Internet Learning Volume 6, Number 2, Fall 2017/Winter 2018 | Page 30

The Effect of Term Length on Student Achievement in Online College Algebra
Test Instrument
Students complete assignments using
a popular third-party online mathematical
software program, MyMath-
Lab. This program is produced by the
textbook company and aligns with the
course content. The software is robust
and contains a very large bank of mathematical
problems. For the final exam,
a set of 25 problems are randomly
pulled for each student. The final exam
is cumulative, covering all major topics
from the course. Students log into the
software program and are given 2.5 h to
complete the exam.
Upon completion of the final
exam, the software program produces
a score. While generally accurate, the
scoring does benefit from additional
instructor review. The software does
not allow for variances in the formatting
of the final answer. Even though
a formatting note accompanies each
problem (such as “Write the answer as
a simplified fraction.”), not all students
adhere to the recommendation. When
this happens, a problem can be mathematically
accurate but marked as incorrect
by the scoring system. An example
is when the student computes the slope
using two points on a line. Since the
slope formula itself is in fraction form,
students often leave the final answer in
fraction form, such as m = 5/1, rather
than merely writing the slope as m = 5.
It is up to the instructor to determine
whether full or partial credit should be
awarded. Having multiple formatting
issues, such as these, can greatly impact
a student’s final score.
Reliability and Validity
Test reliability is the amount of measurement
error in the scores yielded by a
test, where a reliability of at least 0.80 is
generally considered acceptable for use
in research (Gall, Gall, & Borg, 2003).
Since the final exam is algorithmically
randomized for every student, each test
is different which, according to Hodges
and Kim (2010), means that traditional
measures of reliability are impossible to
determine. When developing the tests,
the lead course instructor constructs the
test parameters in MyMathLab. Chapter
and section coverage, assignment difficulty
levels (i.e. easy, moderate, hard,
and very hard question types), and the
estimated time to complete the test are
all selected. Given these parameters,
MyMathLab will generate a unique test
for each student. If the lead course instructor
chose to include four moderate
problems from Chapter 3 of the text, for
instance, then each test would include
four randomly generated problems
from Chapter 3 that are considered of
moderate difficulty.
Validity refers to the appropriateness
of inferences made from test
scores (Gall et al., 2003). Content-related
evidence, through assessment by
a mathematics content expert, was used
to demonstrate the validity of a sample
test. Through a review of the course
learning objectives, it was determined
that the sample test questions appropriately
measure student understanding of
the course content.
29