Internet Learning Volume 3, Number 1, Spring 2014 | Page 48

Measuring Online Course Design
Discussion
We investigated the differences between
students and peer reviewers regarding
the essential standards in three online
courses. When the courses were approved
for design the faculty course developers
were provided a copy of the Quality Matters
Rubrics in the beginning of the course
development process. Instructional designers,
who were certified QM peer reviewers,
were available for individual consultations
during the design and development process.
The faculty course developers were
very familiar with the standards and agreed
that it was essential to incorporate the standards
into online course design processes.
As reported in the results section,
in course A the results reported by students
and peer reviewers differed significantly in
regards to Standard 2.1 (The course learning
objectives describe outcomes that are measurable)
and Standard 2.4 (Instructions to
students on how to meet the learning objectives
are adequate and stated clearly). For
Standard 2.1, the students were asked to report
whether course objectives were clearly
presented in the course syllabus. For Standard
2.4, both reviewers and students were
asked to report whether clear instructions
on how students should meet the learning
objectives are articulated in the course.
Students reported that the instructions
were available. However, reviewers do not
agree as one stated:
Standard 2.4 calls for clear instructions
on how students should meet the learning
objectives. The course design does
a good job in providing students with a
brief introduction to each Chapter topic;
however, it is somewhat difficult to understand
which learning activities, resources,
assignments, and assessments support the
learning objectives for each unit week. It
is important to help students connect the
dots between chapter level objectives and
the assigned activities and assessment for
the week.
Apparently peer reviewers are looking
for above average at approximately 85%.
Students might think the brief introduction
to each chapter provides instructions on
how to achieve the learning objectives. The
overall satisfaction of the course might also
affect students’ rating on the standards as
the majority rated the course as excellent.
A third factor that might contribute to the
difference is the student satisfaction of the
teacher. The responses to the open-ended
questions indicated that the professor was
excellent and cares about student learning,
as one student stated:
The professor always leads a very informative,
fun, and creative class and this one
was not an exception. I learned a plethora
of new things from the reading, assignments,
and independent studies throughout
the semester.
In course B the results reported by
students and peer reviewers differed significantly
regarding Standard 2.2 (The module/
unit learning objectives describe outcomes
that are measurable and consistent with the
course-level objectives) and Standard 3.2
(The course grading policy is stated clearly).
For Standard 2.2, the students were asked
to report whether module/unit objectives
were clearly stated in each unit. For Standard
3.2, both reviewers and students were
asked to report whether grading policy was
clearly articulated in the course. Students
reported that the grading policy was available,
however, the majority of the reviewers
thought that the policy was not clear
enough. One reviewer stated:
Standard 3.2 asks for a clear, written description
on how student's grades will be
calculated, for instance, the total points
for each assignment, the percentages or
weights for each component of the course
47