Internet Learning Volume 5, Number 1, Fall 2016/Winter 2017 | Page 12

was 4.3 and 3.5, respectively.
Online Graduate Course Evaluation from Both Students’ and
Peer Instructors’ Perspectives Utilizing Quality Matters TM
All of the courses received higher
ratings for the key components of QM,
including learning objectives, assessment
and measurement, instructional
materials, course activities and learner
interaction, and course technology
(Categories 2–6). Courses appeared to
require improvement in the following
categories: course orientation, learner
support, and accessibility (Categories 1,
7, and 8).
Inter-rater Reliability between
Students and Peer Instructors
Overall, students evaluated the courses
more positively than the peer instructors
did for all the courses except one
(D004). However, the differences did
not appear to be great (ranged from
0.1 to 0.6). We were more interested in
checking whether student evaluations
were consistent with the evaluations of
peer instructors. In order to see the relationship
between the students’ and instructors’
evaluations, researchers carried
out Pearson’s correlation analysis.
The analysis was conducted separately
for each course. To compare the two
groups’ evaluations, researchers calculated
the average scores of each evaluation
item and used them to calculate the
Pearson’s correlation coefficients. The
analysis revealed that students’ evaluations
of courses were significantly correlated
with the instructors’ evaluations
in all the courses. Those ranged from r
= .34 to r = .67 (see Table 2).
Reviews of Essential Standards
On the basis of the findings, we confirmed
that the courses fulfilled the key
components of QM standards except in
the course technology category. In order
to gain deeper understanding of the
evaluation, researchers reviewed the 21
essential standards (see Table 3).
The findings revealed that the
strongest areas of the courses were
learning objectives, assessment and
measurement, and learner engagement.
Regarding the learning objectives, the
following standards were highly evaluated:
learning objectives or competencies
were (1) suited to the level of
the courses; (2) measureable and consistent
with the course level goals; and
(3) clearly stated from the learner’s
perspective. According to the results,
the courses included learning activities
that promoted active engagement and
achievement of learning goals. Policies
about course grades were stated clearly.
Table 2. Correlations of Evaluations between Students and Instructors
Note: The number of questions for each course is 43. So, the degrees of freedom is 41.
11