Internet Learning Volume 5, Number 1, Fall 2016/Winter 2017 | Page 10

Online Graduate Course Evaluation from Both Students’ and
Peer Instructors’ Perspectives Utilizing Quality Matters TM
tuck, 2015). Regarding student perspectives,
researchers typically have
taken two approaches. One way is to investigate
student satisfaction of courses
and see whether the courses meet QM
standards (Aman, 2009). The other is to
compare students’ and peer reviewers’
evaluations and see how consistent they
are (You et al., 2014).
This study was conducted in Indiana
University’s graduate program in
adult education. That degree program
originated in 1947 as a community service
program providing off-campus,
non-credit courses in adult education
(Treff, 2008). Through a series of organizational
revisions, the program was restructured
within the academy, offering
both Doctor of Philosophy and Doctor
of Education degrees. Moving from the
School of Education to the School of
Continuing Studies in the 1980s, doctoral
students were no longer admitted;
the program became a Master of Science
in Adult Education administered
from the Indianapolis campus (IUPUI),
and was converted to an online format
in 1998. In 2012, the program returned
to the School of Education in Bloomington
as part of Instructional Systems
Technology.
9
In 2015, the program underwent
a self-study in an effort to improve
the quality of the program. That
study involved interviews and surveys
of alumni, currently enrolled students,
and program faculty. In our self-study,
we felt it important to gain the perspectives
of our students, faculty and
outside observers. This current study,
which focuses more directly on specific
online courses, is consistent with
our overall quality improvement effort.
Our objective was to improve our online
courses by comparing them to QM
standards, identify areas of strengths
and weaknesses in our courses, and
identify whether instructor perceptions
are congruent with student perceptions
of our courses and thereby improve the
quality of our graduate adult education
online program.
Methods
This study examined students’ evaluations
of online courses in comparison
to peer instructors’ evaluations of the
same online courses. The evaluations
followed the QM standards.
Measurement
The course evaluation data were collected
from two cohorts: students and
peer instructors. Evaluation items were
adopted from the QM standards. There
were 21 evaluation items organized by
8 categories: (1) course overview introduction,
(2) learning objectives, (3)
assessment and measurement, (4) instructional
materials, (5) course activities
and learner interaction, (6) course
technology, (7) learner support, and (8)
accessibility and usability. Each evaluation
item was rated with a 5-point
Likert scale (1 = strongly disagree, 5 =
strongly agree).
Although the two cohorts used
the same evaluation items, the organization
and procedure were different.
The peer instructors used an evaluation