Internet Learning Volume 3, Number 1, Spring 2014 | Page 110

Internet Learning
nal peer reviewers will increase the level of
training of our faculty overall and better
distribute the course review workload. An
alternative to alleviating our staffing limitations
is to shift to external peer reviews
once faculty members have a better understanding
and buy-in of the peer review process.
Future Research
Two additional research streams are
suggested by this initial study. The first is
expansion of the original project to include
other institutions that are at a similar point
in QM implementation (i.e., hosting voluntary,
internal peer reviews) to increase the
available participant pool. Initially, after we
identified and assessed faculty members’
beliefs, we planned to test the utility of attitudes,
norms, and perceived behavioral control
in predicting faculty intention to participate
in the peer review and then predict
actual behavior from intention. However, at
the close of our data collection period when
the researchers were no longer blind to the
research participation status of our peer review
participants, we realized that our sample
size was too small to support such an
examination. A power analysis confirmed
this concern. Given the R 2 from the current
dataset (i.e., R 2 = 0.32), a sample size of at
least 34 participants would be needed for a
test of the model with power = 0.90 and α =
0.05 (Cohen & Cohen, 1983). Though our
total number of eligible faculty members
was large enough to support such a test, we
were not able to recruit enough participants
for this entirely voluntary, un-incentivized
research study. With a larger sample of faculty
who are being introduced to the QM
peer review at other institutions, a broader
picture of the accommodations that are
made to the process to gain faculty buy-in
could be obtained. In addition, given that
norms are group-specific expectations,
groups of faculty members may hold different
norms at other institutions. Conducting
this study on a larger set of institutions
would allow for more general statements
regarding faculty beliefs and motivations
to comply with expectations regarding the
peer review of courses. Such research may
also shed light on the direction that norms
and attitudes shift as faculty members embrace
peer review as a method of continuous
course improvement.
A second stream of research will
be directed at improving the feedback provided
during the course of internal peer
reviews. A cursory review of comments
provided to faculty course developers at the
close of this initial set of internal reviews
revealed substantial inconsistencies across
reviewers. Given that we firmly believe that
internal peer review is a tool that is helping
our institution build a culture of continuous
course improvement, promoting more
rigorous standards for acceptable reviewer
comments may have the potential to more
efficiently improve the course quality. To
evaluate this prediction, this research team
is planning to systematically examine the
content of the comments provided by our
peer reviewers to evaluate the extent that
feedback provided to faculty course developers
was consistent with the QM training
that reviewers received (e.g., that reviewers
referenced specific Rubric standards and
provided evidence from the course). The
results of this research will shed light on
the nature of comments that peer reviewers
make and suggest areas for revision of training
and minimum content standards for
comments. Follow-up research is planned
to determine whether revisions of training
and comment standards positively affect
the content of reviewer comments and assist
faculty course developers in improving
their courses.
109