Internet Learning Volume 3, Number 1, Spring 2014 | Page 33

Internet Learning
frequently submit different scores may lack
clarity; this could result in the need for
changes to the specific standard or it could
signal a need for more reviewer training.
The standards with lowest rater
agreement table provide an overview of the
revisions made by the 2010 Rubric Committee
for standards that statistically had
the lowest rater agreement. The chart also
provides data on the most recent data analysis
and has been provided to the 2014 Rubric
Committee.
Individual ratings given by a QM
peer review in course reviews reflect, to at
least some extent, that particular reviewer’s
professional/pedagogical opinion, and,
therefore, may vary from the ratings of
the other individual reviewers. However,
markedly lower rater agreement for specific
standards in the QM Rubric is a prompt
to members of the Rubric Committee to focus
attention on those standards during the
regular review and refreshment of the QM
Rubric.
Summary
Regular, robust (breadth and depth)
review and refreshment of the QM
Rubric and processes keep them
current, practical, and applicable across
academic disciplines and academic levels.
The review includes interpretation of educational
research, as well as an emerging
emphasis on research generation. Expertise
from online educators across the United
States plays a critical role in the transparent,
faculty-centered processes. The review
ensures validity in the set of quality standards
within the Rubric. Statistical analyses
of data gathered from formal course reviews
reveals that the peer review process
has been consistently applied across review
types and academic disciplines and points
to the value of QM’s professional development
in which over 28,000 online educators
have participated. The analyses also
provide critical information to the Rubric
Committee on the frequency of met standards
and on the proportion of rater agreement
by specific standards.
Glassick (2000) noted that Boyer’s
scholarship of overlapping discovery, integration,
application, and teaching is “a
hard but worthwhile task” (p. 880). This
article outlines how the dynamic and rigorous
processes adopted by QM continue
to take on that worthwhile task. All aspects
of the QM program are regularly reviewed
and refreshed with and for online teaching
faculty.
References
American Association of Colleges of
Nursing, AACN Task Force on Defining
Standards for the Scholarship of Nursing.
(1999). Defining scholarship for the discipline
of nursing. Retrieved from http://
www.aacn.nche.edu/publications/position/
defining-scholarship
Bernstein, D., & Bass, R. (2005). The scholarship
of teaching and learning. Academe,
91(4), 37–43.
Cervero, R. M., & Wilson, A. L. (1994). The
politics of responsibility: A theory of program
planning practice for adult education.
Adult Education Quarterly, 45(1), 249–268.
Cousin, G., & Deepwell, F. (2005). Designs
for network learning: A community of
practice perspective. Studies in Higher education,
30 (1), 57–66.
Glassick, C. E. (2000). Boyer’s expanded
definitions of scholarship, the standards for
assessing scholarship, and the elusiveness
of the scholarship of teaching. Academic
32