Military Review English Edition July-August 2015 | Page 61

MULTIRATER FEEDBACK

issue; some propose solutions. Many, such as Tim
Kane’s Bleeding Talent, suggest the solution lies in
360-degree feedback, or multirater feedback, with
proposals to incorporate those types of feedback
into officer evaluation reports (OERs), promotion boards, and even central selection list (CSL)
boards.2 Gen. Martin Dempsey, chairman of the
Joint Chiefs of Staff, seems to agree. Army Times
reporter Andrew Tilghman reports that Dempsey
said, “as time passes and the force grows more
comfortable with 360-degree reviews, they may
ultimately be integrated into the command screening process.”3 However, Odierno, in an April 2013
address to students at the U.S. Army Command and
General Staff College, rightly said the Army must
be careful about how it proceeds with implementing multirater feedback into evaluation and leader
development processes.4
Multirater feedback has been one of the fastest
growing assessment instruments in business for
leaders, managers, executives, and employees alike,
so why not use it in the Army as well? This paper examines multirater feedback, its validity and reliability, appropriate uses for it, and its pitfalls.
An understanding of this assessment instrument—how it performs effectively under certain
conditions and how it can be damaging to an organization under other conditions—supports the conclusion that multirater feedback should be used only for
development purposes. Using multirater feedback
directly for performance evaluations, promotion
boards, and CSL boards would lead to improper
selection of future leaders and could needlessly
damage Army leader development, while failing to
address toxic leadership.

What is Multirater Feedback?
Multirater feedback, 360-degree feedback,
multisource assessment and feedback, and similar
assessment approaches share the common characteristic of providing individuals (leaders, managers,
or anyone that interacts with more than one level of
the organization) with feedback on their behaviors
from the perspectives of others. For the purposes
of this article, 360-degree feedback and multirater feedback will be used interchangeably, with the
understanding that 360-degree feedback involves
MILITARY REVIEW  July-August 2015

subordinates, and multirater feedback may or may
not involve subordinates. Although the number of
reports and levels surrounding the individual varies,
the concept remains the same. When used properly,
this instrument can benefit the target individual
and the organization as a whole. From 1982 to 1992,
the number of off-the-shelf 360-degree instruments
being sold quadrupled. Companies spent $152 million on multirater feedback systems in 1992, with
90 percent of Fortune 500 companies using a form
of multirater feedback in 2003.5 Although powerful
as an assessment instrument, multirater feedback
requires certain conditions: a safe learning environment, experienced coaches or counselors, and a
longitudinal development plan.

Development of Multirater
Feedback
Over time, organizations and employees have
sought more fair appraisal and assessment systems,
other than the traditional top-down formal assessments typically prepared by managers about their
subordinates. Well-documented cases of unfair,
inaccurate, or low-quality top-down reports have
demonstrated the need for bottom-up feedback
from subordinates. Feedback from multiple raters
was intended to counter the subjective nature of
top-down ratings, yielding a “fairer and possibly
less biased view than simply relying on superior’s
ratings.”6 However, according to Clive Fletcher,
Caroline Baldry, and Nicole Cunningham-Snell, unless feedback systems are constructed and evaluated
along the lines associated with psychometric tests,
they may produce misleading assessments.7
Multirater feedback instruments request data
from the target individual; the individual’s supervisors, peers, and subordinates; and if applicable, customers or others. The organization categorizes the
behaviors required for performing the job and usually asks respondents to rate the individual’s behaviors as observed along a Likert scale, sometimes also
including a qualitative portion that allows short-answer input.8 The number of observable behavioral
competencies varies between instruments, normally
tailored to the job type and the organization. The
intent of the survey is to present the target individual with a complete picture of his or her behavioral
59