unique, they cannot extrapolate from historical data. Then – and without a hint of
irony – they will claim that therefore they
must rely on their experience. Of course,
as the book argues, expertise and science are both based on past observations
– one of these with much more selective
recall and tendency for flawed inferences
than the other. Managers make just such
a mistake whenever they say that they
can’t make estimates about implementing
a new technology because it is so unique
– even though they have a long history of
implementing new technologies. Using
that same logic, your insurance company
couldn’t possibly compute a life insurance
premium for you because you’re unique
and because you haven’t died yet. In fact,
insurance actuaries know how to extrapolate from larger, more heterogeneous
populations.
The third edition also expands on developments in how big data, social media,
mobile phones and personal measurement devices are making the “we don’t
have enough data” excuse much harder
to justify.
SUMMARY
You can, in fact, measure anything,
in our view, but doing so is sometimes a
challenge even for those who are convinced the claim is true. We simply need
to recognize that the perceived challenge
A NA L Y T I C S
results from some of the same old, entrenched misconceptions. Your problem is
most likely not as unusual as you think;
there are sources of information you can
use, if you think creatively about how to
apply them; calibrated experts can make
good estimates of their uncertainty about
the data points they provide; and calculating the expected value of information
can focus you on collecting the most useful additional data, not wasting effort and
resources on data that won’t help much.
Douglas W. Hubbard (dwhubbard@
hubbardresearch.com) is president of Hubbard
Decision Research in Glen Ellyn, Ill., and an
internationally recognized expert in measurement
and decision analysis. Douglas A. Samuelson
([email protected]), D.Sc., is president
and chief scientist of InfoLogix, Inc., a consulting
and R&D company in Annandale, Va., and a
contributing editor of OR/MS Today and Analytics
magazines. He is a longtime member of INFORMS.
NOTES & REFERENCES
1. Douglas W. Hubbard, 2007, “How to Measure
Anything: Finding the Value of ‘Intangibles’ in
Business,” Wiley; third edition, 2014.
2. S. Lichtenstein and B. Fischhoff, 1980, “Training
for Calibration,” Organizational Behavior and Human
Performance, Vol. 26, No. 2, pp.149-171.
3. Paul Meehl, 1986, “Causes and Effects of My
Disturbing Little Book,” Journal of Personality
Assessment, Vol. 50, pp. 370-375.
4. Douglas A. Samuelson, 2001, “Information
Technology Benefits Assessment,” Encyclopedia of
Operations Research and the Management Sciences,
Second Edition, Springer. (A revised version also
appears in the third edition, 2013.)
5. Philip E. Tetlock, 2006, “Expert Political Judgment:
How Good Is It? How Can We Know?” Princeton,
N.J.: Princeton University Press.
M A Y / J U N E 2 014
|
33