24
doc • Summer 2015
Kentucky
Standardized Mortality
Ratio (SMR)
Common Misunderstandings
By Tuyen Tran, M.D.
Since the Institute of Medicine’s (IOM)
report in 1999, To Err is Human, multiple
stakeholders (government agencies, accreditation bodies, payers, hospitals, providers,
and the public) have responded with various metrics to monitor and interventions
to implement. They wanted to improve the
healthcare and positively effect changes to
prevent the annual avoidable 98,000 deaths
and 1 million injuries related to medical
errors1. Stakeholders demanded positive outcomes and they wanted providers accountable. As the chanting of “First, do no harm!”
continued, hospitals and providers became
mired in performance measures and report
cards. But, are these metrics valid?
Mortality is comprised of three variables: 1)
patient risk-factors (case-mix), 2) random
chance, and 3) quality of care. Despite the
best attempt at risk-adjustment, the calculations cannot account for unmeasured and/
or immeasurable factors. That is, the outputs
of risk adjustment regression models depend
upon the input variables, which can vary
widely. For example, comorbidities (morbid
obesity, dementia, and heart failure, level of
frailty and disability) which impact mortality
estimates are inconsistently documented.
As early as 1847, Ignaz Semmelweis observed
that women delivered by physicians and
medical students had a much higher postdelivery mortality than midwives (13-18%
vs 2% respectively)2. (The differences were
related to lack of hand washing.) Interest
in comparing outcomes among healthcare
providers certainly predates the IOM report.
Of particular interest is the adjusted standardized mortality ratio (SMR, also known by
various other names such as Risk-Adjusted
Hospital Mortality Ratio). It is a comparison
of the observed number of in-hospital deaths
to the number expected based upon the hospital’s case mix. Ratios greater than 1 suggest
unsafe healthcare and ratios less than 1 suggest safe practices. The thought process for
“risk-adjustment” is that if the contributions
from patient case-mix factors are removed,
the residual unexplained variation is related
to quality of care. This is perfect! First, the
end point is concrete and everyone would
consider the outcome of death important.
Second, the information is readily available
from most administrative databases. Third,
the ratio allows for inter-hospital comparisons. Finally, the data are amenable to statistical analyses and graphical displays to facilitate
interpretation.
There is the inevitable desire to conclude that
a favorable SMR (ratio < 1) indicates a safe
hospital, or vice versa. Death in modern hospitals is relatively rare (5-10%), and forensic
clinical analyses of these deaths show that
only 5% are attributable to unsafe care3. Thus,
mathematically, due to the low rates of occurrence, only 8% of hospitals with unfavorable
risk adjusted SMR (ratio >1) will truly be
more “unsafe” than the average hospital. On
the other hand, of the hospitals with favorable SMR, 10 out of 11 of these hospitals
may actually be more “unsafe” than the average hospital. The reason is that most quality issues may result in injury or prolonged
hospital stays; but, they do not cause death4.
Most unsafe practices do not cause death and
most deaths are not the result of unsafe care.
Of course, one preventable injury is one too
many! Despite the tremendous allocation of
resources into the improvement of quality
and safety, the fact is that patients are needlessly harmed as a complication of receiving
healthcare5. Unfortunately, there are no reliable tools to accurately measure the quality of
care for physicians, hospitals, or populations.
And until these valid metrics are found, tension will continue to mount among those
who seek safe quality care (public), those
who have the obligation to protect the public
(policy makers), and the providers who do
wish to deliver safe and quality healthcare.
References
1.
2.
There is the
inevitable desire
to conclude that
a favorable SMR
(ratio < 1) indicates
a safe hospital, or
vice versa.
3.
4.
5.
Kohn LT, Corrigan JM, Donaldson MS, eds.
To err is human: building a safer health system.
Washington, DC: National Academies Press,
1999.
Best M, Neu