6. Outcomes: Do positive outcomes validate a treatment?
Tying back to number five, when we see our clients or patients feel better after our treatment, we often give credit to our treatment for making them feel better. However, this alone cannot be a validation to the positive (or sometimes negative) outcomes because outcomes only measures outcomes, not the effects of the intervention. In other words, outcomes are not reliable factors that indicate our hands-on intervention is the sole or primary cause of pain relief, improved mobility, and other outcomes. This is because many factors other than the intervention can affect the outcome, such as the non-verbal interactions between the therapist and the client or patient, the natural regression of pain, and clients’/patients’ beliefs and prior experience to the intervention and interaction with their previous therapist. Sometimes a positive outcome can happen without any interventions; and we can also say the same thing about negative outcomes which doesn’t always mean that the intervention “sucks.” It could have gotten worse without the intervention.11
Relying on outcomes can also blind us from acknowledging those who do not respond well to the treatment, which is a demonstration of survivorship bias—we notice and acknowledge successful outcomes while ignoring or sweeping aside unsuccessful ones. However, we shouldn’t pooh-pooh outcomes as part of our clinical decision-making. Even though high quality randomized controlled trials are the best method to test how well an intervention works for specific problem, Herbet et al. suggest that we should avoid the being extreme on either side of the decision-making process, where one side relies on outcomes and the other side relies strictly on randomized controlled trials.
They wrote, “We think that, when there is evidence of effects of intervention from high quality clinical trials, a sensible approach to clinical decision-making lies somewhere between the two extremes of the fully empirical approach and a hard-line approach in which clinical decision-making is based only on high quality clinical research without regard to outcome. In this approach, extreme clinical observations (very good or very poor outcomes) are considered likely to be ‘real’ (bias is unlikely to have qualitatively altered the clinical picture), so they are used to guide clinical decision-making.
On the other hand, the qualitative interpretation of typical observations (small improvements in outcome) could
plausibly be altered by bias, so they are essentially ignored. In other words, this approach suggests that clinical decision
making should be influenced by observations of very good and very poor outcomes, but should not be influenced by less extreme observations.”11
When there is no good available evidence, clinical outcomes are likely all we get to use. That could be risky, depending on the nature of the problem and the type of intervention, because low-quality evidence or the lack of it can misguide us.
However, according to Herbert et al., when randomized controlled trials “provide clear evidence of the effects of an intervention from high quality clinical trials, clinical outcome measures become relatively unimportant and measures of the process of care become more useful. When evidence of effects of interventions is strong, we should audit the process of care to see if it is consistent with what the evidence suggests is good practice. When there is little or no evidence (i.e. when practice cannot be evidence-based) we should audit clinical outcomes.”11
When it comes to considering whether an intervention works or doesn’t work, we should take our time (we don’t work in the emergency room) to examine the existing evidence and check our biases. Remember:
“Science is a way of trying not to fool yourself. The principle is that you must not fool yourself, and you are the easiest person to fool.” ~ Richard Feynman
It’s okay to feel frustrated or have a “WTF” moment, and this is a normal experience when learning something new and counter-intuitive. While it is quite overwhelming to see the huge complexity and interactions among different factors that makes up a research paper, it gets easier when we get familiar with recognizing its strengths and weaknesses. In doing so, we can avoid being misled by news headlines and blogs that may misinterpret or exaggerate a study. Also, we can appreciate the larger body of literature that gives us a better picture of a topic, such as whether massage therapy actually does decrease cortisol or not.
Are there any specific areas in research literacy that you would like us to cover in detail? Let us know!
Email Nick at: [email protected]
References
1. Field, T.M., 1998. Massage therapy effects. American Psychologist. 53, 1270e1281.
2. Field, T., Hernandez-Reif, M., Diego, M., Schanberg, S., Kuhn, C. Cortisol decreases and serotonin and dopamine increase following massage therapy. International Journal of Neuroscience. 2005. 115, 1397e1413.
3. Moyer CA, Seefeldt L, Mann ES, Jackley LM. Does massage therapy reduce cortisol? A comprehensive quantitative review. 2011. J Bodyw Mov Ther. Jan;15(1):3-14. doi: 10.1016/j.jbmt.2010.06.001. Epub 2010 Jul 2.
4. Zadro JR, O’Keeffe M, Maher CG. Evidence-based physiotherapy needs evidence-based marketing. Br J Sports Med. Published Online First: 27 October 2018. doi: 10.1136/bjsports-2018-099749.
5. Barash D. Science Needs the Freedom to Constantly Change Its Mind. Aeon. 27 October, 2015.
6. Novella S. In Defense of Prior Probability. Neurologica. 2015, 12 May.
7. Smith CA, de Lacey S, Chapman M, Ratcliffe J, Norman RJ, Johnson NP, Boothroyd C, Fahey P. Effect of Acupuncture vs Sham Acupuncture on Live Births Among Women Undergoing In Vitro Fertilization: A Randomized Clinical Trial. A Randomized Clinical Trial. JAMA. 2018 May 15;319(19):1990-1998. doi: 10.1001/jama.2018.5336.
8. Cook C, Sheets C. Clinical equipoise and personal equipoise: two necessary ingredients for reducing bias in manual therapy trials. J Man Manip Ther. 2011;19(1):55-7.
9. Christiane Nüsslein-Volhard, Banquet speech. NobelPrize.org.
10. O’Connell N. Breaking Research News – The sensory cortex is not a brick. Body in Mind. 2011, 23 Aug.
11. Herbert R, Jamtvedt G, Mead J, Hagen KB. Outcome measures measure outcomes, not effects of intervention. Aust J Physiother. 2005;51(1):3-4.
48 massage & fitness magazine