3. Effect sizes: How big is the difference between groups? Is it relevant to clinical practice?
In research, an effect size tells us how much difference there are between two treatment groups, and knowing it can help us recognize how effective and efficacious the treatment is. If a research paper says that patients who were treated with XYZ intervention had less pain than those who had received no intervention or had a different type of treatment, how big was the difference? After all, technically, researchers could claim a tiny difference between groups can be “better,” in which some people can market that intervention as “effective.”
Effect size, also known as Cohen’s d, is measured on a scale where 0.8 is ranked as “high,” 0.5 as “moderate,” and 0.2 as “low.” Back to the Moyer’s 2011 review on the effect of massage on cortisol levels, the researchers pooled 11 studies on single-dose massage sessions as the first treatment in a series of treatments with a total of 460 participants and found that “[massage therapy] did not reduce cortisol significantly more than control treatments” with the effect size (d) of 0.15. When they examined eight studies of single-dose sessions with 307 participants as the last treatment of a series, they also found the same result (d = 0.15).
Among multiple-dose massage sessions, they also found low effect sizes (d = 0.12) among 16 studies with a total of 598 participants. When these studies are split into adults and children, the adults (n = 508) effect size dropped more than 50 percent (d = 0.05). However, the children group ranked higher with a “moderate” d = 0.052. The last result may seem promising, but Moyer et al cautioned that this data is based on a tiny number of studies and likely is “vulnerable to the file-drawer threat” where many studies with negative outcomes are not published.
Note: Larger effect size does not always mean it’s “better,” nor does a smaller effect size mean it’s “not very good.” You need to consider context of the data to making better meaning. For example, if being active regularly can improve mortality by 0.09 over five to six years, this small effect
can still save many lives by encouraging some people to be more active.
4. Personal and clinical equipoise: How much bias are there in the study? How fair were the interventions compared?
In some ways, manual therapy research is similar to mixed martial arts where one modality is “pitted” against another to see which is “better.” This bias not only may lead them the “everything is a nail” mentality, it also can skew the outcomes of manual therapy research. As with any scientific research, the goal is to be as objective as possible, hence the emphasis of clinical equipoise, which is the assumption that one intervention is NOT better than another. Physiotherapists Chad Cook and Charles Sheets wrote that to have true equipoise, the researchers must have “no preference or is truly uncertain about the overall benefit or harm offered by the treatment to his/her patient. In other words, the clinician has no personal preconceived preferences toward the ability of one or more of the interventions to have a better outcome than another.”8
While this may not seem very practical or even realistic in actual research, it is something researchers should strive for. However, even with the best intentions, researchers sometimes unconsciously set up the experiment in a way that favors the tested intervention or undermines the
control group. It is also possible that the “placement of importance, enthusiasm, or confidence associated with one’ expertise in an intervention” can influence the experiment’s outcome, which also includes how well the patients perceive the treatment.8
Having pure personal and clinical equipoise may be unlikely to avoid to do because it is impossible to blind the practitioner. It would be like working blind-folded while providing the treatment, which is not practical.
5. Is the research basic or translational science? How well could the research be applied to hands-on work?
You might have read something about researchers using fruit flies to understand how genetics, behavior, and certain diseases work. Back in the late 1970s to early 1980s, Dr. Christiane Nüsslein-Vollhard, who is a biologist, made some discoveries about how fruit flies grow and develop with her colleagues Dr. Eric Wieschaus and Dr. Edward B. Lewis. She had no idea that their research on Drosophilas could end up being one of the main staple animals that many biomedical research use in the next three decades.
Cohen’s d compares the mean of two variables to determine how much or how little the effect size is. The further apart the means are, the greater the effect size.
Image: Skbkekas
For more information on effect sizes, check out this short presentation on Cohen's d and why effect sizes matter in clinical research.
46 massage & fitness magazine