The Journal of mHealth Vol 3 Issue 2 (Apr/May 2016) | Page 8

Transforming Delivery: Learning the Fundamentals...
Continued from page 5

The use of this real-time data is important in many of Medic
Mobile’s projects. “You will see community health workers using
an app for the case management of childhood illnesses and from
this you are able to see for example the number of clients they
had in a given time period and how that stacks up relative to peer
health workers.” explains Holeman. “One important indicator is
the number of children followed up within 24, 48 or 72 hours of
a particular health event. For conditions like malaria, that timeliness is deeply important in averting child mortality. In this case
and many others, the opportunities with real-time data are less
about the tools themselves and more about drawing attention to
health workers and the quality of care they provide. That is to
say, when real-time data projec ts are going well, the digital technology often fades into the background, it is the infrastructural
and human features of the health system as a whole that become
more visible. One of the interesting opportunities and challenges
for these projects is to use that data to feed back into the design
process. Ideally you are reshaping the delivery of care and making
changes to the technology at the same time. In my mind, this all
underscores a point that we return to repeatedly throughout the
course: that successful mHealth initiatives typically involve the
integration of new digital tools with routine health worker activities and the re-organization of local health systems.”
Says Pagliari, “As more and more of these real world sources of
mHealth evidence become available our challenge is to make
best use of the data and render it into forms that are easily
interpretable”.
“Trying to differentiate the stages of research within the life
cycles of innovation is also something that we’ve been looking
at over the years,” says Pagliari. “For example, at the innovation
stage, when we are not quite sure how things are going to work,
our research question and methods will be more exploratory and
may use participative design methods or softer forms of evaluation that are much quicker, later moving into more traditional
qualitative and quantitative research methods.”
Despite the raft of new technologies that we see in digital health,
when it comes to scaling mHealth projects there is always a need
to understand accessibility. If a technology is only accessible by
a few owing to infrastructure, device availability or socio-economic limitations then it is unlikely to have a significant impact
when introduced to a wider audience.
“With Medic Mobile it has been interesting to watch the rise and
fall and rise again in popularity of text messaging” says Holeman.
“When we were getting started in 2008 people were already telling
us then that text messaging was basically dead and that before long
smartphones would be almost as cheap as water and everybody
would have them. The popular wisdom was that if you want to
blast large regions with educational messages then text messaging
is fine but if you want to equip health care workers with coordination tools so that they can keep track of actual data then working with text messaging was a quickly dying project. However, we
chose to do it anyway because our approach starts with people! So
we observe what tools people already have in their hands. In the
challenging places where we like to work, they had basic phones.”

6

“We had some very successful early projects using SMS[6, 7] and
when you understand how and why it is that these coor-

April/May 2016

How to Measure the ROI of Health Apps
dinating tools work it is not that shocking.” explains Holeman.
“Most of what community health care workers do is walk. They
are travelling from home to home, so anything that enables
them to know that they need to visit this house today or that
they don’t need to visit a particular house today - basic coordinating tools - can save them hours. In the last year Medic Mobile
has started using more smartphones and tablets to great effect,
but not just anywhere. In many of the places we work less than
a third of the population has electricity at home; that level of
electricity access still presents striking challenges for smartphone
projects. There are workarounds like giving people solar panels
but that requires additional work on their part and it means that
they can’t do their technology-enabled work while the phone
is charging. So if we introduce electricity workarounds, these
are not only another burden on the health workers, they also
become another design consideration. We have to be humble
in our reflection on these difficulties, because it’s far too easy to
fall into a trap of pursuing innovation for its own sake without
actually understanding what problems we are trying to solve.”
“Understanding what new eHealth approaches will be truly
transformational, or are truly innovative is the critical challenge”
agrees Pagliari. “In mHealth, simple but sensitively-designed
interventions may be far more effective than simply having the
latest technology, and may also be considerably cheaper in some
cases. I think this disentangling of the truth from the hype, is
really important for the area, which is why it’s so valuable to be
working with someone like Isaac. With all his work in the lower
income countries it makes you catch yourself when you are talking about all the high tech [digital health] stuff and makes you
think about what is fundamentally important.”
References
1. Pagliari C. (2007) Design and Evaluation in eHealth: Challenges and Implications for an Interdisciplinary Field. Journal of Medical Internet Research 9(2)
2. Pagliari C, Sloan D, Gregor P, Sullivan F, Kahan J, Detmer
D, Oortwijn W, MacGillivray S. (2005) What is eHealth
(4): a scoping exercise to map the field. Journal of Medical
Internet Research 7(1):e9
3. Franklin V, Greene A, Waller A, Greene S, Pagliari C.
Patients’ Engagement With “Sweet Talk” – A Text Messaging Support System for Young People With Diabetes. J Med
Internet Res 2008;10(2):e20
4. Ryan D, Pinnock H, Lee AJ, Tarassenko L, Pagliari C, Sheikh
A, Price D. (2009) The CYMPLA trial. Mobile phone-based
structured intervention to achieve asthma control in patients
with uncontrolled persistent asthma: a pragmatic randomised
controlled trial. Prim Care Respir J. 2009 Dec;18(4):343-5
5. Lee S, Nurmatov U, Nwaru B, Mukherjee M, Grant E,
Pagliari C. (2016) Effectiveness of mHealth for Maternal,
Newborn and Child Health in Low and Middle Income
Countries: Systematic Review. Journal of Global Health
2016; 6; 010401.
6. Holeman I, Pagliari C, Weller D, Grant L, Evans J (2014)
Mobile health for cancer in low to middle income countries.
priorities for research and development. European Journal of
Cancer Care. Doi: 10.1111/ecc.12250 e15)
7. Mahmud N, Rodriguez J, Nesbit J. A text message-based
intervention to bridge the healthcare communica tion gap
in the rural developing world. Technology and Health Care.
2010 Jan 1;18(2):137-44. n

How to Measure the ROI of
Health Apps
By Viktor Bogdanov, Intersog
Health apps have been transforming the
healthcare industry over the last couple
of years. Its impact will continue to grow
as more and more health apps are used to
manage and treat chronic diseases.
Health IT doesn’t follow a traditional
health business model, so it’s often
unclear how the app is helping you make
money, save money, or save lives. As a
result, it can be difficult sometimes to
figure out your ROI for a health app.
Measuring ROI by Calculating
Minutes
However, there are other means of ascertaining the ROI of health apps. One
example is mHealth apps like the Greenway PrimeMOBILE app for Windows
8[1]. This health app calculates minutes
that are saved in each doctor-patient
encounter.
These minutes will add up over time and
can be crucial in freeing up time to attend
to more patients. Doctors can access the
PrimeSUITE EHR from their mobile or
tablet to make notes and access information while making rounds. As a result,
EHRs can be started immediately[2] and
not later on when a computer is free.
Such apps are easier to measure as your
ROI can be calculated based on how
many minutes were saved over a period
of time. The value of such apps goes far
beyond money as the time saved by doctors can be used to save more lives.
Measuring ROI in Drug Trials
Health apps have had a significant
impact on improving patient adherence
and completion of treatment. So when
you use an app alongside a clinical trial,
you’ll see rates of adherence improve dramatically. This in turn can save the company money while conducting a drug
trial. As a result, these numbers can be
added up to identify your ROI.
For instance, as noted by Waracle, their
clinical trial completion rates have seen
an astounding increase from 53% with

no app provided to 82% with an interactive app being used alongside the clinical trial. Let’s assume it costs $15,000 for
each person in a clinical trial. The savings
for pharmaceutical companies would
amount to over $1.5 million for each
drugs trial. This average saving represents
a conservative 10x ROI.
The high volume of data that’s collected
during a drug trial can also be used to
enable better decisions and diminish
health risks of the participants through
early intervention. Further, it will make
it a lot easier and cost effective to obtain
regulatory approval.
This in turn will have impact on how
quickly a treatment can be transferred for
clinical use. Further, the data collected
can drive healthcare app development and
help optimize app from real patient use.
Measuring ROI by Driving
Engagement
Engagement can also be measured to
assess the ROI of mobile health apps.
Studies[3] have found that apps that are
realistic and interactive can enhance clinical skills. Skills like suturing live tissues,
handling of instruments, and the completion of invasive procedures can all be
significantly improved by practicing on a
mobile app. In other words, as medical
professionals get more engaged with the
app[4], your ROI will also increase.
Further, health apps that drive engagement can also be used to increase patient

adherence and reduce readmissions[5].
This in turn can enhance patient experience and increase loyalty[6] as patients
can track their progress and communicate with providers on a regular basis.
With the Internet of Things (IoT) slowly
growing in importance at hospitals,
adoption rates will improve significantly
as it will make patient care coordination
much smoother. Although devices are
getting smart these days, it’s still difficult
to get them to work together. As a result,
platforms where these devices and sensors operate will also get better.
All the data collected from IoT and
health apps will produce a massive
amount of data. This data in turn will be
used to make healthcare more efficient,
cost-effective, and intelligent. Providers
want to essentially cut costs and improve
patient experience and access to services.
As a result, you can expect health apps to
grow in importance much faster as there
are fewer barriers.
References

1. http://www.mhealthnews.com/news/measuring-mhealth-roi-minutes-saved
2. http://www.ehealth.intersog.com/blog/howto-implement-ehrs-the-right-way
3. http://www.ncbi.nlm.nih.gov/pmc/articles/
PMC3102783/
4. http://blog.biodigital.com/5-ways-to-driveengagement-increase-roi-in-your-medical-app
5. http://www.practiceunite.com/guide-toevaluating-roi-of-secure-texting-apps/
6. http://www.xcubelabs.com/our-blog/
calculating-roi-patient-engagementmobile-app/ n

The Journal of mHealth

7