The Journal of mHealth Vol 3 Issue 2 (Apr/May 2016) | Page 24

Evidence and Measuring Digital Health Outcomes

Study Demonstrates the Effectiveness of App for Diagnosing...

Evidence and
Measuring Digital
Health Outcomes
By Keith Nurcombe
Keith Nurcombe has worked in healthcare for over twenty years spending the last few years working with
businesses in the health and technology space, most recently building O2 Health where he was Managing Director until the end of 2012, since then he has been providing consultancy services to businesses.

The art of measuring outcomes is a challenge for all areas of the healthcare community both in business but also in wider
care delivery models.
Healthcare evidence and outcomes is a
really complicated process and often very
difficult to find your way through. This
is often compounded with the belief that
all of this needs to be agreed and worked
out in advance i.e. before you start anything - hence delaying the whole project
and often resulting in some projects not
happening at all.
Lots of really good work has been done
to date in the digital world of healthcare
when it comes to trying to make sure that
we have the kind of outcomes that healthcare providers and payers want to see.
So why do we need to measure and why
outcomes?
There has been a very high level of focus
for many years on outcomes in the healthcare word for all the right reasons, after all
this is about patients and their health and
wellbeing. So, why do we find ourselves in
a place where providers and payers seem
to have set the bench mark for success
even higher for digital health? I have some

22

views as to why this is, and actually I think
we should set our benchmark very high.

ment at points throughout and at the
end of the process.

Patient data - people are very sensitive
about permissions, who owns what,
and who has access to what, in the data
chain when it involves patients and clinical data. This is compounded when data
moves out of clinician control.

2. Think about baseline data before
beginning the project - What do you
need? What are you looking to prove or
achieve and therefore what do you need
in order to judge the success and outcomes at the end of the process? Knowing this will give you an edge if you can
get baseline data upon which it is possible to judge and draw conclusions from.

Digital programs and technology are
still often seen as slightly frightening
concepts and when people don’t really
understand it they can feel the need to
place extra steps, or extra security, in a
process to feel happy.
Digital healthcare is defining new operating models and new ways of working. This
means clinicians and patients need to find
their way through new processes, and as
this may also involve less face-to-face contact then this can often lead to further concerns for patients but especially clinicians.
I think there are some basic rules to follow
in terms of trying to get the best outcomes
for digital healthcare going forward:
1. Always set clear objectives and criteria
for your projects - clarity at the beginning will allow better outcome measure-

3. Always look at the patient and user outcomes as much as you look at the clinical
outcomes - It is always very tempting to
be driven down the route of everything
clinical when actually you need to measure the changes for the patient and the
user as much, if not more, in the digital
healthcare world moving forward.
Let's be clear evidence is king and ROI
for the payer, provider and user are all
critical for success as well. We need to
learn to be master of all in terms of evidence and outcomes if we are going to
compete with the old fashioned non digital ways of doing things involving paper
and data collection from trials etc.
It's here to stay lets embrace it! n

Study Demonstrates the
Effectiveness of App for
Diagnosing Respiratory Diseases
Early results from a clinical trial to assess the accuracy of a
smartphone-based system for diagnosing respiratory diseases
have shown breakthrough performance in paediatric diagnoses.
Preliminary analysis by the research team has highligh ted the
high level of accuracy of the ResApp diagnostic algorithms for
the identification of lower respiratory tract disease. Lower respiratory diseases (such as bronchiolitis and pneumonia) are often
more severe than upper respiratory tract infections (URTIs), and
the ability to differentiate between these two categories is critical
for effective treatment and advice.
The results of the trial have shown ResApp’s diagnostic tool
achieved overall accuracy levels in excess of 90% when used to
differentiate between lower respiratory tract diseases and URTIs
with no lower respiratory tract involvement, and achieved 99%
accuracy when distinguishing between patients with a lower
respiratory tract disease and subjects with no discernible respiratory tract disease.
Traditional diagnostic techniques rely on chest auscultation (listening with a stethoscope), often followed by additional observations (such as oxygen saturation) and tests (chest x-ray, blood
tests, sputum tests). The dataset of 218 lower respiratory tract
disease cases includes 24 cases where auscultation by experienced
paediatric clinical teams was unclear. Only after further observations and tests were these 24 cases correctly diagnosed with
lower respiratory tract disease. In 19 of these cases (80 per cent)
ResApp’s algorithms were able to correctly identify lower respiratory tract involvement without the use of additional clinical
observations or additional tests.
89 per cent was the overall accuracy in differentiating between
lower respiratory tract disease, upper respiratory tract infections,
and healthy patients. In particular cases, the accuracy was much
higher: for instance, for the differential diagnosis of croup, viral
pneumonia, bronchiolitis and upper respiratory tract infections
ResApp’s algorithms achieved accuracy levels between 90 and 98
per cent. In a subset of patients that had been initially cleared by
a doctor but diagnosed with lower respiratory tract disease after
additional clinical testing, the technology diagnosed them correctly 97 per cent of the time.
“We are pleased to again report high levels of accuracy on a
dataset that is more than 50 per cent larger than the previously
used dataset,” Dr. Tony Keating, CEO and Managing Director
of ResApp, said in a statement. “These updated results reaffirm
the algorithm’s clinical accuracy right before we enter the pivotal
studies needed for our upcoming premarket submission to the
US Food and Drug Administration. In addition, the preliminary
results for the separation of bacterial and atypical pneumonia
from viral pneumonia are very exciting as they demonstrate the

April/May 2016

power of ResApp’s algorithm in supporting clinicians in making
critical decisions for patient treatment.”
“These results clearly demonstrate that ResApp’s diagnostic tool
outperforms experienced clinicians using stethoscopes and can
match the results provided by an entire suite of expensive, timeconsuming clinical tests,” continues Keating. “We are confident
that ResApp’s accuracy will improve even further as we enrol
more patients.”
Founded in 2014, ResApp Health is developing smartphone
medical applications based on machine learning algorithms that
use sound alone to diagnose and measure the severity of respiratory conditions without the need for additional hardware.
The system, which uses algorithms that were initially developed
by The University of Queensland with funding from the Bill
and Melinda Gates Foundation, has been designed to be used in
a range of circumstances including telehealth service provision,
emergency departments, regular clinic use by healthcare providers, at-home use by consumers and for global aid and humanitarian organisations to deliver tools for the developing world.
Respiratory disease diagnosis using only the sound of a
patient’s cough
The technology was originally developed by Associate Professor
Udantha Abeyratne based on the premise that coughs and breathing sounds carry vital information on the state of the respiratory
tract. By analysing these sounds the technology makes it possible
to diagnose and measure the severity of a wide range of chronic
and acute diseases such as pneumonia, asthma, bronchiolitis
Continued on page 24

The Journal of mHealth

23