Artificial and Human Intelligence with Digital Twins
details how a machine learning model can be
used to classify the state of a robot motor
which can then be presented to factory
personnel with AR. 3 This article applies the
blueprint concepts to facilities management
after first exploring each concept in depth.
While the various streams of data reach their
conclusions in human perception, the
starting point of a digital twin for a user is
how it is perceived. Thus, the starting point
for this exploration are user interfaces for
digital twins, followed by a discussion of AI.
The operations of a physical twin can be
digitized by sensors, cameras and other such
devices, but those digital streams are not the
only sources of data that can feed the digital
twin. In addition to streaming data,
accumulated historical data can inform a
digital twin. Relevant data could include data
not generated from the asset itself, such as
weather and business cycle data. Also, CAD
drawings and other documentation can help
the digital twin provide context. AI and other
analytical models can take raw data and
process it into forms that help humans
understand the system. Additionally, AI can
make intelligent choices of content on behalf
of the user. Such guidance could be very
welcome to users because user input
facilities are very different from the typical
keyboard and mouse. Finally, as displayed in
the upper right corner of Figure 1, humans
can perceive the system as an intelligent
reality—a technologically enhanced reality
that can aid their cognition and judgement. 2
Human Reality of Digital Twins
Humans have a long history of interfacing
with data through a canon of data
visualization, starting with William Playfair’s
inventions of line, bar and pie charts in the
late 1700’s. Digital twins can present data in
such familiar forms, but the traditions of the
late eighteenth century should not unduly
restrain the expressive power of a digital
twin.
When using mobile technologies such as
tablets, smart phones and AR headsets, the
digital reality is overlaid directly on the
physical reality into a single view, as shown
in Figure 2. AR headsets may be the obvious
choice for this use case, but it is not the only
one. Traditional interfaces rendering 3D
models also allow workers to take advantage
of digital twins.
With the blueprint in Figure 1 as a basis, this
article explores how to create digital twins
that utilize AI and reality technologies to
achieve operational benefits. Any number of
operations could be enhanced with the
techniques described here. For example, the
paper “Augmented Reality (AR) Predictive
Maintenance
System with
Artificial
Intelligence (AI) for Industrial Mobile Robot”
2
M. Thomas, “Intelligent Realities For Workers Using Augmented Reality, Virtual Reality and Beyond.” The Industrial Internet
Consortium Journal of Innovation, Mar 2019. Available: https://www.iiconsortium.org/news/joi-articles/2019-March-Intelligent-
Realities-For-Workers-Using-Augmented-Reality-Virtual-Reality-and-Beyond.pdf
3
Y. Tay et al., “Augmented Reality (AR) Predictive Maintenance System with Artificial Intelligence (AI) for Industrial Mobile
Robot.” SAS Global Forum 2019. Available: https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-
proceedings/2019/3628-2019.pdf
IIC Journal of Innovation
- 57 -