Artificial and Human Intelligence with Digital Twins
In this kind of VR system, the expert can
change to a vantage that the field worker
cannot see. For example, they could view the
system from any angle, go through locked
doors or even go inside of components.
environment around a device and whatever
interacts with the device. The speed is real
time, and connectivity often allows us to
span distances instantly. Advances in
streaming analytics now enable us to
process this real-time data using machine
learning and artificial intelligence.
Unlike the field worker’s digital reality, the
expert’s reality must be created. It could be
created by a 3D artist or with digital artifacts
such as CAD drawings—or some
combination of the two. An artist would
have full control, but utilizing CAD drawings
would be more scalable.
While very simple systems can be twinned
from raw data readings, AI and other
analytical techniques are necessary to make
a human-consumable digital twin of complex
systems. Consider a vehicle. If you see it as
only an object on a map, then a digital twin
can be very simple and easy to digest. There
are only two variables, latitude and
longitude, and the variables are easily
understood by humans. But if you wish to
twin the operations of the vehicle, you are
now dealing with hundreds of megabytes of
data per second and thousands of variables.
While all that data is important for the
operations of the vehicle, that much raw
data would overwhelm the ability of a
human to make sense of it. AI synthesizes
the data so that the digital twin can present
it in a human consumable format.
Conversely, AI enhances the digital twin
experience
by
providing
additional
information about the environment not
otherwise available to the user.
While it is technically possible to convert
CAD drawings directly for use in gaming
engines, CAD drawings tend to be too
detailed for real-time rendering in the
gaming engine style. CAD is purposed
towards creating models that can be handed
to manufacturing or building, while gaming
engines pursue photo realism, believable
lighting and low latency response to changes
in camera position.
Tools exist to optimize CAD drawings for
virtual engines. In the blog post “Design,
build and operate faster with the PiXYZ
Plugin for AEC,” Unity evangelist Kieran
Colenutt outlines a way to perform a
conversion from CAD to model formats
supported by Unity. 8
Underneath the umbrella term of AI are
several specific categories of machine
learning. These are presented in this section
as a toolbox for your digital twin. First, the
general architectural practices of AI are
F ROM I O T S ENSORS TO A RTIFICIAL
I NTELLIGENCE
With IoT, data is collected from sensors on a
device, on neighboring devices, the
8
K. Colenutt, “Design, build and operate faster with the PiXYZ Plugin for AEC,” Unity Blog, Jan 2019. Available:
https://blogs.unity3d.com/2019/01/30/design-build-and-operate-faster-with-the-pixyz-plugin-for-aec/
IIC Journal of Innovation
- 63 -