Intelligent Realities For Workers Using Augmented Reality, Virtual Reality and Beyond
With an intelligent reality application, the
digital twin can be overlaid directly on the
physical twin. When a bus rolls in to the
garage, a fleet manager can view important
output from the bus’s digital twin as an AR
overlay. A simple example is showing an
alert because the bus is overdue an oil
change. But the real power of the digital twin
would come from more nuanced cases that
aren’t simple violations of established single-
dimensional rules. Perhaps the bus is within
the accepted ranges across several aspects
of maintenance, but the digital twin sees
that a combination of near violations greatly
increases the risk of a mission-critical failure.
The AR device can communicate to the
manager who can then take appropriate
action. practices immediately through the HMD.
Such learnings can be deployed back in to
workers’ intelligent realities.
Feeding AI from XR Devices This section considers three use cases for
reality analytics along with architectural
solutions to their problems.
While VR doesn’t offer the same connection
to the physical world, a VR HMD can also
communicate position and orientation of the
workers’ heads. Eye tracking is also making it
in to XR products, including HTC Vive Pro
Eye™ and Microsoft HoloLens2.
Such
information can be used to improve the
simulation as well as strengthen the
understanding of how humans would react
in the physical analogue of the virtual
environment.
E XAMPLE U SE C ASES FOR
I NTELLIGENT R EALITY
When a factory deploys a thousand AR
HMDs to workers, they are also deploying at
least a thousand head mounted cameras.
Those cameras are well-positioned to
provide a rich set of video content. Such
content can be piped through computer
vision and then on to machine learning and
other analytical models. In addition to video
from the cameras, HMDs can transmit
precise information about the position and
orientation of the head of the wearer.
Augmented Reality Chess Coach
In this first use case, the work is developing
Science, Technology, Engineering, and
Mathematics (STEM) skills in young children,
and the workers are parent volunteers that
want to share the STEM benefits of chess
with elementary schools.
Chess is known to aid the development of
STEM skills for students as young as
elementary school and elementary school
chess clubs can provide a venue for youth
For
manufacturing,
an
AR-enabled
workplace could generate machine learning
models that are trained based on head
position, gaze, placement of components in
the workspace, and quality outcomes. Once
trained, such a model could detect small
movements and practices that lead to poor
quality outcomes and suggest better
IIC Journal of Innovation
- 12 -