Artificial and Human Intelligence with Digital Twins
analyses by combining multiple streams of
information. A process for implementing a
computer vision model is as follows.
classification model that identifies images
with a high likelihood of known good or
suspected anomaly images. The power
transformer is an example of this.
If possible, fix the camera to a stable mount
point so that all images will be taken from
the same angle and with the same
proportions. This vastly simplifies the model
training as compared to general object
recognition models, which must capture
objects from many angles. The fixed camera
location also simplifies the process of
determining the location of defects on the
piece.
If you have images that have been labeled
with known defect types, you can create a
more complex classification model that
identifies the various defects. The discrete
parts are an example of this. There might be
previous images labeled with an incorrect
bearing insertion and other images labeled
with incorrect part milling.
If you have good location identification, you
can also break down the images and find the
portions of the image with defects. The
semiconductor wafer is an example here.
You can quantify the expected yield based
on the proportion of the wafer with defects.
Another option is to initially create a model
that finds easily identified features on the
piece. For a power substation, you could
have general instructions on how to point
the camera at a transformer in the
substation. An object recognition model
could identify the bushings on the top of the
transformer. This would provide reference
points to scale the images with images
captured at similar angles, similar to how
facial recognition models determine various
key points on a face. 17 18
When you have trained the model, you can
determine at what latency you can infer and
test new images being captured and if you
need to stream image-by-image and get
immediate results. Alternatively, you might
be able to capture a batch of images and
process in batch. Also determine if the
inferencing can be done in the cloud or
server or if an edge gateway is needed.
You will use the images to create a
classification model using Convolutional
neural networks (CNN). Depending on how
well labeled your data is, you can create
models of various complexity.
A PPLICATIONS TO S MART F ACILITIES
Smart facilities offer a perfect example of
how a digital twin can offer features that
cannot be accomplished in the physical
If you primarily have a collection of known
good images, you can create a binary
17 Long,
Xindian. 2018. “Understanding Object Detection in Deep Learning.” Available:
https://blogs.sas.com/content/subconsciousmusings/2018/11/19/understanding-object-detection-in-deep-learning/
18
Long, Xindian. 2019. “Exploring Computer Vision in Deep Learning: Object Detection and Semantic Segmentation.”
Proceedings of the SAS Global Forum 2019 Conference. Cary, NC: SAS Institute Inc. Available:
https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2019/3317-2019.pdf
IIC Journal of Innovation
- 69 -