AI Trustworthiness Challenges and Opportunities Related to IIoT
Figure 3
or other factors. Situations such as these
may make both humans and/or AI systems
face significant challenges in deciding
correctly. In this case attempting to select
the least bad of several bad choices might be
required. Such situations must be
considered at design time and a model and
rationale of decision making provided, to
increase trust in the system.
“bolted-on” later to an existing system but
must be considered throughout the entire
lifecycle from design to final validation
testing.
One of the articles in the September 2018
issue 10 of the IIC Journal of Innovation refers
to the need for monitoring and maintaining
the levels of trustworthiness throughout the
long lifecycle of IoT system. This is one of the
areas where the use of AI may be beneficial.
AI can analyze sensor data representing
operational KPIs in real-time or near-real-
time and provide indications to humans (HIL)
about the state of the various characteristics
of IoT Trustworthiness.
It is not always obvious exactly what AI
systems are learning or why they are making
the decisions they make. One difficulty with
complex systems is that outcomes derived
from decisions cannot be traced back to
specific points in the decision process. Unlike
a decision tree where each decision can be
traced through the logical process, AI
systems are vastly more opaque, leading to
uncertainly on what parameters to modify to
get the expected results, especially in the
case of unintended consequences.
Decisions often must be made in systems
despite
incomplete
or
inconsistent
information. An example is a medical
diagnosis where a decision on a course of
action may be urgently required despite the
inability to gather sufficient information.
Another example may be the operation of a
vehicle – a decision, potentially avoiding a
crash, may be necessary despite a lack of
information because of inadequate visibility
10
Therefore, the model must consider the
context around the system. For example the
impact of pollutants or a chemical plant
explosion should be considered as part of
the
design
context
and
model.
https://www.iiconsortium.org/news/joi-articles/2018-Sept-IoT-Trustworthiness-is-a-Journey_IGnPower.pdf
- 81 -
June 2019