AI Trustworthiness Challenges and Opportunities Related to IIoT
exclude demographic groups. These
challenges are related to a lack of
transparency and clarity of AI decisions,
making it hard to trust the AI systems in new
situations. A related example is how a
software-based flight envelope protection
compensation system in the Boeing 737
MAX may have been involved in crashes due
to unexpected behavior that the pilots could
not understand 2 .
I NTRODUCTION
Incorporating Artificial Intelligence (AI,
including Machine Learning) technologies
into Industrial Internet of Things (IIoT)
systems can offer business and technology
advancements such as cost reduction and
better performance. Examples include the
benefits of predictive maintenance leading
to reduced outages, better resource
management and scheduling and enhanced
insights into system usage. 1 AI has also been
used to design physical structures, electronic
components, and has even been used to
perform quality assurance testing of
complex systems.
IoT Trustworthiness is defined in the IIC
Vocabulary 3 as the “degree of confidence
one has that the system performs as
expected with characteristics including
safety, security, privacy, reliability and
resilience in the face of environmental
disturbances, human errors, system faults
and attacks.”
AI technologies may also create new
challenges and risks for IoT systems. Trust in
systems depends on having assurance that
they operate correctly, based on evidence
that can be understood. Trust and evidence
in AI leading to trust in systems are essential,
especially in complex systems that are not
easily understood. Some AI systems make it
hard or impossible to understand how a
decision was made, reducing trust in the
system. A related challenge is the need to
prepare and select data properly for training
supervised learning systems. If the data has
been “poisoned” by an attacker or simply
inadvertently is incomplete or skewed, then
the results of the trained system may be
inappropriate. An example of bias is
historical data leading to loan decisions that
This paper describes the risks and challenges
AI can pose to the trustworthiness of an IoT
system as well as how AI can be used to
enhance the trustworthiness of a system. It
is noteworthy that the same technologies
that can lead to trust concerns may also be
applied to improve the trust in systems and
to mitigate risks. Safety, security and
reliability can be improved through the
appropriate use of AI technologies since they
can enable faster response and adaptability
of a system to unforeseen situations. Such
adaptability may itself introduce a loss of
predictability and explainability of the
decisions, so this concern needs to be
1
Throughout this paper we won’t be concerned about the detailed distinctions between Artificial Intelligence and Machine
Learning but treat them as one general area except where necessary.
2 https://www.bloomberg.com/news/articles/2019-05-07/boeing-max-failed-to-apply-safety-lesson-from-deadly-2009-crash
3 IIC Vocabulary 2.1, https://www.iiconsortium.org/pdf/IIC_Vocab_Technical_Report_2.1.pdf
IIC Journal of Innovation
- 76 -