AI Trustworthiness Challenges and Opportunities Related to IIoT
addressed at the design stage. One
approach, for example, is to tag data during
the preparation stage before supervised
learning to include additional information
that is useful for later explanations.
One example is demonstrated in the paper
by Chrabaszcz, Loshchilov and Hutter. In
their paper “Back to Basics: Benchmarking
Canonical Evolution Strategies for Playing
Atari” 4 they describe a method of teaching
computers to play 1980s era video games,
and accidentally discover a previously
unknown exploit in the popular Atari game
Q-Bert. The key point is that AI can be used
to discover exploits that are otherwise hard
to find.
E XAMPLE OF A TTACKING A S YSTEM
U SING AI
AI may be used to probe a system for
vulnerabilities and learn how to attack a
system. This has been demonstrated in a
benign use case of connecting an AI system
to a video game and learning how to defeat
the game in novel ways, as described in this
section. Imagine however, if the game is not
Atari Q-Bert but instead “air traffic control,”
“city traffic light system” or “nuclear power
plant” and the implications should become
clear.
Playing to win and learning to cheat
In 2018, researchers at the University of
Freiburg devised a system whereby an
Artificial Intelligence could learn to play old
Atari video games. Using a method that
simulates natural selection in biology, their
AI system learned how to play a selection of
eight different games using only the video
feed as input.
The notion of using AI to probe a system for
weaknesses comes out of the idea of testing.
One exciting area of research relates to the
automated design and testing of complex
software systems. This capability can
represent a double-edged sword however.
With the right tools, engineers can use AI to
devise novel and robust solutions to a
myriad of problems. With those same tools
potential adversaries can find and exploit
obscure weaknesses to direct sophisticated
attacks.
4
How well did it work? Not only could the AI
play the games well, in some cases it could
outperform humans. This was possible
because the AI system has no concept of
conventional wisdom; instead the AI tries
millions of different strategies without
consideration for how elegant they look. As
a result, the AI was able to discover new and
novel strategies to games that had already
been played by millions of people.
https://arxiv.org/pdf/1802.08842.pdf
- 77 -
June 2019