Evaluating Security of IIoT Testbeds
method to evaluate the security features of
these type of devices with rigor,
repeatability, and a consistent way of
communicating the findings. In our
evaluations, edge gateways included
features such as next generation firewalls,
intrusion detection, and point-to-point
authentication, resolving most of the threats
encountered in the risk analysis. However,
the practical implementations of these
capabilities in the testbeds were not well
specified and are generally difficult to track
as the testbed progresses.
Distinguishing the Testbed from
Eventual Production Deployment
A specific challenge faced by IIoT testbeds is
answering the question whether they are
evaluating the security of the current
testbed or of the eventual production
deployment of the testbed. While some of
the evaluated testbeds did have a
collaborating
partner
with
security
expertise, others did not. Even then, the
testbeds tried hard to make their testbeds
more secure, though omitting security
requirements in the early stages of the
testbed conceptualization and design makes
it challenging for those implementing
security on the testbed and those evaluating
security.
Evaluating Trustworthiness
In the IISF and in the NIST Framework, 10
trustworthiness is described as the
composition of security, safety, privacy,
reliability, and resiliency. As part of the
questionnaire, the TSCG tasked testbeds to
provide qualitative information on their
concerns related to these characteristics.
Every
testbed
provided
relevant
information, as exemplified by the Retail
Video Analytic Testbed which listed privacy
as an issue, or the Smart Factory Machine
Learning Testbed which noted that safety,
reliability and resilience are important.
However, the testbeds were not able to
quantify the relationship between these
characteristics or if they should be evaluated
separately or together. The negative or
positive effects of security controls on other
characteristics, such as the safety at the
edge or the reliability of a components, is
still an evolving research problem.
10
the
C ONCLUSIONS
In this article, we used two case studies to
describe the current state of the art of
security evaluation of IIoT testbeds within
the IIC. To address the challenges
documented, we are evolving the IIC
Security Working Group’s TSCG’s evaluation
methods to focus on particular security
targets for the testbeds. The current work in
the IIC developing Industrial IoT security
maturity models for testbeds – similar to the
Office of Electricity Delivery & Energy
Reliability: Cybersecurity Capability Maturity
National Institute of Standards and Technology (NIST): CPS PWG Cyber-Physical Systems (CPS) Framework Release 1.0, (2016)
IIC Journal of Innovation
- 61 -