Securing the ML Lifecycle
explicitly did not discuss how to increase trustworthiness in ML , although our proposed secure machine learning lifecycle can be considered a fundamental prerequisite .
We propose a simple set of artifacts to support an initial security analysis of a machine learning project .
Our previous discussions emphasized that , at its core , the four main assets in a machine learning lifecycle are the training data , training configuration , resulting trained model , and the final queries to the model and generated results . The usual Confidentiality , Integrity , and Availability triad ( CIA ) still holds true with respect to these assets .
However , the threats differ . Stealing training or configuration data or even the trained model corresponds to some extent to what we know from conventional information security . Yet , the idea of injecting data ( poisoning ) to impact the integrity of the trained model or even causing such a poisoned model to not correctly classify data may not appear immediately apparent . The same applies to an attacker ’ s attempts to infer knowledge about the involved assets .
Figure 5-1 can be used to support such a discussion and allow us to ask questions such as :
• “ Is stealing a confidentially trained model possible in our ML pipeline ?”
• “ Can an attacker poison the training data to reduce the integrity of the trained model ?”
• “ Can an adversarial model evade our classifier due to leaked confidential knowledge about the features used for training ?”
We acknowledge that the very simple matrix we propose is heavily asset-centric , and a different approach may be to focus only on data flows . We also assume that any of these attacks could be performed by an adversarial attacker model . To offer simple mnemonic support , we use the acronym SPIE ( Stealing , Poisoning , Inference , Evasion ) to indicate which phase of the ML lifecycle may be primarily ( though not exclusively ) subject to a certain type of attack .
IIC Journal of Innovation 47