Securing the ML Lifecycle
Attacking the deployed model should be considered from two perspectives , i . e . stealing the model or trying to evade the model .
Stealing the model could again be a rather conventional attack , requiring illegitimate access to files and copying them . But more sophisticated attacks may try to duplicate a model just by posing queries . In a similar fashion , the attacker could try to infer knowledge about the underlying training dataset . These latter two means are often referred to as adversarial attacks , as they are conducted by models trained with the purpose of copying other models 29 .
The attacker may also try and generate adversarial models that work to construct malicious input and yield erroneous model output , while appearing unmodified to human observers 30 . In fact , we are aware that there are many examples of published models used for supporting IT security purposes ( e . g ., phishing detection ) that were immediately attacked by corresponding adversarial models 31 and that successfully evaded the originally proposed and trained classifier .
Models may be made available to customers just like regular software . There are active discussions about how to license trained models as well as training parameters with respect to open-source usage . Technical solutions to embed watermarks in model output also appear to be available 32 . However , we are not aware of any true technical license enforcement embedded in models 33 . Our assumption is that , given the current state of the art , such enforcement can only realistically be done in the operational environment .
29
Ishai Rosenberg , Asaf Shabtai , Yuval Elovici , and Lior Rokach . 2021 . Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain . ACM Comput . Surv . 54 , 5 , Article 108 ( June 2022 ).
30
Nicolas Papernot , Patrick McDaniel , Ian Goodfellow , Somesh Jha , Z . Berkay Celik , and Ananthram Swami . 2017 . Practical Black-Box Attacks against Machine Learning . ASIA CCS ' 17
31
32
Huili Chen , Bita Darvish Rouhani , Cheng Fu , Jishen Zhao , and Farinaz Koushanfar . 2019 . DeepMarks : A Secure Fingerprinting Framework for Digital Rights Management of Deep Learning Models . In Proceedings of the 2019 on International Conference on Multimedia Retrieval ( ICMR ' 19 ).
IIC Journal of Innovation 45