Benoît Coqueret
ABSTRACT
Artificial intelligence, and specifically deep neural networks (DNNs), has rapidly emerged in the past decade as the standard for several tasks from specific advertising to object detection. The performance offered has led DNN algorithms to become a part of critical embedded systems, requiring both efficiency and reliability. In particular, DNNs are subject to malicious examples designed in a way to fool the network while being undetectable to the human observer: the adversarial examples. While previous studies propose frameworks to implement such attacks in black box settings, those often rely on the hypothesis that the attacker has access to the logits of the neural network, breaking the assumption of the traditional black box. In this paper, we investigate a real black box scenario where the attacker has no access to the logits. In particular, we propose an architecture-agnostic attack which solve this constraint by extracting the logits. Our method combines hardware and software attacks, by performing a side-channel attack that exploits electromagnetic leakages to extract the logits for a given input, allowing an attacker to estimate the gradients and produce state-of-the-art adversarial examples to fool the targeted neural network. Through this example of adversarial attack, we demonstrate the effectiveness of logits extraction using side-channel as a first step for more general attack frameworks requiring either the logits or the confidence scores.
Supplemental Material
- Mart'in Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016b. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (Savannah, GA, USA) (OSDI'16). USENIX Association, USA, 265--283.Google Scholar
- Martín Abadi, Paul Barham, Jianmin Chen, Z. Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek Gordon Murray, Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zhang. 2016a. TensorFlow: A system for large-scale machine learning. ArXiv , Vol. abs/1605.08695 (2016).Google Scholar
- Tarek Abdelzaher, Nora Ayanian, Tamer Basar, Suhas Diggavi, Jana Diesner, Deepak Ganesan, Ramesh Govindan, Susmit Jha, Tancrede Lepoint, Benjamin Marlin, Klara Nahrstedt, David Nicol, Raj Rajkumar, Stephen Russell, Sanjit Seshia, Fei Sha, Prashant Shenoy, Mani Srivastava, Gaurav Sukhatme, Ananthram Swami, Paulo Tabuada, Don Towsley, Nitin Vaidya, and Venu Veeravalli. 2018. Toward an Internet of Battlefield Things: A Resilience Perspective. Computer, Vol. 51, 11 (2018), 24--36. https://doi.org/10.1109/MC.2018.2876048Google ScholarDigital Library
- Cédric Archambeau, Eric Peeters, Francc ois-Xavier Standaert, and Jean-Jacques Quisquater. 2006. Template Attacks in Principal Subspaces. In Cryptographic Hardware and Embedded Systems - CHES 2006, Louis Goubin and Mitsuru Matsui (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 1--14.Google Scholar
- Pallavi Sunil Bangare and Kishor P. Patil. 2022. Security Issues and Challenges in Internet of Things (IOT) System. In 2022 2nd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE). 91--94. https://doi.org/10.1109/ICACITE53722.2022.9823709Google ScholarCross Ref
- Lejla Batina, Shivam Bhasin, Dirmanto Jap, and Stjepan Picek. 2019. CSI NN: Reverse Engineering of Neural Network Architectures Through Electromagnetic Side Channel. In USENIX Security Symposium.Google Scholar
- George Becker, Jeremy Cooper, Elke DeMulder, Gilbert Goodwill, Joshua Jaffe, Gary Kenworthy, Tim Kouzminov, Andrew Leiserson, Mark Marson, Pankaj Rohatgi, and Sami Saab. 2013. Test Vector Leakage Assessment ( TVLA ) methodology in practice ( Extended Abstract ).Google Scholar
- Ryad Benadjila, Emmanuel Prouff, Ré mi Strullu, Eleonora Cagli, and Cé cile Dumas. 2020. Deep learning for side-channel analysis and introduction to ASCAD database. Journal of Cryptographic Engineering, Vol. 10, 2 (2020), 163--188. https://doi.org/10.1007/s13389-019-00220--8Google ScholarCross Ref
- Rémi Bernhard, Pierre-Alain Moëllic, and Jean-Max Dutertre. 2019. Impact of Low-Bitwidth Quantization on the Adversarial Robustness for Embedded Neural Networks. 2019 International Conference on Cyberworlds (CW) (2019), 308--315.Google Scholar
- Wieland Brendel, Jonas Rauber, and Matthias Bethge. 2017. Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models. ArXiv , Vol. abs/1712.04248 (2017).Google Scholar
- Nicolas Bruneau, Sylvain Guilley, Annelie Heuser, Damien Marion, and Olivier Rioul. 2017. Optimal side-channel attacks for multivariate leakages and multiple models. Journal of Cryptographic Engineering , Vol. 7 (2017), 331--341.Google ScholarCross Ref
- Eleonora Cagli, Cé cile Dumas, and Emmanuel Prouff. 2017a. Convolutional Neural Networks with Data Augmentation Against Jitter-Based Countermeasures - Profiling Attacks Without Pre-processing. In Cryptographic Hardware and Embedded Systems - CHES 2017 - 19th International Conference, Taipei, Taiwan, September 25--28, 2017, Proceedings (Lecture Notes in Computer Science, Vol. 10529), Wieland Fischer and Naofumi Homma (Eds.). Springer, 45--68. https://doi.org/10.1007/978--3--319--66787--4_3Google ScholarCross Ref
- Eleonora Cagli, Cécile Dumas, and Emmanuel Prouff. 2017b. Kernel Discriminant Analysis for Information Extraction in the Presence of Masking. In Smart Card Research and Advanced Applications, Kerstin Lemke-Rust and Michael Tunstall (Eds.). Springer International Publishing, Cham, 1--22.Google Scholar
- Nicholas Carlini and David A. Wagner. 2016. Towards Evaluating the Robustness of Neural Networks. 2017 IEEE Symposium on Security and Privacy (SP) (2016), 39--57.Google Scholar
- Suresh Chari, Josyula R. Rao, and Pankaj Rohatgi. 2002. Template Attacks. In Cryptographic Hardware and Embedded Systems - CHES 2002, 4th International Workshop, Redwood Shores, CA, USA, August 13--15, 2002, Revised Papers (Lecture Notes in Computer Science, Vol. 2523). Springer, 13--28. https://doi.org/10.1007/3--540--36400--5_3Google ScholarCross Ref
- Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. 2017. ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models. Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security (2017).Google ScholarDigital Library
- Marios Choudary and Markus Kuhn. 2015. Efficient Stochastic Methods: Profiled Attacks Beyond 8 Bits. In Smart Card Research and Advanced Applications, Marc Joye and Amir Moradi (Eds.). Springer International Publishing, Cham, 85--103.Google Scholar
- Omar Choudary and Markus G. Kuhn. 2014. Efficient Template Attacks. In Smart Card Research and Advanced Applications, Aurélien Francillon and Pankaj Rohatgi (Eds.). Springer International Publishing, Cham, 253--270.Google Scholar
- Lei Deng, Guoqi Li, Song Han, Luping Shi, and Yuan Xie. 2020. Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey. Proc. IEEE, Vol. 108, 4 (2020), 485--532. https://doi.org/10.1109/JPROC.2020.2976475Google ScholarCross Ref
- Mathieu Dumont, Kevin Hector, Pierre-Alain Moellic, Jean-Max Dutertre, and Simon Pontié. 2023. Evaluation of Parameter-based Attacks against Embedded Neural Networks with Laser Injection. arxiv: 2304.12876 [cs.CR]Google Scholar
- Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (2015).Google ScholarDigital Library
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org/Google ScholarDigital Library
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014a. Generative Adversarial Nets. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdfGoogle Scholar
- Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014b. Explaining and Harnessing Adversarial Examples. CoRR , Vol. abs/1412.6572 (2014).Google Scholar
- Song Han, Huizi Mao, and William J. Dally. 2015. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding. arXiv: Computer Vision and Pattern Recognition (2015).Google Scholar
- Annelie Heuser, Olivier Rioul, and Sylvain Guilley. 2014. Good Is Not Good Enough. In Cryptographic Hardware and Embedded Systems -- CHES 2014, Lejla Batina and Matthew Robshaw (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 55--74.Google ScholarDigital Library
- Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the Knowledge in a Neural Network. ArXiv , Vol. abs/1503.02531 (2015).Google Scholar
- Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. ArXiv , Vol. abs/1704.04861 (2017).Google Scholar
- Weiwei Hu and Ying Tan. 2017. Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN. ArXiv , Vol. abs/1702.05983 (2017).Google Scholar
- Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. 2016. Densely Connected Convolutional Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 2261--2269.Google Scholar
- Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box Adversarial Attacks with Limited Queries and Information. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 2137--2146. https://proceedings.mlr.press/v80/ilyas18a.htmlGoogle Scholar
- Andrew Ilyas, Logan Engstrom, and Aleksander Madry. 2019. Prior Convictions: Black-box Adversarial Attacks with Bandits and Priors. In International Conference on Learning Representations. https://openreview.net/forum?id=BkMiWhR5K7Google Scholar
- Matthew Jagielski, Nicholas Carlini, David Berthelot, Alexey Kurakin, and Nicolas Papernot. 2019. High Accuracy and High Fidelity Extraction of Neural Networks. In USENIX Security Symposium.Google Scholar
- Raphaël Joud, Pierre-Alain Moëllic, Simon Pontié, and Jean-Baptiste Rigaud. 2023 a. A Practical Introduction to Side-Channel Extraction of Deep Neural Network Parameters. In Smart Card Research and Advanced Applications, Ileana Buhan and Tobias Schneider (Eds.). Springer International Publishing, Cham, 45--65.Google Scholar
- Raphaël Joud, Pierre-Alain Moëllic, Simon Pontié, and Jean-Baptiste Rigaud. 2023 b. A Practical Introduction to Side-Channel Extraction of Deep Neural Network Parameters. In Smart Card Research and Advanced Applications, Ileana Buhan and Tobias Schneider (Eds.). Springer International Publishing, Cham, 45--65.Google Scholar
- Diederik Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6980Google Scholar
- Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2016. Adversarial examples in the physical world. ArXiv , Vol. abs/1607.02533 (2016).Google Scholar
- Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE, Vol. 86, 11 (1998), 2278--2324. https://doi.org/10.1109/5.726791Google ScholarCross Ref
- Yannan Liu, Lingxiao Wei, Bo Luo, and Qiang Xu. 2017. Fault injection attack on deep neural network. In 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). 131--138. https://doi.org/10.1109/ICCAD.2017.8203770Google ScholarDigital Library
- Jianjia Ma. 2020. A higher-level Neural Network library on Microcontrollers (NNoM). https://doi.org/10.5281/zenodo.4158710Google ScholarCross Ref
- Houssem Maghrebi, Thibault Portigliatti, and Emmanuel Prouff. 2016. Breaking Cryptographic Implementations Using Deep Learning Techniques. In Security, Privacy, and Applied Cryptography Engineering - 6th International Conference, SPACE 2016, Hyderabad, India, December 14--18, 2016, Proceedings (Lecture Notes in Computer Science, Vol. 10076), Claude Carlet, Anwar Hasan, and Vishal Saraswat (Eds.). Springer, 3--26. https://doi.org/10.1007/978--3--319--49445--6_1Google ScholarCross Ref
- Stefan Mangard, Elisabeth Oswald, and Thomas Popp. 2007. Power analysis attacks - revealing the secrets of smart cards. Springer.Google Scholar
- Loïc Masure, Nicolas Belleville, Eleonora Cagli, Marie-Angela Cornélie, Damien Couroussé, Cécile Dumas, and Laurent Maingault. 2020. Deep Learning Side-Channel Analysis on Large-Scale Traces. In Computer Security -- ESORICS 2020, Liqun Chen, Ninghui Li, Kaitai Liang, and Steve Schneider (Eds.). Springer International Publishing, Cham, 440--460.Google ScholarDigital Library
- Loïc Masure, Gaëtan Cassiers, Julien Hendrickx, and François-Xavier Standaert. 2022. Information Bounds and Convergence Rates for Side-Channel Security Evaluators. Cryptology ePrint Archive, Paper 2022/490. https://eprint.iacr.org/2022/490 https://eprint.iacr.org/2022/490.Google Scholar
- Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Kurakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang, Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhav Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, and Rujun Long. 2018. Technical Report on the CleverHans v2.1.0 Adversarial Examples Library. arXiv preprint arXiv:1610.00768 (2018).Google Scholar
- Nicolas Papernot, Patrick Mcdaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2015. The Limitations of Deep Learning in Adversarial Settings. 2016 IEEE European Symposium on Security and Privacy (EuroS&P) (2015), 372--387.Google Scholar
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Curran Associates Inc., Red Hook, NY, USA.Google ScholarDigital Library
- Adnan Siraj Rakin, Md Hafizul Islam Chowdhuryy, Fan Yao, and Deliang Fan. 2021. DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories. 2022 IEEE Symposium on Security and Privacy (SP) (2021), 1157--1174.Google Scholar
- Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. 2015. You Only Look Once: Unified, Real-Time Object Detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 779--788.Google Scholar
- David Rolnick and Konrad Paul Kording. 2019. Reverse-engineering deep ReLU networks. In International Conference on Machine Learning.Google Scholar
- Manuele Rusci, Marco Fariselli, Martin Croome, Francesco Paci, and Eric Flamand. 2023. Accelerating RNN-Based Speech Enhancement on a Multi-core MCU with Mixed FP16-INT8 Post-training Quantization. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Irena Koprinska, Paolo Mignone, Riccardo Guidotti, Szymon Jaroszewicz, Holger Fröning, Francesco Gullo, Pedro M. Ferreira, Damian Roqueiro, Gaia Ceddia, Slawomir Nowaczyk, Jo ao Gama, Rita Ribeiro, Ricard Gavaldà, Elio Masciari, Zbigniew Ras, Ettore Ritacco, Francesca Naretto, Andreas Theissler, Przemyslaw Biecek, Wouter Verbeke, Gregor Schiele, Franz Pernkopf, Michaela Blott, Ilaria Bordino, Ivan Luciano Danesi, Giovanni Ponti, Lorenzo Severini, Annalisa Appice, Giuseppina Andresini, Ibéria Medeiros, Guilherme Gracc a, Lee Cooper, Naghmeh Ghazaleh, Jonas Richiardi, Diego Saldana, Konstantinos Sechidis, Arif Canakoglu, Sara Pido, Pietro Pinoli, Albert Bifet, and Sepideh Pashami (Eds.). Springer Nature Switzerland, Cham, 606--617.Google Scholar
- Ahmad Shawahna, Sadiq M. Sait, and Aiman H. El-Maleh. 2019. FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review. IEEE Access , Vol. 7 (2019), 7823--7859.Google ScholarCross Ref
- Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, D. Erhan, Ian J. Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. CoRR , Vol. abs/1312.6199 (2013).Google Scholar
- Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing Machine Learning Models via Prediction APIs. In USENIX Security Symposium.Google ScholarDigital Library
- Jonathan Uesato, Brendan O'Donoghue, A"aron van den Oord, and Pushmeet Kohli. 2018. Adversarial Risk and the Dangers of Evaluating Against Weak Attacks. ArXiv , Vol. abs/1802.05666 (2018).Google Scholar
- Yaman Umuroglu, Nicholas J. Fraser, Giulio Gambardella, Michaela Blott, Philip Leong, Magnus Jahre, and Kees Vissers. 2017. FINN: A Framework for Fast, Scalable Binarized Neural Network Inference. In Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Monterey, California, USA) (FPGA '17). Association for Computing Machinery, New York, NY, USA, 65--74. https://doi.org/10.1145/3020078.3021744Google ScholarDigital Library
- Chang Yue, Peizhuo Lv, Ruigang Liang, and Kai Chen. 2022. Invisible Backdoor Attacks Using Data Poisoning in the Frequency Domain. ArXiv , Vol. abs/2207.04209 (2022).Google Scholar
- Raphael Zingg and Matthias Rosenthal. 2020. Artificial intelligence on microcontrollers. https://digitalcollection.zhaw.ch/handle/11475/20055 Embedded World Conference 2020, Nürnberg, 25.-27. Februar 2020. ioGoogle Scholar
Index Terms
- When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence
Recommendations
Stateful Detection of Black-Box Adversarial Attacks
SPAI '20: Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial IntelligenceThe problem of adversarial examples, evasion attacks on machine learning classifiers, has proven extremely difficult to solve. This is true even in the black-box threat model, as is the case in many practical settings. Here, the classifier is hosted as ...
Security beyond cybersecurity: side-channel attacks against non-cyber systems and their countermeasures
AbstractSide-channels are unintended pathways within target systems that leak internal information, exploitable via side-channel attack techniques that extract the target information, compromising the system’s security and privacy. Side-channel attacks ...
One-Sided Countermeasures for Side-Channel Attacks Can Backfire
WiSec '18: Proceedings of the 11th ACM Conference on Security & Privacy in Wireless and Mobile NetworksSide-channel attacks are currently one of the most powerful attacks against implementations of cryptographic algorithms. They exploit the correlation between the physical measurements (power consumption, electromagnetic emissions, timing) taken at ...
Comments