Remi Desmartin
ABSTRACT
Neural networks are increasingly relied upon as components of complex safety-critical systems such as autonomous vehicles. There is high demand for tools and methods that embed neural network verification in a larger verification cycle. However, neural network verification is difficult due to a wide range of verification properties of interest, each typically only amenable to verification in specialised solvers. In this paper, we show how Imandra, a functional programming language and a theorem prover originally designed for verification, validation and simulation of financial infrastructure can offer a holistic infrastructure for neural network verification. We develop a novel library CheckINN that formalises neural networks in Imandra, and covers different important facets of neural network verification.
- Edward W. Ayers, Francisco Eiras, Majd Hawasly, and Iain Whiteside. 2020. PaRoT: A Practical Framework for Robust Deep Neural Network Training. In NASA Formal Methods - 12th International Symposium, NFM 2020, Moffett Field, CA, USA, May 11-15, 2020, Proceedings(LNCS, Vol. 12229). Springer, 63–84.Google Scholar
- A. Bagnall and G. Stewart. 2019. Certifying True Error: Machine Learning in Coq with Verified Generalisation Guarantees. AAAI (2019).Google Scholar
- R. Boyer and J. Moore. 1979. A Computational Logic. ACM Monograph Series. Academic Press, New York.Google Scholar
- Robert S. Boyer and J Strother Moore. 1988. Integrating decision procedures into heuristic theorem provers: a case study of linear arithmetic. Machine intelligence(1988), 83–124.Google Scholar
- Marco Casadio, Ekaterina Komendantskaya, Matthew L. Daggitt, Wen Kokke, Guy Katz, Guy Amir, and Idan Refaeli. 2022. Neural Network Robustness as a Verification Property: A Principled Case Study. In Computer Aided Verification (CAV 2022)(Lecture Notes in Computer Science). Springer.Google Scholar
- François Chollet 2015. Keras. https://keras.io.Google Scholar
- Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming(ICFP ’00). Association for Computing Machinery, New York, NY, USA, 268–279. https://doi.org/10.1145/351240.351266Google ScholarDigital Library
- Leonardo de Moura and Grant Olney Passmore. 2013. Computation in Real Closed Infinitesimal and Transcendental Extensions of the Rationals. In CADE.Google Scholar
- Remi Desmartin, Grant Passmore, Ekaterina Kmendantskaya, and Matthew L. Daggitt. 2022. CNN Library in Imandra. https://github.com/aisec-private/ImandraNN.Google Scholar
- Remi Desmartin, Grant Passmore, and Ekaterina Komendantskaya. 2022. Neural Networks in Imandra: Matrix Representation as a Verification Choice. https://arxiv.org/abs/2205.09556.Google Scholar
- Remi Desmartin, Grant Passmore, Ekaterina Komendantskaya, and Matthew Daggitt. 2022. CheckINN: Wide Range Neural Network Verification in Imandra (Extended). https://doi.org/10.48550/arXiv.2207.10562 arXiv:2207.10562 [cs].Google Scholar
- Kirsty Duncan, Ekaterina Komendantskaya, Robert J. Stewart, and Michael A. Lones. 2020. Relative Robustness of Quantized Neural Networks Against Adversarial Attacks. In 2020 International Joint Conference on Neural Networks, IJCNN 2020, Glasgow, United Kingdom, July 19-24, 2020. 1–8. https://doi.org/10.1109/IJCNN48605.2020.9207596Google Scholar
- Bruno Dutertre and Leonardo de Moura. 2006. A Fast Linear-Arithmetic Solver for DPLL(T). In Computer Aided Verification. Springer Berlin Heidelberg, 81–94.Google Scholar
- Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin T. Vechev. 2019. DL2: Training and Querying Neural Networks with Logic. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 1931–1941. http://proceedings.mlr.press/v97/fischer19a.htmlGoogle Scholar
- T. Gehr, M. Mirman, D. Drachsler-Cohen, E. Tsankov, S. Chaudhuri, and M. Vechev. 2018. AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. In S&P.Google Scholar
- Dan R. Ghica and Todd Waugh Ambridge. 2021. Global Optimisation with Constructive Reals. In 36th Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2021, Rome, Italy, June 29 - July 2, 2021. 1–13.Google Scholar
- P. W. Grant, J. A. Sharp, M. F. Webster, and X. Zhang. 1996. Sparse matrix representations in a functional language. Journal of Functional Programming 6, 1 (Jan. 1996), 143–170. https://doi.org/10.1017/S095679680000160X Publisher: Cambridge University Press.Google ScholarCross Ref
- Jónathan Heras, María Poza, Maxime Dénès, and Laurence Rideau. 2011. Incidence Simplicial Matrices Formalized in Coq/SSReflect. In Intelligent Computer Mathematics(Lecture Notes in Computer Science), James H. Davenport, William M. Farmer, Josef Urban, and Florian Rabe (Eds.). Springer, Berlin, Heidelberg, 30–44. https://doi.org/10.1007/978-3-642-22673-1_3Google Scholar
- Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2017. Safety Verification of Deep Neural Networks. In Computer Aided Verification - 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 10426). 3–29.Google Scholar
- Kai Jia and Martin Rinard. 2021. Exploiting Verified Neural Networks via Floating Point Numerical Error. In Static Analysis - 28th International Symposium, SAS 2021, Chicago, IL, USA, October 17-19, 2021, Proceedings(Lecture Notes in Computer Science, Vol. 12913). Springer, 191–205.Google Scholar
- Dejan Jovanović and Leonardo de Moura. 2013. Solving non-linear arithmetic. ACM Communications in Computer Algebra 46, 3/4 (Jan. 2013), 104.Google ScholarDigital Library
- G. Katz, C. Barrett, D. Dill, K. Julian, and M. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In CAV.Google Scholar
- Guy Katz, Clark Barrett, David Dill, Kyle Julian, and Mykel Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. arXiv:1702.01135 [cs] (May 2017). http://arxiv.org/abs/1702.01135 arXiv:1702.01135.Google Scholar
- Guy Katz, Derek A. Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljic, David L. Dill, Mykel J. Kochenderfer, and Clark W. Barrett. 2019. The Marabou Framework for Verification and Analysis of Deep Neural Networks. In CAV 2019, Part I(LNCS, Vol. 11561). Springer, 443–452.Google Scholar
- Wen Kokke, Ekaterina Komendantskaya, Daniel Kienitz, Robert Atkey, and David Aspinall. 2020. Neural Networks, Secure by Construction - An Exploration of Refinement Types. In Programming Languages and Systems - 18th Asian Symposium, APLAS 2020, Fukuoka, Japan, November 30 - December 2, 2020, Proceedings(Lecture Notes in Computer Science, Vol. 12470). Springer, 67–85.Google Scholar
- Alexander Kozlov, Ivan Lazarevich, Vasily Shamporov, Nikolay Lyalyushkin, and Yury Gorbachev. 2021. Neural Network Compression Framework for Fast Model Inference. In Intelligent Computing, Kohei Arai (Ed.). Springer International Publishing, Cham, 213–232.Google Scholar
- Raghuraman Krishnamoorthi. 2018. Quantizing deep convolutional networks for efficient inference: A whitepaper. CoRR abs/1806.08342(2018). arXiv:1806.08342http://arxiv.org/abs/1806.08342Google Scholar
- Ori Lahav and Guy Katz. 2021. Pruning and Slicing Neural Networks using Formal Verification. In Formal Methods in Computer Aided Design, FMCAD 2021, New Haven, CT, USA, October 19-22, 2021. 1–10. https://doi.org/10.34727/2021/isbn.978-3-85448-046-4_27Google Scholar
- Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=rJzIBfZAbGoogle Scholar
- Elisabetta De Maria, Abdorrahim Bahrami, Thibaud L’Yvonnet, Amy P. Felty, Daniel Gaffé, Annie Ressouche, and Franck Grammont. 2022. On the use of formal methods to model and verify neuronal archetypes. Frontiers Comput. Sci. 16, 3 (2022), 163404.Google ScholarDigital Library
- Grant Olney Passmore. 2021. Some Lessons Learned in the Industrialization of Formal Methods for Financial Algorithms. In Formal Methods - 24th International Symposium, FM 2021, Virtual Event, November 20-26, 2021, Proceedings(Lecture Notes in Computer Science, Vol. 13047). Springer, 717–721.Google Scholar
- Grant O. Passmore, Simon Cruanes, Denis Ignatovich, Dave Aitken, Matt Bray, Elijah Kagan, Kostya Kanishev, Ewen Maclean, and Nicola Mometto. 2020. The Imandra Automated Reasoning System (System Description). In Automated Reasoning - 10th International Joint Conference, IJCAR 2020, Paris, France, July 1-4, 2020, Proceedings, Part II, Vol. 12167. Springer, 464–471.Google Scholar
- Connor Shorten and Taghi M. Khoshgoftaar. 2019. A survey on Image Data Augmentation for Deep Learning. Journal of Big Data 6, 1 (July 2019). https://doi.org/10.1186/s40537-019-0197-0 Publisher: Springer Science and Business Media LLC.Google ScholarCross Ref
- Joseph Sill. 1998. Monotonic Networks. California Institute of Technology.Google Scholar
- Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin T. Vechev. 2019. An abstract domain for certifying neural networks. PACMPL 3, POPL (2019), 41:1–41:30. https://doi.org/10.1145/3290354Google ScholarDigital Library
- Natalia Slusarz, Ekaterina Komendantskaya, Matthew L. Daggitt, and Robert Stewart. 2022. Differentiable Logics for Neural Network Training and Verification. https://doi.org/10.48550/arXiv.2207.06741 arXiv:2207.06741 [cs].Google Scholar
- Joseph Tassarotti, Koundinya Vajjha, Anindya Banerjee, and Jean-Baptiste Tristan. 2021. A formal proof of PAC learnability for decision stumps. In CPP ’21: 10th ACM SIGPLAN International Conference on Certified Programs and Proofs, Virtual Event, Denmark, January 17-19, 2021. ACM, 5–17.Google ScholarDigital Library
- Antoine Wehenkel and Gilles Louppe. 2019. Unconstrained Monotonic Neural Networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 1543–1553.Google Scholar
- James Wood. 2019. Vectors and Matrices in Agda. https://personal.cis.strath.ac.uk/james.wood.100/blog/html/VecMat.htmlGoogle Scholar
Index Terms
- CheckINN: Wide Range Neural Network Verification in Imandra
Recommendations
NNV 2.0: The Neural Network Verification Tool
Computer Aided VerificationAbstractThis manuscript presents the updated version of the Neural Network Verification (NNV) tool. NNV is a formal verification software tool for deep learning models and cyber-physical systems with neural network components. NNV was first introduced as ...
Neural Network Verification Using Residual Reasoning
Software Engineering and Formal MethodsAbstractWith the increasing integration of neural networks as components in mission-critical systems, there is an increasing need to ensure that they satisfy various safety and liveness requirements. In recent years, numerous sound and complete ...
Branch and Bound for Sigmoid-Like Neural Network Verification
Formal Methods and Software EngineeringAbstractThe robustness of deep neural networks has received extensive attention and is considered to need guarantees by formal verification. For ReLU neural network verification, there are abundant studies and various techniques. However, verifying ...
Comments