Skip to main content
SpringerLink
Log in

Verifying Binarized Neural Networks by Angluin-Style Learning

  • Conference paper
  • First Online:
Theory and Applications of Satisfiability Testing – SAT 2019 (SAT 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11628))

Abstract

We consider the problem of verifying the behavior of binarized neural networks on some input region. We propose an Angluin-style learning algorithm to compile a neural network on a given region into an Ordered Binary Decision Diagram (OBDD), using a SAT solver as an equivalence oracle. The OBDD allows us to efficiently answer a range of verification queries, including counting, computing the probability of counterexamples, and identifying common characteristics of counterexamples. We also present experimental results on verifying binarized neural networks that recognize images of handwritten digits.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Bottom-up compilation constructs constants and literals of a knowledge base and then composes them together using the Apply operation [11]. Top-down compilation recursively conditions the knowledge base and then combines the recursive compilations to obtain the final compilation [13, 27].

  2. 2.

    We first train the network using sigmoid activations, and then at test time we replace the sigmoid activations with step activations, while keeping the learned weights.

  3. 3.

    Ignatiev et al. [16] also subsequently proposed to use (prime) implicants to explain the decisions made by neural networks. While they computed implicants directly, we learned the OBDD of a neural network. Having an OBDD not only facilitates the computation of prime implicants, but it also allows model counting to be performed efficiently [12], which provides more powerful tools for analysis, as we shall show.

  4. 4.

    Note that SDDs are known to be exponentially more succinct than OBDDs [3].

References

  1. Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)

    Article  MathSciNet  Google Scholar 

  2. Bailleux, O., Boufkhad, Y.: Efficient CNF encoding of boolean cardinality constraints. In: Rossi, F. (ed.) CP 2003. LNCS, vol. 2833, pp. 108–122. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45193-8_8

    Chapter  MATH  Google Scholar 

  3. Bova, S.: SDDs are exponentially more succinct than OBDDs. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 929–935 (2016)

    Google Scholar 

  4. Bryant, R.E.: Graph-based algorithms for boolean function manipulation. IEEE Trans. Comput. C–35, 677–691 (1986)

    Article  Google Scholar 

  5. Cadoli, M., Donini, F.M.: A survey on knowledge compilation. AI Commun. 10(3–4), 137–150 (1997)

    Google Scholar 

  6. Cheng, C.-H., Nührenberg, G., Huang, C.-H., Ruess, H.: Verification of binarized neural networks via inter-neuron factoring (Short Paper). In: Piskac, R., Rümmer, P. (eds.) VSTTE 2018. LNCS, vol. 11294, pp. 279–290. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03592-1_16

    Chapter  Google Scholar 

  7. Choi, A., Shi, W., Shih, A., Darwiche, A.: Compiling neural networks into tractable Boolean circuits. In: AAAI Spring Symposium on Verification of Neural Networks (VNN19) (2019)

    Google Scholar 

  8. Choi, A., Xue, Y., Darwiche, A.: Same-decision probability: a confidence measure for threshold-based decisions. Int. J. Approximate Reasoning (IJAR) 53(9), 1415–1428 (2012)

    Article  MathSciNet  Google Scholar 

  9. Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or \(-\)1 (2016)

    Google Scholar 

  10. Darwiche, A.: SDD: a new canonical representation of propositional knowledge bases. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI), pp. 819–826 (2011)

    Google Scholar 

  11. Darwiche, A.: Tractable knowledge representation formalisms. In: Tractability: Practical Approaches to Hard Problems, pp. 141–172. Cambridge University Press (2014)

    Google Scholar 

  12. Darwiche, A., Marquis, P.: A knowledge compilation map. JAIR 17, 229–264 (2002)

    Article  MathSciNet  Google Scholar 

  13. Huang, J., Darwiche, A.: The language of search. J. Artif. Intell. Res. 29, 191–219 (2007)

    Article  MathSciNet  Google Scholar 

  14. Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 4107–4115 (2016)

    Google Scholar 

  15. Hull, J.J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550–554 (1994)

    Article  Google Scholar 

  16. Ignatiev, A., Narodytska, N., Marques-Silva, J.: Abduction-based explanations for machine learning models. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) (2019)

    Google Scholar 

  17. Jha, S., Raman, V., Pinto, A., Sahai, T., Francis, M.: On learning sparse boolean formulae for explaining AI decisions. In: Barrett, C., Davies, M., Kahsai, T. (eds.) NFM 2017. LNCS, vol. 10227, pp. 99–114. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57288-8_7

    Chapter  Google Scholar 

  18. Jha, S., Seshia, S.A.: A theory of formal synthesis via inductive learning. Acta Informatica 54(7), 693–726 (2017)

    Article  MathSciNet  Google Scholar 

  19. Kahlert, L., Krüger, F., Manthey, N., Stephan, A.: Riss solver framework v5. 05 (2015)

    Google Scholar 

  20. Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_5

    Chapter  Google Scholar 

  21. Kearns, M., Vazirani, U.V.: An Introduction to Computational Learning Theory. MIT Press, Cambridge (1994)

    Book  Google Scholar 

  22. Koul, A., Fern, A., Greydanus, S.: Learning finite state representations of recurrent policy networks. In: Proceedings of the Seventh International Conference on Learning Representations (ICLR) (2019)

    Google Scholar 

  23. Leofante, F., Narodytska, N., Pulina, L., Tacchella, A.: Automated verification of neural networks: advances, challenges and perspectives. CoRR abs/1805.09938 (2018). http://arxiv.org/abs/1805.09938

  24. Meinel, C., Theobald, T.: Algorithms and Data Structures in VLSI Design: OBDD - Foundations and Applications. Springer, Heidelberg (1998)

    Book  Google Scholar 

  25. Nakamura, A.: An efficient query learning algorithm for ordered binary decision diagrams. Inf. Comput. 201(2), 178–198 (2005)

    Article  MathSciNet  Google Scholar 

  26. Narodytska, N., Kasiviswanathan, S.P., Ryzhyk, L., Sagiv, M., Walsh, T.: Verifying properties of binarized deep neural networks. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI) (2018)

    Google Scholar 

  27. Oztok, U., Darwiche, A.: A top-down compiler for sentential decision diagrams. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), pp. 3141–3148 (2015)

    Google Scholar 

  28. Pulina, L., Tacchella, A.: An abstraction-refinement approach to verification of artificial neural networks. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 243–257. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_24

    Chapter  Google Scholar 

  29. Selman, B., Kautz, H.A.: Knowledge compilation and theory approximation. J. ACM 43(2), 193–224 (1996)

    Article  MathSciNet  Google Scholar 

  30. Shih, A., Choi, A., Darwiche, A.: Formal verification of Bayesian network classifiers. In: Proceedings of the 9th International Conference on Probabilistic Graphical Models (PGM) (2018)

    Google Scholar 

  31. Shih, A., Choi, A., Darwiche, A.: A symbolic approach to explaining Bayesian network classifiers. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI) (2018)

    Google Scholar 

  32. Tseitin, G.: On the complexity of derivation in propositional calculus. In: Studies in Constructive Mathematics and Mathematical Logic, pp. 115–125 (1968)

    Chapter  Google Scholar 

  33. Wegener, I.: Branching Programs and Binary Decision Diagrams. SIAM, Philadelphia (2000)

    Book  Google Scholar 

  34. Weiss, G., Goldberg, Y., Yahav, E.: Extracting automata from recurrent neural networks using queries and counterexamples. In: Proceedings of the 35th International Conference on Machine Learning (ICML), pp. 5244–5253 (2018)

    Google Scholar 

  35. Weng, T.W., et al.: Towards fast computation of certified robustness for ReLU networks. In: Proceedings of the Thirty-Fifth International Conference on Machine Learning (ICML) (2018)

    Google Scholar 

  36. Zhang, H., Zhang, P., Hsieh, C.J.: RecurJac: an efficient recursive algorithm for bounding jacobian matrix of general neural networks and its applications. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) (2019)

    Google Scholar 

Download references

Acknowledgments

This work has been partially supported by NSF grant #IIS-1514253, ONR grant #N00014-18-1-2561 and DARPA XAI grant #N66001-17-2-4032.

Author information

Authors and Affiliations

  1. Computer Science Department, University of California, Los Angeles, USA

    Andy Shih, Adnan Darwiche & Arthur Choi

Authors
  1. Andy Shih
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adnan Darwiche
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arthur Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andy Shih .

Editor information

Editors and Affiliations

  1. University of Lisbon, Lisbon, Portugal

    Mikoláš Janota

  2. University of Lisbon, Lisbon, Portugal

    Inês Lynce

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shih, A., Darwiche, A., Choi, A. (2019). Verifying Binarized Neural Networks by Angluin-Style Learning. In: Janota, M., Lynce, I. (eds) Theory and Applications of Satisfiability Testing – SAT 2019. SAT 2019. Lecture Notes in Computer Science(), vol 11628. Springer, Cham. https://doi.org/10.1007/978-3-030-24258-9_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-24258-9_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-24257-2

  • Online ISBN: 978-3-030-24258-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation