Skip to main content
SpringerLink
Log in
Book cover

International Conference on Quantitative Evaluation of Systems

QEST 2019: Quantitative Evaluation of Systems pp 165–181Cite as

Fast Falsification of Hybrid Systems Using Probabilistically Adaptive Input

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11785))

Abstract

We present an algorithm that quickly finds falsifying inputs for hybrid systems, i.e., inputs that steer the system towards violation of a given temporal logic requirement. Our method is based on a probabilistically directed search of an increasingly fine grained spatial and temporal discretization of the input space. A key feature is that it adapts to the difficulty of a problem at hand, specifically to the local complexity of each input segment, as needed for falsification. In experiments with standard benchmarks, our approach shows comparable or better performance to existing techniques, while at the same time being relatively simple.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/ERATOMMSD/falstar.

  2. 2.

    https://github.com/decyphir/breach release version 1.2.9.

References

  1. Adimoolam, A., Dang, T., Donzé, A., Kapinski, J., Jin, X.: Classification and coverage-based falsification for embedded control systems. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 483–503. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_24

    Chapter  Google Scholar 

  2. Akazaki, T.: falsification of conditional safety properties for cyber-physical systems with gaussian process regression. In: Falcone, Y., Sánchez, C. (eds.) RV 2016. LNCS, vol. 10012, pp. 439–446. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46982-9_27

    Chapter  Google Scholar 

  3. Akazaki, T., Hasuo, I.: Time robustness in MTL and expressivity in hybrid system falsification. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9207, pp. 356–374. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21668-3_21

    Chapter  Google Scholar 

  4. Annpureddy, Y., Liu, C., Fainekos, G., Sankaranarayanan, S.: S-TaLiRo: a tool for temporal logic falsification for hybrid systems. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 254–257. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19835-9_21

    Chapter  MATH  Google Scholar 

  5. Balkan, A., Tabuada, P., Deshmukh, J.V., Jin, X., Kapinski, J.: Underminer: a framework for automatically identifying nonconverging behaviors in black-box system models. ACM Trans. Embed. Comput. Syst. 17(1), 1–28 (2017)

    Article  Google Scholar 

  6. Bartocci, E., et al.: Specification-based monitoring of cyber-physical systems: a survey on theory, tools and applications. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification. LNCS, vol. 10457, pp. 135–175. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75632-5_5

    Chapter  Google Scholar 

  7. Deshmukh, J., Jin, X., Kapinski, J., Maler, O.: Stochastic local search for falsification of hybrid systems. In: Finkbeiner, B., Pu, G., Zhang, L. (eds.) ATVA 2015. LNCS, vol. 9364, pp. 500–517. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24953-7_35

    Chapter  MATH  Google Scholar 

  8. Dokhanchi, A., Yaghoubi, S., Hoxha, B., Fainekos, G.E.: ARCH-COMP17 category report: preliminary results on the falsification benchmarks. In: Frehse, G., Althoff, M. (eds.) Applied Verification of Continuous and Hybrid Systems (ARCH). EPiC Series in Computing, vol. 48, pp. 170–174. EasyChair (2017)

    Google Scholar 

  9. Dokhanchi, A., et al.: ARCH-COMP18 category report: results on the falsification benchmarks. In: Frehse, G. (ed.) Applied Verification of Continuous and Hybrid Systems (ARCH). EPiC Series in Computing, vol. 54, pp. 104–109. EasyChair (2019)

    Google Scholar 

  10. Donzé, A.: Breach, a toolbox for verification and parameter synthesis of hybrid systems. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 167–170. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_17

    Chapter  Google Scholar 

  11. Donzé, A., Ferrère, T., Maler, O.: Efficient robust monitoring for STL. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 264–279. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_19

    Chapter  Google Scholar 

  12. Donzé, A., Maler, O.: Robust satisfaction of temporal logic over real-valued signals. In: Chatterjee, K., Henzinger, T.A. (eds.) FORMATS 2010. LNCS, vol. 6246, pp. 92–106. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15297-9_9

    Chapter  MATH  Google Scholar 

  13. Dreossi, T., Dang, T., Donzé, A., Kapinski, J., Jin, X., Deshmukh, J.V.: Efficient guiding strategies for testing of temporal properties of hybrid systems. In: Havelund, K., Holzmann, G., Joshi, R. (eds.) NFM 2015. LNCS, vol. 9058, pp. 127–142. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17524-9_10

    Chapter  Google Scholar 

  14. Eddeland, J., Miremadi, S., Fabian, M., Åkesson, K.: Objective functions for falsification of signal temporal logic properties in cyber-physical systems. In: Conference on Automation Science and Engineering (CASE), pp. 1326–1331. IEEE (2017)

    Google Scholar 

  15. Fainekos, G.E., Pappas, G.J.: Robustness of temporal logic specifications for continuous-time signals. Theor. Comp. Sci. 410(42), 4262–4291 (2009)

    Article  MathSciNet  Google Scholar 

  16. Hoxha, B., Abbas, H., Fainekos, G.E.: Benchmarks for temporal logic requirements for automotive systems. In: Frehse, G., Althoff, M. (eds.) Applied veRification for Continuous and Hybrid Systems (ARCH). EPiC Series in Computing, vol. 34, pp. 25–30. EasyChair (2014)

    Google Scholar 

  17. Hu, J., Lygeros, J., Sastry, S.: Towards a theory of stochastic hybrid systems. In: Lynch, N., Krogh, B.H. (eds.) HSCC 2000. LNCS, vol. 1790, pp. 160–173. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-46430-1_16

    Chapter  MATH  Google Scholar 

  18. Igel, C., Hansen, N., Roth, S.: Covariance matrix adaptation for multi-objective optimization. Evol. Comput. 15(1), 1–28 (2007)

    Article  Google Scholar 

  19. Jegourel, C., Legay, A., Sedwards, S.: Importance splitting for statistical model checking rare properties. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 576–591. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_38

    Chapter  Google Scholar 

  20. Jegourel, C., Legay, A., Sedwards, S.: An effective heuristic for adaptive importance splitting in statistical model checking. In: Margaria, T., Steffen, B. (eds.) ISoLA 2014. LNCS, vol. 8803, pp. 143–159. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45231-8_11

    Chapter  Google Scholar 

  21. Jin, X., Deshmukh, J.V., Kapinski, J., Ueda, K., Butts, K.R.: Powertrain control verification benchmark. In: Fränzle, M., Lygeros, J. (eds.) Hybrid Systems: Computation and Control (HSCC), pp. 253–262. ACM (2014)

    Google Scholar 

  22. Jin, X., Donzé, A., Deshmukh, J.V., Seshia, S.A.: Mining requirements from closed-loop control models. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 34(11), 1704–1717 (2015)

    Article  Google Scholar 

  23. Kapinski, J., Deshmukh, J.V., Jin, X., Ito, H., Butts, K.: Simulation-based approaches for verification of embedded control systems: an overview of traditional and advanced modeling, testing, and verification techniques. IEEE Control Syst. Mag. 36(6), 45–64 (2016)

    Article  MathSciNet  Google Scholar 

  24. LaValle, S.M., Kuffner Jr., J.J.: Randomized kinodynamic planning. Int. J. Robot. Res. (IJRR) 20(5), 378–400 (2001)

    Article  Google Scholar 

  25. Lee, R., Kochenderfer, M.J., Mengshoel, O.J., Brat, G.P., Owen, M.P.: Adaptive stress testing of airborne collision avoidance systems. In: IEEE/AIAA 34th Digital Avionics Systems Conference (DASC 2015), pp. 6C2:1–6C2:13 (2015)

    Google Scholar 

  26. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT press, Cambridge (2018)

    MATH  Google Scholar 

  27. Wolpert, D., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Article  Google Scholar 

  28. Yaghoubi, S., et al.: ARCH-COMP19 category report: results on the falsification benchmarks. In: Frehse, G. (ed.) Applied Verification of Continuous and Hybrid Systems (ARCH). EPiC Series in Computing. EasyChair (2019)

    Google Scholar 

  29. Zhang, Z., Ernst, G., Hasuo, I., Sedwards, S.: Time-staging enhancement of hybrid system falsification. In: 2018 IEEE Workshop on Monitoring and Testing of Cyber-Physical Systems (MT-CPS 2018), pp. 3–4. IEEE, April 2018

    Google Scholar 

  30. Zhang, Z., Ernst, G., Sedwards, S., Arcaini, P., Hasuo, I.: Two-layered falsification of hybrid systems guided by Monte Carlo tree search. In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD 2018) (2018)

    Google Scholar 

  31. Zutshi, A., Deshmukh, J.V., Sankaranarayanan, S., Kapinski, J.: Multiple shooting, CEGAR-based falsification for hybrid systems. In: Embedded Software (EMSOFT), pp. 5:1–5:10 (2014)

    Google Scholar 

Download references

Acknowledgement

This work is supported by the ERATO HASUO Metamathematics for Systems Design Project (No. JPMJER1603), JST; and Grants-in-Aid No. 15KT0012, JSPS.

Author information

Authors and Affiliations

  1. Ludwig-Maximilians-University, Munich, Germany

    Gidon Ernst

  2. University of Waterloo, Waterloo, Canada

    Sean Sedwards

  3. National Institute of Informatics, Tokyo, Japan

    Zhenya Zhang & Ichiro Hasuo

Authors
  1. Gidon Ernst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sean Sedwards
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenya Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ichiro Hasuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gidon Ernst .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    David Parker

  2. Saarland University, Saarbrücken, Germany

    Verena Wolf

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ernst, G., Sedwards, S., Zhang, Z., Hasuo, I. (2019). Fast Falsification of Hybrid Systems Using Probabilistically Adaptive Input. In: Parker, D., Wolf, V. (eds) Quantitative Evaluation of Systems. QEST 2019. Lecture Notes in Computer Science(), vol 11785. Springer, Cham. https://doi.org/10.1007/978-3-030-30281-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30281-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30280-1

  • Online ISBN: 978-3-030-30281-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation