Skip to main content
SpringerLink
Log in

Safety-Assured Design and Adaptation of Connected and Autonomous Vehicles

  • Chapter
  • First Online:
Machine Learning and Optimization Techniques for Automotive Cyber-Physical Systems

  • 433 Accesses

Abstract

Design and development of connected and autonomous vehicles (CAVs) are accompanied by a growing concern over the safety of these systems. This chapter will survey recent advances in designing and operating CAVs with safety assurance, with a special focus on CAVs that employ neural network-based components. A diverse but interconnected set of techniques on the verification, design, and runtime adaptation of CAVs will be presented, culminating in a discussion of the outstanding challenges that the field faces and of the promising future directions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Althoff, M.: An introduction to cora 2015. In: Proceedings of ARCH’15. EPiC Series in Computer Science, vol. 34, pp. 120–151. EasyChair (2015)

    Google Scholar 

  2. Alur, R., Courcoubetis, C., Halbwachs, N., Henzinger, T.A., Ho, P.-H., Nicollin, X., Olivero, A., Sifakis, J., Yovine, S.: The algorithmic analysis of hybrid systems. Theor. Comput. Sci. 138(1), 3–34 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  3. Balunovic, M., Vechev, M.: Adversarial training and provable defenses: Bridging the gap. In: International Conference on Learning Representations (2020)

    Google Scholar 

  4. Bernat, G., Cayssials, R.: Guaranteed on-line weakly-hard real-time systems. In: IEEE Real-Time Systems Symposium (RTSS) (2001)

    Google Scholar 

  5. Castane, R., Marti, P., Velasco, M., Cervin, A., Henriksson D.: Resource management for control tasks based on the transient dynamics of closed-loop systems. In: 18th Euromicro Conference on Real-Time Systems (ECRTS’06) (2006)

    Google Scholar 

  6. Cervin, A., Eker, J., Bernhardsson, B., Årzén, K.E.: Feedback–feedforward scheduling of control tasks. Real-Time Syst. 23(1), 25–53 (2002)

    Article  MATH  Google Scholar 

  7. Cervin, A., Velasco, M., Marti, P., Camacho, A.: Optimal online sampling period assignment: theory and experiments. IEEE Trans. Control Syst. Technol. 19(4), 902–910 (2011)

    Article  Google Scholar 

  8. Chen, X., Ábrahám, E., Sankaranarayanan, S.: Flow*: an analyzer for non-linear hybrid systems. In: Proceedings of CAV’13. LNCS, vol. 8044, pp. 258–263. Springer (2013)

    Google Scholar 

  9. Chen, X., Sankaranarayanan, S.: Reachability analysis for cyber-physical systems: are we there yet? In: Proceedings of NFM’22. LNCS, vol. 13260, pp. 109–130. Springer (2022)

    Google Scholar 

  10. Chisci, L., Rossiter, J.A., Zappa, G.: Systems with persistent disturbances: predictive control with restricted constraints. Automatica 37(7) (2001)

    Google Scholar 

  11. Dai, X., Chang, W., Zhao, S., Burns, A.: A dual-mode strategy for performance-maximisation and resource-efficient cps design. ACM Trans. Embed. Comput. Syst. 18(5s) (2019)

    Google Scholar 

  12. Davare, A., Zhu, Q., Di Natale, M., Pinello, C., Kanajan, S., Sangiovanni-Vincentelli, A.: Period optimization for hard real-time distributed automotive systems. In: Design Automation Conference (DAC’07) (2007)

    Google Scholar 

  13. Deng, P., Zhu, Q., Davare, A., Mourikis, A., Liu, X., Natale, M.D.: An efficient control-driven period optimization algorithm for distributed real-time systems. IEEE Trans. Comput. 65(12), 3552–3566 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  14. Dutta, S., Jha, S., Sankaranarayanan, S., Tiwari, A.: Output range analysis for deep feedforward neural networks. In: NASA Formal Methods Symposium, pp. 121–138. Springer (2018)

    Google Scholar 

  15. Dutta, S., Jha, S., Sankaranarayanan, S., Tiwari, A.: Output range analysis for deep feedforward neural networks. In: Proceedings of NFM’18. LNCS, vol. 10811, pp. 121–138. Springer (2018)

    Google Scholar 

  16. Dutta, S., Chen, X., Sankaranarayanan, S.: Reachability analysis for neural feedback systems using regressive polynomial rule inference. In: 22nd ACM International Conference on Hybrid Systems: Computation and Control (HSCC), pp. 157–168 (2019)

    Google Scholar 

  17. Dvijotham, K., Stanforth, R., Gowal, S., Mann, T.A., Kohli, P.: A dual approach to scalable verification of deep networks. In: UAI, vol. 1, p. 2 (2018)

    Google Scholar 

  18. Fan, J., Li, W.: Adversarial training and provable robustness: a tale of two objectives. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 7367–7376 (2021)

    Google Scholar 

  19. Fan, J., Huang, C., Li, W., Chen, X., Zhu, Q.: Towards verification-aware knowledge distillation for neural-network controlled systems. In: 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1–8. IEEE (2019)

    Google Scholar 

  20. Fan, J., Huang, C., Chen, X., Li, W., Zhu, Q.: Reachnn*: a tool for reachability analysis of neural-network controlled systems. In: International Symposium on Automated Technology for Verification and Analysis (2020)

    Google Scholar 

  21. Fawzi, A., Moosavi-Dezfooli, S.-M., Frossard, P.: The robustness of deep networks: a geometrical perspective. IEEE Signal Process. Mag. 34(6), 50–62 (2017)

    Article  Google Scholar 

  22. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conferences on Learning Representations (2015)

    Google Scholar 

  23. Gowal, S., Dvijotham, K., Stanforth, R., Bunel, R., Qin, C., Uesato, J., Arandjelovic, R., Mann, T., Kohli, P.: On the effectiveness of interval bound propagation for training verifiably robust models. Preprint (2018). arXiv:1810.12715

    Google Scholar 

  24. Henzinger, T.A., Kopke, P.W., Puri, A., Varaiya, P.: What’s decidable about hybrid automata? In: Proceedings of the 27th Annual ACM Symposium on Theory of Computing (STOC’95), pp. 373–382. ACM (1995)

    Google Scholar 

  25. Huang, X., Kwiatkowska, M., Wang, S., Wu, M.: Safety verification of deep neural networks. In: International Conference on Computer Aided Verification, pp. 3–29. Springer (2017)

    Google Scholar 

  26. Huang, C., Fan, J., Li, W., Chen, X., Zhu, Q.: Reachnn: reachability analysis of neural-network controlled systems. ACM Trans. Embedd. Comput. Syst. 18(5s), 1–22 (2019)

    Article  Google Scholar 

  27. Huang, C., Li, W., Zhu, Q.: Formal verification of weakly-hard systems. In: The 22nd ACM International Conference on Hybrid Systems: Computation and Control (HSCC) (2019)

    Google Scholar 

  28. Huang, C., Chang, K.-C., Lin, C.-W., Zhu, Q.: Saw: a tool for safety analysis of weakly-hard systems. In: 32nd International Conference on Computer-Aided Verification (CAV’20) (2020)

    Google Scholar 

  29. Huang, C., Fan, J., Chen, X., Li, W., Zhu, Q.: Divide and slide: layer-wise refinement for output range analysis of deep neural networks. In: International Conference on Embedded Software (EMSOFT) (2020)

    Google Scholar 

  30. Huang, C., Xu, S., Wang, Z., Lan, S., Li, W., Zhu, Q.: Opportunistic intermittent control with safety guarantees for autonomous systems. Proccedings of the Design Automation Conference (DAC’20) (2020)

    Google Scholar 

  31. Huang, C., Fan, J., Chen, X., Li, W., Zhu, Q.: Polar: a polynomial arithmetic framework for verifying neural-network controlled systems. Preprint (2021). arXiv:2106.13867

    Google Scholar 

  32. Ivanov, R., Weimer, J., Alur, R., Pappas, G.J., Lee, I.: Verisig: verifying safety properties of hybrid systems with neural network controllers. In: 22nd ACM International Conference on Hybrid Systems: Computation and Control (HSCC), pp. 169–178 (2019)

    Google Scholar 

  33. Ivanov, R., Carpenter, T.J., Weimer, J., Alur, R., Pappas, G.J., Lee, I.: Verifying the safety of autonomous systems with neural network controllers. ACM Trans. Embedd. Comput. Syst. (TECS) 20(1), 1–26 (2020)

    Google Scholar 

  34. Ivanov, R., Carpenter, T., Weimer, J., Alur, R., Pappas, G., Lee, I.: Verisig 2.0: verification of neural network controllers using taylor model preconditioning. In: Silva, A., Rustan, K., Leino, M. (eds.) Computer Aided Verification, pp. 249–262. Springer International Publishing, Cham (2021)

    Chapter  MATH  Google Scholar 

  35. Jiao, R., Liang, H., Sato, T., Shen, J., Chen, Q.A., Zhu, Q.: End-to-end uncertainty-based mitigation of adversarial attacks to automated lane centering. In: 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 266–273 (2021)

    Google Scholar 

  36. Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient smt solver for verifying deep neural networks. In: International Conference on Computer Aided Verification (CAV), pp. 97–117. Springer (2017)

    Google Scholar 

  37. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2016)

    Google Scholar 

  38. Lee, D., Hess, D.J.: Public concerns and connected and automated vehicles: safety, privacy, and data security. Hum. Soc. Sci. Commun. 9(1), 1–13 (2022)

    Google Scholar 

  39. Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Adv. Neural Inf. Process. Syst. 31 (2018)

    Google Scholar 

  40. Liang, H., Wang, Z., Roy, D., Dey, S., Chakraborty, S., Zhu, Q.: Security-driven codesign with weakly-hard constraints for real-time embedded systems. In: 37th IEEE International Conference on Computer Design (ICCD’19) (2019)

    Google Scholar 

  41. Liang, H., Wang, Z., Jiao, R., Zhu, Q.: Leveraging weakly-hard constraints for improving system fault tolerance with functional and timing guarantees. In: 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1–9 (2020)

    Google Scholar 

  42. Liu, X., Huang, C., Wang, Y., Zheng, B., Zhu, Q.: Physics-aware safety-assured design of hierarchical neural network based planner. In: 2022 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS) (2022)

    Google Scholar 

  43. Löfberg, J: Minimax Approaches to Robust Model Predictive Control, vol. 812. University Electronic Press, Linköping (2003)

    Google Scholar 

  44. Lu, J., Issaranon, T., Forsyth, D.: Safetynet: detecting and rejecting adversarial examples robustly. In: Proceedings of the IEEE international conference on computer vision, pp. 446–454 (2017)

    Google Scholar 

  45. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. Preprint (2017). arXiv:1706.06083

    Google Scholar 

  46. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: International Conference on Learning Representations (2018)

    Google Scholar 

  47. Makino, K., Berz, M.: Taylor models and other validated functional inclusion methods. J. Pure Appl. Math. 4(4), 379–456 (2003)

    MathSciNet  MATH  Google Scholar 

  48. Mirman, M., Gehr, T., Vechev, M.: Differentiable abstract interpretation for provably robust neural networks. In: International Conference on Machine Learning, pp. 3578–3586 (2018)

    Google Scholar 

  49. Mundhenk, P., Paverd, A., Mrowca, A., Steinhorst, S., Lukasiewycz, M., Fahmy, S.A., Chakraborty, S.: Security in automotive networks: lightweight authentication and authorization. ACM Trans. Des. Autom. Electron. Syst. 22(2), 25:1–25:27 (2017)

    Google Scholar 

  50. Nedialkov, N.S.: Implementing a rigorous ode solver through literate programming. In: Rauh, A., Auer, E. (eds.) Modeling, Design, and Simulation of Systems with Uncertainties. Mathematical Engineering, vol. 3, pp. 3–19. Springer, Berlin/Heidelberg (2011)

    Chapter  Google Scholar 

  51. NHTSA Media.: U.S. transportation secretary elaine l. chao announces first participants in new automated vehicle initiative web pilot to improve safety, testing, public engagement. NHTSA (2020)

    Google Scholar 

  52. Phillips, G.M.: Interpolation and Approximation by Polynomials. Springer, Berlin (2003)

    Book  MATH  Google Scholar 

  53. Prabhakar, P., Afzal, Z.R.: Abstraction based output range analysis for neural networks. In: Advances in Neural Information Processing Systems, pp. 15788–15798 (2019)

    Google Scholar 

  54. Raghunathan, A., Steinhardt, J., Liang, P.S.: Semidefinite relaxations for certifying robustness to adversarial examples. In: Advances in Neural Information Processing Systems, pp. 10877–10887 (2018)

    Google Scholar 

  55. Ramanathan, P.: Overload management in real-time control applications using (m, k)-firm guarantee. IEEE Trans. Parallel Distrib. Syst. 10(6), 549–559 (1999)

    Article  Google Scholar 

  56. Richards, A.G.: Robust constrained model predictive control. Ph.D Thesis, Massachusetts Institute of Technology, 2005

    Google Scholar 

  57. Roy, D., Chang, W., Mitter, S.K., Chakraborty, S.: Tighter dimensioning of heterogeneous multi-resource autonomous cps with control performance guarantees. In: ACM/IEEE Design Automation Conference (DAC), pp. 1–6 (2019)

    Google Scholar 

  58. Ruan, W., Huang, X., Kwiatkowska, M.: Reachability analysis of deep neural networks with provable guarantees. In: International Joint Conferences on Artificial Intelligence (2018)

    Google Scholar 

  59. Sato, T., Shen, J., Wang, N., Jia, Y., Lin, X., Chen, Q.A.: Dirty road can attack: Security of deep learning based automated lane centering under {Physical-World} attack. In: 30th USENIX Security Symposium (USENIX Security 21), pp. 3309–3326 (2021)

    Google Scholar 

  60. Seshia, S.A., Hu, S., Li, W., Zhu, Q.: Design automation of cyber-physical systems: challenges, advances, and opportunities. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 36(9), 1421–1434 (2017)

    Article  Google Scholar 

  61. Seto, D., Krogh, B., Sha, L., Chutinan, A.: The simplex architecture for safe online control system upgrades. In: American Control Conference (ACC), vol. 6, pp. 3504–3508 (1998)

    Google Scholar 

  62. Siddiqui, F., Lerman, R., Merrill, J.B.: Teslas running autopilot involved in 273 crashes reported since last year. The Washington Post (2022)

    Google Scholar 

  63. Singh, G., Gehr, T., Mirman, M., Püschel, M., Vechev, M.: Fast and effective robustness certification. In: Advances in Neural Information Processing Systems, pp. 10802–10813 (2018)

    Google Scholar 

  64. Singh, G., Gehr, T., Püschel, M., Vechev, M.: Boosting robustness certification of neural networks. In: International Conference on Learning Representations (2019)

    Google Scholar 

  65. Summary Report: Standing general order on crash reporting for automated driving systems. Technical Report, NHTSA, 2022

    Google Scholar 

  66. Summary Report: Standing general order on crash reporting for level 2 advanced driver assistance systems. Technical Report, NHTSA, 2022

    Google Scholar 

  67. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks. International Conferences on Learning Representations (2014)

    Google Scholar 

  68. Tjeng, V., Xiao, K.Y., Tedrake, R.: Evaluating robustness of neural networks with mixed integer programming. In: International Conference on Learning Representations (2019)

    Google Scholar 

  69. Tran, H.-D., Bak, S., Xiang, W., Johnson, T.T.: Verification of deep convolutional neural networks using imagestars. In: International Conference on Computer-Aided Verification (2020)

    Google Scholar 

  70. U.S. Department of Transportation: Using connected vehicle technologies to solve real-world operational problems. USDOT ITS Research - Connected Vehicle Pilot Deployment Program (2022)

    Google Scholar 

  71. Wang, S., Pei, K., Whitehouse, J., Yang, J., Jana, S.: Formal security analysis of neural networks using symbolic intervals. In: 27th {USENIX} Security Symposium ({USENIX} Security 18), pp. 1599–1614 (2018)

    Google Scholar 

  72. Wang, Y., Huang, C., Zhu, Q.: Energy-efficient control adaptation with safety guarantees for learning-enabled cyber-physical systems. In: Proceedings of the 39th International Conference on Computer-Aided Design, ICCAD ’20, New York, NY, USA. Association for Computing Machinery (2020)

    Google Scholar 

  73. Wang, S., Zhang, H., Xu, K., Lin, X., Jana, S., Hsieh, C.-J., Kolter, J.Z.: Beta-crown: efficient bound propagation with per-neuron split constraints for neural network robustness verification. In: Proceedings of NeurIPS’21, vol. 34 (2021)

    Google Scholar 

  74. Wang, Y., Huang, C., Wang, Z., Xu, S., Wang, Z., Zhu, Q.: Cocktail: learn a better neural network controller from multiple experts via adaptive mixing and robust distillation. In: 2021 58th ACM/IEEE Design Automation Conference (DAC), pp. 397–402. IEEE (2021)

    Google Scholar 

  75. Wang, Z., Huang, C., Kim, H., Li, W., Zhu, Q.: Cross-layer adaptation with safety-assured proactive task job skipping. ACM Trans. Embed. Comput. Syst. 20(5s) (2021)

    Google Scholar 

  76. Wang, Y., Huang, C., Wang, Z., Wang, Z., Zhu, Q.: Design-while-verify: correct-by-construction control learning with verification in the loop. In: 59th ACM/IEEE Design Automation Conference, DAC 2022, San Francisco, CA, USA, July 10–14 (2022)

    Google Scholar 

  77. Wang, Z., Huang, C., Zhu, Q.: Efficient global robustness certification of neural networks via interleaving twin-network encoding. In: DATE’22: Proceedings of the Conference on Design, Automation and Test in Europe (2022)

    Google Scholar 

  78. Wiggers, K.: Waymo’s driverless cars were involved in 18 accidents over 20 months. VentureBeat (2020)

    Google Scholar 

  79. Wong, E., Kolter, Z.: Provable defenses against adversarial examples via the convex outer adversarial polytope. In: International Conference on Machine Learning, pp. 5286–5295 (2018)

    Google Scholar 

  80. Wong, E., Rice, L., Kolter, J.Z.: Fast is better than free: revisiting adversarial training. In: International Conferences on Learning Representations (2020)

    Google Scholar 

  81. Zheng, B., Gao, Y., Zhu, Q., Gupta, S.: Analysis and optimization of soft error tolerance strategies for real-time systems. In: 2015 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), pp. 55–64 (2015)

    Google Scholar 

  82. Zhang, H., Weng, T.-W., Chen, P.-Y., Hsieh, C.-J., Daniel, L.: Efficient neural network robustness certification with general activation functions. In: Advances in Neural Information Processing Systems, pp. 4939–4948 (2018)

    Google Scholar 

  83. Zhang, H., Chen, H., Xiao, C., Li, B., Boning, D., Hsieh, C.-J.: Towards stable and efficient training of verifiably robust neural networks. In: International Conference on Learning Representations (2020)

    Google Scholar 

  84. Zhou, H., Li, W., Kong, Z., Guo, J., Zhang, Y., Yu, B., Zhang, L., Liu, C.: Deepbillboard: Systematic physical-world testing of autonomous driving systems. In: 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE), pp. 347–358. IEEE (2020)

    Google Scholar 

  85. Zhou, W., Gao, R., Kim, B., Kang, E., Li, W.: Runtime-safety-guided policy repair. In: Deshmukh, J., Ničković, D. (eds.) Runtime Verification, pp. 131–150. Springer International Publishing, Cham (2020)

    Google Scholar 

  86. Zhu, Q., Sangiovanni-Vincentelli, A.: Codesign methodologies and tools for cyber–physical systems. In: Proceedings of the IEEE 106(9), 1484–1500 (2018)

    Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge the support from the US National Science Foundation (NSF) awards CCF-1646497, CCF-1834324, CNS-1834701, CNS-1839511, IIS-1724341, CNS-2038853, the US Office of Naval Research (ONR) grant N00014-19-1-2496, and the US Air Force Research Laboratory (AFRL) under contract number FA8650-16-C-2642.

Author information

Authors and Affiliations

  1. University of Dayton, Dayton, OH, USA

    Xin Chen

  2. Boston University, Boston, MA, USA

    Jiameng Fan, Wenchao Li & Zhilu Wang

  3. University of Liverpool, Liverpool, UK

    Chao Huang

  4. Northwestern University, Evanston, IL, USA

    Ruochen Jiao, Xiangguo Liu, Yixuan Wang, Weichao Zhou & Qi Zhu

Authors
  1. Xin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiameng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruochen Jiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenchao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiangguo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yixuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Zhilu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Weichao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Qi Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qi Zhu .

Editor information

Editors and Affiliations

  1. NVIDIA, Santa Clara, CA, USA

    Vipin Kumar Kukkala

  2. Colorado State University, Fort Collins, CO, USA

    Sudeep Pasricha

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chen, X. et al. (2023). Safety-Assured Design and Adaptation of Connected and Autonomous Vehicles. In: Kukkala, V.K., Pasricha, S. (eds) Machine Learning and Optimization Techniques for Automotive Cyber-Physical Systems. Springer, Cham. https://doi.org/10.1007/978-3-031-28016-0_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28016-0_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28015-3

  • Online ISBN: 978-3-031-28016-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation