Skip to main content
SpringerLink
Log in

Automatic Scenario Generation for Decision Algorithm Performance Evaluation of Autonomous Vehicle via Scenario Parameter Sweeping Method

  • Published:
International Journal of Automotive Technology Aims and scope Submit manuscript

Abstract

Autonomous driving algorithms are always in operation during normal driving and require responses to many more driving situations than ADAS, which assists the driver only in specific risk scenarios. However, due to the absence of a quantitative algorithm evaluation method, most algorithms are evaluated in their own test scenarios. Even in similar cases in identical scenarios, different decisions may be made depending on small differences such as the speed or location of surrounding vehicles. Therefore, one representative scenario cannot cover all cases and it is difficult to quantitatively compare and evaluate algorithms. In this study, the parameters constituting the scenario are determined for lane change and intersection scenarios, typical scenarios in autonomous driving research. Then, all key parameters are swept within a certain range, and all cases that are slightly different are generated even in identical scenario. Simulations are performed automatically on the generated cases, and the vehicle risk is calculated on all cases based on the time to occupancy (TTO). Using this evaluation method, edge cases in which the autonomous algorithm may have weaknesses can be found.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

a y :

lateral acceleration, m/s2

L :

driving distance, m

l w :

line width, m

P :

vehicle position

P activation :

activation point, x

P collision :

collision point, x

P start :

start position, x

P lc :

lane change position, x

S collision :

scenario execution driving distance, m

S shift :

collision point shift, m

t y :

cut-in execution time, sec

V :

vehicle speed, m/s

ego:

ego vehicle

sur:

surrounding vehicle

cut:

cut-in maneuver

rel:

relative (position, speed)

References

  • Bae, J. J., Lee, M. S. and Kang, N. (2020). Partial and full braking algorithm according to time-to-collision for both safety and ride comfort in an autonomous vehicle. Int. J. Automotive Technology 21, 2, 351–360.

    Article  Google Scholar 

  • Funke, J., Brown, M., Erlien, S. M. and Gerdes, J. C. (2016). Collision avoidance and stabilization for autonomous vehicles in emergency scenarios. IEEE Trans. Control Systems Technology 25, 4, 1204–1216.

    Article  Google Scholar 

  • Gelbal, S. Y., Aksun-Guvenc, B. and Guvenc, L. (2020). Collision avoidance of low speed autonomous shuttles with pedestrians. Int. J. Automotive Technology 21, 4, 903–917.

    Article  Google Scholar 

  • Hwang, S., Lee, K., Jeon, H. and Kum, D. (2022). Autonomous vehicle cut-in algorithm for lane-merging scenarios via policy-based reinforcement learning nested within finite-state machine. IEEE Trans. Intelligent Transportation Systems, 1–13.

  • Isermann, R., Schorn, M. and Stählin, U. (2008). Anticollision system PRORETA with automatic braking and steering. Vehicle System Dynamics 46, S1, 683–694.

    Article  Google Scholar 

  • Jeon, H. S., Kum, D. S. and Jeong, W. Y. (2018). Traffic scene prediction via deep learning: Introduction of multichannel occupancy grid map as a scene representation. IEEE Intelligent Vehicles Symposium (IV), Suzhou, China.

  • Jeon, S., Lee, K. and Kum, D. (2022). Overtaking decision and trajectory planning in highway via hierarchical architecture of conditional state machine and chance constrained model predictive control. Robotics and Autonomous Systems, 104014.

  • Ji, J., Khajepour, A., Melek, W. W. and Huang, Y. (2016). Path planning and tracking for vehicle collision avoidance based on model predictive control with multiconstraints. IEEE Trans. Vehicular Technology 66, 2, 952–964.

    Article  Google Scholar 

  • Kim, H., Shin, K., Chang, I. and Huh, K. (2018). Autonomous emergency braking considering road slope and friction coefficient. Int. J. Automotive Technology 19, 6, 1013–1022.

    Article  Google Scholar 

  • Kim, J. and Kum, D. (2017). Collision risk assessment algorithm via lane-based probabilistic motion prediction of surrounding vehicles. IEEE Trans. Intelligent Transportation Systems 19, 9, 2965–2976.

    Article  Google Scholar 

  • Lee, J., Jung, U. and Song, B. (2020). Critical scenario generation for collision avoidance of automated vehicles based on traffic accident analysis and machine learning. Trans. KSAE 28, 11, 817–826.

    Article  Google Scholar 

  • Lee, K. and Kum, D. (2019). Collision avoidance/mitigation system: Motion planning of autonomous vehicle via predictive occupancy map. IEEE Access, 7, 52846–52857.

    Article  Google Scholar 

  • Nam, H., Choi, W. and Ahn, C. (2019). Model predictive control for evasive steering of an autonomous vehicle. Int. J. Automotive Technology 20, 5, 1033–1042.

    Article  Google Scholar 

  • Park, C., Jeong, N. T., Yu, D. and Hwang, S. H. (2019). Path generation algorithm based on crash point prediction for lane changing of autonomous vehicles. Int. J. Automotive Technology 20, 3, 507–519.

    Article  Google Scholar 

  • Shim, T., Adireddy, G. and Yuan, H. (2012). Autonomous vehicle collision avoidance system using path planning and model-predictive-control-based active front steering and wheel torque control. Proc. Institution of Mechanical Engineers, Part D: J. Automobile Engineering 226, 6, 767–778.

    Google Scholar 

  • Yoon, S., Jeon, H. and Kum, D. (2019). Predictive cruise control using radial basis function network-based vehicle motion prediction and chance constrained model predictive control. IEEE Trans. Intelligent Transportation Systems 20, 10, 3832–3843.

    Article  Google Scholar 

  • Zhang, Y., Chen, H., Waslander, S. L., Gong, J., Xiong, G., Yang, T., and Liu, K. (2018). Hybrid trajectory planning for autonomous driving in highly constrained environments. IEEE Access, 6, 32800–32819.

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1F1A1068376) and the Technology Innovation Program (No. 20019115) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).

Author information

Authors and Affiliations

  1. MORAI Inc., 501 Teheran-ro, Gangnam-gu, Seoul, 06168, Korea

    Jiwon Jung

  2. Department of Future Mobility, Gachon University, Gyeonggi, Seongnam-si, 13120, Korea

    Kibeom Lee

Authors
  1. Jiwon Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kibeom Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kibeom Lee.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jung, J., Lee, K. Automatic Scenario Generation for Decision Algorithm Performance Evaluation of Autonomous Vehicle via Scenario Parameter Sweeping Method. Int.J Automot. Technol. 23, 1383–1391 (2022). https://doi.org/10.1007/s12239-022-0121-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12239-022-0121-z

Key Words

Search

Navigation