Abstract
In pedestrian-to-vehicle collision accidents, adapting safety measures ahead of time based on actual pose of pedestrians is one of the core objectives for integrated safety. It can significantly enhance the performance of passive safety system when active safety maneuvers fail to avoid accidents. This study proposes a deep learning model to estimate 3D pose of pedestrians from images. Since conventional pedestrian image datasets do not have available pose features to work with, a computer graphics-based (CG) framework is established to train the system with synthetic images. Biofidelic 3D meshes of standing males are first transformed into several walking poses, and then rendered as images from multiple view angles. Subsequently, a matrix of 50 anthropometries, 10 gaits and 12 views is built, in total of 6000 images. A two-branch convolutional neural network (CNN) was trained on the synthetic dataset. The model can simultaneously predict 16 joint landmarks and 14 joint angles of pedestrian for each image with high accuracy. Mean errors of the predictions are 0.54 pixels and − 0.06°, respectively. Any specific pose can then be completely reconstructed from the outputs. Overall, the current study has established a CG-based pipeline to generate photorealistic images with desired features for the training; it demonstrates the feasibility of leveraging CNN to estimate the pose of a walking pedestrian from synthesized images. The proposed framework provides a starting point for vehicles to infer pedestrian poses and then adapt protection measures accordingly for imminent impact to minimize pedestrian injuries.
Similar content being viewed by others
Abbreviations
- ARS:
-
Adaptive restraint system
- BMI:
-
Body mass index
- CDC:
-
Center for Disease Control
- CG :
-
Computer graphics
- CV:
-
Computer vision
- CNN:
-
Convolutional neural networks
- DOF:
-
Degree of freedom
- FEA:
-
Finite element analysis
- HBM:
-
Human body model
- HIC:
-
Head injury criterion
- SBSM:
-
Statistical body shape model
- VRU:
-
Vulnerable road user
References
World Health Organization: Global status report on road safety 2018. https://www.who.int/violence_injury_prevention/road_safety_status/2018/en/ (2020). Accessed 1 July 2020
Cicchino, J.B.: Effectiveness of forward collision warning and autonomous emergency braking systems in reducing front-to-rear crash rates. Accid. Anal. Prev. 99, 142–152 (2017)
Rosen, E.: Autonomous emergency braking for vulnerable road users. Paper presented at the 2013 IRCOBI Conference, Gothenburg, Sweden, 11–13 September 2013
Karlow, J., Bruce, M., Heudorfer, B., Meißner, D., Igawa, T.: Active Intervention into Passive Systems: From Passive Safety to Safe Driving. SAE Technical Paper (2006)
Schoeneburg, R., Breitling, T.: Enhancement of active & passive safety by future pre-safe® systems. Paper Presented at the 19th International Technical Conference on the Enhanced Safety of Vehicles (ESV), Washington, DC, 6–9 June 2005
Eichberger, A., Wallner, D., Hirschberg, W., Cresnik, R.: A situation based method to adapt the vehicle restraint system in frontal crashes to the accident scenario. Paper presented at the 21st International Technical Conference on the Enhanced Safety of Vehicles (ESV), Stuttgart, Germany, 15–18 June 2009
Meier, A., Gonter, M., Kruse, R.: Precrash classification of car accidents for improved occupant safety systems. Procedia Technol. 15, 198–207 (2014)
Untaroiu, C.D., Adam, T.J.: Performance-based classification of occupant posture to reduce the risk of injury in a collision. IEEE. Trans. Intell. Transp. Syst. 14(2), 565–573 (2013)
Adam, T.J., Untaroiu, C.D.: Identification of occupant posture using a Bayesian classification methodology to reduce the risk of injury in a collision. Transp. Res. Part. C. Emerg. Technol. 19(6), 1078–1094 (2011)
Hu, J., Flannagan, C.A., Bao, S., McCoy, R.W., Siasoco, K.M., Barbat, S.: Integration of active and passive safety technologies—a method to study and estimate field capability. Stapp. Car. Crash. 59, 269–296 (2015)
Tang, J., Zhou, Q., Nie, B., Yasuki, T., Kitagawa, Y.: Influence of pre-impact pedestrian posture on lower extremity kinematics in vehicle collisions. SAE. Int. J. Transp. Saf. 4(2), 278–288 (2016)
Tang, J., Zhou, Q.: Intrinsic relationship between anatomy, kinematics and injuries of knee joint under various vehicle-to-pedestrian impact orientations. Paper presented at the 2020 IRCOBI Asia Conference, Beijing, China. 1–2 June 2021
Chen, H., Poulard, D., Crandall, J.R., Panzer, M.B.: Pedestrian response with different initial positions during impact with a mid-sized sedan. Paper Presented at the 24th International Technical Conference on the Enhanced Safety of Vehicles (ESV), Gothenburg, Sweden, 8–11 June 2015
Untaroiu, C.D., Meissner, M.U., Crandall, J.R., Takahashi, Y., Okamoto, M., Ito, O.: Crash reconstruction of pedestrian accidents using optimization techniques. Int. J. Impact. Eng. 36(2), 210–219 (2009)
Wang, L., Shi, J., Song, G., Shen, I.-F.: Object detection combining recognition and segmentation. Paper presented at the 8th Asian Conference on Computer Vision, Tokyo, Japan, 18–22 November 2007
Li, X., Li, L., Flohr, F., Wang, J., Xiong, H., Bernhard, M., Pan, S., Gavrila, D.M., Li, K.: A unified framework for concurrent pedestrian and cyclist detection. IEEE. Trans. Intell. Transp. Syst. 18(2), 269–281 (2017)
Li, K., Wang, X., Xu, Y., Wang, J.: Density enhancement-based long-range pedestrian detection using 3-D range data. IEEE. Trans. Intell. Transp. Syst. 17(5), 1368–1380 (2016)
Park, B.K., Lumeng, J.C., Lumeng, C.N., Ebert, S.M., Reed, M.P.: Child body shape measurement using depth cameras and a statistical body shape model. Ergonomics 58(2), 301–309 (2015)
Reed, M.P., Manary, M.A., Schneider, L.W.: Methods for Measuring and Representing Automobile Occupant Posture. SAE Technical Paper (1999)
Tang, J., Zhou, Q., Nie, B., Hu, J.: Obesity effects on pedestrian lower extremity injuries in vehicle-to-pedestrian impacts: a numerical investigation using human body models. Traffic. Inj. Prev. 21(8), 569–574 (2020)
Zhang, K., Cao, L., Fanta, A., Reed, M.P., Neal, M., Wang, J.T., Lin, C.H., Hu, J.: An automated method to morph finite element whole-body human models with a wide range of stature and body shape for both men and women. J. Biomech. 60, 253–260 (2017)
Fryar, C.D., Gu, Q., Ogden, C.L., Flegal, K.M.: Anthropometric reference data for children and adults: united states, 2011–2014, 2016 Vital and Health Statistics, Series 3. Anal. Stud. 3(39), 1–46 (2016)
Chidester, A.B., Isenberg, R.A.: Final Report—The pedestrian crash data study. Paper presented at the 17th International Technical Conference on the Enhanced Safety of Vehicles (ESV), Amsterdam, The Netherlands, 4–7 June 2001
Levine, D., Richards, J., Whittle, M.W.: Whittle’s Gait Analysis. Elsevier, Oxford (2012)
Cao, Z., Martinez, G.H., Simon, T., Wei, S.-E., Sheikh, Y.A.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern. Anal. Mach. Intell. 66, 1–15 (2019)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Paper presented at the International Conference on Learning Representations 2015, San Diego, CA, 7–9 May 2015
Huang, G., Liu, Z., Maaten, L. van der, Weinberger, K.Q.: Densely connected convolutional networks. Paper presented at the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, 21–26 July 2017
He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. IEEE Trans. Pattern. Anal. Mach. Intell. 42(2), 386–397 (2020)
Zou, T., Zha, A., Liu, Q., Simms, C.: Pedestrian gaits observed from actual pedestrian-vehicle collisions. Int. J. Crashworth. 54, 1–23 (2020)
Tang, J., Hu, J., Nie, B., Zhou, Q.: An algorithm for rapid adjustment of lower extremity posture of a pedestrian model. Paper presented at 2017 IRCOBI Conference, Antwerp, Belgium, 13–15 September 2017
Zhou, X., Zhu, M., Pavlakos, G., Leonardos, S., Derpanis, K.G., Daniilidis, K.: MonoCap: monocular human motion capture using a CNN coupled with a geometric prior. IEEE Trans. Pattern. Anal. Mach. Intell. 41(4), 901–914 (2019)
Acknowledgements
This study was supported by Ministry of Science and Technology of China (Grant Number 2017YFE0118400) and National Natural Science Foundation of China (Grant Number 51975313).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all the authors, the corresponding author states that there is no conflict of interest.
Rights and permissions
About this article
Cite this article
Tang, J., Zhou, Q. A Computer Graphics-Based Framework for 3D Pose Estimation of Pedestrians. Automot. Innov. 4, 338–349 (2021). https://doi.org/10.1007/s42154-021-00142-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42154-021-00142-4