Moritz Gebert
ABSTRACT
Current interior sensing systems already enable the detection of critical driver states such as drowsiness or inattention. In order to extend the system’s capabilities, this work firstly investigates a possible detection of driver fainting via an interior sensing camera. An approach that supports the simulation of driver fainting is developed and realized in a parked vehicle as well as during manual and automated driving, with 61 participants in total. Moreover, multiple instructed intentional movements with for- and side-ward movements of the body are recorded. Classification models are developed based on features that are derived from head and body pose data. These models are then applied to the complete video streams that include various waiting and driving scenarios. The best classification results are seen with Random Forest classifiers with up to 84% true positive detections and 0.33 false positive detections per hour. The majority of false positive detections were seen during automated driving. Implications and options for future research are discussed.
- John. Allum, Michael Gresty, Emiliy Keshner, and Charlotte Shupert. 1997. The Control of Head Movements during Human Balance Corrections. Journal of Vestibular Research 7, 2-3 (May 1997), 189–218. https://doi.org/10.3233/VES-1997-72-309Google Scholar
Cross Ref
- Gustavo E. A. P. A. Batista, Eamonn J. Keogh, Oben Moses Tataw, and Vinícius M. A. de Souza. 2014. CID: an efficient complexity-invariant distance for time series. Data Mining and Knowledge Discovery 28, 3 (May 2014), 634–669. https://doi.org/10.1007/s10618-013-0312-3Google ScholarDigital Library
- Tobias Baumgartner and Rainer Surges. 2019. Synkope, epileptischer oder psychogener Anfall? Der Weg zur richtigen Diagnose. DMW - Deutsche Medizinische Wochenschrift 144, 12 (June 2019), 835–841. https://doi.org/10.1055/a-0629-0362Google ScholarCross Ref
- Michele Brignole, Carlo Menozzi, Lorella Gianfranchi, Daniele Oddone, Gino Lolli, and Antonio Bertulla. 1991. Neurally mediated syncope detected by carotid sinus massage and head-up tilt test in sick sinus syndrome. The American Journal of Cardiology 68, 10 (Oct. 1991), 1032–1036. https://doi.org/10.1016/0002-9149(91)90491-3Google ScholarCross Ref
- Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 1 (Jan. 2021), 172–186. https://doi.org/10.1109/TPAMI.2019.2929257Google ScholarCross Ref
- Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2022. Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges, and Opportunities. Comput. Surveys 54, 4 (May 2022), 1–40. https://doi.org/10.1145/3447744Google ScholarDigital Library
- Antonio Criminisi. 2011. Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning. Foundations and Trends® in Computer Graphics and Vision 7, 2-3 (2011), 81–227. https://doi.org/10.1561/0600000035Google ScholarDigital Library
- Alfonso Delgado-Bonal and Alexander Marshak. 2019. Approximate Entropy and Sample Entropy: A Comprehensive Tutorial. Entropy 21, 6 (May 2019), 541. https://doi.org/10.3390/e21060541Google ScholarCross Ref
- Mayank Dutia. 1991. The muscles and joints of the neck: Their specialisation and role in head movement. Progress in Neurobiology 37, 2 (Jan. 1991), 165–178. https://doi.org/10.1016/0301-0082(91)90026-WGoogle ScholarCross Ref
- Mica R. Endsley. 1995. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors: The Journal of the Human Factors and Ergonomics Society 37, 1 (March 1995), 32–64. https://doi.org/10.1518/001872095779049543Google ScholarCross Ref
- Alberto Fernandez, Salvador Garcia, Francisco Herrera, and Nitesh V. Chawla. 2018. SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary. Journal of Artificial Intelligence Research 61 (April 2018), 863–905. https://doi.org/10.1613/jair.1.11192Google ScholarCross Ref
- Rikard Fredriksson, Michael G. Lenné, Sjef van Montfort, and Colin Grover. 2021. European NCAP Program Developments to Address Driver Distraction, Drowsiness and Sudden Sickness. Frontiers in Neuroergonomics 2 (Nov. 2021), 786674. https://doi.org/10.3389/fnrgo.2021.786674Google ScholarCross Ref
- Ben D. Fulcher and Nick S. Jones. 2014. Highly Comparative Feature-Based Time-Series Classification. IEEE Transactions on Knowledge and Data Engineering 26, 12 (dec 2014), 3026–3037. https://doi.org/10.1109/tkde.2014.2316504Google ScholarCross Ref
- Refaat Hanna. 2009. The Contribution of Medical Conditions to Passenger Vehicle Crashes / United States (Report No. DOT HS 811 219).Technical Report. U.S. Department of Transportation, National Highway Traffic Safety Administration. http://www-nrd.nhtsa.dot.gov/Pubs/811219.pdfGoogle Scholar
- Riender Happee, Edo de Bruijn, Patrick A. Forbes, and Frans C.T. van der Helm. 2017. Dynamic head-neck stabilization and modulation with perturbation bandwidth investigated using a multisegment neuromuscular model. Journal of Biomechanics 58 (June 2017), 203–211. https://doi.org/10.1016/j.jbiomech.2017.05.005Google ScholarCross Ref
- Md. Al Mehedi Hasan, Mohammed Nasser, Shamim Ahmad, and Khademul Islam Molla. 2016. Feature Selection for Intrusion Detection Using Random Forest. Journal of Information Security 07, 03 (2016), 129–140. https://doi.org/10.4236/jis.2016.73009 Number: 03.Google ScholarCross Ref
- Sonja Hermann. 2005. Exploring Sitting Posture and Discomfort Using Nonlinear Analysis Methods. IEEE Transactions on Information Technology in Biomedicine 9, 3 (Sept. 2005), 392–401. https://doi.org/10.1109/TITB.2005.854513Google ScholarDigital Library
- Geoffrey L. Heyer, Caitlin Schmittauer, and Monica P. Islam. 2016. The Clinical and Electroencephalographic Spectrum of Tilt-Induced Syncope and “Near Syncope” in Youth. Pediatric Neurology 62 (Sept. 2016), 27–33. https://doi.org/10.1016/j.pediatrneurol.2016.05.007Google ScholarCross Ref
- Jianping Hua, Zixiang Xiong, James Lowey, Edward Suh, and Edward R. Dougherty. 2004. Optimal number of features as a function of sample size for various classification rules. Bioinformatics 21, 8 (11 2004), 1509–1515. https://doi.org/10.1093/bioinformatics/bti171 arXiv:https://academic.oup.com/bioinformatics/article-pdf/21/8/1509/48973153/bioinformatics_21_8_1509.pdfGoogle ScholarCross Ref
- Youichi Kobayashi. 2006. Diagnosis and Treatment of Syncope. Journal of Arrhythmia 22, 2 (2006), 74–85.Google ScholarCross Ref
- Arief Koesdwiady, Ridha Soua, Fakhreddine Karray, and Mohamed S. Kamel. 2017. Recent Trends in Driver Safety Monitoring Systems: State of the Art and Challenges. IEEE Transactions on Vehicular Technology 66, 6 (June 2017), 4550–4563. https://doi.org/10.1109/TVT.2016.2631604Google ScholarCross Ref
- Miron B. Kursa and Witold R. Rudnicki. 2010. Feature Selection with the Boruta Package. Journal of Statistical Software 36, 11 (2010), 1–13. https://doi.org/10.18637/jss.v036.i11 Number: 11.Google ScholarCross Ref
- Thomas Lempert, M. Bauer, and D. Schmidt. 1994. Syncope: A videometric analysis of 56 episodes of transient cerebral hypoxia. Annals of Neurology 36, 2 (Aug. 1994), 233–237. https://doi.org/10.1002/ana.410360217Google ScholarCross Ref
- Huagui Li, Marc Weitzel, Arthur Easley, William Barrington, and John Windle. 2000. Potential risk of vasovagal syncope for motor vehicle driving. The American Journal of Cardiology 85, 2 (Jan. 2000), 184–186. https://doi.org/10.1016/S0002-9149(99)00647-5Google ScholarCross Ref
- Tianpeng Li, Jiansheng Chen, Chunhua Hu, Yu Ma, Zhiyuan Wu, Weitao Wan, Yiqing Huang, Fuming Jia, Chen Gong, Sen Wan, and Luming Li. 2018. Automatic Timed Up-and-Go Sub-Task Segmentation for Parkinson’s Disease Patients Using Video-Based Activity Classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering 26, 11 (Nov. 2018), 2189–2199. https://doi.org/10.1109/TNSRE.2018.2875738Google ScholarCross Ref
- Pascal Madeleine. 2012. Dynamics of Seated Computer Work Before and After Prolonged Constrained Sitting. Journal of Applied Biomechanics 28, 3 (July 2012), 297–303. https://doi.org/10.1123/jab.28.3.297Google ScholarCross Ref
- Wilson E. Marcilio and Danilo M. Eler. 2020. From explanations to feature selection: assessing SHAP values as feature selection mechanism. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, Recife/Porto de Galinhas, Brazil, 340–347. https://doi.org/10.1109/SIBGRAPI51738.2020.00053Google ScholarCross Ref
- Stephan Matzka. 2021. Künstliche Intelligenz in den Ingenieurwissenschaften: maschinelles Lernen verstehen und bewerten. Springer Vieweg, Wiesbaden.Google Scholar
- Nobuyasu Nakano, Tetsuro Sakura, Kazuhiro Ueda, Leon Omura, Arata Kimura, Yoichi Iino, Senshi Fukashiro, and Shinsuke Yoshioka. 2020. Evaluation of 3D Markerless Motion Capture Accuracy Using OpenPose With Multiple Video Cameras. Frontiers in Sports and Active Living 2 (May 2020), 50. https://doi.org/10.3389/fspor.2020.00050Google ScholarCross Ref
- Frederik Naujoks, Christian Purucker, Katharina Wiedemann, and Claus Marberger. 2019. Noncritical state transitions during conditionally automated driving on german freeways: Effects of non–driving related tasks on takeover time and takeover quality. Human factors 61, 4 (2019), 596–613.Google Scholar
- Sandro Ninni, Claude Kouakam, William Szurhaj, Guillaume Baille, Didier Klug, Dominique Lacroix, and Philippe Derambure. 2019. Usefulness of head-up tilt test combined with video electroencephalogram to investigate recurrent unexplained atypical transient loss of consciousness. Archives of Cardiovascular Diseases 112, 2 (Feb. 2019), 82–94. https://doi.org/10.1016/j.acvd.2018.08.004Google ScholarCross Ref
- Lars-Göran Öst. 1988. Applied relaxation: Description of an effective coping technique. Cognitive Behaviour Therapy 17, 2 (1988), 83–96.Google Scholar
- Sofiane Ramdani, Benoît Seigle, Julien Lagarde, Frédéric Bouchara, and Pierre Louis Bernard. 2009. On the use of sample entropy to analyze human postural sway data. Medical Engineering & Physics 31, 8 (Oct. 2009), 1023–1031. https://doi.org/10.1016/j.medengphy.2009.06.004Google ScholarCross Ref
- Bruno ML Rocha, Rita V Gomes, Gonçalo JL Cunha, Beatriz MV Silva, Rita Pocinho, Rui Morais, Inês Araújo, and Cândida Fonseca. 2019. Diagnostic and therapeutic approach to cardioinhibitory reflex syncope: A complex and controversial issue. Revista Portuguesa de Cardiologia (English Edition) 38, 9 (2019), 661–673.Google ScholarCross Ref
- Neil J. Salkind and Kristin Rasmussen (Eds.). 2007. Encyclopedia of measurement and statistics. SAGE Publications, Thousand Oaks, Calif.Google Scholar
- Pascal Schneider, Raphael Memmesheimer, Ivanna Kramer, and Dietrich Paulus. 2019. Gesture Recognition in RGB Videos UsingHuman Body Keypoints and Dynamic Time Warping. arxiv:1906.12171 [cs.CV]Google Scholar
- Thomas Schreiber and Andreas Schmitz. 1997. Discrimination power of measures for nonlinearity in a time series. Physical Review E 55, 5 (May 1997), 5443–5447. https://doi.org/10.1103/PhysRevE.55.5443Google ScholarCross Ref
- Sharon Shmuely, Prisca R. Bauer, Erik W. van Zwet, J. Gert van Dijk, and Roland D. Thijs. 2018. Differentiating motor phenomena in tilt-induced syncope and convulsive seizures. Neurology 90, 15 (April 2018), e1339–e1346. https://doi.org/10.1212/WNL.0000000000005301Google ScholarCross Ref
- Gulbadan Sikander and Shahzad Anwar. 2019. Driver Fatigue Detection Systems: A Review. IEEE Transactions on Intelligent Transportation Systems 20, 6 (June 2019), 2339–2352. https://doi.org/10.1109/TITS.2018.2868499Google ScholarCross Ref
- Dan Sorajja, Gillian C. Nesbitt, David O. Hodge, Phillip A. Low, Stephen C. Hammill, Bernard J. Gersh, and Win-Kuang Shen. 2009. Syncope While Driving: Clinical Characteristics, Causes, and Prognosis. Circulation 120, 11 (Sept. 2009), 928–934. https://doi.org/10.1161/CIRCULATIONAHA.108.827626Google ScholarCross Ref
- Karen H.E. Søndergaard, Christian G. Olesen, Eva K. Søndergaard, Mark de Zee, and Pascal Madeleine. 2010. The variability and complexity of sitting postural control are associated with discomfort. Journal of Biomechanics 43, 10 (July 2010), 1997–2001. https://doi.org/10.1016/j.jbiomech.2010.03.009Google ScholarCross Ref
- Sergios Theodoridis and Konstantinos Koutroumbas. 2009. Pattern recognition (4th ed ed.). Academic Press, Burlington, MA. OCLC: 610009838.Google Scholar
- Thomas Lempert, M. Bauer, and D. Schmidt. 2019. Syncope video, what happens when you faint, including seizure-like activity.https://www.youtube.com/watch?v=lF2TF5OUakIGoogle Scholar
- J. Gert van Dijk, Roland D. Thijs, Erik van Zwet, Martijn R. Tannemaat, Julius van Niekerk, David G. Benditt, and Wouter Wieling. 2014. The semiology of tilt-induced reflex syncope in relation to electroencephalographic changes. Brain 137, 2 (Feb. 2014), 576–585. https://doi.org/10.1093/brain/awt332Google ScholarCross Ref
- Peter Welch. 1967. The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms. IEEE Transactions on Audio and Electroacoustics 15, 2 (June 1967), 70–73. https://doi.org/10.1109/TAU.1967.1161901Google ScholarCross Ref
- Typ Whinnery and Estrella M. Forster. 2017. The first sign of loss of consciousness. Physiology & Behavior 179 (Oct. 2017), 494–503. https://doi.org/10.1016/j.physbeh.2017.06.022Google ScholarCross Ref
- David A. Winter. 2009. Biomechanics and Motor Control of Human Movement (1 ed.). John Wiley & Sons, Hoboken, New Jersey. https://onlinelibrary.wiley.com/doi/book/10.1002/9780470549148Google Scholar
- Xia Xiaomao, Zhang Xudong, and Wang Yuanfang. 2019. A Comparison of Feature Selection Methodology for Solving Classification Problems in Finance. Journal of Physics: Conference Series 1284, 1 (Aug. 2019), 012026. https://doi.org/10.1088/1742-6596/1284/1/012026 Number: 1.Google ScholarCross Ref
- Ying Xu, Dennis Nguyen, Armin Mohamed, Cheryl Carcel, Qiang Li, Mansur A. Kutlubaev, Craig S. Anderson, and Maree L. Hackett. 2016. Frequency of a false positive diagnosis of epilepsy: A systematic review of observational studies. Seizure 41 (Oct. 2016), 167–174. https://doi.org/10.1016/j.seizure.2016.08.005Google ScholarCross Ref
- Daniel Zwillinger and Stephen Kokoska. 2000. CRC standard probability and statistics tables and formulae. Chapman & Hall/CRC, Boca Raton.Google Scholar
Index Terms
- Towards In-vehicle Driver Fainting Detection
Recommendations
Driver Distraction Assessment Using Driver Modeling
SMC '13: Proceedings of the 2013 IEEE International Conference on Systems, Man, and CyberneticsCharacterizing individual human drivers is of increasing interest for applications like adaptive driver assistance or monitoring. Describing the human driver by means of control-theoretic driver models constitutes a promising approach. In this paper, we ...
How Do Automation Modes Influence the Frequency of Advanced Driver Distraction Warnings? A Simulator Study
AutomotiveUI '23 Adjunct: Adjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsDriver monitoring systems detect driver distraction and will become a standard safety feature in vehicles soon. Advanced driver distraction warnings prompt drivers through different modalities to keep their attention on the road. With added driving ...
Driver’s fatigue recognition based on yawn detection in thermal images
AbstractThe paper presents a novel approach to driver’s fatigue recognition based on yawn detection using thermal imaging. This work writes into the active drivers’ assisting systems which can warn on driver’s drowsiness based on continuous ...
Comments