Skip to main content
SpringerLink
Log in

Image-based fall detection and classification of a user with a walking support system

  • Research Article
  • Published:
Frontiers of Mechanical Engineering Aims and scope Submit manuscript

Abstract

The classification of visual human action is important in the development of systems that interact with humans. This study investigates an image-based classification of the human state while using a walking support system to improve the safety and dependability of these systems.We categorize the possible human behavior while utilizing a walker robot into eight states (i.e., sitting, standing, walking, and five falling types), and propose two different methods, namely, normal distribution and hidden Markov models (HMMs), to detect and recognize these states. The visual feature for the state classification is the centroid position of the upper body, which is extracted from the user’s depth images. The first method shows that the centroid position follows a normal distribution while walking, which can be adopted to detect any non-walking state. The second method implements HMMs to detect and recognize these states. We then measure and compare the performance of both methods. The classification results are employed to control the motion of a passive-type walker (called “RT Walker”) by activating its brakes in non-walking states. Thus, the system can be used for sit/stand support and fall prevention. The experiments are performed with four subjects, including an experienced physiotherapist. Results show that the algorithm can be adapted to the new user’s motion pattern within 40 s, with a fall detection rate of 96.25% and state classification rate of 81.0%. The proposed method can be implemented to other abnormality detection/classification applications that employ depth image-sensing devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alami R, Albu-Schaeffer A, Bicchi A, et al. Safe and dependable physical human-robot interaction in anthropic domains: State of the art and challenges. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing: IEEE, 2006

    Google Scholar 

  2. Bedaf S, Gelderblom G J, De Witte L. Overview and categorization of robots supporting independent living of elderly people: What activities do they support and how far have they developed. Assistive Technology, 2015, 27(2): 88–100

    Article  Google Scholar 

  3. WHO. Good Health Adds Life to Years. Global Brief for World Health Day 2012. 2012. Retrieved form http://www.who.int/ageing/publications/whd2012_global_brief/en/

    Google Scholar 

  4. Stevens J A, Thomas K, Teh L, et al. Unintentional fall injuries associated with walkers and canes in older adults treated in US emergency departments. Journal of the American Geriatrics Society, 2009, 57(8): 1464–1469

    Article  Google Scholar 

  5. Noury N, Fleury A, Rumeau P, et al. Fall detection-principles and methods. In: Proceedings of 29th Annual International Conference of the IEEE on Engineering in Medicine and Biology Society. Lyon: IEEE, 2007

    Google Scholar 

  6. Tong L, Song Q, Ge Y, et al. HMM-based human fall detection and prediction method using tri-axial accelerometer. IEEE Sensors Journal, 2013, 13(5): 1849–1856

    Article  Google Scholar 

  7. Huynh Q T, Nguyen U D, Irazabal L B, et al. Optimization of an accelerometer and gyroscope-based fall detection algorithm. Journal of Sensors, 2015, 2015: 452078

    Article  Google Scholar 

  8. Bourke A K, Lyons GM. A threshold-based fall-detection algorithm using a bi-axial gyroscope sensor. Medical Engineering & Physics, 2008, 30(1): 84–90

    Article  Google Scholar 

  9. Auvinet E, Multon F, Saint-Arnaud A, et al. Fall detection with multiple cameras: An occlusion-resistant method based on 3-D silhouette vertical distribution. IEEE Transactions on Information Technology in Biomedicine, 2011, 15(2): 290–300

    Article  Google Scholar 

  10. Hirata Y, Hara A, Kosuge K. Motion control of passive intelligent walker using servo brakes. IEEE Transactions on Robotics, 2007, 23(5): 981–990

    Article  Google Scholar 

  11. Taghvaei S, Kosuge K. HMM-based state classification of a user with a walking support system using visual PCA features. Advanced Robotics, 2014, 28(4): 219–230

    Article  Google Scholar 

  12. Stone E E, Skubic M. Fall detection in homes of older adults using the Microsoft Kinect. IEEE Journal of Biomedical and Health Informatics, 2015, 19(1): 290–301

    Article  Google Scholar 

  13. Wang Y, Wu K, Ni L M. WiFall: Device-free fall detection by wireless networks. IEEE Transactions on Mobile Computing, 2016, 16(2): 581–594

    Article  Google Scholar 

  14. Daher M, El Najjar M E B, Khalil M. Automatic fall detection system using sensing floors. International Journal of Computing and Information Sciences, 2016, 12(1): 75–82

    Article  Google Scholar 

  15. Purwar A, Un Jeong D, Chung W Y. Activity monitoring from realtime triaxial accelerometer data using sensor network. In: Proceedings of International Conference on Control, Automation and Systems. Seoul: IEEE, 2007, 2402–2406

    Google Scholar 

  16. Aguilar P A, Boudy J, Istrate D, et al. A dynamic evidential network for fall detection. IEEE Journal of Biomedical and Health Informatics, 2014, 18(4): 1103–1113

    Article  Google Scholar 

  17. Shi G, Chan C S, Li W J, et al. Mobile human airbag system for fall protection using MEMS sensors and embedded SVM classifier. IEEE Sensors Journal, 2009, 9(5): 495–503

    Article  Google Scholar 

  18. Zhao K, Jia K, Liu P. Fall detection algorithm based on human posture recognition. In: Pan J S, Tsai P W, Huang H C, eds. Advances in Intelligent Information Hiding and Multimedia Signal Processing. Smart Innovation, Systems and Technologies, Vol 64. Cham: Springer, 2017, 119–126

    Article  Google Scholar 

  19. Hsieh C Y, Liu K C, Huang C N, et al. Novel hierarchical fall detection algorithm using a multiphase fall model. Sensors (Basel), 2017, 17(2): 307

    Article  Google Scholar 

  20. Perry J T, Kellog S, Vaidya SM, et al. Survey and evaluation of realtime fall detection approaches. In: Proceedings of 6th International Symposium on High-Capacity Optical Networks and Enabling Technologies. Alexandria: IEEE, 2009, 158–164

    Google Scholar 

  21. Rougier C, Meunier J, St-Arnaud A, et al. Fall detection from human shape and motion history using video surveillance. In: Proceedings of 21st International Conference on Advanced Information Networking and Applications Workshops. Niagara Falls: IEEE, 2007, 875–880

    Google Scholar 

  22. Taghvaei S, Jahanandish M H, Kosuge K. Autoregressive moving average hidden Markov model for vision-based fall prediction—An application for walker robot. Assistive Technology, 2017, 29(1): 19–27

    Article  Google Scholar 

  23. Skubic M, Harris B H, Stone E, et al. Testing non-wearable fall detection methods in the homes of older adults. In: IEEE 38th Annual International Conference of Engineering in Medicine and Biology Society. IEEE, 2016, 557–560

    Google Scholar 

  24. Hirata Y, Hara A, Kosuge K. Motion control of passive intelligent walker using servo brakes. IEEE Transactions on Robotics, 2007, 23(5): 981–990

    Article  Google Scholar 

  25. Mazhelis O. One-class classifiers: A review and analysis of suitability in the context of mobile-masquerader detection. South African Computer Journal, 2007, 36: 29–48

    Google Scholar 

  26. Yamato J, Ohya J, Ishii K. Recognizing human action in timesequential images using hidden Markov model. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 1992

    Google Scholar 

  27. Goswami A, Peshkin M A, Colgate J E. Passive robotics: An exploration of mechanical computation. In: Proceedings of IEEE International Conference on Robotics and Automation. Cincinnati: IEEE, 1990, 279–284

    Chapter  Google Scholar 

  28. Hirata Y, Komatsuda S, Kosuge K. Fall prevention control of passive intelligent walker based on human model. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Nice: IEEE, 2008, 1222–1228

    Google Scholar 

  29. Gonzalez R C, Woods R E. Image Processing. Digital Image Processing. Upper Saddle River: Addison-Wesley Publishing Co., Inc., 1977

    MATH  Google Scholar 

  30. Rencher A C. Methods of Multivariate Analysis. 2nd ed. New York: John Wiley & Sons, 2003

    MATH  Google Scholar 

  31. Rabiner, L, Juang B. An introduction to hidden Markov models. IEEE ASSP Magazine, 1986, 3(1): 4–16

    Article  Google Scholar 

Download references

Acknowledgements

The experiments were conducted with the help of Dr. Ryushiro Kawazoe, who is an experienced physical therapist and CEO of Kumasuma Inc., Dr Takuro Hatsukari from the Paramount Bed Company, Tokyo 136-8670, Japan, and the members of the System Robotics Laboratory, Tohoku University, Japan.

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Shiraz University, Shiraz, 71936-16548, Iran

    Sajjad Taghvaei

  2. Department of Bioengineering and Robotics, Tohoku University, Sendai, 980-8579, Japan

    Kazuhiro Kosuge

Authors
  1. Sajjad Taghvaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kazuhiro Kosuge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sajjad Taghvaei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Taghvaei, S., Kosuge, K. Image-based fall detection and classification of a user with a walking support system. Front. Mech. Eng. 13, 427–441 (2018). https://doi.org/10.1007/s11465-017-0465-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11465-017-0465-7

Keywords

Search

Navigation