Skip to main content

Advertisement

SpringerLink
Log in

User performance evaluation and real-time guidance in cloud-based physical therapy monitoring and guidance system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The effectiveness of traditional physical therapy may be limited by the sparsity of time a patient can spend with the physical therapist (PT) and the inherent difficulty of self-training given the paper/figure/video instructions provided to the patient with no way to monitor and ensure compliance with the instructions. In this paper, we propose a cloud-based physical therapy monitoring and guidance system. It is able to record the actions of the PT as he/she demonstrates a task to the patient in an offline session, and render the PT as an avatar. The patient can later train himself by following the PT avatar and getting real-time guidance on his/her device. Since the PT and user (patient) motion sequences may be misaligned due to human reaction and network delays, we propose a Gesture-Based Dynamic Time Warping algorithm that can segment the user motion sequence into gestures, and align and evaluate the gesture sub-sequences, all in real time. We develop an evaluation model to quantify user performance based on different criteria provided by the PT for a task, trained with offline subjective test data consisting of user performance and physical therapist scores. Moreover, we design three types of guidance which can be provided after each gesture based on user score, and conduct subjective tests to validate their effectiveness. Experiments with multiple subjects show that the proposed system can effectively train patients, give accurate evaluation scores, and provide real-time guidance which helps the patients learn the tasks and reach the satisfactory score with less time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Alexiadis DS et al (2011) Evaluating a dancer's performance using kinect-based skeleton tracking. In: Proc. of the 19th ACM international conference on Multimedia (MM’11). Scottsdale

  2. Ali Z, Muhammad G, Alhamid MF (2017) An automatic health monitoring system for patients suffering from voice complications in smart cities. IEEE Access 5:3900–3908

    Article  Google Scholar 

  3. Amazon Web Services. [Online]. Available: https://aws.amazon.com/

  4. Ananthanarayan S et al (2013) Pt Viz: towards a wearable device for visualizing knee rehabilitation exercises. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). Paris

  5. Anderson F, Grossman T, Matejka J, Fitzmaurice G (2013) YouMove: enhancing movement training with an augmented reality mirror. In: Proceedings of the 26th annual ACM symposium on User interface software and technology (UIST’13). St Andrews

  6. Aziz K et al (2016) Smart real-time healthcare monitoring and tracking system using GSM/GPS technologies. In: Big Data and Smart City (ICBDSC’16). Muscat

  7. Bau O, Mackay WE (2008) OctoPocus: a dynamic guide for learning gesture-based command sets. In: Proceedings of the 21st annual ACM symposium on User interface software and technology (UIST’08). Monterey

  8. Berndt DJ, Clifford J (1994) Using Dynamic Time Warping to Find Patterns in Time Series. In: KDD workshop, vol 10, no. 16

  9. Catarinucci L et al (2015) An IoT-aware architecture for smart healthcare systems. IEEE Internet of Things Journal 2(6):515–526

    Article  Google Scholar 

  10. Chang YJ, Chen SF, Huang JD (2011) A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res Dev Disabil 32(6):2566–2570

    Article  Google Scholar 

  11. Chang CY et al (2012) Towards pervasive physical rehabilitation using Microsoft Kinect. In: Pervasive Computing Technologies for Healthcare (PervasiveHealth’12). San Diego

  12. Choden P et al (2017) Volatile urine biomarkers detection in type II diabetes towards use as smart healthcare application. In: Knowledge and Smart Technology (KST’17). Chonburi

  13. Dey S, Liu Y, Wang S, Lu Y (2013) Addressing response time of cloud-based mobile applications. In: Proceedings of the first international workshop on Mobile cloud computing & networking. ACM

  14. Doyle J, Bailey C, Dromey B, Scanaill CN (2010) BASE-An interactive technology solution to deliver balance and strength exercises to older adults. In: 2010 4th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth’10). Munich

  15. Freeman D, Benko H, Morris MR, Wigdor D (2009) ShadowGuides: visualizations for in-situ learning of multi-touch and whole-hand gestures. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS’09). Banff

  16. Gwet KL (2008) Intrarater reliability. Wiley encyclopedia of clinical trials. John Wiley and Sons, Hoboken, pp. 1–14

  17. Jack D et al (2001) Virtual reality-enhanced stroke rehabilitation. Neural Systems and Rehabilitation Engineering, IEEE Transactions on 9(3):308–318

    Article  Google Scholar 

  18. Kahol K, Tripathi P, Panchanathan S (2004) Automated gesture segmentation from dance sequences. In: Proceedings of the Sixth IEEE International conference on Automatic Face and Gesture Recognition (FGR’04). Seoul

  19. Kim D, Song J, Kim D (2007) Simultaneous gesture segmentation and recognition based on forward spotting accumulative HMMs. Pattern Recogn 40(11):3012–3026

    Article  Google Scholar 

  20. Kinect. [Online]. Available: www.xbox.com/en-US/kinect

  21. Lange B et al (2011) Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In: Engineering in Medicine and Biology Society (EMBC’11). Boston

  22. Linktropy. [Online]. Available: http://www.apposite-tech.com/products/

  23. Lu Y, Liu Y, Dey S (2015) Cloud mobile 3D display gaming user experience modeling and optimization by asymmetric graphics rendering. IEEE Journal of Selected Topics in Signal Processing 9(3):517–532

    Article  Google Scholar 

  24. Mirelman A, Patritti BL, Bonato P, Deutsch JE (2010) Effects of virtual reality training on gait biomechanics of individuals post-stroke. Gait & posture 31(4):433–437

    Article  Google Scholar 

  25. Müller M (2007) Information retrieval for music and motion, vol 2. Springer, Heidelberg

    Book  Google Scholar 

  26. Nkosi MT, Mekuria F (2010) Cloud computing for enhanced mobile health applications. In: Cloud Computing Technology and Science (CloudCom’10). Indianapolis

  27. Seber GA, Lee AJ (2012) Linear regression analysis, vol 936. Wiley, New York

    MATH  Google Scholar 

  28. Sodhi R, Benko H, Wilson A (2012) LightGuide: projected visualizations for hand movement guidance. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). Austin

  29. Tang R, Yang XD, Bateman S, Jorge J, Tang A (2015) Physio@ Home: Exploring visual guidance and feedback techniques for physiotherapy exercises. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). Seoul

  30. The Microsoft documentation for Kinect 2.0. [Online]. Available: https://msdn.microsoft.com/en-us/library/microsoft.kinect.jointtype.aspx

  31. The U.S. Mobile App Report by comScore. [Online]. Available: http://www.comscore.com/Insights/Presentations-and-Whitepapers/2016/The-2016-US-Mobile-App-Report

  32. Unity. [Online]. Available: https://unity3d.com/

  33. Wei W, Lu Y, Printz C, Dey S (2015) Motion Data Alignment and Real-Time Guidance in Cloud-Based Virtual Training System. In: Proc. of Wireless Health (WH'15). Bethesda

  34. Yang Z et al (2016) An IoT-cloud based wearable ECG monitoring system for smart healthcare. J Med Syst 40(12):286

    Article  Google Scholar 

  35. Yurtman A, Barshan B (2014) Detection and evaluation of physical therapy exercises by dynamic time warping using wearable motion sensor units. In: Information Sciences and Systems (SIU’14). Trabzon

Download references

Funding

This work is funded by the National Science Foundation under grant number IIS-1522125.

Author information

Authors and Affiliations

  1. Mobile Systems Design Lab, Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA, 92093, USA

    Wenchuan Wei, Yao Lu & Sujit Dey

  2. Department of Rehabilitation Services, University of California San Diego, La Jolla, CA, 92093, USA

    Eric Rhoden

Authors
  1. Wenchuan Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric Rhoden
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sujit Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenchuan Wei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, W., Lu, Y., Rhoden, E. et al. User performance evaluation and real-time guidance in cloud-based physical therapy monitoring and guidance system. Multimed Tools Appl 78, 9051–9081 (2019). https://doi.org/10.1007/s11042-017-5278-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5278-5

Keywords

Search

Navigation