Abstract
Parkinson’s disease is a progressive nervous system disorder afflicting millions of patients. Among its motor symptoms, bradykinesia is one of the cardinal manifestations. Experienced doctors are required for the clinical diagnosis of bradykinesia, but sometimes they also miss subtle changes, especially in early stages of such disease. Therefore, developing auxiliary diagnostic methods that can automatically detect bradykinesia has received more and more attention. In this article, we employ a two-stage framework for bradykinesia recognition based on the video of patient movement. First, convolution neural networks are trained to localize keypoints in each video frame. These time-varying coordinates form motion trajectories that represent the whole movement. From the trajectory, we then propose novel measurements, namely stability, completeness, and self-similarity, to quantify different motor behaviors. We also propose a periodic motion model called PMNet. An encoder--decoder structure is applied to learn a low dimensional representation of a motion process. The compressed motion process and quantified motor behaviors are combined as inputs to a fully-connected neural network. Different from the traditional means, our solution extends the application scenario outside the hospital and can be easily transplanted to conduct similar tasks. A commonly used clinical assessment is served as a case study. Experimental results based on real-world data validate the effectiveness of our approach for bradykinesia recognition.
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A system for large-scale machine learning. In Proceedings of USENIX Symposium on Operating Systems Design and Implementation, Vol. 16. 265--283.Google Scholar
- Eduard Bakstein, Kevin Warwick, Jonathan Burgess, Øyvind Stavdahl, and Tipu Aziz. 2010. Features for detection of Parkinson’s disease tremor from local field potentials of the subthalamic nucleus. In International Conference on Cybernetic Intelligent Systems. 1--6.Google ScholarCross Ref
- Andrea Bandini, Silvia Orlandi, Hugo Jair Escalante, Fabio Giovannelli, Massimo Cincotta, Carlos A. Reyes-Garcia, Paola Vanni, Gaetano Zaccara, and Claudia Manfredi. 2017. Analysis of facial expressions in Parkinson’s disease through video-based automatic methods. Journal of Neuroscience Methods 281, 1 (2017), 7--20.Google ScholarCross Ref
- Andrea Bandini, Silvia Orlandi, Fabio Giovannelli, Andrea Felici, Massimo Cincotta, Daniela Clemente, Paola Vanni, Gaetano Zaccara, and Claudia Manfredi. 2016. Markerless analysis of articulatory movements in patients with Parkinson’s disease. Journal of Voice 30, 6 (2016), 766--e1.Google ScholarCross Ref
- J. Cancela, M. Pansera, M. T. Arredondo, J. J. Estrada, M. Pastorino, L. Pastor-Sanz, and J. L. Villalar. 2010. A comprehensive motor symptom monitoring and management system: The bradykinesia case. In International Conference of the IEEE Engineering in Medicine and Biology Society. 1008--1011.Google Scholar
- François Chollet et al. 2015. Keras. Retrieved from https://github.com/fchollet/keras/.Google Scholar
- Jonathan Chung, Sarah A. Chau, Nathan Herrmann, Krista L. Lanctôt, and Moshe Eizenman. 2018. Detection of apathy in alzheimer patients by analysing visual scanning behaviour with RNNs. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery 8 Data Mining. ACM, 149--157.Google ScholarDigital Library
- Bryan T. Cole, Serge H. Roy, and S. Hamid Nawab. 2011. Detecting freezing-of-gait during unscripted and unconstrained activity. In International Conference of the IEEE Engineering in Medicine and Biology Society. 5649--5652.Google Scholar
- Esther Cubo, Pablo Martinez Martín, Jesus A. Martin-Gonzalez, Carmen Rodríguez-Blázquez, Jaime Kulisevsky, and ELEP Group Members. 2010. Motor laterality asymmetry and nonmotor symptoms in Parkinson’s disease. Movement Disorders 25, 1 (2010), 70--75.Google ScholarCross Ref
- Peng Dai, Femida Gwadry-Sridhar, Michael Bauer, and Michael Borrie. 2016. Bagging ensembles for the diagnosis and prognostication of alzheimer’s disease. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI’16). 3944--3950.Google ScholarDigital Library
- E. R. l. Dorsey, R. Constantinescu, J. P. Thompson, K. M. Biglan, R. G. Holloway, K. Kieburtz, F. J. Marshall, B. M. Ravina, G. Schifitto, A. Siderowf, and C. M. Tanner. 2007. Projected number of people with Parkinson disease in the most populous nations, 2005 through 2030. Neurology 68, 5 (2007), 384--386.Google ScholarCross Ref
- Saba Emrani, Anya McGuirk, and Wei Xiao. 2017. Prognosis and diagnosis of parkinson’s disease using multi-task learning. In Proceedings of International Conference on Knowledge Discovery and Data Mining. 1457--1466.Google ScholarDigital Library
- S. Fahn, R. L. Elton, and Members of the UPDRS Development Committee. 1987. Unified Parkinson’s disease rating scale. Recent Developments in Parkinson’s Disease 2 (1987), 153--164.Google Scholar
- Valery L. Feigin, Amanuel Alemu Abajobir, Kalkidan Hassen Abate, Foad Abd-Allah, Abdishakur M. Abdulle, Semaw Ferede Abera, Gebre Yitayih Abyu, Muktar Beshir Ahmed, Amani Nidhal Aichour, Ibtihel Aichour, et al. 2017. Global, regional, and national burden of neurological disorders during 1990--2015: A systematic analysis for the Global Burden of Disease Study 2015. The Lancet Neurology 16, 11 (2017), 877--897.Google ScholarCross Ref
- W. R. Gibb and A. J. Lees. 1988. The relevance of the Lewy body to the pathogenesis of idiopathic Parkinson’s disease.Journal of Neurology, Neurosurgery 8 Psychiatry 51, 6 (1988), 745--752.Google Scholar
- Christopher G. Goetz, Barbara C. Tilley, Stephanie R. Shaftman, Glenn T. Stebbins, Stanley Fahn, Pablo Martinez-Martin, Werner Poewe, Cristina Sampaio, Matthew B. Stern, Richard Dodel, Bruno Dubois, Robert Holloway, Joseph Jankovic, Jaime Kulisevsky, Anthony E. Lang, Andrew Lees, Sue Leurgans, Peter A. LeWitt, David Nyenhuis, C. Warren Olanow, Olivier Rascol, Anette Schrag, Jeanne A. Teresi, Jacobus J. van Hilten, and Nancy LaPelle. 2008. Movement disorder society-sponsored revision of the unified Parkinson’s disease rating scale (MDS-UPDRS): Scale presentation and clinimetric testing results. Movement Disorders 23, 15 (2008), 2129--2170.Google ScholarCross Ref
- Alex Graves. 2013. Generating sequences with recurrent neural networks. ArXiv e-prints (Aug. 2013). arxiv:cs/1308.0850Google Scholar
- Margaret M. Hoehn and Melvin D. Yahr. 1998. Parkinsonism: Onset, progression, and mortality. Neurology 50, 2 (1998), 318--318.Google ScholarCross Ref
- Ji-Won Kim, Jae-Ho Lee, Yuri Kwon, Chul-Seung Kim, Gwang-Moon Eom, Seong-Beom Koh, Do-Young Kwon, and Kun-Woo Park. 2011. Quantification of bradykinesia during clinical finger taps using a gyrosensor in patients with Parkinson’s disease. Medical 8 Biological Engineering 8 Computing 49, 3 (2011), 365--371.Google Scholar
- Xiaoli Liu, Peng Cao, André R. Gonçalves, Dazhe Zhao, and Arindam Banerjee. 2018. Modeling alzheimer’s disease progression with fused laplacian sparse group lasso. ACM Transactions on Knowledge Discovery from Data 12, 6 (2018), 65.Google Scholar
- O. Martinez-Manzanera, E. Roosma, M. Beudel, R. W. K. Borgemeester, Teus van Laar, and Natasha M. Maurits. 2016. A method for automatic and objective scoring of bradykinesia using orientation sensors and classification algorithms. IEEE Transactions on Biomedical Engineering 63, 5 (2016), 1016--1024.Google ScholarCross Ref
- Ziad S. Nasreddine, Natalie A. Phillips, Valérie Bédirian, Simon Charbonneau, Victor Whitehead, Isabelle Collin, Jeffrey L. Cummings, and Howard Chertkow. 2005. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society 53, 4 (2005), 695--699.Google ScholarCross Ref
- Khalil Niazmand, Karin Tonn, Anastasios Kalaras, Urban M. Fietzek, Jan-Hinnerk Mehrkens, and Tim C. Lueth. 2011. Quantitative evaluation of Parkinson’s disease using sensor based smart glove. In International Symposium on Computer-Based Medical Systems. 1--8.Google Scholar
- Khalil Niazmand, Karin Tonn, Anastasios Kalaras, S. Kammermeier, Kai Boetzel, Jan-Hinnerk Mehrkens, and Tim C. Lueth. 2011. A measurement device for motion analysis of patients with Parkinson’s disease using sensor based smart clothes. In International Conference on Pervasive Computing Technologies for Healthcare. 9--16.Google Scholar
- K. Niazmand, K. Tonn, Y. Zhao, U. M. Fietzek, F. Schroeteler, K. Ziegler, A. O. Ceballos-Baumann, and T. C. Lueth. 2011. Freezing of Gait detection in Parkinson’s disease using accelerometer based smart clothes. In IEEE Biomedical Circuits and Systems Conference. 201--204.Google Scholar
- M. Pastorino, J. Cancela, M. T. Arredondo, M. Pansera, L. Pastor-Sanz, F. Villagra, M. A. Pastor, and J. A. Martin. 2011. Assessment of bradykinesia in Parkinson’s disease patients through a multi-parametric system. In International Conference of the IEEE Engineering in Medicine and Biology Society. 1810--1813.Google Scholar
- Werner Poewe, Klaus Seppi, Caroline M. Tanner, Glenda M. Halliday, Patrik Brundin, Jens Volkmann, Anette-Eleonore Schrag, and Anthony E. Lang. 2017. Parkinson disease. Nature Reviews Disease Primers 3, 1 (2017), 17013.Google ScholarCross Ref
- George Rigas, Alexandros T. Tzallas, Markos G. Tsipouras, Panagiota Bougia, Evanthia E. Tripoliti, Dina Baga, Dimitrios I. Fotiadis, Sofia G. Tsouli, and Spyridon Konitsiotis. 2012. Assessment of tremor activity in the Parkinson’s disease using a set of wearable sensors. IEEE Transactions on Information Technology in Biomedicine 16, 3 (2012), 478--487.Google ScholarDigital Library
- Arash Salarian, Heike Russmann, Christian Wider, Pierre R. Burkhard, Franios J. G. Vingerhoets, and Kamiar Aminian. 2007. Quantification of tremor and bradykinesia in Parkinson’s disease using a novel ambulatory monitoring system. IEEE Transactions on Biomedical Engineering 54, 2 (2007), 313--322.Google ScholarCross Ref
- Sofija Spasojević, Tihomir V. Ilić, Ivan Stojković, Veljko Potkonjak, Aleksandar Rodić, and José Santos-Victor. 2017. Quantitative assessment of the arm/hand Movements in Parkinson’s Disease Using a Wireless armband Device. Frontiers in Neurology 8 (2017), 388.Google ScholarCross Ref
- Nooritawati Md Tahir and Hany Hazfiza Manap. 2012. Parkinson disease gait classification based on machine learning approach. Journal of Applied Sciences 12, 2 (2012), 180--185.Google ScholarCross Ref
- Shaohua Teng, Naiqi Wu, Haibin Zhu, Luyao Teng, and Wei Zhang. 2018. SVM-DT-based adaptive and collaborative intrusion detection. IEEE/CAA Journal of Automatica Sinica 5, 1 (2018), 108--118.Google ScholarCross Ref
- PeiYun Zhang, Sheng Shu, and MengChu Zhou. 2018. An online fault detection model and strategies based on SVM-grid in clouds. IEEE/CAA Journal of Automatica Sinica 5, 2 (2018), 445--456.Google ScholarCross Ref
- Wenying Zhang, Huaguang Zhang, Jinhai Liu, Kai Li, Dongsheng Yang, and Hui Tian. 2017. Weather prediction with multiclass support vector machines in the fault detection of photovoltaic system. IEEE/CAA Journal of Automatica Sinica 4, 3 (2017), 520--525.Google ScholarCross Ref
- Christian Zimmermann and Thomas Brox. 2017. Learning to estimate 3D hand pose from single RGB images. In International Conference on Computer Vision.Google ScholarCross Ref
- Daphne G. M. Zwartjes, Tjitske Heida, Jeroen P. P. Van Vugt, Jan A. G. Geelen, and Peter H. Veltink. 2010. Ambulatory monitoring of activities and motor symptoms in Parkinson’s disease. IEEE Transactions on Biomedical Engineering 57, 11 (2010), 2778--2786.Google ScholarCross Ref
Index Terms
- Bradykinesia Recognition in Parkinson’s Disease via Single RGB Video
Recommendations
Estimating bradykinesia severity in Parkinson's disease by analysing gait through a waist-worn sensor
Bradykinesia is a cardinal symptom of Parkinson's disease (PD) and describes the slowness of movement revealed in patients. Current PD therapies are based on dopamine replacement, and given that bradykinesia is the symptom that best correlates with the ...
Estimating bradykinesia in Parkinson's disease with a minimum number of wearable sensors
CHASE '17: Proceedings of the Second IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering TechnologiesMonitoring motor function of patients with Parkinson's disease (PD) over long periods of time is essential in order to improve symptom management and avoid complications. Wearable technologies can be useful in this context as long as they do not ...
Static Weight Transferences in Patients with Parkinson's Disease using the Nintendo Wii Balance Board
REHAB '15: Proceedings of the 3rd 2015 Workshop on ICTs for improving Patients Rehabilitation Research TechniquesIdiopathic Parkinson's Disease (PD) is a neurological disorder with a large number of motor and non-motor alterations that produce an impact on basic and instrumental Activities of Daily Living. These motor disorders are categorized by tremors, rigidity,...
Comments