Abstract
Human action recognition aims at extracting features on top of human skeletons and estimating human pose. It has received increasing attention in recent years. However, existing methods capture only the action information while in a real world application such as cognitive assessment, we need to measure the executive functioning that helps psychiatrists to identify some mental disease such as Alzheimer, Schizophrenia and ADHD. In this paper, we propose a skeleton-based action recognition named Mind-In-Action (MIA) for cognitive assessment. MIA integrates a pose estimator to extract the human body joints and then automatically measures the executive functioning employing the distance and elbow angle calculation. Three score functions were designed to measure the executive functioning: the accuracy score, the rhythm score and the functioning score. We evaluate our model on two different datasets and show that our approach significantly outperforms the existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., Hu, W.: Channel-wise topology refinement graph convolution for skeleton-based action recognition. In: ICCV, pp. 13359–13368 (2021)
Cormier, E.: Attention deficit/hyperactivity disorder: a review and update. J. Pediatr. Nurs. 23(5), 345–357 (2008)
Duan, H., Zhao, Y., Chen, K., Lin, D., Dai, B.: Revisiting skeleton-based action recognition. In: CVPR (2022)
Elmi, S., Bell, M., Tan, K.L.: Deep-Cogn: skeleton-based human action recognition for cognitive behavior assessment. In: IEEE 34th ICTAI, pp. 692–699 (2022)
Farha, Y.A., Gall, J.: MS-TCN: multi-stage temporal convolutional network for action segmentation. In: IEEE CVPR, pp. 3575–3584 (2019)
Gattupalli, S., Ghaderi, A., Athitsos, V.: Evaluation of deep learning based pose estimation for sign language recognition. In: PETRA, p. 12. ACM (2016)
Ishikawa, Y., Kasai, S., Aoki, Y., Kataoka, H.: Alleviating over-segmentation errors by detecting action boundaries. In: IEEE WACV, pp. 2321–2330. IEEE (2021)
Johansson, G.: Visual perception of biological motion and a model for its analysis. Percept. Psychophysics 14(2), 201–211 (1973)
Li, S.J., AbuFarha, Y., Liu, Y., Cheng, M.M., Gall, J.: MS-TCN++: multi-stage temporal convolutional network for action segmentation. CoRR, abs/2006.09220 (2020)
McClelland, M.M., Cameron, C.E.: Self-regulation in early childhood: improving conceptual clarity and developing ecologically valid measures. Child Dev. Perspect. 6(2), 136–142 (2012)
McClelland, M.M., et al.: Predictors of early growth in academic achievement: the head-toes-knees-shoulders task. Front. Psychol. 5(2), 599 (2014)
Song, Y.-F., Zhang, Z., Shan, C., Wang, L.: Stronger, faster and more explainable: a graph convolutional baseline for skeleton-based action recognition. In: MM, pp. 1625–1633. ACM (2020)
Song, Y.F., Zhang, Z., Shan, C., Wang, L.: Constructing stronger and faster baselines for skeleton-based action recognition. IEEE Trans. Patterns Pattern Anal. Mach. Intell. 45, 1474–1488 (2021)
Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: CVPR, pp. 5693–5703 (2019)
Tran, D., Wang, H., Torresani, L., Feiszli, M.: Video classification with channel-separated convolutional networks. In: ICCV, pp. 5552–5561 (2019)
Tucha, O.: The history of attention deficit hyperactivity disorder. ADHD Attention Deficit Hyperactivity Disord. 2(4) (1999)
Wang, D., Hu, D., Li, X., Dou, D.: Temporal relational modeling with self-supervision for action segmentation. CoRR, abs/2012.07508 (2020)
Wilson, A.D., Golonka, S.: Embodied cognition is not what you think it is. Front. Psychol. 12(2) (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Elmi, S., Navuluru, S.K., Bell, M. (2023). Mind in Action: Cognitive Assessment Using Action Recognition. In: Strauss, C., Amagasa, T., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2023. Lecture Notes in Computer Science, vol 14146. Springer, Cham. https://doi.org/10.1007/978-3-031-39847-6_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-39847-6_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-39846-9
Online ISBN: 978-3-031-39847-6
eBook Packages: Computer ScienceComputer Science (R0)