Abstract
Extracting local features is a key technique in the field of human motion prediction. However, Due to incorrect partitioning of strongly correlated joint sets, existing methods ignore parts of strongly correlated joint pairs during local feature extraction, leading to prediction errors in end joints. In this paper, a Motion Chain Learning Framework is proposed to address the problem of prediction errors in end joints, such as hands and feet. The key idea is to mine and build strong correlations for joints belonging to the same motion chain. To be specific, all human joints are first divided into five parts according to the human motion chains. Then, the local interaction relationship between joints on each motion chain is learned by GCN. Finally, a novel Weights-Added Mean Per Joint Position Error loss function is proposed to assign different weights to each joint based on the importance in human biomechanics. Extensive evaluations demonstrate that our approach significantly outperforms state-of-the-art methods on the datasets such as H3.6M, CMU-Mocap, and 3DPW. Furthermore, the visual result confirms that our Motion Chain Learning Framework can reduce errors in end joints while working well for the other joints.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arus, E.: Biomechanics of Human Motion: Applications in the Martial Arts. CRC Press, Boca Raton (2012)
Bartlett, R.: Introduction to Sports Biomechanics: Analysing Human Movement Patterns. Routledge (2014)
Cai, Y., et al.: Learning progressive joint propagation for human motion prediction. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part VII. LNCS, vol. 12352, pp. 226–242. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58571-6_14
Chen, S., Liu, B., Feng, C., Vallespi-Gonzalez, C., Wellington, C.: 3D point cloud processing and learning for autonomous driving: impacting map creation, localization, and perception. IEEE Signal Process. Mag. 38(1), 68–86 (2020)
Cui, Q., Sun, H., Yang, F.: Learning dynamic relationships for 3D human motion prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6519–6527 (2020)
Dang, L., Nie, Y., Long, C., Zhang, Q., Li, G.: MSR-GCN: multi-scale residual graph convolution networks for human motion prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11467–11476 (2021)
Ding, P., Yin, J.: Towards more realistic human motion prediction with attention to motion coordination. IEEE Trans. Circuits Syst. Video Technol. 32(9), 5846–5858 (2022)
Fan, C., Zhang, R., Ming, Y.: MP-LN: motion state prediction and localization network for visual object tracking. Vis. Comput. 1–16 (2021)
Fragkiadaki, K., Levine, S., Felsen, P., Malik, J.: Recurrent network models for human dynamics. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4346–4354 (2015)
Gui, L.Y., Wang, Y.X., Ramanan, D., Moura, J.M.: Few-shot human motion prediction via meta-learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 432–450 (2018)
Ionescu, C., Papava, D., Olaru, V., Sminchisescu, C.: Human3. 6M: large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1325–1339 (2013)
Jain, A., Zamir, A.R., Savarese, S., Saxena, A.: Structural-RNN: deep learning on spatio-temporal graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5308–5317 (2016)
Li, C., Zhang, Z., Lee, W.S., Lee, G.H.: Convolutional sequence to sequence model for human dynamics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5226–5234 (2018)
Li, M., Chen, S., Zhang, Z., Xie, L., Tian, Q., Zhang, Y.: Skeleton-parted graph scattering networks for 3D human motion prediction. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022, Part VI. LNCS, vol. 13666, pp. 18–36. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20068-7_2
Liu, X., Yin, J., Liu, J., Ding, P., Liu, J., Liu, H.: TrajectoryCNN: a new spatio-temporal feature learning network for human motion prediction. IEEE Trans. Circuits Syst. Video Technol. 31(6), 2133–2146 (2020)
Liu, Z., et al.: Towards natural and accurate future motion prediction of humans and animals. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10004–10012 (2019)
Lu, T.W., Chang, C.F.: Biomechanics of human movement and its clinical applications. Kaohsiung J. Med. Sci. 28, S13–S25 (2012)
Ma, T., Nie, Y., Long, C., Zhang, Q., Li, G.: Progressively generating better initial guesses towards next stages for high-quality human motion prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6437–6446 (2022)
Mao, W., Liu, M., Salzmann, M., Li, H.: Learning trajectory dependencies for human motion prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9489–9497 (2019)
Pavllo, D., Feichtenhofer, C., Auli, M., Grangier, D.: Modeling human motion with quaternion-based neural networks. Int. J. Comput. Vision 128, 855–872 (2020)
Qin, Y., Chi, X., Sheng, B., Lau, R.W.: GuideRender: large-scale scene navigation based on multi-modal view frustum movement prediction. Vis. Comput. 1–11 (2023)
Song, S., Chau, L.P., Lin, Z.: Portrait matting using an attention-based memory network. Vis. Comput. 1–14 (2023)
Von Marcard, T., Henschel, R., Black, M.J., Rosenhahn, B., Pons-Moll, G.: Recovering accurate 3D human pose in the wild using IMUs and a moving camera. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 601–617 (2018)
Vukotić, V., Pintea, S.-L., Raymond, C., Gravier, G., van Gemert, J.C.: One-step time-dependent future video frame prediction with a convolutional encoder-decoder neural network. In: Battiato, S., Gallo, G., Schettini, R., Stanco, F. (eds.) ICIAP 2017, Part I. LNCS, vol. 10484, pp. 140–151. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68560-1_13
Wang, H., Ho, E.S., Shum, H.P., Zhu, Z.: Spatio-temporal manifold learning for human motions via long-horizon modeling. IEEE Trans. Visual Comput. Graphics 27(1), 216–227 (2019)
Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Zhang, T., Cao, Y., Zhang, L., Li, X.: Efficient feature fusion network based on center and scale prediction for pedestrian detection. Vis. Comput. 39(9), 3865–3872 (2023)
Acknowledgements
This work was supported in part by Natural Science Foundation Project of CQ (No. CSTC2021JCYJ-MAXMX0062), National Natural Science Foundation of China (No. 62002121 and 62072183), Shanghai Science and Technology Commission (No. 21511100700, 22511104600), the Open Project Program of the State Key Lab of CAD &CG (No. A2203), Zhejiang University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Z., Chen, L., Li, C., Wang, C., He, G. (2024). Learning Local Features of Motion Chain for Human Motion Prediction. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14497. Springer, Cham. https://doi.org/10.1007/978-3-031-50075-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-50075-6_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-50074-9
Online ISBN: 978-3-031-50075-6
eBook Packages: Computer ScienceComputer Science (R0)