Skip to main content
SpringerLink
Log in

Enhancing human action recognition via structural average curves analysis

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Human action recognition typically requires a large amount of training samples, which is often expensive and time-consuming to create. In this paper, we present a novel approach for enhancing human actions with a limited number of samples via structural average curves analysis. Our approach first learns average sequences from each pair of video samples for every action class and then gather them with original video samples together to form a new training set. Action modeling and recognition are proposed to be performed with the resulting new set. Our technique was evaluated on four benchmarking datasets. Our classification results are superior to those obtained with the original training sets, which suggests that the proposed method can potentially be integrated with other approaches to further improve their recognition performances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Here, one notes that the normalized warping path do not need to be equally spaced.

  2. http://cvrc.ece.utexas.edu/SDHA2010/Aerial_View_Activity.html.

  3. http://mrl.isr.uc.pt/experimentaldata/public/uc-3d/.

  4. https://github.com/rasmusbergpalm/DeepLearnToolbox.

References

  1. Mahbub, U., Imtiaz, H., Ahad, M.A.R.: Action recognition based on statistical analysis from clustered flow vectors. Signal Image Video Process. 8(2), 243–253 (2014)

    Article  Google Scholar 

  2. Shao, L., Zhen, X., Tao, D., Li, X.: Spatio-temporal laplacian pyramid coding for action recognition. IEEE Trans. Cybern. 44(6), 817–827 (2014)

    Article  Google Scholar 

  3. Pei, L., Ye, M., Zhao, X., Xiang, T., Li, T.: Learning spatio-temporal features for action recognition from the side of the video. Signal Image Video Process. 10, 199–206 (2016)

    Article  Google Scholar 

  4. Keçeli, A.S., Kaya, A., Can, A.B.: Combining 2d and 3d deep models for action recognition with depth information. Signal Image Video Process. (2018). https://doi.org/10.1007/s11760-018-1271-3

    Google Scholar 

  5. Wang, L., Qiao, Y., Tang, X.: Action recognition with trajectory-pooled deep-convolutional descriptors. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4305–4314. IEEE (2015)

  6. Chaaraoui, A.A., Padilla-Lpez, J.R., Climent-Prez, P., Flrez-Revuelta, F.: Evolutionary joint selection to improve human action recognition with rgb-d devices. Expert Syst. Appl. 41(3), 786–794 (2014)

    Article  Google Scholar 

  7. Duan, L., Xu, D., Tsang, I.H., Luo, J.: Visual event recognition in videos by learning from web data. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1667–1680 (2012)

    Article  Google Scholar 

  8. Zhou, F., De la Torre, F.: Generalized time warping for multi-modal alignment of human motion. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1282–1289. IEEE (2012)

  9. Guha, T., Ward, R.K.: Learning sparse representations for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1576–1588 (2012)

    Article  Google Scholar 

  10. Theodorakopoulos, I., Kastaniotis, D., Economou, G., Fotopoulos, S.: Pose-based human action recognition via sparse representation in dissimilarity space. J. Vis. Commun. Image Represent. 25(1), 12–23 (2014)

    Article  Google Scholar 

  11. Zhang, D., Gatica-Perez, D., Bengio, S., McCowan, I.: Semi-supervised adapted hmms for unusual event detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 611–618. IEEE (2005)

  12. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE (2008)

  13. Seo, H.J., Milanfar, P.: Action recognition from one example. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 867–882 (2011)

    Article  Google Scholar 

  14. Rodriguez, M.D., Ahmed, J., Shah, M.: Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE (2008)

  15. Neverova, N., Wolf, C., Taylor, G.W., Nebout, F.: Multi-scale deep learning for gesture detection and localization. In: Computer Vision-ECCV 2014 Workshops, pp. 474–490. Springer International Publishing (2014)

  16. Schmid, M.F., Booth, C.R.: Methods for aligning and for averaging 3d volumes with missing data. J. Struct. Biol. 161(3), 243–248 (2008)

    Article  Google Scholar 

  17. Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  18. Amit, K., Kaustubh, K., Srikanth, C.V.: Ramasubramanian: Towards fast, view-invariant human action recognition. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8. IEEE (2008)

  19. Ahmadi, S.A., Sielhorst, T., Stauder, R., Horn, M., Feussner, H., Navab, N.: Recovery of surgical workflow without explicit models. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 420–428. Springer, Berlin (2006)

  20. Boudaoud, S., Rix, H., Meste, O.: Core shape modelling of a set of curves. Comput. Stat. Data Anal. 54(2), 308–325 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  21. Morlini, I., Zani, S.: Estimation of the structural mean of a sample of curves by dynamic time warping. Data Analysis, Classification and the Forward Search, pp. 39–48. Springer, Berlin (2006)

    Chapter  Google Scholar 

  22. Xie, X., De Vylder, J., Van Cauwelaert, D., Veelaert, P., Philips, W., Aghajan, H.: Average track estimation of moving objects using ransac and dtw. In: Proceedings of the International Conference on Distributed Smart Cameras, Article No. 28. ACM (2014)

  23. Seltzer, M.L., Yu, D., Wang, Y.: An investigation of deep neural networks for noise robust speech recognition. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7398–7402. IEEE (2013)

  24. Liu, Y., Shriberg, E., Stolcke, A., Hillard, D., Ostendorf, M., Harper, M.: Enriching speech recognition with automatic detection of sentence boundaries and disfluencies. IEEE Trans. Audio Speech Lang. Process. 14(5), 1526–1540 (2006)

    Article  Google Scholar 

  25. Chen, C., Jafari, R., Kehtarnavaz, N.: UTD-MHAD: a multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. In: IEEE International Conference on Image Processing, pp. 168–172. IEEE (2015)

  26. Lu, G., Kudo, M.: Learning action patterns in difference images for efficient action recognition. Neurocomputing 123, 328–336 (2014)

    Article  Google Scholar 

  27. Jung, M., Hwang, J., Tani, J.: Multiple spatio-temporal scales neural network for contextual visual recognition of human actions. In: Joint IEEE International Conferences on Development and Learning and Epigenetic Robotics, pp. 235–241. IEEE (2014)

Download references

Acknowledgements

The work is supported by National Natural Science Foundation of China (61403232, 61327003), Natural Science Foundation of Shandong Province, China (ZR2014FQ025), and Young Scholars Program of Shandong University (YSPSDU, 2015WLJH30).

Author information

Authors and Affiliations

  1. Key Laboratory of High-Efficiency and Clean Mechanical Manufacture of MOE, National Demonstration Center for Experimental Mechanical Engineering Education, School of Mechanical Engineering, Shandong University, Jinan, 250061, China

    Shichen Zeng, Guoliang Lu & Peng Yan

Authors
  1. Shichen Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoliang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoliang Lu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, S., Lu, G. & Yan, P. Enhancing human action recognition via structural average curves analysis. SIViP 12, 1551–1558 (2018). https://doi.org/10.1007/s11760-018-1311-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1311-z

Keywords

Search

Navigation