Skip to main content
SpringerLink
Log in

Human body recognition based on the sparse point cloud data from MIMO millimeter-wave radar for smart home

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human body recognition is widely used in smart home. The current mainstream perception modalities, i.e., camera and wearable device, are vulnerable under challenging lighting conditions and poor convenience. On the other hand, Multi-human body recognition remains as one of the most challenging tasks in a dynamic and complex environment. In this work, we introduce the low-cost multiple-input-multiple-output (MIMO) millimeter-wave radar without exposing user’s private information for human body recognition in smart home. We propose a human body recognition scheme with the clustering based on the human body tracking using the sparse point cloud data of MIMO millimeter-wave radar. Firstly, the possible position of human body is predicted based on Kalman filter. Then, the point cloud data is clustered based on the human body shape in the prediction range of the human position. Finally, label tags are used to mark the human body targets detected by each frame of the radar. We apply human body recognition to validate the effectiveness of the proposed scheme. It can achieve single-person and double-person recognition using the sparse point cloud data of MIMO millimeter-wave radar. The results show that our proposed scheme reduces the error probability by 23.4% for the single-person recognition and by 31.1% for the double-person recognition. Extensive evaluations on the application of human activity recognition well demonstrate the practicability of the proposed scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abramovich YI, Frazer GJ, Johnson BA (2013) Principles of modeSelective MIMO OTHR. IEEE Trans Aerosp Electron Syst 46(3):1839–1868

    Article  ADS  Google Scholar 

  2. Banan A, Nasiri A, Taheri-Garavand A (2020) Deep learning-based appearance features extraction for automated carp species identification. Aquac Eng 89:102053–102053

    Article  Google Scholar 

  3. Chalabi NE, Attia A, Bouziane A, Akhtar Z (2021) Particle swarm optimization based block feature selection in face recognition system. Multimedia Tools Appl 80:33257–33273

    Article  Google Scholar 

  4. Clemente C, Pallotta L, Maio AD, Soraghan JJ, Farina A (2015) A novel algorithm for radar classification based on doppler characteristics exploiting orthogonal Pseudo-Zernike polynomials. IEEE Trans Aerosp Electron Syst 51(1):417–430

    Article  ADS  Google Scholar 

  5. Erol B, Amin MG, Boashash B, Ahmad F, Zhang YD (2016) Wideband radar based fall motion detection for a generic elderly. 50th Asilomar Conference on Signals. Systems and Computers

  6. Fan Y, Bao J, Aljumaily MSL, Li H (2019) Communications via frequency-modulated continuous-wave radar in millimeter wave band. IEEE Global Communications Conference

  7. Gurbuz SZ, Clemente C, Balleri A, Soraghan JJ (2017) Micro-Doppler-based in-home aided and unaided walking recognition with multiple radar and sonar systems. IET Radar Sonar Navig 11(1):107–115

    Article  Google Scholar 

  8. Hai D, Himed B (2009) A Virtual Antenna Beamforming (VAB) approach for radar systems by using orthogonal coding waveforms. IEEE Trans Antennas Propag 57(2):425–435

    Article  ADS  Google Scholar 

  9. Ji Y, Yang Y, Xu X, Shen HT (2018) One-shot learning based pattern transition map for action early recognition. Sig Process 143(1):364–370

    Article  Google Scholar 

  10. Ji XP, Cheng J, Feng W, Tao DP (2018) Skeleton embedded motion body partition for human action recognition using depth sequences. Sig Process 143(1):56–58

    Article  Google Scholar 

  11. Jin F, Sengupta A, Cao SY, Wu YJ (2020) MmWave radar point cloud segmentation using GMM in multimodal traffic monitoring. IEEE International Radar Conference

  12. Jiang X, Zhang Y, Yang Q, Deng B, Wang H (2020) Millimeter-wave array radar-based human gait recognition using multi-channel threedimensional convolutional neural network. Sensors 20(19):5466–5466

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  13. Jokanović B, Amin M (2018) Fall detection using deep learning in range-doppler radars. IEEE Trans Aerosp Electron Syst 54(1):180–189

    Article  ADS  Google Scholar 

  14. Laura G, Jiménez JM, Miran T, Jaime L (2018) Wireless technologies for IoT in smart cities. Netw Protoc Algorithm 10(1):23–64

    Article  Google Scholar 

  15. Laura G, Lorena P, Miran T, Jaime L (2018) System for detection of emergency situations in Smart City Environments Employing Smartphones. In: 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp 266–272

  16. Liang L, Popescu M, Skubic M, Rantz M, Cuddihy P (2011) Automatic fall detection based on Doppler radar motion signature. In: 2011 5th International Conference on Pervasive Computing Technologies for Healthcare, pp 222–225

  17. Markopoulos PP, Zlotnikov S, Ahmad F (2019) Adaptive radar-based human activity recognition with L1-Norm linear discriminant analysis. IEEE J Electromagn Rf Microwaves Med Biol 3(2):120–126

    Article  Google Scholar 

  18. Pham C, Nguyen L, Nguyen A, Nguyen N, Nguyen VT (2021) Combining skeleton and accelerometer data for human fine-grained activity recognition and abnormal behaviour detection with deep temporal convolutional networks. Multimedia Tools Appl 80(19):28919–28940

    Article  Google Scholar 

  19. Qian K, He ZY, Zhang XY (2020) 3D Point cloud generation with millimeter-wave radar. Proc ACM Interact Mob Wearable Ubiquitous Technol 4(4):1–23

  20. Ricci R, Balleri A (2015) Recognition of humans based on radar micro-doppler shape spectrum features. Sonar Navig IET Radar 9(9):1216–1223

    Article  Google Scholar 

  21. Schumann O, Hahn M, Dickmann J, Wöhler C (2018) Semantic segmentation on radar point clouds. 21st International Conference on Information Fusion, pp 2179–2186

  22. Varshney N, Bakariya B, Kushwaha AKS (2022) Human activity recognition using deep transfer learning of cross position sensor based on vertical distribution of data. Multimedia Tools Appl 81:22307–22322

    Article  Google Scholar 

  23. Wan EA, Van der Merwe R, Nelson AT (2000) Dual estimation and the unscented transformation. Advances in Neural Information Processing Systems. NIPS 1999, pp 666–672

  24. Wang M, Zhang YD, Cui G (2019) Human motion recognition exploiting radar with stacked recurrent neural network. Digit Signal Proc 87(2):125–131

    Google Scholar 

  25. Wang M, Cui G, Yang X, Kong L (2018) Human body and limb motion recognition via stacked gated recurrent units network. IET Radar Sonar Navig 12(9):1046–1051

    Article  Google Scholar 

  26. Wei L, Yang X, Chao Z, Shuai L, Ning L (2016) DRFM range false target cancellation method based on slope-varying LFM chirp signal. In: 2016 IEEE 13th International Conference on Signal Processing (ICSP)

  27. Zhai W, Zhang Y (2014) Motion parameters estimation of high-speed moving target for stepped frequency chirp signal. In: 2014 International Radar Conference

  28. Zhu SY, Xu JH, Guo HQ, Liu QW, Wu SE, Wang HG (2018) Indoor human activity recognition based on ambient radar with signal processing and machine learning. In: 2018 IEEE International Conference on Communications, pp 1–6

Download references

Acknowledgements

This work is granted by the Science and Technology Department of Jilin Province, China [Grant No. 20220101153JC]. Some or all data during the study are available from the corresponding author by request (Email: tjguo_ciomp@hotmail.com). Authors express their gratitude to IoT Innovation Lab at UTS in Australia for their generously provided the experimental equipment. Authors would also like to thank anonymous reviewers and editors provided many helpful comments on the manuscript.

Author information

Authors and Affiliations

  1. College of Instrumentation & Electrical Engineering, Jilin University, Changchun, China

    Xiaohua Zhou, Xinkai Meng & Jianbin Zheng

  2. Global Big Data Technologies Center, University of Technology Sydney, Sydney, Australia

    Gengfa Fang

  3. Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Science, Changchun, China

    Tongjian Guo

Authors
  1. Xiaohua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinkai Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianbin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gengfa Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tongjian Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tongjian Guo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, X., Meng, X., Zheng, J. et al. Human body recognition based on the sparse point cloud data from MIMO millimeter-wave radar for smart home. Multimed Tools Appl 83, 22055–22074 (2024). https://doi.org/10.1007/s11042-023-15700-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15700-7

Keywords

Search

Navigation