Elsevier

Pattern Recognition

Volume 43, Issue 8, August 2010, Pages 2904-2910
Pattern Recognition

Infrared gait recognition based on wavelet transform and support vector machine

https://doi.org/10.1016/j.patcog.2010.03.011Get rights and content

Abstract

To detect human body and remove noises from complex background, illumination variations and objects, the infrared thermal imaging was applied to collect gait video and an infrared thermal gait database was established in this paper. Multi-variables gait feature was extracted according to a novel method combining integral model and simplified model. Also the wavelet transform, invariant moments and skeleton theory were used to extract gait features. The support vector machine was employed to classify gaits. This proposed method was applied to the infrared gait database and achieved 78%–91% for the probability of correct recognition. The recognition rates were insensitive for the items of holding ball and loading package. However, there was significant influence for the item of wearing heavy coat. The infrared thermal imaging was potential for better description of human body moving within image sequences.

Introduction

Biometric recognition refers to an automatic recognition of individuals based on feature derived from their anatomical and/or behavioral characteristics [1], [2]. We can obtain the anatomical characteristics of the human beings by measuring their fingerprint, face, iris, hand shape and so on, and extract the behavioral characteristics from their behaviors, such as gait, voice and keystroke. Unlike conventional recognition techniques of passwords or ID cards, biometric recognition technique is safer and more reliable because it is almost impossible to be disguised, shared, and misplaced [3]. This makes biometric recognition much more difficult to be abused than traditional methods of identification.

Gait recognition, as one of new fields in biometric recognition, has unique capability to recognize people at a distance when other biometrics might not be perceivable. Furthermore, it is difficult to disguise gait without hampering progress, which is of particular interest in scene of crime analysis [4]. Gait can be detected and measured at low resolution, and therefore it can be used in situations where face or iris information is not available in high enough resolution for recognition [5]. However, there are a lot of challenges in gait recognition, such as foreground segmentation, clothing of the subject, walking speed change, carrying objects. In general, sensors of video collection include digital video (DV), professional camera and surveillance camera in gait recognition. It is difficult for these sensors to remove noises from complex background, illumination variation and carrying objects. Fig. 1 illustrates some influencing factors.

In order to improve the detection accuracy, we applied the technique of Infrared (IR) thermal imaging to recognize gaits. Human body is a natural emitter of infrared ray. Usually, the temperature of human body is different from that of background. This leads to the different energy distribution and gray-scale difference between background and human body in thermal image. Furthermore, if we use the technique of infrared thermal imaging, the environment will give little influence on the body detection. Especially, targets can be watched at night by using infrared thermal camera. Fig. 2 gives the comparison between conventional image and infrared image when subject walks with a package. The conventional images were taken outdoors in the afternoon. The outdoor lighting was very stable. Otherwise the complex environment would make the silhouette extraction more difficult. The package changes human body silhouette in the conventional image as shown in Fig. 2(b) and the impact from package is difficult to remove. However, Fig. 2(c) reveals that the package is invisible in infrared image because of different temperature between human body and the package. The clean silhouette can be generated for the infrared image as illustrated in Fig. 2(d). The infrared pattern facilitates silhouette extraction. So we collected infrared gait video with FLIR A40M thermal camera in our experiments. Daoliang Tan [6] used infrared camera to collect gait video at night. This technique is difficult to remove the influence from noises such as carrying object and illumination variations. Bhanu and Han [7], [8] used FLIR SC2000 long-wave infrared thermal camera to deal with human activity recognition. Each subject was asked to walk slowly, walk fast and run forward and backward. There were five subjects to be tested in their experiments. However, they did not analyze the situation for carrying objects. For this reason, we used the IR thermal camera to collect gait video and established a database of Infrared Thermal Gait with 23 subjects. Each subject was asked to walk with four conditions: walking in a natural manner, holding ball, loading package and walking with heavy coat.

In addition, we used a novel method that the integral model was combined with the simplified model to obtain multi-variables gait features. The integral model is based on body silhouettes. By using this integral model, we can identify persons by analyzing motion information of silhouettes, such as walking speed, area change of silhouettes. The simplified model is based on the anatomic principles to simplify human body. But there is some limitation to extract gait feature by using single model (integral model or simplified model). The integral model based on silhouettes is excellent to recognize subjects fast and directly. But there is an indirect connection between the silhouette and the gait. Also complex background can influence the silhouette extraction. So the gait feature based on integral model is easy to be disturbed by background [9], [10], [11]. If we only use the simplified model to extract gait feature, some information (such as body width) will be lost after simplifying the human body. On the side, the gait feature generally includes behavioral characteristic or anatomical characteristic. How can we obtain the effective features including not only behavioral characteristic, but also anatomical characteristic? For this reason, this paper gives a novel method that integral model is combined with simplified model to obtain multi-variables information.

At the same time, the wavelet transform (WT) and invariant moments were used to extract gait parameters. Also the grey-level gait energy image (GEI) was simplified to obtain body skeleton and to extract gait parameters. The GEI [12], [13], [14], [15] reflects major shapes of silhouettes and their changes over gait cycle. So we extracted human silhouettes from the infrared video and obtained the GEI in this paper. These gait parameters were fitted together and presented to support vector machine for classification. Fig. 3 illustrates the technical flow chart.

Section snippets

Infrared video collection

In our experiments, the infrared thermal camera A40M was used to collect gait video. This thermal camera is a tool for infrared detection technology. The kernel of camera is the infrared detector which can translate the infrared ray into electrical signal. The signal processing system can generate infrared image. In this way, the infrared gait video can be collected by using this thermal camera. Detection distance can take some impact on the quality of infrared image. The image of whole body

Results and discussion

This method was applied to infrared thermal gait database. There were 23 subjects (13 males and 10 females). Their age was from 19 years to 30 years. Each subject was asked to walk with four conditions: normal walking, loading package of known weight (5 kg), holding volleyball and wearing heavy coat. There are as many as four video sequences for each subject: (normal walking)×(loading package)×(holding volleyball)×(wearing heavy coat). So there are 92 (23×4) sequences in this infrared database.

Conclusions

Human body is a natural emitter of infrared ray and the temperature is not similar to that of surroundings. So the infrared image is not easy to be disturbed by complex background and illumination variations. Also IR imagery appears more robust for silhouette extraction. In view of this, we used infrared thermal camera to collect gait video and established an infrared thermal gait database. To test the recognition performance, we used double-model to extract gait features and achieved 78%–91%

Acknowledgments

The authors would like to acknowledge the Neural Engineering and Rehabilitation Lab of Tianjin University (TUNERL) and Dr. Hongzhi Qi, Penghai Li, Zhongxing Zhou, Ce Peng and Longlong Cheng for their valuable contributions to this research. This research was supported by National Natural Science Foundation of China (Nos. 30970875, 90920015 and 60501005), Joint Project of National Natural Science Foundation of China–Royal Society of Edinburgh of UK (No. 30910494/C1009), The National High

About the Author—ZHAOJUN XUE received the Bachelor's degree in mechanical design and automation at Shandong University, Jinan, China, received his Master's degree in biomedical engineering from Tongji University, Shanghai, China, and completed his Doctor's degree in biomedical engineering at Tianjin University, Tianjin, China. He is presently a postdoctoral fellow at Tianjin University. His research interests include biometric recognition, image processing, and pattern recognition.

References (19)

  • S. Prabhakar et al.

    Biometric recognition: security and privacy concerns

    IEEE Security & Privacy

    (2003)
  • K. Delac, M. Grgic. A survey of biometric recognition methods, in: 46th International Symposium Electronics in Marine,...
  • A.K. Jain

    Technology: biometric recognition

    Nature

    (2007)
  • M.S. Nixon, J.N. Carter, Advances in automatic gait recognition, in: Proceedings of Sixth IEEE International Conference...
  • L. Lee, W.E.L. Grimson, Gait analysis for recognition and classification, in: Proceedings of the Fifth IEEE Conference...
  • D. Tan, K. Huang, S. Yu, et al., Efficient night gait recognition based on template matching. in: Proceedings of the...
  • B. Bhanu, J. Han, Kinematic-based human motion analysis in infrared sequences [A], in: Proceedings of Sixth IEEE...
  • J. Han, B. Bhanu, Human activity recognition in thermal infrared imagery, in: Proceedings of the IEEE Workshop on...
  • Z. Xue et al.

    Gait recognition based on body silhouette area feature

    Computer Engineering

    (2008)
There are more references available in the full text version of this article.

Cited by (135)

  • In-home Health Monitoring using Floor-based Gait Tracking

    2022, Internet of Things (Netherlands)
  • Convolutional neural-network-based automatic dam-surface seepage defect identification from thermograms collected from UAV-mounted thermal imaging camera

    2022, Construction and Building Materials
    Citation Excerpt :

    Over the past two decades, machine-learning techniques have been applied to the target segmentation of thermograms. Commonly employed methods include support vector machines (SVMs) [28,29], fuzzy c-means (FCM) [30], and k-means methods [31,32,33,34]. Dit Leksir et al. [28] proposed an approach base on SVM that can help to detect faults of electrical installations in thermograms.

View all citing articles on Scopus

About the Author—ZHAOJUN XUE received the Bachelor's degree in mechanical design and automation at Shandong University, Jinan, China, received his Master's degree in biomedical engineering from Tongji University, Shanghai, China, and completed his Doctor's degree in biomedical engineering at Tianjin University, Tianjin, China. He is presently a postdoctoral fellow at Tianjin University. His research interests include biometric recognition, image processing, and pattern recognition.

About the Author—DONG MING received his B.S., M.S. and Ph.D. degrees in Biomedical Engineering at Tianjin University (TU), China, in 1999, 2002 and 2004, respectively. He worked as a research associate in Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong, from 2002 to 2003 and was a visiting scholar in Division of Mechanical Engineering and Mechatronics, University of Dundee, UK, from 2005 to 2006. He joined TU faculty in College of Precision Instruments and Optoelectronics Engineering as an associate professor of biomedical engineering since 2006. Now he is the head of Neural Engineering and Rehabilitation Laboratory of TU and the deputy director of Tianjin Key Laboratory of Biomedical Detecting Techniques and Instruments. His major research interests include neural engineering, rehabilitation engineering, sports science, biomedical instrumentation and signal/image processing.

About the Author—WEI SONG received her B.S. and M.S. degrees in Biomedical Engineering at Tianjin University (TU), China, in 2007 and 2009, respectively. She is now working as a research associate in Department of Orthopaedics and Traumatology, Li Ka Shing Faculty of Medicine, University of Hong Kong. Her major research interests include neural engineering, rehabilitation engineering, and signal/image processing.

About the Author—BAIKUN WAN received his B.S. degree in nuclear electronics at University of Science and Technology of China, Hefei, China, in 1968 and received his M.S. degree in Plasma Physics at Institute of Physics, Chinese Academy of Sciences, Beijing, China, in 1982. He was a visiting scholar in Department of Applied Electronics, Tokyo Institute of Technology, Japan, from 1985 to 1987 and was a senior visiting scholar in Division of Medical Electronics, Tokyo Medical and Dental University, Japan, in 2002. Since 1996, he has been the Professor in College of Precision Instruments and Optoelectronics Engineering and the dean of Department of Biomedical Engineering, Tianjin University. His research interests include the development of quantitative techniques and models for a better understanding of biological systems, aiding clinical diagnosis and treatment, medical imaging and signal processing, etc.

About the Author—SHIJIU JIN received his B.S. and M.S. degrees in Precision Instruments at Tianjin University (TU), China, in 1970 and 1981, respectively. He is now the Professor in College of Precision Instruments and Optoelectronics Engineering, Tianjin University. His research interests include the development of precision measurement techniques.

View full text