Skip to main content
SpringerLink
Log in

Recognizing the human attention state using cardiac pulse from the noncontact and automatic-based measurements

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

User attention state recognition when interacting with a monitor or undertaking a specific task represents a crucial issue in many domains and applications, such as e-learning, driving and network video conferences. However, for the consideration of convenience and practicality in these situations, it is inescapable and necessary to develop a noncontact and automatic monitoring system to analyze, recognize and predict what kind of attention state in individuals during a task execution without delay. The elaborated technique presented here has achieved efficient cardiac pulse estimation based on the face image captured by the noncontact, automatic and webcam-based measurement method. After collecting cardiac pulse signals, various features extraction methods are presented to obtain key features from the raw data which is related to the attention state or not. The experiment result shows that it is possible to estimate humans attention state based on the technique presented here. The proposed technique may be useful for monitoring person for the purpose of health care, psychological testing, online learning or security, etc.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Allen J (2007) Photoplethysmography and its application in clinical physiological measurement. Physiol Meas 28(3):R1

    Article  Google Scholar 

  • Asteriadis S, Karpouzis K, Kollias S (2008) A neuro-fuzzy approach to user attention recognition. In: Artificial Neural Networks-ICANN 2008, pp 927–936. Springer

  • Banzhaf W, Nordin P, Keller RE, Francone FD (1998) Genetic programming: an introduction, vol 270. Morgan Kaufmann San Francisco

  • Belle A, Hargraves RH, Najarian K (2012) An automated optimal engagement and attention detection system using electrocardiogram. Comput Math Methods Med 2012. doi:10.1155/2012/528781

  • Beymer D, Flickner M (2003) Eye gaze tracking using an active stereo head. In: Computer vision and pattern recognition, 2003. Proceedings. 2003 IEEE computer society conference on, vol 2, pp II–451. IEEE

  • Bousefsaf F, Maaoui C, Pruski A (2014) Remote assessment of physiological parameters by non-contact technologies to quantify and detect mental stress states. In: Control, decision and information technologies (CoDIT), 2014 International conference on, pp 719–723. IEEE

  • Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27

    Article  Google Scholar 

  • Chang H, Yang M, Yang J (2016) Learning a structure adaptive dictionary for sparse representation based classification. Neurocomputing 190:124–131

    Article  Google Scholar 

  • Ferreira C (2002) Gene expression programming in problem solving. In: Soft Computing and Industry, pp 635–653. Springer, London

  • Ferreira C, Gepsoft U (2008) What is gene expression programming

  • Garbey M, Sun N, Merla A, Pavlidis I (2007) Contact-free measurement of cardiac pulse based on the analysis of thermal imagery. IEEE Trans Biomed Eng 54(8):1418–1426

    Article  Google Scholar 

  • Gu B, Sheng VS (2016) A robust regularization path algorithm for v-support vector classification

  • Gu B, Sheng VS, Tay Keng Y, Romano W, Li S (2015) Incremental support vector learning for ordinal regression. IEEE Trans Neural Netw Learn Syst 26(7):1403–1416

    Article  MathSciNet  Google Scholar 

  • Harris FJ (1978) On the use of windows for harmonic analysis with the discrete fourier transform. Proceedings of the IEEE 66(1):51–83

    Article  Google Scholar 

  • Hennessey C, Noureddin B, Lawrence P (2006) A single camera eye-gaze tracking system with free head motion. In: Proceedings of the 2006 symposium on Eye tracking research & applications, pages 87–94. ACM

  • Huelsbusch M, Blazek V (2002) Contactless mapping of rhythmical phenomena in tissue perfusion using PPGI. In: Medical Imaging 2002, pp 110–117. International Society for Optics and Photonics

  • Hyvärinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans Neural Netw 10(3):626–634

    Article  Google Scholar 

  • Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4):411–430

    Article  Google Scholar 

  • Hyvärinen A, Karhunen J, Oja E (2004) Independent component analysis, vol 46. Wiley, Hoboken

    Google Scholar 

  • Jiang D, Zhijian W, Kang L, Cao B, Li K (2006) A new method used in gene expression programming: GRCM. J Syst Simul 18(6):1466–1468

    Google Scholar 

  • Jiang D, Wang Z, Sun H, Du Y (2010) A unified fitness calculation method for automatic modeling algorithms. In: Intelligent control and automation (WCICA), 2010 8th World congress on, pp 1569–1573. IEEE

  • Lakens D (2013) Using a smartphone to measure heart rate changes during relived happiness and anger. IEEE Trans Affect Comput 2:238–241

    Article  Google Scholar 

  • Lee C, Jang MG (2009) Fast training of structured svm using fixed-threshold sequential minimal optimization. ETRI J 31(2):121–128

    Article  Google Scholar 

  • Lewandowska M, Rumiński J, Kocejko T, et al. (2011) Measuring pulse rate with a webcama non-contact method for evaluating cardiac activity. In: Computer science and information systems (FedCSIS), 2011 Federated conference on, pp 405–410. IEEE

  • Li X, Hu B, Dong Q, Campbell W, Moore P, Peng H (2011) Eeg-based attention recognition. In: Pervasive computing and applications (ICPCA), 2011 6th International conference on, pp 196–201. IEEE

  • Liang Z, Sun J, Lin Q, Zhihua D, Chen J, Ming Z (2016) A novel multiple rule sets data classification algorithm based on ant colony algorithm. Appl Soft Comput 38:1000–1011

    Article  Google Scholar 

  • Lindsley DB (1950) Emotions and the electroencephalogram

  • Lowd D, Domingos P (2005) Naive bayes models for probability estimation. In: Proceedings of the 22nd international conference on machine learning, pp 529–536. ACM

  • Monkaresi H, Hussain MS, Calvo RA (2014) Using remote heart rate measurement for affect detection. In: FLAIRS Conference

  • Palmieri F, Fiore U, Castiglione A (2014) A distributed approach to network anomaly detection based on independent component analysis. Concurr Comput Pract Exp 26(5):1113–1129

    Article  Google Scholar 

  • Pavlidis I, Dowdall J, Sun N, Puri C, Fei J, Garbey M (2007) Interacting with human physiology. Comput Vis Image Underst 108(1):150–170

    Article  Google Scholar 

  • Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23(10):1175–1191

    Article  Google Scholar 

  • Poh MZ, McDuff DJ, Picard RW (2010a) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18(10):10762–10774

    Article  Google Scholar 

  • Poh MZ, Swenson NC, Picard RW (2010b) Motion-tolerant magnetic earring sensor and wireless earpiece for wearable photoplethysmography. IEEE Trans Inf Technol Biomed 14(3):786–794

    Article  Google Scholar 

  • Qi X, Zhao G, Shen L, Li Q, Pietikäinen M (2016) Load: local orientation adaptive descriptor for texture and material classification. Neurocomputing 184:28–35

    Article  Google Scholar 

  • Shen J, Tan H, Wang J, Wang J, Lee S (2015) A novel routing protocol providing good transmission reliability in underwater sensor networks. J Internet Technol 16(1):171–178

    Google Scholar 

  • Song X, Zhang J, Han Y, Jiang J (2016) Semi-supervised feature selection via hierarchical regression for web image classification. Multimed Syst 22(1):41–49

    Article  Google Scholar 

  • Van Loan C (1992) Computational frameworks for the fast Fourier transform, vol 10. SIAM

  • Verkruysse W, Svaasand LO, Nelson JS (2008) Remote plethysmographic imaging using ambient light. Opt Express 16(26):21434–21445

    Article  Google Scholar 

  • Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Computer vision and pattern recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE computer society conference on, vol 1, pp I–511. IEEE

  • Wang XZ, Ashfaq RAR, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196

    Article  MathSciNet  Google Scholar 

  • Wei L, Tian Y, Wang Y, Ebrahimi T, Huang T (2012) Automatic webcam-based human heart rate measurements using laplacian eigenmap. In: Computer vision–ACCV 2012, pp 281–292. Springer

  • Wen X, Shao L, Xue Y, Fang W (2015) A rapid learning algorithm for vehicle classification. Inf Sci 295:395–406

    Article  Google Scholar 

  • Whitley D (1994) A genetic algorithm tutorial. Stat Comput 4(2):65–85

    Article  Google Scholar 

  • Wieringa FP, Mastik F, Van der Steen AFW (2005) Contactless multiple wavelength photoplethysmographic imaging: a first step toward spo2 camera technology. Ann Biomed Eng 33(8):1034–1041

    Article  Google Scholar 

  • Wilson N, Johnston FD, Rosenbaum FF, Erlanger H, Kossmann CE, Hecht H, Cotrim N, de Oliveira RM, Scarsi R, Barker PS (1944) The precordial electrocardiogram. Am Heart J 27(1):19–85

    Article  Google Scholar 

  • Xia Z, Wang X, Sun X, Liu Q, Xiong N (2014a) Steganalysis of LSB matching using differences between nonadjacent pixels. Multimed Tools Appl 75(4):1974–1962

  • Xia Z, Wang X, Sun X, Wang B (2014b) Steganalysis of least significant bit matching using multi-order differences. Sec Commun Netw 7(8):1283–1291

    Article  Google Scholar 

  • Xia Z, Wang X, Sun X, Wang Q (2016) A secure and dynamic multi-keyword ranked search scheme over encrypted cloud data. IEEE Trans Parallel Distrib Syst 27(2):340–352

  • Xia Z, Wang X, Sun X, Liu Q, Xiong N (2016) Steganalysis of lsb matching using differences between nonadjacent pixels. Multimed Tools Appl 75(4):1947–1962

    Article  Google Scholar 

  • Xie S, Wang Y (2014) Construction of tree network with limited delivery latency in homogeneous wireless sensor networks. Wirel Pers Commun 78(1):231–246

    Article  Google Scholar 

  • Yang R, Wang Z (2015) Cross-oriented choquet integrals and their applications on data classification. J Intell Fuzzy Syst 28(1):205–216

    MathSciNet  MATH  Google Scholar 

  • Yuwu L, Lai Z, Fan Z, Cui J, Zhu Q (2015) Manifold discriminant regression learning for image classification. Neurocomputing 166:475–486

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank anonymous reviewers for their very detailed and helpful review. This study was funded by National Natural Science Foundation of China (61502291, 61573157 and 61561024), the Cultivation Project for Outstanding Young Teachers in Higher Education Institutions of Guangdong Province (YQ2015070), the Characteristic Innovation Project in Higher Education Institutions of Guangdong Province (2015KTSCX039, 2015GXJK037), the Shantou University National Foundation Cultivation Project (NFC15005) and the Science Foundation of Jiangxi University of Science and Technology (NSFJ2015-K13).

Author information

Authors and Affiliations

  1. Department of Computer Science, Shantou University, 515063, Shantou, China

    Dazhi Jiang, Bo Hu & Yifei Chen

  2. School of Computer and Software, Nanjing University of Information Science and Technology, 210044, Nanjing, China

    Yu Xue

  3. College of Mathematics and Informations, South China Agricultural University, 510642, Guangzhou, China

    Wei Li

  4. School of Information Engineering, JiangXi University of Science and Technology, 341000, Ganzhou, China

    Wei Li

  5. College of Computer Science and Software Engineerig, Shenzhen University, 518060, Shenzhen, China

    Zhengping Liang

Authors
  1. Dazhi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yifei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhengping Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengping Liang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, D., Hu, B., Chen, Y. et al. Recognizing the human attention state using cardiac pulse from the noncontact and automatic-based measurements. Soft Comput 22, 3937–3949 (2018). https://doi.org/10.1007/s00500-017-2604-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2604-9

Keywords

Search

Navigation