Abstract
Remote photoplethysmography (rPPG) can achieve contactless human vital signs monitoring, but its signal quality is limited by the remote operation nature. In practical applications, improving the rPPG signal quality becomes an essential task. As a remote imaging technique, rPPG utilizes a camera to capture a video of a skin area, especially the facial area, then focuses on a particular sub-area as the region of interest (ROI). In this paper, we investigated a novel adaptive ROI (AROI) approach for improving the rPPG signal quality. In this approach, block-based spatial-temporal division is performed on a captured face video. Based on these segmented video pipelines, the spatial-temporal quality distribution of the rPPG signals is estimated using a signal-to-noise ratio (SNR) feature. Afterwards, AROIs are calculated through mean-shift clustering and adaptive thresholding in SNR maps. As the AROI can be dynamically adjusted according to the spatial-temporal quality distribution of rPPG signals on the face, the quality of the final recovered rPPG signal is improved. The performance of the proposed AROI approach was evaluated with both still and moving subjects. Compared to conventional ROI methods for rPPG, the proposed AROI obtained a higher accuracy in heart rate measurement. And the state-of-the-art motion-resistant rPPG techniques can be effectively enhanced through being integrated with the AROI.
Similar content being viewed by others
References
Allen J (2007) Photoplethysmography and its application in clinical physiological measurement. Physiol Meas 28:R1–R39. doi:10.1088/0967-3334/28/3/R01
Bay H, Ess A, Tuytelaars T, Gool LV (2008) SURF: speeded up robust features. Comput Vis Image Und 110:346–359. doi:10.1007/11744023_32
Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307–310. doi:10.1016/S0140-6736(86)90837-8
Bland JM, Altman DG (2007) Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat 17:571–582. doi:10.1080/10543400701329422
Bousefsaf F, Maaoui C, Pruski A (2014) Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate. Biomed Signal Process Control 8:568–574. doi:10.1016/j.bspc.2013.05.010
Comaniciu D, Meer P (2002) Mean shift: A robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24:603–619. doi:10.1109/34.1000236
Crowe JA, Damianou D (1992) The wavelength dependence of the photoplethysmogram and its application to pulse oximetry. In: Proc IEEE Annu Int Conf Eng Med Biol Soc. p 2423–2424. doi:10.1109/IEMBS.1992.5761522
Feng L (2015a) Block-based adaptive ROI for remote photoplethysmography (still subject) https://youtu.be/7bR8pEM5bnc. Accessed 24 July 2015
Feng L (2015b) Block-based adaptive ROI for remote photoplethysmography (moving subject) https://youtu.be/TR_fkny2tt0. Accessed 24 July 2015
Feng L, Po LM, Xu X, Li Y, Ma R (2014) Motion resistant remote imaging photoplethysmography based on optical properties of skin. IEEE Trans Circuits Syst Video Technol 25:879–891. doi:10.1109/TCSVT.2014.2364415
Fitzpatrick TB (1975) Sun and skin (Soleil et peau). J Med Esthétique 2:33–34. doi:10.1159/000251345
Haan G, Jeanne V (2013) Robust pulse rate from chrominance-based rPPG. IEEE Trans Biomed Eng 60:2878–2886. doi:10.1109/TBME.2013.2266196
Han J, Zhang D, Hu X, Guo L, Ren J, Wu F (2015) Background prior-based salient object detection via deep reconstruction residual. IEEE Trans Circuits Syst Video Technol 25:1309–1321. doi:10.1109/TCSVT.2014.2381471
Hayes MJ, Smith PR (1998) Artifact reduction in photoplethysmography. Appl Opt 37:7437–7446. doi:10.1364/AO.37.007437
He Z, Yi S, Cheung YM, You X, Tang YY (2016) Robust object tracking via key patch sparse representation. IEEE Trans Cybern 47:354–364. doi:10.1109/TCYB.2016.2514714
Holton BD, Mannapperum K, Lesniewski PJ, Thomas JC (2013) Signal recovery in imaging photoplethysmography. Physiol Meas 34:1499–1511
Huang J, You X, Yuan Y, Yang F, Lin L (2010) Rotation invariant iris feature extraction using Gaussian Markov random fields with non-separable wavelet. Neurocomputing 73:883–894. doi:10.1016/j.neucom.2009.09.016
Jacob B, Chen J, Huang Y, Cohen I (2009) Pearson correlation coefficient. In: Cohen I (ed) Noise reduction in speech processing. Springer, Berlin, pp 1–4
Jones MJ, Rehg JM (2002) Statistical color models with application to skin detection. Int J Comput Vis 46:81–96. doi:10.1023/A:1013200319198
Kawasaki N (2011) Improving motion robustness of contact-less monitoring of heart rate using video analysis. MS thesis, Department of Mathematics and Computer Science, Eindhoven University of Technology
Lempe G, Zaunseder S, Wirthgen T, Zipser S, Malberg H (2013) ROI selection for remote photoplethysmography. Bildverarbeitung für die Medizin p:99–103. doi:10.1007/978-3-642-36480-8_19
Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proc IJCAI. p 674–679 https://cecas.clemson.edu/~stb/klt/lucas_bruce_d_1981_1.pdf
Ma X, Liu Q, He Z, Zhang X, Chen WS (2016) Visual tracking via exemplar regression model. Knowl-Based Syst 106:26–37. doi:10.1016/j.knosys.2016.05.028
Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst, Man, Cybern, Syst SMC-9(1):62–66. doi:10.1109/TSMC.1979.4310076
Poh M, McDuff DJ, Picard RW (2010) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18:10762–10774. doi:10.1364/OE.18.010762
Sun Y, Hu S, Azorin-Peris V, Greenwald S, Chambers J (2011) Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise. J Biomed Opt 16:077010–1–077010‑9. doi:10.1117/1.3602852
Sun Y, Hu S, Azorin-Peris V, Kalawsky R, Greenwald S (2013) Noncontact imaging photoplethysmography to effectively access pulse rate variability. J Biomed Opt 18:061205–061205. doi:10.1117/1.JBO.18.6.061205
Tarassenko L, Villarroel M, Guazzi A, Jorge J, Clifton DA, Pugh C (2014) Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol Meas 35:807–831. doi:10.1088/0967-3334/35/5/807
Tomasi C, Kanade T (1991) Detection and tracking of point features. Carnegie Mellon University Technical Report CMU-CS-91-132 https://cecas.clemson.edu/~stb/klt/tomasi-kanade-techreport-1991.pdf
Tsouri GR, Kyal S, Dianat S, Mestha LK (2012) Constrained independent component analysis approach to nonobtrusive pulse rate measurements. J Biomed Opt 17:077011–1–077011‑4. doi:10.1117/1.JBO.17.7.077011
Verkruysse W, Svaasand LO, Nelson JS (2008) Remote plethysmographic imaging using ambient light. Opt Express 16:21434–21445. doi:10.1364/OE.16.021434
Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proc CVPR. p I-511–I-518. doi: 10.1109/CVPR.2001.990517
Wu HY, Rubinstein M, Shih E, Guttag JV, Durand F, Freeman WT (2012) Eulerian video magnification for revealing subtle changes in the world. ACM Trans Graph 31:65–1–65–8. doi:10.1145/2185520.2185561
Zarzoso V, Comon P (2010) Robust independent component analysis by iterative maximization of the kurtosis contrast with algebraic optimal step size. IEEE Trans Neural Netw 21:248–261. doi:10.1109/TNN.2009.2035920
Zhang D, You X, Wang P, Yanushkevich SN, Tang YY (2009) Facial biometrics using nontensor product wavelet and 2D discriminant techniques. Int J Pattern Recogn 23:521–543. doi:10.1142/S0218001409007260
Zhang D, Han J, Han J, Shao L (2016) Cosaliency detection based on intrasaliency prior transfer and deep intersaliency mining. IEEE Trans Neural Netw Learn Syst 27:1163–1176. doi:10.1109/TNNLS.2015.2495161
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Po, LM., Feng, L., Li, Y. et al. Block-based adaptive ROI for remote photoplethysmography. Multimed Tools Appl 77, 6503–6529 (2018). https://doi.org/10.1007/s11042-017-4563-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-4563-7