Resting and Postexercise Heart Rate Detection From Fingertip and Facial Photoplethysmography Using a Smartphone Camera: A Validation Study

Background: Modern smartphones allow measurement of heart rate (HR) by detecting pulsatile photoplethysmographic (PPG) signals with built-in cameras from the fingertips or the face, without physical contact, by extracting subtle beat-to-beat variations of skin color. Objective: The objective of our study was to evaluate the accuracy of HR measurements at rest and after exercise using a smartphone-based PPG detection app. Methods: A total of 40 healthy participants (20 men; mean age 24.7, SD 5.2 years; von Luschan skin color range


Introduction
There are over 100,000 health-related apps in the health and fitness category in the Google Play Store and the iTunes App Store designed for mobile devices (ie, smartphones or tablets) [1,2].The number of health-related apps is increasing by 25% each year [3].Mobile device usage has constantly been on the rise over the last few years [4], with an estimated 6.9 billion subscriptions globally [5].In the United States, 64% of adults own at least one smartphone [6], and 62% of smartphone owners have used their phone to obtain health information.In Europe, 50% of citizens own a smartphone [7].In the Asia Pacific region [8], the number of smartphone users is estimated to reach 1.2 billion in 2017.With technological advances and the increasing trend of mobile device usage, mHealth is becoming popular and is seen as an opportunity to promote health and fitness, provide health maintenance, or enhance lifestyle management [9][10][11].Compared with home health devices and computers, personal mobile devices with health-related apps installed are more convenient, portable, and accepted [12,13].Older adults are more likely to own a smartphone than a computer [11].An estimated 19 million people use mobile health devices worldwide [14].
Over the last few years, multiple smartphone devices and apps were developed to facilitate heart rate (HR) monitoring.Monitoring of HR during exercise can be used to assess fitness level and intensity of exercise.It is also useful for patients who are taking medications that affect HR to guide disease management.Recently, several smartphone apps capable of measuring HR and detecting arrhythmia have also been reported [15,16].Modern smartphones allow measurement of HR by detecting pulsatile photoplethysmographic (PPG) signals with built-in cameras from the fingertips or the face without physical contact by extracting subtle beat-to-beat variations of skin color that is similar to HR fluctuations [17,18].The PPG signal is typically recorded by placing a finger over the camera lens, which measures color changes due to fingertip blood volume changes [19].Facial PPG recording using the smartphone camera is a novel method of detecting pulsatile PPG signal without physical contact [20,21].
HR estimation using the built-in smartphone cameras may provide readily accessible, inexpensive, and user-friendly means to measure HR without additional hardware such as wrist bands or watches.However, validation of HR measurements detected from smartphone-based PPG apps is limited [22][23][24].This study aimed to evaluate the accuracy of HR measurements at rest and after exercise using the Cardiio-Heart Rate Monitor + 7 Minute Workout (Cardiio) smartphone-based PPG detection app compared with a continuous electrocardiogram (ECG) as the reference.

Participants and Recruitment
We recruited 40 healthy participants between 18 and 40 years old with no current medical conditions and who were not taking regular medications.Demographics characteristics were collected.Body height and weight were measured under standard anthropometry procedures, and body mass index was calculated as weight in kilograms divided by height in square meters.Blood pressure was measured using an automatic blood pressure monitor (Tango M2, SunTech Medical, Inc., NC, USA) before and after testing procedures.The participants were evaluated for facial skin color by using the von Luschan skin color chart (range 1-36) [25].This study was approved by the Joint Chinese University of Hong Kong -New Territories East Cluster Clinical Research Ethics Committee (CREC Ref. No. 2016.550).

Study Setup
We set up 2 iPhones (iPhone 6S; Apple, Inc, Cupertino, CA, USA) and a 12-lead ECG treadmill (GE Series 2000, GE Medical Systems Information Technologies Inc, Milwaukee, WI, USA) for HR measurements.Backdrop and background light intensity was standardized during signal acquisition and was captured in the unit of lux.The Cardiio (Cardiio Inc, Cambridge, MA, USA) smartphone app was installed in the 2 iPhones for facial and fingertip PPG detection.HR measurements were taken by continuous 12-lead ECG, and facial and fingertip PPG detection simultaneously.Each participant was given at least a 5-minute rest interval before testing began.We took 3 measurements of resting HR and averaged them for analysis before exercise.Participants were then tested on a motorized treadmill using the Bruce test protocol from stage 1 to stage 4 (2.7, 4.0, 5.5, and 6.8 km/h) [26].Each participant underwent 2 sequential tests on the treadmill to achieve (1) moderate-intensity exercise, defined as 50% to 70% maximum HR, and (2) vigorous-intensity exercise, defined as 70% to 85% maximum HR.Maximum HR was calculated as 220 beats/min minus the participant's age [27].HR recordings were acquired immediately when moderate-and vigorous-intensity exercise was achieved based on HR from the continuous ECG and repeated successively until return to resting HR.

12-Lead Electrocardiogram
Continuous 12-lead ECG was our reference standard of HR measurement.We compared HRs detected by the Cardiio smartphone app with simultaneous 12-lead ECG recordings.

Facial PPG Detection
We asked participants to sit in front of an iPhone placed upright on a desk approximately 30 cm away.Once the Cardiio smartphone app was activated, a large circle appeared on the screen, and each participant needed to position the image of his or her entire face within the circle so that it was captured by the front camera (Figure 1).Participants were instructed to hold still for 20 seconds during each measurement.The continuous pulsatile PPG signal from the face detected by the camera was displayed on the bottom of the iPhone screen.An estimated HR measurement analyzed by the Cardiio smartphone app was displayed as a result.Participants were asked to keep their movements to a minimum and not to speak during measurement.

Fingertip PPG Detection
Fingertip PPG signals were recorded for each participant by placing their index fingertip over the iPhone's back camera for 20 seconds.Once the finger was placed in contact and illuminated by the adjacent LED flash, a continuous pulsatile PPG signal from the fingertip was detected by the camera and displayed on the bottom of the iPhone screen (Figure 1).It was analyzed by the Cardiio smartphone app for an estimated HR measurement and displayed as a result.Participants were asked to keep their movements to a minimum and not to speak during measurement.

Statistical Analysis
We present numerical results as mean (SD) or median (interquartile range, IQR).Pearson correlation (r) and coefficient of determination were applied to determine the nature of the relationship between the HR measurements made by the Cardiio smartphone app (face and fingertip PPG detection) and by the standard 12-lead ECG.We performed the paired Student t test and Wilcoxon signed rank test to determine the difference between the means and medians measured by both devices.Root mean square error (RMSE) was performed to evaluate the spread of errors between predicted and observed values.Bland-Altman plots were used to examine agreement between ECG and PPG-estimated HR.All statistical analyses were performed using IBM SPSS statistical software (IBM SPSS Statistics for Windows, version 22.0; IBM Corporation).All analyses were 2-tailed, and P values of <.05 were considered statistically significant.
We recorded a total of 665 fingertip and 665 facial PPG measurements.Boxplots were used to determine extreme outliers for any observation outside the upper and lower fences that were 3 times the IQR.Potential outliers were checked for accuracy before exclusion.Failed measurements (n=4 facial measures) and extreme outliers were possibly caused by the monitor losing fingertip skin contact or misalignment of the face.We excluded 69 outliers for PPG measurements representing unrealistic HR values (n=34 fingertip measures and n=35 facial measures), resulting in 631 fingertip and 626 facial PPG measurements for analysis.

Characteristics of Participants
Table 1 summarizes participants' characteristics.Of the 40 healthy participants, 50% (n=20) were men, and the mean (SD) age was 24.7 (5.2) years.The median von Luschan skin color among participants was 23 (IQR [19][20][21][22][23][24][25]; the lightest skin color was 14 and the darkest was 27.Male participants had a higher body mass index, resting systolic blood pressure (SBP), and resting diastolic blood pressure (DBP) than the female participants.There were no significant differences in postexercise SBP and DBP between the sexes.The median backdrop and background light intensity during signal acquisition was 199 lux (IQR 127-249).

Fingertip PPG-Estimated Heart Rate
We analyzed 80 averaged resting HR values from 234 PPG measurements, and overall 397 postmoderate-and postvigorous-intensity exercise HR values.

Facial PPG-Estimated Heart Rate
We analyzed 80 averaged resting HR values from 237 facial PPG measurements and a total of 389 postexercise HR values.
The Cardiio smartphone app had 95% of resting HR measurements fall within -1.72 and +2.30 beats/min, while postexercise HR measurements were wider and within -6.36 and +8.37 beats/min, as illustrated in the Bland-Altman plots (Figure 3).

Principal Findings
According to the literature, HR monitors are considered accurate and regarded as excellent when r ≥.93 and RMSE<6.8%[28,29].The American National Standards Institute/Association for the

RenderX
Advancement of Medical Instrumentation EC-13 standard states that the accuracy requirements for HR monitors are RMSE≤5 beats/min or ≤10%, whichever is greater [30].Our results showed that the Cardiio smartphone app using both fingertip and facial PPG detection can be regarded as excellent and considered accurate in measuring HR at rest, and after moderateto vigorous-intensity exercise.
PPG estimation of resting HR from both the fingertip and the face demonstrated very high accuracy.The RMSE were less than 1.5% (RMSE=1.40%from fingertip and RMSE=1.44%from facial PPG measurements).The errors between PPG-estimated resting HR and ECG HR were <1%, as illustrated by the coefficients of determination (Figure 2).
In particular, the results of fingertip PPG measurements were consistent and accurate in the different physical activity levels tested in this study.Fingertip PPG-estimated HRs were strongly correlated with both resting and postexercise ECG HR (all r ≥.99; Table 4).HR was underestimated with mean differences of <0.5 beats/min or median differences of 0 beats/min at rest and after exercise compared with the reference ECG (Table 3).
For facial PPG measurements, the accuracy of HR estimation was better at rest and diminished with exercise.The coefficient of determination of facial PPG-estimated HR was stronger with resting ECG HR (R 2 >.99) than postexercise ECG HR (R 2 ≥.96) (Table 3).For postexercise HR detection, facial PPG underestimated HR values with mean differences of approximately 1 beat/min or median differences of 1-2 beats/min when compared with ECG HR (Table 3).Possible explanations are blushing and excessive facial motion due to heavy breathing after exercise leading to misalignment of the face with the camera.
Overall, the Bland-Altman plots showed better agreement between ECG and fingertip-estimated HR than between ECG and facial PPG-estimated HR, and that 95% limits of agreement were wider for facial than for fingertip PPG estimation (Figure 3).The results demonstrated the ability of a PPG-based smartphone app to provide meaningful and accurate readings at rest and after exercise.

Limitations
Limitations of our study include that (1) this was a convenience sample of healthy young adults, which may limit generalizability; and (2) the method of HR detection from fingertip and facial PPG using iPhone's front and back cameras, which may not apply to other smartphone cameras.We recommend further evaluation of the accuracy of taking HR measurements in a real-world environment under nonstandardized background lighting and in expanded exercise settings.We suggest conducting further studies including a larger number of participants with an extended age range and more skin tone colors to increase generalizability.This study did not include patients with arrhythmia or other heart-related problems, which is an area for future investigation.Future work could also study user feedback relating to health-sensing mobile apps.

Conclusions
Smartphone use is becoming ubiquitous.People are increasingly relying on smartphones and health-related apps for health care purposes.Smartphone apps must be validated for their accuracy, reliability, and effectiveness in providing health care benefits.Capturing a PPG signal with the built-in smartphone cameras may provide a readily accessible and inexpensive means to measure HR.Our results have demonstrated that HR detection by the Cardiio smartphone app is accurate at rest and after moderate-and vigorous-intensity exercise in a healthy young adult sample.Both fingertip and facial PPG have shown high accuracy in measuring resting HR with reference to an ECG.Although touchless facial PPG detection is more convenient, fingertip PPG is more accurate for HR detection after exercise.

Figure 1 .
Figure 1.(A) Cardiio smartphone app.(B) Setup to acquire photoplethysmographic (PPG) signals from the participant's face.(C) Obtaining PPG signals from the fingertip.(D) Example of a report produced by the Cardiio smartphone app.

Figure 3
Figure3presents the Bland-Altman plots with 95% limits of agreement.The Cardiio smartphone app had 95% of resting HR measurements fall within -1.98 and +2.07 beats/min, while postexercise HR measurements were within -3.81 and +4.32 beats/min.

Figure 2 .
Figure 2. Scatter plots comparing measurements of heart rate (HR) estimated from the Cardiio smartphone phone app photoplethysmographic (PPG) signals and from a reference electrocardiogram (ECG).P<.001 for all correlations.(A) Resting estimated HR from fingertip PPG signals.(B) Resting estimated HR from facial PPG signals.(C) Postexercise HR from fingertip PPG signals.(D) Postexercise HR from facial PPG signals.

Figure 3 .
Figure 3. Bland-Altman plots of limits of agreement in resting heart rate (HR) estimated from the Cardiio smartphone app and a reference electrocardiogram (ECG).(A) Resting estimated HR from fingertip PPG signals.(B) Resting estimated HR from facial PPG signals.(C) Postexercise HR from fingertip PPG signals.(D) Postexercise HR from facial PPG signals.

Table 1 .
Baseline characteristics of the study participants.

Table 3 .
Accuracy of measuring heart rate (HR) using Cardiio smartphone app compared with reference electrocardiogram.

Table 4 .
Correlation test (r) a of heart rate between reference electrocardiogram (ECG) and Cardiio smartphone app.P<.001 for all correlations (correlation is significant at the .01level, 2-tailed). a