Automated analysis of spontaneous eye blinking in patients with acute facial palsy or facial synkinesis

Although patients with facial palsy often complain of disturbed eye blinking which may lead to visual impairment, a blinking analysis is not part of routine grading of facial palsy. Twenty minutes of spontaneous eye blinking at rest of 30 patients with facial palsy (6 with acute palsy; 24 patients with facial synkinesis; median age: 58 years, 67% female), and 30 matched healthy probands (median age: 57 years; 67% female) was smart phone video recorded. A custom computer program automatically extracted eye measures and determined the eye closure rate (eye aspect ratio [EAR]), blink frequency, and blink duration. Facial Clinimetric Evaluation (FaCE), Facial Disability Index (FDI) were assessed as patient-reported outcome measures. The minimal EAR, i.e., minimal visible eye surface during blinking, was significantly higher on the paretic side in patients with acute facial palsy than in patients with synkinesis or in healthy controls. The blinking frequency on the affected side was significantly lower in both patient groups compared to healthy controls. Vice versa, blink duration was longer in both patient groups. There was no clear correlation between the blinking values and FaCE and FDI. Blinking parameters are easy to estimate automatically and add a functionally important parameter to facial grading.


Automated blinking analyses
The Jena Facial Palsy Toolbox (JeFaPaTo) was used for the analyses 14 .In brief, JeFaPoTo performs first an automatic face detection in the imported video.Then, using the mediapipe library 468 facial landmarks and 52 blend shape features are extracted 21,22 .With the landmarks around the eye, the eye aspect ratio (EAR; Fig. 1) can be calculated for both eyes over all frames of the video 23 .EAR describes the ratio between the vertical and horizontal distance between the landmarks, resulting in a detailed behavior approximation of the upper and lower eyelids.Hence, the EAR is characterizing the eye openness in each frame and invariant to the distance of the eye to the camera.The EAR is getting close to zero when closing the eye in a healthy person.The lower EAR, the better is the eye closure function.Furthermore, the blinks for both eyes were detected and counted.

Statistics
All statistical analyses were performed using IBM SPSS Statistics 25 (Chicago, IL).The results had exploratory character as no measurements with the tool box had been performed in patients with facial palsy before.Hence, no data were available to determine a concrete assumption on the blinking in patients compared to healthy probands.Nevertheless, we performed a power analysis to get an idea of a sufficient sample size.Primary outcome measure was blinking per 20 min.Normal average spontaneous blinking is about 15/min, i.e. 300/20 min.Pooled for patients (acute and chronic palsy), we assumed a reduction to 100/20 min.Further, we assumed the same standard deviation of 200/20 min in probands and patients.Based on these assumptions, the power calculation revealed at a test level of alpha = 0.05, in each group (probands and patients) N = 23 participants had to be analyzed (two-sided independent samples) with a power of 95%.Therefore, we decided to include N = 30 patients and N = 30 matched healthy probands into the study.
Nominal and ordinal data are presented as absolute values and relative values in percentage.The results of the metric parameters are presented as means ± standard deviation (SD), median and range, if not otherwise indicated.In order to proof the hypothesis that spontaneous blinking parameters were impaired in patients compared to healthy controls, one factor analysis of variance (ANOVA) with post-hoc Bonferroni correction for multiple testing was used for all independent blinking parameters of all three subgroups (acute palsy, synkinesis, healthy).As the healthy controls had no paretic side, it was necessary to define which side (left/right) should be compared to the paretic side in the patients and which side to the contralateral side.The results for all blinking parameters for left and right side were not different in the controls (see "Results").Therefore, it was determined to compare the paretic side of patients to the left side of controls.The contralateral side of patients was compared to the right side of controls.The Wilcoxon test was used to compare dependent parameters between two subgroups (paretic versus contralateral side).In order to proof the hypothesis that impaired blinking correlated to impaired quality of life, Spearman's rho was used to perform the correlations analysis between the blinking parameters and the results of the PROMs.As the correlation analyses had exploratory character, no correction for multiple test was performed.P values < 0.05 were considered significant.

Ethics statement
Written informed consent was obtained from all participants.Informed consent has also been obtained to publish the facial images in a publication.The ethics committee of the Jena University Hospital approved the study (No. 2019-1539).

Characteristics and facial-specific quality of the life of the of patients and the healthy controls
Most of the patients with acute facial palsy had an idiopathic facial palsy (83.3%).Infection (37.5%) and trauma/ tumor (33.3%) were to most frequent etiologies in the patients with facial synkinesis.The House-Brackmann grading varied from grade II to grade VI in the patients with acute palsy and from grade II to grade V in the patients with facial synkinesis.The median Sunnybrook Composite Score for the patients with acute facial palsy and for patients with facial synkinesis was 36.5 and 67, respectively.The median Sunnybrook Synkinesis Score of the patients with facial synkinesis was 6.5.More details are given in Table 1.The results of the facial-specific quality of life assessments are shown in Table 2.The one-way ANOVAs revealed that there was a statistically significant difference in all quality of life scores between at least two groups (Supplemental Table 1).As expected, the FaCE and FDI parameters were normal in the healthy controls.The FaCE and FDI domains were all decreased

Comparison of blinking on the paretic and the contralateral side
All blinking analysis parameters for both facial sides are listed for the in Table 3.There was no side difference in the healthy controls (all p > 0.05).In patients with acute facial palsy, the ratio of the length of the palpebral fissure height (i.e. the highest to lowest point of the palpebral fissure) on the diseased side to the contralateral side was 96 ± 7%.The average and maximum EAR were not different (p = 0.248 and p = 0.345, respectively).The minimal EAR was greater on the paretic side, i.e. the ability to close the eye was lower (p = 0.028).The number of blink in 20 min and therefore also the blinking frequency was reduced on the paretic side (both p = 0.027).
The same was seen for the subset of blinks with complete eye closure (both p = 0.028).The average duration of the blinks showed no side difference (p = 0.180).In patients with facial synkinesis, nearly all parameters were changed on the paretic side.The ratio of the length of the palpebral fissure height was 88 ± 18%.There was a trend to lower ratio compared to patients with acute facial palsy (p = 0.080).The average and the maximum EAR were reduced (p = 0.005 and p = 0.006, respectively), whereas the minimum EAR was larger than on the contralateral side (p = 0.004).The maximal EAR on the contralateral side in patients with facial synkinesis was reached the highest values from all sides.It might be that these patients actively make their contralateral eye more open to cope with the synkinesis on the paretic side.Blinking frequency was reduced on the post-paralytic synkinetic side (p < 0.001).The same was seen when only analyzing the blinks with complete eye closure (p < 0.001).There was a non-significant trend of a longer average duration of each blink on the synkinetic side (p = 0.080).The time course of the average number of blinks and the average duration of each blink during the 20 min observation time is shown in Fig. 2. The time course of only the number of blinks with complete eye closure is shown in Supplement Fig. 2. The number of blinks varied from minute to minute in healthy probands and in the patients.In contrast, the duration of each blink was relatively constant except for patients with acute palsy.Here, the duration of the blinks varied considerably.

Comparison of blinking between heathy probands and both patient groups
The comparison of the paretic side in the two patient groups and the left side in the healthy probands is shown in Table 4.The one-way ANOVAs revealed that there was a statistically significant difference in most blinking parameters on the paretic side (left side in controls) between at least two groups (Supplemental Table 2).Hence, the objective to show that the used tool box allowed an automated and objective confirmation of the impaired blinking in patients was confirmed.Regarding the EAR, only the minimum EAR, i.e. best eye closure showed significant lower values (better closure) for healthy probands compared to patients with acute facial palsy (p < 0.001) and lower values for patients with synkinesis than for patients with acute palsy (p = 0.015).The absolute number and hence also the blinking frequency was lower in both patient groups (acute palsy and patients with synkinesis) than in healthy probands (p = 0.034 and p = 0.034, respectively).The parameter were not different between the patient groups (p = 0.893).The same was seen when only the blinks with complete eye closure were examined.
The average duration of the blinks was longer in both patients groups (p < 0.001 and p = 0.027, respectively), and also significantly longer in patients with acute palsy compared to the patients with facial synkinesis (p = 0.011).
The comparison of the contralateral side in the two patient groups and the left side in the healthy probands is shown in Table 5.The one-way ANOVAs revealed that there was no statistically significant difference in most blinking parameters on the contralateral side (right side in controls) between at least two groups (Supplemental Table 3).A difference between at least two groups was seen only for the average EAR on the contralateral side.Most parameters were not different between the three groups (all p > 0.05).Only the average EAR was higher (i.e. the eyes were more open) in the group of patients with synkinesis than in patients with acute palsy (p = 0.029) and also than in healthy probands (p = 0.017).

Correlation analysis between the blinking parameters and quality of life
An overview about the correlation analyses is given in Table 6.No correlations were seen for almost all blinking parameter and PROM values (all p > 0.05).Only in patients with acute facial palsy a better FaCE Eye Comfort www.nature.com/scientificreports/ was correlated to a higher blinking frequency (rho = 0.845; p = 0.034).Hence, the hypothesis that impaired spontaneous blinking is correlated to impaired quality was not confirmed.

Discussion
The main objective, to establish the tool box for use in clinical routine and to show the feasibility to measure objectively spontaneous blinking parameters, has been achieved by the presented study.Furthermore, the hypothesis, that impaired blinking in patients with facial palsy could be measured automatically, could be confirmed.
In contrast, the hypothesis that the blinking impairment correlates with impaired quality of life could not be confirmed.
Blinking is a dynamic facial nerve related facial function important for corneal protection and optimal vision.Spontaneous blinking consists of a stereotypic rapid downward movement of the upper eyelid and a subsequent upward movement completing the blink.This is not the same as the eye closure typically performed during facial function assessment with facial grading systems.There is no established facial grading tool for routine use estimating eye blinking function 24 .Terzis and Bruno suggested in 2002 a subjective 5-stage scoring system for grading of blinks 25 .However, this system was never used again by others.
The present study confirms that blinking is reduced on the paretic side in patients with acute facial palsy.Moreover, impaired blinking could also be confirmed for patients with facial synkinesis.The use of an automated image analysis tool allowed an easy, not time-consuming, and reliable quantification of the blinking frequency and of the blinking duration.The later was significantly prolonged in patients with facial palsy.The automated method also allowed a precise quantification of the eye opening area or the degree of eye closure by calculation of the EAR.The present results show that these parameters are also not adequately covered by facial-specific PROMs  www.nature.com/scientificreports/probands.About 7 blinks per minute, i.e. about half of all blinks produced a complete eye closure in healthy probands.In the literature using manual methods for counting, the spontaneous blink rate in adults between 50 and 70 years varies between 11 and 22 blinks per minute, i.e. the presented results fall into this range (see Fig. 5 in: 27 ).Due to classical EMG studies, the duration of the orbicularis oculi activation during a normal spontaneous blink is about 280-300 ms 33 .This fits well to the present results as EMG activity is seen before the movement occurs 33,34 .The blink frequency was decreased and the blink duration was increased both in patients with acute facial palsy and in patients with facial synkinesis.The latter is shown for the first time.Furthermore, it seems that the patients with acute facial palsy try to compensate the disturbed ipsilateral blinking with a longer blink duration on the contralateral side.It would therefore be interesting to develop a training program for voluntary blinking and to see whether this improves patients' quality of life.The sample size was too small to be able to give a definitive answer here.Studies on blinking in patients with facial palsy were so far focused on patients with acute palsy.We clearly show that disturbed blinking is also an important factor for patients with facial synkinesis.The EAR as a measure of the eye openness or closeness is a very robust parameter in automated video analysis 23 , but was not yet used as parameter in patients with facial palsy.The minimal EAR as measure of minimal openness (maximal closeness) of the eye again was disturbed not only in patients with acute facial palsy but also in patients with facial synkinesis.Schulz et al. 13 used the margin reflex distance (MRD) as parameter of the eye openness.They showed an increased MRD in patients with acute facial palsy going back to normal after recovery.Patients with facial synkinesis were not evaluated.
The present study has limitations.The group of patients with acute facial palsy in this first study using the JeFaPaTo was very small.A larger group will obtain more robust but probably not other results.Spontaneous blinks were analyzed.The other two types, voluntary and reflex blinks were not yet investigated 2 .Furthermore, spontaneous blinking was only analyzed at rest.A patient with synkinesis who is talking or smiling may close his eyes unexpectedly.This is analyzed in an ongoing study.Then, blinking activity during daily activity depends on several internal and external factors, including age, ocular surface status, level of mental activity, changes in visual processing or attention 27,35 .It would be worthwhile to analyze if spontaneous blinking is differently changed during attention and social communications tasks in patients with facial palsy compared to healthy probands 35 .
Some classical blink parameter like blink peak velocity and amplitude (that are also disturbed in the patients) are not yet implemented in the tool 36,37 .Furthermore, asymmetry of blinking might be perceived as disturbing for the patients 34 .This parameter should be implemented in the software, too.A 6-year old smart phone allowing videos with 240 frames per second was used.Nowadays, many smartphone allow such a frame rate and are ubiquitously available.Nevertheless, JeFaPaTo would also allow analysis of videos with lower frame rate.The other way round, JeFaPaTo would also allow to upload data from ultrahigh-speed cameras like they are used for basic research questions related to eye blinking 38 .
Beyond the addition of further blink parameters to the software it will be worthwhile to use the tool also for other oculofacial disease and conditions with disturbed eye blinking.Typical examples are hemifacial spasm, blepharospasm, Grave's disease, and Parkinson's disease, but also research conditions like sleep deprivation or settings with alternating attention 13,27,32,39 .

Conclusions
Automated, objective and fast analysis of spontaneous eye blinking is feasible in patients with facial palsy and postparalytic facial syndrome with synkinesis.Blinking is decreased and blink duration is prolonged not only in patients with acute facial palsy but also in patients with facial synkinesis.Although the number of blinks with complete eye closure remains decreased on patients with facial synkinesis compared to healthy controls, the minimal openness of the eye surface returns nearly back to normal.All aspects of blinking seem not to be covered by typical facial-specific PROMs, as the FDI and the FaCE do not correlate to the results of the blinking analyses.Automated blinking analysis should be used in routine grading of facial palsy, in clinical studies, and to compare groups of patients from different institutions.

Figure 1 .
Figure 1.Explanation of the eye aspect ratio (EAR).EAR describes the ratio between the vertical and horizontal distance between the automatically detected landmarks.The formula for the calculation based on the landmarks is shown in the figure.EAR is characterizing the eye openness in each frame and invariant to the distance of the eye to the camera.The EAR is getting close to zero when closing the eye in a healthy person.The dynamics from normal openness to minimal openness during eye closure over time is shown form left to the right.

Figure 2 .
Figure 2. Automated blinking analysis over 20 min for the patients with acute facial palsy (blue line), patients with facial synkinesis (red line), and healthy probands (grey line).(A, B) Average number of blinks per minute (mean ± standard error of the mean).(C, D) Average blink duration in ms (mean ± standard error of the mean).(A, C) Paretic side of the patients, left side of the healthy probands.(B, D) Contralateral side of the patients, right side of the healthy probands.

Table 1 .
Characteristics of the healthy control group and the two patients groups.M, mean; SD, standard deviation.

Table 2 .
Facial-specific quality of life of the healthy control group, acute facial palsy and postparalytic synkinesis group.FaCE, Facial Clinimetric Evaluation; FDI, Facial Disability Index; M, mean; SD, standard deviation.Significant values are in[bold].*ANOVA,post-hoctest; additional data is given in Supplemental Table1.

Table 3 .
Automated blinking analysis of the healthy control group, acute facial palsy and postparalytic synkinesis group comparing the paralytic with the contralateral side.Significant values are in[bold].*Ratio of height/width of the eye opening, average of the first 3 s of the blink-free interval.**At least pupil covered.

Table 4 .
Blinking on the paretic/left* side, comparison of the healthy control group, acute facial palsy and postparalytic synkinesis group.Significant values are in [bold].*ANOVA, post-hoc test, additional data is given in Supplemental Table 2; **left side in healthy controls.

Table 5 .
Blinking on the contralateral/right* side, comparison of the healthy control group, acute facial palsy and postparalytic synkinesis group.Significant values are in[bold].*ANOVA,post-hoctest, additional data is given in Supplemental Table3; **right side in healthy controls.