Simple and Unbiased OSA Prescreening: Introduction of a New Morphologic OSA Prediction Score

Introduction

Obstructive sleep apnea (OSA) is a chronic condition caused by recurrent partial (hypopnea) or complete (apnea) upper-airway collapse during sleep and presents with a wide range of symptoms, including snoring and daytime sleepiness.^1,2 The estimated global prevalence of moderate to severe OSA amounts to over 425 million adults.^3–5 OSA leads to significant performance impairment, increased risk for accidents and elevated risk for metabolic and cardiovascular disease, depression, dementia, and stroke. OSA is directly associated with increased overall morbidity and mortality.^6–14

Diagnosis of OSA can be complex.¹⁵ The most accurate way to diagnose OSA is the objective overnight recording of sleep and respiration, either in a sleep laboratory with polysomnography or at home with a portable polygraph system. However, these methods are expensive, time-consuming, and often not readily available.^4,16 Less expensive and early pre-screening tools include patient-reported symptoms with questionnaires or prediction scores based on risk factors or a combination of these.¹⁵ While questionnaires are practical, they rely on subjective symptom reporting, reaching very limited diagnostic accuracy and little differentiation of OSA severity.^17–20 OSA prediction scores often include some objective risk factors such as oral cavity, neck circumference, body-mass index (BMI), and craniofacial photography.²¹ However, these scores were mainly evaluated within very specific clinical risk populations (eg, obese patients, pregnant women, children, etc.), did not differentiate OSA severity, and can be less practical and of limited statistical accuracy.^22–28

Given the existing evidence, a recent review on OSA prescreening tools for the US Preventive Services Task Forces concluded that the accuracy and clinical utility of current screening tools is not sufficient.¹⁵ Thus, to identify patients at risk in a timely and cost-efficient manner, a well-performing early screening step in the diagnostic process is needed. The early and objective risk prediction of OSA and the differentiation of severity is important in a clinical context. The exclusion of OSA is essential for effective differential diagnosis. Equally important is the detection of severe OSA due to its high risk of cardiovascular disease, morbidity and overall mortality.^29,30

Our objective was to develop and test a simple OSA prediction score less prone to subjective biases as an early and effective screening tool with general clinical practicability. We aimed to include only simple to obtain objective features based on anthropometric measures and intraoral visibility, with a special focus on the accurate diagnosis of severe OSA, due to its clinical importance.

Patients and Methods

Participants and Data Collection Procedure

Our study used patient data that was collected as part of the Sleep Apnea Global Interdisciplinary Consortium (SAGIC). The Consortium is a collaborative, worldwide research project on sleep apnea in existence since 2011 with a major focus on personalized medicine approaches, craniofacial risk factors and the genetics of OSA. The participating sleep centers (University of Pennsylvania, Ohio State University, University of Sydney, University of Western Australia, University of Iceland, University of San Paulo, Korea University, Chang Gung Memorial Hospital, and the University Hospital Charité Berlin) collect data through questionnaires, standardized facial and intraoral photography, saliva samples, and polysomnography or polygraphy.

The morphological score was developed and validated with data from German SAGIC patients, using anthropometric measures and intraoral photography collected as part of SAGIC. The patients had to be 18 years or older with suspected OSA (eg, symptoms such as daytime sleepiness and non-restful sleep that were not explained by other factors). Exclusion criteria included pregnancy, history of psychiatric or neurological disorders (eg, stroke, epilepsy, or brain damage) or other relevant medical diagnoses (eg, requiring medications including beta-blocker, antihistamine, etc.) that may influence sleep, or a currently ongoing OSA therapy (eg, continuous positive airway pressure therapy). The patients underwent overnight sleep-laboratory polysomnography (SOMNOmedics GmbH, Randersacker, Germany) or portable polygraphy (Nox-T3, CareFusion Respiratory, CA, USA or Embletta, Natus Medical Incorporated, WI, USA), and completed a SAGIC questionnaire (including demographic questions, medication and medical history, etc). Anthropometric measurements (neck, waist, and hip), facial and intraoral photography were collected by the study team. Visual scoring of sleep and respiration were performed by certified sleep specialists according to AASM 2012 (American Academy of Sleep Medicine) criteria.³¹

Participation and patient recruitment for SAGIC was approved by the Institutional Ethics and Scientific Review Committee of the Charité – Universitätsmedizin Berlin, Germany (EA 1/245/12). Patients signed an informed consent form.

Intraoral Facial Photography and Modified Friedman Tongue Position

Classification of pharyngeal visibility was included in our scoring system based on intraoral photographs, although, in clinical practice classification may also be performed directly. The photographs were taken following a standardized SAGIC protocol and have been shown to provide useful anatomical data for OSA risk prediction.^32,33 In brief, a digital camera (Canon PowerShot SX 120 and 170 IS) and laser beam (single, class IIIa, 4.0–5.0 milliwatt green laser module) were connected and mounted on a monopod with tilt functionality (Manfrotto 682B professional monopod with detachable legs). Patients were prepared (eg, hair tied back, glasses removed) and certain facial landmarks (eg, gonion profile, orbital rim, etc.) were marked for craniofacial pictures (detailed description in Lee et al 2009).³² Patients were seated in correct posture and camera alignment (eg, camera approx. 40cm from subject). First, craniofacial pictures were taken (frontal, profile and neck extended photographs, described in more detail in Lee et al 2009).³² Secondly, intra-oral pictures were taken using standardized instructions on how to open the mouth. The intraoral area was photographed in eight positions, four with the tongue in normal position and four with extended tongue, each first without and then with added tools such as phonation and tongue depressor (described in more detail in Schwab et al 2017).³³

We used a modification of the Friedman Tongue Position (FTP) for an objective classification of the intraoral visibility. The FTP allows for an evaluation of the tongue’s position in relation to pillars, uvula, soft and hard palate. As the FTP uses a neutral tongue position for visibility classification, we only analyzed the intraoral pictures with neutral tongue position. As our cohort consisted of OSA risk patients, we adapted the FTP categories by including phonation and tongue depression to allow for a better differentiation of the higher categories. Patients were categorized the following way (Figure 1): I. Visibility of the uvula and palatal arch in neutral tongue position without added tools, II. Visibility of the uvula and palates in neutral tongue position only with phonation, III. Visibility of the uvula and palates in neutral tongue position only with tongue depressor, IV. No visibility of the uvula and palatal arch with or without phonation or tongue depression.

Figure 1 Modified Friedman tongue position.

Statistics

Statistical analysis was performed with SPSS (IBM SPSS Statistics, version 25). Non-parametric Spearman rank correlations were calculated between the AHI of the test cohort and several OSA risk factors to identify possible prediction variables for the score. We used non-parametric correlations as the AHI and some of the risk variables were not normally distributed. Significance level was set at p = 0.01. We weighted the identified score variables with 1–3 points based on previous studies and prediction ability.²⁴ To test the predictive ability of the score and identify score thresholds, the receiver-operating characteristics (ROC) and curve were calculated with a test cohort of n = 150 patients. The ROC curve is defined by sensitivity (true positive rate) on the y-axis and 1-specificity (false positive rate) on the x-axis. The area under the ROC curve (ROC-AUC) was used as a measure of model-predictive performance. The ROC-AUC is interpreted as the probability that a randomly selected subject with the outcome of interest (eg, case) has a larger model predicted probability than a randomly selected subject without the outcome (eg, control); thus, ROC-AUC values closer to 1.0 represent better performance.³⁴ The ROC curves were used to identify score thresholds to best predict patients with severe OSA (AHI ≥ 30) and patients with no OSA (AHI ≤ 5). Additionally, the positive and negative predictive values, the positive and negative likelihood ratios, and the test accuracy with a 95% confidence interval were calculated. These variables are based on the two-by-two outcome table of true positives, true negatives, false positives (Type I error), and false negatives (Type II error) (see Supplementary Table 1 for equations). The ROC analysis was repeated with an independent cohort of n = 50 patients to validate the results.

Results

Test and Validation Cohort

First, a total of 166 SAGIC patients were recruited to develop and test the scoring system; 16 patients were excluded from analysis due to missing or incomplete data. The remaining 150 patients made up the test cohort. Afterwards, another 50 patients were recruited as part of SAGIC and assigned to the validation cohort to validate the scoring system. The cohorts did not differ in general demographics (Table 1). Both cohorts consisted of slightly more men (test cohort: 60%; validation cohort: 56%), a mean age of almost 60 years (58±12 years; 60±12 years) and an overall mean BMI of 30±7 kg/m² (representing the overweight category). Both cohorts presented a similar range of AHI. The test cohort showed a range from 0.0 to 115.0 events/hour (mean±SD AHI: 19.7±22.0 events/hour), the validation cohort a range from 0.1 to 114.2 events/hour (mean±SD AHI: 28.2±24.5 events/hour).

Table 1 Demographics of SAGIC Patients, Separated by Test and Validation Cohort

Score Development

For the score, six objective and easy to assess variables that highly correlated with the AHI of the test cohort were included: Modified FTP (r = 0.40, p = 0.001), waist circumference (r = 0.66, p = 0.001), neck circumference (r = 0.56, p = 0.001), BMI (r = 0.53, p = 0.001), male sex (r = 0.34, p = 0.001) and age for women (r = 0.53, p = 0.001). As age was only marginally significant for the entire test cohort (r = 0.20, p=0.02) and not associated with AHI in men alone (r = 0.02, p = 0.860), we decided to use age only as a prediction variable for women. We used the same weighting system as Deflandre et al, assigning each variable with up to three points, a higher point indicating a higher AHI and OSA risk, resulting in a score range from 0 to 15 points (Table 2).²⁴

Table 2 Variables of the Scoring System with Weighting Points

The score was applied to the test cohort to evaluate its overall predictive abilities for the detection of severe OSA (AHI ≥ 30) as well as the exclusion of OSA (AHI ≤ 5), and to identify the best possible score threshold for these categories (Supplementary Table 2).

For distinguishing patients with severe OSA (AHI ≥ 30, n = 34) and those without severe OSA (AHI < 30, including patients with an AHI below 5 and therefore no OSA at all, n = 116), the ROC-AUC was 0.90 (95% CI: 0.84, 0.95) with p < 0.001. Figure 2 displays the ROC curve with sensitivity and 1-specificity for different thresholds. The score threshold of 8 showed the best balance of predictive values with both, a high sensitivity of 82% (95% CI: 65%, 93%) and a high specificity of 82% (95% CI: 74%, 88%). As a higher score indicates a higher OSA risk, a score of 8 or more points was chosen for the prediction of patients with severe OSA and achieved a test accuracy of 82% (95% CI: 75%, 88%). Table 3 presents the statistical details including the positive and negative predictive values and the population independent positive and negative likelihood ratios.

Table 3 Test Cohort (n = 150) – Receiver-Operating Characteristics for a Score ≥8 (Severe OSA), Score ≤5 (Exclusion of OSA) and Score Range 5–8 (Mild to Moderate OSA)

Figure 2 Test cohort – receiver-operating curve and score thresholds for predicting severe obstructive sleep apnea (apnea–hypopnea index ≥30). Abbreviations: ROC, receiver-operating characteristics; SEN, sensitivity; SPE, specificity. Notes: Left: ROC curve for score prediction of severe obstructive sleep apnea (apnea–hypopnea index ≥ 30) within the test cohort. The curve is defined by sensitivity (true positive rate) on the y-axis and 1-specificity (false positive rate) on the x-axis. Points above the diagonal of the plot represent good results, better than random. Right: Sensitivity and 1-specificity for different score thresholds. Highlighted (blue, bold): A score of 8 or above presents the best predictive values.

Next (Figure 3, Table 3), we identified the best possible score threshold to distinguish patients without OSA (exclusion of OSA, AHI < 5, n = 39) and patients with OSA (AHI ≥ 5, n = 111). The ROC-AUC was 0.89 (95% CI: 0.83, 0.94) with p < 0.001. The score threshold of 5 showed the best predictive values with a sensitivity of 72% (95% CI: 55%, 85%) and a specificity of 83% (95% CI: 75%, 89%). As a lower score indicates a lower OSA risk, a score of 5 or less was chosen to predict patients with no OSA. The test accuracy for this score was 80% (95% CI: 73%, 86%).

Figure 3 Test cohort – receiver-operating curve and score thresholds for predicting exclusion of obstructive sleep apnea (apnea–hypopnea index < 5). Abbreviations: ROC, receiver-operating characteristics; SEN, sensitivity; SPE, specificity. Notes: Left: ROC curve for score prediction of exclusion of obstructive sleep apnea (apnea–hypopnea index <5) within the test cohort. The curve is defined by sensitivity (true positive rate) on the y-axis and 1-specificity (false positive rate) on the x-axis. Points above the diagonal of the plot represent good results, better than random. Right: Sensitivity and 1-specificity for the different score thresholds. Highlighted (blue, bold): A score of 5 or below presents the best predictive values.

Having defined the upper threshold score to identify patients with severe OSA (AHI ≥ 30; score ≥ 8) and the lower threshold to rule out OSA (AHI < 5; score ≤ 5), we verified the middle category for the diagnosis of mild-to-moderate OSA. Based on the two-by-two outcome table of true and false positives and negatives (Supplementary Table 2), we calculated sensitivity, specificity, predictive values, and likelihood ratios (Table 3). In our test cohort, scores above 5 and below 8 predicted mild to severe OSA (5 ≤ AHI < 30) with a sensitivity of 49% (95% CI: 38%, 61%), a specificity of 78% (95% CI: 67%, 87%) and a test accuracy of 63% (95% CI: 55%, 71%).

Score Validation

We retested the scoring system with the validation cohort (n = 50, Table 1) by assessing the prediction power and the accuracy of the score thresholds. The ROC curve was significant for both, the prediction of severe OSA (ROC-AUC = 0.89 with 95% CI: 0.79, 0.90; p < 0.001) and exclusion of OSA (ROC-AUC=0.96 with 95% CI: 0.90, 1.00; p = 0.001). The thresholds were confirmed as best prediction cut-off values (Table 4, Supplementary Table 3, Supplementary Figures 1 and 2). The score threshold 8 presented a sensitivity of 83% (95% CI: 59%, 96%), specificity of 88% (95% CI: 71%, 97%), and a test accuracy of 86% (95% CI: 73%, 94%) for detecting severe OSA. The score threshold of 5 achieved a sensitivity of 100% (95% CI: 48%, 100%), a specificity of 78% (95% CI: 63%, 89%) and a test accuracy of 80% (95% CI: 66%, 90%) for the exclusion of OSA. For the middle category (5 < score < 8), the score reached a sensitivity of 52% (95% CI: 32%, 71%), a specificity of 91% (95% CI: 72%, 99%) and a test accuracy of 70% (95% CI: 55%, 82%).

Table 4 Validation Cohort (n = 50) – Receiver-Operating Characteristics for a Score ≥8 (Severe OSA), Score ≤5 (Exclusion of OSA) and Score Range 5–8 (Mild to Moderate OSA)

Discussion

This study was undertaken to develop and test an OSA prescreening tool which utilizes objective measurements and is easy to apply. The score is based exclusively on common anthropometric measures and intraoral visibility. It was developed as a morphological score and does not contain subjective questions, thereby reducing bias related to subjective evaluation measurements.

It differentiates the prediction between severe OSA and the exclusion of OSA with high test accuracy, sensitivity, specificity, and general applicability. It represents an effective and valuable new addition to the clinical tools addressing the most relevant diagnostic questions in suspected OSA patients; however, it does not predict an OSA syndrome (OSAS).

The majority of OSA patients remain undiagnosed with estimates of unrecorded cases of at least 80–90%.^35,36 The past decades have shown that OSA is a heterogeneous disease, with different phenotypes and degrees of severity leading to different outcomes.^37,38 For the sleep physician, an early diagnostic tool is especially relevant to effectively prepare for further diagnosis or treatment. Especially the exclusion of OSA is important in establishing a correct differential diagnosis when presented with common symptoms like severe snoring, trouble staying asleep, or daytime fatigue. OSA can be excluded when the patient presents an AHI smaller than 5 events/hour and without specific symptoms. The prescreening is also especially relevant because an OSA patient without symptoms are often not recognized. This is of great importance for patients with severe OSA, as it may be associated with elevated cardiovascular risk.^29,30 If a patient presents specific symptoms, then prescreening with our score may only indicate the severity of the disease. A continuing diagnostic is necessary, independent of the AHI revealed during prescreening. So, if a patient during prescreening displays an AHI smaller than 5 but with clinically relevant symptoms, a sleep specialist or a physician trained in sleep medicine will initiate further diagnostical steps. With an AHI of 8 or larger but without symptoms, the prescreening is important to identify OSA patients otherwise overlooked and to initiate appropriate and often extremely necessary further treatment based on relevant co-morbidity.

Currently, no simple prescreening system – not prone to subjective biases and applicable for wide clinical use without expensive and time-consuming assessment of overnight sleep – exists that could answer these questions with adequate accuracy.¹⁵ The score presented in this study is a simple to use prescreening tool.

Previous scoring systems for predicting OSA have been limited. Next to the time-consuming and expensive systems of overnight sleep assessment (eg, polysomnography) or imaging of airway measurements, the less expensive and easily applied screening instruments include questionnaires, containing items that are subjective measures, potentially associated with inaccuracy.^4,39 Moreover, often the results do not differentiate the predicted outcome in a clinically meaningful way. The popular Epworth Sleepiness Scale (ESS), assessing daytime sleepiness with eight simple questions, only reaches a medium sensitivity of 53% and a medium specificity of 59% for the general diagnosis of OSA.^40,41 The Berlin Questionnaire predicts general OSA without differentiation with a high sensitivity of 93%, but at a cost of a very low specificity of 16%.⁴² Another questionnaire, the STOP-Bang differentiates in predicting low, moderate, and severe OSA with a medium sensitivity of 77% and specificity of 45% for having a moderate to severe OSA.⁴³

While other prediction scores are based on more objective OSA risk factors that are less prone to bias, they are often validated within very specific populations (eg, obese patients, pregnant women, children, etc.) and do not differentiate OSA severity and lack accuracy.^22–28 The Morphometric Equation, introduced by Kushida, Efron and Guilleminault in 1997, uses only known anatomical OSA risk-factors (oral cavity, neck circumference, BMI) to calculate a probability of OSA. Their score reached a high sensitivity of 98% and specificity of 100% for predicting the presence of OSA, but it was not possible to differentiate degrees of OSA severity. The application within certain OSA patients failed.^26,44,45 The P-SAP (perioperative sleep apnea prediction) score, developed and validated within a specific surgical population, includes several objective demographical, medical, and airway measures, as well as snoring as a symptom. The score is lacking either in sensitivity or specificity (lower scores with a sensitivity of 94% and specificity of 32%, higher scores with a sensitivity of 24% and specificity of 91%).²⁷ The NoSAS (neck, obesity, snoring, age, sex) score, validated within a sleep clinic, includes objective measures and the subjective symptom of snoring. While sensitivity of 94% was quite high for general OSA, specificity was lacking (30%).²³ Deflandre et al attempted to differentiate and predict the common severity levels of OSA with the DES-OSA prediction score.²⁴ It is based on the Mallampati Score, distance between thyroid and chin, neck circumference, BMI, gender. While the score was able to differentiate between mild, moderate, and severe OSA with a good combination of relatively high sensitivity (75–83%) and specificity (72–77%), it did not focus on the exclusion of OSA.

We developed a morphological score without subjective questions in order to reduce a possible subjective bias. It was specifically developed for clinical application within sleep medicine. However, its easy applicability may make it a useful tool for general clinical practice as well. The development of our scoring system was guided by previous scores, such as the DES-OSA.²⁴ The Mallampati Score and the distance between thyroid and chin used by Deflandre et al were replaced with other variables that were highly correlated with the AHI and easier to assess. In particular, the distance between thyroid and chin is difficult to measure with high-risk OSA patients and has been criticized as not being an adequate tool for airway assessment.⁴⁶ Instead of the Mallampati Score with protruded tongue position, we used a modified version of the FTP with a neutral tongue position. The scale was modified to adapt for higher values of OSA patients, and therefore to achieve a more even distribution between the newly adapted four categories with our patient cohort: 7% fit into category I, 33% into category II, 33% into category III, and 26% into category IV. While we used the intraoral photographs obtained as part of SAGIC data collection, intraoral visibility can also be classified without photographs during routine clinical practice. We suggest that the modified FTP is added to the clinical routine when other OSA-related symptoms are seen and/or the patients score high on the more commonly used scores for pharyngeal visibility (eg, Mallampati Score).

Our score was developed on the one hand, to predict severe OSA due to its high risk of cardiovascular comorbidities. Here, with a score of 8 or higher, the accuracy of the test was above 80%. The score reached a high sensitivity and specificity in both the test cohort (82% sensitivity and 82% specificity) and validation cohort (83% sensitivity and 88% specificity). It also reached population-independent likelihood ratios in an acceptable range in both the test cohort (positive likelihood ratio 4.6, negative likelihood ratio 0.2) and validation cohort (positive likelihood ratio 6.9, negative likelihood ratio 0.2). A positive likelihood ratio above 4 and a negative likelihood ratio below 0.3 indicate good applicability of the scoring system with a general population.

On the other hand, the score was supposed to indicate the exclusion of OSA, which was important for an effective differential diagnosis and to avoid unnecessary and costly diagnostic procedures. Here, a “true positive” would mean recognizing a non-OSA patient correctly as not having OSA. A score of 5 or lower showed a test accuracy of 80% in both, the test and the validation cohort, indicating a good test efficacy. The score reached a high specificity of 83% (test cohort), identifying the “negatives” (OSA patients) correctly. The sensitivity of 72% in the test cohort, recognizing the “positives” (non-OSA patients), was not as high but still in an acceptable range. Within the validation cohort, the specificity decreased to 78% and the sensitivity increased to 100%; however, the validation cohort was relatively small, and results may not be as conclusive. Also, the positive likelihood ratios within the test cohort with over 4 indicated a strong general applicability. The score had a higher likelihood of recognizing non-OSA patients and correctly excluding them (true positives) than of diagnosing non-OSA patients with OSA (false negative, Supplementary Table 2).

Limitations

There are certain limitations to the development of our scoring system. This score was developed to focus on the detection of severe OSA and on the exclusion of OSA, an often overlooked feature in a prescreening instrument. The score was not intended to differentiate between the severity of OSA (medium or moderate OSA). It is a prescreening tool that will give first information on how to proceed and which further diagnostic steps to take. It is also noteworthy, that this is a score for screening of OSA, not OSAS. At this time, the prescreening did not focus on further clinical symptoms or cardiovascular symptomatic. This could be included in further studies. Also, we based the score on previous scores and accepted the predictive value of certain high-risk variables. However, we confirmed that those variables were highly correlated with the AHI. Another limitation was the selection bias based on using the SAGIC cohort with suspected OSA patients. OSA was only suspected and nonetheless, 25% of our test cohort and 10% of our validation cohort had an AHI below 5, presenting a complete variation of OSA severity categories within the cohorts. Also, the likelihood ratios, which are population-independent, had values in an acceptable range, indicating a good applicability of the scoring system with a general population and in general clinical practice. However, the validation cohort was quite small and certain results may not be as conclusive as necessary.

We recommend that the score be retested within a larger and more diverse sample, especially regarding the predictive capabilities of the exclusion of OSA due to some marginal values. At that point, it would also be interesting to compare AHI assessment with full overnight polysomnography versus portable polygraphy. Also, to confirm the score’s applicability within general clinical practice, the sample should include patients not recruited from sleep clinics, such as patients seen in a primary care setting. The score may need to be adapted if applied in a younger population and regarding sex (eg, waist and neck circumference) and international application, as studies have demonstrated ethnic and sex variations with regard to anatomical risk factors and OSA severity.⁴⁷

Conclusion

This score introduces a new early and easily applied prescreening tool for OSA. It combines objective and precise morphological measurements with predictive power, focusing on answering the most relevant clinical questions, especially regarding the identification of severe OSA patients independent of clinical symptoms. Also, without specific clinical symptoms, a relevant OSA can be excluded. We believe that it can be a valuable tool for widespread screening applicable in general clinical practice and may reduce unnecessary expensive testing in some scenarios.