Introduction

Retinopathy of prematurity (ROP) is the most common disease leading to childhood blindness among preterm patients in countries reporting high survival rates.1, 2, 3

ROP severe enough to require treatment usually occurs around the 37–38th postconceptional week (PCA: defined as gestational age (GA) at birth plus weeks of life). There is, therefore, a time interval during which the occurrence of ROP may be predicted and the disease may be promptly treated.

The prevalence of ROP ranges from 0 to 30%, depending on the quality of neonatal care delivered to infants and the set of risk factors that patients present individually. Therefore, screening programs to detect ROP in preterm infants at neonatal intensive care units (NICUs), as well as qualified ophthalmologists to provide this type of ophthalmologic care, are required to diagnose the few cases requiring treatment in order to avoid disease progression and blindness. Initial ROP screening and eye examination should be performed between the 4th and 6th weeks after birth, and repeated according to the findings from baseline examination every 1 or 2 weeks until retinal vascularization is completed, around the 42nd PCA. In case of ROP onset, examinations should be performed at shorter time intervals. Appropriate screening sessions are costly and demand a heavy workload. Moreover, repeated ophthalmological examinations may lead to stress and physical impairment in systemically compromised infants.

Several risk factors have been reported as involved in the development of ROP but birth weight (BW) and GA are considered the most important risk factors for disease onset.4, 5

Severe ROP develops after few weeks of age (usually around the 36th to 38th PCA) and, in this way, repeated eye examinations are necessary to detect a single case of ROP requiring treatment. In order to reduce the number of eye examinations performed in preterm infants, we developed a scoring system (ROPScore) that, if applied at 6 weeks of life, we hypothesize it may serve as a better predictor than BW and GA for the occurrence of any stage ROP or ROP severe enough to require treatment among very low BW (VLBW) preterm infants. This scoring system will allow for a reduction in the number of eye examinations performed in the same patient during ROP screening, as tests are repeated in those patients who are more likely to develop the disease. This study describes the development of ROPScore and compares the performance of the score against the BW and GA in order to predict the onset of ROP.

Patients and methods

Study design

This is a prospective institution-based cohort study including all VLBW preterm infants admitted to the institution and screened for ROP between October 2002 and July 2009.

Setting

The study was conducted at the NICU of the Hospital de Clínicas de Porto Alegre, a university-based tertiary hospital, located in an urban area with approximately 3 million inhabitants. Almost all patients are admitted through the Brazilian Public Health System. In 2002 a neonatal screening program for detection of ROP was implemented, and since then over 95% of patients admitted to the NICU are effectively examined once a week by ophthalmologists.

Population

The sample included all VLBW preterm infants (born with BW≤1500 g and/or GA≤32 weeks) who survived from the initial ophthalmological examination, performed between the 4th and 6th weeks after birth, to the 45th PCA. There were no exclusion criteria.

Clinical outcomes

Clinical outcomes were defined as the onset of any stage ROP and the development of ROP severe enough to require treatment. Staging of disease was recorded according to the 1984/1987 International Classification of ROP6, 7 and always corresponded to the highest stage of ROP observed during patient follow-up. Severe ROP was defined according to the Early Treatment for Retinopathy of Prematurity Randomized Trial.8

Selection of variables to generate ROPScore

The selection of variables to generate ROPScore was based on an analysis of risk factors for ROP in this cohort of patients. The 16 variables initially analyzed were: BW; GA (evaluated by obstetric history, early obstetric ultrasound, and confirmed by newborn infant clinical examination); weight gain proportional to BW measured at 6 weeks of life (defined as the patient's weight measured at 6 weeks of life minus BW, divided by BW); gender; multiple or single gestation; 5-min Apgar score; use of oxygen in mechanical ventilation or by nasal continuous positive airway pressure; use of erythropoietin, indomethacin or surfactant; occurrence of sepsis, meningitis, patent ductus arteriosus or intraventricular hemorrhage at any grade; needing for blood transfusions; and infant small for GA (<10th percentile). Sepsis and meningitis were diagnosed by clinical examination and microbiological culture. Intraventricular hemorrhage was diagnosed by cranial ultrasound. Clinical diagnosis of sepsis was based on the presence of three or more of the following signs: apnea, difficulty breathing, cyanosis, tachycardia or bradycardia, poor perfusion or shock, irritability, lethargy, hypotonia or seizures, abdominal distension, vomiting, diet intolerance, gastric residues, hepatomegaly, temperature instability or fever, petechiae or purpura, and appearance of poor health.

Development of ROPScore

All variables selected to generate the score were considered statistically significant for both ROP outcomes after univariate analysis and logistic regression (multivariate analysis). The score was established after linear regression, taking into account the impact of each variable in relation to ROP onset.

Statistical analysis

All statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS), version 15.0. for Windows (SPSS Inc., Chicago, IL, USA). The Student t-test and the χ2 test (univariate analysis) were used to compare patients who developed and who did not develop ROP (clinical outcomes).

Establishing ROPScore accuracy (sensitivity/specificity)

Receiver operating characteristic (ROC) curve was used to assess the accuracy of continuous values of the score to predict any stage or severe ROP, as well as the respective cutoff points for sensitivity and specificity and the positive predictive values (PPV) and negative predictive values (NPV). PPV, in the present study, refers to the probability of a preterm infant with a score above the best cutoff point developing ROP, whereas NPV corresponds to the probability of a preterm infant with a score below the best cutoff point not developing any stage and/or severe ROP.9

ROPScore ROC curve results were compared with ROC curve results for BW and GA individually, because BW and GA are usually considered the best predictors of ROP.

Presentation of ROPScore

All variables selected to generate ROPScore were entered into an Excel spreadsheet (Microsoft) for practical use by ophthalmologists during screening sessions (Figure 1).

Figure 1
figure 1

ROPScore on an Excel spreadsheet for use in clinical practice during screening examinations in preterm infants.

Full size image

Ethical aspects

The study protocol was approved by the Research Ethics Committee of the Hospital de Clínicas de Porto Alegre (protocol no. 09-253) and Federal University of São Paulo. The protocol is consistent with the principles of the Helsinki Declaration of 1995 (revised in Edinburgh in 2000).

Results

The study included data on 474 patients. Mean BW and GA for the entire cohort were 1217.3±272 g and 30.3±2.2 weeks. The demographic characteristics of the cohort are shown in Table 1. The prevalence of any stage ROP in our sample was 110/474 patients (23.2%). The incidence of ROP severe enough to require treatment in this study was 24/474 patients (5.0%).

Table 1 Demographic characteristics of all 474 patients included in the study
Full size table

After univariate analysis for any stage ROP and for severe ROP, the scoring system was developed based on BW, GA, proportional weight gain from birth to 6 weeks of life, use of oxygen in mechanical ventilation, and need for blood transfusions from birth to 6 weeks of life.

Linear regression analysis of BW, GA, proportional weight gain, use of mechanical ventilation, and need for blood transfusions revealed linear coefficients (β) of −0.004, −0.263, −1.258, +1.920, and +1.980, respectively. These numbers were applied as weighted values in the final calculation of the score, as follows: 24.847−0.004*C2−0.263*C3−1.258*C7+1.920*C4+1.980*C5.

The final continuous values of the score followed a parametric distribution, ranging from 9.1 to 21.6, with the best cutoff point for sensitivity and specificity established as 11 for ROP and 14.5 for severe ROP. An ROPScore cutoff point of 11 showed 94% sensitivity, 26% specificity, a PPV of 28% (95% CI 23.3–32.3), and an NPV of 93% (95% CI 86.8–96.9) for any stage ROP. For severe ROP, a cutoff point of 14.5 showed 96% sensitivity, 56% specificity, a PPV of 10.5% (95% CI 7.0–15.1), and an NPV of 99.6% (95% CI 98.1–99.9).

The area under the ROC curve for ROPScore values was 0.77 (P<0.001; 95% CI 0.72–0.82) to predict any stage ROP and 0.88 (P<0.001; 95% CI 0.82–0.94) to predict severe ROP. These values were significantly higher for ROPScore than for BW (0.71; P<0.001; 95% CI 0.66–0.77) and GA (0.69; P<0.001; 95% CI 0.63–0.75), when measured separately (Table 2).

Table 2 Area under the ROC curve for ROPScore compared with BW and GA for any stage ROP and for severe ROP
Full size table

Discussion

ROP is characterized as a two-phase disease. Phase 1 occurs after premature birth until 30–32 weeks’ PCA and refers to the interruption in the natural course of retinal vasculogenesis due to premature birth.10 Phase 2 occurs between 32 and 34 weeks’ PCA. This phase is characterized by hypoxia-induced retinal neovascularization similar to that observed in other proliferative retinopathies.11

Currently, if preterm infants at potential risk of developing severe ROP were early identified, they could receive an even more careful and individual perinatal care concerning their various risk factors. This approach would facilitate planning the best moment to introduce eye examination, because sicker children, as well as those with a worse postnatal prognosis, are at greater risk for the onset of ROP. This more appropriate perinatal management of preterm patients at risk of developing advanced stages of the disease could lead to an efficient prevention of severe ROP, reducing rates of childhood blindness.12

Several risk factors have been reported as involved in the development of ROP, but BW and GA are considered the most important related risk factors.4, 5 ROPScore was developed based on the main risk factors for the onset of ROP and it includes data on BW, GA, and weight gain proportional to BW measured at 6 weeks of life. Currently, low weight gain after premature birth is widely accepted as a predictive factor for later development of ROP, being considered superior to BW and GA alone as predictors.12, 13, 14, 15, 16, 17, 18 In addition, ROPScore also takes into account the use of oxygen in mechanical ventilation and the need for blood transfusions. All these risk factors are involved in the process of ROP development and can be easily identified by ophthalmologists during the initial ROP screening eye examination.19, 20, 21, 22, 23, 24, 25

The use of predictive scores is very important in neonatology. Several scoring systems, from the classic Apgar score to modern illness severity scores such as CRIB, SNAP, and SNAPPE-II, are routinely used as predictors of several comorbidities. Previous studies have attempted to demonstrate the usefulness of CRIB, SNAP, and SNAPPE to predict ROP. The authors reported that none system showed sufficient power to predict long-term clinical outcomes or ROP-induced vision impairment, and that cumulative SNAP could be considered as an independent risk factor for progression from moderate to severe ROP. However, after adjusting for other risk factors, cumulative SNAP was unable to accurately predict the desired outcome (threshold ROP).26, 27

SNAPPE-II has been used as a standard score in the NICU of our Institution since 2004. SNAPPE-II estimates illness severity using nine physiological and laboratory parameters collected within the first 12 h of life of preterm patients. The score can predict with great accuracy the risk of mortality during hospitalization and may also have some importance as an indicator for the development of other prematurity-related morbidities. The score had been previously tested by our research group in order to determine whether it would be a good predictor for later development of ROP. A positive association between high SNAPPE-II scores and ROP onset was observed only after univariate analysis. However, after adjustments by logistic regression and by the results obtained using ROC curves, no improvement was observed in the performance of SNAPPE-II as a predictor of later onset of ROP compared with BW and GA.28 A possible explanation for the poor performance of illness severity scores as predictors of ROP may lie in the fact that such scores are obtained on the infant's first day at the NICU and do not necessarily reflect the clinical behavior of patients in the weeks that follow premature birth. Patients with a high SNAPPE-II score, and consequent increased morbidity on hospital admission, may frequently have a more stable clinical course developing less comorbidities. On the other hand, patients who are healthier at birth, and consequently have a lower score on admission, may present a much more unstable clinical course in the weeks following birth, which may predispose infants to the onset of ROP.28

In order to be effective in terms of ROP detection, a scoring system should incorporate variables that take into account not only the first days after birth, but also subsequent weeks of life, when several physiological or pathological processes occur. In this sense, ROPScore includes weight gain proportional to BW measured at 6 weeks of life, in addition to the use of oxygen in mechanical ventilation and the need for blood transfusions up to 6 weeks of life. Thus, in theory, at least three of the five variables composing the score would be analyzed in the first 6 weeks of life of patients, unlike BW and GA that are defined at birth. Recently, clinical prediction models to early identify patients in risk of ROP were presented. WINROP29, 30, 31 and PINT-ROP32 were presented using postnatal longitudinal weight gain measurements. ROPScore, by the other hand, uses the BW and the weight measured at completed 6 weeks of life as weighted values in the final calculation of the score.

The accuracy of ROPScore to predict the onset of any stage ROP or severe ROP was determined by ROC curves and the respective cutoff points for sensitivity and specificity of the continuous values of the score. A cutoff point of 11 was established for any stage ROP and 14.5 for severe ROP. At these cutoff points, ROPScore showed 94% sensitivity and 26% specificity for any stage ROP, and 96% sensitivity and 56% specificity for severe ROP. Therefore, the higher the score achieved, the greater the statistical probability of an infant developing ROP. For the cutoff point of 11, PPV and NPV for any stage ROP were established. PPV, in the present study, refers to the probability of a preterm infant with a score above 11 developing ROP, whereas NPV corresponds to the probability of a preterm infant with a score below 11 not developing any stage ROP or severe ROP. Predictive values at the established cutoff points 11 and 14.5 safely reflect the global clinical usefulness of the score for the onset of any stage and severe ROP based on the prevalence of the disease in the study setting.9

ROPScore is a robust predictor of ROP onset. This scoring system includes risk factors for ROP that are easy to record. The score is simple enough to be routinely used by ophthalmologists or by the NICU staff during screening examination for detection of ROP. ROPScore is presented in an Excel spreadsheet (Microsoft) and it can be used in advance of the first ophthalmological examination by a member of the NICU staff who inserts in the respective column the BW in grams (first row) and the GA in weeks (second row). The numbers 1 or 0 (zero) should be inserted in the third and fourth rows, respectively, if the preterm received any blood transfusion from birth to the 6th week of life (insert number 1 if the baby received blood transfusion or insert number 0 if not) and if the preterm used or not used oxygen-therapy under mechanical ventilation from birth to the 6th week of life. The last insertion (in the fifth row) is the baby weight (in grams) at completed 6 weeks of life. The Excel spreadsheet automatically calculates the proportional weight gain and the final score for that considered patient (Figure 1). The staff member presents for the ophthalmologist the scores for each infant to be examined in this screening session. According to the score, the ophthalmologist in charge will decide which babies should receive more or less frequent re-evaluations in the weeks following the initial examination. Higher scores (above 14.5) lead to more evaluations because these babies have more risk to develop severe ROP than patients with lower scores. It is worth to mention that the use of the score did not influence the usual established criteria for inclusion of preterm infants in the screening. The ROPScore can be helpful in order to reduce the excessive number of eye examinations performed in preterm infants that achieved low scores.

Summarizing, ROPScore is more accurate than BW and GA to predict the occurrence of ROP in VLBW preterm infants. ROPScore is a promising tool that aims to reduce the excessive number of eye examinations performed in VLBW preterm infants. Further studies are warranted to validate the actual usefulness of the score. The internal validation process of ROPScore is currently underway, with the score being applied prospectively to our own patients who are not included in this initial cohort. The external validation of ROPScore is being conducted by prospectively applying the score to a cohort of patients from another population in different countries.