Modified NEOMOD score as a neonatal mortality prediction tool in a medium‐income country: A validation diagnostic test study

Abstract Background and Aims Multiple organ dysfunction (MOD) is a potentially reversible physiological disorder that involves two or more systems. Modified NEOMOD (Neonatal Multiple Organ Dysfunction score) scale could be a useful instrument to measure MOD and predict mortality. Our aim was to validate modified NEOMOD in patients from a neonatal intensive care unit (NICU) of a middle‐income country. Methods Diagnostic test study. Preterm newborns admitted NICU were included. Daily values were collected from birthday to Day 14. MOD was defined as at least one point in two or more systems. The lowest score is 0 and the maximum is 16. The outcome variable was mortality. Secondary outcomes were bronchopulmonary dysplasia, retinopathy of prematurity (ROP), late‐onset neonatal sepsis (LONS), intraventricular hemorrhage (IVH) and length of hospital stay. Area under the curve (AUC) and Hosmer–Lemeshow test were calculated to evaluate scale discrimination and calibration. Logistic regression was used to estimate the association between daily modified NEOMOD score and death. Results We included 273 patients who met the inclusion criteria. MOD incidence was 74.4%. The median gestational age in patients with MOD was 30 (interquartile range [IQR]: 27–33) and in patients without MOD it was 32 (IQR: 31–33) (p < 0.001). There were 40 deaths (14.6%), 38 (18.7%) from the MOD group and 2 (2.9%) from non‐MOD group. On accumulated Day 7, AUC was 0.89 (95% confidence interval [CI]: 0.83–0.95). Modified NEOMOD had good calibration (X 2 = 2.94, p = 0.982). DBP (12.8% vs. 2.9%, p = 0.001), ROP (3.9% vs. 0%, p = 0.090), IVH (33% vs. 12.9%, p < 0.001), and LONS (36.5% vs. 8.6%, p < 0.001) frequency was higher in the MOD group than non‐MOD group. Length of hospital stay also was higher in MOD group (median 21 days [IQR 7–44] vs. median 5 days [IQR 4–9], p = 0.004). Conclusion Modified NEOMOD scale presents good discrimination and calibration for death in preterm children. This scale could help in clinical decision‐making in real‐time.


| BACKGROUND
Prematurity is defined as any birth occurring before 37 weeks of gestation. Worldwide, 15 million preterm babies are born each year, representing between 4% and 18% of all newborns. 1 Prematurity is the main cause of death in children under 5 years of age, causing approximately one million deaths in 2015, of which three-quarters have preventable causes. Therefore, prematurity is considered a public health problem. 2,3 Complications of prematurity are responsible for substantially higher morbidity and mortality rates in preterm infants than in term infants. The risk of complications is inversely proportional to the gestational age. 4,5 Multiple-organ dysfunction (MOD) is defined as a potentially reversible physiological disorder that involves two or more systems that are not initially involved in the disorder responsible for admission to the neonatal intensive care unit (NICU). 6 MOD increases the risk of morbidity and mortality in newborns, and its incidence is inversely proportional to gestational age. 7 Although MOD is defined worldwide, problems persist when studying each patient individually, making it difficult for the medical team to make clinical decisions. Clinical decision-making about newborns is complex due to its strong psychological and social components Clinical decision making for newborns is complex due to strong psychological and social components. 8 Although patients' caregivers believe that decisions made about patients are in line with management guidelines, they believe that the will of the physician may also influence them. 9 Factors such as (a) the difficulty of establishing a boundary between the management of patient complications and falling into medical futility, (b) the uncertainty about whether aggressive treatment will lead to a reasonably healthy child or a child with sequelae, and (c) the inability to determine what is an acceptable ratio between the probabilities of those two outcomes makes it difficult for the clinical team to make decisions and know what to offer and what not to offer to the patient's parents. 10 The use of mortality prediction scores allows the team to have an objective tool based on the physiological state of the patient that will guide the direction of treatment.
Several scoring systems have been designed to perform this function in newborns. One of them, the Neonatal Multiple Organ Dysfunction (NEOMOD) score, was created by Janota et al. 11 and modified by Çentikaya et al. 12 This scoring system allows the analysis of eight organ systems, predicts mortality according to the level of involvement of each, is easy to use, and the required variables may be available in newborn units of high-complexity hospitals in developing countries. 12,13 Unlike other existing mortality prediction scores, 14,15 where only variables related to birth and the first hours of life are considered, the NEOMOD score and its subsequent modification allow the probability of death to be estimated at any time during the first 28 days of life. Thus, clinically, it may come closer to the definition of MOD and provide accurate information to the treating clinician for timely and targeted decision-making.
The modified NEOMOD scale was created with two objectives: the first is a real-time objective diagnostic tool diagnosis of multiple organ dysfunction, and the second is a mortality prediction scale where the increase in the score of the scale implies a greater probability of death. 12 Although the usefulness of NEOMOD has been extensively studied with good results, this scoring system should be evaluated in different population samples to confirm its universal applicability. The objective of this study is to evaluate the predictive capacity of the modified NEOMOD score for mortality in newborns with gestational age below 37 weeks in the NICU of a third-level hospital in a developing country.

| METHODS
Colombia is a middle-income country with approximately 50 million inhabitants. 16

| Study population and data collection
This is a diagnostic test validation study. All patients admitted to NICU from January 2018 to December 2019 were included. Children who were born at term (more than 36 weeks and 6 days of

| Modified NEOMOD score
The NEOMOD scoring system was developed by Janota et al. 11 and modified by Çetinkaya et al. 12 The dysfunction of eight organ systems (central nervous system, cardiovascular system, kidneys, respiratory system, gastrointestinal system, hemocoagulation balance, acid-base balance, and microvascular system) is classified as moderate (1 point) or severe (2 points) every 24 h from Days 1 to 28 of life (Table 1). The maximum possible score of the modified NEOMOD is 16 points. The sensitivity of this scale is 84.1%, and the specificity is 78%, with an area under the receiver operating characteristic curve (AUC) of 0.943 ± 0.021. 12 The daily values required by the scoring system were collected from the day of birth until 14th day of life. The primary outcome variable was mortality. Secondary outcomes were bronchopulmonary dysplasia (BPD; defined as the need for supplemental oxygen for more than 28 days), 18 retinopathy of prematurity (ROP) (stage 3 or higher), 19 necrotizing enterocolitis (according to the Bell scale), 20 intraventricular hemorrhage (IVH) (9), late onset sepsis and length of hospital stay.

| Sample size
The sample size calculation was performed using the formula proposed by Riley et al. According to the validation study performed by Centinkaya et al., 12 the sensitivity of the scale is 84.1% and the specificity 78.0%. Based on this information, with a confidence level of 95% and a precision of 5%, with an eventual proportion of 74% and a minimum expected value of 68%, the required sample size is 264 patients.

| Statistical analysis
The dichotomous variables are expressed as percentages. Continuous variables are expressed as median and interquartile range (IQR) or mean and standard deviation, depending on their distribution. The normality of the variables was estimated with the Shapiro-Wilk test.
The population was divided between newborns diagnosed and not diagnosed with MOD to compare the variables of interest. Fisher's exact test or the χ 2 test were used for categorical variables and the ttest or Mann-Whitney test for continuous variables. The level of statistical significance adopted was p < 0.05.
The incidence of MOD was calculated as the number of cases of MOD detected divided by the total number of patients admitted to the NICU within the study period. Mortality was calculated in the groups with MOD and without MOD, taking as the numerator the number of deaths in each group and as the denominator the total population of each group.
The discrimination ability of the NEOMOD score was estimated by the area under the curve, with a confidence interval of 95%. The ROC curve was constructed taking the worst values recorded for each system per day over the entire patient follow-up. Calibration was evaluated with hosmer-lemeshow test. The analyses were performed with the statistical package Stata ® 14.0 (Stata Corp).
The study was approved by the nstitutional review board, given its retrospective nature, informed consent was exempted (# 1368).   Table 3. The median cumulative score at  even more so in a newborn whose physiology is highly variable, 28 for which reason it cannot be limited to independent daily evaluations. 7 The sensitivity of the modified NEOMOD score was 84.1%, its specificity was 78%, and its AUC was 0.943 ± 0.021. 12

| Limitations and strengths
This study has several limitations. The first is the potential information bias due to its retrospective nature; after 14 days, fewer than half the data on the variables used in the score were available.
Although the score allows replacing missing data with normal values, this can lead to an underestimation of the actual number of MODs.
This study also has strengths. This is the first study to validate the modified NEOMOD score in the Latin American population of a middle-income country, which allows it to be used in this population group in the NICUs of Latin American countries.

| Next steps
In addition to the results and the certainty of being able to use the modified NEOMOD score in our population, the present writingreview and editing.

ACKNOWLEDGMENTS
We thank the Clinical Research Center of Fundación Valle del Lili for all the support provided during the study.

CONFLICT OF INTEREST STATEMENT
The authors declare no conflict of interest.

DATA AVAILABILITY STATEMENT
The datasets generated and/or analyzed during the current study are not publicly available to protect the rights of research participants according to Helsinki declaration but are available from the corresponding author on reasonable request.

ETHICS STATEMENT
The study was approved by the institutional review board, and given its retrospective nature, informed consent was exempted (# 1368).

TRANSPARENCY STATEMENT
The lead author Laura Torres-Canchala affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.