https://orcid.org/0000-0002-3287-5181
Figures
Abstract
In many low-income countries, over five percent of hospitalized children die following hospital discharge. The lack of available tools to identify those at risk of post-discharge mortality has limited the ability to make progress towards improving outcomes. We aimed to develop algorithms designed to predict post-discharge mortality among children admitted with suspected sepsis. Four prospective cohort studies of children in two age groups (0–6 and 6–60 months) were conducted between 2012–2021 in six Ugandan hospitals. Prediction models were derived for six-months post-discharge mortality, based on candidate predictors collected at admission, each with a maximum of eight variables, and internally validated using 10-fold cross-validation. 8,810 children were enrolled: 470 (5.3%) died in hospital; 257 (7.7%) and 233 (4.8%) post-discharge deaths occurred in the 0-6-month and 6-60-month age groups, respectively. The primary models had an area under the receiver operating characteristic curve (AUROC) of 0.77 (95%CI 0.74–0.80) for 0-6-month-olds and 0.75 (95%CI 0.72–0.79) for 6-60-month-olds; mean AUROCs among the 10 cross-validation folds were 0.75 and 0.73, respectively. Calibration across risk strata was good: Brier scores were 0.07 and 0.04, respectively. The most important variables included anthropometry and oxygen saturation. Additional variables included: illness duration, jaundice-age interaction, and a bulging fontanelle among 0-6-month-olds; and prior admissions, coma score, temperature, age-respiratory rate interaction, and HIV status among 6-60-month-olds. Simple prediction models at admission with suspected sepsis can identify children at risk of post-discharge mortality. Further external validation is recommended for different contexts. Models can be digitally integrated into existing processes to improve peri-discharge care as children transition from the hospital to the community.
Citation: Wiens MO, Nguyen V, Bone JN, Kumbakumba E, Businge S, Tagoola A, et al. (2024) Prediction models for post-discharge mortality among under-five children with suspected sepsis in Uganda: A multicohort analysis. PLOS Glob Public Health 4(4): e0003050. https://doi.org/10.1371/journal.pgph.0003050
Editor: Collins Otieno Asweto, University of Embu, KENYA
Received: February 1, 2024; Accepted: April 4, 2024; Published: April 29, 2024
Copyright: © 2024 Wiens et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Study materials including the study protocol, consent form, data collection tools, de-identified participant data, data dictionary, and the analysis code are available on request to the corresponding author (Matthew O. Wiens, matthew.wiens@bcchr.ca) or to the Institute for Global Health at BC Children’s and Women’s Hospital (Jessica Trawin, jessica.trawin@cw.bc.ca) or through the published protocol (https://doi.org/10.5683/SP3/QRUMNQ) and dataset (https://doi.org/10.5683/SP3/REPMSY). Owing to the sensitive nature of clinical data, access to the de-identified data is granted on a case-by-case basis and will require the signing of a data sharing agreement.
Funding: The study was supported by funds from Grand Challenges Canada (MW; grant #TTS-1809-1939, https://www.grandchallenges.ca/), Thrasher Research Fund (MW; grant #13878, https://www.thrasherresearch.org/), BC Children’s Hospital Foundation (https://www.bcchf.ca/), and Mining4Life (https://mining4life.org/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Morbidity and mortality secondary to sepsis disproportionately affect children in low- and middle-income countries, where >85% of global cases and deaths occur [1]. Lower income regions are plagued by poorly resilient health systems, widespread socio-economic deprivation, and unique vulnerabilities, including malnutrition. Reducing the overall sepsis burden requires a multi-pronged strategy that addresses three periods along the care continuum–pre-facility, facility and post-facility [2]. Of these, post-facility issues have been largely neglected in research, policy, and practice [3].
Robust epidemiological data for pediatric post-discharge mortality in the context of sepsis and severe infection have been limited [4]. Growing evidence points to a significant burden of post-discharge mortality, which accounts for as many deaths as the acute hospital phase of illness [5,6]. While comorbid conditions such as malnutrition and anemia have been linked to risk, other factors such as illness severity (at admission and discharge), prior hospitalizations, and underlying social vulnerability, are also independently associated with poor post-discharge outcomes [7]. However, we lack simple data-driven methods to identify those at highest risk of mortality.
Current epidemiological evidence has demonstrated critical gaps in care following discharge [8]. Most post-discharge deaths occur at home, rather than during a subsequent readmission, indicating poor health utilization among the most vulnerable. Effective healthcare utilization is often hampered by poverty, community and family social dynamics, and poorly linked and unresponsive health facilities [9–11]. Providing quality care during and after discharge is a significant challenge in many facilities, in part due to severely strained human and material resources.
Effective solutions to improving the transition of care from hospital to home within poorly resourced health systems must be child-centred and focused on identifying the most vulnerable children [12]. In this study, we aim to update the development and validation of clinical prediction models that identify children, admitted with suspected sepsis, who are at risk of post-discharge mortality [13].
Materials and methods
Study design and approvals
Four independently funded, prospective observational cohort studies were conducted with a primary objective of generating model-building data: two among children under six months and two among children 6–60 months of age. These studies were approved by the Mbarara University of Science and Technology Research Ethics Committee (No. 05/11-11, 10-Nov-2011; and No. 15/10-16, 27-Jan-2017) and the University of British Columbia–Children’s and Women’s Health Centre of BC Research Ethics Board (H10-01927, 01-Dec-2011; and H16-02679, 09-May-2017). Written informed consent was obtained from the parent or legal guardian of all study participants. This manuscript adheres to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement [14].
Study setting and population
Subjects were enrolled from six hospitals in Uganda (S1 Text). These facilities serve catchments of 30 districts with a population of approximately 8.2 million individuals, including approximately 1.4 million children under five years [15], in a mix of urban and rural areas, reflecting a representative sampling of the Ugandan pediatric population.
All study cohorts had identical eligibility criteria. Any child admitted with suspected sepsis was eligible. Suspected sepsis was defined as children admitted with a proven or suspected infection (as determined by the treating medical team). We previously demonstrated that 90% of children enrolled using these criteria meet the international pediatric sepsis consensus conference (IPSCC) definition [16]. The IPSCC defines sepsis as the presence of the systemic inflammatory response syndrome alongside a suspected or proven infection.
The first cohort (enrolment 13-Mar-2012 to 13-Jan-2014) was used previously to report a predictive model for post-discharge mortality in 6-60-month-olds [13]. The second and third cohorts were the primary enrolment for the present analysis, and were defined by age range: 0-6-month-olds (enrolment 11-Jan-2018 to 30-Mar-2020) and 6-60-month-olds (enrolment 13-Jul-2017 to 02-Jul-2019); these data have been previously reported [6]. The fourth cohort enrolled only 0-6-month-olds (enrolment 31-Mar-2020 to 05-Aug-2021) in order to understand how the early COVID-19 period impacted post-discharge outcomes. Protocols and procedures were largely overlapping, and the same research staff were involved in data collection during all four enrolment periods [17].
Data collection
Data collection tools are available through the Smart Discharges Dataverse [17]. Data collection procedures were previously described (also S1 Text) [6,13]. Briefly, trained study nurses collected clinical, social, and demographic data from consented participants at hospital admission; largely overlapping between the two age groups, some variables were specific to 0-6-month-olds. These variables were our candidate predictors and were selected based on clinical and contextual knowledge of possible factors relating to post-discharge mortality, using a modified Delphi process to identify promising variables in each age group [18,19].
Study nurses recorded discharge diagnosis and status (died, discharged, discharged against medical advice, referred). A field officer contacted enrolled children by phone two and four months after discharge, with an in-person visit at six months to determine mortality status and, if applicable, date of death. All data were collected using encrypted study tablets and uploaded to a Research Electronic Data Capture (REDCap) database hosted at the BC Children’s Hospital Research Institute (Vancouver, Canada) [20,21].
Model development
Outcome definition and ascertainment.
The primary outcome of the prediction model was post-discharge mortality within six months of discharge, analyzed as a binary outcome. While data were available to build a time-to-event prediction model, time of death was considered irrelevant for modelling mortality. Complete six-month follow-up data for vital status was available for 98% of our cohort.
Prediction model performance was evaluated primarily using area under the receiver operating characteristic curve (AUROC). We also reported the precision-recall curve and area under the precision-recall curve (PR-AUC), which are more appropriate for imbalanced datasets [22].
Variable selection.
Recognising the challenges of implementing large prediction models in resource-constrained settings, we determined a priori to develop three models for each age group and restricted each model to eight variables drawing from a different pool of available predictors: one model focused solely on commonly-available clinical variables; one model focused on commonly-available clinical and social variables; and one model used any candidate predictor variable (Fig 1). This approach aimed to reduce the impact of missingness in an implementation scenario. A feature of our modelling approach (elastic net regression) was that final model size could not be pre-specified, often resulting in large models. Therefore, we conducted two rounds of variable selection.
To prioritize parsimony, the first variable selection round reduced the list of possible predictors to two subsets: one including only the most relevant clinical variables; and a second including only the most relevant clinical and social variables. Variables included in these subsets were determined a priori, based on clinical significance and ease of measurement in low-resource settings. These subsets were used to derive intermediary models that were either clinically-focused or clinically- and socially-focused; the full candidate predictor list for each age group was also used to derive intermediary models that used any available variable (Tables B and C in S1 Text).
The second variable selection round involved ranking the importance of variables from each intermediary model, which was calculated as the weighted sums of the absolute regression coefficients [23]. The top eight unique variables (e.g., temperature and its quadratic term were considered a single unique variable) were selected based on average ranking from 10-fold cross-validation of the intermediary models. If an interaction term was ranked in the top eight variables, both interaction terms were included. This second variable selection round produced a family of final models to predict mortality within six months post-discharge (M6PD) that used only the eight top-ranked variables in each age group: models using only clinical variables, denoted by M6PD-C0-6 for 0-6-month-olds and M6PD-C6-60 for 6-60-month-olds; models using clinical and social variables, denoted by M6PD-CS0-6 and M6PD-CS6-60; and models using any of available predictor variable, denoted by M6PD-A0-6 and M6PD-A6-60 (Fig 1).
Statistical analysis
The primary study sample size was determined to accomplish three aims: to explore the epidemiology of post-discharge mortality, as previously reported [6]; to develop prediction models; and as a control period for a later interventional phase. The estimated sample size was determined as 2,117 and 1,551 for the 0-6-month and 6-60-month cohorts, respectively (S2 Text). All analyses were conducted using R version 4.2.2 (R Foundation for Statistical Computing, Vienna, Austria) [24], reported in detail in S2 Text.
Results
Study population
During the four enrolment periods, a total of 22,166 consecutively admitted children were screened and 8,810 enrolled (Fig 2). Among 0-6-month-olds (n = 3,665), 3,424 (93.4%) survived to discharge. Complete 6-month outcomes were available for 3,349 (97.8%) of these children, forming the full dataset for model development in this age group. Among 6-60-month-olds (n = 5,145), 4,916 (95.5%) survived to discharge. Complete 6-month outcomes were available for 4,830 (98.2%) of these children, forming the full dataset for model development in this age group.
Mortality within 6 months of discharge occurred in 257 (7.7%) 0-6-month-olds, with median (interquartile range [IQR]) time to death of 31 (9–80) days, and in 233 (4.8%) 6-60-month-olds, with time to death of 36 (11–105) days (Fig A in S1 Text). Missing data were minimal (Table 1).
These cohorts’ clinical and demographic details have been previously described (Table 1) [6,13]. The mean ±standard deviation [SD] age was 2.1 ±1.8 months with 1,884 (56.3%) male in the 0-6-month group, and 21.7 ±13.7 months with 2,670 (55.3%) male in the 6-60-month group. Poor growth/malnutrition was common, with 463 (13.8%) 0-6-month-olds and 668 (13.8%) 6-60-month-olds classified as severely underweight (weight-for-age z-score <-3) and similar weight-for-age z-score distributions in both age groups. Discharge diagnoses recorded by the clinical team could be overlapping in the case of multiple diagnoses (Table D in S1 Text). Most predictor variables considered were associated with post-discharge mortality (Table 1).
Prediction models
The intermediary variable models were large (coefficients, performance metrics, and variable importance reported in S3–S5 Texts). The models derived using all candidate predictors (intermediary any variable model) included 41 unique variables in the 0-6-month model and 19 unique variables in the 6-60-month model. Applied to the entire dataset for each age group, the AUROC was 0.81 (95%CI 0.79 to 0.84) for the 0-6-month model and 0.79 (95%CI 0.77 to 0.82) for the 6-60-month model, with average AUROCs of 0.77 (range 0.69–0.87) and 0.76 (range 0.71–0.81) across the 10 cross-validations, respectively; the PR-AUC was 0.27 for the 0-6-month model and 0.18 for the 6-60-month model, with average PR-AUCs of 0.22 (range 0.13–0.31) and 0.16 (range 0.11–0.21) across the 10 cross-validations, respectively. Calibration was good at low predicted probabilities, with a Brier scores of 0.07 (range 0.06–0.07) for the 0-6-month model and 0.04 (range 0.04–0.05) for the 6-60-month model. Calibration decreased at higher predicted probabilities, although there were almost no individuals with probabilities >40%. In both age groups, mid-upper arm circumference (MUAC) was identified as the variable with the highest importance.
The final models are summarized in Table 2, and detailed in S6–S8 Texts, including all model terms, their coefficients, and plots outlining the relative importance of coefficients in each model.
Final 0-6-month models
The M6PD-C0-6 model, using only simple clinical variables, included weight-for-age z-score (mean rank [rm] = 1.4, selection frequency [sf] = 10), MUAC (rm = 1.6, sf = 10), feeding status (rm = 3.4, sf = 10), SpO2 (rm = 5.8, sf = 9), duration of illness (rm = 6.2, sf = 9), age × jaundice (rm = 7.8, sf = 7), and bulging fontanelle (rm = 8.3, sf = 8) (Table A in S3 Text). The AUROC was 0.77 (95%CI 0.74 to 0.80) and PR-AUC was 0.23 when applied to the entire 0-6-month dataset (Fig 3), while the average AUROC and PR-AUC across the internal 10 cross-validations were 0.75 (range 0.63–0.85) and 0.23 (range 0.11–0.33), respectively (Table 2A and S6 Text). Setting the sensitivity to 80%, the corresponding probability threshold was 0.058; at this threshold, positive and negative predictive values were 14% and 97%, respectively. Calibration at low predicted probabilities was good, with a Brier score of 0.07 (Fig 3 and Fig C in S6 Text). Calibration at probabilities beyond 30–40% was poor, but sample sizes were very small in this range.
The points on the receiver operating characteristic (ROC), precision recall (PR), and gain curve plots indicate co-ordinates for the probability threshold at sensitivity = 80%, with positive predictive value (PPV) and negative predictive value (NPV) also reported at this threshold.
The M6PD-CS0-6 model, using social and clinical variables, was nearly identical in performance to M6PD-C0-6; the variables were largely overlapping with only fontanelle status replaced by travel time required to reach hospital (Table 2A and S7 Text). M6PD-A0-6 that used any available variable, was identical to M6PD-CS0-6 (Table 2A and S8 Text).
Final 6-60-month models
The M6PD-C6-60 model, using only clinical predictors, included nine variables (the 8th best-performing variable included an interaction with a new variable; Table B in S3 Text): MUAC (rm = 1, sf = 10), SpO2 (rm = 2.7, sf = 10), weight-for-age z-score (rm = 2.8, sf = 10), time since prior admission (rm = 4.7, sf = 10), abnormal coma score (rm = 5.8, sf = 9), temperature (rm = 6.4, sf = 9), HIV status (rm = 6.5, sf = 9) and age × respiratory rate (rm = 9.1, sf = 2). The AUROC was 0.74 (95%CI 0.72 to 0.79) and PR-AUC was 0.17 when applied to the entire 6-60-month dataset (Fig 4), with an average AUROC of 0.73 (range 0.67–0.77) and average PR-AUC of 0.16 (range 0.10–0.19) across the 10 cross-validations (Table 2B and S6 Text). Setting sensitivity to 80%, the corresponding probability threshold was 0.036; at this threshold, positive and negative predictive values were 0.08 and 0.98, respectively. Calibration across risk strata was good with a Brier score of 0.04 (Fig 4 and Fig D in S6 Text).
The points on the receiver operating characteristic (ROC), precision recall (PR), and gain curve plots indicate co-ordinates for the probability threshold at sensitivity = 80%, with positive predictive value (PPV) and negative predictive value (NPV) also reported at this threshold.
The M6PD-CS6-60 model, which used clinical and social variables, was almost identical to M6PD-C6-60, with only home water source and water disinfection practices replacing coma score (Table 2B and S7 Text). M6PD-A6-60 was similar to M6PD-CS6-60, with water disinfection practices replaced by hemoglobin; performance metrics were nearly identical (Table 2B and S8 Text).
Discussion
Using four large, objective-driven, prospective cohorts of under-5 children admitted with suspected sepsis, we derived and internally-validated prediction models for post-discharge mortality using only admission data. Their performance to predict mortality up to six months post-discharge was good, suggesting potential utility to improve post-discharge outcomes by linking individual risk to interventional intensity [25]. Data-driven, child-centred approaches to post-discharge care have been strongly advocated for [5,26,27]. Our robust, cross-validated models utilized data from multiple sites, captured over eight years, and should spur focus on external validation outside Uganda.
Several recent studies have shown that post-discharge mortality can be closely linked to a variety of key risk factors, such as malnutrition and disease severity [4–6]. Our results affirm this through formal model development using varied sets of few, objective, and easy-to-collect variables typically available in most settings where such models would be used. In a model deployment context, however, the general approach of developing a single model may not always be sufficient since missingness at the point-of-care may be common. Having multiple simplified models with similar performance, as we saw in our models, may help alleviate these kinds of logistical barriers to implementation [28,29].
Without an effective intervention, risk prediction has limited utility. Understanding discharge as a dynamic process encompassing the time between admission and re-integration into community care is integral to our focus on admission factors [30]. Early identification allows post-discharge risk to be incorporated into discharge planning from the outset. Significant challenges in preparing caregivers for discharge and the transition home have been identified, suggesting that early planning is an essential component of effective peri-discharge care [30].
Choosing risk probability thresholds to classify post-discharge mortality as a binary outcome depends on many factors, including availability of human resources, baseline risk, risk tolerance, and impact on patients/caregivers. Though the thresholds chosen may prove useful in some settings, choice of both the threshold and number of thresholds must be informed by local context and constitutes a critically important consideration for deployment of this, or any, risk model [31].
Although internal validation can justify using models within the region in which they are derived, external validation using different data sources (ideally several) from different regions is essential [32], using both existing and future data [5]. Consequently, we have several prospective studies underway, and will establish data sharing agreements with other collaborators to enable use of their collected data. However, not every conceivable implementation region for any given model can be subjected to external validation. A more pragmatic approach is developing a region-specific model-updating process, integrated over the life-course of the model. Calibration drift due to secular trends, the measured impact of the model itself, and peculiarities of each individual site are key considerations in model deployment [33]. Digitization of the healthcare system will help establishing these processes [34].
As health systems in low-income countries increasingly adopt electronic health records, incorporating algorithms to augment care decisions has tremendous potential to improve outcomes and facilitate adoption of these digital systems [35,36]. Using routinely-collected variables can allow models to run without additional user input and automatically prompt follow-up guidance to the medical team and patient, encouraging adoption and linkage to interventional programs. Furthermore, such systems can report baseline risk data and, when linked to follow-up programs, data on readmission and mortality to national-level health management information systems, such as DHIS2 [37]. These data can be used in model calibration and updating, ensuring site-specific validity. Contextually-validated digital clinical decision support systems utilizing risk algorithms are increasingly recognized as essential to achieving universal health coverage, especially in low- and middle-income countries [38,39].
Limitations
This study has several limitations. While our models performed well with internal cross-validation, demonstrating good performance in planned external validation is essential to encourage adoption. Second, our models do not accommodate missing data for predictor variables. While missing data rates were very low, this is unlikely to represent true rates of missingness in real-world practice. We developed a family of models, varying in number and type of predictors, which produced similar performance, to partially address this limitation. Future research will explore more robust methods for addressing missing data, including building sub-models to allow for every possible combination of missing variable [29]. Third, these models were developed in the absence of a proven program to utilize a risk-based approach to care, limiting their current utility. While merely knowledge of individual risk can change behaviour and may influence provision of peri-discharge care, risk-informed approaches to follow-up care are also currently under investigation [40]. Fourth, calibration was good at most observed risk levels, but there were very few patients with predicted risk greater than 40–50%, so calibration beyond these probabilities could not be assessed. Regardless, our models should perform adequately for implementation purposes using the optimal threshold cut-offs identified. Finally, the added value of these models may be questioned in the light of previously published models [40–42]. Our models were based on purposively built cohorts, with a priori stakeholder engagement regarding relevant variables and their measurement timing, and were uniquely developed within the clinical rubric of suspected sepsis, which is increasingly recognized as a global health priority.
Conclusion
Post-discharge mortality in the context of suspected sepsis occurs frequently in children under five years old, but those at highest risk can be identified using simple clinical criteria, measured at admission. Being able to select from a range of prediction models, with similar performance parameters, may support wider implementation of digital risk-stratification tools in different clinical settings. Future work must focus on both external validation as well evaluation of how risk-stratified care can improve post-discharge outcomes.
Supporting information
S1 Text. Details of study cohorts and variables used for the full and intermediary models.
https://doi.org/10.1371/journal.pgph.0003050.s002
(DOCX)
S3 Text. Intermediary clinical variable models–variable importance.
https://doi.org/10.1371/journal.pgph.0003050.s004
(DOCX)
S4 Text. Intermediary clinical and social variable models–variable importance.
https://doi.org/10.1371/journal.pgph.0003050.s005
(DOCX)
S5 Text. Intermediary any variable models–performance metrics, coefficients and variable importance.
https://doi.org/10.1371/journal.pgph.0003050.s006
(DOCX)
S6 Text. Final clinical variable models, M6PD-C0-6 and M6PD-C6-60 –performance metrics and coefficients.
https://doi.org/10.1371/journal.pgph.0003050.s007
(DOCX)
S7 Text. Final clinical and social variable models, M6PD-CS0-6 and M6PD-CS6-60 –performance metrics and coefficients.
https://doi.org/10.1371/journal.pgph.0003050.s008
(DOCX)
S8 Text. Final any variable models, M6PD-A0-6 and M6PD-A6-60 –performance metrics and coefficients.
https://doi.org/10.1371/journal.pgph.0003050.s009
(DOCX)
Acknowledgments
We would like to acknowledge all past and present members of the Smart Discharges Research program for their efforts in data collection, administration, logistics support, and all study activities, including but not limited to: Tumwebaze Godfrey, Agaba Collins, Tumukunde Goreth, Naturinda Mackline, Assimwe Abibu, Nakafero Joan, Kiiza Israel, Kitenda Julius, Kamba Ayub, Kuguminkiriza Brenda, Kabajasi Olive, Kembabazi Brenda, Happy Annet, Tusingwire Fredson, Nuwasasira Agaston, Ankatse Christine, Naturinda Rabecca, Nabawanuka Abbey Onyachi, Kamazima Justine, Kairangwa Racheal, Ounyesiga Thomas, Mwoya Yuma, Twebaze Florence, Bulage Mary, Tugumenawe Darius, Tuhame Dyonisius, Twesigye Leonidas, Kamusiime Olivia, Ainembabazi Harriet, Abaho Samuel, Nakabiri Zaituni, Naigaga Shaminah, Kisame Zorah, Babirye Clare, Kayegi Maliza, Opuko Wilson, Mwaka Savio, Baryahirwa Hassan, Mutungi Alexander, Charlene Kanyali, Catherine Kiggundu, Alexia Krepiakevich, Brooklyn Nemetchek, Jessica Trawin, Maryum Chaudhry, Peter Lewis, Rishika Bose, Sahar Zandi Nia, Tamara Dudley, and Cherri Zhang. Without their effort and support, this study would not have been possible.
References
- 1. Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. 2020;395: 200–211. pmid:31954465
- 2. Rudd KE, Kissoon N, Limmathurotsakul D, Bory S, Mutahunga B, Seymour CW, et al. The global burden of sepsis: barriers and potential solutions. Crit Care. 2018;22: 232. pmid:30243300
- 3. Wiens MO, Kissoon N, Holsti L. Challenges in pediatric post-sepsis care in resource limited settings: a narrative review. Transl Pediatr. 2021;10: 2666–2677. pmid:34765492
- 4. Knappett M, Nguyen V, Chaudhry M, Trawin J, Kabakyenga J, Kumbakumba E, et al. Pediatric post-discharge mortality in resource-poor countries: a systematic review and meta-analysis. EClinicalMedicine. 2024;67: 102380. pmid:38204490
- 5. Childhood Acute Illness and Nutrition (CHAIN) Network. Childhood mortality during and after acute illness in Africa and south Asia: a prospective cohort study. Lancet Glob Heal. 2022;10: e673–e684. pmid:35427524
- 6. Wiens MO, Bone JN, Kumbakumba E, Businge S, Tagoola A, Sherine SO, et al. Mortality after hospital discharge among children younger than 5 years admitted with suspected sepsis in Uganda: a prospective, multisite, observational cohort study. Lancet Child Adolesc Heal. 2023;7: 555–566. pmid:37182535
- 7. Nemetchek B, English L, Kissoon N, Ansermino JM, Moschovis PP, Kabakyenga J, et al. Paediatric postdischarge mortality in developing countries: a systematic review. BMJ Open. 2018;8: e023445. pmid:30593550
- 8. Paul S, Tickell KD, Ojee E, Oduol C, Martin S, Singa B, et al. Knowledge, attitudes, and perceptions of Kenyan healthcare workers regarding pediatric discharge from hospital. Prazeres F, editor. PLoS One. 2021;16: e0249569. pmid:33891601
- 9. Krepiakevich A, Khowaja AR, Kabajaasi O, Nemetchek B, Ansermino JM, Kissoon N, et al. Out of pocket costs and time/productivity losses for pediatric sepsis in Uganda: a mixed-methods study. BMC Health Serv Res. 2021;21: 1252. pmid:34798891
- 10. Nemetchek B, Khowaja A, Kavuma A, Kabajaasi O, Olirus Owilli A, Ansermino JM, et al. Exploring healthcare providers’ perspectives of the paediatric discharge process in Uganda: a qualitative exploratory study. BMJ Open. 2019;9: e029526. pmid:31494611
- 11. English L, Kumbakumba E, Larson CP, Kabakyenga J, Singer J, Kissoon N, et al. Pediatric out-of-hospital deaths following hospital discharge: a mixed-methods study. Afr Health Sci. 2016;16: 883–891. pmid:28479878
- 12. Ahmed SM, Brintz BJ, Talbert A, Ngari M, Pavlinac PB, Platts-Mills JA, et al. Derivation and external validation of a clinical prognostic model identifying children at risk of death following presentation for diarrheal care. PLOS Glob public Heal. 2023;3: e0001937. pmid:37368867
- 13. Wiens MO, Kumbakumba E, Larson CP, Ansermino JM, Singer J, Kissoon N, et al. Postdischarge mortality in children with acute infectious diseases: derivation of postdischarge mortality prediction models. BMJ Open. 2015;5: e009449. pmid:26608641
- 14. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement. Ann Intern Med. 2015;162: 55–63. pmid:25560714
- 15. Uganda Bureau of Statistics. Population & Censuses. 2022 [cited 5 Dec 2023]. Available: https://www.ubos.org/explore-statistics/20/.
- 16. Goldstein B, Giroir B, Randolph A, International Consensus Conference on Pediatric Sepsis. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med. 2005;6: 2–8. pmid:15636651
- 17. Wiens M, Kissoon N (Tex), Ansermino JM, Barigye C, Businge S, Kumbakumba E, et al. Smart Discharges to improve post-discharge health outcomes in children: A prospective before-after study with staggered implementation. In: Borealis, V1 [Internet]. 2023 [cited 5 Dec 2023]. Available: https://doi.org/10.5683/SP3/QRUMNQ.
- 18. Wiens MO, Kissoon N, Kumbakumba E, Singer J, Moschovis PP, Ansermino JM, et al. Selecting candidate predictor variables for the modelling of post-discharge mortality from sepsis: a protocol development project. Afr Health Sci. 2016;16: 162–9. pmid:27358628
- 19. Nemetchek BR, Liang L, Kissoon N, Ansermino MJ, Kabakyenga J, Lavoie PM, et al. Predictor variables for post-discharge mortality modelling in infants: a protocol development project. Afr Health Sci. 2018;18: 1214–1225. pmid:30766588
- 20. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42: 377–81. pmid:18929686
- 21. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95: 103208. pmid:31078660
- 22. Jeni LA, Cohn JF, De La Torre F. Facing Imbalanced Data Recommendations for the Use of Performance Metrics. Int Conf Affect Comput Intell Interact Work [proceedings] ACII. 2013;2013: 245–251. pmid:25574450
- 23. Kuhn M. Building Predictive Models in R Using the caret Package. J Stat Softw. 2008;28.
- 24.
R Core Team. R: A language and environment for statistical computing. In: R Founcation for Statistical Computing, Vienna, Austria [Internet]. 2021 [cited 5 Dec 2023]. Available: https://www.r-project.org/.
- 25. Wiens MO, Kumbakumba E, Larson CP, Moschovis PP, Barigye C, Kabakyenga J, et al. Scheduled Follow-Up Referrals and Simple Prevention Kits Including Counseling to Improve Post-Discharge Outcomes Among Children in Uganda: A Proof-of-Concept Study. Glob Heal Sci Pract. 2016;4: 422–434. pmid:27628107
- 26. Akech S, Kwambai T, Wiens MO, Chandna A, Berkley JA, Snow RW. Tackling post-discharge mortality in children living in LMICs to reduce child deaths. Lancet Child Adolesc Heal. 2023;7: 149–151. pmid:36682368
- 27. Wiens MO, Kissoon N, Kabakyenga J. Smart Hospital Discharges to Address a Neglected Epidemic in Sepsis in Low- and Middle-Income Countries. JAMA Pediatr. 2018;172: 213–214. pmid:29379956
- 28. Janssen KJM, Vergouwe Y, Donders ART, Harrell FE, Chen Q, Grobbee DE, et al. Dealing with missing predictor values when applying clinical prediction models. Clin Chem. 2009;55: 994–1001. pmid:19282357
- 29. Hoogland J, van Barreveld M, Debray TPA, Reitsma JB, Verstraelen TE, Dijkgraaf MGW, et al. Handling missing predictor values when validating and applying a prediction model to new patients. Stat Med. 2020;39: 3591–3607. pmid:32687233
- 30. Kabajaasi O, Trawin J, Derksen B, Komugisha C, Mwaka S, Waiswa P, et al. Transitions from hospital to home: A mixed methods study to evaluate pediatric discharges in Uganda. PLOS Glob public Heal. 2023;3: e0002173. pmid:37703267
- 31. Wynants L, van Smeden M, McLernon DJ, Timmerman D, Steyerberg EW, Van Calster B. Three myths about risk thresholds for prediction models. BMC Med. 2019;17: 192. pmid:31651317
- 32. Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M. External validation of prognostic models: what, why, how, when and where? Clin Kidney J. 2020;14: 49–58. pmid:33564405
- 33. Otokiti AU, Ozoude MM, Williams KS, Sadiq-onilenla RA, Ojo SA, Wasarme LB, et al. The Need to Prioritize Model-Updating Processes in Clinical Artificial Intelligence (AI) Models: Protocol for a Scoping Review. JMIR Res Protoc. 2023;12: e37685. pmid:36795464
- 34. Davis SE, Greevy RA, Lasko TA, Walsh CG, Matheny ME. Detection of calibration drift in clinical prediction models to inform model updating. J Biomed Inform. 2020;112: 103611. pmid:33157313
- 35. Mitchell M, Kan L. Digital Technology and the Future of Health Systems. Heal Syst Reform. 2019;5: 113–120. pmid:30908111
- 36. Sharma V, Ali I, van der Veer S, Martin G, Ainsworth J, Augustine T. Adoption of clinical risk prediction tools is limited by a lack of integration with electronic health records. BMJ Heal Care Informatics. 2021;28: e100253. pmid:33608259
- 37. University of Oslo. The world’s largest health information management system—developed through global collaboration led by UiO. In: dhis2 [Internet]. [cited 5 Dec 2023]. Available: https://dhis2.org/.
- 38. Endalamaw A, Erku D, Khatri RB, Nigatu F, Wolka E, Zewdie A, et al. Successes, weaknesses, and recommendations to strengthen primary health care: a scoping review. Arch Public Health. 2023;81: 100. pmid:37268966
- 39. Manyazewal T, Woldeamanuel Y, Blumberg HM, Fekadu A, Marconi VC. The potential use of digital health technologies in the African context: a systematic review of evidence from Ethiopia. NPJ Digit Med. 2021;4: 125. pmid:34404895
- 40.
ClinicalTrials.gov. Smart Discharges to Improve Post-discharge Health Outcomes in Children. In: NCT05730452 [Internet]. 2023 [cited 5 Dec 2023]. Available: https://clinicaltrials.gov/ct2/show/NCT05730452.
- 41. Madrid L, Casellas A, Sacoor C, Quintó L, Sitoe A, Varo R, et al. Postdischarge Mortality Prediction in Sub-Saharan Africa. Pediatrics. 2019;143: e20180606. pmid:30552144
- 42. Ngari MM, Fegan G, Mwangome MK, Ngama MJ, Mturi N, Scott JAG, et al. Mortality after Inpatient Treatment for Severe Pneumonia in Children: a Cohort Study. Paediatr Perinat Epidemiol. 2017;31: 233–242. pmid:28317139