Pulse oximetry for children with pneumonia treated as outpatients in rural Malawi

Abstract Objective To investigate implementation of outpatient pulse oximetry among children with pneumonia, in Malawi. Methods In 2011, 72 health-care providers at 18 rural health centres and 38 community health workers received training in the use of pulse oximetry to measure haemoglobin oxygen saturations. Data collected, between 1 January 2012 and 30 June 2014 by the trained individuals, on children aged 2–59 months with clinically diagnosed pneumonia were analysed. Findings Of the 14 092 children included in the analysis, 13 266 (94.1%) were successfully checked by oximetry. Among the children with chest indrawing and/or danger signs, those with a measured oxygen saturation below  90% were more than twice as likely to have been referred as those with higher saturations (84.3% [385/457] vs 41.5% [871/2099]; P < 0.001). The availability of oximetry appeared to have increased the referral rate for severely hypoxaemic children without chest indrawing or danger signs from 0% to 27.2% (P < 0.001). In the absence of oximetry, if the relevant World Health Organization (WHO) guidelines published in 2014 had been applied, 390/568 (68.7%) severely hypoxaemic children at study health centres and 52/84 (61.9%) severely hypoxaemic children seen by community health workers would have been considered ineligible for referral. Conclusion Implementation of pulse oximetry by our trainees substantially increased the referrals of Malawian children with severe hypoxaemic pneumonia. When data from oximetry were excluded, retrospective application of the guidelines published by WHO in 2014 failed to identify a considerable proportion of severely hypoxaemic children eligible only via oximetry.


Introduction
Among children with pneumonia, hypoxaemia is common, predicts mortality and is a marker of severe illness. [1][2][3] Pulse oximetry -hereafter called oximetry -is the standard tool for non-invasively measuring peripheral arterial haemoglobin oxygen saturation. In low-income countries, however, access to oximetry has lagged behind access to life-saving oxygen treatment. [3][4][5][6][7] In rural areas in low-income countries, there is interest in training community and health-centre-based health workers in oximetry -and making oximetry more widely available -so that more hypoxaemic children at risk of death can be referred to hospitals. 8 Oximetry requires negligible infrastructure, is portable, non-invasive and user-friendly and offers a more accurate and objective way to identify hypoxaemia in children than clinical signs alone. 3,8,9 Among practitioners and caregivers faced with decisions on the care of a child with severe pneumonia, the results of oximetry may be more persuasive than a clinical assessment alone. 10 In late 2011, as part of a three-year prospective, observational study of the impact of a 13-valent pneumococcal conjugate vaccine, we introduced oximetry into two districts of central Malawi. One aim was to evaluate the usefulness of oximetry during the care, by rural health workers, of children aged 2-59 months with pneumonia.

Methods
In our two study districts of Lilongwe and Mchinji, we assessed, prospectively, the quality of oxygen saturation measurement by oximetry and the impact of such measurement on referral decisions that were made according to the latest relevant Malawian guidelines. At the time of our study, the Malawian guidelines on pneumonia care at outpatient facilities and in the community 11 were consistent with the World Health Organization's (WHO's) pre-2014 guidelines. 12, 13 We also estimated retrospectively the number of our hypoxaemic study children who would not have been referred if -as is usual for rural Malawi -oximetry had not been available and if the WHO 2014 guidelines on the integrated management of childhood illnesses -which do not recommend referral of pneumonia cases with chest indrawing at outpatient facilities 14 -had been followed.

Prospective routine care
In 2011, in our two study districts, we trained 110 health-care providers -i.e. 72 rural public-sector practitioners at 18 health centres and 38 community health workers (CHW) -in the use of oximetry, the keeping of medical records and other aspects of the care of children younger than five years with pneumonia. These providers had been selected, by the investigators and the Malawian Ministry of Health, because they worked in areas with consistent health services and were considered to Objective To investigate implementation of outpatient pulse oximetry among children with pneumonia, in Malawi. Methods In 2011, 72 health-care providers at 18 rural health centres and 38 community health workers received training in the use of pulse oximetry to measure haemoglobin oxygen saturations. Data collected, between 1 January 2012 and 30 June 2014 by the trained individuals, on children aged 2-59 months with clinically diagnosed pneumonia were analysed. Findings Of the 14 092 children included in the analysis, 13 266 (94.1%) were successfully checked by oximetry. Among the children with chest indrawing and/or danger signs, those with a measured oxygen saturation below 90% were more than twice as likely to have been referred as those with higher saturations (84.3% [385/457] vs 41.5% [871/2099]; P < 0.001). The availability of oximetry appeared to have increased the referral rate for severely hypoxaemic children without chest indrawing or danger signs from 0% to 27.2% (P < 0.001). In the absence of oximetry, if the relevant World Health Organization (WHO) guidelines published in 2014 had been applied, 390/568 (68.7%) severely hypoxaemic children at study health centres and 52/84 (61.9%) severely hypoxaemic children seen by community health workers would have been considered ineligible for referral. Conclusion Implementation of pulse oximetry by our trainees substantially increased the referrals of Malawian children with severe hypoxaemic pneumonia. When data from oximetry were excluded, retrospective application of the guidelines published by WHO in 2014 failed to identify a considerable proportion of severely hypoxaemic children eligible only via oximetry.

Outpatient pulse oximetry in Malawi
Eric D McCollum et al.
be representative, in terms of paediatric health care, of rural Malawi. In these study areas, daily clinics in rural health centres were run by nurses or non-physician clinicians called clinical officers or medical assistants while salaried CHW -called health surveillance assistantsran weekly village clinics. None of the study health centres provided oxygen. The training lasted one day, cost approximately 13 United States dollars (US$) per trainee, was based on videos and small-group practical sessions and was run by a paediatric pulmonologist. Trainees were given lessons in the use, on children, of oximeters fitted with clip probes designed for adult fingers (Acare Technology, Xinzhuang, Taiwan, China). As supplied, by Lifebox Foundation (London, England), these oximeters and probes cost approximately US$ 250 and US$ 25 each, respectively. The trainees were advised to use a probe on a hallux -i.e. big toe -if the patient weighed less than 10 kg or was younger than two years and on an index finger of a heavier or older patient. An oximetric measure-ment was considered interpretable if it showed consistent, high-amplitude plethysmographic waveforms associated with a stable oxygen saturation. Providers were trained to diagnose children with pneumonia according to the Malawian 2000 guidelines supplemented with oximetry. 11-13 Any findings of general danger signs, indrawing or severe hypoxaemia -i.e. an oxygen saturation of less than 90% -were to be considered indicative of the need for hospital referral. At the training's conclusion, participants were evaluated and, if necessary, trained further. We retrained all of the trainees in early 2013.
After a three-month pilot study, all of the trained providers collected data on the children with clinically diagnosed pneumonia who they encountered between 1 January 2012 and 30 June 2014. The community-based and facility-based providers collected such data on routine care forms -as used for integrated community case management and the International Union Against Tuberculosis and Lung Disease's Child Lung Health Programme, respectivelymodified to include oximetry measurement results. 12,15 The providers were not asked to perform any additional duties apart from the oximetry and they did not receive incentives. Members of the research team made monthly supervisory visits to each trained provider, at a monthly cost of about US$ 7 per trainee. Such visits provided opportunities to review the implementation of guidelines, appraise records, make direct observations of providers performing oximetry and interpreting the results and study the maintenance logs for the oximeters so that battery use, cleaning frequency and functionality could be determined. Any performance deficits observed were addressed.

Mid-study quality assessment
In April 2013, we selected randomlyand separately assessed -the oximetric performance of 24 of our trained providers. We used 60 children without pneumonia -each awaiting elective surgery at Kamuzu Central Hospital in  Lilongwe -as the test subjects. The results of the oximetric examination of the same subjects, by a paediatric pulmonologist (EDM), were used as the gold standard. Each provider was also asked to complete a multiple-choice and shortanswer survey on their experience and problems with -and use of -oximetry.

Missed hypoxaemia referrals
We investigated, retrospectively, the effects that applying different sets of guidelines with different oxygen saturation referral thresholds would have had on the referral to hospital of the paediatric pneumonia cases that our study providers encountered. With the data collected by the facility-based providers, we applied the Malawian 2000 guidelines -that recommend the referral of patients at health facilities because of indrawing -and the WHO 2014 guidelines -that do not recommend such referral. Our aim was to estimate the numbers of moderately hypoxaemic children -i.e. children with oxygen saturations of 90-92% -and severely hypoxaemic children, among all eligible children and also among all hypoxaemic children, who would not have been referred if oximetry had been unavailable. We conducted a similar analysis of the data from the CHW but, for pneumonia cases seen by such workers, both the Malawian and WHO guidelines that we considered recommend referral because of indrawing.

Data analysis
For our analyses we used three sets of data: (i) the routine care forms completed by the trained providers; (ii) the maintenance logs for the oximeters; and (iii) the results of the mid-study quality assessment. Normally distributed data were described using means and standard deviations and compared in Student's t-tests. Nonparametric data were described using medians and interquartile ranges and compared in Wilcoxon Mann-Whitney tests. Proportions were compared in Pearson χ 2 tests. For the quality assessment, the level of agreement between each provider and the expert (EDM) was expressed as a weighted kappa. To account for the accuracy of the oximeter used, 16 oxygen saturation values that differed by no more than two percentage points were considered to be in full agreement. Agreements that gave weighted kappas of no more than 0.00 or of 0.01-0.19, 0.20-0.39, 0.40-0.59, 0.60-0.79 and 0.80-1.0 were categorized as poor, slight, fair, moderate, substantial and perfect, respectively. 17 All analyses were performed using Stata version 13.1 (StataCorp. LP, College Station, USA).

Ethical approval
The ethical boards of University College London (protocol 2006/002) and the Malawi Ministry of Health (protocol 941) approved the study protocol and did not require written consent, from the cases, for the collection of data on the routine care of pneumonia cases.

Mid-study quality assessment
Twenty-two of the 24 randomly selected providers responded to the survey questions and were observed measuring oxygen saturation (Table 1 and Table 2). Nearly 94% (1222) of the 1301 successful measurements made by providers were within two percentage points of the expert's measurements. The weighted kappa for the overall level of agreement between the providers and the expert (0.41; Table 2) indicated moderate agreement.

Prospective routine care
During our study, we introduced 56 oximeters with probes. Thirteen (23.2%) oximeters and 24 (42.9%) probes were replaced during the study period.
Over the study period, the providers reportedly measured the oxygen saturations of 13 266 (94.1%) of the 14 092 children with pneumonia who they reviewed (Table 3 and Fig. 1). Although severe hypoxaemia was indicated by a greater proportion of the oxygen saturations recorded by facility-based providers than by CHW [9.3% (568/6087) vs 1.2% (84/7179); P < 0.001], moderate hypoxaemia showed the opposite trend [8.9% (543/6087) vs 10.3% (627/7179); Compared with the pneumonia cases described by the facility-based providers, those described by the CHW were, in general, older and had slower respiratory rates, higher median oxygen saturations and lower prevalences of indrawing and eligibility for referral according to the Malawian guidelines enhanced with oximetry (Table 3). Among the cases described by facilitybased providers, those with failed oxygen saturation measurements were,    Patients were considered to have a general danger sign if they were abnormally sleepy, had convulsions, were not breastfeeding or drinking, were vomiting everything they ingested, showed stridor when calm, were infected with -or had been exposed to -human immunodeficiency virus and/or had severe malnutrition.

Research
Outpatient pulse oximetry in Malawi Eric D McCollum et al.
in general, younger, weighed less and showed a higher prevalence of general danger signs than those with successful measurements (Table 3). Among the cases described by CHW, those with failed oxygen saturation measurements had lower prevalences of general danger signs and referral eligibility than those with successful measurements (Table 3).
Together, the trained providers were more than twice as likely to have referred a case who was clinically eligible for referral when the child had severe hypoxaemia than when the child did not (84.3% [385/457] vs 41.5% [871/2099]; P < 0.001; Fig. 1). If we assume that no clinically ineligible child with severe hypoxaemia would have been referred in the absence of oximetry, the availability of such oximetry appears to have increased referrals of such children by 27.2% -i.e. from zero to 27.2% (53/195; P < 0.001). The results of the trained providers' responses to their oximetric measurements -i.e. in terms of referring pneumonia cases -are summarized in Fig. 1 severely hypoxaemic children) -would not have been referred (Fig. 2) (Fig. 3).

Discussion
This study examines the outpatient use of oximetry by front-line health workers at health centres and in the communities of a low-resource country. We analysed more than 15 400 oxygen saturation values, collected by health providers over three years and across two central Malawian districts, to assess the quality of the oximetric measurements, decisionmaking and -under existing clinical guidelines and the assumption that oximetry was unavailable -the failure to refer moderately or severely hypoxaemic cases. In general, we showed that the overall burden of hypoxaemia in our study area was high and that implementation of oximetry in Malawi, by frontline facility-based providers and CHW, would probably increase the referral of severely hypoxaemic children during routine pneumonia care. While we concluded that the quality of the oximetric measurements made by our trained providers was generally good, equipment durability and referral-related decision-making needs improvement. In the absence of oximetry, guidelines that do not recommend referral because of chest indrawing alone -e.g. the WHO 2014 guidelines on the integrated management of childhood illnesses -hamper the referral of substantial proportions of moderately or severely hypoxaemic Malawian children who may be at high risk of early death.
Our results show that, while not without challenges, rural health workers can use oximetry with moderate reliability. Although the overall weighted kappa that we calculated for the level of provider-expert agreement (0.41) compares favourably with the kappa reported, for inter-observer agreement on child chest indrawing, in the United Republic of Tanzania (0.33), 18 it was lower than we expected. The probable reasons for the relatively low weighted kappa we calculated include the natural temporal variation in one child's oxygen saturation, the influences of movement artefact and poor perfusion on measurement accuracy and the inherent accuracy of the oximeters that were used.
We found the failure of health-care providers using an oximeter to measure oxygen saturation to be associated with certain patient characteristics. In health centres, for example, failure tended to occur with patients who were sicker, younger and smaller than most of the other pneumonia cases that our trained providers encountered. We suggest that a failed measurement in a health centre could be a referral indication. Poor extremity perfusion in a dehydrated or septic child increases the likelihood of measurement failure, especially if the  probe is not ideally sized. 6 Our trained CHW appeared to struggle most with recording the oxygen saturation of mildly ill toddlers. As such children are often active, agitated and anxious with strangers, they may represent a particular challenge to providers who are trying to minimize movement artefact as they measure a child's oxygen saturation. 6 While we considered using paediatric clip or wrap probes in our study, they were more expensive than adult finger clips at the time of our study. One of our aims was to investigate the implementation of equipment that might be affordable when used on a large scale. In low-income countries, low-cost, durable paediatric probes that fit a range of patient sizes while providing precise measurements in less than 30 seconds are likely to be an important factor in the successful implementation of oximetry at a national level. The oximeters we investigated were designed for use in the operating theatres of low-resource countries -not in rural communities. Lifebox Foundation is now designing a low-cost oximeter for use, by front-line health workers, on children (I Walker and I Wilson, personal communication, 2016).

Fig. 2. Estimated effects of the guidelines applied on hospital referrals among paediatric pneumonia cases investigated by health-centre-based health-care providers, Malawi, 2012-2014
Although the referral-related decision-making we investigated sometimes appeared inconsistent, we expected this since, in Malawi, the referral of a child is typically a joint decision, between the provider and caregiver, that is influenced by many social factors -e.g. the childcare available and domestic responsibilities and finances. Our results indicate that, when making referral decisions, providers might perform substantially better when clinically eligible children are also identified as severely hypoxaemic. The data we analysed do not provide contextual, qualitative information as to why some severely hypoxaemic children were referred when others were not. In the future, we plan to conduct focus group discussions to understand the providers' decision-making better. Despite the providers' mixed performance in making referral decisions, their use of oximetry led to the referral of about 248 [385-(41.5%*457)+53=248] severely hypoxaemic children who would probably not have been referred in the absence of oximetry (Fig. 1).
When oximetry is not available -i.e. the normal situation in rural Malawi -providers who follow the Malawian 2000 guidelines or, to a greater extent, the WHO 2014 guidelines are likely to under-refer children with severe hypoxaemia. Among hospitalized children with pneumonia 3,9 and, it seems, also among outpatients with pneumonia in Malawi, oximetry offers a more reliable method of identifying hypoxaemia than the observation of clinical signs. While research in developing countries has focused on severe hypoxaemia's strong association with poor outcomes, 1 recent evidence from a systematic review 19 and from a study of hospitalized Malawian children 20 indicates that even moderate hypoxaemia is also a predictor of death in childhood pneumonia. Given this evidence -and the fact that many moderately hypoxaemic children progress to severe hypoxaemia 21 -all outpatients with pneumonia who have oxygen saturations below 93% should perhaps be referred. We need more outcome-based community-level research on this topic.
Our study had two main limitations. First, the data we analysed were largely collected during routine care and, although such follow-up is recommended, few Malawian children are re-evaluated after they have completed pneumonia treatment. We were therefore unable to determine patient outcomes and were even unable to confirm whether or not referred patients went to hospital. Second, since the study was observational and lacked a control group, our results should be interpreted with caution.