Introduction

Cataract is now one of the most common causes of avoidable blindness in children in Low and Middle Income Countries (LMIC) [1,2,3]. Visual outcomes following childhood cataract surgery in Sub-Saharan Africa (SSA) are poor, with only 44–62% of children achieving a postoperative visual acuity of 6/18 or better [4, 5]; late presentation is a major factor in this [6, 7]. In contrast, in high-income countries 72–91% achieve postoperative visual acuity of 6/18 or better [8, 9]. There is also a vast differential in morbidity and mortality from retinoblastoma between high and low-income countries with a mortality of 70% in SSA [10, 11], compared with <3% in high-income regions [12, 13].

Both childhood cataract and retinoblastoma can be detected by the red reflex test using a direct ophthalmoscope (DO). Current World Health Organisation (WHO) documents relating to the detection of childhood eye disease do not recommend red reflex screening and refer only to vision testing and torchlight examination in primary eye care facilities [14]. This is a pragmatic decision relating to scarcity of and inexperience in using DO [15, 16]. However torchlight examination will only detect advanced cases of cataract and retinoblastoma. In contrast red reflex screening of neonates using a DO, is standard practice in many high-income countries [17].

In recent decades, the successful implementation of public health intervention programmes with vitamin A supplementation and high coverage measles immunisation in SSA have led to a marked reduction in blinding corneal scarring. If the health workers who administer vitamin A and vaccinations to young children could also be trained to detect cataract and other pathology early, there is potential to achieve similar reductions in other causes of childhood blindness, through early detection and prompt treatment [18]. Although red reflex screening is ideally carried out aged 4–6 weeks, presentation lag times for congenital cataract and retinoblastoma are measured in years rather than weeks in our population [6, 7], and therefore opportunistic use of the vaccination encounters during the first year of life for screening would be a potentially significant improvement.

It is currently unclear which might be the most suitable screening approach for early childhood eye disease in a primary care setting. Although WHO pragmatically recommend torchlight, our experience and unpublished pilot data suggest that this results in under-ascertainment of cataract and retinoblastoma.

We were interested in potential alternatives to the standard DO for assessing the red reflex. A novel low-cost and easy to use DO (Arclight) has been developed [19]. The device, which uses a light emitting diode (LED), is charged by a small solar panel in the casing, and costs USD $7.50 when purchased in bulk. A second device is Peek Retina [20], which was developed as an adaptor for smartphones to permit examination of the retina (rather than to specifically examine the red reflex). It is comprised of a small adaptor with prisms and LED, which is attached on a smartphone and aligned with the phone camera. The prism provides co-axial illumination and uses the observation systems of the camera to capture images of the retina. Peek Retina is not designed for red reflex assessment but holds potential for differentiating normal and abnormal red reflex based on the co-axial light source. A third new device is the CatCam, a prototype comprising a modified smartphone with a co-axial infra-red LED and infra-red sensitive camera, which has an advantage of assessing the fundus reflex without causing pupil constriction and enabling transfer of digital images for remote reading. These new technologies offer the potential to make red reflex testing easier and more acceptable to primary health care workers. However, these devices have not been validated as screening tools for cataract or retinoblastoma in children.

Therefore, we aimed to compare the sensitivity and specificity of these three new screening devices with each other and with torchlight. Secondly, one of the devices (Arclight) was used in a pilot screening programme in primary health care clinics providing services for young children, to investigate its feasibility and acceptability as a screening tool for childhood eye disease in the primary care setting. The paper covers two separate but related studies-sensitivity analysis requires an enriched sample and therefore a hospital-based study whereas real life feasibility requires community screening assessment, for which detection rates but not sensitivity can be measures.

Methods

Diagnostic accuracy study

To compare the sensitivity and specificities of four different screening methods for cataract and retinoblastoma performed by nurses, we conducted a prospective, cross-sectional, hospital-based, enriched sample study comparing their diagnostic accuracy to the results of a full clinical examination performed by a consultant paediatric ophthalmologist. The study was designed according to ‘Standards for reporting diagnostic accuracy studies’ [21].

We recruited participants from children attending the paediatric ophthalmology clinic at Kilimanjaro Christian Medical Centre (KCMC), Moshi, Tanzania between November 2016 and March 2017. For this study, we required a mixed group of young children some of whom had cataract or retinoblastoma and some of whom did not. Prior to recruitment, all new patients presenting to the clinic underwent a preliminary red reflex test assessment by a junior paediatric ophthalmologist, independent of the other study procedures, using a DO without pupil dilation. Children under 5 years from consenting families were then recruited as potential positives (abnormal red reflex) or potential negatives (normal red reflex). All patients who had an abnormal red reflex or normal reflex were included in the first series from which the enriched sample was selected. Children with other obvious eye pathologies and whose carers were unwilling to provide consent were excluded.

Following recruitment, each child had both eyes examined using each of the four screening tests being compared: torchlight, Arclight, PeekRetina (model EC2Y5EJ, UK- using Sony 23 compact) and CatCam (prototype modified Google Nexus 5X with co-axial infra-red LED peak wavelength 860 nm). The examinations were conducted by four different ophthalmic nurses, with a different nurse performing each test on the child. The order of the tests was randomised for each child, using a simple random number table. The devices were rotated throughout the study so that each nurse assessed children using all four methods. The nurses were masked to each other’s findings. The nurse made a subjective judgement as to whether the red reflex was normal or abnormal. Examinations were performed in a dimly lit room and the pupils were not pharmacologically dilated.

Immediately after the nurses’ screening, all children had their pupils dilated using Tropicamide 0.8% with Phenylephrine hydrochloride 5%. They were then examined by a masked consultant paediatric ophthalmologist (GF) using a slit lamp and indirect ophthalmoscopy for the presence or absence of ocular disease, and if present, the diagnosis was established. This was the reference standard assessment. Each child was then assigned a final status as “true positive” or “true negative” based on the ophthalmologist’s findings. Only children with cataract or retinoblastoma were defined as true positives.

Arclight screening in the primary care setting—feasibility study

To investigate the feasibility of red reflex screening in a primary health care setting in Tanzania we conducted a prospective observational study. We recruited 24 Reproductive and Child Health (RCH) nurses who regularly examine children in 12 local RCH clinics. The nurses attended a 2-day training seminar for red reflex screening with three new devices, Arclight, prototypes of Peek Retina and CatCam. At the end the trainer (GF) (1) assessed each nurse to check they had grasped the technique and were observed performing it with a baby of 6 months or under and (2) asked each trained nurse to provide feedback on the learning experience using a structured questionnaire.

Following the feedback during the nurse training, we selected the Arclight in the prospective screening study. Although CatCam performed better in the initial hospital-based study, it was not possible to test CatCam in this large study due to the limited availability of devices at that time. Moreover, the good performance of Arclight in the initial comparative study and the positive qualitative feedback from the 24 RCH nurses meant that this was considered a good alternative.

Following training, the 24 RCH nurses examined children <5 years who were attending their RCH clinic for growth monitoring and immunisation and had never been examined before. Recruitment took place between February 2017 and June 2017. Children with serious medical conditions or whose carers were unwilling to provide consent were excluded. The nurses performed the red reflex screening using the Arclight in a dimly lit room. The pupils were not pharmacologically dilated. The RCH nurses completed a questionnaire about their experience of using the Arclight for red reflex screening.

If a child “screened positive” on red reflex examination in one or both eyes in the RCH clinic, they were referred to the paediatric eye clinic-KCMC an average of 5 km distance. They were re-examined by a paediatric ophthalmologist (GF) using a slit lamp and dilated indirect ophthalmoscopy to determine whether or not there was media opacity or other pathology. Children who screened negative were not referred for examination by the ophthalmologist in this part of the study.

Statistical analysis

Data were double entered and managed in Access (Microsoft). The analysis was performed in STATA Version 14 (StataCorp)

For the initial comparison study a sample of 40 positive cases in the whole sample was estimated to provide at least +/− 15% precision to estimate the sensitivity of the test (based on a sensitivity of 50%). Negatives were also recruited to ‘mask’ the testers and to estimate the specificity.

For the feasibility study, we estimated that 100 Arclight examinations per screener would provide adequate screening experience and ability to detect cataract in the general population. We based the required sample size on the observed prevalence rate of cataract 1.18% in the pilot study and estimated that a sample size of 2400 would detect cases with +/− 1.05% precision at 95% confidence level. An intraclass correlation coefficient of 0.05 was taken into account to adjust for cluster variance between nurses [22].

The sensitivity, specificity, and area under the curve of each of the four methods were estimated along with their confidence intervals. A scatter plot of sensitivity against specificity was plotted to compare the four methods. We used descriptive statistics to report nurse’s learning experiences with Arclight, Peek Retina and CatCam and the challenges of red reflex screening using Arclight in RCH clinics. In all screened failures, positive predictive values were determined.

Results

Diagnostic accuracy study

We recruited 101 children who had a mean age of 33.4 (range 2.0–60.0 months) and 58.4% were male. Of the 59 males and 42 females, 26 males and 15 females (total 41) were found to be “true positive” by the reference standard ophthalmologist’s examination: 37 had cataract and four had retinoblastoma. The remaining 60 children were designated as “true negative” by the reference examination (20 had mild allergic conjunctivitis, 13 with abnormal red reflex (eight had refractive error and five had strabismus), seven had nasolacrimal duct obstruction, three had optic atrophy, three had blepharitis, two had bacterial conjunctivitis, two had sub-conjunctiva haemorrhage and ten were truly normal).

The sensitivity and specificity values of the four techniques, relative to the reference standard ophthalmologist’s examination, are reported in Table 1.

Table 1 Results of screening tests for detection of cataract and retinoblastoma using four different screening methods (torchlight, Arclight, Peek Retina and CatCam).
Full size table

The torchlight assessment had a very low sensitivity of 7.3%. The other three tests had high to very high sensitivities (>90%) and specificities. The CatCam performed the best of the four tests, with only 1/41 “true positive” child not identified and no false positive results. The estimated sensitivity and specificity were highest for CatCam, followed by Arclight and Peek Retina, Fig. 1.

Fig. 1: Area Under ROC Curve.
figure 1

Sensitivity, specificity and confidence intervals of CatCam, Arclight, Peek Retina and Pentorch. (ROC, Receiver Operating Characteristic).

Full size image

Community nurses screened 2728 children aged <5 years between February 2017 and June 2017 (Fig. 2). Their median age was 9.0 (IQR: 4–17 months) and 1259 (46.2%) were male. They identified 24 (0.88%, 95% CI: 0.31–1.48%) children who were considered to have an abnormal red reflex using the Arclight: “screened positive”.

Fig. 2: A flow chart representing a protocal for community screening.
figure 2

Number of participants screened by community nurses and the outcomes.

Full size image

All children who “screened positive” were referred to KCMC and examined by the paediatric ophthalmologist. Seven of these children had the target conditions: six cataracts (four bilateral, two unilateral) and one retinoblastoma. This gives a prevalence estimate of 1.5/1000 (95% CI: 0.40–3.75/1000) for bilateral cataract and 0.37/1000 (95% CI: 0.00–2.04/1000) for retinoblastoma. All seven children with target conditions (cataract and retinoblastoma) needed surgery. Retinoblastoma cases were all group E according to the International Classification of Retinoblastoma and were all enucleated. Seventeen of 24 (70.8%) “screened positive” children did not have either target condition (29.2% positive predictive value (95% CI 12.62–51.09)), however, seven had other ocular pathologies (one corneal opacity, two strabismus, four refractive error)-these can also cause abnormal red reflex. One child underwent strabismus surgery.

After three months of screening using Arclight, the majority of nurses (23/24) completed the questionnaire, and all had additional comments. They reported that they could differentiate a normal from an abnormal red reflex, 3/23 (13%) very easily, 9/23 (39%) easily and 11/23 (48%) with some difficulty. Three quarters (74%) of nurses reported that examining a neonate less than 4 weeks old was more difficult than an older child. Very quick examination (duration 30 s to 1 min) was reported by 8/23 (35%) nurses; an average of 2–3 min 10/23 (43%) nurses; an average of 4–5 min 3/23 (13%) while 2/23 (9%) took longer than 5 min to elicit a red reflex. A third of nurses (35%) reported that learning Arclight needs less than a day, another third (39%) thought a full day was required and the remainder (26%) thought two or more days were needed. All nurses reported that parents were happy and willing to let their children be screened. 18/23 (78%) reported that there was a suitable space in their facilities for screening. However, 9/23 (39%) reported that screening interfered with their day-to-day activities because of staff shortages, large numbers of children, increased workload, and multiple responsibilities.

Discussion

Community screening is not a suitable environment for accurately estimating the sensitivity of a screening test where the disease is rare (in this case an estimated prevalence of 1–2 per thousand). Therefore, we initially compared the four diagnostic tests in a hospital-based diagnostic accuracy study, using a sample enriched with true positives. The weakness of that approach is that the screening test may be less sensitive in the community when performed by non-specialist nurses in a less optimal environment, and on younger patients (the hospital patients are older because we need true positives and our children present late in the absence of a screening programme). It was not practical to cross check all the cases in the community phase of the study because it would have involved stationing paediatric ophthalmologists in several in RCH clinics for 3 months. We tried to limit this weakness by (1) selecting ophthalmic clinic nurses in the comparative study who had no previous training in red reflex testing and (2) ensuring that the RCH nurses were able to perform red reflex testing after their training during the workshop.

The comparison study showed that assessment with torchlight was the least sensitive device (7%). Assessment with CatCam had the highest sensitivity (97.6%; 95% CI: 87.1–99.9), possibly because infra-red light does not cause pupil constriction, so the pupillary diameter is larger in this test. The sensitivity of assessment with Arclight ophthalmoscope was also very good (92.7%; 95% CI: 80.1–98.5) which agrees closely with the sensitivity of 93.8% reported by Ruttum MS et al. [23] where 3rd-year medical students were given a 30 min introduction to ophthalmoscope and retinoscope and then allowed to examine eight children with congenital cataract and eight age-matched controls with normal findings.

Although CatCam performed better than Arclight in the comparative study, only one prototype device was available which meant it could not be used in the prospective RCH clinic study. Arclight had already been demonstrated to have a sensitivity of (92.7%; 95% CI: 80.1–98.5) and was readily available, so was used in this second phase.

Both CatCam and Peek Retina prototypes were used for the study. Catcam is a prototype modified smartphone device which is not commercially available. It does not test red-reflex but instead images the infra-red reflex. Both this and a separate UK proof-of-concept study have indicated that infra-red reflex imaging may make childhood cataract screening more accurate. A large UK newborn population screening study is proposed to test this hypothesis using the more recent standalone prototype, Neocam. If superiority is confirmed, the technology may be commercialised. Given the manufacturing costs it would be expected to retail for under GBP 5000, though might be available for less than this in developing countries. Peek Retina, a smartphone add-on designed for dilated retinal examination, specifically optic disc assessment, is commercially available for GBP 180, however the product is frequently shared with partners in LMICs for free or at a significant discount to further the mission of Peek’s work.

Our data suggest that if 10,000 children are screened using red reflex testing, ninety children are expected to be referred as screening failures, 22 (0.22%) of whom are expected to have true cataract (15 bilateral); 4 (0.37%) retinoblastoma and 30 (0.3%) children are expected to have other conditions such as refractive errors and strabismus, and 34 will be normal.

Our prevalence figures are higher than previously reported. Published estimates of retinoblastoma incidence (thought to be globally consistent) are 1 per 15–20,000 live births [12]. However, the lower end of our confidence interval for prevalence would be consistent with that incidence. Cataract prevalence data for children are scarce and variable; our prevalence estimate is higher than published estimates from SSA and India i.e. 0.4–8.5 per 10,000 children [2, 24]. Our prevalence figures could be an underestimate, in that sensitivity of Arclight in the eye clinic was 92.7 percent and may be slightly less in the community because of less than optimal conditions and the number of false negatives was not established. Conversely, our estimate may be biased upwards if news had circulated in the community that eye screening was taking place in certain health centres and carers who suspected an eye problem in their child preferentially sought out these centres. However, it was in the same order of magnitude as the pilot study prevalence figure from a separate large community sample, suggesting consistency though the same phenomenon could have happened here too.

These findings are of public health significance in view of the consequences of delay in diagnoses and adherence to referral. Attention needs to be given to address the human resource issues identified by the nurses who conducted this exercise in their own workplaces. Our study shows that more than 50% of non-ophthalmic nurses found Arclight easy to use although screening infants (with smaller palpebral fissures and pupils as well as limited co-operation) did present some difficulties. The majority reported feasibly short learning times and subsequent examination times, as has been found in other studies [16, 20]. This means that Arclight red reflex screening by RCH nurses is feasible as well as being both sensitive and specific. This method offers the potential for screening for congenital cataract and retinoblastoma to become an integral component of primary child health activities. A potential advantage of a digital imaging system such as CatCam is the facility for telemedicine and, potentially, automated image analysis, which may decrease the training required for rural screeners in the future. The disadvantage may be initial cost.

In summary we recommend a change of WHO guidelines from vision testing and torchlight examination to red reflex testing at the primary care facilities. We have shown that screening with CatCam and Arclight are sensitive and specific, and that screening with Arclight is feasible in the community. Red reflex screening yields significantly higher prevalence estimates of cataract in children than previously reported in the region, highlighting the public health importance of this intervention.

Summary

What was known before

  • Lag times to cataract surgery in high-income countries are measured in terms of weeks rather than months and years in low-income countries because babies undergo neonatal red reflex screening.

  • Current WHO documents relating to the detection of childhood eye disease in low resource settings refer only to vision testing and torchlight examination in primary health care facilities, rather than red reflex testing.

What this study adds

  • Arclight and CatCam have high sensitivity than torchlight, are easy to learn and use by primary health care nurses.

  • Use of penlight is poor quality care and should be removed from guidelines.

  • The community pick up rate for both cataract and retinoblastoma suggested higher prevalence for both conditions than previous studies, emphasising not only greater sensitivity of red reflex testing over torchlight but its public health importance.