Assessment of cataract surgical outcomes in settings where follow-up is poor: PRECOG, a multicentre observational study

Background Poor follow-up after cataract surgery in developing countries makes assessment of operative quality uncertain. We aimed to assess two strategies to measure visual outcome: recording the visual acuity of all patients 3 or fewer days postoperatively (early postoperative assessment), and recording that of only those patients who returned for the fi nal follow-up examination after 40 or more days without additional prompting.


Introduction
Unoperated cataract remains the most common cause of blindness worldwide, 1 even thoug h the disorder can be eff ectively and inexpensively treated with a standard procedure. When treated by skilled surgeons, 90% of patients can achieve good vision (best-corrected visual acuity of 6/12 or better), [2][3][4] and an equal proportion are satisfi ed with their surgical result. 5 Poor surgical outcomes and inadequate access to surgery are major impediments to the reduction of blindness from cataract, particularly in low-resource settings. [6][7][8][9][10][11][12][13] Improving surgical capacity by training additional surgeons and providing equipment could help to address both issues, 14,15 but success also depends on monitoring surgical quality. Visual acuity after cataract surgery has traditionally been measured weeks to months after the operation, since wound healing can change refractive power, and gradual resolution of common complications such as corneal oedema can substantially improve vision. Less often, visual decline from surgical complications can also occur.
In many developing countries, postoperative follow-up rates are as low as 20-30%, 16 because of poor transportation infrastructure, costs to patients, and failure to communicate the benefi ts of returning, 17 which can include distribution of corrective glasses. Low rates of postoperative follow-up and uncertainty about whether returning patients are representative of the operated cohort make assessment of performance against objective standards diffi cult. WHO 18 recommends that 80% of patients should have uncorrected (without refraction) visual acuity of 6/18 or better in the operated eye, but does not stipulate a specifi c time after surgery for assessment.
In the past 20 years, phacoemulsifi cation and smallincision cataract surgery, both of which use small, rapidly healing cataract surgical wounds, have been widely adopted. Although studies have shown that vision changes during healing are reduced with these techniques, [19][20][21] only a few studies have examined whether early visual assessment is predictive of fi nal vision. 16,22 An international group of organisations with a focus on eye health undertook the Prospective Review of Early Cataract Outcomes and Grading (PRECOG) study to test new approaches for the measurement of cataract surgical quality in settings with poor rates of follow-up. Our hypothesis was that, even in these settings, data that accurately represent visual outcomes for an operated cohort can be obtained. Our aims were to measure the correlation for individual patients and for hospitals between visual acuity at 3 or fewer days postoperatively and at 40 or more days postoperatively, when acuity should have stabilised; to assess the extent to which the visual acuity of patients who return without additional prompting to the clinic after 40 or more days for followup examination is representative of the entire operative cohort; and to establish evidence-based targets for good visual acuity results for hospitals.

Study design and participants
A geographically diverse range of large and small, urban and rural, and public and private hospitals in developing countries were invited to take part in the PRECOG study through ORBIS International (New York, NY, USA), the Fred Hollows Foundation (Sydney, NSW, Australia), Helen Keller International (New York, NY, USA), the International Association for the Prevention of Blindness Latin American Offi ce (Miami, FL, USA), and the Aravind Eye Care System (Madurai, India). The only requirement for hospitals was that surgical output be suffi cient to complete recruitment within 6 months. The principal investigator (NC) did a single, 1-day training session in each country, during which the study investigators at participating centres reviewed the study protocol and forms. Investigators at the coordinating centre (Zhongshan Ophthalmic Centre, Guangzhou, China) regularly reviewed submitted forms and provided feedback to regional coordinators, who discussed any protocol or data entry issues with participating centres.
Each hospital enrolled 40-120 consecutive patients aged 30 years or older scheduled to undergo surgery for adult-onset cataract. We chose this range of number of patients per hospital to provide a representative sample of surgical procedures at each centre, while minimising the burden of postoperative follow-up. Patients had been diagnosed with visually signifi cant cataract, and could have any visual acuity in the operative eye, but could not have ocular comorbidities apparent during preoperative examination. If comorbidities initially masked by lens opacity were discovered postoperatively, patients were not censored, since this situation is common in developing-country settings. Patients underwent surgery by any widely used method, including phacoemulsifi cation, small-incision cataract surgery, or extracapsular cataract extraction.
The study protocol was approved by the institutional review boards at the coordinating centre and at other participating organisations. Some hospitals did not have their own institutional review boards, in which case review was done by the sponsoring organisation. All participants provided written informed consent, and the principles of the Declaration of Helsinki were followed throughout.

Preoperative examination
All patients underwent preoperative ocular examinations by an ophthalmologist with slit lamp and dilation of the pupil. Patients' demographic information and cataract surgical histories were recorded, as were uncorrected visual acuity and best-corrected visual acuity (with glasses) in both eyes, presence of ocular comorbidities, and biometric measurements to identify the power of the intraocular lens needed for surgery.
Visual acuity was assessed at each hospital with that hospital's usual charts and at the recommended distance (usually 4 m). Tumbling E charts were used in all locations. Measurements were made separately for each eye, beginning with the right, in a well-lit area of the clinic. After correctly identifying the direction of more than half of the optotypes on the uppermost line (usually equivalent to a visual acuity of 3/60), patients moved to the next and then to successively lower lines. The lowest line on which more than half of the  optotypes were correctly read was recorded as the patient's visual acuity. Patients unable to read the top line at the standard distance attempted to read it from 1 m, and visual acuity was divided by the standard distance. Throughout testing, the examiner ensured that the non-tested eye was fully occluded, and that the patient maintained the proper distance from the chart and did not squint (which creates an optical pinhole eff ect that improves vision).

Early postoperative examination
The early postoperative examination was done within 72 h after surgery, at hospital discharge in most centres. Data recorded included dates of surgery and of examination, examiner's identity (the operating surgeon was not allowed to do the examination), uncorrected and best-corrected visual acuity in the operated eye, type of surgery done (phacoemulsifi cation, small-incision cataract surgery, or extra-capsular cataract extraction), identity of the operative eye, presence of intraoperative or perioperative complications, and reason for postoperative uncorrected visual acuity of 6/60 or worse in the operative eye, if relevant. All patients were instructed to return for a fi nal examination 40 days after surgery, with additional visits in the intervening period at the discretion of the centre.

Final follow-up examination
Final follow-up examinations were completed on all patients who returned without further prompting to the clinic 40 or more days after surgery. 40 days after enrolment of the fi nal patient, clinic staff could use methods such as telephone calls and transport subsidies to encourage unexamined patients to return. 3 months after enrolment of the fi nal patient, clinic staff began home visits for unexamined patients, with a target of examining at least 90% of enrolees. Clinics maintained a log to record whether each patient had returned without further prompting, had returned after a study intervention to promote follow-up, or was examined at home. The fi nal examination included pupil dilation by an ophthalmologist. Data recorded included date and location of examination, examiner's identity (the operating surgeon was not allowed to do the examination), identity of operative eye, uncorrected and best-corrected visual acuity (assessed with the same chart as was used in previous examinations, irrespective of location), presence of postoperative complications, and reason for uncorrected visual acuity of 6/60 or worse in the operative eye, if relevant.

Statistical analysis
All visual acuity data were log-transformed. We used Spearman's rank correlation coeffi cient (r s ) to assess the correlation between patients' early and late postoperative (fi nal follow-up) uncorrected visual acuity measurements. Visual outcome for each hospital was defi ned as the proportion of patients with uncorrected visual acuity of 6/18 or better (ie, good surgical outcome by WHO standards 18 ) minus the proportion with uncorrected visual acuity of 6/60 of worse (blindness, as defi ned in the USA and some other countries). 23 The visual outcome based on assessment of uncorrected visual acuity at fi nal followup for all patients at a hospital was treated as the standard, to which the visual outcomes based on early assessment of all patients and fi nal follow-up assessment of only the patients who returned without additional prompting were compared by use of Spearman's rank correlation coeffi cient. We classifi ed hospitals on the basis of visual outcome into high (top 25%), middle (middle 50%), and low (bottom 25%) performance, and compared the proportion of hospitals that ranked in the same category when the standard data (fi nal follow-up for all patients) were used and when data from either of the two strategies under investigation (early assessment and late assessment of only those who returned unprompted) were used. All statistical analyses were done with Stata 11.0 (StataCorp, College Station, TX, USA).

Role of the funding source
None of the funders had any role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had fi nal responsibility for the decision to submit for publication.   Completion of fi nal follow-up examination was greater than 90% for all three world regions and for all countries apart from Indonesia (86% [189/221]). Overall, 1831 (51%) of 3581 patients whose mode of follow-up was recorded returned to the clinic for the fi nal examination without additional study-related prompting or inducement (table 2). Median time between hospital discharge and fi nal examination was 98 days (IQR 71-171).
Most patients underwent small-incision cataract surgery, with the rest undergoing either phacoemulsifi cation surgery or extra-capsular cataract extraction (table 2). Most patients were blind (uncorrected visual acuity of 6/60 or worse) in the operative eye before surgery, and most had substantial improvements in visual acuity at the early postoperative examination and at the fi nal follow-up examination (table 2).
We calculated the proportion of patients with uncorrected visual acuity of 6/18 or better minus the proportion with uncorrected visual acuity of 6/60 or worse (visual outcome) for each hospital, with results from the fi nal follow-up assessment for all patients taken as the standard. Visual outcome from late (fi nal followup) postoperative examination for all patients was highly correlated with that from early postoperative examination for all patients (r s =0·74, p<0·0001; fi gure 1) and that from late postoperative examination of those who returned unprompted only (r s =0·86, p<0·0001; fi gure 2). The correlation between visual outcome from late postoperative assessment of all patients and of those who returned unprompted only persisted when only hospitals with unprompted follow-up rates of less than 50% (n=17; r s =0·71, p= 0·002) or less than 30% (n=10; r s =0·63, p=0·05) were included in the analysis. Table 3 shows the 25th, 50th, 75th, and 90th centiles for visual outcome, and the proportion of patients with good uncorrected visual acuity and poor uncorrected visual acuity (blindness), for the diff erent assessment strategies. Among low-ranking hospitals at the early examination, the proportion with poor visual acuity was similar to that with good visual acuity, which means that the visual outcome (diff erence) was small or even negative (fi gure 1). Although early and late visual acuity are strongly correlated at both patient and hospital levels, substantial improvement in vision still occurred between the time of the early and late postoperative assessments. We classifi ed hospitals by visual outcome as high (top 25%), middle (middle 50%), and low (bottom 25%) performance, with classifi cation by visual outcome based on late (≥40 days postoperatively) uncorrected visual acuity for all patients as the standard. Hospital classifi cation based on this standard was the same as that based on early postoperative assessment in 27 of 40 hospitals (68%), and was the same as that based on late assessment of only those who returned unprompted in 31 of 37 hospitals (84%) (table 4). When only hospitals with unprompted follow-up rates of less than 50% were included in this comparison, 11 of 17 were in the same classifi cation as the standard; the fi gure for hospitals with less than 30% follow-up was seven of ten.
Adjustment of visual outcome at early postoperative assessment for patient factors (age, sex, surgery type, and presence of surgical complications) and hospital characteristics (annual surgical volume, private vs public, and urban vs rural) did not further improve the hospital-level correlation between visual outcome at early assessment and late assessment for all patients (data not shown).

Discussion
Training programmes are urgently needed to reduce the burden of unoperated cataract, and such programmes depend on the ability to assess quality of surgical outcomes. This ability in turn depends on easy-to-use and reliable metrics. Existing standards 18 are neither evidencebased nor validated in settings with low postoperative follow-up. Common practice has been to assess patients' visual acuity at their fi nal follow-up visit, whenever this visit takes place. Since the time between surgery and the fi nal visit can vary greatly between patients, acuity is recorded at diff erent times in the ocular healing process, which increases variability in the outcome.
We have shown that two alternative strategies, early postoperative assessment of all patients and later assessment of only those who return unprompted to the clinic, provide a suffi ciently valid measure of hospital performance to be useful to programme planners (panel). Although some hospitals might fi nd it simpler in practice to measure visual acuity at the time of discharge on all patients, other centres where follow-up rates are not very low might prefer to assess later visual acuity on those who return without prompting, in view of the slightly greater accuracy of this method.
Classifi cation with these measures might distinguish between hospitals capable of serving as training centres (high performance) and those in acute need of further training (low performance), and in most cases will be representative of a hospital's actual performance record based on fi nal visual acuity for all patients (defi nitive data for which are not obtainable without the substantial, resource-intensive eff ort made in this research context). We chose to use an unconventional outcome measure to include the proportion of patients with both good and poor visual acuity after surgery, because of the equal importance of achieving good outcomes and avoiding bad ones. When only the proportion of patients with visual acuity of 6/18 or better was used as the outcome measure, our results were substantially the same (data not shown). In view of our results, we suggest that WHO 25th centile 50th centile 75th centile 90th centile

Late postoperative assessment of all patients
Proportion with visual acuity of 6/18 or better 51% 64% 75% 83% Proportion with visual acuity of 6/60 or worse 13% 6% 4% 1% Visual outcome 39% 57% 71% 78% Early postoperative assessment was done within 3 days after surgery; late postoperative assessment was done 40 or more days after surgery. The visual outcome was defi ned as the proportion of patients with visual acuity of 6/18 or better minus the proportion with visual acuity of 6/60 or worse at each hospital. *Those who returned unprompted are patients who returned for follow-up assessment after 40 or more days as instructed, without additional study-related prompting or inducement.  targets for cataract surgery be modifi ed to include evidence-based measures, including specifi c targets for early assessment of visual acuity, potentially based on the data presented here. Extra-capsular cataract extraction surgery, which uses a larger wound than newer surgical techniques and requires sutures, is generally expected to require more time for healing and visual recovery than more modern surgical methods. As our cohort shows, this technique is still widely practised in some regions. That early bestcorrected visual acuity correlates equally well with later best-corrected vision in these cases as in those with more modern surgery in which smaller wounds are used (small-incision cataract surgery and phacoemulsifi cation) is therefore encouraging. Re fraction (measurement of the strength of corrective lenses needed for optimum visual acuity) improved the correlation between early and late visual acuity for extra-capsular cataract extraction. However, refraction did not improve the correlation between early and late visual acuity for our study cohort as a whole, the large majority of whom underwent other types of surgery. The automated devices needed for refraction are expensive, and the training needed to refract accurately without complex machinery is substantial.
Strengths of our study include the geographical breadth and range of participating centres with respect to various characteristics. High follow-up rates were achieved in all of the regions studied with the use of home visits, meaning that our estimates of fi nal follow-up results for all patients are unlikely to be biased. Although a few hospitals had high rates of unprompted return for followup, the fact that overall only half of patients returned unprompted is consistent with previous work, 16 and emphasises the need for strategies to measure cataract visual outcomes in settings with poor follow-up.
Our study also has several limitations. Large studies typically standardise eye charts and testing distances between centres, often with charts designed for research. 27 We believed that allowing hospitals to use equipment similar to that used in their usual practice was important, although the vision measurement protocol was carefully standardised. Diff erences in charts might have increased variability in our measurements. Systematic audits of visual acuity and other data reported by hospitals were not practical to undertake. Thus, we cannot exclude the possibility of inaccurate or even deliberately falsifi ed data, although we attempted to prevent such occurrences. Inaccuracies were minimised by onsite training and careful feedback to hospitals about issues encountered early in the study. The risk of data falsifi cation was reduced by prohibiting surgeons from assessing the visual acuity of their own patients. Although we cannot rule out the possibility of exaggerated good results, only fi ve of the 40 hospitals in the study achieved the WHO target of 80% late uncorrected visual acuity of 6/18 or better.
Although outcomes from both early assessment of all patients and late assessment of only those who returned unprompted are well correlated with those from late assessment of all patients, these correlations are not perfect-a fact that must be understood and accepted by users of these data. We attempted to improve the hospital-level association between visual outcomes based on early assessment and on late assessment for all patients by adjusting for hospital and patient factors, but were unable to do so.
Finally, although the range of participating hospitals was wide, it does not represent a random sample of the institutions that do cataract surgery in developing countries. Many hospitals had ties with international development-focused non-governmental organisations, and doctors might have received additional training with support from these organisations. However, since only a small proportion of hospitals reached the WHO outcome standard, there is no clear evidence to suggest that ours was an unusually strong cohort of centres.
Despite its limitations, our study validates practical new strategies to assess the quality of cataract surgery, usable in settings where follow-up rates are low, extra-capsular

Systematic review
We searched PubMed on April 11, 2013, using the terms "cataract surgery" and "early", cross-indexed with "vision", "visual acuity", "outcome", and "result" for articles published in any language since Jan 1, 1970. Few previous studies have sought to examine the correlation between early and later postoperative visual acuity after cataract surgery. Investigators of a small study of four centres in Indonesia 22 (two of which were also participants in our study) reported that even with large-incision surgery, early assessment of postoperative visual acuity was representative of fi nal vision. Osher and colleagues 24 noted uncorrected visual acuity of 6/12 or better at both 1 day and 5 weeks postoperatively in more than 95% (n=98 at 1 day, n=97 at 5 weeks) of 100 consecutive best-case scenario eyes that underwent phacoemulsifi cation. Other reports 19,25,26 have suggested that refractive power measured as early as 30 min after modern cataract surgery is well correlated with later refractive power. However, only the study by Briesen and colleagues 26 was concerned with assessment of surgical quality in settings with poor follow-up, and the investigators did not assess the usefulness of early visual acuity measurement. Limburg and colleagues 16 reported a low correlation between early and late postoperative visual acuity after cataract surgery, perhaps because of diff erences in follow-up timing between centres, and possible data entry problems. Patients who return without additional prompting or encouragement to the clinic after cataract surgery might be expected to represent a biased sample (eg, they might be more likely to have symptomatic surgical complications that aff ect visual acuity than are patients who do not return). Although few studies have addressed this question, the investigators of the study in Indonesia 22 noted that the visual acuity of returning patients could be representative of the entire operated cohort.

Interpretation
Our results validate the use of early assessment of all patients and of later assessment of only those patients who return for follow-up to measure the quality of cataract surgery in settings where follow-up is poor. Which of these approaches hospitals will prefer will probably depend on specifi c circumstance: measuring visual acuity at hospital discharge might be logistically easier in some settings, although later assessment, as long as patient return rates are reasonable, might be somewhat more accurate.
cataract extraction is still practised, and refraction is impractical. This ability to measure quality is an essential requirement for programmes designed to reduce the burden of the world's most common cause of blindness.