Introduction

Cataract surgery is one of the most performed surgical procedures in the United Kingdom. The recent British Ophthalmological Surveillance Unit study reported an overall UK postcataract surgery endophthalmitis incidence rate of 0.14% or 1 in 700 operations,1 which is very similar to the rate found in a systematic review of 215 published studies.2 Although this condition is rare, it has a poor prognosis, with only 53% of cases achieving a final visual acuity of 20/40 or better.3

The rate of postcataract surgery endophthalmitis may be consistent for a period of time. However, the exact number is not expected to be identical every month but varies around a long-term mean value. The variation is inherent in the process itself, which is reasonably stable and likely to be predictable within certain limits. With a special cause or ‘unnatural’ variation, one or more points vary in an unpredictable manner. This is a ‘signal’ that the process has changed. Various methods such as the Monte-Carlo4 and binomial theorem probability5 have been described to detect these ‘signals’ or what we more commonly described as early outbreaks.

The control chart is a method that has been used widely in healthcare at operational, clinical, strategic, and public policy level6, 7, 8, 9, 10 to effectively detect unusual variations in a system. We feel that this method has a useful role to detect variations in the rates of endophthalmitis and early outbreaks in postcataract surgery endophthalmitis.

The control chart has control limits and certain guidelines to detect unusual variations in a system. Any point that falls outside the upper and lower control limits is deemed unusual. These limits are usually set at 3 standard deviations (SD) However, for a situation with unacceptably serious outcome, the control limit may be tightened to 2 SD.11 The obvious disadvantage of this action is an increase in false-positive rate. The probability of a point falling beyond either control limit set at this level by chance is approximately 0.05. Other guidelines recommended11 for suggesting an unusual variation in a control chart are as follows:

  1. 1)

    2 out of 3 consecutive points between the control limit and one SD below this level.

  2. 2)

    A run. This occurs when seven or more consecutive points fall on either side of the centre line.

  3. 3)

    A trend. When six consecutive points move steadily upwards or downwards.

We used this approach to retrospectively analyse our departmental postcataract surgery endophthalmitis over a 10-year study period. We also discussed and compared the different possible statistical methods available at present.

Materials and methods

All cataract extractions from 01/07/97 to 30//06/07 (10 years) at the Royal Gwent Hospital were analysed. Patients who underwent combined procedures (eg, trabeculectomy and penetrating keratoplasty) were excluded. Patient details were obtained from theatre logbooks. Patients' records were requested for those diagnosed with postoperative endophthalmitis.

Acute-onset endophthalmitis was defined as any patient with a clinical diagnosis of presumed infectious endophthalmitis occurring within 6 weeks of cataract surgery.7

The annual incidence for each year was calculated. The control chart method was used to analyse the data to investigate the pattern of unusual variation and the possibility of an outbreak retrospectively.

Results

During the study period, a total of 21 032 cataract operations were performed in the Royal Gwent Hospital. There were 35 cases of acute-onset endophthalmitis. There were 21 female and 14 male patients. The mean age was 79.7 years (range: 69–93). The annual incidences (1 July to 30 June) of postcataract surgery endophthalmitis during the study period were 0.17, 0.16, 0.21, 0.15, 0.19, 0.40, 0.09, 0.05, 0.10, and 0.08% respectively. The mean annual incidence over the 10-year period was 0.16%.

A line graph was plotted for the number of non-infected cases between endophthalmitis cases against each endophthalmitis case (see Figure 1). There was only 34 points for 35 cases of endophthalmitis since the first entry for the number of non-infected cases was taken after the first endophthalmitis case. We used the control chart method to detect a rare event and set the control limits accordingly (see Appendix for control limit calculation).

Figure 1
figure 1

Control chart for the number of non-infected eyes between each case of endophthalmitis.

Full size image

The points at which an outbreak occurred need to be compared with a stable baseline mean before the outbreak. In our case series, we suspected that there may be unusual variations and clustering between points 18–26. Therefore, the baseline mean was calculated from the data before this point, that is, from point 1–17. The mean was 588.4. The lower control limit set at 2 SD, which was at 108.2 of cases. The criteria aforementioned were applied to test point 18–26. The first point that fell below the control limit was 24. Two out of three consecutive points (points 18 and 19) fell between the first and second SD, suggesting an unusual variation in the system. There was also seven consecutive points (points 18–24) below the mean baseline, which was another indicator of an unusual variation.

However, the decision to suspend operating and start an investigation was not instigated until point 26. This was noted after five cases of endophthalmitis that occurred within 3 months in the beginning of 2003. Operating was suspended and a multidisciplinary team investigated the outbreak. No specific cause was found despite extensive investigations, but additional steps were taken to ensure sterility. Povidine iodine in conjunctival sac was also introduced. The interval between point 24 (the first point below the control limit) and point 26 was 20 days.

After the additional steps have been applied, a new mean was calculated from point 27–34. This was 1154.6 of non-infected cases between endophthalmitis cases (see Appendix). Compared with the mean of 195.2 from point 18–26, there is a clinically and statistically significant improvement (P=0.0006).

Discussion

The beginning of an outbreak of postcataract surgery endophthalmitis is not always easily identifiable. It may be tempting to classify a few closely presented cases of endophthalmitis as an epidemic, which may result in expensive investigation and suspension of operating theatre. Conversely, these cases could be passed off as a normal variation, delaying investigations that could have prevented subsequent cases.

The purpose of this study was to use the control chart to improve the monitoring of an endophthalmitis outbreak. As postcataract surgery endophthalmitis is usually a rare event, this method of plotting a control chart of non-infective cases between endophthalmitis cases is preferred to plotting the number of infective cases or endophthalmitis incidence against time as mentioned by Anuradha and Feyi-Waboso.12 The conventional control chart recommendation is to set the lower control limit at 3 SD. This may be acceptable when dealing with fairly non-serious outcome, for example, the number of days it takes to mail the patient an outpatient appointment; however, when patients' sight are involved, one could argue not to wait until a data point exceeds 3 SD before taking action. Therefore, a case can be made for using 2 SD as the lower control limit in this situation as in certain chemical industries that could not afford risking much variation in their production line. However, this will lead to an increase in false-positive rate and should be taken into consideration.13 We are aware that the control limits of the 2 SD may be an arbitrary value for endophthalmitis-outbreak monitoring. The right balance needs to be struck between sensitivity and specificity, and this process could only be obtained in a rigorous statistical analysis of large pools of data.

For the purpose of endophthalmitis-outbreak monitoring, the first two guidelines for control chart are more useful in detecting an early outbreak, as fewer points are required. However, the guideline for a ‘run’ could be altered to an appropriate level on the basis of previous data, for example, five rather seven consecutive points fall on the same side of the centerline. Similarly, detection of a ‘trend’ could be tightened accordingly. The latter two criteria are, however, useful to delineate the ‘drift’ or tendency in the system.

A control chart should have about 16–20 data points to apply the four guidelines described above.11, 14 With fewer data entry points, there is an increased danger of missing special causes (type-II error). Conversely, with more than 35 data entry points, there is increasing likelihood of finding special causes due to chance (type-I error). The most common special causes in these instances would be from a ‘trend’, and the points outside the control limits that tend to regress towards a mean. One way to avoid this is by sampling less frequently. For example, if there is a new case of endophthalmitis, the control chart could be used over the most recent 16–20 cases to detect a special cause. However, this entails good book-keeping of not just the number of endophthalmitis cases but the number of non-infected cases between infected ones.

Other methods of identification of an outbreak that have been described are the Monte-Carlo4 and binomial theorem probability.5 The main advantages of the control chart over these methods are that it allows the data to be analysed over a recent period of time and the control limits set according to recent performance. The 2 SD control limits are by no means rigid and we recommend individualization of the thresholds and control chart guidelines accordingly to the departmental record. However, this could only be achieved with frequent usage of the control chart through time and experience. The control chart also enables data in one period of time to be compared with another. The graphical presentation allows large amount of data to be easily understood without understanding complex statistics. Improvement trends are also notable on the control chart.

The main drawback of the control chart method in accurately detecting an unusual variation in rare events like endophthalmitis is that the system would need to have at least a ‘background’ mean to start off with. Ideally, to achieve this, it needs to meet the minimal data points (16 cases of endophthalmitis in the system). Other statistical methods may be more suitable for a newly established department.

The control chart is potentially a sensitive method to detect an outbreak of endophthalmitis. Its control limits are dynamic and factor in recent performance to detect an early outbreak of postcataract surgery endophthalmitis. We advocate that appropriate measures or investigations should be performed when certain signals emerged from a control chart. Further studies are required to identify a more accurate level of control limit appropriate for the identification of an outbreak or unusual variation.