Diagnostic accuracy of electronic surveillance tool for catheter-associated urinary tract infections in tertiary care hospitals

Abstract Background: Automated systems have been developed to reduce labor-intensive manual recordings during nosocomial infection surveillance. The diagnostic accuracies of these systems have differed in various settings. Methods: We designed this meta-analysis to evaluate the diagnostic accuracy of an electronic surveillance tool for catheter-associated urinary tract infections (CAUTIs) in tertiary care hospitals. We systematically searched databases such as Medline, Scopus, Cochrane library and Embase (from inception until November 2019) for relevant studies. We assessed the quality of trials using the diagnostic accuracy studies-2 tool, and performed a meta-analysis to obtain a pooled sensitivity and specificity for electronic surveillance. We included 6 studies with 16,492 patients in the analysis. Results: We found a pooled sensitivity of electronic diagnostic surveillance for CAUTIs of 97.5% (95% confidence interval [CI], 67.6–99.9%) and a pooled specificity of 92.6% (95% CI, 55.2–99.2%). The diagnostic odds ratio was 494 (95% CI, 89–2747). The positive likelihood ratio was 13.1 (95% CI, 1.63–105.8) and the negative likelihood ratio 0.02 (95% CI, 0.001–0.40). A bivariate box plot indicated the possibility of heterogeneity between the included studies. Conclusion: Our review suggests that electronic surveillance is useful for diagnosing CAUTIs among hospitalized patients in tertiary care hospitals due to its high sensitivity and specificity.


Introduction
The Centre for Disease Control and Prevention classified healthcare-associated infections (HAI) into 4 major types: pneumonia, surgical site infections, urinary tract infections (UTIs), and blood-stream infections. [1] However, UTIs are considered the most common form of nosocomial infection accounting for about 40% of the cases. [2] More than 80% of the UTIs are associated with urinary catheterization. [3] Almost onefourth of all hospitalized patients undergo urinary catheterization; and the majority (90%) of catheterized patients belong to a high-risk trauma class. [4] Hence, following best practices for catheterization is essential to prevent catheter-associated urinary tract infections (CAUTIs) and reduce the caseload of nosocomial infections among these patients in tertiary care hospitals. However, many healthcare providers forget to check whether their patients have a urinary catheter in place.
Routing monitoring of variables differs based on the HAI type. According to the Centre for Disease Control and Prevention guidelines for healthcare-associated urinary tract infections, healthcare professionals should comprehensively monitor the variables related to patients' signs, symptoms, urinary routine and culture tests, blood tests, antibiotic use, presence of invasive devices, and clinical or radiological evidence of infection during hospitalization. [1] Hence, surveillance should involve bedside investigation procedures and review of medical records including care charts prepared by nurses, laboratory reports, treatment charts, radiographic examination findings, and healthcare records [5] ; but collecting and analyzing all this information is time-consuming and costly.
Few hospitals have an established surveillance system for monitoring CAUTIs. [3] Medical and health charts recorded manually by nurse practitioners communicate vital information about the patients to other members of the healthcare team. Automated nosocomial infection surveillance systems (including some for ventilator-associated pneumonia, central line-associated bloodstream infections, and device associated denominators) have been developed to reduce the labor-intensive manual recording. [6][7][8][9][10] The diagnostic accuracies of electronic surveillance systems have been investigated in various settings with different results. However, no systematic efforts have evaluated the diagnostic accuracy of these systems. We designed this meta-analysis to evaluate the diagnostic accuracy of an electronic surveillance tool for CAUTIs in tertiary care hospitals.

Material and methods
2.1. Inclusion criteria 2.1.1. Type of studies. We included all full-text articles examining the diagnostic accuracy of electronic surveillance tools for CAUTIs in tertiary care hospitals irrespective of their study design. The studies reported sensitivity and specificity or provided data to calculate their values. We excluded unpublished data, case reports, and studies with sample sizes smaller than 10. The ethical approval was not necessary for meta-analysis.

Search strategy
We performed an extensive and systematic electronic search in databases such as Medline, Scopus, Cochrane library, and Embase using medical subject headings and free-text terms like "Validation Studies," "Urinary Tract Infections," "Urinary Catheterization," "Catheter-Associated Urinary Tract Infections," "Nosocomial Infections," "Electronic Surveillance," "Nursing Records," "Sensitivity," "Specificity," "Diagnosis," "Manual Records," and "Diagnostic Accuracy Studies." The articles in the search were those from inception until November 2019 without language restrictions. We hand-searched a reference list of primary trials to find relevant articles to include in our review.

Selection of studies
Two authors independently performed the primary screening of titles, keywords and abstracts, and retrieved relevant full-text articles. The same authors then independently performed a secondary screening of the retrieved articles and identified the studies satisfying the inclusion criteria. Disagreements during the selection of studies were resolved through consensus.

Data extraction and management
The primary investigator extracted data to obtain the relevant variables from the studies. The extracted data included: study setting and design, inclusion and exclusion criteria, reference standards, index test, total participants, comorbidities, mean age, sensitivity, and specificity. We manually entered data into the STATA software (StataCorp., CollegeStation, TX). Both authors double-checked data to ensure correct entries by comparing data in the review and in the study reports.

Risk of bias assessment in included studies
Two authors independently evaluated the risk of bias using the quality assessment of diagnostic accuracy studies-2 tool. [11] The quality assessment of diagnostic accuracy studies-2 tool consists of the following domains assessed for risk of bias and applicability: patient selection, index test, reference standard, and time interval, that is, (flow and timing) of the outcome assessments. We graded the studies as low, high, or unclear based on the presence of any bias among them.

Statistical analysis
We performed the meta-analysis using the STATA 14.2 software (StataCorp.). We applied the bivariate meta-analysis method to obtain the pooled estimate of diagnostic accuracy measures (sensitivity, specificity, PLR, NLR, and DOR) for the electronic surveillance system. We constructed a summary receiver operator characteristic curve (sROC) and produced a 95% predictive ellipse within the ROC space. We also calculated the area under the curve (AUC) in the sROC. The diagnostic value was better the closer the AUC value was to 1.
We generated a forest plot to graphically represent the studyspecific and pooled estimates of sensitivity and specificity. We assessed heterogeneity graphically using a bivariate boxplot. We could not explore potential sources of heterogeneity with subgroup analysis or meta-regression as there were <10 studies included in the review. We used the Metandi command package for analyses.

Selection of studies
Or systematic search for studies reporting the diagnostic accuracy of electronic surveillance systems for CAUTIs yielded 1773 records (673 from Medline, 547 from Scopus, 412 from Embase, and 141 from the Cochrane library). After the first screening stage (title, abstract and keywords), we retrieved 122 relevant studies. We assessed the full texts of these studies for eligibility criteria, and identified 4 potential studies from the reference list of the primary articles. Finally, we included 6 studies satisfying the inclusion criteria with 16,492 participants (Fig. 1). [12][13][14][15][16][17] 3.2. Characteristics of the studies included Table 1 summarizes the characteristics of the included studies. Half of the studies were prospective and half retrospective. Five were conducted in the United States of America and one in Taiwan (Lo et al [16] ). The age of the participants ranged from 55 to 72.5 years. The sample sizes of the studies varied from 175 to 11,251. All the studies had index tests as the electronic surveillance system and reference standards as manual chart/record reviews by infection preventionists, nurses, or physicians.

Diagnostic performance of electronic surveillance
We found 6 studies evaluating the diagnostic performance of electronic surveillance systems for CAUTIs. Figure 3 shows the forest plot with individual study sensitivities and specificities.    Figure 5 shows the bivariate boxplot testing the heterogeneity between the included studies; 2 out of 6 studies are outside of the circle in the plot indicating the presence of heterogeneity between the studies. Since there were <10 studies, we could not investigate the source of heterogeneity by sub-group analysis or metaregression. We also omitted testing for publication bias due to the low number of studies.

Discussion
CAUTIs are considered preventable, in spite of being the most common nosocomial infection in tertiary care hospitals. [18] Most hospitals do not have a surveillance system to monitor the placement of urinary catheters, the duration of placement, and development of CAUTIs in hospitalized patients. [19] Manual chart reviews by nurses or healthcare professionals are a laborintensive and time-consuming process. [20] Electronic surveillance is considered an effective strategy to prevent different types of HAI including CAUTIs. [21] Thus, we conducted this metaanalysis to evaluate CAUTI surveillance systems and find the pooled estimates for their diagnostic performance among hospitalized patients in tertiary care hospitals. We found a pooled sensitivity and specificity of electronic diagnostic surveillance for CAUTIs of 97.5% and 92.6%, respectively. Since this was the first review and meta-analysis on electronic surveillance for CAUTIs, we could not compare our results to others. However, we compared our findings with those of surveys of electronic surveillance systems for other forms of Table 1 Characteristics of the included studies (n = 6).   nosocomial infections (like those for ventilator-associated pneumonia, [22] surgical site infections, [23,24] and bloodstream infections [25] ). We found that the sensitivity and specificity were highest for surgical site infection surveillance systems followed by those for CAUTIs (our data), and then by surveillance systems for bloodstream infections. Ventilator-associated pneumonia surveillance systems had showed the lowest published sensitivity (even going below the 50% range). Based on these findings, we believe the electronic surveillance systems can be applied for almost all the major HAI in tertiary care hospitals. Our calculated PLR was 13.1 (95% CI, 1.63-105.8) and the NLR was 0.02 (95% CI, 0.001-0.40), suggesting that the CAUTI electronic surveillance system can be used as a confirmatory and exclusion tool. Our bivariate box plot indicated the possibility of heterogeneity between the studies we analyzed. However, we were not able to explore the source of heterogeneity due to the limited number of eligible studies. We also did not perform a publication bias assessment for the same reason.
The strengths of our study include its comprehensive nature, including all 6 eligible studies with 16,492 participants. This is the first review providing a pooled sensitivity and specificity for diagnostic electronic surveillance for CAUTIs. But, we are also aware of the study's limitations. First, some of the studies in our review had a high risk of bias and this may have affected our pooled estimates. Also, we found significant heterogeneity between the studies included in the meta-analysis. This limits the power of the interpretation of the pooled results. The publication bias could not be assessed due to the small number of studies included. Finally, most studies were conducted in the United States of America; and, therefore, the generalizability of our findings is limited.
However, we believe that our study provides valuable insights regarding the diagnostic accuracy of electronic surveillance systems for CAUTIs. We found that electronic surveillance systems provide both high sensitivity and specificity (>90%), and allow for confirmation and exclusion of CAUTIs with utmost certainty. Our findings suggest that automated or electronic surveillance tools with case definitions in line with international guidelines for diagnosis of CAUTIs should be implemented in tertiary care hospitals to monitor the patients with a urinarycatheter to identification and management of the infection. However, further high-quality studies in other geographical regions are needed to ensure applicability throughout the world.

Conclusion
In all, we found that diagnostic electronic surveillance is highly useful for CAUTIs among hospitalized patients due to its high sensitivity and specificity. Our results suggest that this surveillance modality can be used for CAUTI screenings in tertiary care hospitals as it is efficient and time-saving.

Author contributions
Yuehua Shen designed the project; Yuehua Shen and Hongmei Cui were involved in data collection and data analysis; Yuehua Shen prepared the manuscript; Hongmei Cui edits the manuscript; all authors read and approved the final manuscript.   Shen and Cui Medicine (2021) 100:39 www.md-journal.com