Published online Jan 02, 2019.
https://doi.org/10.3346/jkms.2019.34.e16
Diagnostic Test Accuracy Review of Cytology for Squamous Intraepithelial Lesion and Squamous Cell Carcinoma of Uterine Cervix
Abstract
Background
Even though cervico-vaginal smears have been used as a primary screening test for cervical carcinoma, the diagnostic accuracy has been controversial. The present study aimed to evaluate the diagnostic accuracy of cytology for squamous intraepithelial lesion (SIL) and squamous cell carcinoma (SqCC) of the uterine cervix through a diagnostic test accuracy (DTA) review.
Methods
A DTA review was performed using 38 eligible studies that showed concordance between cytology and histology. In the DTA review, sensitivity, specificity, diagnostic odds ratio (OR), and the area under the curve (AUC) on the summary receiver operating characteristic (SROC) curve were calculated.
Results
In the comparison between abnormal cytology and histology, the pooled sensitivity and specificity were 93.9% (95% confidence interval [CI], 93.7%–94.1%) and 77.6% (95% CI, 77.4–77.8%), respectively. The diagnostic OR and AUC on the SROC curve were 8.90 (95% CI, 5.57–14.23) and 0.8148, respectively. High-grade squamous intraepithelial lesion (HSIL) cytology had a higher sensitivity (97.6%; 95% CI, 94.7%–97.8%) for predicting HSIL or worse histology. In the comparison between SqCC identified on cytology and on histological analysis, the pooled sensitivity and specificity, diagnostic OR, and AUC were 92.7% (95% CI, 87.3%–96.3%), 87.5% (95% CI, 87.2%–87.8%), 865.81 (95% CI, 68.61–10,925.12), and 0.9855, respectively. Geographic locations with well-organized screening programs had higher sensitivity than areas with insufficient screening programs.
Conclusion
These results indicate that cytology had a higher sensitivity and specificity for detecting SIL and SqCC of the uterine cervix during primary screening.
Graphical Abstract
INTRODUCTION
A cervico-vaginal smear, including the conventional smear and liquid-based cytology, is a simple and inexpensive test for the prediction of squamous intraepithelial lesion (SIL) or squamous cell carcinoma (SqCC) of the uterine cervix.1 These tests have contributed to a decrease in the incidence of cervical cancer, especially in geographic areas supported by well-organized screening programs.1 Although several studies have reported on the diagnostic accuracy of the cervico-vaginal smear, results showed a wide range of estimated sensitivity compared to the specificity.1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38 Because the diagnostic accuracy can be affected by variable factors, including study time, geographic area, and population,1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38 it should be fully elucidated based on these standardized parameters, including the diagnostic grades of cytology. We tried to establish the universally acceptable value beyond the limitations of individual studies. A diagnostic test accuracy (DTA) review should be performed to confirm the cytology test outcomes of the uterine cervix.
To evaluate the diagnostic accuracy of cytology, the concordance rates between cytology and histology of the uterine cervix were investigated. In addition, the present study aimed to evaluate the diagnostic accuracy of cytology for SIL and SqCC of the uterine cervix through DTA review. A subgroup analysis based on the number of patients and study location was also conducted.
METHODS
Published study search and selection criteria
Relevant articles were obtained by searching the PubMed databases through January 31, 2018. There was no time limit for the start. These databases were searched using the following key words: ‘(Uterine Cervical Neoplasms OR Uterine Cervical Dysplasia OR Cervical Intraepithelial Neoplasia OR ((cervix OR cervical OR cervico*) AND (cancer* OR carcinoma OR adenocarcinoma OR neoplas* OR dysplas* OR dyskaryos*)) OR (CIN OR CINII* OR CIN2* OR CINIII* OR CIN3*) AND (SIL OR HSIL OR H-SIL OR LSIL OR L-SIL OR ASCUS OR ASC-US).’ The titles and the abstracts of all searched articles were screened for exclusion. Review articles, including the previous meta-analysis, were also screened to obtain additional eligible studies. Search results were then reviewed and articles were included if the study investigated the uterine cervix and there was information regarding the concordance between cytology and histology. The articles were excluded when they were case reports or non-original articles or non-English language publications.
Data extraction
Data from all eligible studies1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38 were extracted by two independent authors. Extracted data included the following: first author's name, year of publication, study location, dates of the research, methodology of cytologic examination, and number of patients analyzed. For the meta-analysis, we extracted all data associated with the concordance between cytology and histology in various categories of comparison.
Statistical analyses
The review of DTA was performed using the Meta-Disc program (version 1.4; Unit of Clinical Biostatics, the Ramon y Cajal Hospital, Madrid, Spain). In order to calculate the pooled sensitivity and specificity, individual data were collected from each eligible study in various categories of comparison. The summary receiver operating characteristic (SROC) curve was initially constructed by plotting ‘sensitivity’ and ‘1-specificity’ of each study, and the curve fitting was performed through linear regression using the Littenberg and Moses linear model. Because the data were heterogeneous owing to differences in various methodology and populations, the accuracy data were pooled by fitting a SROC curve and measuring the value of the area under the curve (AUC). An AUC close to 1 indicates a strong test and an AUC close to 0.5 is considered as a poor test. In addition, the diagnostic odds ratio (OR) was calculated by the Meta-Disc program. The estimated values were those that predict abnormal histology of abnormal cytology. In addition, the estimated values of cytologic low-grade squamous intraepithelial lesion (LSIL), high-grade squamous intraepithelial lesion (HSIL), and SqCC were predicted to histologic LSIL, HSIL or worse, and SqCC. To obtain the detailed information, a subgroup analysis based on number of patients, was conducted.
To obtain the results of concordance between abnormal cytology and histology, the Comprehensive Meta-Analysis software package was used (Biostat, Englewood, NJ, USA). The concordance was measured by agreement rates between HSIL identified with cytology and histology and between SqCC identified with cytology and histology. Because the eligible studies used various cytologic methods, including conventional and liquid-based preparations, in various populations, a random-effects model was more suitable than a fixed-effects model. Heterogeneity between the studies was checked using the Q and I2 statistics and presented using P values. To assess publication bias, Begg's funnel plot and Egger's test were used. The results were considered statistically significant at P < 0.05.
RESULTS
Selection and characteristics
A total of 3,314 reports were searched and screened in the database. Due to insufficient information on concordance, 3,155 reports were excluded. An additional 48 reports were excluded owing to results reported on other diseases, 45 were excluded because they were non-original, and 28 articles were excluded because they were written in non-English language. Finally, 38 studies were included in the present analysis (Fig. 1 and Table 1), which provided data on 302,148 patients. Information on the concordance between abnormal cytology and histology test results is shown in Table 1.
Fig. 1
Flow chart of study search and selection methods.
Table 1
Main characteristics of the eligible studies
DTA review of cytology
A DTA review was conducted to elucidate the diagnostic accuracy using cytology in uterine cervix. In the comparison between abnormal cytology and histology, the pooled sensitivity and specificity values were 93.9% (95% confidence interval [CI], 93.7%–94.1%) and 77.6% (95% CI, 77.4%–77.8%), respectively (Fig. 2). The diagnostic OR and AUC on the SROC curve were 8.90 (95% CI, 5.57–14.23) and 0.8112, respectively (Fig. 3). A subgroup analysis based on the number of included patients of each eligible study (≥ 1,000 and < 1,000) and study locations (areas with well-organized versus insufficient screening programs) was conducted. In the subgroup that included the larger number of patients, the pooled sensitivity and specificity, diagnostic OR and AUC on the SROC curve were 94.9% (95% CI, 94.8%–95.1%), 77.8% (95% CI, 77.5%–78.0%), 22.91 (95% CI, 10.70–49.04), and 0.8963, respectively. However, the pooled sensitivity and specificity of the subgroup with a smaller number of patients was 71.1% (95% CI, 69.3%–72.9%) and 73.6% (95% CI, 72.2%–75.0%), respectively. Next, in the subgroup analysis based on study location, areas with well-organized screening programs had a higher sensitivity than areas with insufficient screening programs (94.9% vs. 71.1%).
Fig. 2
The forest plots for the sensitivity and specificity of abnormal cytology in predicting SIL or SqCC in uterine cervix. (A) Sensitivity. (B) Specificity.
SIL = squamous intraepithelial lesion, SqCC = squamous cell carcinoma, CI = confidence interval.
Fig. 3
SROC curve of abnormal cytology in predicting SIL or SqCC in uterine cervix.
SROC = summary receiver operating characteristic, SIL = squamous intraepithelial lesion, SqCC = squamous cell carcinoma, AUC = area under the curve, SE = standard error, Q* = the point where sensitivity and specificity are equal.
In the comparison between LSIL identified with cytology and LSIL identified with histology, the pooled sensitivity and specificity, diagnostic OR, and AUC were 80.5% (95% CI, 78.7%–81.2%), 80.6% (95% CI, 80.2%–81.0%), 11.80 (95% CI, 5.30–26.29), and 0.8339, respectively (Table 2). For predicting HSIL or worse histology, the sensitivity and specificity of LSIL cytology were 97.6% (95% CI, 97.4%–97.8%) and 71.7% (95% CI, 71.3%–72.0%), respectively. The diagnostic OR and AUC were 64.49 (95% CI, 29.04–143.20) and 0.9444, respectively. The pooled sensitivity and specificity, diagnostic OR, and AUC of cytologic SqCC were 92.7% (95% CI, 87.3%–96.3%), 87.5% (95% CI, 87.2%–87.8%), 865.81 (95% CI, 68.61–10,925.12), and 0.9855 for predicting SqCC in histology. In the subgroup analysis, those that used conventional cytology and well-organized screening programs had a higher sensitivity and lower specificity than subgroups that used liquid-based cytology and lacked screening programs.
Table 2
Sensitivity, specificity, diagnostic OR and AUC of SROC curve in cases with histologic confirmation
DISCUSSION
In daily practice, screening tests use cytology and/or the human papillomavirus (HPV) test to predict SIL and SqCC of the uterine cervix. However, it is difficult to obtain information on diagnostic accuracy of cytology and the HPV test from individual studies. Previous studies show that the ranges of sensitivities and specificities of cytology and HPV test varied widely.39 In the eligible studies, sensitivities and specificities of cytology ranged from 22.4% to 99.4% and 0.0% to 99.0%, respectively.1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38 Therefore, it is useful to assess the diagnostic accuracy of a screening test to predict the presence of SIL and SqCC in the uterine cervix by performing a meta-analysis, including a DTA review. To the best of our knowledge, the present study is the first to assess the diagnostic accuracy of cytology for predicting SIL and SqCC in the uterine cervix.
In the present DTA review, regardless of the diagnostic grade of cytology, its diagnostic accuracy was initially evaluated for the prediction of abnormal histology. The sensitivity and specificity of cytology were 93.9% and 77.6%, respectively. In a subgroup analysis based on the number of patients, the larger subgroup showed a higher sensitivity than the smaller subgroup (94.9% vs. 71.1%). Eligible studies with a small number of patients might affect the sensitivity and specificity, since patient cohort sizes ranged from 13 to 50,701. In addition, experiences of cytopathologists and cytotechnologists may be important for the diagnostic accuracy of cytologic examination. Recent automated cytoscreening systems can also be helpful for effective screening. Results of this DTA review show that cytology is a useful screening test in the prediction of SIL or SqCC histology.
In the DTA review for the diagnostic accuracy of cytology, index should be cytology and comparator test should be histology. However, in previous studies, colposcopy was included in the comparator test.39 Cases with negative colposcopic findings were considered as true negative in these studies.39 However, because colposcopy is not a confirmative examination, specificity might be overestimated due to the increase in true negative cases. Therefore, cytology and histology should be compared to properly evaluate the diagnostic accuracy. The present study included only patients with histologic confirmation, but not those who underwent colposcopic examination.
In a previous DTA review, the sensitivity of cytology and HPV test were 65.87%–75.51% and 92.60%–95.13%, respectively.39 However, in this study, cytology was compared between atypical squamous cells of undetermined significance (ASC-US) or worse cytology and HSIL histology. The true positive rate and sensitivity were decreased because patients who underwent LSIL histology were considered false positives in abnormal cytology. The sensitivity of cytology was higher in our study compared to the previous DTA review. Therefore, overestimation of specificity could be possibly considered. In addition, the previous DTA review only included studies that assessed both cytology and HPV tests. The estimated value for diagnostic OR and AUC on SROC, which are useful in comparing various tests, were not shown. In summary, the superiority of the HPV test for accurately diagnosing SIL or SqCC in the uterine cervix cannot be proven in the previous DTA review. In addition, in other DTA review,40 the pooled sensitivity of cytology with HSIL or worse was 79.4% for predicting cancer. However, this review did not show results for other parameters, such as specificity, diagnostic OR, AUC on SROC. The estimated values of overall abnormal cytology and LSIL were not found in the previous review.40
In practice, ASC-US cytology usually requires a repeat smear and/or an HPV test. An ancillary test, such as the HPV test, may be useful because the confirmative information in the repeat smear cannot be obtained. However, the gradient correlation between HPV test and histology is unclear. The advantage of cytology is its ability to predict histologic abnormalities which can help with patient management, compared to that of an HPV test. After a cytologic preparation, HPV tests using the remaining cytologic specimen can be performed. The presence of ASC-US cytology groups, which can increase the false-positive rate and decrease sensitivity. In the previous study, the rate of ASC-US cytology was less than 5.0%.12 However, in the Republic of Korea which has a well-organized screening system, the rate of ASC-US were 0.045% in 432,691 women who had screening tests.1 Therefore, an ancillary HPV test can be more useful in patients with ASC-US cytology. In areas with insufficient screening systems, the effectiveness of a cytologic examination is not fully elucidated. In addition, in areas with a well-organized screening system, the usefulness of an HPV test as the primary screening test is unclear. Primary screening tests should not be selected by simply considering the sensitivity. Availability of screening systems may be important for choosing the screening method to help diagnose SIL or SqCC of the uterine cervix.
In a subanalysis of the ATHENA study, co-testing using cytology and the HPV test has no advantage compared with the HPV test alone.12 However, this study did not enroll patients without an HPV test. This criterion could decrease the sensitivity and true positive cases of cytology. In addition, this report compared ASC-US and worse cytology with HSIL or worse confirmed with histology. Therefore, because sensitivity can differ by patient populations, the diagnostic accuracy of the screening test in the general population can differ between individual studies. The results showed that sensitivity of cytology in our results (96.9%) was higher than that of the HPV test sensitivity for HSIL or worse with histology as shown in Castle's report (88.2%). In addition, in our study, the estimated concordance rates were 93.1% (95% CI, 84.7%–97.1%) and 98.8% (95% CI, 69.0%–100.0%) for HSIL and SqCC cytology, respectively.
There are some limitations in the current DTA review. First, the comparisons between various cytologic abnormalities and histologic abnormalities were conducted in the present DTA review. ASC-US/atypical squamous cells, cannot exclude HSIL (ASC-H) cytology belongs to the heterogeneous diagnostic category. However, the diagnostic accuracy of ASC-H could not be performed due to insufficient information included in the eligible studies. Second, the aim of the present DTA review was to elucidate the diagnostic accuracy of cytology. Thus, the effectiveness between cytology and HPV test was compared with the results of previous reports.12, 39 Third, the number of patients in the individual studies did not apply to exclusion criteria in the present DTA review. The eligible studies with a smaller number of patients showed far from average estimation. However, the effects of studies with a smaller number of patients on overall estimated values were insignificant. Therefore, the diagnostic accuracy of cytology using individual studies with a smaller number of patients should be accurately interpreted. Fourth, histologic examinations include a punch biopsy, loop electrocautery excision procedure, conization, or hysterectomy in the uterine cervix. Sampling error can occur with histologic examinations, such as a punch biopsy. However, in the present DTA review, a detailed evaluation based on histologic methodology could not be conducted due to insufficient information on eligible studies.
In conclusion, our results show that cytology has higher sensitivity and specificity for the prediction of SIL or SqCC, regardless of the diagnostic grade of cytology. The diagnostic accuracy of cytology as a primary screening test was re-confirmed in the present DTA review. Therefore, cytology is one of the most sensitive and confirmative primary screening tests for SIL and SqCC.
Funding:This study was supported by a research grant received from the Korean Society of Cytopathology in 2018.
Disclosure:The authors have no potential conflicts of interest to disclose.
Author Contributions:
Conceptualization: Pyo JS, Kang G.
Data curation: Pyo JS, Yoon HK.
Formal analysis: Pyo JS, Kang G.
Investigation: Pyo JS, Kim HJ.
Methodology: Pyo JS, Kang G.
Writing - original draft: Pyo JS.
Writing - review & editing: Pyo JS, Yoon HK, Kim HJ.
References
-
Belinson JL, Qiao YL, Pretorius RG, Zhang WH, Rong SD, Huang MN, et al. Shanxi Province cervical cancer screening study II: self-sampling for high-risk human papillomavirus compared to direct sampling for human papillomavirus and liquid based cervical cytology. Int J Gynecol Cancer 2003;13(6):819–826.
-
-
Canda MT, Demir N, Sezer O, Doganay L, Ortac R. Clinical results of the liquid-based cervical cytology tool, Liqui-PREP, in comparison with conventional smears for detection of squamous cell abnormalities. Asian Pac J Cancer Prev 2009;10(3):399–402.
-
-
Cárdenas-Turanzas M, Nogueras-Gonzalez GM, Scheurer ME, Adler-Storthz K, Benedet JL, Beck JR, et al. The performance of human papillomavirus high-risk DNA testing in the screening and diagnostic settings. Cancer Epidemiol Biomarkers Prev 2008;17(10):2865–2871.
-
-
Castle PE, Stoler MH, Wright TC Jr, Sharma A, Wright TL, Behrens CM. Performance of carcinogenic human papillomavirus (HPV) testing and HPV16 or HPV18 genotyping for cervical cancer screening of women aged 25 years and older: a subanalysis of the ATHENA study. Lancet Oncol 2011;12(9):880–890.
-
-
Hutchinson ML, Zahniser DJ, Sherman ME, Herrero R, Alfaro M, Bratti MC, et al. Utility of liquid-based cytology for cervical carcinoma screening: results of a population-based study conducted in a region of Costa Rica with a high incidence of cervical carcinoma. Cancer 1999;87(2):48–55.
-
-
Iftner T, Becker S, Neis KJ, Castanon A, Iftner A, Holz B, et al. Head-to-head comparison of the RNA-based aptima human papillomavirus (HPV) assay and the DNA-based hybrid capture 2 HPV test in a routine screening population of women Aged 30 to 60 years in Germany. J Clin Microbiol 2015;53(8):2509–2516.
-
-
Salmerón J, Lazcano-Ponce E, Lorincz A, Hernández M, Hernández P, Leyva A, et al. Comparison of HPV-based assays with Papanicolaou smears for cervical cancer screening in Morelos State, Mexico. Cancer Causes Control 2003;14(6):505–512.
-
-
Wu R, Belinson SE, Du H, Na W, Qu X, Wu R, et al. Human papillomavirus messenger RNA assay for cervical cancer screening: the Shenzhen Cervical Cancer Screening Trial I. Int J Gynecol Cancer 2010;20(8):1411–1414.
-
-
Zhu J, Norman I, Elfgren K, Gaberi V, Hagmar B, Hjerpe A, et al. A comparison of liquid-based cytology and Pap smear as a screening method for cervical cancer. Oncol Rep 2007;18(1):157–160.
-
-
Koliopoulos G, Nyaga VN, Santesso N, Bryant A, Martin-Hirsch PP, Mustafa RA, et al. Cytology versus HPV testing for cervical cancer screening in the general population. Cochrane Database Syst Rev 2017;8:CD008587
-
-
Castanon A, Landy R, Michalopoulos D, Bhudia R, Leaver H, Qiao YL, et al. Systematic review and meta-analysis of individual patient data to assess the sensitivity of cervical cytology for diagnosis of cervical cancer in low- and middle-income countries. J Glob Oncol 2017;3(5):524–538.
-