Elsevier

Gynecologic Oncology

Volume 162, Issue 1, July 2021, Pages 107-112
Gynecologic Oncology

Validation of American College of Radiology Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US): Analysis on 1054 adnexal masses

https://doi.org/10.1016/j.ygyno.2021.04.031Get rights and content

Highlights

  • The ACR O-RADS US has a good diagnostic performance in differentiating malignant from benign adnexal lesions.

  • The ACR O-RADS US is applicable to radiologists with different experience even freshman.

  • Explicit sub-classification into two groups among O-RADS 4 lesions showed better stratification of the intermediate risk.

Abstract

Objective

To assess the diagnostic performance and inter-observer agreement of the American College of Radiology (ACR) Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US).

Methods

From January 2016 to December 2018 a total of 1054 adnexal lesions in 1035 patients with pathologic results from two hospitals were retrospectively included. Each lesion was assigned to an O-RADS US category according to the criteria. Kappa (κ) statistics were applied to assess inter-observer agreement between a less experienced and an expert radiologist.

Results

Of the 1054 adnexal lesions, 750 were benign and 304 were malignant. The malignancy rates of O-RADS 5, O-RADS 4, O-RADS 3, and O-RADS 2 lesions were 89.57%, 34.46%, 1.10%, and 0.45% respectively. Area under the receiver operating characteristic curve was 0.960 (95% CI, 0.947–0.971). The optimal cutoff value for predicting malignancy was >O-RADS 3 with a sensitivity and specificity of 98.7% (95% CI, 0.964–0.996) and 83.2% (95% CI, 0.802–0.858) respectively. When sub-classifying multilocular cysts and smooth solid lesions in O-RADS 4 lesions as O-RADS 4a lesions and the rest cystic lesions with solid components as O-RADS 4b lesions, the malignancy rate were 17.02% and 42.57% respectively, which showed better risk stratification (P < 0.001). The inter-observer agreement between a less-experienced and an expert radiologist of O-RADS categorization was good (κ = 0.714).

Conclusions

The ACR O-RADS US provides effective malignancy risk stratification for adnexal lesions with high reliability for radiologists with different experience. Sub-grouping of O-RADS 4 lesions into two groups facilitated better stratification of the intermediate risk.

Introduction

Preoperative assessment of adnexal lesions is important because the presumed diagnosis determines the management. Ultrasound is routinely used as a sensitive method for evaluating adnexal masses [1]. The International Ovarian Tumor Analysis (IOTA) consensus nomenclature and definitions has improved discrimination of adnexal masses by proposing “Simple Rules” and “ADNEX” models [[2], [3], [4]]. Although the diagnostic performance of these models is high [[5], [6], [7], [8]], their acceptance has been limited in clinical practice in some countries such as the United States. Other scoring systems or mathematical models have also been proposed [[9], [10], [11], [12], [13]], while given that these models have not been validated prospectively, they have not been accepted to introduce into clinical practice internationally.

In 2018, the American College of Radiology (ACR) Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US) committee released the white paper of ovarian-adnexal reporting lexicon for US14, aimed to improve the standardization of describing adnexal masses by incorporating and assessing published commonly recognized ultrasound features. Then, the lexicon was applied to a US risk stratification classification based on the analysis of IOTA phase 1–3 database, consisted of six categories O-RADS 0–5 [4,[15], [16], [17], [18], [19]], defined by prevalence of malignancy based on the IOTA ADNEX model which has been proved as an accurate model to assign an individual risk to every lesion [4,20]. O-RADS 1 category indicate normal or physiologic ovary; O-RADS 2 category refers to lesions that are almost certainly benign with a risk of malignancy <1%; O-RADS 3 category refers to lesions with a low risk of malignancy (1% -10%); O-RADS 4 category refers to lesions with an intermediate risk of malignancy (10% -50%); O-RADS 5 category refers to lesions with a high risk of malignancy defined as ≥50%. The new system includes all types of adnexal lesions and categorizes a malignancy risk, facilitating reliable interpretations and ultimately recommending appropriate management. Nonetheless, the recommended risk of O-RADS 4 lesions is in a wide range between 10% and 50%, which remains indeterminate for these lesions. Moreover, the utilization of this risk stratification system still needs further external validation especially on a broad population beyond European population on which the system was based. More evidence is warranted as well to test the inter-observer variability and the diagnostic performance in less-experienced radiologists since the lesion characterization of O-RADS is apparently based on subjective assessment.

Therefore, the purpose of this study was to evaluate the diagnostic performance of O-RADS US risk stratification and its inter-observer agreement between a less-experienced and an expert radiologist on a retrospective cohort with US detected lesions and known pathological outcomes.

Section snippets

Ethics approval

This retrospective study was reviewed and approved by the participating hospitals review board. The requirement to obtain written informed patient consent was waived.

Study design and setting

This was a retrospective, diagnostic study including patients diagnosed with suspected adnexal mass at a tertiary-care general hospital and a cancer center, from January 2016 to December 2018. The inclusion criteria were patients with primary clinical diagnosis of adnexal mass by transvaginal or transrectal US (for virgin patients)

Patient characteristics

A total of 1054 ovarian and adnexal lesions in 1035 patients who had undergone US examination and biopsy or surgery were included in this study (714 lesions from a tertiary-care general hospital and 340 lesions from a cancer center). 750 (71.2%) lesions were benign and 304 (28.8%) lesions were malignant proven by pathological results. Patients selection was illustrated as flowchart in Fig. 1. Their clinical characteristics and laboratory results were summarized in Table 1. Malignant tumors were

Discussion

This study validated the ACR O-RADS US risk stratification on a large Chinese cohort and investigated the inter-observer agreement for O-RADS categorization and key terms used in the lexicon. The overall results suggested that the use of O-RADS provides effective malignancy risk stratification for adnexal lesions with high reliability for radiologists with different experience.

Referred to pathology, the malignancy rate of each O-RADS category were comparable to the recommended risk of the

Funding source

The present study was supported by China Postdoctoral Science Foundation (2020M672988).

Authors' contribution

Lan Cao: investigation, data curation, writing - original draft. Wei Mingjie: investigation, data curation, writing - original draft. Liu Ying: investigation, resources, data curation. Fu Juan: resources, data curation. Zhang Honghuan: resources, supervision. Huang Jing: resources, supervision. Pei Xiaoqing: conceptualization, supervision, writing - review & editing. Zhou Jianhua: conceptualization, supervision, writing - review & editing.

Declaration of Competing Interest

The authors declare no conflict of interest.

References (31)

  • J.L. Alcázar et al.

    External validation of IOTA simple descriptors and simple rules for classifying adnexal masses

    Ultrasound Obstet. Gynecol.

    (2016)
  • D. Timmerman et al.

    Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group

    BMJ.

    (2010)
  • L. Ameye et al.

    A scoring system to differentiate malignant from benign masses in specific ultrasound-based subgroups of adnexal tumors

    Ultrasound Obstet. Gynecol.

    (2009)
  • F. Amor et al.

    GI-RADS reporting system for ultrasound evaluation of adnexal masses in clinical practice: a prospective multicenter study

    Ultrasound Obstet. Gynecol.

    (2011)
  • R. Biagiotti et al.

    Predicting ovarian malignancy: application of artificial neural networks to transvaginal and color Doppler flow US

    Radiology.

    (1999)
  • Cited by (0)

    1

    Equal contributors

    View full text