Effectiveness of Training in CT Colonography Interpretation: Review of Current Literature

International guidance recommends that readers be speciﬁcally trained before embarking on independent interpretation of CT colonography (CTC) examinations. Systematic comparison of both international training requirements and the effectiveness of CTC training is lacking in the published literature. Therefore, we identiﬁed available international training standards for CTC and performed a review of studies published in the last 20 years to assess the impact of CTC interpretation training on reader diagnostic accuracy. A wide variation in training requirements was observed. Studies of the effectiveness of CTC reader training were heterogenous in methodology, with large variation in sample size and the type of training administered. Although training in CTC interpretation improves reader sensitivity overall, it has varying impact on speciﬁcity. Consensus agreement on the best way to train and assess readers in CTC interpretation may lead to lasting improvements in reader performance.


C T colonography (CTC) is a whole-colon investigation
commonly used in the detection of colorectal cancer and precursor polyps. Patients drink an oral cathartic agent to cleanse the bowel, with oral contrast medium used to tag any residual stool; these can be combined using a single agent with osmotic laxative effect, such as Gastrografin (diatrizoate meglumine and diatrizoate sodium solution, Bracco Imaging, USA). 1 High-resolution 2-and 3-dimensional (2D, 3D) CT images of the gas-distended colon are then obtained. Interpretation of these CTC images is time-consuming and differs from routine abdominopelvic CT. It therefore requires additional skills and training for adequate diagnostic accuracy. As such, it is recognized that specific training in CTC interpretation is necessary for accurate independent reporting.
Lack of training in CTC interpretation will lead to perceptual errors when detecting colorectal polyps and cancers, which in turn contributes to poorer outcomes. 2 There is currently no consensus on the most effective method to train readers in CTC interpretation. Many current guidelines and standards for the practice of CTC advocate a minimum case number as the main training requirement. [3][4][5] However, they do not provide details regarding the distribution or abnormalities that should be included. Moreover, there is wide variation in the number of training cases that is regarded as a suitable basic training requirement, with no clear logic as to why these numbers were selected. Although many studies have been conducted to assess the impact of interpretation training on CTC readers, these have not been comprehensively reviewed or incorporated into societal guidelines for CTC training. Here, we (a) review current international consensus guidance for CTC training prior to independent practice and (b) review the published literature regarding training methods for CTC interpretation. In a subsequent article, we draw on these data and the educational literature to make recommendations for best practice in CTC interpretation training. 6

Current International CTC Training Guidelines
We performed an internet search for training guidelines from the 10 most populous countries according to the World Bank. 7 Search terms included the country name and variations of "CT colonography", "CTC", "CTC training", "CTC accreditation", "CTC guidelines", and "radiology society" or "college". In addition, where they could be identified, we searched individual country or region websites, as well as regional or international radiology organizations which were known to have previously published guidance regarding CTC. Where references were identified regarding CTC training guidelines or minimum standard stipulations for radiologists, these were retrieved, and the source information interrogated.

Literature Review of Training Methods for CTC Interpretation
We searched PubMed using the following Medical Subject Headings (MeSH) and free-text terms: (1) (CT OR comput* AND tomogra*) AND (colonograp* OR virtua* AND colono*); (2) train* OR test* OR perform* OR experien* OR error*; (3) 1 AND 2; relating to CT colonography training and performance and examined reference lists from relevant articles. Inclusion criteria were any type of reader performance or diagnostic accuracy study assessing radiologist diagnostic accuracy or yield either before and after an interpretation training intervention, or simply after interpretation training. The search was restricted to dates January 1, 2001 to December 31, 2021, and only articles published in English were eligible. Excluded studies were those that assessed reader sensitivity without a training intervention of any kind, those that only assessed the performance of computer-aided detection (CAD) and those that did not involve CTC interpretation or report sensitivity. Abstracts were screened by a single researcher (AEO) and the full text of potentially eligible studies was retrieved for further assessment.
Once eligibility was confirmed, a single researcher (AEO) extracted the following characteristics for each article: (a) author and publication year; (b) number of sites, readers and faculty participating in the training intervention; (c) training components of the intervention; (d) background of readers undergoing training (eg, radiologist or radiographer/technician); (e) reader and trainer career CTC experience; (f) number of test cases completed per reader; (g) number of lesions (polyps and/or colorectal cancers) assessed in test(s) both pre-and post-training intervention; (h) characteristics of such lesions; and (i) summary measures of the effect of training on reader diagnostic accuracy, including sensitivity and specificity where available. If data were not available in the original research article report, supplementary materials were examined, but no additional data was sought from primary research authors.
The data were extracted to a Microsoft Excel spreadsheet and summarized with descriptive statistics. Due to considerable heterogeneity of initial reader experience and pre-training diagnostic accuracy, quantitative synthesis (meta-analysis) was not attempted.

International CTC Training Guidelines
We identified several national and international regulatory bodies that have established training recommendations and guidelines for CTC readers (Table 1). These include standards which refer to both the CTC service ("service level guidance"), and the competency and performance of CTC readers ("reader level guidance").
Although the importance of training in CTC interpretation prior to independent practice is recognized, none of these bodies mandate the completion of a specific program or training package prior to independently reporting CTC. The Royal Australian and New Zealand College of Radiologists (RANZCAR) is the only body to accredit (or certify) "CTC specialists", a status achieved by (a) fulfilling minimum CTC training requirements stipulated by the College, and (b) maintaining an audited CTC logbook. The Joint Guidance from the British Society of Gastrointestinal and Abdominal Radiology (BSGAR) and the Royal College of Radiologists (RCR) is the only guidance to stipulate objective, quantitative performance indicators for CTC readers, including targets for polyp identification rates and positive predictive values. However, this primarily applies to readers who have already been trained and are reporting in routine practice, rather than those undergoing initial basic training.
There is no consensus on the minimum number of validated scans to be interpreted before achieving reporting competency, and recommendations vary widely between published guidelines. for example, the ACR, CAR and RANZCR all suggest a minimum of 50 training cases, whereas the ESGAR were unable to form a consensus recommendation, and the BSGAR-RCR recommends a minimum of 175 validated cases (by endoscopy or surgery), with over 300 training cases as an "aspirational" standard.

CTC Training Literature: Study and Reader Characteristics
Initial searching identified 987 abstracts, which was subsequently refined to 27 full text studies for further analysis (Fig. 1). Of these, 8 did not include a training intervention or were focused on CAD performance and were excluded, leaving 19 articles for data extraction and analysis. Table 2 provides a summary of the included articles, involving 372 readers at 143 centers interpreting 1183 lesions.
Where specified, most (85%; 11/13) of these studies were focused on training novice or inexperienced readers (0-500 CTC cases), [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24] apart from 2 multi-center studies, which trained experienced readers who had reported more than 500. 2,25 Accordingly, most of these studies only assessed diagnostic accuracy after the training intervention (rather than pre-and post-training). Three studies assessed the effect of training with and without CAD used concurrently during interpretation, and all observed some benefit of CAD use. 15,22,23 Five studies measured interpretation times before and after training, and all except one, 15 observed a decrease in interpretation time. 8,10,12,16 CTC Training Literature: Training and Testing Methods Employed Several different training components are described, which can be broadly categorized into (a) hands-on training workshops; (b) trainee-directed independent reading of relevant CTC literature; (c) didactic lectures on CTC-related topics (technique, software applications, interpretation, and pitfalls); and (d) reading of practice cases (independently or supervised; whole data sets or partial studies).
Most studies (79%; 15/19) contained passive, standardized (ie, non-tailored) teaching components for example, lectures, self-directed reading, or case review, and were heavily trainer focused. Seven studies used only a single expert trainer. Only 1 study emphasized individualized feedback on reader performance which was provided by a panel of experts. 2 Where used, the format of the training workshops varied according to study design, but was typically 1 or 2 days (ranging from half a day to 4 days), with teaching delivered by several expert faculty in 1:1, 2:1 or small group tutorial setting. 2,8,11,15,20,23,25 If specified, definitions of a faculty expert again varied, but were usually a board-certified abdominal or gastrointestinal radiologist with a career experience of 300 CTC cases. Only one study described training given to expert faculty in how to specifically train readers in CTC. 2 All studies incorporated an element of CTC interpretation testing in either a formative (throughout the training) or       Post-training per-lesion sensitivity for 6 mm lesions varied widely between studies, from 51%, up to 100%, 16,17,24 and can be attributed to differing sample sizes (number of readers, test cases and lesions), reader experience and difficulty of the lesions selected for testing. Generally, studies which included 'hard-to-detect' lesions (eg, those assessing the impact of CAD on radiologist interpretation, or recruiting experienced readers being trained on subtle cases) observed lower sensitivities compared to those that did not. 2,9,11,17 All studies where pre-and post-training assessments were administered observed a significant improvement in reader sensitivity, 2,12,15,16,20,23,25 except 2. 10,13 The impact of training on specificity was varied, with some studies reporting reduced specificity after training. 2,15 Other studies found an improvement in specificity as reader experience increased; 8,10,25 one study observed a trend towards improving specificity in the 12 months after training, implying a gradual improvement in lesion characterisation. 2

Discussion
CT colonography (CTC) is highly sensitive for large (10 mm) polyps and colorectal cancers and has good sensitivity for 6 to 9 mm polyps. 26 Accurate interpretation of these studies in routine clinical practice is essential if the outstanding diagnostic accuracy demonstrated in research studies is to be translated to the real world. Earlier CTC training studies advocated focusing on a minimum number of CTC cases read prior to independent reporting; however, simple case review, even with endoscopic correlation, does not necessarily lead to improved sensitivity, as the same errors may be perpetuated. 10 Accumulation of CTC caseload has a differing effect on reader sensitivity, with a third of readers not reaching competency even after reviewing 175 CTC cases. 12 Studies which have focused on delivering feedback to readers, with or without specific individualized training, have observed significant gains in reader sensitivity. 2,8,9,25 Therefore, the development of expertise requires training and feedback in addition to clinical experience. 27 In this study, we found that various institutions globally have provided recommendations for radiologist training prior to independent practice, but these recommendations are highly variable. The minimum number of cases varies from >50 to (under ideal circumstances) >300, with limited detail regarding what these cases should be composed of, other than that they should be validated endoscopically (or via surgery). This heterogeneity in recommendations is perhaps due to the relatively small number of published articles investigating the topic of CTC reader training. Our literature review identified only 19 such articles, with variable study designs and training interventions.
Although, in general, reader training was associated with superior detection rates, several early studies found no effect on reader sensitivity. 10,13 Larger, more recent studies which focused on a broad selection of relevant cases and careful targeted feedback to those undergoing training have shown greater benefits. 2,12,25 Several studies have shown a considerable difference in pre-training ability to interpret CTC; Heresbach et al. 11 found that the best predictor of final accuracy was initial reader sensitivity at the start of training, and Liedenbaum et al. 12 found that some readers were unable to achieve adequate diagnostic accuracy despite prolonged training. The number of cases to which a reader is exposed is only a single facet of training, and a combination of an individual's aptitude, CTC case mix, training and feedback methods are all important to maximize performance. Generally, although specificity may reduce immediately after training, this tended to improve over time as skills of lesion characterization are being developed and refined.
In summary, we found considerable variability in national and international guidance recommendations for CTC training prior to independent practice. The published literature shows there is a clear benefit of training, which can be maximized through individualization and reader feedback. Optimizing reader training in CTC will require a more sophisticated program that should ideally be standardized to ensure all those learning how to interpret CTC can achieve the high accuracy that the technique has obtained in research trials.