Physician review of image registration and normal structure delineation

Abstract Introduction Image registration and delineation of organs at risk (OARs) are key components of three‐dimensional conformal (3DCRT) and intensity‐modulated radiotherapy (IMRT) treatment planning. This study hypothesized that image registration and OAR delineation are often performed by medical physicists and/or dosimetrists and are not routinely reviewed by treating physicians. Methods An anonymous, internet‐based survey of medical physicists and dosimetrists was distributed via the MEDPHYS and MEDDOS listserv groups. Participants were asked to characterize standard practices for completion and review of OAR contouring, target volume contouring, and image registration at their institution along with their personal training in these areas and level of comfort performing these tasks. Likert‐type scales are reported as Median [Interquartile range] with scores ranging from 1 = “Extremely/All of the time” to 5 = “Not at all/Never.” Results Two hundred and ninety‐seven individuals responded to the survey. Overall, respondents indicated significantly less frequent physician review (3 [2–4] vs 2 [1–3]), and less confidence in the thoroughness of physician review (3 [2–4] vs 2 [1–3], P < 0.01) of OAR contours compared to image registration. Only 19% (95% CI 14–24%) of respondents reported a formal process by which OAR volumes are reviewed by physicians in their clinic. The presence of a formal review process was also associated with significantly higher perceived thoroughness of review of OAR volumes compared to clinics with no formal review process (2 [2–3] vs 3 [2–4], P < 0.01). Conclusion Despite the critical role of OAR delineation and image registration in the 3DCRT and IMRT treatment planning process, physician review of these tasks is not always optimal. Radiotherapy clinics should consider implementation of formal processes to promote adequate physician review of OARs and image registrations to ensure the quality and safety of radiotherapy treatment plans.


| INTRODUCTION
The advent of three-dimensional conformal radiotherapy (3DCRT) and intensity-modulated radiotherapy (IMRT) now allows delivery of radiotherapy in a highly conformal manner. A key component of the 3DCRT and IMRT treatment planning process is the accurate delineation of target volumes (TVs) and surrounding organs at risk (OARs). While the appropriate demarcation of TVs is critical to ensuring that patients' disease is adequately treated, accurate OAR delineation is crucial for inverse treatment planning and ensures that the dose to OARs is correctly determined.
Despite the critical role of TV and OAR accuracy in delivering safe and effective treatment with 3DCRT and IMRT, delineation of these structures is often associated with a steep learning curve with significant potential for error. For example, upon central review of the contours from RTOG 0529 it was found that the small and large bowel were incorrectly contoured 60% and 45% of the time, respectively. 1 In an attempt to standardize the way in which TVs and OARs are contoured, numerous consensus guidelines and contouring atlases have been developed. [2][3][4][5][6][7] Another strategy to help ensure TVs and OARs are accurately delineated is image registration, during which diagnostic image sets are fused to the image sets obtained at the time of simulation. Although image registration can improve the ability to accurately demarcate volumes of interest, this is dependent upon the accuracy of the registration of diagnostic image sets to the image sets obtained at the time of simulation, a process that can have many sources of error. 8 Initially published by ASTRO in 2012, Safety is No Accident, provides a framework to help ensure the quality and safety of radiotherapy treatments. The most recent version of Safety is No Accident specifically defines the roles and responsibilities of members of the radiotherapy team and states that among other tasks, the treating radiation oncologist has the responsibility to "define the [TVs] on the images obtained during simulation," "confirm registration," and "review [OARs] delineated by planning staff for accuracy." Furthermore, it is recommended that "after the radiation oncologist defines [TVs] and normal tissues, when possible, another physician should review and confirm the contours before treatment planning begins." 9 This study hypothesized that in line with the framework provided by Safety is No Accident, TVs are delineated by the treating physician in the majority of clinics; while image registration and OAR contouring are typically performed by a dosimetrist or medical physicist.
However, we also hypothesized that the accuracy of OAR contours and image registrations are not routinely reviewed by treating physicians and that formal peer review of these steps prior to the treatment planning process is suboptimal.

2.A | Survey
An anonymous, internet-based survey was developed with input from radiation oncologists, medical physicists, and dosimetrists at multiple institutions. Questions in the survey included multiple choice, Yes-No, multiple-item Likert-type scale (1 = Extremely/All of the time, 2 = Quite/Most of the time, 3 = Moderately/Some of the time, 4 = Slightly/Rarely, and 5 = Not at all/Never), and free-response formats.

2.A.1 | Demographics/practice patterns
The survey (File S1) was divided into four main sections. Branching logic was used in multiple instances to ask relevant follow-up questions and to allow participants to elaborate on specific responses to questions. The first section of the survey consisted of general questions to establish the participant's job title, clinical role, and clinic demographics.

2.A.2 | OARs
The second section consisted of two subsections focused on OAR delineation. In the first subsection, participants were asked to list the job title of individuals who routinely enter OAR volumes in their clinic. Participants who enter OAR volumes as part of their clinical responsibilities were subsequently asked to characterize their confidence in their ability to accurately contour OARs in general as well as for specific commonly contoured OARs. The list of specific OARs that participants were asked to rate their confidence contouring was developed from several institutional scorecards used for plan evaluation with subsequent input from physicians, dosimetrists, and medical physicists who contributed to the survey design. Participants were also asked about any formal or informal training they had received for contouring OARs. In the second subsection, participants were asked to characterize the review of OARs by the physicians at their clinic including the frequency and thoroughness of physician review of OAR volumes along with the presence or absence of a formal process for physician review of OAR volumes.

2.A.3 | Image registration
The third section focused on image registration; in the first subsection participants were asked to detail the individuals at their clinic who routinely register diagnostic image sets with image sets obtained at the time of simulation and if applicable were subsequently asked to characterize their confidence in their ability to register images accurately. The second subsection focused on physician review of image registration.

2.A.4 | TVs
Participants were also asked to describe the individuals responsible for entry of TVs at their institution; however, given the scope of this study, participants were not asked to characterize TV entry or evaluation further. The final section of the survey asked participants to characterize how frequently changes or errors in components of the TURCHAN ET AL.

| 81
3DCRT and IMRT treatment planning process necessitating changes to the treatment plan are discovered in their clinic. The survey was closed to responses on February 28, 2019.

2.C | Statistics
Statistical analysis was performed using JMP®, Version 14 (SAS Institute Inc., Cary, NC). All tests were two-sided. The χ 2 test was used to compare among discrete variables and the t test among continuous variables. Differences between medians were assessed using the

3.A | Demographics/practice patterns
Two hundred and ninety-seven individuals responded to the survey (estimated response rate 4%). Of those who responded, 213 (74%) were medical physicists, 69 (24%) were dosimetrists, and 4 (1%) were medical physics residents or postdoctoral fellows. Further details regarding the demographics of the respondents are given in Table 1.
When asked to characterize the process of image registration and OAR and target delineation, respondents reported that dosimetrists and/or medical physicists are solely responsible for importing the primary image set, registering diagnostic images with the primary image set, and entering OAR contours at their clinics 95% (95% CI 93-97%), 89% (95% CI 86-92%), and 63% (95% CI 58-68%) of the time, respectively. 69% (95% CI 64-74%) responded that, at their clinic, only physicians enter TVs. Further details regarding which member(s) of the radiotherapy team perform these tasks in the represented clinics are shown in Table 2.

3.B | Image registration
One hundred and eighty-four (81%, 95% CI 77-85%) respondents reported performing registration of diagnostic images with the primary image set as part of their clinical responsibilities. Overall, respondents are confident in their ability to appropriately register diagnostic images with the primary image set (Likert-type score 2 [1][2]). Median confidence of respondents in their ability to appropriately register diagnostic images with the primary image set varied significantly on Likert-type scale by diagnostic imaging modality with higher confidence for CT than for MRI or PET: 1 [1-2], 2 [1][2], and 2 [1-2] (P < 0.01). Of the respondents who report performing registration of diagnostic images with the primary image set as part of their clinical responsibilities, 128 (67%, 95% CI 60--4%) indicated that they relied on the assistance of automated image registration software for "all" or "most" of the image registration that they perform (Likert-type score 2 [2][3]), and confidence on Likert-type scale in the accuracy of registration performed with automated software was rated as a 2 [2][3].
When asked to characterize how often physicians at their institution reviewed diagnostic image registration performed by others, the median Likert-type score (from 1 = always to 5 = never) was 2 [1][2][3].
When asked to rate their confidence in the thoroughness of physician review of diagnostic image registration, respondents gave a median Likert-type score of 2 [1][2][3]. While most respondents indicated that in cases where they are uncertain of the accuracy of image registration they explicitly ask the treating physician to review the registration, 19 (9%, 95% CI 5-13%) indicated that they do not ask the treating physician to review the registration. When asked to characterize the reason that they do not ask the treating physician to review image registration when they were uncertain of the accuracy, respondents most commonly indicated that they did not feel as though the treating physician would review the registration thoroughly and/or asking the physician to do so would not improve the accuracy. Of the respondents who reported contouring OARs as part of their clinical responsibilities, 87 (67%, 95% CI 60-74%) indicated that they "never" or "rarely" use auto-segmentation or other automated processes to assist in contouring OARs (Likert-type score 4 [3][4]).

3.C | Contouring
When asked to indicate their confidence in the ability of auto-segmentation and other automated processes to assist in the accurate delineation of OARs, respondents gave a median Likert-type score of Moreover, 32 (20%, 95% CI 15-25%) individuals at clinics without a formal review process stated that physicians "rarely" or "never" Moreover, errors in OAR delineation necessitating changes to the treatment plan were more likely to be caught prior to patients beginning treatment in clinics with a formal review process in place (n = 13, 35%, 95% CI 20-50%) compared to clinics with no formal review process in place (n = 26, 16%, 95% CI 10-22%; P = 0.02).

| DISCUSSION
As hypothesized, this study found that, in line with the framework provided by Safety is No Accident, image registration and OAR delineation are performed primarily by dosimetrists and/or medical physicists in the majority of clinics surveyed. Accurate image registration is critical to the 3DCRT and IMRT treatment planning process to ensure accurate delineation of target and avoidance structures, while accurate OAR characterization ensures that the dose to each avoidance structure of interest can be correctly determined and evaluated by the treating physician. Interobserver variability in these aspects of the treatment planning process has been demonstrated previously and can be both dosimetrically and clinically significant. 5,7,11 Given this, Safety is No Accident recommends that the treating radiation oncologist reviews image registration and OAR delineation performed by others to ensure the accuracy of these tasks.
As hypothesized, respondents indicated deficiencies in the frequency and thoroughness of physician review of image registration and OAR contours. Moreover, only 19% (95% CI 14-24%) of respondents reported a formal process by which OAR volumes are reviewed by physicians in their clinic, creating a potential quality and T A B L E 3 Frequency and confidence contouring common organs at risk. (Likert-type scale 1 = "Extremely" to 5 = "Not at all" Incident Learning System (RO-ILS) database, which identified "problematic plan" errors including "poor image fusion" and errors related to the target and normal structures. 12 Moreover, multiple studies performing failure mode and effect analysis (FMEA) have identified issues with inaccurate image registration and volume delineation as potential threats to the quality and safety of the care delivered in radiotherapy clinics. 13,14 When asked to characterize their confidence in their ability to accurately delineate specific normal structures from a list of common OARs, respondents' confidence significantly varied based on the OAR, as shown in Table 3. While respondents were uniformly confident in their ability to accurately contour structures such as the lung and the bladder, they acknowledged uncertainty in their ability to accurately contour OARs such as the duodenum and the sacral plexus. Most notably, despite 70% (95% CI 62-78%) of respondents stating that they are responsible for contouring the brachial plexus when clinically applicable, 21% (95% CI 13-29%) acknowledged being "not at all comfortable" with their ability to do so accurately.
This high rate of uncertainty combined with the possibly debilitating and irreversible consequences of brachial plexopathy underscores the potential for adverse events as a result of inaccurate delineation of OARs and the need for review of avoidance structures by the treating physician. 15 Given the critical nature of accurate OAR delineation to ensure safety and quality in radiotherapy treatment plans, Safety is No Accident recommends that all physicians should review OARs delineated by dosimetrists and medical physicists to ensure the accuracy of these contours. This study indicates that physician review of OAR contours is suboptimal among the represented clinics and that, on average, respondents were only "moderately" confident in the thoroughness of physician review of OAR contours. One potential way to improve physician review of OAR contours is the implementation of a formal process for OAR review within clinics. Our results demonstrate that physician review of OAR contours, as judged by medical physicists and dosimetrists, is significantly more frequent and thorough in clinics with a formal review process in place and the presence of a formal review process significantly increases the number of treatment plan changes attributed to inaccurate OAR delineation that are caught prior to patients starting treatment. Despite this, only 19% (95% CI 14-24%) of respondents indicated the presence of such a process in their clinic, providing a potential area where the quality and safety of the radiotherapy treatment process could be improved.
Respondents who indicated the presence of a formal review process in their clinic were asked to further characterize this process.
Most responses described a system in which the dosimetrist and/or medical physicist contours OARs and the OARs are then labeled in some manner to indicate that they have not yet been reviewed. The treating physician subsequently reviews the OARs and then changes the label to indicate that they have reviewed and verified the accuracy of the OAR contours. Moreover, a few respondents indicated the presence of prospective contouring rounds in their clinic, during which physician peer review of the clinical plan as well as OAR and TV contours is performed. This practice is in line with recommendations in Safety is No Accident and has been shown to improve peer review and treatment standardization and decrease delays in treatment as a result of modifications that need to be made to the treatment plan after the treatment planning process has been completed. 16,17 Aside from improving physician review, another potential strategy to improve OAR accuracy is the implementation of additional  10 Under the assumption that the overlap in subscription to these two groups is minimal, this represents a response rate of approximately 4%. Although the potential for nonresponder bias increases when response rates fall below expected levels, it is important to note that expected response rate for a sur-

| CONCLUSION
Despite the critical role of OAR delineation and image registration in the 3DCRT and IMRT treatment planning process, physician review of these tasks, as reported by an international cohort of surveyed dosimetrists and medical physicists, is at times suboptimal. Radiotherapy clinics should consider the use of a formal review processes for OAR delineation and image registration to ensure the quality and safety of treatment plans, the implementation of which has been incorporated into a recently described framework for promoting safety and quality in radiotherapy clinics. 20 Another potential strategy to improve the quality and safety of the treatment delivered by radiation therapy clinics is the implementation of standardized training programs for individuals who contour normal structures and perform image registration as part of their clinical responsibilities.
Further studies to develop optimal training and systems-based practices for OAR delineation and image registration are underway.
Moreover, further studies are warranted to characterize physicians' perception of this process and objectively evaluate the extent and quality of physician review of OARs and image registration along with how well these measures correlate with accuracy of OAR delineation and image registration.

ACKNOWLEDG MENTS
The authors made the following contributions to the manuscript: