Clinical Implementation of a Proton Dose Verification System Utilizing a GPU Accelerated Monte Carlo Engine

Purpose: To develop a clinical infrastructure that allows for routine Monte Carlo dose calculation verification of spot scanning proton treatment plans and includes a simple biological model to aid in normal tissue protection. Materials and Methods: A graphical processing unit accelerated Monte Carlo dose engine was used as the calculation engine for dose verification on spot scanning proton plans. An infrastructure was built around this engine that allows for seamless exporting of treatment plans from the treatment planning system and importing of dose distribution from the Monte Carlo calculation via DICOM (digital imaging and communications in medicine). An easy-to-use Web-based interface was developed so that the application could be run from any computer. In addition to the standard relative biological effectiveness ¼ 1.1 for proton therapy, a simple linear equation dependent on dose-weighted linear energy transfer was included. This was used to help detect possible high biological dose in critical structures. Results: More than 270 patients were treated at our proton center in the first year of operation. Because most plans underwent multiple iterations before final approval, more than 1000 plans have been run through the system from multiple users with minimal downtime. The average time from plan export to importing of the Monte Carlo doses was less than 15 minutes. Treatment plans have been modified based on the nominal Monte Carlo dose or the biological dose. Conclusion: Monte Carlo dose calculation verification of spot scanning proton treatment plans is feasible in a clinical environment. The 3-dimensional dose verification, particularly near heterogeneities, has resulted in plan modifications. The biological dose data provides actionable feedback for end of range effects, especially in pediatric patients.


Introduction
Proton beam therapy (PBT) is emerging as a promising form of external beam radiation therapy for cancer due to its superior ability to spare healthy tissue while treating the clinical target to the required tumoricidal dose. At the present time, PBT can be delivered in 3 modes: passive scattering, uniform scanning, and pencil beam scanning (PBS). Of these 3 delivery modes, PBS is the most advanced as it allows for intensity modulated PBT (IMPT). http://theijpt.org Mayo Clinic is establishing a PBT program at 2 of its sites, one in Rochester, Minnesota, and the other in Phoenix, Arizona. The PBT facilities (Hitachi Ltd, Tokyo, Japan) at both sites are identical in all meaningful aspects; each features a synchrotron capable of producing proton beams with energies ranging from 71.3 MeV to 228.8 MeV, 4 gantry treatment rooms, and 1 fixed beam treatment room. All treatment rooms are equipped with a PBS nozzle for the delivery of IMPT. Treatment planning is carried out using the PBT treatment-planning software (TPS) Eclipse, version 13.5 (Varian Medical Systems, Palo Alto, California).
As part of a comprehensive quality assurance (QA) program, we identified a need for an independent secondary verification of the dose calculation carried out in the Eclipse TPS. The practice of performing an independent secondary dose calculation, which is well established in photon external beam therapy [1] is slowly being adopted in PBT. Secondary dose calculations for passive scattering PBT [2][3][4], uniform scanning [5,6], and PBS [7,8] have recently been reported. The requirements to use the second check at both centers, provide superior accuracy to the commercial dose calculation, and run quickly enough to be deployable in the clinic for every patient prompted us to implement a graphical processing unit (GPU) based fast Monte Carlo (MC) dose calculation engine [9].
In addition to a more accurate dose calculation, the MC calculation allows the inclusion of a relative biological effectiveness (RBE) model based on the dose-weighted linear energy transfer (LETd) of the protons [10]. This was deemed valuable given the uncertainty [11,12] of the proton RBE as a function of linear energy transfer (LET), the complexity of IMPT plans, and the recent reports of necrosis following PBT [13,14].
The purpose of this study is to report on a novel, clinically practical, and innovative design that allows the sharing of computational resources over large geographical distances utilizing a fast GPU-based MC algorithm for routine secondary proton dose calculation and RBE information for clinical use in a PBS system.

Graphical Processing Unit Accelerated Monte Carlo Dose Calculation
The functional details and benchmarking of the GPU accelerated MC dose calculation are described elsewhere [9,15]. In short, the Compute Unified Device Architecture framework, which is a parallel computing platform and application programming interface, was used to implement GPU kernels for the following tasks: (1) simulation of beam spots from our various scanning nozzle configurations; (2) proton propagation through the simulated patient based on computed tomography (CT) image data; (3) modeling of the intranuclear cascade stage of nonelastic interactions when they occur; (4) simulation of nuclear evaporation; and (5) statistical error estimates on the dose.
Validation of the GPU MC calculation consisted of secondary particle yield calculations in proton collisions with therapeutically relevant nuclei, dose calculations in homogeneous phantoms, and recalculations of complex head and neck treatment plans from a commercially available TPS, all of which were compared with Geant 4.9.6p2/TOPAS [16,17].
Commissioning of the GPU MC code consisted of matching the lateral spot shape, depth and width of the distal edge, and number of protons per monitor unit for each of the 97 pencil beam energies, which ranged from 70 to 232 MeV, available for treatment to the measured values. The conversion of CT Hounsfield units to material and density was also commissioned following the formalism in Zacharatou et al [18], including material overrides for implanted device, i.e. titanium or stainless steel implants, details given in Zhao et al [5] Extensive comparisons to measurements were conducted to validate the commissioning of the GPU MC.

Simplified Biological Model
Many different proton RBE models have been proposed [12,[19][20][21]. These models are usually a function of LETd, the a and b parameters from the linear quadratic formula, and the physical dose (PD). In the system described here, in addition to the conventional 1.1 constant, a simple linear model that is independent of a and b is used.
During the MC simulation, LETd [10] is calculated along with the PD distribution. For each voxel of the CT data set the biological dose (BD) was computed from PD and LETd. In addition to the conventional 1.1*PD dose file, a dose file with an LETd weighted BD is created based on the following equation:

System Configuration
The requirements of the application presented several unique challenges. Because this application was to be used clinically, it needed to be built using institutionally approved tools to ensure reliability and maintainability, so the user interface and scheduler services were developed using common Windows technologies including C# programming language, .NET framework, Microsoft SQL Server databases, and Visual Studio development environment. The system also had to interoperate with a TPS. This interoperability was achieved through the use of a standard medical imaging format and communications protocol, namely DICOM (digital imaging and communications in medicine). The dose simulation engines, for performance reasons, were developed to be executed on Linux-based servers. To seamlessly interact with Linux executables from a Windows application, Python-based Web services were employed on the Linux side, allowing simple Web requests to be made from the Windows servers. A file share was created, allowing data to be accessed from all servers, regardless of the operating system. These last 2 were critical to the successful integration of a system with some key components developed and deployed for Windows and others that must be run under Linux. For improved usability, the user interface is Web based so users can run anywhere, provided the user is within the institutional firewall. Automatic authorization is provided using an Active Directory service.

Digital Imaging and Communications in Medicine Handling
The MC dose verification system, called 2nd Check, begins with a DICOM export from the TPS to the 2nd Check DICOM listener. The export filter is set up to push all chosen DICOM files to the listener running as a service on a Windows server. To successfully complete the dose verification, the user must export the desired radiation therapy ION plan along with the associated radiation therapy dose, radiation therapy structure set, and all CT images associated with the plan .
To prepare the dose files for import back into the TPS, a windows service called the Import Preparer makes a copy of the original radiation therapy plan file for each newly generated dose file. New unique identifiers are created and properly matched for each of the new radiation therapy plan and radiation therapy dose files, which is needed for successful import back into the TPS. These files are then all copied over to an outgoing directory that is already preset within Eclipse, greatly simplifying the import process.

User Experience and Data Management
One of the key design considerations was to keep the process for users as smooth and intuitive as possible. The user experience should consist of signaling to the system that they are ready to run the 2nd Check and clearly see the status of the process. Another requirement was that inputs would need to be made from across the institutional network on a variety of devices. The last requirement was to build a system with a high degree of fault tolerance and segment independence.
The functionality of all pieces of this system is based on virtual concepts created in a relational database. The primary virtual concept is the queue. All activity the user expects to occur automatically happens in a virtual queue.
To meet the requirement for the user experience, it was decided to build a Web application to accommodate the run anywhere philosophy. The contents of the website were chosen for simplicity and ease of navigation. When the user enters the website, the first page is simply the instruction and any bulletins with simple tabs across the top. The tabs take users to where they need to be, either looking for patients or viewing the current state of the queue. Patient navigation occurs by searching for either patient ID or the patient name. Once the patient is found, the user can drill down to the plans.
At this level, users will see the actions available for their plans or its current state in the QA system. The queue is simply the view of all plans currently moving through all steps in relation to their current state and the time they entered the queue.
The plans remain in the system until they are removed. Two processes exist to remove a plan from the system; deleting the patient or importing all of the patients plans back into the TPS.

Clinical Workflow
After the treatment plan is exported and results are brought back into the TPS, as detailed in the sections ''DICOM Handling'' to :User Experience and Data Management,'' the clinical evaluation is performed. The first step in the evaluation is performed by the medical physicist and consists of comparing the physical dose calculated by the TPS and the MC system (both include the conventional 1.1 biological factor). Line profiles in each CT slice, particularly in heterogeneous areas, are examined, dose-volume histogram statistics are evaluated, and dose output is examined. If major issues are detected, such as dose differences .5%, the root cause is explored, and typically, a new planning approach is devised. Minor issues are noted and brought to the treating physician's attention during their review. The medical physicist then evaluates the BD distribution. This step primarily includes evaluating sensitive critical structures with the key feature of interest being a comparison of the enhanced target BD compared with enhanced critical structure dose. As a note, independent of this system, the robustness of the plan with respect to stopping power and position uncertainties are also evaluated at this stage; this can be done in the TPS or via the MC system. In routine practice, the TPS is generally used. If only minor issues are found (ie, sporadic dose differences ,2%), the plan is prepared for the physician review with the minor issues noted verbally or in writing. The physician reviews the TPS dose distribution along with the MC and BD distributions. If major discrepancies between the MC and TPS dose (ie, systematic differences .2%) or concerns based on the BD exist that cannot be easily remedied by plan modification; the physician is contacted for clinical guidance.

Calculation Times
The calculation time is broken up into the following 5 components: (1) DICOM export from the TPS, (2) submission of the plan to the GPU queue, (3) MC conversion and calculation, (4) conversion of the dose files (PD, BD, LETd) to DICOM, and (5) DICOM import to the TPS.
The DICOM export is typically around 1 minute but depends primarily on the size of the CT data set. The plan is generally in the queue for less than 30 seconds. The system is designed to handle several submissions simultaneously, but occasionally all servers may be occupied and queue time can be up to 10 minutes. The MC calculation time can range from 3 minutes for a 2-field plan with a clinical target volume ,200 cm 3 to 9 minutes for a multiple field plans with a clinical target volume .1000 cm 3 . Currently, 10 8 protons are simulated for each plan, divided among all the proton spots according to their relative weight. Conversion of the MC dose cubes to DICOM generally takes 2 minutes; this time also includes archiving and transferring the files to the import directory. Importing the DICOM dose cubes into the TPS takes 2 minutes. The entire process takes anywhere from 7 to 18 minutes with about half of the time occupied by the MC calculation. This meets our initial clinical specification of ,15 minutes for the majority of the plans. Figure 1 represents a case where the dose calculation from the TPS and the 2nd Check system agree quite well. Figure 2 shows a case where the TPS and MC dose disagree slightly. The MC dose distribution is 2.2% lower than that from the TPS. This difference is attributed to the small target size (~2 cm in diameter) and the complex heterogeneous surroundings. This plan would be suitable for renormalization based on the MC dose calculation. Figure 3 is an example of a large disagreement between the TPS and MC dose. This error in the TPS calculation would not be picked up by a patient-specific measurement on a flat water phantom as in that case both calculations agreed to within 2% in dose. Upon investigation it was determined that the beam was traveling parallel to a large bone/air interface, which resulted in the discrepancy.

Clinical Results
Shown in Figure 4 is the dose colorwash for the LETd weighted biological plan. This is an example of a plan modification based on the BD. The top image is the initial plan and the bottom image is after mitigation for high BD. In the region of the biological hot spots (arrows), the physical dose of the initial plan is 2% higher than the modified plan; however, the BD is approximately 10% higher. A slight margin and gantry angle change are the main differences in the plan; the optimization objectives are similar.
More than 270 patients were treated at our proton center in the first year of operation, all of whom used this system. Because most plans underwent multiple iterations before final approval, more than 1000 plans have been run through the system from multiple users with minimal downtime. The average time from plan export to importing of the MC doses is less than 15 minutes. Modifications of the treatment plans have been made based on either the nominal MC dose or the BD. An exact quantification of the percent of plans that have been modified based on this system is difficult because both the MC dose Figure 2. The dose colorwash of the treatment planning system (TPS) generated distribution (left) and the Monte Carlo generated dose distribution (right) for a central brain tumor with a clinical target volume (CTV) (red) that is approximately 2 cm in diameter. The graph in the center displays the dose profile through the center of the CTV, with red representing the TPS calculation. Note that the Monte Carlo dose distribution is 2.2% lower than that from the TPS in the center region of the CTV. This difference is attributed to the small target size and the complex heterogeneous surroundings. This plan would be suitable for renormalization. and BD are included in the instructions to the dosimetrists. One example of this instruction is for breast treatments. For those treatments, there is a constraint on the brachial plexus of less than 60 Gy as represented in the BD.

Discussion
In this report, we demonstrate that a GPU-based MC dose second-check system is feasible and has been implemented for routine use on every patient treated with spot scanned protons in an enterprise-wide infrastructure allowing fast, accurate, and simple dose second checks to be integrated into the patient care path. Early work by Kooy et al [2] demonstrated the need for and value of a calculative infrastructure that was independent of the planning and delivery system. Based on scattered PBT, the system took into account a limited number of routinely used treatment components to allow for an independent quality check. Sahoo et al [3], also working with scattered beams, demonstrated that an monitor unit-check algorithm similar to that in photon beam therapy was possible if each physical characteristic and variable was taken into account. In both cases, an independent and accurate second check was created. More recently, Mackin [7] reported on the use of an analytical dose second-check algorithm to perform patient-specific spot scanning QA with intent to reduce or eliminate time-consuming measurements. The current work extends beyond each of these in that it is dedicated to spot scanning, creates a highly accurate and very efficient GPU-based MC second-check system, is integrated into a robust and enterprise-wide interface, calculates and compares physical dose with the commercial TPS, and provides key insight into LET and simple BD distributions.
Recent reports of brainstem injury in patients undergoing proton therapy have highlighted the importance of understanding the biologic dose and end of range effects, especially in pediatric patients [8,9]. Although RBE is dependent on multiple factors, understanding differences in LET along the proton path, particularly at the Bragg peak is important because it is an actionable parameter, especially in spot-scanning systems. The BD 2nd Check process described herein provides visual feedback of areas of relative intensification of biologic dose. By visually inspecting these BD maps for clinical treatment plans, clinicians can easily see the effects of beam arrangements, optimization parameters, and planning margins that may result in regions of biological intensification. However, because the calculation is based on a simple model, the absolute BDs are Figure 3. The dose colorwash of the treatment planning system generated distribution (left) and the Monte Carlo generated dose distribution (right) for a head and neck treatment. There exists a notable region where the Monte Carlo calculation results in an approximately 15% increased dose. Upon investigation it was determined that the beam was traveling parallel to a large bone/air interface near the trachea and vertebral bodies, resulting in the discrepancy, which is not located immediately adjacent to those interfaces. currently unknown. Further refinements in understanding proton LET from cell models, as well as patient data, can be incorporated in future models.
In the initial implementation now in clinical use, end-to-end second check per patient has met required times of less thañ 15 minutes, running from anywhere on the network, and by any physics or dosimetry staff member. Current limitations of the system currently are the queue/scheduler, which can result in longer wait times if multiple plans are in queue. In addition, a fixed number of protons are currently used in the simulation, but we anticipate that improvement in calculation times can be achieved with a statistical error-based stopping criterion. Finally, this system does not replace patient-specific machine-based measurement since machine performance cannot be predicted by the software; however, with the use of this system to verity 3-dimensional dose calculation, patient-specific measurement at multiple depths would no longer be necessary.

Conclusion
MC dose calculation verification of spot scanning proton treatment plans is feasible and practical in the clinical environment. The 3-dimensional dose verification, particularly near heterogeneities, has resulted in plan modifications. The BD data provide actionable feedback for end-of-range effects, especially in pediatric cases.

ADDITIONAL INFORMATION AND DECLARATIONS
Conflicts of Interest: The authors have no conflicts of interest to disclose. . The dose colorwash for the dose-weighted linear energy transfer weighted biological dose calculation for the initial plan (top) and after modification (bottom). The modification was done to reduce the high biological dose indicated by the yellow arrows. In the region of the biological hot spots, the physical dose of the initial plan is 2% higher than the modified plan, but the biological dose is~10% different. Small modifications of margin and gantry angle resulted in an improved biologic dose profile in critical locations such as the brainstem and spinal cord.