Analysis of patient‐specific quality assurance for Elekta Unity adaptive plans using statistical process control methodology

Abstract The Elekta Unity MR‐linac utilizes daily magnetic resonance imaging (MRI) for online plan adaptation. In the Unity workflow, adapt to position (ATP) and adapt to shape (ATS) treatment planning options are available which represent a virtual shift or full re‐plan with contour adjustments respectively. Both techniques generate a new intensity modulated radiation therapy (IMRT) treatment plan while the patient lies on the treatment table and thus adapted plans cannot be measured prior to treatment delivery. A statistical process control methodology was used to analyze 512 patient‐specific IMRT QA measurements performed on the MR‐compatible SunNuclear ArcCheck with a gamma criterion of 3%/2 mm using global normalization and a 10% low dose threshold. The lower control limit (LCL) was determined from 68 IMRT reference plan measurements, and a one‐sided process capability ratio (Cp,l) was used to assess the pass rates from 432 measured ATP and 80 measured ATS plans. Further analysis was performed to assess differences between SBRT or conventional fractionation pass rates and to determine whether there was any correlation between the pass rates and plan complexity. The LCL of the reference plans was determined to be a gamma pass rate of 0.958, and the Cp,l of the measured ATP plans and measured ATS plans were determined to be 1.403 and 0.940 for ATP and ATS plans, respectively, while a Cp,l of 0.902 and 1.383 was found for SBRT and conventional fractionations respectively. For plan complexity, no correlation was found between modulation degree and gamma pass rate, but a statistically significant correlation was observed between the beam‐averaged aperture area and gamma pass rate. All adaptive plans passed the TG‐218 guidelines, but the ATS and SBRT plans tended to have a smaller beam‐averaged aperture area with slightly lower gamma pass rates.

generated for each fraction while the patient lies on the treatment table, patient-specific QA measurements using traditional phantoms are not possible before the treatment commences. Since the daily-adapted treatment plans cannot be measured using current equipment before the daily-adapted plan is delivered to the patient, an investigation into the QA pass rates of the adapted plans compared to the initial reference plan is needed to provide insight into the variations between the daily adapted plans on the Unity MR-linac. To our knowledge, the variability or consistency of the gamma pass rates for each daily ATS or ATP plan to their reference plan has not been investigated.
For this study, statistical process control (SPC) is used to compare the results of the measured gamma pass rates of the adaptive ATP and ATS plans to the measured gamma pass rates of the original reference treatment plans. Further analysis on the plan complexity, determined by calculating the modulation degree and beamaveraged aperture area, was performed for the reference, ATP and ATS plans to determine if there were any correlations between these metrics and the measured gamma pass rates. Historically, SPC has been used in radiotherapy studies to determine the effect of a procedure change in an experimental population, such has the dosimetric changes to a treatment plan when using alternative planning techniques. [9][10][11][12] Statistical process control is a method of statistical analysis that is used to evaluate the function efficiency of a reference process and monitor the variation of additional experimental processes. As such, SPC can provide a statistical expectation of how a given ATP or ATS treatment will be delivered based on the deliverability of its reference plan.

2.A | Elekta unity adaptive planning
During each daily-fractionated treatment, the patients were instructed to lie on the table with proper immobilization in place.
Following an MRI acquisition, a physicist and physician worked together to generate an adapted treatment plan, either ATP or ATS.
ATP uses a rigid registration to align the daily MR and planning CT datasets. The CT dataset or reference plan is treated as the primary dataset, and the daily MRI is shifted to align with the primary dataset. After the registration is approved, there are four options for optimizing the treatment plan within the ATP workflow using: (a) original segments, (b) adapt segments, (c) optimize weights, and (d) optimize shapes. 8 A plan calculated with "original segments" aligns the two datasets and uses the original MLC pattern on the new patient position. This is only appropriate for instances where there are minimal shifts. A plan calculated with "adapt segments" moves the MLC positions using a segment aperture morphing (SAM) algorithm 13 and projects the fluence to the new patient position through the adapted MLC positions. A plan calculated with "optimize weights" uses the SAM algorithm to modify the MLC positions and optimizes the beam weights to best match the original DVH parameters under the new patient position conditions. Finally, a plan calculated with "optimize shapes" is a re-optimization of both the MLC shapes and weights using a warm-start gradient descent optimization algorithm that aims to match the reference plan DVH. Machine deliverability constraints are imposed in each optimization loop of the warm-start optimizer.
Plans calculated under the ATS workflow experience a full deformable image registration (DIR) of the CT to the MR. In many cases, the DIR adapted contours need to be further adjusted to match the daily anatomy as seen on the MRI. Following contour generation and approval, ATS uses the full Monaco Hyperion optimizer with the DVH constraints set during the reference plan creation. These constraints can be adjusted in real-time during the reoptimization process as needed. As a result, an entirely new re-optimized plan is generated from both ATP and ATS procedures with the exception of ATP original segments, albeit through different means. Both ATP and ATS adapted planning workflows are performed while the patient lies on the treatment table within the MRlinac bore. As a result, it is not possible to perform QA of the adapted plan before the patient is treated. The policy at our institution was to have the patient-specific QA completed prior to the subject's next treatment fraction. where an initial reference plan was created.

2.A.1 | IMRT plan creation and QA measurements
Prior to any treatment, the reference treatment plan was recalculated on the MR-compatible SunNuclear ArcCheck (Melbourne, FL), measured and compared using a gamma criterion of 3%/2 mm with global normalization and a 10% low dose threshold. Due to the design of the Unity MR-linac, the ArcCheck phantom is first placed on the QA platform and translated into the bore. A CT scan of the ArcCheck was imported into the Monaco treatment planning system (TPS) and the relative electron densities were determined as described by Snyder et al. 14,15 The alignment of the ArcCheck phantom utilizes a custom QA platform since the Unity MR-linac only has a sagittal laser, which is insufficient to properly align the phantom by itself. As a result, the alignment reproducibility is dependent on the consistency of how the ArcCheck phantom and platform are setup on the treatment couch. The setup for the ArcCheck phantom for QA measurements is shown in Fig. 1

2.A.2 | IMRT QA Plan measurement uncertainty
The QA platform was calibrated to isocenter during commissioning and verified on a monthly basis to ensure it was within recommended tolerances. The QA platform calibration process consists of placing a phantom supplied by Elekta with radio-opaque BBs at known loca-  Figure 2 shows the directional components of the QA platform and the ArcCheck phantom. Table 1 outlines the tolerances, separated into the directional component, for both the QA platform and the ArcCheck phantom.
The total directional (X or Y) uncertainty was calculated using the following equation.
where σ AC is the directional component uncertainty of the ArcCheck phantom and σ QAP is the directional component uncertainty of the QA platform. Table 2 outlines the total uncertainty (σ total ) calculated for both the X and Y directions.
The SNC software allows for a virtual shift of the measured data in both the X and Y directions.
Based on the above uncertainty analysis, up to a 2 mm shift in either the X or Y directions is allowed.

2.A.3 | Statistical process control analysis
Traditionally, SPC utilizes the lower control limit (LCL), upper control limit (UCL), and the center line. The center line was defined to be the mean of the reference data. The LCL was determined using data from the reference IMRT measurements, where μ w is the mean of the reference data, σ w is the standard deviation of the reference data, and L is the desired distance of control limits from the central line, or in other words the number of standard deviations. The calculation for this study was based on an L of 3 representing three standard deviations from the mean. Because there is no clinical significance of an upper bound when examining gamma pass rates, the UCL was not calculated.
In addition, a one-sided process capability ratio ðC p,l Þ was used to assess the adaptive plan results, where μ is the mean of the experimental data, σ is the standard deviation of the experimental data, and LCLis the value calculated from the reference data using Eq. (2). A C p,l value above 1 indicates that the variability of the test data, adaptive plan gamma pass rates, was within the inherent variability of the process (reference plan pass rates).
For the SPC analysis, the process capability ratios were determined for ATP and ATS plans using all of the measured data. When calculating the process capability ratios for the SBRT and conventional fractionation plans, only the first fraction of the week was used in the conventional fractionation arm to balance the number of measurements between the shorter fraction SBRT and longer conventional fractionation schemes.

2.A.4 | Correlation between QA gamma pass rates and plan complexity
Plan complexity was determined through calculation of the modulation degree, as given in Eq. (4), for each reference, ATP, and ATS plan in this study.
In Eq. (4), MU total represents the total number of MUs for the plan, UArea i is the open beam aperture area for beam i which is a union of all the segments for that beam, Area i,j is the aperture area for segment j in beam i, and MU i,j is the MU associated with segment j in beam i. The beam-averaged aperture area was defined as the average of all beam specific aperture areas for a given plan. A Spearman's rank correlation coefficient was used to determine if there was any correlation between modulation degree or beam-averaged aperture area to the measured gamma pass rates on the Sun Nuclear ArcCheck.

3.A | SPC analysis of Elekta unity adapted plans
A total of 65 subjects were a part of this initial MR-linac study. The patients' age ranged from 3 to 89 yr with a median age of 67.5 yr.
Some patients had boost plans or multiple treatment sites, resulting in a total of 68 reference plans. Any boost or multiple site treatment resulted in a new treatment plan and was thus treated as a new data point. Figure 3 depicts the sites that were treated and the ratio of  The C p,l , determined for the 432 measured ATP plans and 80 measured ATS plans, was calculated to be 1.403 and 0.940, respectively. The C p,l for the SBRT and conventional fractionation schemes was determined to be 0.902 and 1.383 respectively. Figure 5 shows the gamma pass rates for the SBRT and conventional fractionation analysis.

3.B | Analysis of plan complexity with QA pass rates
The maximum and minimum IMRT modulation degree was calculated to be 3.49 and 1.18, respectively, where the maximum modulation degree came from a pancreas SBRT plan. The plan modulation degree against gamma pass rate is shown in Fig. 6. A Spearman's rank correlation was performed to determine if any correlation existed between the plan modulation degree and gamma pass rate.
The Spearman's rank coefficients were calculated to be 0: Analysis of the QA pass rate compared to the beam-averaged aperture area (Fig. 7) using a Spearman's rank correlation yielded coefficients of 0:5 p ¼ 10 À5 , 0:57ðp < 10 À6 Þ, and 0:68ðp< 10 À6 Þ for the reference, ATP and ATS plans respectively. Thus, there is a mild, but statistically significant, correlation between gamma pass rate and beam-averaged aperture area.

3.C | Sensitivity to machine issues
The initial patient-specific QA performed for one reference and a few adapted plans showed pass rates below the TG-218 recommended tolerance of 0.950. Despite these measurements being above the TG-218 action threshold, they were suspiciously low compared to the other measurements. Upon further investigation, many beam interrupts resulting from gantry encoder errors occurred during the delivery of these QA plans. It was ultimately determined that a loose gantry drive gear was causing the machine faults and the gantry drive assembly was replaced. All plans measured during the time period associated with the loose gantry drive gear were reran with a fully functioning machine and no errors were observed during the delivery of these QA measurements ( Table 3). The reran plan gamma pass rates were statistically significantly higher than the initial gamma pass rates (p ¼ 7 Â 10 À6 , paired T-test).

| DISCUSSION
SPC methodology is a useful tool to extract details about a process such as patient-specific QA results. From statistical methods, a lower control limit, which in this study represents three standard deviations from the mean, can be defined. Using the process capability ratio C p,l , additional populations can be compared to a reference population such as the reference IMRT QA pass rate. The SPC methodology was useful in this work to highlight differences between the different adaptive planning methods and fractionation schemes. However, it should be noted that based on Eq. (3) that the C p,l is very sensitive to the standard deviation and the degrees of freedom of the sample population in question. Thus, one consideration when using the C p,l metric as an evaluation tool is its sensitivity to minor changes in population statistics.
Based on the results, ATS and SBRT adaptive plans had an increased variability compared to the reference plans which resulted were of SBRT plans with the smaller average beam aperture, whereas only 11% of the ATP adaptive plans (48 out of 432) were SBRT. Thus, it seems as though SBRT plans, with the smaller beamaveraged aperture area, is driving the C p,l values of the ATS plans below 1.0. Through the Spearman's rank correlation test, the beam aperture area was mildly correlated with gamma pass rate, where smaller beam aperture areas led to a lower gamma pass rate. These results were found to be statistically significant. It was also found that gamma pass rate was not correlated with modulation factor for the step-and-shoot IMRT plans studied. Beam aperture area was observed to have a more significant correlation to the gamma pass rates than the modulation degree. SBRT plans had the lowest beamaveraged aperture area and the lowest C p,l of 0.902. Similarly, ATS plans had a slightly larger beam-averaged aperture area and a correspondingly higher C p,l of 0.940.
In the context of this work, the one-sided process capability ratio can be used to describe how much of a distribution of adaptive plans' gamma pass rates are contained within the variability of the reference plans' gamma pass rates. For example, a C p,l value equal to 1.00, 0.67, and 0.33 indicate that the gamma pass rates measured for a subset of adaptive plans were all above the LCL, which was set from the distribution of gamma pass rates for the reference plans at the one-side confidence level of k = 3, k = 2, and k = 1 respectively.
Assuming that the distribution of measured gamma pass rates is An evaluation of the gamma pass rates for QA plans that were delivered during the period of gantry encoder errors showed suspiciously lower pass rates for a few deliveries. During this time, our standard gantry spoke shot and Winston-Lutz tests were performed but did not identify any issues with the machine delivery.
It is reasonable to think that the loose gantry drive gear caused errors in the gantry position during delivery, but this would only yield slight differences in the spoke shot angles or Winston-Lutz angles delivered. Consequently, this would be unlikely to cause detectable errors in these standard tests. However, as described here, the patient-specific QA was able to identify that an error was indeed occurring that was impacting the delivery of treatment plans.

| CONCLUSION
As radiation oncology treatment planning continues to evolve to include MRIgRT patient-specific treatment plans, clinics not only need to have confidence in the adapted treatment plan, but also in their treatment machine. This paper presents the results from our institution for the first 68 patients on the Unity MR-linac. The lower control limit (LCL) for the reference plans were determined to be 0.958, and the process capability ratio for ATP and ATS was found to be 1.403 and 0.940 respectively. When analyzing the data as SBRT or conventional fractionation schemes, the C p,l was found to be 0.902 and 1.383, respectively. All measurements were above TG-218 recommended tolerance providing confidence in the Unity MR-linac's performance in generating and delivering adapted plans. It was found that the beam-averaged aperture area was correlated with QA pass rate, where smaller aperture areas led to lower pass rates. In our analysis, both ATS and SBRT plans measured during this study have lower beam-averaged aperture areas and C p,l <1:0. However, all adaptive plan gamma pass rates were above TG-218 recommendations and the SPC analysis shows that adaptive plans can be expected to have acceptable pass rates provided the reference plans do. It was also found that modulation degree was not correlated with QA pass rate for either reference or adaptive plans.

A U T H O R C O N T R I B U T I O N S
We confirm that all coauthors contributed this work and agreed with the submission of this manuscript to JACMP.

CONFLI CT OF INTEREST
No conflict of interest.