Selecting a PRO-CTCAE-based subset for patient-reported symptom monitoring in prostate cancer patients: a modified Delphi procedure

Background Clinician-based reporting of adverse events leads to underreporting and underestimation of the impact of adverse events on prostate cancer patients. Therefore, interest has grown in capturing adverse events directly from patients using the Patient-Reported Outcomes (PROs) version of the Common Terminology Criteria for Adverse Events (CTCAE). We aimed to develop a standardized PRO-CTCAE subset tailored to adverse event monitoring in prostate cancer patients. Materials and methods We used a mixed-method approach based on the ‘phase I guideline for developing questionnaire modules’ by the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life group, including a literature review, and interviews with patients (n = 30) and health care providers (HCPs, n = 16). A modified Delphi procedure was carried out to reach consensus on the final subset selected from the complete PRO-CTCAE item library. Results Fourteen multidisciplinary HCPs and 12 patients participated in the Delphi rounds. Ninety percent agreed on the final subset, consisting of: ‘ability to achieve and maintain erection’, ‘decreased libido’, ‘inability to reach orgasm’, ‘urinary frequency’, ‘urinary urgency’, ‘urinary incontinence’, ‘painful urination’, ‘fecal incontinence’, ‘fatigue’, ‘hot flashes’, ‘feeling discouraged’, ‘sadness’, and ‘concentration’. From 16 articles identified in the literature review, the following adverse events for which no PRO-CTCAE items are available, were included to the recommendation section: ‘nocturia’, ‘blood and/or mucus in stool’, ‘hemorrhoids’, ‘hematuria’, ‘cystitis’, ‘neuropathy’, and ‘proctitis’. Conclusions The obtained PRO-CTCAE-subset can be used for multidisciplinary adverse event monitoring in prostate cancer care. The described method may guide development of future PRO-CTCAE subsets.


INTRODUCTION
Prostate cancer is the second most common malignancy in men worldwide, with w1.4 million new registered cases in 2020. 1 It is estimated that the incidence of prostate cancer in Europe increases with w30% from 2015 to 2040. 2 The incidence is associated with increasing age, with the highest incidence in men >65 years of age. 2 Most patients are nowadays diagnosed in a curable stage, likely caused by increased awareness, and diagnostic and treatment improvements over the last decades. 3 Five-year survival rates in European countries are currently 83%-88%. 2,4 Prostate cancer treatment can have a significantly negative and potentially remaining effect on patients' healthrelated quality of life (HRQoL). For prostatectomy, negative effects on urinary continence and erectile functioning are still reported after 5 years of follow-up. Radiotherapy is associated with nocturia and worse bowel function. 5,6 Optimal strategies to monitor and treat adverse events of treatment are warranted to overcome declined HRQoL.
Currently, clinicians report adverse events using the Common Terminology Criteria for Adverse Events (CTCAE). However, this clinician-based approach is criticized: clinicians structurally underreport and underestimate (the severity of) adverse events. [7][8][9] Furthermore, agreement between clinicians when reporting adverse events was found to be moderate, with intraclass correlation coefficients ranging from 0.46 to 0.71. 10 To overcome the gap between what clinicians report and what patients experience, interest grows in capturing adverse events directly from patients using Patient-Reported Outcomes (PROs), which may enhance its precision and comprehensiveness. Recently, Basch et al. (2016) demonstrated that integrating PRO monitoring into routine care is associated with clinical benefits, including decreased emergency room visits, improved overall survival, and less hospitalization compared to clinician-reported adverse event monitoring. 11 Subsequently, the National Cancer Institute (NCI) developed a PRO version of the CTCAE for use in clinical care and research settings. Their item library consists of 124 items evaluating the frequency, severity, and/or activity interference of 78 symptomatic adverse events. 12 Implementing the full PRO-CTCAE library in daily clinical practice for adverse event monitoring, however, is demanding and inappropriate for every tumor type or treatment. Selecting adverse events relevant for cancer types and treatments from this library is critically important to measure without bias: 13 not all adverse events are relevant for each study population, intervention, context, objective, and setting. Therefore, the purpose of this study was to develop a standardized PRO-CTCAE item subset for unambiguously monitoring adverse events in prostate cancer patients.

MATERIALS AND METHODS
To develop a standardized PRO-CTCAE item subset, we used a mixed-method approach based on the 'phase I guideline for developing questionnaire modules' by the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life group. 14 This guideline was originally developed to select relevant items for inclusion in modules specific to tumor site, treatment modality, or HRQoL dimension. Our approach included a comprehensive literature study, and semi-structured interviews with patients (n ¼ 30) and health care providers (HCPs, n ¼ 16) to identify PRO-CTCAE items most relevant for prostate cancer patients. Subsequently, we used a modified Delphi procedure to reach consensus on the final selection of items ( Figure 1). Additionally, analyses were executed stratified by treatment modality to determine whether there were items that cover treatment-specific adverse events. The Consolidated criteria for reporting qualitative research (COREQ) checklist was used for reporting about the interviews and focus groups.
The Netherlands Cancer Institute institutional review board (IRB) declared that formal approval from an ethics committee was not required, as the Dutch Medical Research (Human Subjects) Act did not apply for this study, and approved local execution of the study (IRBd19-334). All study participants gave informed consent for participating in the study.

PRO-CTCAE item library
The input was the completely validated PRO-CTCAE item library. From the 78 PRO-CTCAE items, 74 items were applicable to men; four items applicable to women were excluded in further analyses (i.e. 'irregular periods/vaginal bleeding'; 'missed expected menstrual period'; 'vaginal discharge'; 'vaginal dryness').

Comprehensive literature study
A literature search was conducted to identify the most important adverse events and potentially missing PRO-CTCAE items relevant for prostate cancer. The following search terms were used in PubMed: 'prostate cancer' OR 'prostatic neoplasms' OR 'prostatic adenocarcinoma' AND 'toxicities' OR 'adverse effects' OR 'side effects' AND 'radiotherapy' OR 'hormonal therapy' OR 'surgery' OR 'and chemotherapy'. See Supplementary Table S1

Patient interviews
Prostate cancer patients who received curative treatment (n ¼ 20) and palliative care (defined as patients with metastasized disease and/or not treated with curative intent; n ¼ 10) between September 2020 and January 2021, recruited from community (n ¼ 2) and academic hospitals (n ¼ 2) in the Netherlands, were invited to participate in semi-structured interviews. The number of included patients was dependent on data saturation (the point where no new information emerged from the interviews). Purposive sampling was used to select a broad range of patients in terms of age, treatment type, and disease state. Inclusion criteria were: !18 years of age, currently undergoing primary prostate cancer treatment or 6 months after completing treatment, ability to provide informed consent, and ability to complete the questionnaire in Dutch or English; the latter two were based on the judgment of their treating physician or nurse. Interviews were systematically documented and recorded on audiotape by the interviewer. Based on the EORTC guideline for developing questionnaire modules, 14 we developed a scripted interview guide consisting of the following steps. Firstly, patients were asked to describe their symptoms. Subsequently, they completed the full PRO-CTCAE item library; the interviewer then asked what symptoms meant to them, the extent to which they had experienced these, and if they had experienced any other symptoms. Next, they were asked to score the relevance of each item on a four-point scale (1 ¼ not relevant, 4 ¼ highly relevant; items with a mean score ! 2 were considered relevant), and to comment on the item description wordings. Also, they indicated whether there were important adverse events missing in the PRO-CTCAE list.

HCP interviews
HCPs treating prostate cancer patients were invited to participate in a semi-structured interview. The HCPs were equally distributed as specialists working in two community and two academic hospitals in the Netherlands, consisting of radiation oncologists, urologists, medical oncologists, and nurse practitioners.
A scripted interview guideline was developed following the EORTC guideline for developing questionnaire modules. 14 Firstly, HCPs were asked to score the relevance of each PRO-CTCAE item on a four-point scale (1 ¼ not relevant, 4 ¼ highly relevant; items with a mean score ! 2 were considered relevant), and to comment on the item description wordings and to explain their score. Subsequently, they were asked to add items when they felt these were missing.

The modified Delphi method
All medical specialists and patients who participated in the semi-structured interviews received an invitation to participate in the modified Delphi procedure. This is a method aimed at reaching consensus in a structured way, based on experts' opinions on a specific subject. Information and feedback is gathered through multiple questionnaire rounds, which are completed anonymously to avoid bias that may result from desire to group conformity. The method is targeted at making best use of already available information. 15 We followed the recommendations of the RAND modified Delphi method, [16][17][18][19] set up in three rounds. The first two rounds consisted of web-based questionnaires through an online survey program (Castor Electronic Data Capture). The results from each previous round were fed back anonymously in the introduction of the next round. The third round consisted of an expert and patient focus group meeting evaluating the results from the previous rounds. Lastly, all results and conclusions from the last round were sent to all participants for final consensus. The threshold for consensus was set at !75% agreement for each round. Items were excluded for the next round if <25% of both HCPs and patients considered an item to be relevant. If no consensus was reached, the item was transferred to the next round. The Delphi rounds are described in more detail in the following sections.

Expert and patient recruitment
All medical specialists and patients who participated in the semi-structured interviews received an invitation to participate in the Delphi survey rounds. Of those, 14 HCPs (88%) and 12 patients (40%) agreed to participate.

First Delphi round
The first round aimed to determine the relevance of the PRO-CTCAE items suggested in the interviews and retrieved from the literature. Questions were formulated uniformly and in the same order for HCPs and patients (from high to low relevance as scored during the interviews). For example: 'Do you think that the adverse event "urinary frequency" is relevant and should be added in the final subset?'. Participants were asked to score the relevance of each item on a four-point scale (1 ¼ not relevant, 4 ¼ highly relevant; items with a mean score ! 2 were considered relevant). The consensus for items was calculated and summarized.

Second Delphi round
The second round started with the summarized overview from the first round, administered as follows: (i) if !75% of patients but not HCPs scored an item as 'relevant', the item was only asked in the second round of the HCP survey, and vice versa; (ii) if an item scored between 25% and 75% in both expert groups, the item was asked in both HCPs and patients surveys. Participants were again asked to score the relevance of each item on a four-point scale (1 ¼ not relevant, 4 ¼ highly relevant; items with a mean score ! 2 were considered relevant).

Third Delphi round
In the third Delphi round, we aimed to reach consensus for the final items and to evaluate the included and excluded PRO-CTCAE items from the previous rounds and to give recommendation for emerged adverse events that are currently not included within the PRO-CTCAE item list. In a focus group meeting including both patient representatives as well as a multidisciplinary HCP group, moderated by one of the coordinating investigators, the expert panel strived to create one subset for patients treated with curative or palliative intent.

Finalization of the outcome subset
The final items list was presented to all Delphi participants by e-mail for formal agreement. Additional adverse events mentioned by both patients and HCPs but beyond the scope of the PRO-CTCAE were included in the recommendations section.

ESMO Open
E. Feldman et al.

Delphi procedure
Fourteen HCPs (88%) and 12 patients (40%) agreed to participate in the first round. The results from all Delphi rounds are presented in Table 2. Overall consensus (!75%) for inclusion was reached for five items: 'ability to achieve and maintain erection', 'decreased libido', 'urinary frequency', 'urinary urgency', and 'urinary incontinence'. Items that were scored as relevant by HCP, but not by patients, were 'fatigue', 'diarrhea', 'pain when urinating', 'hot flashes', 'fecal incontinence', 'breast swelling and tenderness', and 'feeling discouraged'. Items that were scored as relevant by patients, but not by the HCP, were 'delayed orgasm' and 'ejaculation'. Twenty-six items were excluded. Consensus was not reached for 46 items, which were transferred to the second round.
The third Delphi round was a focus group expert meeting for which a heterogeneous group of experts involved in prostate cancer care (N ¼ 10, radiation oncologists, urologists, medical oncologists, and nurse practitioners; and patient representatives, n ¼ 2) participated. The experts agreed uniformly on the nine items included following round 2. From the 11 items for which no consensus was reached in the second round, the following six were included: 'inability to reach orgasm', 'painful urination', 'feeling discouraged', 'sadness', 'anxiety', and 'concentration'. The following decisions were made. Firstly, 'ejaculation' was exchanged with 'inability to reach orgasm': Although 'delayed orgasm' and 'inability to reach orgasm' had the same relevance score (56%), the expert panel agreed that the spectrum of sexual dysfunction was best covered by the items 'inability to reach orgasm' and 'ability to achieve and maintain erection', making it redundant to add an extra item. Secondly, 'painful urination' was added due to its high score during the interviews and the first Delphi round. Furthermore, 'feeling discouraged' and 'sadness' were both added in order to cover the whole spectrum of patients' emotional well-being; 'anxiety' was eliminated as it was covered by the former two items and scored lower than these. Lastly, low relevance was expressed for 'concentration' (32%) and 'memory' (28%) in the Delphi rounds, but the expert panel expressed that it was of clinical importance to include at least one item covering cognitive problems. 'Concentration' was favored over 'memory' based on the higher relevance score, correlation between the two items, and importance expressed by the patients. Eighty-two percent agreed on the 55 items that were excluded in the previous rounds; two participants (9%) did not answer this question. Analyses stratified by treatment modality did not lead to the inclusion of additional items in the core subset. For radiotherapy, 'bloating', 'sleeplessness', 'shortness of breath', and 'increased sweating' were suggested to complement the subset. For radical prostatectomy, suggested items were 'bloating', 'decreased appetite', and 'constipation'. For hormone therapy (with or without additional treatments), the suggested item was 'dry mouth'. No items were suggested specifically for chemotherapy treatment.

DISCUSSION
In this multicenter mixed-methods study, we developed a consensus-based PRO-CTCAE subset applicable for patientreported adverse event monitoring during and after prostate cancer treatment. Out of 74 PRO-CTCAE items applicable to men, 13 items relevant in a multidisciplinary clinical setting for prostate cancer patients were included in the final outcome subset: 'ability to achieve and maintain erection', 'decreased libido', 'inability to reach orgasm', 'urinary frequency', 'urinary urgency', 'urinary incontinence', 'painful urination', 'fecal incontinence', 'fatigue', 'hot flashes', 'feeling discouraged', 'sadness', and 'concentration'. The final subset was agreed on by 91% of the participants.
Assessment of adverse events and HRQoL in clinical practice yields interesting information, as it will give longitudinal data about the effects of treatment from an unselected population. Primary treatment decisions have a distinct impact on prostate cancer patients for a longer period after treatment. 6 However, current evidence is still insufficient to precisely guide men about symptom risks and its subsequent effect on HRQoL. 20 More specifically, the number of relevant studies is still limited, with a high heterogeneity in methodology, applied measurements, and outcome reporting. 6,20 Measuring a pre-specified set of outcomes through a (selection of) validated outcome measure(s) following recommended methods of reporting could reduce study heterogeneity and lead to more specific treatment recommendations in the future. 20 Toward this goal, several core outcome sets were developed already. On behalf of The Prostate Cancer Working Group from the NCI Symptom Management and Health-Related Quality-of-Life Steering Committee, Chen et al. 21 suggested five items for clinical trials of localized prostate cancer patients ('urinary incontinence', 'urinary obstruction and irritation', 'bowel-related symptoms', 'sexual dysfunction', and 'hormonal symptoms') and four domains for advanced cancer ('pain', 'fatigue', 'mental well-being', and 'physical well-being'). They, however, suggest a range of validated outcome measures to choose from, therefore still adding to the heterogeneity described above. 6,20 The PRO-CTCAE subset developed in our study describes outcomes that are already operationalized in the PRO-CTCAE item library. Furthermore, Chen et al. 21 describe that PRO data interpretation remains challenging, with the exact meaning of scores and score changes remaining unclear. A benefit of the PRO-CTCAE is the assessment of frequency, severity, and/or activity interference of each symptomatic adverse event 12 that are combined into a CTCAE grade following a validated algorithm, where symptoms that worsened by !2 points or reached an absolute grade !3 are considered clinically relevant. 11,22 The International Consortium for Health Outcomes Measurement developed two separate sets for prostate cancer including adverse events, HRQoL, baseline characteristics, and survival, 23,24 but do not specifically state their rationale for this separation. 23,24 Our subset is applicable to both localized and advanced prostate cancer as sensitivity analyses showed no notably different recommendations for which items to include.
We used a structured mixed-method approach based on the 'phase 1 guideline for developing questionnaire modules' by the EORTC Quality of Life group, 14 allowing contributions from the literature, and patient and clinician perspectives. The current study may therefore serve as a guide for future development of tumor-specific PRO-CTCAEbased subsets incorporating the patients' perspective in clinical monitoring of cancer treatments. Furthermore, our sample consisted of a clinical heterogeneous patient population across different time frames, reflecting the prostate cancer population. Even though patients treated with RT (with or without additional treatment modality) were slightly overrepresented, all other common treatment modalities were also represented. Therefore, the subset allows supervision of patient adverse events during the full course of treatment and the subsequent follow-up trajectory.
The present study is limited by the national setting; international multicenter verification is still required in order to test psychometric properties. Nonetheless, since international treatment guidelines are largely comparable and the literature review revealed no additional adverse events, the subset will probably be useful in an international setting as well. Furthermore, a relatively large proportion of highly educated patients was included. As health literacy is correlated with education level, but can also differ among age and sex, 25,26 our study would have benefitted from including patients with a variety of educational levels. Nonetheless, the PRO-CTCAE questionnaire is validated in a heterogeneous patient group with varying educational levels. Therefore, the subset will probably be applicable across all educational levels. Educational levels may have affected the patients' performance status before their cancer diagnosis, and the prevalence of comorbid diseases. This may have affected the adverse effects patients and HRQoL patients experienced and thus reported, with higher educated people reporting more favorable HRQoL, 27 leading to an underestimation of the adverse effects included in the subset. We have partially evaded this by including patients in a palliative care setting, but interpretation of results could be improved by information on the patients' baseline health status.

Conclusions
To conclude, this study presents a PRO-CTCAE-based subset for multidisciplinary adverse events monitoring in prostate cancer patients during and after cancer treatment. It is anticipated that monitoring adverse events with the current PRO-CTCAE based subset for prostate cancer patients will ultimately provide a more representative reflection of patients' experiences. Furthermore, the current study may serve as a guide for future development of tumor-specific PRO-CTCAE-based subsets incorporating the patients' perspective in clinical monitoring of cancer treatments. The use of the PRO-CTCAE subset could facilitate remote monitoring: the current study was carried out during the COVID-19 pandemic, which widely impacted the health care system. Clinical implementation of the PRO-CTCAE subset could enable remote monitoring while providing crucial patient care and decreasing the risk of spreading the virus during direct contact between patient and HCPs. Furthermore, considering the high prevalence of prostate cancer, 2,4,28 remote monitoring could also take off pressure of prostate cancer care in general. This highlights the importance of the use of PRO tools for future clinical care.

GLOSSARY
Common Terminology Criteria for Adverse Events (CTCAE): reporting system for adverse events reporting, completed by clinicians. Delphi procedure: method aimed at reaching consensus in a structured way, based on experts' opinions on a specific subject. Information and feedback is gathered through multiple questionnaire rounds, which are completed anonymously to avoid bias that may result from desire to group conformity. The method is targeted at making best use of already available information. Patient-Reported Outcomes (PROs): Validated questionnaires about health-related quality of life (e.g. functioning, symptoms, adverse events, etc.) that are completed by patients.