Validation and implementation of a mobile app decision support system for prostate cancer to improve quality of tumor boards

Certified Cancer Centers must present all patients in multidisciplinary tumor boards (MTB), including standard cases with well-established treatment strategies. Too many standard cases can absorb much of the available time, which can be unfavorable for the discussion of complex cases. In any case, this leads to a high quantity, but not necessarily a high quality of tumor boards. Our aim was to develop a partially algorithm-driven decision support system (DSS) for smart phones to provide evidence-based recommendations for first-line therapy of common urological cancers. To assure quality, we compared each single digital decision with recommendations of an experienced MTB and obtained the concordance.1873 prostate cancer patients presented in the MTB of the urological department of the University Hospital of Cologne from 2014 to 2018 have been evaluated. Patient characteristics included age, disease stage, Gleason Score, PSA and previous therapies. The questions addressed to MTB were again answered using DSS. All blinded pairs of answers were assessed for discrepancies by independent reviewers. Overall concordance rate was 99.1% (1856/1873). Stage specific concordance rates were 97.4% (stage I), 99.2% (stage II), 100% (stage III), and 99.2% (stage IV). Quality of concordance were independent of age and risk profile. The reliability of any DSS is the key feature before implementation in clinical routine. Although our system appears to provide this safety, we are now performing cross-validation with several clinics to further increase decision quality and avoid potential clinic bias.


Framework or theory involved in the methods
The use and potential of expert-curated-decision support versus AI-based approaches for providing oncology treatment recommendations was described by the work of Bungartz et al in Nature Biotechnology 1 . For the purpose of good comparability, we considered it a legitimate approach to methodically use the preliminary work on Watson for Oncology as also published in PLOS One for our work. [2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21] In particular, the work of Yu et al on the use of Watson for Oncology in 201 prostate cancer patients provided a useful comparison here using the same methods, noting the quantitatively superior analysis of 1873 patients in our work. 16

Rationale for statistical analysis including appropriate statistical analyses/methods
Descriptive statistics and data analysis were carried out using IBM´s statistics software SPSS Version 25 and Microsoft Excel. The patient characteristics age, cancer stage, risk stratification, Gleason Score, and PSA-level (prostate specific antigen) were documented. Descriptive statistics were depicted as number of percentages or mean ± standard deviation (SD). After assigning patients to the concordant or the non-concordant group, a chi-squared test was used to compare categorical variables and the Mann-Whitney U test was applied to compare ordinal variables between the groups. Multivariate logistic regression analysis was used to analyze the association between the concordance rate and clinicopathological data. Statistical significance was assumed if the p-value was < 0.05 for all statistical analysis. Graphics, charts and tables were generated using SPSS, Microsoft Excel and Power Point.

Rationale for the study design and why a design like RCT wasn't chosen
The reason for the study rationale is provided in point 1 described above. Randomization was not useful given that the data were collected retrospectively. Of course, the respective pairs (App and tumor board) of each treatment decision were blinded for the examination of concordance to avoid investigator bias. However, the current work should serve as a basis to enable a prospective evaluation. Currently, the technical implementation of supporting query algorithms is taking place through an API into existing hospital information systems, which will enable automatic therapy recommendations based on EasyOncology. This will then provide the opportunity for a prospective cohort study, with clinics with decisionsupported tumor boards and clinics without digital tumor board support serving as cohorts to be compared.

Inadequate details on the EasyOncology application and this could be improved in the main text
The content of EasyOncology was developed by experienced specialists from all relevant oncological specialties with the intention to provide evidence-based and unbiased first-line treatment recommendations for the most common cancer entities. The intuitive user interface and quality of the application led to a top 3 ranking in a worldwide comparison of 157 oncological applications in 2017. The content and therapeutic recommendations are based on expert-curated knowledge that is assimilated and interpreted from written and visual sources alike. More precisely, given oncological guidelines (i.e. S3-guidelines, NCCN), approval status of therapeutics and best-clinical practice of major cancer centers build the basis for EO's data platform. Query algorithms have been developed to effectively support the decision-making process, especially in complex cancer cases. EasyOncology was certified as a medical device in July 2020. To assure quality of given treatment recommendations, a continuous comparison with real-word treatment recommendations of various MTDs has been implemented. For this study, clinicopathological data of an uro-oncological MTD, namely the patient characteristics age, cancer stage, risk stratification, Gleason Score, and PSA-level (prostate specific antigen), were assessed to provide treatment recommendations using EasyOncology.

Funding source of this study-Needs to be explicit. How was this study funded?
The study was conducted as part of the authors' university research activities and did not receive separate funding. The EasyOncology app was developed and published by the company Easy Medical