Artificial Intelligence in Oncological Hybrid Imaging

 

 Permissions and Reprints

Abstract

Background Artificial intelligence (AI) applications have become increasingly relevant across a broad spectrum of settings in medical imaging. Due to the large amount of imaging data that is generated in oncological hybrid imaging, AI applications are desirable for lesion detection and characterization in primary staging, therapy monitoring, and recurrence detection. Given the rapid developments in machine learning (ML) and deep learning (DL) methods, the role of AI will have significant impact on the imaging workflow and will eventually improve clinical decision making and outcomes.

Methods and Results The first part of this narrative review discusses current research with an introduction to artificial intelligence in oncological hybrid imaging and key concepts in data science. The second part reviews relevant examples with a focus on applications in oncology as well as discussion of challenges and current limitations.

Conclusion AI applications have the potential to leverage the diagnostic data stream with high efficiency and depth to facilitate automated lesion detection, characterization, and therapy monitoring to ultimately improve quality and efficiency throughout the medical imaging workflow. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based therapy guidance in oncology. However, significant challenges remain regarding application development, benchmarking, and clinical implementation.

Key Points: 

• Hybrid imaging generates a large amount of multimodality medical imaging data with high complexity and depth.

• Advanced tools are required to enable fast and cost-efficient processing along the whole radiology value chain.

• AI applications promise to facilitate the assessment of oncological disease in hybrid imaging with high quality and efficiency for lesion detection, characterization, and response assessment. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based oncological therapy guidance.

• Selected applications in three oncological entities (lung, prostate, and neuroendocrine tumors) demonstrate how AI algorithms may impact imaging-based tasks in hybrid imaging and potentially guide clinical decision making.

Citation Format

• Feuerecker B, Heimer M, Geyer T et al. Artificial Intelligence in Oncological Hybrid Imaging. Fortschr Röntgenstr 2023; 195: 105 – 114

Zusammenfassung

Hintergrund Der Stellenwert künstlicher Intelligenz (KI) hat in der medizinischen Bildgebung in den letzten Jahren deutlich zugenommen. Aufgrund der enormen Datenmengen und strukturierbaren Aufgaben im diagnostischen Workflow hat die KI in der onkologischen Hybridbildgebung besonders vielversprechende Anwendungsgebiete für die Läsionsdetektion, die Läsionscharakterisierung und die Therapiebeurteilung. Vor dem Hintergrund rasanter Entwicklungen im Bereich des Machine Learning (ML) und des Deep Learning (DL) ist von einer zunehmenden Bedeutung in der onkologischen Hybridbildgebung auszugehen mit Potenzial, die klinische Therapiesteuerung und patientenrelevante Ergebnisse zu verbessern.

Methode und Ergebnisse Diese narrative Übersichtsarbeit fasst die Evidenz in verschiedenen aufgabenbezogenen Anwendungen der Bildanalyse von KI im Bereich der onkologischen Hybridbildgebung zusammen. Nach Einführung in das Thema der KI werden ausgewählte Beispiele exploriert, vor dem Hintergrund aktueller Herausforderungen und im Hinblick auf die klinische Relevanz in der Therapiesteuerung diskutiert.

Schlussfolgerung Der Einsatz von KI bietet vielversprechende Anwendungen der Detektion, der Charakterisierung und der longitudinalen Therapiebeurteilung im Bereich der onkologischen Hybridbildgebung. Schlüsselherausforderungen liegen in den Bereichen der Entwicklung von Algorithmen, der Validierung und der klinischen Implementierung.

Kernaussagen: 

• Mit der onkologischen Hybridbildgebung werden große Datenvolumen aus 2 bildgebenden Modalitäten erzeugt, deren strukturierte Analyse komplex ist.

• Für die Datenanalyse werden neue Methoden benötigt, um eine schnelle und kosteneffiziente Beurteilung in allen Aspekten der diagnostischen Wertschöpfungskette zu ermöglichen.

• KI verspricht, die diagnostische Auswertung der onkologischen Hybridbildgebung zu vereinfachen und wesentliche Verbesserungen in Qualität und Effizienz bei der Erkennung, Charakterisierung und dem longitudinalen Monitoring onkologischer Erkrankungen zu ermöglichen. Ziel ist reproduzierbare, strukturierte, quantitative diagnostische Daten für die evidenzbasierte onkologische Therapiesteuerung zu generieren.

• Selektierte Anwendungsbeispiele in 3 ausgewählten Tumorentitäten (Lungenkarzinom, Prostatakarzinom, Neuroendokrine Tumore) zeigen wie KI-gestützte Applikationen einen wesentlichen Beitrag in der automatisierten Bildanalyse leisten und eine weitere Individualisierung von Therapien ermöglichen könnten.

^* Benedikt Feuerecker and Maurice M. Heimer contributed equally as first author.

Publication History

Received: 13 March 2022

Accepted: 11 July 2022

Article published online:
28 September 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

• References

• 1 Moore JH. et al. Preparing next-generation scientists for biomedical big data: artificial intelligence approaches. Per Med 2019; 16 (03) 247-257
  CrossrefPubMedGoogle Scholar
• 2 Rueckel J. et al. Artificial Intelligence Algorithm Detecting Lung Infection in Supine Chest Radiographs of Critically Ill Patients With a Diagnostic Accuracy Similar to Board-Certified Radiologists. Crit Care Med 2020; 48 (07) e574-e583
  CrossrefPubMedGoogle Scholar
• 3 Oosterhoff JHF, Doornberg JN, Doornberg C. Machine Learning, Artificial intelligence in orthopaedics: false hope or not? A narrative review along the line of Gartner’s hype cycle. EFORT Open Rev 2020; 5 (10) 593-603
  CrossrefPubMedGoogle Scholar
• 4 Sollini M. et al. Artificial intelligence and hybrid imaging: the best match for personalized medicine in oncology. Eur J Hybrid Imaging 2020; 4 (01) 24
  CrossrefPubMedGoogle Scholar
• 5 Capobianco E. High-dimensional role of AI and machine learning in cancer research. Br J Cancer 2022; 126 (04) 523-532
  CrossrefPubMedGoogle Scholar
• 6 Shiyam Sundar LK. et al. Potenzials and caveats of AI in hybrid imaging. Methods 2021; 188: 4-19
  CrossrefPubMedGoogle Scholar
• 7 Zaharchuk G. Next generation research applications for hybrid PET/MR and PET/CT imaging using deep learning. Eur J Nucl Med Mol Imaging 2019; 46 (13) 2700-2707
  CrossrefPubMedGoogle Scholar
• 8 Aide N. et al. New PET technologies – embracing progress and pushing the limits. Eur J Nucl Med Mol Imaging 2021; 48 (09) 2711-2726
  CrossrefPubMedGoogle Scholar
• 9 Hosny A. et al. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18 (08) 500-510
  CrossrefPubMedGoogle Scholar
• 10 Pe’er D. et al. Tumor heterogeneity. Cancer Cell 2021; 39 (08) 1015-1017
  CrossrefPubMedGoogle Scholar
• 11 Ju W. et al. Random Walk and Graph Cut for Co-Segmentation of Lung Tumor on PET-CT Images. IEEE Trans Image Process 2015; 24 (12) 5854-5867
  CrossrefPubMedGoogle Scholar
• 12 Pfannenberg C. et al. Practice-based evidence for the clinical benefit of PET/CT-results of the first oncologic PET/CT registry in Germany. Eur J Nucl Med Mol Imaging 2019; 46 (01) 54-64
  CrossrefPubMedGoogle Scholar
• 13 Mottet N. et al. EAU-EANM-ESTRO-ESUR-SIOG Guidelines on Prostate Cancer-2020 Update. Part 1: Screening, Diagnosis, and Local Treatment with Curative Intent. Eur Urol 2021; 79 (02) 243-262
  CrossrefPubMedGoogle Scholar
• 14 AWMF. Leitlinienprogramm Onkologie (Deutsche Krebsgesellschaft, Deutsche Krebshilfe, AWMF): S3-Leitlinie Prostatakarzinom, Langversion 6.0, 2021, AWMF Registernummer: 043/022OL. (abgerufen am: 02.08.2021) http://www.leitlinienprogramm-onkologie.de/leitlinien/prostatakarzinom/
  PubMedGoogle Scholar
• 15 Zhao X. et al. Tumor co-segmentation in PET/CT using multi-modality fully convolutional neural network. Phys Med Biol 2018; 64 (01) 015011
  CrossrefPubMedGoogle Scholar
• 16 Bradshaw TJ. et al. Nuclear Medicine and Artificial Intelligence: Best Practices for Algorithm Development. J Nucl Med 2021; DOI: 10.2967/jnumed.121.262567.
  CrossrefPubMedGoogle Scholar
• 17 Pinto Dos Santos D, Dietzel M, Baessler B. A decade of radiomics research: are images really data or just patterns in the noise?. Eur Radiol 2021; 31 (01) 1-4
  CrossrefPubMedGoogle Scholar
• 18 Gatta R. et al. Integrating radiomics into holomics for personalised oncology: from algorithms to bedside. Eur Radiol Exp 2020; 4 (01) 11
  CrossrefPubMedGoogle Scholar
• 19 Holzinger A. et al. Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip Rev Data Min Knowl Discov 2019; 9 (04) e1312
  CrossrefPubMedGoogle Scholar
• 20 Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016; 278 (02) 563-577
  CrossrefPubMedGoogle Scholar
• 21 Sepehri S. et al. Comparison and Fusion of Machine Learning Algorithms for Prospective Validation of PET/CT Radiomic Features Prognostic Value in Stage II-III Non-Small Cell Lung Cancer. Diagnostics (Basel) 2021; 11 (04) DOI: 10.3390/diagnostics11040675.
  CrossrefPubMedGoogle Scholar
• 22 Chartrand G. et al. Deep Learning: A Primer for Radiologists. Radiographics 2017; 37 (07) 2113-2131
  CrossrefPubMedGoogle Scholar
• 23 Litjens G. et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88
  CrossrefPubMedGoogle Scholar
• 24 Sanli Y. et al. Neuroendocrine Tumor Diagnosis and Management: (68)Ga-DOTATATE PET/CT. Am J Roentgenol 2018; 211 (02) 267-277
  CrossrefPubMedGoogle Scholar
• 25 El-Galaly TC. et al. FDG-PET/CT in the management of lymphomas: current status and future directions. J Intern Med 2018; 284 (04) 358-376
  CrossrefPubMedGoogle Scholar
• 26 Rakheja R. et al. Necrosis on FDG PET/CT correlates with prognosis and mortality in sarcomas. Am J Roentgenol 2013; 201 (01) 170-177
  CrossrefPubMedGoogle Scholar
• 27 Bettinelli A. et al. A Novel Benchmarking Approach to Assess the Agreement among Radiomic Tools. Radiology 2022; 211604 DOI: 10.1148/radiol.211604.
  CrossrefPubMedGoogle Scholar
• 28 van Timmeren JE. et al. Radiomics in medical imaging-“how-to” guide and critical reflection. Insights Imaging 2020; 11 (01) 91
  CrossrefPubMedGoogle Scholar
• 29 Li K, Zhang R, Cai W. Deep learning convolutional neural network (DLCNN): unleashing the potential of (18)F-FDG PET/CT in lymphoma. Am J Nucl Med Mol Imaging 2021; 11 (04) 327-331
  PubMedGoogle Scholar
• 30 Mongan J, Moy L, Kahn Jr CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. Radiol Artif Intell 2020; 2 (02) e200029
  CrossrefPubMedGoogle Scholar
• 31 Lambin P. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017; 14 (12) 749-762
  CrossrefPubMedGoogle Scholar
• 32 Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health 2021; 3 (03) e195-e203
  CrossrefPubMedGoogle Scholar
• 33 Luchini C, Pea A, Scarpa A. Artificial intelligence in oncology: current applications and future perspectives. Br J Cancer 2022; 126 (01) 4-9
  CrossrefPubMedGoogle Scholar
• 34 Sibille L. et al. (18)F-FDG PET/CT Uptake Classification in Lymphoma and Lung Cancer by Using Deep Convolutional Neural Networks. Radiology 2020; 294 (02) 445-452
  CrossrefPubMedGoogle Scholar
• 35 Blanc-Durand P. et al. Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG-PET/CT for total metabolic tumour volume prediction using a convolutional neural network. Eur J Nucl Med Mol Imaging 2021; 48 (05) 1362-1370
  CrossrefPubMedGoogle Scholar
• 36 Qu YH. et al. The correlation of (18)F-FDG PET/CT metabolic parameters, clinicopathological factors, and prognosis in breast cancer. Clin Transl Oncol 2021; 23 (03) 620-627
  CrossrefPubMedGoogle Scholar
• 37 Lohmann P. et al. PET/MRI Radiomics in Patients With Brain Metastases. Front Neurol 2020; 11: 1
  CrossrefPubMedGoogle Scholar
• 38 Nakajo M. et al. Machine learning based evaluation of clinical and pretreatment (18)F-FDG-PET/CT radiomic features to predict prognosis of cervical cancer patients. Abdom Radiol (NY) 2022; 47 (02) 838-847
  CrossrefPubMedGoogle Scholar
• 39 AWMF. Leitlinienprogramm Onkologie (Deutsche Krebsgesellschaft, Deutsche Krebshilfe, AWMF): Prävention, Diagnostik, Therapie und Nachsorge des Lungenkarzinoms, Langversion 1.0, 2018, AWMF-Registernummer: 020/007OL. 2018 (Zugriff am: 09.04.2022) http://leitlinienprogramm-onkologie.de/Lungenkarzinom.98.0.html
  PubMedGoogle Scholar
• 40 Kandathil A. et al. Role of FDG PET/CT in the Eighth Edition of TNM Staging of Non-Small Cell Lung Cancer. Radiographics 2018; 38 (07) 2134-2149
  CrossrefPubMedGoogle Scholar
• 41 UyBico SJ. et al. Lung cancer staging essentials: the new TNM staging system and potential imaging pitfalls. Radiographics 2010; 30 (05) 1163-1181
  CrossrefPubMedGoogle Scholar
• 42 Thakur SK, Singh DP, Choudhary J. Lung cancer identification: a review on detection and classification. Cancer Metastasis Rev 2020; 39 (03) 989-998
  CrossrefPubMedGoogle Scholar
• 43 Venkadesh KV. et al. Deep Learning for Malignancy Risk Estimation of Pulmonary Nodules Detected at Low-Dose Screening CT. Radiology 2021; 300 (02) 438-447
  CrossrefPubMedGoogle Scholar
• 44 Binczyk F. et al. Radiomics and artificial intelligence in lung cancer screening. Transl Lung Cancer Res 2021; 10 (02) 1186-1199
  CrossrefPubMedGoogle Scholar
• 45 van Klaveren RJ. et al. Management of lung nodules detected by volume CT scanning. N Engl J Med 2009; 361 (23) 2221-2229
  CrossrefPubMedGoogle Scholar
• 46 Werner S. et al. Accuracy and Reproducibility of a Software Prototype for Semi-Automated Computer-Aided Volumetry of the solid and subsolid Components of part-solid Pulmonary Nodules. Fortschr Röntgenstr 2022; 194 (03) 296-305
  Article in Thieme ConnectPubMedGoogle Scholar
• 47 Borrelli P. et al. Freely available convolutional neural network-based quantification of PET/CT lesions is associated with survival in patients with lung cancer. EJNMMI Phys 2022; 9 (01) 6
  CrossrefPubMedGoogle Scholar
• 48 Wallis D. et al. An [18F]FDG-PET/CT deep learning method for fully automated detection of pathological mediastinal lymph nodes in lung cancer patients. Eur J Nucl Med Mol Imaging 2022; 49 (03) 881-888
  CrossrefPubMedGoogle Scholar
• 49 Mu W. et al. Non-invasive decision support for NSCLC treatment using PET/CT radiomics. Nat Commun 2020; 11 (01) 5228
  CrossrefPubMedGoogle Scholar
• 50 Fendler WP. et al. (68)Ga-PSMA PET/CT: Joint EANM and SNMMI procedure guideline for prostate cancer imaging: version 1.0. Eur J Nucl Med Mol Imaging 2017; 44 (06) 1014-1024
  CrossrefPubMedGoogle Scholar
• 51 Hofman MS. et al Prostate-specific membrane antigen PET-CT in patients with high-risk prostate cancer before curative-intent surgery or radiotherapy (proPSMA): a prospective, randomised, multicentre study. Lancet 2020; 395 10231 1208-1216
  CrossrefPubMedGoogle Scholar
• 52 Calais J. et al. Potenzial Impact of (68)Ga-PSMA-11 PET/CT on the Planning of Definitive Radiation Therapy for Prostate Cancer. J Nucl Med 2018; 59 (11) 1714-1721
  CrossrefPubMedGoogle Scholar
• 53 Grubmuller B. et al. PSMA Ligand PET/MRI for Primary Prostate Cancer: Staging Performance and Clinical Impact. Clin Cancer Res 2018; 24 (24) 6300-6307
  CrossrefPubMedGoogle Scholar
• 54 Kostyszyn D. et al. Intraprostatic Tumor Segmentation on PSMA PET Images in Patients with Primary Prostate Cancer with a Convolutional Neural Network. J Nucl Med 2021; 62 (06) 823-828
  CrossrefPubMedGoogle Scholar
• 55 Erle A. et al. Evaluating a Machine Learning Tool for the Classification of Pathological Uptake in Whole-Body PSMA-PET-CT Scans. Tomography 2021; 7 (03) 301-312
  CrossrefPubMedGoogle Scholar
• 56 Gafita A. et al. qPSMA: Semiautomatic Software for Whole-Body Tumor Burden Assessment in Prostate Cancer Using (68)Ga-PSMA11 PET/CT. J Nucl Med 2019; 60 (09) 1277-1283
  CrossrefPubMedGoogle Scholar
• 57 Gafita A. et al. Tumor sink effect in (68)Ga-PSMA-11 PET: Myth or Reality?. J Nucl Med 2021; DOI: 10.2967/jnumed.121.261906.
  CrossrefPubMedGoogle Scholar
• 58 Moazemi S. et al. Decision-support for treatment with (177)Lu-PSMA: machine learning predicts response with high accuracy based on PSMA-PET/CT and clinical parameters. Ann Transl Med 2021; 9 (09) 818
  CrossrefPubMedGoogle Scholar
• 59 Moazemi S. et al. Estimating the Potenzial of Radiomics Features and Radiomics Signature from Pretherapeutic PSMA-PET-CT Scans and Clinical Data for Prediction of Overall Survival When Treated with (177)Lu-PSMA. Diagnostics (Basel) 2021; 11 (02) DOI: 10.3390/diagnostics11020186.
  CrossrefPubMedGoogle Scholar
• 60 Papp L. et al. Supervised machine learning enables non-invasive lesion characterization in primary prostate cancer with [(68)Ga]Ga-PSMA-11 PET/MRI. Eur J Nucl Med Mol Imaging 2021; 48 (06) 1795-1805
  CrossrefPubMedGoogle Scholar
• 61 Fehrenbach U. et al. Automatized Hepatic Tumor Volume Analysis of Neuroendocrine Liver Metastases by Gd-EOB MRI-A Deep-Learning Model to Support Multidisciplinary Cancer Conference Decision-Making. Cancers (Basel) 2021; 13 (11) DOI: 10.3390/cancers13112726.
  CrossrefPubMedGoogle Scholar
• 62 Hicks RJ. et al. ENETS standardized (synoptic) reporting for molecular imaging studies in neuroendocrine tumours. J Neuroendocrinol 2021; e13040 DOI: 10.1111/jne.13040.
  CrossrefPubMedGoogle Scholar
• 63 Dromain C. et al. ENETS standardized (synoptic) reporting for radiological imaging in neuroendocrine tumours. J Neuroendocrinol 2021; e13044 DOI: 10.1111/jne.13040.
  CrossrefPubMedGoogle Scholar
• 64 Ambrosini V. et al. Consensus on molecular imaging and theranostics in neuroendocrine neoplasms. Eur J Cancer 2021; 146: 56-73
  CrossrefPubMedGoogle Scholar
• 65 Werner RA. et al. SSTR-RADS Version 1.0 as a Reporting System for SSTR PET Imaging and Selection of Potenzial PRRT Candidates: A Proposed Standardization Framework. J Nucl Med 2018; 59 (07) 1085-1091
  CrossrefPubMedGoogle Scholar
• 66 Gao X, Wang X. Deep learning for World Health Organization grades of pancreatic neuroendocrine tumors on contrast-enhanced magnetic resonance images: a preliminary study. Int J Comput Assist Radiol Surg 2019; 14 (11) 1981-1991
  CrossrefPubMedGoogle Scholar
• 67 Luo Y. et al. Preoperative Prediction of Pancreatic Neuroendocrine Neoplasms Grading Based on Enhanced Computed Tomography Imaging: Validation of Deep Learning with a Convolutional Neural Network. Neuroendocrinology 2020; 110 (05) 338-350
  CrossrefPubMedGoogle Scholar
• 68 Atkinson C. et al. Radiomics-Based Texture Analysis of (68)Ga-DOTATATE Positron Emission Tomography and Computed Tomography Images as a Prognostic Biomarker in Adults With Neuroendocrine Cancers Treated With (177)Lu-DOTATATE. Front Oncol 2021; 11: 686235
  CrossrefPubMedGoogle Scholar
• 69 Liberini V. et al. Impact of segmentation and discretization on radiomic features in (68)Ga-DOTA-TOC PET/CT images of neuroendocrine tumor. EJNMMI Phys 2021; 8 (01) 21
  CrossrefPubMedGoogle Scholar
• 70 Liberini V. et al. The Challenge of Evaluating Response to Peptide Receptor Radionuclide Therapy in Gastroenteropancreatic Neuroendocrine Tumors: The Present and the Future. Diagnostics (Basel) 2020; 10 (12) DOI: 10.3390/diagnostics10121083.
  CrossrefPubMedGoogle Scholar
• 71 Onner H, Abdulrezzak U, Tutus A. Could the skewness and kurtosis texture parameters of lesions obtained from pretreatment Ga-68 DOTA-TATE PET/CT images predict receptor radionuclide therapy response in patients with gastroenteropancreatic neuroendocrine tumors?. Nucl Med Commun 2020; 41 (10) 1034-1039
  CrossrefPubMedGoogle Scholar
• 72 Wehrend J. et al. Automated liver lesion detection in (68)Ga DOTATATE PET/CT using a deep fully convolutional neural network. EJNMMI Res 2021; 11 (01) 98
  CrossrefPubMedGoogle Scholar
• 73 Sonni I. et al. Impact of (68)Ga-PSMA-11 PET/CT on Staging and Management of Prostate Cancer Patients in Various Clinical Settings: A Prospective Single-Center Study. J Nucl Med 2020; 61 (08) 1153-1160
  CrossrefPubMedGoogle Scholar
• 74 Barrio M. et al. The Impact of Somatostatin Receptor-Directed PET/CT on the Management of Patients with Neuroendocrine Tumor: A Systematic Review and Meta-Analysis. J Nucl Med 2017; 58 (05) 756-761
  CrossrefPubMedGoogle Scholar
• 75 Yin Q. et al. Associations between Tumor Vascularity, Vascular Endothelial Growth Factor Expression and PET/MRI Radiomic Signatures in Primary Clear-Cell-Renal-Cell-Carcinoma: Proof-of-Concept Study. Sci Rep 2017; 7: 43356
  CrossrefPubMedGoogle Scholar
• 76 Srimathi S, Yamuna G, Nanmaran R. An Efficient Cancer Classification Model for CT/MRI/PET Fused Images. Curr Med Imaging 2021; 17 (03) 319-330
  CrossrefPubMedGoogle Scholar