Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.


Introduction
Hepatocellular Carcinoma (HCC), the most common primary liver malignancy, is linked to high mortality rates and stands as a leading cause of cancer-related deaths worldwide (1).Accurate diagnosis and staging of HCC are crucial for improving patient survival rates and treatment outcomes.However, early diagnosis of HCC presents a significant challenge, especially for individuals with chronic liver disease.A notable characteristic of liver cancer is its strong association with liver fibrosis, with over 80% of hepatocellular carcinomas (HCCs) developing in fibrotic or cirrhotic livers (2).This indicates that liver fibrosis plays a vital role in the liver's premalignant environment.
Medical imaging techniques, including Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Ultrasound (US), play an essential role in the diagnosis and staging of HCC, supplementing clinical findings, biological markers, and blood tests.CT scans provide detailed cross-sectional images of the liver, aiding in the identification and characterization of tumors (3).MRI offers superior soft tissue contrast, making it invaluable for assessing the extent of liver cancer (4).US, a non-invasive and costeffective imaging modality, can detect liver tumors by generating liver images using sound waves (5).However, each of these imaging methods has its limitations.For instance, CT scans expose patients to ionizing radiation, potentially heightening the risk of radiationinduced cancer.Moreover, CT scans can be expensive and less accessible in certain healthcare settings.While MRI can produce high-quality images, it can be time-consuming and may not be suitable for patients with claustrophobia or those with metal implants.US has limitations in image quality, particularly in patients with obesity or excessive intestinal gas.Recently, advanced MRI techniques, such as MR Elastography (MRE) and gadoxetic acid-enhanced MRI, have been introduced for liver imaging.These techniques provide high-resolution images without the harmful effects of radiation (6).MRE measures the stiffness of liver tissue, which can assist in differentiating between benign and malignant liver tumors.Gadoxetic acid-enhanced MRI offers dynamic imaging of the liver and can enhance the detection and characterization of HCC.
Diagnosing HCC poses significant challenges.These challenges arise from the prevalence of typical radiological features that are common to other liver tumors or benign conditions.Such similarities in imaging characteristics can lead to misdiagnosis or delayed diagnosis.As a result, patients with liver lesions exhibiting these typical features may require histological confirmation or rigorous monitoring to ensure accurate diagnosis and appropriate treatment.
In recent years, the potential of Artificial Intelligence (AI) techniques in diagnosing HCC has been the subject of extensive research.These techniques have been explored for various purposes such as detecting and evaluating HCC, facilitating treatment, and predicting treatment response (7)(8)(9)(10)(11)(12)(13). Numerous studies have investigated the use of AI models in conjunction with different modalities, including electronic health record (EHR) reports, clinical parameters, biological markers, and blood test results, for diagnosing liver cancer (14,15).AI techniques have emerged as powerful tools capable of extracting valuable insights from voluminous EHRs and developing multimodal AI methods.These methods provide a more comprehensive and accurate depiction of the liver's internal structure and function.
While many researchers have shown interest in exploring the potential of AI techniques in liver cancer research, there remains a gap in comprehensively evaluating the implementation of singlemodal and multi-modal AI techniques for diagnosing HCC.This study aims to bridge this gap by providing a comprehensive review of the most recently developed AI-based techniques that utilize both single and multi-modal data for diagnosing HCC.AI-based techniques hold the potential to enhance early diagnosis, improve diagnostic accuracy, and improve treatment outcomes for patients with HCC.This pivotal area of research could lead to significant advancements in liver cancer diagnosis and prediction.

Methodology and materials
This research explores the application of AI methodologies in diagnosing and prognosticating primary liver cancer, specifically HCC.The objective is to encapsulate the latest and most relevant discoveries in this rapidly evolving field.
A thorough literature review was conducted using databases such as PubMed, Scopus, Semantic Scholar, IEEEXplore, and Web of Science, up until March 31, 2024.During this process, several key terms such as "artificial intelligence", "deep learning", "machine learning", "liver cancer", "hepatocellular carcinoma", "multimodal", "medical imaging", "US", "CT", and "MRI" were searched in the title and/or abstract or all field.References from relevant articles were examined to identify additional qualifying publications.
An expert review of the eligible literature was carried out, and the most informative and pertinent citations were chosen for inclusion.The studies selected were those that integrated AI techniques with medical imaging datasets, including US, CT, and MRI, in conjunction with Electronic Health Records (EHR) and clinical parameters.Studies that did not utilize medical imaging techniques or AI models specifically targeting primary liver cancer were excluded.
The search was confined to peer-reviewed articles, conference proceedings, dissertations, and book chapters published in English from January 2010 to March 2024.These publications were retrieved, screened, and reviewed by the authors.One researcher then undertook the data extraction, focusing on the methods and results of each study.
As depicted in Figure 1, our study selection process began with 1334 records.After removing 885 duplicates, we screened 450 records.The title and abstract screening led to the exclusion of 240 studies, leaving 210 for full-text review.Following a comprehensive evaluation, 177 articles (7, were deemed suitable for this study.We categorized the modalities into four groups: US (n = 34), CT (n = 95), MRI (n = 34), and multi-modal (n=19).The characteristics of the included studies are detailed in Tables 1-10.

Artificial intelligence techniques
AI techniques, including Machine Learning (ML) and Deep Learning (DL), have been extensively investigated in application and interest within the field of liver cancer research (187)(188)(189)(190). ML utilizes data to develop algorithms that can identify specific behavioral patterns and build predictive models.The objective of ML is to create a model that leverages statistical dependencies and correlations within a dataset, eliminating the need for explicit programming.This process is divided into two stages: training and validation.During the training stage, the model is exposed to a portion of the available data (training dataset).In the validation stage, the model's performance is evaluated on a separate subset of Flowchart of study selection.the dataset (test dataset) to assess its ability to generalize its training performance to unseen data.Well-known ML algorithms, such as Support Vector Machines (SVM) and Artificial Neural Networks (ANNs), have been applied in HCC management (191,192).DL technology, a subset of ML, has shown remarkable efficacy in the analysis of liver images.This is largely due to its ability to process large volumes of data through multiple layers of artificial neurons.These neurons are engineered to emulate the intricate structure of the human brain and its biological neural networks.A unique characteristic of DL algorithms is that these layers of features are not manually constructed with human expertise.Rather, they are autonomously learned from data using a general-purpose learning procedure.This facilitates an end-to-end mapping from the input to the output, essentially converting the image into classification methods.In ML methods, success is contingent upon accurate segmentation and the selection of expert-designed features.DL approaches can surmount these limitations as they can identify the regions of the image most associated with the outcome through self-training.Moreover, they can discern the features of the region that informed the decision through multiple layers.Convolutional Neural Networks (CNNs) are presently the most prevalent DL algorithms employed for the diagnosis and management of HCC (193)(194)(195).The uniqueness of CNNs compared to Fully Connected Networks lies in their ability to capture spatial hierarchies through convolutional and pooling layers, their parameter efficiency due to shared weights, and their effectiveness in processing structured data like images and videos.The fundamental principles of CNNs include local connections, shared weights, pooling, and the use of numerous layers.These components collectively enhance the accuracy and efficiency of the entire system.A standard CNN model is composed of an input layer, an output layer, and several hidden layers.These hidden layers encompass convolutional layers, pooling layers, and fullyconnected layers.By repeatedly applying convolution and pooling, fully-connected layers are subsequently utilized for classification or predictions.There exists a variety of layer combinations, and numerous Deep Neural Network (DNN) architectures have been successfully implemented for HCC diagnosis and prediction.These include Fully Convolutional Networks (FCNs) (196), 3D U-Net (197), Recurrent Neural Networks (RNNs) (198), Graph Convolutional Networks (GCNs) (199,200), Generative Adversarial Networks (GANs) (16,201), AlexNet (202), and VGGNet-19 (203).These models are specifically engineered to eliminate fully connected layers and restore spatial dimensions, thereby augmenting DL capabilities even when there is a scarcity of labeled data.However, it is imperative to address domain adaptation and dataset bias to ensure the success of transfer learning (TL).This is because these factors can significantly influence the performance and generalizability of the models.
In contrast to CNNs, Fully Convolutional Networks (FCNs) are engineered to preserve spatial information, thereby enhancing their effectiveness for pixel-level predictions.This attribute renders FCNs particularly apt for liver tumor segmentation, as they employ convolutional layers in lieu of fully connected ones (196).
U-Net, conversely, utilizes an encoder-decoder model equipped with skip connections.This architecture enables it to amalgamate local and global context information, thereby augmenting object localization precision.Despite the limitations posed by scarce training data, 3D U-Net has exhibited remarkable results in the classification of liver lesions (197).
RNNs, encompassing Long Short-Term Memory (LSTM) and Gated Recurrence Unit (GRU), are specifically tailored to scrutinize sequential data by capturing temporal dependencies.These models have been successfully deployed for predicting HCC recurrence post liver transplantation (198).By addressing the vanishing gradients issue and capitalizing on temporal dependencies, they have substantially enhanced prediction accuracy.
Graph Convolutional Networks (GCNs) offer a variety of techniques for graph convolution, which are instrumental in clinically predicting Microvascular Invasion (MVI) in Hepatocellular Carcinoma (HCC) (199).These techniques include spectral-based and spatial-based GCN approaches, each carrying unique computational implications.DenseGCN, a contemporary architecture, has been introduced for the identification of liver cancer.It integrates advanced techniques such as similarity network fusion and denoising autoencoders, significantly boosting detection accuracy (200).
Generative Adversarial Networks (GANs) have demonstrated their value in generating synthetic images and augmenting data across a range of medical applications.In the realm of liver tumor detection, Tripartite GAN offers a cost-effective and non-invasive alternative by generating contrast-enhanced MRI images, eliminating the need for contrast agent injection (201).Another promising application is the Mask-Attention GAN, which generates realistic tumor images in CT scans for training and evaluation purposes (16).
Transfer Learning (TL) strategies have been employed in the field of medical imaging to mitigate overfitting issues arising from limited data.Within the TL framework, knowledge can be shared and transferred between different tasks.The workflow comprises two steps: pretraining on a large dataset and fine-tuning on the target dataset.Essentially, by fine-tuning the DL architecture, the knowledge gleaned from one dataset can be transferred to a dataset procured from another center.

AI-based US techniques
US is recommended in clinical guidelines for the detection of HCC in patients with cirrhosis.However, its efficacy can be influenced by several factors, including operator experience, equipment quality, and patient morphology.Previous studies have indicated that the sensitivity of HCC detection using conventional US ranges from 59% to 78% (204).To enhance sensitivity and specificity, various US modalities have been explored.For instance, Contrast-Enhanced Ultrasound (CEUS) has been demonstrated to improve the sensitivity of HCC detection.These models serve as invaluable tools for predicting HCC recurrence, guiding treatment decisions, and improving patient outcomes.This study investigates the most recently developed AI-based approaches for evaluating detection, prognostication, treatment response, and survival in HCC.Table 1 provides a summary of the results from studies evaluating AI-based US approaches for HCC diagnosis.

Diagnosis of focal liver lesions
This section outlines the recently developed AI-based US models for diagnosing HCC.These applications encompass diagnosing focal liver lesions (FLLs), distinguishing between benign and malignant liver lesions, differentiating HCC from focal nodular hyperplasia (FNH), cirrhotic parenchyma (PAR), and intrahepatic cholangiocarcinoma (ICC) (see Table 1).Among these studies, Bharti et al. (21) proposed a Support Vector Machine (SVM) model that integrates three classifiers using B-mode US data

Characterization of focal liver lesions
In a study conducted by Virmani et al. (7), a Neural Network Ensemble (NNE) model was proposed to distinguish a normal liver from four distinct liver lesions, achieving an impressive accuracy of 95%.The diagnoses for the included liver lesions were confirmed through experienced radiologists, clinical follow-ups, and other associated findings.
In 2017, Hassan et al. ( 20) introduced an ANN model that achieved a classification accuracy of 97.2% for benign and malignant FLLs.In 2019, Schmauch et al. (22) developed a supervised DL model, specifically a CNN, utilizing a French radiology public challenge dataset for diagnosing FLLs.The model was capable of detecting FLLs and categorizing them as benign (such as cyst, FNH, and angioma) or malignant (like HCC, metastasis), achieving a mean AUC of 0.935 and 0.916 in the training dataset.Despite promising results, further validation is required due to the limited number of images used for training.
In 2020, Yang et al. (23) conducted a multicenter study to develop a Deep Convolutional Neural Network (DCNN) using an US database, along with background and clinical parameters (such as HBV, HCV, lesion margin, morphology) for characterizing FLLs.The model achieved an AUC of 0.924 for distinguishing benign from malignant lesions in the external validation dataset.The model demonstrated superior accuracy compared to clinical radiologists and CECT, albeit slightly lower than Contrast-Enhanced Magnetic Resonance Imaging (CE-MRI) (87.9%).This approach could potentially enhance radiologists' performance and reduce the reliance on CECT/CEMR and biopsy.
In 2021, Mao et al. ( 25) developed various ML-based models for distinguishing primary liver cancer and secondary liver cancer by extracting radiomic features from US images.The Logistic Regression (LR) model outperformed other ML models in this study.Ren et al. (30) applied a Support Vector Machine (SVM) model in B-mode US for predicting the pathological grading of HCC, achieving an AUC of 0.874 in the test set.The same research group also developed another SVM model for differentiating HCC from Intrahepatic Cholangiocarcinoma (ICC), yielding good performances (31).In these studies, liver lesions were pathologically confirmed and used as the standard reference.
In 2017, Guo et al. (40) demonstrated that a multiple-kernel learning-based model could enhance the sensitivity, specificity, and   42) developed two DL-based models using CEUS data to predict the two-year progression-free survival of HCC patients undergoing either Radiofrequency Ablation or Surgical Resection.The models achieved C-index values of 0.726 and 0.741 for Radiofrequency Ablation and Surgical Resection, respectively.When the Surgical model was applied to predict outcomes for patients initially treated with Ablation, it suggested that approximately 17.3% of Ablation patients could potentially experience a longer two-year progressionfree survival if they underwent Surgery.Conversely, the Ablation predictive model indicated that 27.3% of Surgical patients might achieve a longer two-year progression-free survival if they had received Ablation treatment.These CEUS-based models provide accurate survival assessments for HCC patients and facilitate optimal treatment selection.Furthermore, the same research group employed a DL model to quantitatively analyze CEUS videos (43).They developed three models to predict personalized responses of HCC patients after their first TACE session.The CEUS-based model outperformed the other two ML models, achieving a higher AUC value (0.93 vs 0.80 vs 0.81).

Evaluation prognostication, treatment response and survival in HCC
In another study, Ma et al. ( 44) applied a Radiomics model in dynamic CEUS to predict early and late recurrence in patients with an HCC lesion less than 5cm in diameter after Thermal Ablation.The prediction model yielded an AUC of 0.84 for early recurrence and a Cindex of 0.77 for late recurrence in the test group.The proposed model, which combines CEUS, US Radiomics, and clinical factors, performed well in predicting early HCC recurrence after Ablation and could stratify the high risk of late recurrence.
Lastly, Liu et al. (16) introduced DL models in CEUS to predict the two-year progression-free survival rate of HCC patients, demonstrating exceptional accuracy in guiding treatment decisions.Other researchers have incorporated additional pattern recognition classifiers into DCNN algorithms using CEUS to improve the diagnosis of FLLs.However, previous studies only involved small sample sizes, thus standardized imaging data or external validations are required to validate the model's generalizability across populations.

AI-based CT techniques
Numerous research groups have explored the application of AI in liver cancer research, specifically leveraging CT scan technology.This section delves into AI-based CT methodologies for diagnosing and predicting HCC.Tables 2 and 3 encapsulate selected studies, which can be categorized into three distinct groups: segmentation of liver and liver tumors, characterization of FLLs, and evaluation of prognostication, treatment response, and survival in HCC patients.

Segmentation of liver and liver tumors
The segmentation of liver and liver tumors plays a crucial role in assessing tumor burden, detecting early recurrence, extracting image features, and formulating treatment plans.The manual segmentation of liver and liver lesions is a significant challenge and is time-consuming due to the extensive range of radiographic features in HCC.AI-driven CT models have emerged as powerful tools for the automatic segmentation of liver and liver tumors.Table 2 provides a summary of recently developed AI-driven CT models for segmentation of liver and liver tumors.Since 2017, the Liver Tumor Segmentation Challenge (LiTS) has been encouraging researchers to create AI models for the automatic segmentation of liver tumors.This challenge utilizes a multinational dataset of CT images, known as LiTS17, which  Over the past few years, this challenge has seen participation from more than 280 research teams worldwide, with models based on Fully Convolutional Networks (FCN) or U-Net achieving top scores for the segmentation of liver and liver tumors.At present, the highest-scoring model, MAD-UNet (83), has achieved Dice score of 0.9727 for the segmentation of liver using the LiTS17 dataset.While these results are promising, there is a notable variability in both the imaging characteristics of liver tumors and their delineation.This highlights the need for universal and standardized methods for liver tumor segmentation.

Characterization of focal liver lesions
Table 3 summarizes the results of studies that have evaluated AI-based CT models for diagnosing HCC.Mokrane et al. ( 106) developed a ML model using 13,920 CT images from 189 patients.This model was able to distinguish HCC from non-HCC lesions in cirrhotic patients, achieving AUC values of 0.81 and 0.66 in the training and external validation datasets, respectively.In 2019, Khan et al. ( 107) developed a SVM model that classified FLLs as benign or malignant, achieving an accuracy of 98.3%.Das et al. (52) proposed a CAD system based on a watershed transform and Gaussian Mixture Model (GMM) for accurate and automated liver lesion detection using CT scan data.The liver was first separated using the watershed transform method, and the liver lesion was segmented using the GMM algorithm.Texture features were extracted and fed into a DNN model to automatically classify three types of liver tumors, including hemangioma, HCC, and metastatic carcinoma.The proposed model achieved a classification accuracy of 99.38% and a Jaccard index of 98.18%.
In 2020, Li et al. ( 108) developed a CAD system using ANN, SVM, and CNN models for diagnosing three types of HCC lesions, including nodular, diffuse, and massive.The experimental results demonstrated that the CNN model outperformed both the ANN and SVM models in classifying nodular and massive lesions, but not diffuse lesions.
In 2021, Mao et al. (25) developed a gradient boosting-based model using clinical parameters and CECT data for pathological grading of HCC.The combined model exhibited the best performance with an AUC of 0.8014 in the test set.Shi et al. (109) compared the performance of a DL-based three-phase CECT model with a four-phase CT protocol for distinguishing HCC from other FLLs.The DL-based three-phase CECT protocol without precontrast achieved a similar diagnostic accuracy (85.6%) to the fourphase CT protocol (83.3%).These findings suggest that omitting the pre-contrast phase might not compromise accuracy while reducing a patient's radiation dose.Several CNN-based models have been developed using CT data for diagnosing HCC.In 2018, Yasaka et al. (110) proposed a CNN model using three-phase CT for distinguishing malignant liver lesions from indeterminate and benign liver lesions.The proposed model achieved a median AUC of 0.92 in the test set.In 2021, Krishnan et al. (114) introduced a novel multi-level ensemble architecture for detecting and classifying HCC from other FLLs.This innovative approach highlights the potential of ensemble techniques in improving the specificity and sensitivity of liver cancer diagnosis using CT imaging.
In 2023, Manjunath et al. (115) developed a novel DL model using CT data to detect and classify liver tumors.The experimental results demonstrated that the proposed model improved accuracy, Dice similarity coefficient, and specificity compared to existing algorithms, emphasizing the continuous evolution of DL models for precise liver cancer diagnosis.

Prognostication of HCC
Numerous research groups have focused their efforts on the applications of AI models using CT and CECT images for the prognostication of HCC.Table 4 provides a summary of the results from studies that evaluated AI-based CT models for HCC prognostication.Among these studies, Peng et al. (131)  In 2022, Yang et al. (132) conducted an investigation of various AI models using CECT data from 283 patients.The aim was to predict MVI in patients with HCC.The experimental results revealed that the DL-based clinical-radiological model achieved the best performance with an accuracy of 96.47%, a sensitivity of 90.91%, a specificity of 97.30%, a precision of 89.4%, an F1 score of 87%, and an AUC of 0.909.

AI-based MRI methods
To date, the application of AI models in MRI for diagnosing HCC has not been extensively adopted.The development of MRI features poses technical challenges and incurs substantial costs, resulting in a scarcity of published studies with relatively small sample sizes.This section explores the progression of AI-based MRI models for the diagnosis of HCC.

Segmentation of liver and liver tumors
In recent years, a multitude of research groups have focused on the applications of AI models utilizing MRI data for the automated segmentation of the liver and liver tumors.Table 5 encapsulates the AI-based MRI models recently developed for the segmentation of liver and liver tumors.Among the various studies, the most remarkable performance was delivered by Hossain et al. (139), who pioneered a cascaded network to address anatomical ambiguity.This model, which employs T1-weighted MRI data for liver segmentation, exhibited an impressive performance with a Dice coefficient of 0.9515, Intersection over Union (IoU) of 0.921, and an accuracy of 99.7%.
More recently, Gross et al. ( 140) developed a DCNN model using T1-weighted MRI data from 470 patients for liver segmentation.The results suggested that the proposed DCNN model achieved mean Dice values of 0.968, 0.966, and 0.928 in the training, validation, and public testing datasets, respectively.The model exhibited a promising performance with 73% and 75% sensitivities for validation and testing datasets, respectively.The performance evaluation compared the automatically detected lesions with manual segmentation.The mean Dice score values between the identified lesions using the CNN model and manual segmentations were 0.64 and 0.68 for the validation and testing datasets, respectively.

Characterization of focal liver lesions
The advancements in CNN-based MRI models for diagnosing HCC have significantly enhanced the accuracy, efficiency, and precision of lesion classification and detection.From distinguishing different types of FLLs to detecting targeted HCC, these CNN-based models have showcased remarkable performance metrics and potential clinical utility.Further research and validation studies are essential to fully assess the capabilities of these models in clinical settings, paving the way for personalized and effective treatment strategies in liver cancer management.

Prognostication of HCC
A select number of research groups have ventured into the application of AI models and MRI-based data for HCC prognostication.Table 7 encapsulates a summary of studies evaluating AI-based MRI models for this purpose.
In 2021, Gao et al. (168) scrutinized various AI models using T2-weighted MRI data from 225 patients to predict Microvascular Invasion (MVI) in patients with HCC.The H-DARnet model outshone others, achieving an accuracy of 82.6%, a sensitivity of 79.5%, a specificity of 73.8%, and an AUC of 0.775.
Wei et al. (187) investigated the fusion DL model and the Contrast-Dependent Learning Model (CDLM) using gadoxetic acid-enhanced MRI (EOB-MRI) data from 225 patients for predicting MVI in patients with HCC.Both models exhibited robust performance, with the Fusion DL model achieving an accuracy of 89.4%, a sensitivity of 78.1%, a specificity of 95.3%, and an AUC of 0.93.The CDLM model achieved an accuracy of 92.4%, a sensitivity of 93.9%, a specificity of 91.6%, and an AUC of 0.962 in the training dataset.
In 2023, Chen et al. (169) explored four models (KNN, SVM, Lasso, and DNN) using T2-weighted MRI data from 144 patients for predicting Transarterial Chemoembolization (TACE) outcomes in patients with HCC.Among these, the Lasso model achieved the best performance.
These studies underscore the potential of AI models in conjunction with MRI data for predicting HCC, demonstrating promising results in terms of accuracy, sensitivity, specificity, and AUC.Further research in this area could catalyze significant advancements in the early detection and treatment of HCC.

AI-based multi-modal techniques
AI-based multi-modal techniques are swiftly ascending to prominence in the realm of medical imaging, attributed to its extraordinary ability to amplify diagnostic accuracy and forecast outcomes.AI-based multi-modal model integrates multiple modalities, such as medical imaging data, Electronic Health Records (EHR) and clinical parameters, thereby substantially enhancing the efficacy of AI algorithms.AI-based multi-modal models have proven successful in predicting treatment responses, evaluating survival rates, and staging a multitude of diseases.Such techniques have been deployed in a plethora of studies pertaining to liver imaging applications, yielding encouraging results.The continued exploration and refinement of these techniques hold great promise for the future of medical imaging and patient care.

Segmentation of liver and liver tumors
Table 8 encapsulates a summary of studies that evaluate AIbased multi-modal models for the segmentation of liver and liver tumors.Among the various studies, the most remarkable performance was demonstrated by Hille et al. (173).They explored the SWTR-Unet model using a combination of 61,440 MRI images and 189,600 CT images for the segmentation of both the liver and hepatic lesions.The proposed multi-modal model achieved Dice coefficients of 0.98 and 0.81 for the segmentation of the liver and hepatic lesions, respectively.

Diagnosis of HCC
AI-based multi-modal models offer a comprehensive and robust approach to HCC diagnosis, enabling disease prediction, classification, treatment response prediction, survival rate determination, and disease staging.The outcomes of studies evaluating AI-based multi-modal models for HCC diagnosis are summarized in Table 9.
In 2020, The research findings demonstrated the capacity of multi-modal DL to accurately identify HCC.This study underscores the potential of ML algorithms in assisting the early detection and diagnosis of HCC, which may lead to improved patient outcomes.Li et al. ( 179) investigated a ML-based multi-modal model using three-phase CEUS data from 266 patients and a radiologist's score for evaluating the diagnostic accuracy when differentiating between atypical Hepatocellular Carcinoma (aHCC) and Focal Nodular Hyperplasia (FNH).The proposed model achieved the highest AUC of 0.93 in aHCC and FNH differentiation.
In 2022, Liu et al. (180) proposed a DL model to detect malignancy by combining clinical parameters and CEUS data from 303 patients.The model achieved the best performance with AUC values of 0.969 and 0.957 and accuracies of 96% and 94% in the IntraVenous (IV) and ExtraVenous (EV) groups, respectively.Further research is necessary to identify the optimal combination of modalities and variables for specific medical tasks.The development of standardized protocols and datasets is critical to facilitate the comparison and reproducibility of multi-modal AI models in medical image analysis.

Prognostication of HCC
A multitude of studies have explored the use of AI-based multimodal models for prognostication of HCC.The insights from these studies are compiled in Table 10.Among these, a significant contribution was made by Sun et al. (183), who implemented a hybrid model combining GhostNet and CNN models.This integrated model leveraged CT data and clinical parameters to predict the response of TACE treatment in HCC patients.The proposed method exhibited remarkable performance, achieving an accuracy of 98% and an AUC of 0.98.This model demonstrated its potential in predicting TACE treatment responses, thereby assisting healthcare providers in devising personalized treatment plans and making informed decisions.This approach shows promise in improving patient outcomes and raising the bar in clinical practice.

Challenges and future directions
In the past decade, AI models' application in medical imaging for HCC diagnosis and prediction has emerged as a significant research area.While individual medical imaging methods such as US, CT, and MRI have been explored (205)(206)(207)(208), there is a lack of comprehensive reviews focusing on AI-based models using both single and multi-modal modalities.This study aims to fill that gap, reviewing AI models developed for HCC diagnosis and prediction using both single and multi-modal methods from January 2010 to March 2024.
Despite AI-based diagnostic models not significantly improving overall diagnostic accuracy for pathologists, they have shown increased precision within specific subgroups.However, several challenges must be addressed before integrating these models into clinical workflows.The efficacy of AI models depends on both the models' accuracy and the quality of the datasets used.Factors such as biases, mislabeling, lack of standardization, and missing data can undermine these datasets.Overfitting and spectrum biases are prevalent issues in AI-based medical imaging models.Therefore, the need for standardized methods for AI-based data analysis and comprehensive strategies to tackle missing data is evident.
AI tools intended for medical applications could be categorized as medical devices and must adhere to pertinent regulations.Both the FDA and the European Commission have initiated plans to tackle this issue.Intellectual property concerns, particularly those associated with post-marketing modifications, could pose safety risks.The performance of AI models is intimately linked to the training dataset.The importance of large datasets is paramount, and the promotion of data sharing is necessary, which brings forth ethical and privacy considerations.The clinical performance of AI and the requirement for post-approval validation are significant issues.The development of explainable AI models is vital for securing clinicians' trust and reliance on AI-based CAD systems.Customized prospective clinical trials are indispensable to fully comprehend the role of AI in HCC management.
Looking ahead, the integration of AI in HCC management presents an exciting frontier in medical science.As we continue to refine AI models and address the challenges, we move closer to a future where AI plays a pivotal role in personalized patient care.The potential of AI to analyze vast amounts of data and make precise predictions can lead to early detection and more effective treatment strategies for HCC.This not only improves patient outcomes but also paves the way for a new era in healthcare, where technology and human expertise work hand in hand for the betterment of patient care.
Several strategies are essential for the future of AI in HCC diagnosis and prediction.First, the development of standardized methods for AIbased data analysis and comprehensive strategies to handle missing data are crucial.Second, universal approaches to handle missing data and improve data quality are vital for enhancing the robustness and reliability of DL-based diagnostic tools.Promoting data sharing initiatives can facilitate the availability of large, diverse datasets necessary for training and validating DL models.
In addition to the aforementioned strategies, the exploration of advanced technologies such as transfer learning can further enhance the role of AI in HCC diagnosis and prediction.This technology can adapt pre-trained DL models to new tasks with limited labeled data.This addresses the challenge of acquiring extensive datasets in medical imaging, a common hurdle in the healthcare sector.Federated Learning (FL) is emerging as a transformative trend in healthcare.It enables a collaborative approach to ML development across multiple institutions, eliminating the need for direct data sharing.This innovative method involves the exchange of model parameters only, thereby ensuring the privacy of individual datasets.In the context of liver cancer, where patient data is both sensitive and heavily regulated, FL offers a unique advantage.It allows for the integration of fragmented healthcare data sources while preserving privacy.This enhances the scope and accuracy of ML models, making them more effective and reliable.As such, FL is poised to become an invaluable tool for future research and clinical implementation in liver cancer treatment.It offers the potential to significantly advance patient care, marking a new era in the field of liver cancer treatment.
The development of explainable AI models is another critical step towards earning the trust and reliance of clinicians on AI-based CAD systems.The synergy of researchers, clinicians, and policymakers is a cornerstone in propelling innovation and setting the gold standard for the application of AI techniques in liver cancer care.A comprehensive approach is required to augment AI techniques for HCC diagnosis and management.This involves addressing key aspects such as interpretability, accuracy, data integration, ethical considerations, and validation processes.By tackling these areas, we can tap into the full potential of AI technology, leading to a revolution in HCC diagnosis and prediction.Customized prospective clinical trials are paramount to gain a complete understanding of the role of AI in HCC management.Regulatory bodies like the FDA and the European Commission have kick-started plans to address the regulatory compliance of AI-based diagnostic tools.These plans demand further development and implementation.The challenges and future directions underscore the intricacy of incorporating AI in HCC diagnosis and prediction.However, with persistent research and development, AI holds the promise to bring about a paradigm shift in this field.

Conclusions
This paper offers an exhaustive exploration of AI-driven models for the diagnosis and prediction of HCC, leveraging both medical imaging data and additional clinical information.The potential of AI-based methodologies in diagnosing HCC is vast, yet several hurdles need to be overcome before they can be seamlessly incorporated into clinical workflows to enhance patient diagnosis and treatment outcomes.Despite the presence of challenges such as data quality, model overfitting, regulatory compliance, and the necessity for explainable AI models, the potential advantages are considerable.AI models have the capacity to augment precision within specific patient subgroups.Furthermore, the development of standardized methods for data analysis can significantly bolster the robustness and reliability of these tools.Navigating these intricacies, it becomes evident that a multi-pronged strategy is essential to fully harness the transformative power of AI technology in revolutionizing

Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

FIGURE 1
FIGURE 1 Surgery, Transcatheter Arterial Chemoembolization (TACE), and Microwave Ablation are widely recognized as treatment methods for liver cancer.Each method requires meticulous candidate evaluation to ensure optimal therapeutic effectiveness (38-40).Wu et al.(203) employed ResNet18 in B-mode US to predict HCC recurrence after Microwave Ablation.The model achieved C-index values of 0.695, 0.715, 0.721, and 0.721 for early relapse, late relapse, and relapse-free survival in HCC patients, respectively.Liu et al. ( In 2015, Li et al. (49) introduced a DCNN for the segmentation of liver tumors in CT scans, achieving a precision rate of 82.67%.In 2017, Vivanti et al. (50) examined a CNN-based segmentation model for the automatic detection of recurrence during follow-up, achieving a true positive rate of 86% for lesions larger than 5 mm (28).Subsequently, Sun et al. (51) and Das et al. (52) conducted comprehensive studies on the automatic segmentation of tumors in the liver using CNN-based architectures such as Fully Convolutional Networks (FCNs) and U-Net.In 2017, Sun et al. (51) proposed an FCNs model for the segmentation of liver tumors, achieving high accuracy.

In 2019 ,
Todoroki et al. (111) developed a CNN-based model using multiphasic CT images for detecting and classifying five types of FLLs.Ben-Cohen et al. (91) introduced a FCN architecture with sparsity-based false positive reduction for liver tumor detection, outperforming traditional models.By employing the FCN-4s model and sparsity-based fine-tuning, they successfully detected 94.7% of small lesions, surpassing the performance of the U-Net model.In 2021, Zhou et al. (112) proposed a multi-modality and multiscale CNN model for automatically detecting and classifying FLLs in multi-phasic CT.The model obtained an average test precision of 82.8%, recall of 93.4%, and F1-score of 87.8%.The model achieved average accuracies of 82.5% and 73.4% for the binary and six-class classification, respectively.In this study, the classification performance of the model was placed between a junior and senior physician's evaluation.This preliminary study showed that this CNN-based model can accurately locate and classify FLLs, and could assist inexperienced physicians in reaching a diagnosis in clinical practice.Similarly, Ponnoprat et al. (113) constructed a twostep model based on CNN and SVM for distinguishing HCC and intrahepatic cholangiocarcinoma (ICC), and the model achieved a classification accuracy of 88%.
HCC diagnosis and treatment.With ongoing research and development, AI stands poised to usher in a paradigm shift in the field of HCC diagnosis and prediction, ultimately leading to enhanced patient outcomes and heralding a new epoch in healthcare.Writingreview & editing.AA: Investigation, Writingreview & editing.

TABLE 1 AI
-based US approaches for HCC diagnosis.

TABLE 2
AI-driven CT models for segmentation of liver and liver tumors.

TABLE 3
AI-based CT models for diagnosing HCC.

TABLE 4
AI-based CT models for HCC prognostication.

TABLE 5
AI-based MRI models for liver and liver tumors segmentation.

TABLE 6
AI-based MRI models for diagnosing HCC.

TABLE 7 AI
-based MRI models for HCC prognostication.

TABLE 8
Summary of studies evaluating AI-based multi-modal models for liver and liver tumors segmentation.

TABLE 9 AI
-based multi-modal models for diagnosing HCC.

TABLE 10 AI
-based multi-modal models for prognostication of HCC.

TABLE 10 Continued
proposed a novel AI model based on conventional Machine Learning (cML) and DL methods.This model utilized CT data from 310 patients to predict TACE in patients with HCC.The experimental results demonstrated that the proposed model achieved AUC values of 0.995 and 0.994 in the training and testing datasets, respectively.In 2021, Jiang et al. (81) developed a 3D CNN using CT data from 405 patients.This model was designed to predict Microvascular Invasion (MVI) in patients with HCC and obtained commendable AUC values of 0.98 and 0.906 in the training and testing datasets, respectively.

Table 6
(158)sulates the advancements in AI-based MRI models for diagnosing HCC.These models have shown promise in improving the detection and classification of FLLs, including HCC.In 2019, Hamm et al.(158)proposed a CNN model capable of classifying six types of FLLs, namely adenoma, cyst, Focal Nodular Hyperplasia (FNH), HCC, Intrahepatic Cholangiocarcinoma (ICC), and metastases.The model demonstrated an impressive overall accuracy of 92%, with sensitivity values spanning from 60% to 100%, and specificity values between 89% and 99%.This study highlighted the potential of DL in accurately identifying various types of FLLs.Wang et al. (159) developed an interpretable DL model using MRI images.The model achieved a positive predictive rate of 76.5% and a sensitivity of 82.9% for classifying FLLs.The interpretability of this model enhances its clinical utility by offering insights into the decision-making process.Trivizakis et al. (160) employed a 3D CNN model with Diffusion-Weighted Magnetic Resonance (DW-MR) data to classify primary and metastatic liver tumors.The model achieved an accuracy of 83%, underscoring the potential of DL in enhancing liver tumor recognition, particularly in datasets with limited size and disease specificity.In 2020, Zhen et al. (161) pioneered several CNN models, including a distinctive model that utilizes unenhanced MR images for liver tumor diagnosis, thereby eliminating the need for contrast agent injection.This innovative approach demonstrated a performance on par with experienced radiologists, suggesting a potential reduction in patient discomfort and risks associated with contrast agents.Kim et al. (162) introduced a CNN model that achieved an impressive AUC of 0.97, a sensitivity of 94%, and a specificity of 99% for HCC detection using a training dataset of 455 patients.In a validation dataset of 45 patients, the model maintained an AUC of 0.90, sensitivity of 87%, and specificity of 93% for HCC detection.This study underscored the capability of deep learning models in accurately identifying HCC, a critical step in early diagnosis and treatment planning.Wu et al. (16) developed a DL model based on multiphase, contrast-enhanced MRI to differentiate between different grades of liver tumors for HCC diagnosis.The model utilized a CNN to classify the Liver Imaging Reporting and Data System tumor grades of liver lesions based on MRI data acquired at three-time points.The DL CNN model achieved high accuracy, sensitivity, precision, and AUC, providing valuable clinical guidance for differentiating between intermediate LR-3 liver lesions and more likely malignant LR-4/LR-5 lesions in HCC diagnosis.In 2021, Wan et al. (163) proposed a CNN architecture based on multi-scale and multi-level fusion (MMF-CNN) for detecting liver lesions in MRI images.The model's effectiveness was confirmed through comparative analysis with other DL models, emphasizing its potential to improve diagnostic accuracy and efficiency.The proposed MMF-CNN architecture is a promising approach to accurately and efficiently detect liver lesions in MRI images, which can significantly improve patient outcomes.Oestmann et al. (164) presented a CNN model that employs multiphasic MR images to differentiate between HCC and non-HCC lesions.The model demonstrated high sensitivities and specificities for both lesion types, achieving 92.7% and 82.0%sensitivities for HCC and non-HCC lesions, respectively, and specificities of 82.0% for both HCC and non-HCC lesions.The research underscored the importance of accurately distinguishing between HCC and non-HCC lesions to guide appropriate treatment strategies for liver cancer patients.Bousabarah et al. (148) proposed a CNN for detecting and segmenting HCC using multiphase contrast-enhanced MRI data.
This model was used to diagnose HCC using MRI data and clinical parameters, including age, gender, tumor markers, liver function, and other relevant factors.The study confirmed the potential of combining medical imaging and clinical data to improve HCC diagnosis, emphasizing the importance of such techniques in enhancing healthcare outcomes.In 2021, Gao et al. (118) employed a multi-modal model based on the VGG16 architecture to detect HCC in CT images.The study aimed to determine the model's accuracy in detecting HCC by incorporating eight EHR parameters, including age, gender, platelet count, bilirubin levels, tumor markers, and hepatitis B virus status.