Explainable artificial intelligence in breast cancer detection and risk prediction: A systematic scoping review

With the advances in artificial intelligence (AI), data-driven algorithms are becoming increasingly popular in the medical domain. However, due to the nonlinear and complex behavior of many of these algorithms, decision-making by such algorithms is not trustworthy for clinicians and is considered a black-box process. Hence, the scientific community has introduced explainable artificial intelligence (XAI) to remedy the problem. This systematic scoping review investigates the application of XAI in breast cancer detection and risk prediction. We conducted a comprehensive search on Scopus, IEEE Explore, PubMed, and Google Scholar (first 50 citations) using a systematic search strategy. The search spanned from January 2017 to July 2023, focusing on peer-reviewed studies implementing XAI methods in breast cancer datasets. Thirty studies met our inclusion criteria and were included in the analysis. The results revealed that SHapley Additive exPlanations (SHAP) is the top model-agnostic XAI technique in breast cancer research in terms of usage, explaining the model prediction results, diagnosis and classification of biomarkers, and prognosis and survival analysis. Additionally, the SHAP model primarily explained tree-based ensemble machine learning models. The most common reason is that SHAP is model agnostic, which makes it both popular and useful for explaining any model prediction. Additionally, it is relatively easy to implement effectively and completely suits performant models, such as tree-based models. Explainable AI improves the transparency, interpretability, fairness, and trustworthiness of AI-enabled health systems and medical devices and, ultimately, the quality of care and outcomes.

relatively easy to implement effectively and completely suits performant models, such as tree-based models.Explainable AI improves the transparency, interpretability, fairness, and trustworthiness of AI-enabled health systems and medical devices and, ultimately, the quality of care and outcomes.

| INTRODUCTION
Breast cancer (BC) is one of the most common cancers with high morbidity and mortality globally.Early detection and treatment significantly increase the chances of survival [1].With the growing interest in artificial intelligence (AI), computer-aided diagnosis (CAD) based on AI has become a valuable tool for the detection, classification, and diagnosis of cancer biomarkers and morphological features.
Compared to rule-based systems [2] that require human intervention in decision-making, AI models can learn from medical data and generate new patterns by themselves.However, AI systems are susceptible to several biases [3], mostly stemming from low-quality datasets, faulty algorithms, and human cognitive biases that may lead to inaccurate decisions, predictions, or inferences [4].Additionally, many AI systems have raised concerns among clinicians about accountability, fairness of AI algorithms, and lack of transparency [5], a critical factor for high-stakes domains such as healthcare, where a minor error in decision-making can lead to irreparable consequences [6].
Despite significant progress over the last years in terms of fine-tuning [7] and optimizing [8][9][10][11] AI algorithms to tackle supervised and unsupervised tasks, a considerable number of these algorithms remain enigmatic, classified as Black-box and are yet to be demystified.To this end, the scientific community has started investigating techniques and methods to make AI algorithms more understandable, explainable, and interpretable.In recent years, explainable artificial intelligence (XAI), coined by DARPA [12], has emerged as a notable and noteworthy topic of discussion in the AI community.The rationale behind XAI lies in the assumption that such techniques establish rules for more trustworthy AI systems by making them more transparent, understandable, interpretable, safe, and reliable while making a decision or recommending an action [13,14].
Typically, AI models are evaluated based on their prediction errors [15] without providing enough transparency to the end users throughout this process.As Figure 1 shows, XAI methods are formulated to be applied to the result and provide transparency; however, the human-in-the-loop (HITL) concept must also be applied to achieve trustworthiness.XAI is basically employing human-agent interaction methods to utilize human knowledge and intuition to comprehend the rationale behind the results it gains [16].As Figure 1 illustrates, a result generated by the AI model passes through a suitable XAI method.Then, the human agent benefits from the transparency created by the XAI to validate, confirm, or enhance the predictions [17].It should be noted that the human agent's decision is based on the collaboration between clinicians and XAI experts.If the results are not correct or satisfying, the XAI module investigates the model, data, or both and reruns the outcomes to the AI system throughout an iterative process until a consensus over the results is reached.
Most of the review articles on XAI models investigate the application of XAI in general healthcare [18][19][20].Although there have been reviews of the subject on other types of cancer [21], to the best of our knowledge, our contribution to reviewing existing XAI technologies in breast cancer screening, risk detection, and prediction is distinctive both in terms of scope and breadth.This paper provides a comprehensive summary of published studies and then elaborates on the background and concepts associated with the XAI methods used in these studies.Finally, we highlight the most popular XAI methods and explain the rationale behind their popularity.
The remainder of this paper is organized as follows.Section 2 covers the background, including the introduction to AI models, the concept of accuracy-explainability tradeoff, and the classification of XAI methods.Section 3 describes the research method.Section 4 provides tabulated results of implemented XAI methods in breast cancer research.Section 5 discusses the details and elaborates on the utilized methods.Finally, Section 6 concludes our survey by highlighting XAI's achievements, strengths, and limitations and discussing future research opportunities.

| Overview of AI models
AI models employ data-driven algorithms to reach decisions or identify explanatory patterns.Machine learning (ML) algorithms fall into three types: regression and classification, which are supervised, and clustering, which is unsupervised.If the output is a continuous variable, we deal with regression, but when it is discrete labels or categories, then we use classification [22].Clustering algorithms identify and group similar data points based on their characteristics.The most popular ML models used in breast cancer studies are listed in Table 1.
Deep learning (DL) is a subcategory of ML that may be supervised or unsupervised.Unlike traditional machine learning, DL models require much less manual human intervention since they automate the feature extraction, saving time and resources.DL models are capable of addressing model accuracy and performance for unstructured data such as speech, images, videos, or texts, whereas classical ML models do not function effectively.
In BC datasets, we usually deal with high-dimensional, multimodal [30] structured and unstructured data, which are often big, noisy, and sparse, making them challenging to analyze.Thanks to neural networks' universal approximation [31,32] and the advantage of auto-differentiation [33], deep learning models can be applied to many of these problems.DL models can learn the complex nonlinear relationships between the features and target variables, making them viable data-driven models that enable new discoveries in breast cancer classification and detection.Frequently used DL models and their variants in BC studies are listed in Table 2.
As Figure 2 illustrates, the accuracy-explainability trade-off refers to the balance between the accuracy of an AI model and its explainability [56].The goal of any AI model is to generate highly accurate results.From the XAI perspective, the models must be explainable.However, achieving both accuracy and explainability is far from trivial.Regarding explainability, AI models can be black-box, white-box, or gray-box [12,57], as depicted in Figure 2.
White-box models are intrinsically transparent and explainable [58].However, they are limited to learning only linear associations between input features and the target variable.Although white-box models may not achieve high accuracy levels, they offer human-understandable explanations.In contrast, black-box models are nontransparent by nature [59].While these models may have outstanding performance, they suffer from a lack of explainability.Gray-box models strike a balance between accuracy and explainability.Generally, any data-driven learning algorithm, including black-and-white-box models, is considered a gray box [57,60].For a gray-box model, connections from input data to model output can be explained despite not being fully transparent [60].

| Classification of XAI methods
The transparency of AI systems can be addressed from different perspectives.The results of XAI methods can be presented in various ways, including numerical, rules, textual, visual, or a combination of these [61].As Figure 3 depicts, three critical factors for categorizing XAI methods of explanation exist: scope, stage, and type.The scope of explainability is either local or global.The local method aims to shed light on an AI model for a specific input [62,63] and explains why a particular decision was made by highlighting the input feature influencing the model's output.However, this approach cannot find a general relationship between input features and outputs [64].The global method provides a broader understanding by analyzing the model's overall structure and general patterns across the entire data set or a larger subset, helping users understand its biases, limitations, and general decision-making patterns [62,63].
Intrinsic and post hoc [65] refer to the stage of explanation.The intrinsic approach refers to using whitebox models which are interpretable by nature.The post hoc approach relates to explainable methods that "explain the model predictions after the training and inference processes" [62,64].Generally, the post hoc approach, compared to intrinsic models (white-box models) [65], is more accurate since post hoc explainable methods must be applied to black-box models' prediction, and black-box models tend to perform better in results.Although some of these methods, such as rule extraction [66] and tree extraction [67], can turn black-box models into white-box, there is a complexity-accuracy trade-off [57,64].Additionally, post hoc methods are either model-specific or model-agnostic.Model-specific methods are designed to explain specific black-box models by investigating their internal factors and interactions [64].For example, many techniques are developed to analyze DL models, which attempt to find the contribution of artificial neurons on their final decisions through backpropagation (backprop) error [68][69][70][71].Model-agnostic methods provide explanations independent of a specific AI model.Some common post hoc XAI methods are tabulated in Appendix A. XAI methods can be classified based on the type of explanations they offer [64].As Figure 3 demonstrates, there are four types of explanations: feature importance, white-box model, example-based XAI, and visual explanations [64].For the first type, XAI methods create numbers/values for the input features to express the feature's importance.For the second type, XAI methods "create a white-box model that mimics the original blackbox model and is inherently explainable" [64].The example-based type, also known as data point [65], uses samples from the training datasets to explain the model's action.For the last type, XAI methods offer a type of explainability based on purely visual explanations [64].

| RESEARCH METHOD
This systematic review is carried out using the preferred reporting items on systematic reviews and meta-analysis (PRISMA) [72] guideline in three steps as follows: Step 1: Identifying studies-As mentioned in the introduction, our focus for this paper was on studies that examined existing XAI methods in breast cancer research.We conducted a comprehensive search utilizing some of the most popular and trusted citation platforms [73], including Scopus, IEEE Xplore, PubMed, and Google Scholar (first 50 citations) from January 2017 to July 2023 using the combination of keywords and MeSH terms described in Table 3.A total of 193 studies were included in this step.
Step 2: Selecting the studies-In Step 2, we selected articles for inclusion based on specific criteria: they had to be original studies published in peer-reviewed English-language journals, utilizing at least one XAI methodology within the context of breast cancer.Two reviewers (Amirehsan Ghasemi and Soheil Hashtarkhani), screened citations by title and abstract, excluding various types of irrelevant papers, such as different review papers (n = 16), those discussing XAI but unrelated to breast cancer (n = 18), those discussing breast cancer but unrelated to XAI (n = 20), preprints awaiting peer review (n = 7), conference papers (n = 34), duplicate titles (n = 37), and nonresearch materials like books, dissertations, editorials, and technical notes (n = 12), resulting in the exclusion of 144 studies.Subsequently, 49 articles underwent fulltext scrutiny, with inaccessible or irrelevant articles being excluded.This left us with 30 articles that met the inclusion criteria for our comprehensive review.Figure 4 illustrates a summary of our search strategy and steps.
T A B L E 3 Explored databases and the results.
Step 3: Data extraction and summarization-A data extraction form was developed in Google Sheets, consisting of eight variables, including authors, year, the aim of the study (objective), data set(s), data type, important features, type of AI (ML or DL), and the explained model.Two reviewers (Amirehsan Ghasemi and Soheil Hashtarkhani) extracted data from all included articles, and any disagreement was resolved by consensus.SHAP [116] offer local and global explanations based on the Shapley value [117], a solution concept used in cooperative game theory.In SHAP, the input features of an observation act as players in a game, and the prediction serves as the reward.SHAP computes the average marginal contribution of each player to the reward [64,65] and ensures that the distribution of reward among players is fair [18,118].In BC studies, SHAP can potentially find the contribution of biomarkers (players  important features in Table 4) to the prediction (reward objective in Table 4).
As provided in Table 4, most studies (12/13) implemented ensemble ML learning as the predictors.Only in one study (1/13) [89] did authors first utilize LR-based to discriminate between upregulated and regular expression of HER2 protein, then pathologists' diagnoses (IHC) in conjunction with fluorescent in situ hybridization (IHC + FISH) were used as the training outputs.In Chakraborthy et al. [74], SHAP showed that "by boosting the B cell and CD8 + T cell fractions or B cell and NK T cell fractions in the tumor microenvironment (TME) to levels above their inflection points, the survival rate of BC patients could increase by up to 18%."In Rezazadeh et al. [76], texture analysis of the ultrasound images based on the gray-level co-occurrence matrix (GLCM) predicted the likelihood of malignancy of breast tumors.SHAP was used to find the most critical features: GLCM correlation and GLCM energy within different pixel distances along the 90°direction.
In summary, SHAP emerged as the most frequently used XAI method in the BC studies (13/30).Notably, no DL models were used in conjunction with SHAP.Instead, tree-based ensemble learning ML models, specifically XGBoost (9/13 studies), were the most widely used models.This can be attributed to the high-speed SHAP algorithm, which is well-suited for tree-based models such as XGBoost, Catboost, GBM, AdaBoost, and so on [18].

| Local interpretable model agnostic explanations (LIME)
LIME [119] provides a local explanation using a surrogate model.As outlined in Table 5, LIME is utilized in 5 out of 30 studies to explain the model prediction by highlighting the contribution of the most important features.LIME creates a linear local surrogate mode that is intrinsically interpretable around a sample (data point) and improves transparency by producing feature importance values.The surrogate model in LIME modifies some parts of the given features and generates perturbed instances to understand how the output changes.The perturbation depends on the nature of the input sample.For instance, one method to perturb an image is by replacing certain parts with gray color [120].In Kaplun et al. [90], to explain the image classification, LIME puts a mask of yellow pixels to highlight the important image segments the model focuses on to make the decision.
In Adnan et al. [93], the authors have implemented SHAP in conjunction with LIME to explain that a small number of highly compact and biological gene cluster features resulted in similar or better performance than classifiers built with many more individual genes.With training on smaller gene clusters, LIME proved that the classifiers have better AUC than the original classifiers except in RF and rSVM.In Saarela and Jauhiainen [92], the authors used linear and nonlinear ML classifiers with As a model-agnostic method, LIME was used to explain various ML models, including RF, SVM, ensemble learning, and a shallow DL learning model, as detailed in Kaplun et al. [90].LIME only offers a local interpretation, and compared to SHAP, when a large volume of predictions needs to be explained, it has a higher speed and can be a more excellent alternative.In summary of the model-agnostic methods used in studies (18/30), SHAP was preferred over LIME to shed light on the most important features.This is because SHAP is relatively easy to implement and provides both local and global explanations, and compared to LIME, it has a higher speed on the global-level explanation for high-performance ensemble ML models.

| Post hoc XAI methods: Modelspecific
Several XAI methods are specifically designed for different DL architectures focusing on the feature importance type of explanation.Most of these methods are propagation-based and enjoy the availability of gradients computed during the training.

| Class activation map (CAM)
CAM [121] is a local backpropagation-based method that uses a global average pooling (GAP) layer after the last convolutional layer, followed by the classification layer to identify the most discriminative regions of an image in the convolutional neural network (CNN) [34].This technique combines a linear weighted sum of the feature maps to gain a heatmap that highlights class-specific regions of the image.CAM is limited to existing networks that have the described architecture.
As listed in Table 6, (5/30) studies have used CAM to determine how accurately the CNN model localized the breast masses.Qi et al. [98] proposed two CNN-based networks, the Mt-Net and the Sn-Net, to identify malignant tumors and recognize solid nodules step-bystep.To enable the two networks to collaborate effectively, CAM was introduced as an enhancement mechanism to improve the accuracy and sensitivity of the classification results for both networks.

| Gradient-weighted class activation mapping (Grad-CAM)
Grad-CAM [122] is a local backpropagation-based method that uses the feature maps produced by the last layer of a CNN to create a coarse localization classspecific heatmap where the hot part corresponds to a particular class.Grad-CAM is based on CAM but is not limited to fully connected CNNs.Grad-CAM can be applied to any CNN architecture without retraining or architectural modification as long as the layers are differentiable.
As detailed in Table 7, (4/30) studies used Grad-CAM to determine how accurately the CNN model localized the breast masses.The authors of article [104] have also investigated the performance of the model-agnostic LIME in conjunction with Grad-CAM to investigate the aspects and utilities of two different XAI methods in explaining the misclassification of breast masses.The results highlight the usability of XAI in understanding the mechanism of used AI models and their failures, which can provide valuable insights toward explainable CAD systems.In Gerbasi et al. [107], the authors also implemented Deep SHAP, a high-speed approximation algorithm for computing SHAP values in DL models, to produce maps visually interpreting the classification results, which in the maps, pink pixels strongly contributed to the final predicted class (malignant), and the blue pixels contributed to the prediction of opposite class (benign).

| Gradient-weighted class activation mapping++ (Grad-CAM++)
Grad-CAM++ [123] is a local backpropagation-based method built upon Grad-CAM to enhance visual explanations of CNN.Compared to Grad-CAM, it provides better visual explanations of model predictions in terms of better localization of objects and explaining occurrences of multiple objects of a class in a single image [123].As listed in Table 8, only one study (1/30) used Grad-CAM++.To et al. [110] developed an ensemble learning-based approach to locate cancerous regions in DUV whole-slide images (WSI).It used Grad-CAM++ on a pretrained DenseNet169 model to generate regional significance maps to classify each WSI confidently as cancerous or benign.LRP [124] is a local propagation-based approach.LRP calculates the relevance score for a specific output at the classifier layer.It proceeds backward, exploits the DL structure, and calculates each neuron's explanatory factors (relevance R) for each layer during the backward pass until it reaches the input image [18,124].Based on the computed relevance score, LRP generates a heatmap with highlighted critical regions that can be used to explain the prediction.Two studies (2/30) have implemented LRP; the details are described in Table 9.
Grisci et al. [112] introduced relevance aggregation, an XAI approach based on LRP that combines the relevance derived from several samples as learned by a neural network and generates scores for each input feature.The study results showed that the model could correctly identify which input features or relevant ones are the most important for the model's predictions, facilitate knowledge discovery, and help identify incorrect or irrelevant rules or machine biases in the case of the poorly trained implemented AI model.Chereda et al. [114] extended the procedure of LRP to make it available for GCN to explain its decisions.They tested it on a large BC genomic data set.They showed that the model, named graph layer-wise relevance propagation (GLRP), provides patient-specific molecular subnetworks that agree with clinical knowledge and can identify common, novel, and potentially druggable drivers of tumor progression.
In summary, 12 out of 30 studies used model-specific XAI methods, and CAM and Grad-CAM were the most used models, respectively.

| Clinical applications
Figure 6 illustrates the diverse applications of each XAI method across various clinical scenarios.The studies primarily focused on either diagnosing/classifying breast cancer or conducting survival/prognosis analyses of patients.Within these study types, image recognition techniques were employed on radiology data, or alternative approaches utilizing clinical and demographic data were explored.Notably, SHAP was frequently utilized in clinical data analysis studies rather than image recognition studies.This preference may be attributed to the computational resource intensity of SHAP, posing challenges in handling the high-dimensional feature space inherent in image data.Conversely, techniques such as CAM and Grad-CAM are computationally less intensive, which makes them a better choice for image processing tasks, especially in real-time applications.In diagnosis/classification studies, the primary objective was to employ supervised learning methods for distinguishing between healthy and diseased patients, facilitating early detection.XAI models played a crucial role in helping clinicians comprehend and validate intricate patterns and features that influence diagnostic outcomes.In survival/prognosis models, clinicians sought to predict the onset of events such as mortality or metastasis in patients.XAI methods proved instrumental in interpreting and elucidating the contribution of each factor to a patient's outcome measure.This interpretability makes the models more understandable, usable, and trustworthy for both clinicians and patients, fostering a perception and interpretation of the predictions and building confidence in the decision-making process.

| Future directions
The rapid evolution of AI models, as evidenced by advanced frameworks such as GPT and generative AI-based models, is significantly transforming the healthcare applications landscape.As these models continue to advance and become more intricate, the necessity for XAI methods becomes increasingly imperative.In the healthcare domain, where precision and interpretability are of paramount importance, the demand for robust XAI techniques is expected to grow.Future research should prioritize the refinement and advancement of XAI methodologies to effectively uncover the intricacies of advanced AI models in healthcare contexts.The synergy between the rapid advancements in AI technologies and the evolving landscape of XAI is crucial, shaping the trajectory of personalized healthcare and ensuring that these innovative models translate into tangible benefits for both clinicians and patients.

| Study limitations
This systematic scoping review has some limitations that warrant consideration.While efforts were made to minimize publication bias, excluding non-English language articles, non-access, and gray literature may have resulted in the omission of some valuable information.Additionally, despite our best efforts to construct a comprehensive search strategy across multiple databases using combinations of Boolean search strings and MeSH terms, the diverse terminology associated with XAI methods and breast cancer may have led to the inadvertent omission of certain studies.Moreover, we only investigated the existing established XAI methods; however, XAI schemes based on or independent of these methods could be observed in a few studies.To ensure the integrity and credibility of the study, we did not consider some of the studies with low or no citations in this survey.

| CONCLUSIONS
We systematically reviewed breast cancer studies that successfully implemented the existing XAI methods to their model predictors.In summary, SHAP was the most used model-agnostic method.The frequent use of this method with tree-based ensemble ML models is related to the speed and compatibility that SHAP provided for these models.Grad-CAM and CAM were widely used model-specific XAI methods in these studies.We noticed that other explanatory methods, as provided in Appendix A, have not been used in breast cancer studies and can still be examined and compared as future endeavors.
Additionally, the XAI methods used in the selected studies only provided a sanity check to the model's predictor results.As was mentioned in the introduction, finding the biases in the model and data can be achieved using explainability methods that were either missing or only mentioned in a few of the studies and should be investigated for further studies.Moreover, although the clinical applications of XAI methods were investigated in our study, the results generated by these methods were not evaluated by oncologists.Therefore, to provide trustworthiness, the reliability of the results through clinical evaluation is needed.Researchers have already used XAI domain-specific explanations to improve understanding, interpretation, trustworthiness, and reliability of the results in different medical domains for evaluating health interventions [125], disease causal pathway analysis [126], mental health surveillance and precision resource allocation [127], precision dermatology and disease diagnosis [128], immune response predictors [129], and investigating the links between socioenvironmental risk factors and Alzheimer's disease [130].
Potential challenges associated with the application of XAI, especially when dealing with complex multimodal clinical and medical data, include but are not limited to the availability of data in appropriate temporal and geographic resolutions; its representativeness, diversity, and types of modalities involved, semantic heterogeneity, fusion of heterogeneous data streams, AI-readiness of clinical data sets [131], and algorithmic and human biases in explanations that addressing them can increase the efficiency and acceptance of multimodal XAI schemes.
Addressing these challenges is key to the widespread acceptance of multimodal XAI models and algorithms in cancer care delivery and treatment.

5 , 1 |
Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cai2.136 by University Of Tennessee Health Science Center, Wiley Online Library on [12/07/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License Almost 30 studies have been identified in the literature utilizing XAI methods in breast cancer settings.XAI methods used in these studies include SHAP, LIME, CAM, Grad-CAM, Grad-CAM++, and LRP.The number of studies and explained AI models for each method are shown in Figure 5. Tables 4-9 provide tabular representations of the included studies.A detailed description of the results is provided in the discussion section (Section 5). 5 | DISCUSSION 5.Post hoc XAI methods: Modelagnostic 5.1.1 | SHapley Additive exPlanations (SHAP)

T A B L E 7 5 ,
List of studies that used Grad-class activation map.Learning (FL) to preserve the privacy of patient data Gerbasi et al. [107] 2023 Proposing a fully automated and visually explained model to analyze raw mammograms with microcalcifications INbreast data set [108] (train and test), CBIS-DDSM [109] (used to implement the classification algorithm) Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cai2.136 by University Of Tennessee Health Science Center, Wiley Online Library on [12/07/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License

F I G U R E 6
Explainable artificial intelligence (XAI) methods are applied in different clinical applications of breast cancer literature.Numbers inside the bubbles represent the number of studies.

14 of 22 |
CANCER INNOVATION 27709183, 2024, 5, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cai2.136 by University Of Tennessee Health Science Center, Wiley Online Library on [12/07/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License T A B L E 2 List of popular deep learning (DL) models.
List of studies that used SHapley Additive exPlanations.List of studies that used class activation map.
[97]B L E 4 CANCER INNOVATION 27709183, 2024, 5, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cai2.136 by University Of Tennessee Health Science Center, Wiley Online Library on [12/07/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)onWileyOnlineLibrary for rules of use; OA articles are governed by the applicable Creative Commons License T A B L E 4 (Continued) 27709183, 2024, 5, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cai2.136 by University Of Tennessee Health Science Center, Wiley Online Library on [12/07/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)onWileyOnlineLibrary for rules of use; OA articles are governed by the applicable Creative Commons License T A B L E 5 List of studies that used local interpretable model agnostic explanations.27709183,2024,5,Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cai2.136 by University Of Tennessee Health Science Center, Wiley Online Library on [12/07/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)onWileyOnlineLibrary for rules of use; OA articles are governed by the applicable Creative Commons LicenseLIME to understand how they differ in explaining features' importance.It was found that the nonlinear model (RF) offered better explainability as it focused on fewer features (9) compared to the nonlinear model (all except one feature).Deshmukh et al.[97]used LIME to quantify the impact of patient and treatment characteristics on BC distant metastasis.It reached the results of different impacts ranging from high impacts, such as the nonuse of adjuvant chemotherapy, to moderate impact of carcinoma with medullary features cancer type, to a low impact of oral contraception use.
T A B L E 8 List of studies that used Grad-class activation map++.List of explainable artificial intelligence studies that used layerwise relevance propagation.Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/cai2.136 by University Of Tennessee Health Science Center, Wiley Online Library on [12/07/2024].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License