Efficient Interpretation of Deep Learning Models Using Graph Structure and Cooperative Game Theory: Application to ASD Biomarker Discovery

Abstract

Discovering imaging biomarkers for autism spectrum disorder (ASD) is critical to help explain ASD and predict or monitor treatment outcomes. Toward this end, deep learning classifiers have recently been used for identifying ASD from functional magnetic resonance imaging (fMRI) with higher accuracy than traditional learning strategies. However, a key challenge with deep learning models is understanding just what image features the network is using, which can in turn be used to define the biomarkers. Current methods extract biomarkers, i.e., important features, by looking at how the prediction changes if “ignoring” one feature at a time. However, this can lead to serious errors if the features are conditionally dependent. In this work, we go beyond looking at only individual features by using Shapley value explanation (SVE) from cooperative game theory. Cooperative game theory is advantageous here because it directly considers the interaction between features and can be applied to any machine learning method, making it a novel, more accurate way of determining instance-wise biomarker importance from deep learning models. A barrier to using SVE is its computational complexity: $2^N$ given N features. We explicitly reduce the complexity of SVE computation by two approaches based on the underlying graph structure of the input data: (1) only consider the centralized coalition of each feature; (2) a hierarchical pipeline which first clusters features into small communities, then applies SVE in each community. Monte Carlo approximation can be used for large permutation sets. We first validate our methods on the MNIST dataset and compare to human perception. Next, to insure plausibility of our biomarker results, we train a Random Forest (RF) to classify ASD/control subjects from fMRI and compare SVE results to standard RF-based feature importance. Finally, we show initial results on ranked fMRI biomarkers using SVE on a deep learning classifier for the ASD/control dataset.

A Appendix: Proof of Theorem 2

For any subset $A\subset \mathcal {N}$, we use the short notation and , noting that $A=U_{r}(A)\cup V_{r}(A)$. Rewriting Eq. (2) as

$$ \varPhi _{r}(v_{\varvec{X}})=\frac{1}{|\mathcal {N}|}\sum _{U\subseteq \mathcal {N}_{r}\setminus \{r\}}\sum _{A\subseteq \mathcal {N},U_{r}(A)=U}\left( \begin{array}{c} |\mathcal {N}|-1\\ |A| \end{array}\right) ^{-1}(v_{\varvec{X}}(A\cup \{r\})-v_{\varvec{X}}(A)), $$

and using

$$ \sum _{A\subseteq \mathcal {N},U_{r}(A)=U}\left( \begin{array}{c} |\mathcal {N}|-1\\ |A| \end{array}\right) ^{-1}=\frac{|\mathcal {N}|}{|\mathcal {N}_{r}|}\left( \begin{array}{c} |\mathcal {N}_{r}|-1\\ |U|-1 \end{array}\right) ^{-1}, $$

the expected error between $\hat{\varPhi }_{r}^{C}(v_{\varvec{X}})$ and $\varPhi _{r}(v_{\varvec{X}})$ is

$$ \mathbb {E}[\vert \hat{\varPhi }_{r}^{C}(v_{\varvec{X}})\ -\ \varPhi _{r}(v_{\varvec{X}})\vert ]\le \frac{1}{|\mathcal {N}|}\sum _{U\subseteq \mathcal {N}_{r}\setminus \{r\}}\sum _{A\subseteq \mathcal {N},U_{r}(A)=U}\left( \begin{array}{c} |\mathcal {N}|-1\\ |A| \end{array}\right) ^{-1}\mathbb {E}[\vert \varDelta _{r}^{{\varvec{X}}}(U,A)\vert ] $$

where

$$\begin{aligned} \varDelta _{r}^{{\varvec{X}}}(U,A)&=(v_{\varvec{X}}(U\cup \{r\})-v_{\varvec{X}}(U))-(v_{\varvec{X}}(A\cup \{r\})-v_{\varvec{X}}(A))\\&=\log \frac{p(Y\vert X_{\mathcal {N}\setminus U})}{p(Y\vert X_{\mathcal {N}\setminus (U\cup \{r\})})}-\log \frac{p(Y\vert X_{\mathcal {N}\setminus (U\cup V)})}{p(Y\vert X_{\mathcal {N}\setminus (U\cup V\cup \{r\}})}, \end{aligned}$$

with V short for $V_{r}(A)$. Let $W=\mathcal {N}\setminus (\mathcal {N}_{r}\cup V)$, $Z=\mathcal {N}_{r}\setminus (\{r\}\cup U)$. Then

$$\begin{aligned} \varDelta _{r}^{{\varvec{X}}}(U,A)&=\log \frac{p(Y\vert X_{W\cup V\cup Z\cup \{r\}})p(Y\vert X_{W\cup Z})}{p(Y\vert X_{W\cup V\cup Z})p(Y\vert X_{W\cup Z\cup \{r\}})}. \end{aligned}$$

(9)

Since $X_{r}\perp X_{V}\vert X_{Z}$, we have $p(X_{V}\vert X_{W\cup Z\cup \{r\}})=p(X_{V}\vert X_{W\cup Z})$, and

$$ (\star )=\frac{p(X_{W\cup V\cup Z\cup \{r\}})p(X_{W\cup Z})}{p(X_{W\cup V\cup Z})p(X_{W\cup Z\cup \{r\}})}=\frac{p(X_{W\cup Z\cup \{r\}})p(X_{V}\vert X_{W\cup Z\cup \{r\}})p(X_{W\cup Z})}{p(X_{W\cup Z})p(X_{V}\vert X_{W\cup Z})p(X_{W\cup Z\cup \{r\}})}=1. $$

We can multiply the quotient in Eq. (9) by $(\star )$,

$$\begin{aligned} \varDelta _{r}^{{\varvec{X}}}(U,A)&=\log \frac{p(Y\vert X_{W\cup V\cup Z\cup \{r\}})p(Y\vert X_{W\cup Z})}{p(Y\vert X_{W\cup V\cup Z})p(Y\vert X_{W\cup Z\cup \{r\}})}\frac{p(X_{W\cup V\cup Z\cup \{r\}})p(X_{W\cup Z})}{p(X_{W\cup V\cup Z})p(X_{W\cup Z\cup \{r\}})}\\&=\log \frac{p(X_{W\cup V\cup \{r\}}\vert Y,X_{Z})p(Y,X_{Z})p(Y,X_{Z})p(X_{W}\vert Y,X_{Z})}{p(Y,X_{Z})p(X_{W\cup V}\vert Y,X_{Z})p(Y,X_{Z})p(X_{W\cup \{r\}}\vert Y,X_{Z})}. \end{aligned}$$

We have $p(X_{W\cup V\cup \{r\}}\vert Y,X_{Z})=p(X_{W\cup V}\vert Y,X_{Z})p(X_{r}\vert Y,X_{Z})$, since $X_{W\cup V}\perp X_{r}\vert Y,X_{Z}$. So

$$ \varDelta _{r}^{{\varvec{X}}}(U,A)=\log \frac{p(X_{W\cup V}\vert Y,X_{Z})p(X_{r}\vert Y,X_{Z})p(X_{W}\vert Y,X_{Z})}{p(X_{W\cup V}\vert Y,X_{Z})p(X_{W\cup \{r\}}\vert Y,X_{Z})}. $$

Since $X_{W}\perp X_{r}\vert Y,X_{Z}$, we have $p(X_{W\cup \{r\}}\vert Y,X_{Z})=p(X_{W}\vert Y,X_{Z})p(X_{r}\vert Y,X_{Z})$. Hence $\varDelta _{r}^{{\varvec{X}}}(U,A)=\log 1=0$. Therefore we have $\mathbb {E}[\vert \hat{\varPhi }_{r}^{C}(v_{\varvec{X}})-\varPhi _{r}(v_{\varvec{X}})\vert ]=0$.