Covariant Image Representation with Applications to Classification Problems in Medical Imaging

Abstract

Images are often considered as functions defined on the image domains, and as functions, their (intensity) values are usually considered to be invariant under the image domain transforms. This functional viewpoint is both influential and prevalent, and it provides the justification for comparing images using functional $\mathbf {L}^p$-norms. However, with the advent of more advanced sensing technologies and data processing methods, the definition and the variety of images has been broadened considerably, and the long-cherished functional paradigm for images is becoming inadequate and insufficient. In this paper, we introduce the formal notion of covariant images and study two types of covariant images that are important in medical image analysis, symmetric positive-definite tensor fields and Gaussian mixture fields, images whose sample values covary i.e., jointly vary with image domain transforms rather than being invariant to them. We propose a novel similarity measure between a pair of covariant images considered as embedded shapes (manifolds) in the ambient space, a Cartesian product of the image and its sample-value domains. The similarity measure is based on matching the two embedded low-dimensional shapes, and both the extrinsic geometry of the ambient space and the intrinsic geometry of the shapes are incorporated in computing the similarity measure. Using this similarity as an affinity measure in a supervised learning framework, we demonstrate its effectiveness on two challenging classification problems: classification of brain MR images based on patients’ age and (Alzheimer’s) disease status and seizure detection from high angular resolution diffusion magnetic resonance scans of rat brains.

Appendices

Appendix 1

In this section, we show how to factor out DSC basis functions in formulating the gradient descent direction w.r.t. the DSC coefficients. Here, we only show details of derivatives w.r.t. ${\mathcal {A}}_{nml}$. Derivatives w.r.t. $\mathcal {B}_{nml}$ and $\mathcal {C}_{nml}$ can be done in the same way described in the following. In this section we consider Eq. 14 and in the following sections Eqs. 15 and Eq. 16 will be considered. Here, we only consider the image graphs of Gaussian mixture fields.

Equation 14 is rewritten as

$$\begin{aligned}&\frac{\partial }{\partial {\mathcal {A}}_{nml}}{\mathbf {Dist}}^2(\mathbf {X}_{1\mathcal {R}},\mathbf {X}_2 \circ \gamma ) \nonumber \\&\quad =\frac{\partial }{\partial {\mathcal {A}}_{nml}} ({\uplambda }\underbrace{(U^{2}+V^{2}+X^{2})}_{Part1} + \underbrace{{\mathbf {Dist}}_F^2(I_{1\mathcal {R}},I_{2} \circ \gamma )}_{Part2})\nonumber \\ \end{aligned}$$

(44)

In the Part1 of Eq. 44, factoring out basis functions is very straight forward, therefore, we only show how to do this in the Part2 below.

$$\begin{aligned}&\frac{\partial }{\partial {\mathcal {A}}_{nml}}{\mathbf {Dist}}_F^2(I_{1\mathcal {R}},I_{2} \circ \gamma ) \nonumber \\&\quad =\varvec{\eta }^{\top }\frac{\partial \mathbf A _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}}\varvec{\eta } -2\varvec{\eta }^{\top }\frac{\partial \mathbf C _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}} (\varvec{\rho } \circ \gamma ) \nonumber \\&\qquad + 2(\varvec{\rho } \circ \gamma )^{\top }\mathbf B \frac{\partial (\varvec{\rho } \circ \gamma )}{\partial {\mathcal {A}}_{nml}}-2\varvec{\eta }^{\top }\mathbf C _{\mathcal {R}} \frac{\partial (\varvec{\rho } \circ \gamma )}{\partial {\mathcal {A}}_{nml}}, \end{aligned}$$

(45)

Factoring out basis functions in Eq. 45 requires evaluation Eqs. 40, 41, and 42 explicitly. First, we need to evaluated derivatives of Jacobian matrix, $\mathbf {J}$. Given

$$\begin{aligned} \mathbf {J} = \left( \begin{array}{ccc} 1+U_{u} &{} U_{v} &{} U_{w} \\ V_{u} &{} 1+V_{v} &{} V_{w} \\ W_{u} &{} W_{v} &{} 1+W_{w} \end{array} \right) , \end{aligned}$$

(46)

the first order derivative w.r.t. ${\mathcal {A}}_{nml}$ is given

$$\begin{aligned} \frac{\partial \mathbf {J}}{\partial {\mathcal {A}}_{nml}} = \left( \begin{array}{ccc} \frac{\partial }{\partial u} {\varPhi }_{U_{nml}} &{} \frac{\partial }{\partial v} {\varPhi }_{U_{nml}}&{} \frac{\partial }{\partial w} {\varPhi }_{U_{nml}} \\ 0 &{} 0 &{} 0 \\ 0 &{} 0 &{} 0 \end{array} \right) . \end{aligned}$$

(47)

Once we notice that

$$\begin{aligned}&\frac{\partial \mathbf {J}}{\partial {\mathcal {A}}_{nml}} \mathbf {u}_{i{\mathcal {R}}} \nonumber \\&\quad = \left( \begin{array}{c} \underbrace{u_{i{\mathcal {R}} 1}\frac{\partial }{\partial u} {\varPhi }_{U_{nml}} + u_{i{\mathcal {R}} 2}\frac{\partial }{\partial v} {\varPhi }_{U_{nml}} +u_{i{\mathcal {R}} 3}\frac{\partial }{\partial w} {\varPhi }_{U_{nml}}}_{{\varDelta }_i} \\ 0 \\ 0 \end{array} \right) ,\nonumber \\ \end{aligned}$$

(48)

where the subscript i of ${\varDelta }$ is same with i of $\mathbf {u}_{i{\mathcal {R}}}$, and rewrite $\mathbf {J}^{-1}$ as

$$\begin{aligned} \mathbf {J}^{-1} = \left( \begin{array}{ccc} J_{11}^{-1} &{} J_{12}^{-1} &{} J_{13}^{-1} \\ J_{21}^{-1} &{} J_{22}^{-1} &{} J_{23}^{-1} \\ J_{31}^{-1} &{} J_{32}^{-1} &{} J_{33}^{-1} \end{array} \right) = \left( \begin{array}{c|c|c} &{} &{} \\ \mathbf {\mathbf {J}^{-1}_1} &{} \mathbf {\mathbf {J}^{-1}_2} &{} \mathbf {\mathbf {J}^{-1}_3} \\ &{} &{} \end{array} \right) \end{aligned}$$

(49)

then, we can factor out basis functions in Eq. 43 as

$$\begin{aligned} \frac{\partial \mathbf {u}_{i{\mathcal {R}}} }{\partial {\mathcal {A}}_{nml}} = ( \mathbf {J}^{-1}_1 - ( \mathbf {u}_{i{\mathcal {R}}}^{\top } \mathbf {J}^{-1}_1 ) \mathbf {u}_{i{\mathcal {R}}} ){\varDelta }_i \end{aligned}$$

(50)

and in the same way,

$$\begin{aligned} \frac{\partial \mathbf {u}_{i{\mathcal {R}}} }{\partial \mathcal {B}_{nml}}= & {} ( \mathbf {J}^{-1}_2 - ( \mathbf {u}_{i{\mathcal {R}}}^{\top } \mathbf {J}^{-1}_2 ) \mathbf {u}_{i{\mathcal {R}}} ){\varDelta }_i \end{aligned}$$

(51)

$$\begin{aligned} \frac{\partial \mathbf {u}_{i{\mathcal {R}}} }{\partial \mathcal {C}_{nml}}= & {} ( \mathbf {J}^{-1}_3 - ( \mathbf {u}_{i{\mathcal {R}}}^{\top } \mathbf {J}^{-1}_3 ) \mathbf {u}_{i{\mathcal {R}}}){\varDelta }_i . \end{aligned}$$

(52)

With Eq. 50, we can rewrite Eqs. 40 and 41 as

$$\begin{aligned}&\left. \frac{\partial \mathbf A _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}} \right| _{i,j} \nonumber \\&\quad = a_{ij}\mathbf {u}_{i{\mathcal {R}}}^{\top }\mathbf {u}_{j{\mathcal {R}}} \big [ \mathbf {u}_{i{\mathcal {R}}}^{\top } ( \mathbf {J}^{-1}_1 - ( \mathbf {u}_{j{\mathcal {R}}}^{\top } \mathbf {J}^{-1}_1 ) \mathbf {u}_{j{\mathcal {R}}} ){\varDelta }_j \nonumber \\&\qquad + ( \mathbf {J}^{-1}_1 - ( \mathbf {u}_{i{\mathcal {R}}}^{\top } \mathbf {J}^{-1}_1 ) \mathbf {u}_{i{\mathcal {R}}} )^\top \mathbf {u}_{j{\mathcal {R}}}^{\top } {\varDelta }_i \big ] \nonumber \\&\quad \doteq {\mathcal {J}}_{ij}^1 {\varDelta }_j +\mathcal J_{ij}^2 {\varDelta }_i \end{aligned}$$

(53)

and

$$\begin{aligned}&\left. \frac{\partial \mathbf C _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}}\right| _{i,j} \nonumber \\&\quad = b_{ij}\mathbf {u}_{i{\mathcal {R}}}^{\top }\mathbf {v}_{j} ( \mathbf {J}^{-1}_1 - ( \mathbf {u}_{i{\mathcal {R}}}^{\top } \mathbf {J}^{-1}_1 ) \mathbf {u}_{i{\mathcal {R}}} ) ^\top \mathbf {v}_{j} {\varDelta }_i \nonumber \\&\quad \doteq \mathcal {Z}_{ij} {\varDelta }_i , \end{aligned}$$

(54)

where

$$\begin{aligned} a_{ij}= & {} \frac{\beta }{(2\pi det({\varSigma }_{i\mathcal R}+{\varSigma }_{j{\mathcal {R}}}))^{3/2}} \ \ , \\ b_{ij}= & {} \frac{\beta }{(2\pi det({\varSigma }_{i{\mathcal {R}}}+{\varGamma }_{j{\mathcal {R}}}))^{3/2}} . \end{aligned}$$

Finally we factor out basis functions and their derivatives in Eq. 45 as follows.

$$\begin{aligned}&\varvec{\eta }^{\top }\frac{\partial \mathbf A _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}}\varvec{\eta } -2\varvec{\eta }^{\top }\frac{\partial \mathbf C _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}} (\varvec{\rho } \circ \gamma ) \nonumber \\&\quad = \eta _i \left( \frac{\partial \mathbf A _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}} \right) _{ij} \eta _j -2\eta _i \left( \frac{\partial \mathbf C _{\mathcal {R}}}{\partial {\mathcal {A}}_{nml}} \right) _{ij} (\varvec{\rho } \circ \gamma )_j \nonumber \\&\quad = \eta _i({\mathcal {J}}_{ij}^1 {\varDelta }_j +{\mathcal {J}}_{ij}^2 {\varDelta }_i)\eta _j -2\eta _i \mathcal {Z}_{ij} (\varvec{\rho } \circ \gamma )_j {\varDelta }_i \nonumber \\&\quad = \eta _i[({\mathcal {J}}_{ij}^1 u_{j\mathcal {R}1}+{\mathcal {J}}_{ij}^2 u_{i\mathcal {R}1})\eta _j -2 \mathcal {Z}_{ij}(\varvec{\rho } \circ \gamma )_j u_{i\mathcal {R}1}] \frac{\partial {\varPhi }_{U_{nml}}}{\partial u} \nonumber \\&\qquad +\,\eta _i[({\mathcal {J}}_{ij}^1 u_{j\mathcal {R}2}+{\mathcal {J}}_{ij}^2 u_{i\mathcal {R}2})\eta _j -2 \mathcal {Z}_{ij}(\varvec{\rho } \circ \gamma )_j u_{i\mathcal {R}2}] \frac{\partial {\varPhi }_{U_{nml}}}{\partial v} \nonumber \\&\qquad +\,\eta _i[({\mathcal {J}}_{ij}^1 u_{j\mathcal {R}3}+{\mathcal {J}}_{ij}^2 u_{i\mathcal {R}3})\eta _j -2 \mathcal {Z}_{ij}(\varvec{\rho } \circ \gamma )_j u_{i\mathcal {R}3}] \frac{\partial {\varPhi }_{U_{nml}}}{\partial w}\nonumber \\ \end{aligned}$$

(55)

$$\begin{aligned}&2(\varvec{\rho } \circ \gamma )^{\top }\mathbf B \frac{\partial (\varvec{\rho } \circ \gamma )}{\partial {\mathcal {A}}_{nml}}-2\varvec{\eta }^{\top }\mathbf C _{\mathcal {R}} \frac{\partial (\varvec{\rho } \circ \gamma )}{\partial {\mathcal {A}}_{nml}}\nonumber \\&\quad = 2((\varvec{\rho } \circ \gamma )^{\top }\mathbf B - \varvec{\eta }^{\top }\mathbf C _{\mathcal {R}}) \left. \frac{\partial \varvec{\rho } }{\partial u } \right| _{u+U} {\varPhi }_{U_{nml}} \end{aligned}$$

(56)

Appendix 2

In this section, we show how to evaluate Eqs. 15 and 16. Denote $J_{\gamma }$ the determinant of the Jacobian, $\mathbf {J}_{\gamma }$ defined in Eq. 7. Its derivative w.r.t. ${\mathcal {A}}_{nml}$ defined in Eq. 12 can be written as

$$\begin{aligned} \frac{\partial }{\partial {\mathcal {A}}_{nml}} J_{\gamma }= & {} (1+V_v+W_w+V_vW_w-V_wW_v)\frac{\partial {\varPhi }_{U_{nml}}}{\partial u} \nonumber \\&\quad +(V+wW+u-V_uW_w-V_u)\frac{\partial {\varPhi }_{U_{nml}}}{\partial v} \nonumber \\&\quad +(V_uW_v-V_vW_u-W_u)\frac{\partial {\varPhi }_{U_{nml}}}{\partial w} \end{aligned}$$

(57)

Given the derivative of $\mathbf {J}_{\gamma }$, evaluations of Eqs. 15 and 16 is straightforward:

$$\begin{aligned}&\frac{\partial }{\partial {\mathcal {A}}_{nml}} \sqrt{\kappa _2({\varOmega }_{2}\circ \gamma )} J_{\gamma } \nonumber \\&\quad =\left. \frac{1}{2\sqrt{\kappa _{2}}}\frac{\partial \kappa _{2}}{\partial u} \right| _{u+U} {\varPhi }_{U_{nml}} J_{\gamma } + \sqrt{\kappa _{2}} \frac{\partial J_{\gamma } }{\partial {\mathcal {A}}_{nml}} , \end{aligned}$$

(58)

and

$$\begin{aligned}&\frac{\partial }{\partial {\mathcal {A}}_{nml}}(J_{\gamma }-1)\log (J_{\gamma }) \nonumber \\&\quad =\left[ \log (J_{\gamma }) + 1 - \frac{1}{J_{\gamma }} \right] \frac{\partial }{\partial {\mathcal {A}}_{nml}}J_{\gamma }. \end{aligned}$$

(59)