Abstract
The obsession with how the brain and behavior are related is a challenge for cognitive neuroscience research, for which functional magnetic resonance imaging (fMRI) has significantly improved our understanding of brain functions and dysfunctions. In this paper, we propose a novel multi-modal spatial cerebral graph based on an attention mechanism called MSCGATE that combines both fMRI modalities: task-, and rest-fMRI based on spatial and cerebral features to preserve the rich complex structure between brain voxels. Moreover, it attempts to project the structural-functional brain connections into a new multi-modal latent representation space, which will subsequently be inputted to our trace regression predictive model to output each subject’s behavioral score. Experiments on the InterTVA dataset reveal that our proposed approach outperforms other graph representation learning-based models, in terms of effectiveness and performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Demirci, O., Clark, V.P., Magnotta, V.A., et al.: A review of challenges in the use of fMRI for disease classification/characterization and a projection pursuit application from a multi-site fMRI schizophrenia study. Brain Imaging Behav. 2(3), 207–226 (2008)
Mihalik, A., Ferreira, F.S., Rosa, M.J., et al.: Brain-behaviour modes of covariation in healthy and clinically depressed young people. Sci. Rep. 9, 1–11 (2019)
Sui, J., Adali, T., Yu, Q., et al.: A review of multivariate methods for multimodal fusion of brain imaging data. J. Neurosci. Methods 204(1), 68–81 (2012)
Sui, J., Pearlson, G.D., et al.: In search of multimodal neuroimaging biomarkers of cognitive deficits in schizophrenia. Biol. Psychiat. 78(11), 794–804 (2015)
Blackmon, K., Barr, W.B., Kuzniecky, R., et al.: Phonetically irregular word pronunciation and cortical thickness in the adult brain. Neuroimage 51(4), 1453–1458 (2010)
Saygin, Z.M., Osher, D.E., et al.: Anatomical connectivity patterns predict face selectivity in the fusiform gyrus. Nat. Neurosci. 15(2), 321–327 (2012)
Du, W., Calhoun, V.D., et al.: High classification accuracy for schizophrenia with rest and task FMRI data. Front. Hum. Neurosci. 6(145) (2012)
Hanachi, R., Sellami, A., Farah, I.: Interpretation of human behavior from multi-modal brain mri images based on graph deep neural networks and attention mechanism. In: Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications 4: VISAPP, pp. 56–66 (2021)
Jiang, J., Ma, J., Chen, C., et al.: SuperPCA: A superpixelwise pca approach for unsupervised feature extraction of hyperspectral imagery. IEEE Trans. Geosci. Remote Sens. 56(8), 4581–4593 (2018)
He, X., Cai, D., Yan, S., Zhang, H.-J.: Neighborhood preserving embedding. In: Tenth IEEE International Conference on Computer Vision (ICCV 2005), pp. 1208–1213 (2005)
Ma, L., Crawford, M.M., Tian, J.: Anomaly detection for hyperspectral images based on robust locally linear embedding. J. Infrared Milli Terahz Waves 31, 753–762 (2010)
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk. In: Proceedings of the 20th ACM SIGKDD International Conference On Knowledge Discovery and Data Mining - KDD 2014 (2014)
Dong, Y., Chawla, N.V., Swami, A.: Metapath2vec: Scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2017), pp. 135–144. Association for Computing Machinery, New York (2017)
Grover, A., Leskovec, J.: Node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp, 855–864. Association for Computing Machinery (2016)
Wu, Z., Pan, S., Chen, F., Long, G., et al.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 1–21 (2020)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), pp. 1025–1035. Curran Associates Inc., Red Hook (2017)
Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal deep learning. In: ICML, pp. 689–696 (2011)
Alam, M.T., Kumar, V., Kumar, A.: A Multi-view convolutional neural network approach for image data classification. In: 2021 International Conference on Communication information and Computing Technology (ICCICT), pp. 1–6 (2021)
Sun, C., Yuan, Y.-H., Li, Y., Qiang, J., Zhu, Y., Shen, X.: Multi-view fractional deep canonical correlation analysis for subspace clustering. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds.) ICONIP 2021. LNCS, vol. 13109, pp. 206–215. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92270-2_18
Bhatt, G., Jha, P., Raman, B.: Representation learning using step-based deep multi-modal autoencoders. Pattern Recogn. 95, 12–23 (2019)
Khosla, M., Setty, V., Anand, A.: A comparative study for unsupervised network representation learning. IEEE Trans. Knowl. Data Eng. (2020)
Mikolov, T., Chen, K.: Greg Corrado. Efficient Estimation of Word Representations in Vector Space, Jeffrey Dean (2013)
Zhang, Z., Cui, P., Zhu, W.: Deep learning on graphs: A survey. CoRR (2018)
Kadri, H., Ayache, S., Huusari, R., Rakotomamonjy, A., Ralaivola, L.: Partial trace regression and low-rank kraus decomposition. In: International Conference on Machine Learning (2020)
Koltchinskii, V., Lounici, K., Tsybakov, A.B.: Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. Ann. Stat. 39(5), 2302–2329 (2011)
Fan, J., Gong, W., Zhu, Z.: Generalized high dimensional trace regression via nuclear norm regularization. J. Econom. 212(1), 177–202 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Hanachi, R., Sellami, A., Farah, I.R. (2023). BS-GAENets: Brain-Spatial Feature Learning Via a Graph Deep Autoencoder for Multi-modal Neuroimaging Analysis. In: de Sousa, A.A., et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2021. Communications in Computer and Information Science, vol 1691. Springer, Cham. https://doi.org/10.1007/978-3-031-25477-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-25477-2_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25476-5
Online ISBN: 978-3-031-25477-2
eBook Packages: Computer ScienceComputer Science (R0)