Latent Neural Source Recovery via Transcoding of Simultaneous EEG-fMRI

Abstract

Simultaneous EEG-fMRI is a multi-modal neuroimaging technique that combines the advantages of both modalities, offering valuable insights into the spatial and temporal dynamics of neural activity. In this paper, we aim to address the inference problem inherent in this technique by employing the transcoding framework. Transcoding refers to the process of mapping from a specific encoding (modality) to a decoding (the latent source space), and subsequently encoding the latent source space back to the original modality. Our proposed method focuses on developing a symmetric approach, which involves a cyclic convolutional transcoder capable of transcoding EEG to fMRI and vice versa. Importantly, our method does not rely on any prior knowledge of either the hemodynamic response function or lead field matrix. Instead, it leverages the temporal and spatial relationships between the modalities and latent source spaces to learn these mappings. By applying our method to real EEG-fMRI data, we demonstrate its efficacy in accurately transcoding the modalities from one to another, as well as recovering the underlying source spaces. It is worth noting that these results are obtained on previously unseen data, further emphasizing the robustness and generalizability of our approach. Furthermore, apart from its ability to enable symmetric inference of a latent source space, our method can also be viewed as a form of low-cost computational neuroimaging. Specifically, it allows for the generation of an ‘expensive’ fMRI BOLD image using ‘low-cost’ EEG data. This aspect highlights the potential practical significance and affordability of our approach in the field of neuroimaging research.

This work is supported by the Army Research Laboratory under Cooperative agreement number W911NF-10-2-0022.