Fusing Multiple Neuroimaging Modalities to Assess Group Differences in Perception–Action Coupling

In the last few decades, noninvasive neuroimaging has revealed macroscale brain dynamics that underlie perception, cognition, and action. Advances in noninvasive neuroimaging target two capabilities: 1) increased spatial and temporal resolution of measured neural activity; and 2) innovative methodologies to extract brain-behavior relationships from evolving neuroimaging technology. We target the second. Our novel methodology integrated three neuroimaging methodologies and elucidated expertise-dependent differences in functional (fused EEG-fMRI) and structural (dMRI) brain networks for a perception-action coupling task. A set of baseball players and controls performed a Go/No-Go task designed to mimic the situation of hitting a baseball. In the functional analysis, our novel fusion methodology identifies 50-ms windows with predictive EEG neural correlates of expertise and fuses these temporal windows with fMRI activity in a whole-brain 2-mm voxel analysis, revealing time-localized correlations of expertise at a spatial scale of millimeters. The spatiotemporal cascade of brain activity reflecting expertise differences begins as early as 200 ms after the pitch starts and lasts up to 700 ms afterwards. Network differences are spatially localized to include motor and visual processing areas, providing evidence for differences in perception-action coupling between the groups. Furthermore, an analysis of structural connectivity reveals that the players have significantly more connections between cerebellar and left frontal/motor regions, and many of the functional activation differences between the groups are located within structurally defined network modules that differentiate expertise. In short, our novel method illustrates how multimodal neuroimaging can provide specific macroscale insights into the functional and structural correlates of expertise development.

the living brain, from optical methods to new types of nanoprobes and sensors [1]- [3], observing in vivo activity in the healthy human brain will be mostly based on magnetic resonance (MRI) and electromagnetic imaging methods for the foreseeable future. In terms of functional imaging, both electroencephalography (EEG) and blood-oxygen-level-dependent functional magnetic resonance imaging (BOLD fMRI) are extremely common noninvasive methods for observing human brain function. Magnetoencephelography (MEG), a close cousin of EEG, is also used to noninvasively measure brain activity, though its costs are more prohibitive than EEG. EEG and fMRI also can be acquired simultaneously, which is appealing given that both methods are complementary in terms of their basic physiological measurements. EEG is a direct measurement of neural "mass" activity and provides high temporal resolution and dynamics (at timescales of milliseconds) while fMRI is an indirect measurement of neural activity, based on hemodynamic changes, and offers high spatial resolution given its noninvasive acquisition (spatial resolution in millimeters). Separately, these modalities have made enormous contributions to human behavioral, psychological, and clinical neuroscience. When leveraged in a fused representation, for example when simultaneously acquired, they have been shown to provide new insight into the macroscale dynamical networks that underlie function in the human brain [4]- [11].
In addition to function, understanding the structural organization of the human brain is central to understanding why some brains may function differently than others. Classically, structural differences underlying experiencedependent plasticity were thought to occur in the gray matter regions of the cortex, but recently, research has suggested that neural adaptations can be seen in the white matter fiber tracts as well [12], [13]. A diffusion MRI (dMRI) scan captures the directional diffusion of water within a voxel, and the location and direction of the white matter tracts are inferred from the constrained movement of water molecules (see [14] for a review). Changes in fiber tract connections have been found over a range of temporal scales, including research on professional concert pianists in their thirties that found that differences in fiber tract organization reflected the number of practicing hours during adolescence [15], while another study found structural effects in adults after six weeks of juggling practice [16]. Converging evidence from across the lifespan indicates that variability in white matter fiber tract connections correlates with between subject variability in task performance [17], [18].
In this paper, we demonstrate, with a comprehensive example, that an integrated analysis of fused EEG-fMRI functional imaging together with structural DTI provides unique insight into difference in brain networks between groups of individuals having different perception-action coupling proficiencies. Our specific example focuses on differences in perception-action coupling between experts (i.e., baseball players) and a nonexpert control group during a Go/No-Go task based on baseball pitch discrimination. Deciding on whether to swing at an incoming baseball pitch is a complex task with a very low success rate worth millions of dollars. To be able to quickly predict a 90-mph pitch trajectory and have the motor control to place a bat on the 3-in diameter ball in less than 400 ms has been referred to as "clearly an impossible task" [19]. However, after hours of training, many professional athletes are able to succeed (although with at best a 1/3 success rate) and have become experts in this specific type of perception-action coupling.
Recently our group has shown, using only EEG, that temporally specific neural correlates of a rapid Go/No-Go decision differed between players versus controls [20]. Players, overall, performed better at the task compared to controls both in terms of accuracy and faster response times, i.e., players shifted their speed-accuracy tradeoff curve instead of moving along the same curve as defined by the controls. Players also showed differences in their task-evoked EEG components. Players had larger and earlier EEG components for Correct Go and Correct No-Go trial types, relative to controls. These differences were found to be most likely in the inhibition response during the No-Go trials. Source localization suggested that players have stronger cortical sources in the supplementary motor area (SMA) for Correct No-Go trials and the fusiform gyrus for Correct Go trials. This work offered evidence that there are distinct spatiotemporal neural differences between baseball players and controls during a baseball-like perceptual decision making task. However, the EEG, by itself, only provides a partial picture and does not enable a comprehensive investigation of the structural and functional networks underlying these differences.
Below we describe an approach and corresponding results that use whole brain BOLD fMRI and simultaneously collected EEG and confirms these previous player versus control differences found in our EEG-only experiments [20]. In addition, we describe novel EEG/ fMRI fusion techniques and apply these to this data set as a way to further elucidate functional networks underlying differences in this particular type of expertise. Specifically, a single-trial sliding window linear discrimination analysis of the EEG is used to construct a temporally precise, neurally-derived rating of expertise for each subject. The rating is used as a covariate of interest in the fMRI model, identifying both the regions in the brain that correlate with differences in expertise as well as the timing of these differences. We analyze these results with respect to additional differences we identify in a structural network analysis, enabled by advanced connectivity methods informed by the functional analysis. We conclude that advances in understanding the human brain will be enabled by a more integrated analysis of structural and functional neuroimaging within the context of complex behavior.

A. Subjects
The study included 14 division I collegiate baseball players (all male, 19.57 AE 2.4 years) and 24 nonbaseball player controls (all male, 20.92 AE 2.7 years) with an age range of 18-30 years. Three of the controls were not used in the task-based analysis due to movement during the fMRI scanning and one expert was excluded from structural analysis due to low quality diffusion image reconstruction. Controls had no professional or collegiate baseball experience. All subjects reported no history of neurological problems and had normal or corrected vision and all gave informed consent according to the guidelines and approval of the Columbia University Institutional Review Board.

B. Behavioral Paradigm
This behavioral paradigm has been applied and described previously in [20] and [21] and is reproduced here for completeness.
The experimental session involves a training session prior to the simultaneous fMRI/EEG data acquisition. In the training, subjects familiarized themselves with the different pitch types and completed practice trials until they scored an accuracy of at least 60% (above the random chance accuracy of 50%). At the beginning of each trial, a single letter corresponding to the pitch ("F" for fastball, "C" for curveball, and "S" for slider) was shown on the screen (Horizontal view 0.28˚and vertical view 0.28˚) for a mean time of 819 AE 3.1 ms. While the letter was on the screen, a horizontal bar (horizontal extent 3.93˚, vertical 0.28˚) shrank (horizontally) at a constant rate to either the left or right side of the screen. If the pitch following the letter cue came from a left handed pitcher, then the horizontal bar shrank toward the right, and if the pitch came from a right handed pitcher, then the horizontal bar shrank toward the left. After the horizontal bar shrank completely to either the left or the right, the pitch started from that point on the left or right side of the screen (i.e., pitches from left-handed pitchers started from the right side of the screen, and vice versa).
Subjects used the VisuaStim Digital System (Resonance Technology) 600 Â 800 goggle display to view 450 simulated baseball pitches (five blocks of 90 trials, three different types of pitches) from the viewpoint of a baseball catcher (at the end of the baseball's trajectory). While viewing these pitches, subjects completed a Go/ No-Go task by determining if each pitch matched its prestimulus cue. The program optseq2 [22] was used to select a mean jittered interstimulus interval (ISI) that enabled the rapid presentation of fMRI events without overlap from the hemodynamic responses (mean of 3000 ms and SE of 225 ms). Each subject was instructed to respond by pressing a keyboard button with the index finger of his right hand if the prestimulus cue matched the type of pitch that followed it ("Go" trials). In addition, in order for a "Go" response to be correct, the subject needed to respond while the ball was still in the screen. If the prestimulus cue and the pitch did not match, the subject was instructed to withhold his response ("No-Go" trials). Feedback was given after every trial (for both "Go" and "No-Go" trials) in the form of a "+" for correct responses in "Go" trials and correct withholding of responses in "No-Go" trials and a "" for incorrect "Go" and "No-Go" responses. 60% of the trials were "Go" and 40% of the trials were "No-Go." Overall accuracy for the task was determined by calculating the percent of trials with a correct response (the subject responded while the ball was still on the screen for "Go" trials and withheld his response for "No-Go" trials). Go accuracy was determined by calculating what percent of all the "Go" trials had a correct response (subject responded and this response happened while the ball was still on the screen). "No-Go" accuracy was determined by calculating what percent of all the "No-Go" trials the subject correctly withheld his response.
Similar to our previous work, we simulated each pitch via a differential equation solver in Matlab 2010a (Mathworks, Natick, MA, USA) (see [20]- [23] for details) and presented these using PsychToolbox [24]. Pitches were simulated using six-coupled differential equations. Each of the three pitches-fastball, curveball, and slider-have well-defined initial conditions. To create each pitch, we varied the initial velocity and the rotation angle. For each simulated pitch, an isoluminant green circle was plotted on a gray background for every frame of the trajectory. The size of the circle increased as it approached the viewer, so as to give the illusion of depth. When the ball crossed "home plate," the circle disappeared.

C. Structural MRI and Simultaneous fMRI-EEG Data Acquisition
A 3T Philips Achieva MRI scanner (Philips Medical Systems) with an eight-channel SENSE head coil was used to collect MRI data. For each task block, functional echo planar imaging (EPI) data sensitive to blood oxygenated level-dependent (BOLD) contrast were collected (2-s TR, 20-ms TE, 64 Â 64 matrix, and 35 interleaved slices, 240 repetitions). After the task based data collection, a 5-min resting state scan was collected. Whole brain T1-weighted anatomical images ð1 Â 1 Â 1 mmÞ and single high volume EPI images ð2 Â 2 Â 2 mmÞ were also obtained to help with registration. DTI was acquired along 50 directions with a b-value of 1500 s/mm 2 (as well Vol. 105, No. 1, January 2017 | Proceedings of the IEEE 85 as one image with no diffusion weighting) with a voxelsize of 2 Â 2 Â 2 mm 3 (TR ¼ 8996 ms, TE ¼ 80 ms, FOV ¼ 224 mm, 75 axial slices AC/PC aligned encompassing the whole brain, SENSE Factor ¼ 2).
Simultaneous and continuous EEG data were acquired with a custom built MR-compatible EEG system [7], [9], [25]. This system included a differential amplifier and a bipolar EEG cap with 36 Ag/AgCl electrodes (including the left and right mastoids) arranged as 43 bipolar pairs. In order to minimize noise from subject head motion in the main magnetic field and from inductive pickup from magnetic gradient pulses, we used twisted bipolar pair leads. The 488-Hz-sampled EEG was synchronized with the scanner clock at the start of each functional image acquisition by sending a transistortransistor logic (TTL) pulse to the recording computer. This was used in the gradient artifact removal during the offline EEG data preprocessing steps. 10-k resistors were built into each electrode to ensure subject safety, and all electrode impedances were kept below 20 k.

D. EEG Preprocessing
EEG preprocessing was done with Matlab (Mathworks, Natick, MA, USA). First, gradient artifact removal was performed using a template subtraction algorithm [7]. Then, a software-based 0.5-Hz high-pass filter was used to remove direct current (dc) drifts, a 60-Hz (harmonic) notch filter to minimize line noise artifacts, and a 100-Hz low-pass filter were applied before resampling the data to 256 Hz. These filters were designed to be zero phase to minimize delay distortions. Stimulus events-i.e., countdown, pitch type, responses-were recorded on separate channels.
After filtering, ICA was run using EEGLAB [26] and the FastICA [27] algorithm to remove eye-blink artifacts and other non-EEG artifacts. In stimulus-locked epoching (from 1500 to 2000 ms), the average baseline was removed using data from 200 to 0 ms. An automatic artifact epoch rejection algorithm from EEGLAB was run to remove all epochs that exceeded a probability threshold of five standard deviations from the average. Trials where the subject's response time (RT) was earlier than 100 ms from pitch onset were excluded from further analysis.
Ballistocardiogram (BCG) artifacts were removed from the continuous gradient-free data using a principal components analysis method [7], [25]. First, the data were low passed at 4 Hz to extract the signal within the frequency range in which BCG artifacts are observed and then the first two principal components were determined. The channel weightings corresponding to those components were projected onto the broadband data and subtracted out. These BCG-free data were then rereferenced from the 43 bipolar channels to the 34-electrode space to calculate scalp topographies of EEG discriminating components.

E. Behavioral Analysis
Percent error rates and RTs were analyzed. Errors were broken down into both omissions and commissions, i.e., no-responses and late responses in Go trials and button presses in No-Go trials. Repeated-measures ANOVAs on each behavioral measure were carried out using Trial type (two levels: Go, No-Go) as the within-subject factor and group (player/control) as the between subject factor. The Greenhouse-Geisser (GG) epsilon correction was applied to adjust the degrees of freedom of the F ratios where necessary, and post hoc comparisons were also made in order to determine the significance of contrasts by applying the Bonferroni procedure ðalpha ¼ 0:05Þ.

F. Single-Trial Analysis of EEG
Our analysis focused on a single-trial approach to discriminate between a set of stimulus or response conditions. First, we considered only behaviorally correct trials. Regularized logistic regression was used as a linear classifier to find an optimal projection for discriminating between behaviorally correct Go and behaviorally correct No-Go trials over a specific temporal window [28]. This approach has been previously applied to identify neural components underlying rapid perceptual decision making [7], [9], [20], [23], [29]. Specifically, we defined a training window starting at either a prestimulus or poststimulus onset time , with a duration of , and used logistic regression to estimate a spatial weighting vector that maximally discriminates between EEG sensor array signals X for each class (e.g., Go versus No-Go trials) In (1), X is an N Â T matrix (N sensors and T time samples). The result is a "discriminating component" that is specific to activity correlated with each condition, while minimizing activity correlated with both task conditions. For our experiments, the duration of the training window ðÞ was 50 ms and the center of the window ðÞ was varied across time in 25-ms steps. We used the reweighted least squares algorithm to learn the optimal discriminating spatial weighting vector [30]. We quantified the performance of the linear discriminator by the area under the receiver operator characteristic (ROC) curve, referred to here as AUC, using a leave-one-out procedure. We used the ROC AUC metric to characterize the discrimination performance as a function of sliding our training window from 0-ms prestimulus to 1000-ms poststimulus (i.e., varying ). We quantified the statistical significance of AUC in each window ðÞ using a label permutation procedure. We randomized the labels for each trial (i.e., trial was a Correct Go or a Correct No-Go) and retrained the classifier. This was done 1000 times for each subject, and the AUC values from these permutations were used to establish a p-value for the mean AUC in each time window. All significant results have been corrected for multiple comparisons using a Bonferroni correction at p G 0:05.

G. fMRI Preprocessing
Using FSL (FMRIB Software Library; [31]), we performed bias-field correction on all images to adjust for field distortion artifacts caused by the EEG wires. We then performed slice-timing correction, motion correction, 0.01-Hz high-pass filtering, and 5-mm full-width half-maximum spatial smoothing on the functional data. Motion correction provided motion parameters that were included as confounds in the subsequent GLM. To help reduce noise in our fMRI data, MELODIC de-noising was applied to the functional data using the methodology described in [32]. Functional and structural images were then registered to a standard Montreal Neurological Institute (MNI) brain template after brain extraction, and each subject's image registration was checked manually to ensure proper alignment.

H. Player Versus Control Traditional fMRI Analysis
We first ran a traditional fMRI analysis using eventrelated and RT variability regressors in a GLM. The event-related regressors were composed of boxcar functions with unit amplitude and onset and offset matching those of the stimuli (Correct Go, Correct No-Go, Incorrect No-Go, and a bad trial/Incorrect Go trial regressors). RT variability was modeled using parametric amplitude boxcars with onset/offset matching the stimulus, and these were orthogonalized to the respective event-related regressors. Orthogonalization was performed in FSL using its Gram-Schmidt procedure [33] to decorrelate the RT regressor from all other event-related regressors. All regressors were convolved with the canonical hemodynamic response function (HRF), and temporal derivatives were included as confounds of no interest. A fixed effects model was used to model activations across runs, and a mixed effects approach (FLAME 1 þ 2 [31]) was used to compute the contrasts for traditional players versus controls to identify activation patterns for the Correct Go and Correct No-Go conditions as well as the difference between Correct Go versus Correct No-Go contrasts. Statistical image results for these traditional analyses were thresholded at z 9 1:8, and clusters were corrected for multiple comparisons at p ¼ 0:05 [34], [35].

I. Player Versus Control EEG-fMRI Fusion Analysis
We created a novel methodology to fuse EEG-fMRI data and study group differences. First, we identify time windows in the EEG signal that discriminate between players and controls, and we then use these time windows as regressors in a GLM analysis of the fMRI data.
In accordance with previous methods [7], [9], [10], our methodology uses EEG trial-to-trial variability to index a brain signal of interest and predict subject expertise between players and controls. First, a sliding window linear discrimination analysis, based on logistic regression, was run separately on Correct Go and Correct No-Go trials to classify each trial as belonging to either a player or control. Instead of processing the data within subjects, the time window ðÞ data were pooled across subjects to create a data n Â s matrix, n-trials (7213 for Correct Go, 2791 for Correct Go) by s-subjects (35). A sevenfold logistic regression was run independently for each trial type, where for each fold the data from one player and two controls were held out for testing. The time window center defining the data input for the logistic regression was varied across the trial, starting from 0 ms from stimulus onset and shifted by 25 ms until the final window at 1000 ms. The accuracies of the classifiers were assessed using the AUC [ Fig. 1(a)].
After applying logistic regression to the time windows, each subject's y-values [see (1) and Fig. 1(b)] were averaged and divided by the standard deviation of the individual subject's trial-wise y-values to create an overall "expertise-rating" y-value, Y ;Subj n [ Fig. 1(c)]. For each time window in each data set, the AUC was computed by comparing the overall expertise y-value, Y ;Subj n , to the ground-truth label of player or control. The significance of the AUC for each time window was determined using a permutation test. The label of player or control was randomly assigned 1000 times for each window, the sevenfold logistic regression was performed, and a distribution of values for AUCs with random labels were used to compute a significance threshold for the null hypothesis that there was no EEG marker of expertise. Windows were considered significant if they passed an FDRcorrected threshold of p G 0:05.
To fuse the EEG results with fMRI, we used time windows that discriminated players from controls in EEG analysis as regressors of interest in a GLM that correlated subject expertise with fMRI activation. Resulting statistical parametric maps for these analyses were thresholded at z 9 1:8, and clusters were corrected for multiple comparison at p ¼ 0:05 [34], [35].

J. Structural Connectivity Differences Between Player and Control
Diffusion MRI analysis [36] was performed using DSI Studio (http://dsi-studio.labsolver.org/) and Matlab (MathWorks, Inc.; Natick, MA, USA) on 37 subjects (13 experts, 24 novices). Diffusion data were reconstructed using q-space diffeomorphic reconstruction (QSDR [37]) with a diffusion sampling length ratio of 1.25 and an output resolution of 2 mm. Whole-brain fiber tractography [38] was performed 1000 times for each participant to minimize the impact of any bias in the tractography parameter scheme on streamline generation. Across the 1000 iterations, values were randomly sampled for QAbased fiber termination thresholds (between 0.01 and 0.10), turning angle thresholds (between 40˚and 80˚), and smoothing (between 50% and 80%) to ensure results are robust to these parameter choices, while using constant values for step size (1 mm) and min/max fiber lengths (10 mm/400 mm). Each iteration generated 250000 streamlines, and with a fixed streamline count, differences in the number of estimated fiber tracts can be interpreted as differences in the strength of connection [39]. For each subject, we derived a whole brain structural connectivity matrix by averaging the tractography estimates across the 1000 iterations to estimate the strength of connection between all 116 brain regions in the AAL atlas.
The 116 Â 116 structural matrix for each subject was used as input in an ROI connectivity analysis. The Louvain modularity algorithm implemented in the Brain Connectivity Toolbox (BCT, [40]) was run on the group average unthresholded streamline connectivity matrix. The order of nodes in the connectivity matrix was reorganized based on the representative modularity partition with five modules. After partitioning, the number of streamline edges differing between groups with a threshold of p G 0:05 uncorrected was assessed for each of the identified modules both for within-module and between-module connections. This type of analysis has been used to identify structural differences between autistic patients and controls [41]. A permutation test was implemented to assess if the distribution of significant edges within and across modules deviated from a null hypothesis that there was no network difference between experts and controls. The label of player or control was randomly assigned 5000 times, and for each permutation, a count of significant edges between and within modules for each group (player 9 control and control 9 player) was calculated. Each module-to-module count was then compared to its permutation distribution to determine significance at p G 0:05. Table 1 presents group data for response times and error rates for Go and No-Go trials. A two-way ANOVA on the response times showed a significant effect for the Group ðp ¼ 0:0024Þ. Trial type ðp ¼ 0:24Þ and the Group Â Trial interaction ðp ¼ 0:24Þ did not pass our significance threshold of p G 0:05. The two-way ANOVA for error rates showed a significant main effect for Group ðp G 0:001Þ, Trial Type ðp G 0:001Þ, but the Group Â Trial interaction ðp ¼ 0:944Þ was not significant.

B. Confirming Previous EEG-Only Neural Correlates of Player Versus Control Differences
Given that we acquired our EEG and fMRI simultaneously, which can result in reduced SNR relative to separate acquisitions, we first sought to confirm results we reported in a previous EEG-only acquisition and analysis [20]. Fig. 2(a) shows the mean (across subjects, separated by group) performance (area under the ROC curve: AUC) for stimulus-locked EEG components discriminative of Correct Go versus Correct No-Go discrimination trials. We found that players and controls had similarly shaped discrimination curves; however, players exhibited an earlier rise and larger peak than controls. Both groups showed no significant early discrimination (discrimination before 300 ms); however, discrimination rose sharply to a maximum AUC of 0.87 at 500 ms for players and 0.79 at 525 ms for controls. The players' discrimination curves also were shifted 25 ms earlier relative to that for the controls. To test for significant discrimination differences between players and controls, we computed an independent groups t-test at each window. Shaded regions indicated significant differences (p G 0:05 FDR corrected) in discrimination activity between players and controls. Players show significantly higher discrimination than controls from 400 to 500 ms. These results are all consistent with our previous EEG-only analysis [20].

C. Player Versus Control Traditional fMRI Results
We conducted a traditional GLM analysis to demonstrate what an fMRI-only analysis would reveal on the differences between players versus controls. After computing the traditional fMRI contrasts for each subject, an independent two groups t-test was run comparing the players' and controls' Correct Go and Correct No-Go subject level beta estimates. Significant clusters showing higher activations for players were found in both the Correct Go and Correct No-Go trial types [ Fig. 2(b) and (c)]. Activations were located in the temporal fusiform gyrus, middle temporal gyrus, anterior cingulate, presupplementary motor and supplementary motor cortices for Correct Go trials. Similarly for Correct No-Go trials, activations were found in the presupplementary motor area, supplemental motor areas, middle temporal gyrus, and fusiform gyrus.

D. Unique Functional Network Differences Between Player Versus Control Revealed via EEG-fMRI Fusion
The AUCs for sliding window logistic regression that discriminated expertise in Correct Go and Correct No-Go trials are plotted in Fig. 3(a). Sliding window AUC results for classifying each subject by expertise based on the sevenfold logistic regression y-values are plotted in Fig. 3(b). The expertise rating y-values from the significant windows from the EEG analysis [ Fig. 3(b)] were used as covariates of interest in finding areas of the brain that correlate with the EEG expertise measures derived across the trial duration. For Correct Go trials, significant clusters were found in almost all of the significant EEG expertise windows [ Fig. 3(c)]. Negative correlations were found more in the earlier time windows (G 400 ms). Significant positive correlation clusters were found in regions overlapping with the traditional player 9 control contrasts. Significant negative correlation clusters were found during the 125-ms window in the intracalcarine cortex and precuneus, during the 275-ms window in the superior lateral occipital cortex (LOC) and middle frontal gyrus, and during the 375-ms window in the inferior LOC. For positive correlations, significant clusters were found distributed across the entire cortex and trial duration. Significant positive correlation clusters were found during the 250-ms window in the precentral gyrus and SMA, during the 275-ms window in the superior frontal gyrus (SFG) and middle temporal gyrus (MTG), during the 375-ms window in the hippocampus and fusiform gyrus, during the 425-ms window in the posterior cingulate, and during the 525-ms window in the supra marginal gyrus among others (Table 2).
For Correct No-Go trials, significant clusters were found in almost all of the significant time windows [ Fig. 3(d)]. Negative correlations were found more in the later time windows (9 400 ms) after the peak of expertise discrimination. Significant positive correlation clusters were found in regions overlapping with the traditional player 9 control contrasts. However, significant clusters in the correlation analysis are distributed far more broadly than in the traditional analysis. Positively correlated clusters were found during the 225-ms window in the SMA, temporal and frontal poles, during the 275-ms window in the SFG, lingual gyrus, and central opercular cortex, during the 375-ms window in the hippocampus and inferior LOC. Negatively correlated clusters were found during the 250-ms window in the middle frontal gyrus (MFG), during the 350-ms window in the occipital pole, during the 375-ms window in the subcallosal cortex, during the 400-ms window in the superior frontal gyrus, during the 425-ms window in the preSMA, and during the 475-ms window in the inferior LOC among others (Table 3).

E. Structural Connectivity Differences Between Player Versus Control
To complement our functional connectivity analysis, we also analyzed the DTI data to identify structural networks  that differentiate the two groups [42]. Whole brain tractography analysis showed a robust structural network of nodes derived from the AAL atlas across all subjects [ Fig. 4(a)]. The modular organization of the structural network found five communities of nodes across the brain [ Fig. 4(b)]. Communities were roughly organized by hemisphere (LH-1,4;RH-2,3), cortical/subcortical motor and frontal regions (modules 1 and 2), visual/occipital regions (modules 3 and 4), and cerebellar regions (module 5). We first examined expertise-related structural effects by directly comparing the strength of structural connectivity between brain regions, and we found that players had 1.7 times as many connections as the controls, 548 in players versus 328 in controls [p G 0:05, uncorrected, Fig. 4(c)]. Additionally, the difference in structural connectivity was mostly found between modules [ Fig. 4

IV. DISCUSSION
In this work, we used multimodal neuroimaging to identify structural and functional brain networks that differentiate a group of baseball players from a control group  when they performed a Go/No-Go task designed to mimic the situation of hitting a baseball. Below we discuss how our novel multimodal fusion approach advances our understanding of the structural and functional correlates of expertise, specifically expertise in hitting a baseball, while also relating it to previous work on rapid decision making.
This study demonstrates that simultaneously acquired EEG-fMRI can be used to infer functional networks and offer confirmatory evidence for source localization findings, including those estimated from EEG-only acquisitions [20]. Since source localization is an ill-posed problem, the localization cannot be considered conclusive; however, simultaneous EEG-fMRI enables a withinsubject and within-trial comparison of the brain regions identified in functional analyses from the complementary neuroimaging methodologies. Here, we show that activity in fusiform gyrus that was identified in our traditional fMRI analysis of group differences between players and controls [ Fig. 2(b) and (c)] matched our EEG findings that players have a larger activation in a source localized in the fusiform region. This confirmatory, multimodal result adds to the growing literature that the fusiform gyrus plays a significant role in the expertise-dependent visual object recognition [43]- [47]. Players also had a larger activation in the middle temporal gyrus (MTG) specifically in the left visual area MT/V5 complex which may also give players superior performance as this area is implicated in motion processing. Another area where players exhibited stronger activations was the supra-marginal gyrus. This area is part of the action observation network (AON) and plays a role in the somatosensory processing stream. Surprisingly, we also see activation in the SMA in both the Correct Go and Correct No-Go player/control contrasts. The location of this activation is similar to the area found in our previous Correct No-Go EEG source localization results [20], providing confirmatory evidence that players preferentially activate their SMA, relative to controls, during this baseball-like task. SMA regions, including the pre-SMA, have a known role in motor learning [48]- [51] and critical involvement during Go/No-Go tasks which probe inhibitory control circuits [20], [52]- [56].
In addition to confirming previous results, simultaneously acquired EEG-fMRI allows for a more comprehensive understanding of the differences between players and controls with respect to the spatiotemporal cascade of activity across the brain. Our novel methodology identifies multiple poststimulus 50-ms windows with predictive EEG neural correlates of expertise and fuses these temporal windows with fMRI activity in a whole-brain 2-mm voxel analysis, revealing time-localized correlations of expertise at a spatial scale of millimeters. Many of the significant regions found in the fusion analysis were also observed in the traditional player versus controls contrast, though the fusion analysis enabled a deconstruction of this activity across time. Additionally, some areas that significantly correlate with expertise in the fused analysis were not present in the traditional analysis, including the SFG, the hippocampus, and all regions with significant negative correlations [ Fig. 3(c) and (d)]. These novel spatiotemporal findings suggest that the fused approach may provide more sensitivity than a traditional fMRI-only GLM analysis. Interestingly, the fused approach identified regions in early visual processing areas-the temporal occipital fusiform cortex (TOFC), parahippocampal gyrus, and paracingulate gyrus. Activity in these regions were significantly correlated with expertise, and they are the same areas known to be used in Bar's visual prediction theory [57]. In addition, significant positive correlations in the right SFG at 275 and 300 ms for Correct Go and Correct No-Go trials maps directly to a region known to integrate information from the visual processing areas [58]. This functional evidence taken together helps to support the theory that expertise-specifically, sportive expertise-can produce more efficient neural processing for domain specific perceptual tasks [59]- [61].
This novel fusion methodology is fully data-driven and uses the entire EEG sensor and fMRI voxel space to identify the functional cascade that differentiates two groups. To date, the majority of EEG-fMRI studies use correlative measures to inspect the EEG-informed BOLD modulations [62], and relatively little previous work has used EEG-fMRI fusion methodologies to identify differences between subject populations. One recent exception is [63], who used a joint independent component analysis (jICA, [64]) with simultaneous EEG-fMRI to show that schizophrenic patients have marked differences in processing oddball stimuli compared to controls, but their methodology only used a single electrode for the EEG analysis and requires user supervision to determine ICA components.
Our data-driven methodology takes a more complementary approach to fuse neural information across EEG and fMRI methodologies since it is well known that each neuroimaging measurement may reflect characteristics of (c) Structural connectivity matrix group differences (p G 0:05, uncorrected) displaying expert baseball players 9 controls (red) and controls 9 players (blue) for streamline counts in the connectivity matrix and 3-D brain space. (d) Number of player 9 control (red) and control 9 player between group connections differing for within-module and between-module connections (left) and specific module-to-module results (right). Significance (p G 0:05 FDR corrected) for module-to-module differences was computed by 5000 permutations. Significant module-to-module differences are marked by " ** ." different populations of cells [65]. Here, we use all EEG electrodes to identify temporal windows with neural correlates of expertise that can successfully classify players versus controls and then use this temporal information in a whole-brain GLM analysis of fMRI data to investigate which regions of the brain covary with the predictive EEG signals of expertise. While our results confirm the promise of our EEG-fMRI fusion approach, future research should continue to explore additional methods to extract the strengths of each neuroimaging modality and mitigate known weaknesses, allowing additional hybrid analyses to expand our understanding of relationships between complementary neuroimaging signals.
Our fusion approach for simultaneously collected EEG and fMRI data provides a functional mapping of expertise related differences between the players and controls. It is also important to identify and understand if there are structural differences between the groups. Structural connectivity analysis showed that the players have significantly more coherent structural connections between cerebellar and left frontal/motor regions [ Fig. 4(d)]. These trends point to the players having neuroplastic changes specific to motor processing regions of the brain. This is more clearly shown in the overlap between the functional activations from the EEG-fMRI fusion and the structural connectivity (Fig. 5). This fronto-cerebellar pattern is particularly interesting given that there is a well-established pattern of connectivity between lateral frontal areas and lateral regions of the cerebellum, consistent with the location of expertise predicting activity in our task [66]. Rather than regulating motor coordination, as is usually assumed with cerebellar pathways, these cortical-cerebellar networks are thought to regulate the integration of high-level executive and attention processes that are critical for efficient, adaptive decision making [67]. We found that expert players have greater network-level communication, at both the structural and functional levels, between these fronto-cerebellar circuits. This between-module communication is a plausible neural substrate that can explain the improved behavioral performance at a sensory discrimination task with minimal movement control demands.
In summary, our results indicate a difference in the unfolding of cognitive processes for players versus controls and that these functional differences may at least be partially a result of differences in structural networks between the groups. We find correlative evidence that these macroscale neural differences translate into higher behavioral accuracies and faster response times in players. The spatiotemporal cascade reflecting these differences between the groups begins as early as 200 ms after the pitch starts and lasting up to 700 ms afterwards. Network differences are spatially localized to include motor and visual processing areas, providing evidence for differences in perception-action coupling between the groups. These findings reinforce many studies implicating these areas in mediating visual prediction and expertise [43], [47], [57], [68]- [70]. We also find that our results confirm many prior fMRI studies showing that athletes have stronger activations in the action observation network while they observed or listened to the domain of their expertise [59]- [61], [71]- [76].
In general, our approach illustrates how multimodal neuroimaging can provide specific macroscale insights into the functional and structural correlates of expertise development. This approach, however, may also capture underlying physiology that can account for variability in performance, whether it arises from between subject differences due to genetic or experimental factors or from within subject variability due to fluctuations in attention, interest, etc. Future work should examine the sensitivity of this multimodal approach to capture variability across varying levels of expertise, providing a framework to reveal how brain connectivity enables superior performance. h  Functional EEG-fMRI fusion results for Correct Go trials summed across significant temporal windows displayed on the MNI brain. Structural modular regions that were found to have significantly more player 9 novice streamlines between them (module 1-green and module 5-purple) are also displayed. Functional fusion activations of player 9 control overlap with the structural regions that show differences between groups.