Relating Cognition to both Brain Structure and Function: A Systematic Review of Methods

Introduction: Cognitive neuroscience explores the mechanisms of cognition by studying its structural and functional brain correlates. Many studies have combined structural and functional neuroimaging techniques to uncover the complex relationship between them. In this study, we report the first systematic review that assesses how information from structural and functional neuroimaging methods can be integrated to investigate the brain substrates of cognition. Procedure: Web of Science and Scopus databases were searched for studies of healthy young adult populations that collected cognitive data and structural and functional neuroimaging data. Results: Five percent of screened studies met all inclusion criteria. Next, 50% of included studies related cognitive performance to brain structure and function without quantitative analysis of the relationship. Finally, 31% of studies formally integrated structural and functional brain data. Overall, many studies consider either structural or functional neural correlates of cognition, and of those that consider both, they have rarely been integrated. We identified four emergent approaches to the characterization of the relationship between brain structure, function, and cognition; comparative, predictive, fusion, and complementary. Discussion: We discuss the insights provided in each approach about the relationship between brain structure and function and how it impacts cognitive performance. In addition, we discuss how authors can select approaches to suit their research questions. Impact statement The relationship between structural and functional brain networks and their relationship to cognition is a matter of current investigations. This work surveys how researchers have studied the relationship between brain structure and function and its impact on cognitive function in healthy adult populations. We review four emergent approaches of quantitative analysis of this multivariate problem; comparative, predictive, fusion, and complementary. We explain the characteristics of each approach, discuss the insights provided in each approach, and how authors can combine approaches to suit their research questions.


INTRODUCTION
Cognitive function and adaptive behaviour rely on structure and dynamics of largescale neural networks (Friston, 2002).Early cognitive neuroscience separately assessed how properties and characteristics of brain structure and function might impact upon performance of cognitive tasks.Research employing the structural modality focused on studying physical properties of the brain, such as cytoarchitecture and neuronal integrity, whereas research employing functional approaches assessed characteristics of neuronal activity observed during performance of cognitive tasks and during rest (Rykhlevskaia et al., 2008).However, in recent years some attempts have been made to integrate the two approaches.Authors have begun to investigate how structure and function of the human brain relate to each other by assessing correspondence between findings from the two modalities (Johansen-Berg et al., 2004;Rykhlevskaia et al., 2008).This comparative approach produces a more complete understanding of healthy cognitive function across human lifespan (de Kwaasteniet et al., 2013;Guye et al., 2010;Hahn et al., 2013;Salami et al., 2016;van den Heuvel & Fornito, 2014;Minghui Wang et al., 2016).
There is a complex relationship between brain structure and function (Rykhlevskaia et al., 2008;Suárez et al., 2020).Independent labs have found a striking similarity between patterns of white matter fibres and functionally meaningful parcellations of the cortex (Greicius et al., 2009;Johansen-Berg et al., 2004;Jung et al., 2017).For example, the most central nodes of functional networks are directly and strongly connected by white matter tracts (Greicius et al., 2009).Some studies have focused on a temporal association of activity across remote regions, which is interpreted as interaction across these regions and commonly referred to as functional connectivity (Friston, 2002).Studies that compare patterns of structural white matter connectivity and functional connectivity have found moderate correspondence in structural and functional connectivity (Honey et al., 2009;Parker et al., 2003;Sporns et al., 2011;Wang et al., 2013).This indicates that there are many regions that are not directly connected, but still can show functional interactions (Ashourvan et al., 2019;Hagmann et al., 2008;Honey et al., 2009;Honey et al., 2010;Liao et al., 2015;Røge et al., 2017;Sun et al., 2017;Thomas et al., 2009).This implies that there are regions that are indirectly connected with each other and evidence demonstrates that accounting for indirect connections improves correspondence between structural and functional connectivity (Honey et al., 2009).This evidence illustrates that there is a complex and non-trivial relationship between brain structure and function.As a result of this complexity, it becomes challenging to interpret patterns of results in cognitive neuroimaging investigations when neural structure and function diverge, yet it is possible that divergence provides important information about the mechanisms involved.
Researchers have demonstrated that both regional and inter-regional relationships between brain structure and function can profoundly influence cognition in healthy and clinical populations.For example, one study investigated structural and functional differences across two aging groups with good and poor episodic memory (Persson et al., 2006).It was found that that severe decline in episodic memory was uniquely associated with reduced integrity of white matter in the anterior part of the corpus callosum and increased activity in right prefrontal cortex during episodic encoding.It was argued that the unique activity in right frontal regions observed for the older group with memory impairment may have been a compensatory mechanism for the structural disruption.In another example, both SC and FC have both been found to be decreased in temporal lobe epilepsy patients as compared to controls (Liao et al., 2011;Zhang et al., 2011).Furthermore, similarity between SC and FC has been found to be decreased in people with epilepsy (Chiang et al., 2015).In particular, this decoupling was then modulated by duration of epilepsy and structural changes to individual regions, which were unique to patients with left versus right temporal lobe epilepsy.Unique patterns of disruption of coupling between brain structure and function have been reported in other aspects of aging including emotion processing, executive function, language, motor function inhibition (Ford & Kensinger, 2014;Hu et al., 2013;Mander et al., 2017;Ritchie et al., 2018;Sun et al., 2017), and clinical disorders including schizophrenia, depression autism, stroke, dementia and many others (Anderson et al., 2011;Carter et al., 2009;Cocchi et al., 2014;Hojjati et al., 2018;Min Wang et al., 2016;Weinstein et al., 2011).In addition, several regression studies suggest that the relationship between structure and function contributes unique variance to explanation of cognitive performance (Dhamala et al., 2021;Mansour et al., 2021;Robinson et al., 2021).
To provide an overview of trends and developments within this research field, the present systematic review assesses how researchers have attempted to combine structural and functional brain imaging data in healthy adult populations.The review considers the findings to date, relevant methodological considerations, and outstanding areas that need to be addressed.Through this we hope to gain a better understanding of the state of the field and highlight the potential of combining structural and functional neuroimaging data.

METHODOLOGY
The present work has been conducted in accordance with the guidelines for systematic reviews (Moher et al., 2009) The following information was recorded during data collection: cognitive task, neuroimaging acquisition protocols and paradigms, neuroimaging data pre-processing, outcome measures of neuroimaging data, scales of neuroimaging analysis, methods of integrating information about brain structure and function.

Literature Search and study characteristics
The process of identification, screening and selection of studies presented in Figure 1 has been obtained from PRISMA guidelines (Moher et al., 2009).First, 1923 articles were identified during database search and 251 records duplicates were removed.Then, 1513 records were removed during screening of titles and abstracts.Next, 159 articles were submitted to full-text assessment of eligibility, which resulted with further elimination of 58 articles.Overall, the selection process resulted with 102 articles that were included in our literature review and accounted for 5% of initial search results.Figure 2 illustrates the number of identified papers for each year.Figure 2 A time line represents count of selected publications for each year.

Cognitive Domains
The selected papers covered a wide variety of cognitive domains and their respective processes with many papers investigating multiple processes across many domains.Figure 3 illustrates the frequency of cognitive processes across the selected literature.Overall, the most investigated cognitive domain was language (featured in 21 articles), followed by memory (featured in 19 articles) and working memory (featured in 19 articles).
Figure 3 A distribution of cognitive processes across selected literature.Occurrence of cognitive processes was counted and if an article has investigated multiple cognitive process then the count was the fraction of all processes featured in the article.

Neuroimaging data and data analysis
The selected papers have shown an even balance between analysis of magnetic resonance imaging (MRI) and diffusion tensor imaging (DTI) (Figure 4), but functional MRI (fMRI) dominated the research field (Figure 5).The functional imaging was dominated by task paradigms, but resting state paradigm has also featured in many articles (Figure 6).
Figure 4 A distribution of structural neuroimaging methods across selected literature.
Figure 5 A distribution of functional neuroimaging methods across selected literature.
Figure 6 A distribution of functional neuroimaging paradigms across selected literature.

Methods of integrating structural and functional data
Studies were categorised by the types of inference they made about the relationship between brain structure and function: i) indirect, ii) semi-direct and iii) direct.First, studies that conducted separate analysis of brain structure and function without any quantitative evidence of their relationship were classified as making indirect inferences.Second, semidirect inference referred to studies where authors have not provided statistical analyses of how brain structure and function relate to each other, but the analyses of each modality allowed some inference about how the modalities can relate.To illustrate, some studies have obtained common measures of properties of both structure and function but they have not tested how similar or different these measures are across modalities.In another example, some authors have used the topological location of results obtained for one modality to narrow down analysis of the other modality through informing location of ROI.
Third, direct inference was defined as inference made with quantitative evidence to support the interpretation of results.The use of these methods of inference across studies is shown in Figure 7. Indirect inference was most common as it was used in 55 studies (53% of selected articles); flowed by direct inference in 29 studies (28%) and 4 of these latter studies used both direct and semi-direct inference (4%).14 studies (14%) used semi-direct inference alone.
Figure 7 The percentage of structural and functional neuroimaging studies using different types of inference to understand their relationship.Direct inference was defined as inference backed by quantitative analysis of similarity in patterns observed between approaches.Semi-direct direct inference was defined as inference where analyses did not conduct direct inference, but some inference about how the modalities relate was still possible.Indirect inference was defined as inference made when neural structure and function were analysed separately.
The following methods of comparing brain structure and function were used by studies that only employed semi-direct inference.One study obtained volumetric measures in each neuroimaging dataset, one study assessed how much impact genetics had on each neuroimaging dataset, and one study obtained rich club coefficient in each neuroimaging dataset, and 11 studies used results from one modality to determine ROIs for the analysis of the other modality.As further illustrated in Figure 7, four studies combined direct and semidirect approaches.Those studies used the results-driven ROI approach and conducted direct comparisons between structure and function through various methods, including correlation analyses and impact of structural information on models of effective connectivity.
The variety of direct inferences in quantitative analyses across studies are shown in

Discussion
The relationship between brain structure and function may have profound consequences for understanding cognition (Ford & Kensinger, 2014;Hu et al., 2013;Mander et al., 2017;Ritchie et al., 2018;Sun et al., 2017).In this systematic review we determined how structural and functional neuroimaging methods have been integrated to study cognitive function and adaptive behaviour.A search was conducted across three databases in accordance with PRISMA guidelines (Moher et al., 2009).We assessed the prevalence of studies combining structural and functional neuroimaging data for explaining cognition and evaluated their choice of methods.The results demonstrate that there are to date relatively few studies attempting to combine structural and functional neuroimaging data and most studies use that do use indirect methods to infer the relationship between brain structure and function without formally relating these measures.Here, we consider what these findings mean for the field and how the shift towards direct inference with quantitative methods can lead to greater insight into how the structure and function of the brain combine to effect cognition.
First, this systematic review assessed the prevalence of studies quantitatively combining structural and functional data to explain cognition.Only 5% of the initial search results met the eligibility criteria, i.e. 102 out of 1923 studies examining links between structural and functional data and cognition in healthy adults.Investigations that address this subject have the potential to produce more complete understandings of healthy cognitive function, particularly given that combining brain structure and function information explains more variance in cognitive performance than either modality alone (Dhamala et al., 2021;Jiang et al., 2019;Robinson et al., 2021).These studies can also provide new insight, by determining e.g.causal interactions between regions (Sokolov, Zeidman, Erb, Ryvlin, Friston, et al., 2018;Sokolov, Zeidman, Erb, Ryvlin, Pavlova, et al., 2018), or how new learning and training can result in neuroplasticity (Sun et al., 2016;Yang et al., 2019).Understanding relationships between brain structure, function and cognition can also provide insight into how these relationships breakdown in neurological and psychiatric disorders.The present review's selection criteria eliminated investigations of atypical populations, but some of the selected papers were then implemented in investigations into mechanisms of disease and recovery.To illustrate, Yang et al. (2019) investigated the effects of mindfulness training and were considered in later studies of general wellbeing of healthy populations (Tortella et al., 2021), improvements in cognitive function of diabetic patients (Alipor et al., 2019), and recovery from depression (van der Velden et al.).In another example, Jung et al. ( 2018) investigated how organisation of structural and functional connectivity relates to language, work that had direct implications in understanding language deficits in semantic variant primary progressive aphasia (Battistella et al., 2019) and temporal lobe epilepsy (Black, 2020).This illustrates the potential impact from a deeper understanding of how brain structure and function integrate.
Having explored the prevalence of research on neural substrates of cognition in the literature, the present review considered methods for integrating across modalities.Three approaches to relating structure and function were observed: (i) a direct inference based on quantitative evidence, (ii) semi-direct inference based on closely related or similar processing steps, (iii) indirect inference based on separate analysis of the two approaches.
Indirect inference was the most common approach.During this process, experimenters initially investigated how brain structure or function impacted cognitive function, next they inferred the degree of similarity between the two modalities.However, this process of investigation does not provide any information about how much additional explanatory information is gained by analysing convergent structural and functional neural data.This means that research is focused on exploring the similarities and differences between brain structure and function, but it is still largely unknown how much the nature of this relationship impacts upon mechanisms of cognitive performance.Here, we recommend easily remedying this ambiguity by implementing measures of explained and residual variance in cognitive performance.By comparing the amount of explained variance in multimodal models, compared to single modality investigations, we can start to build better and perhaps more realistic models of cognition and its neural underpinnings.This will advance our understanding of which structural and functional properties are the most important and impactful on cognition, and how these may become vulnerable in the case of disease.
The present review also focused on methods used to make direct inferences.We can roughly place these methods under the umbrellas of four approaches; i) comparative approach, ii) predictive approach, iii) fusion approach and iv) complementary approach.
First, the comparative approach assesses differences in characteristics of each network through measures of distance and significance testing.We have found that data was most commonly related across modalities with analysis of similarity such as correlation and quantification of results cluster overlap (Noonan et al., 2018).They were used to assess both linear and non-linear relationships between a variety of structural and functional properties.Structural properties included volumetrics values like cortical thickness and cortical surface area (Robinson et al., 2021), microstructure like grey and white matter directionality (e.g.fractional anisotropy) (Chavan et al., 2015), white matter connectivity like number of tract streamlines connecting region pairs (Sokolov, Zeidman, Erb, Ryvlin, Friston, et al., 2018), and symmetry scores assigned to reflect how symmetric these properties are across the two brain hemispheres (Josse et al., 2009).Functional properties have included properties of evoked activity like strength, count of activated voxels and laterality of activation (Zuo et al., 2016), and strength of FC as reflected by the correlation in signal intensities across remote regions (Robinson et al., 2021).Finally, inferential statistics have been used to assess if organisational characteristics of structural and functional networks significantly differ (Jung et al., 2018).The second approach of relating brain structure and function involves production of predictive models of cognition.Authors have employed multiple regression (Ford & Kensinger, 2014;Jung & Kim, 2020;Putnam et al., 2008), canonical correlation analysis (Han et al., 2020;Lerman-Sinkoff et al., 2017), partial least squares (Dzafic et al., 2019) and connectome-based predictive modelling (Jiang et al., 2019).
One study started with sparsity-constrained principal component regression in each modality and the selected features of each modality were fed-forwards to a lasso analysis that integrated the models to a single model (Robinson et al., 2021).Another study produced predictions of cognitive performance separately using structural and functional connectivity and then calculated the average of the two predictions (Bajaj et al., 2021).
Next, the fusion approach was observed where structural and functional information was fused prior to relating them to cognition.One such approach involved Independent Component Analysis, which was conducted on both connectivity sets and then canonical correlation analysis conducted on the resulting components (Lerman-Sinkoff et al., 2017).In another study, brain structure and function were fused, where structural connectivity produced a prior distribution of functional connectivity, which was then related to cognition (Xue et al., 2015).Finally, the complementary approach used information about structural characteristics to better understand the functional models of cognition.For example, one study evaluated how much additional variance is explained when structural priors are added to functional models (Kohno et al., 2017).Another group assessed if structural characteristics can be predictive of functional neural interactions observed between cognitive tasks (Chica et al., 2017).Finally, Sun et al. (2012) calculated the ratio of the effective connectivity observed during tasks with underlying direct structural white matter connections.It is important to note that some work may use several of these four general approaches.For example, partial least squares method can be used to fuse structural, functional and cognitive data, because it finds linear combination of predictors variables that covary with the response variable and projects all the information into a new space.
Thus, we see that this method can both be used as a predictive and as fusion method depending on the kinds of research questions that the authors wish to address.
Each one of these approaches has their strengths in addressing specific research questions.The comparative approach allows us to understand what properties differ across structure and function.Consequently, the interpretation of results is easier and more meaningful as it explains what makes each modality unique and why unique patterns of results may be observed.However, it is important to remember that specific features of functional connectivity may be more difficult to be compared against features of brain structure.For example, there is evidence to demonstrate that there is indirect functional connectivity where functional connections can be observed in the absence of direct structural connections.In addition, functional connectivity can be negative, where a pair of remote regions shows a negative association in their signal strength.It is currently unclear how these unique functional characteristics should be compared against structural characteristics.Investigations that undertake complementary approach may be more suited to explore how much of brain function is related to brain structure, as they explore how prior information about brain structure can impact on the relationship between brain function and cognitive performance.The limitation of such investigations is however that they reflect little information about how neural function shapes neural structure, thus they provide a one-sided view of the relationship.Next, the predictive and fusion approaches have the capacity to effectively approach the multivariate nature of this research field.For example, Robinson et al. (2021) have started from a series of univariate regression models, composed of either structural or functional information, and implemented a stacked approach to eventually develop a combined model that is multivariate.In another example, Dhamala et al. ( 2021) have produced three regression models; using only structural connectivity, using only functional connectivity and using both.The challenge of such approaches remains that they may witness suppressor variables, where the variance in the response data accounted for by one variable may impact the beta weights of another variable (Lancaster, 1999).In addition, to our knowledge, so far research has focused on construction of linear models while interactions between structure and function have remained unexplored.Research into mediation effects in regression models will be necessary to more fully explore how the relationship between brain structure and function serves cognitive function.Thus we see that every approach can be used to answer slightly different questions about the relationship between brain structure and function, and authors can be creative in how they integrate several approaches to produce very refined models of cognition.
This review also considered what data was acquired and how it was prepared for analysis.This highlighted a number of limitations.First, it became apparent that fMRI protocols have taken clear dominance over other functional imaging techniques in this research field.As mentioned in the introduction of this review, fMRI method suffers from low temporal resolution and is not a direct measure of neural activity.It is essential to dedicate more research in the future to neuroimaging data with higher temporal resolution, such as EEG.This would allow more direct study of neural signals with millisecond precision.
Consequently, signals that are not effectively reflected in the BOLD response could be studied, such as mismatch negativity which occurs 150 milliseconds following stimulus onset and is recognised as a marker of detection of stimulus irregularity (Näätänen, 1995).
Second, there was a clear dominance of experimental protocols employing cognitive task performance over resting state fMRI.Resting state has been subjected to scrutiny and debate, as mental state and mental processes of the subjects are uncontrolled (Damoiseaux et al., 2006;Poldrack & Devlin, 2007;Smith et al., 2009).In contrast, task paradigms are carefully designed to engage and manipulate a cognitive process of interest and it is more clear what mental state was evoked in participants.However, resting state paradigms show moderate to high test-retest reliability and replicability across datasets and laboratories (Biswal et al., 2010;Buckner et al., 2009;Shehzad et al., 2009;Zuo, Di Martino, et al., 2010;Zuo, Kelly, et al., 2010).This means that resting state may allow easier comparison of results across independent research laboratories.Further, resting state produces consistent activation of a specific set of regions known as default mode network (Greicius et al., 2002).
The function of this network has been related to cognitive functions, including but not limited to task switching, learning, and social cognition (McCormick & Telzer, 2018;Smith et al., 2018;Spunt et al., 2015).Further, its abnormal function has been implicated in many disorders such as dementia, schizophrenia, epilepsy, anxiety and depression and autism (Broyd et al., 2009).Third, many studies have employed correlation analysis as a method of relating brain structure and function.However, mediation analysis and partial correlation analyses were largely not employed.This is problematic, because it has been demonstrated that two regions can display functional connectivity in the absence of direct structural links between them and the similarity between functional and structural networks increases when indirect structural links are permitted in the analysis (Hagmann et al., 2008;Honey et al., 2009).This means that studies which ignore indirect links between regions may find less similarity between brain structure and function than studies that would account for those links.
To conclude, the present review was conducted to survey the prevalence of studies

Figure 1
Figure1Flow diagram obtained from PRISMA guidelines for systematic reviews(Moher et

Figure 8 .
Figure 8. Analysis of similarity (e.g.correlation, cosine) were the most common method of

Figure 8
Figure 8 Methods used to relate structural and functional data in studies that made direct integrating brain structure and function for understanding cognition and detail the methods used in these analyses.Integrating structure and function and cognition is key for a full understanding of brain function and cognitive function through lifespan, disease and recovery.This review demonstrated that the relationship between brain structure and function and cognitive function is still largely underexplored.Inferences about the relationship between neural structure and function and cognitive function were indirect, semi-direct or direct, depending on what kind of evidence was used to support the interpretation of that relationship.Direct inference was not as common as indirect inference, and we have provided a brief discussion of available and previously used approaches to handling this multivariate analysis.
Papers were selected for this review with the following process: papers were identified from databases, duplicate articles were removed, titles and abstracts were screened, and finally in-text elimination was conducted.As part of in-text elimination process, the articles without cognitive outcome were excluded.Article selection process was conducted by MCL.