Elsevier

NeuroImage

Volume 124, Part A, 1 January 2016, Pages 442-454
NeuroImage

Longitudinal reproducibility of default-mode network connectivity in healthy elderly participants: A multicentric resting-state fMRI study

https://doi.org/10.1016/j.neuroimage.2015.07.010Get rights and content

Highlights

  • We implement a multi-site 3 T MRI protocol for resting state fMRI in 13 sites.

  • We acquire across-session test–retest (TRT) data on 64 healthy elderly participants.

  • Despite harmonization strong phantom and brain tSNR differences remain across sites.

  • TRT error of regional DMN functional connectivity is consistent across sites.

  • ICA yields more reliable DMN connectivity measurements relative to SBA.

Abstract

To date, limited data are available regarding the inter-site consistency of test–retest reproducibility of functional connectivity measurements, in particular with regard to integrity of the Default Mode Network (DMN) in elderly participants. We implemented a harmonized resting-state fMRI protocol on 13 clinical scanners at 3.0 T using vendor-provided sequences. Each site scanned a group of 5 healthy elderly participants twice, at least a week apart. We evaluated inter-site differences and test–retest reproducibility of both temporal signal-to-noise ratio (tSNR) and functional connectivity measurements derived from: i) seed-based analysis (SBA) with seed in the posterior cingulate cortex (PCC), ii) group independent component analysis (ICA) separately for each site (site ICA), and iii) consortium ICA, with group ICA across the whole consortium. Despite protocol harmonization, significant and quantitatively important inter-site differences remained in the tSNR of resting-state fMRI data; these were plausibly driven by hardware and pulse sequence differences across scanners which could not be harmonized. Nevertheless, the tSNR test–retest reproducibility in the consortium was high (ICC = 0.81). The DMN was consistently extracted across all sites and analysis methods. While significant inter-site differences in connectivity scores were found, there were no differences in the associated test–retest error. Overall, ICA measurements were more reliable than PCC-SBA, with site ICA showing higher reproducibility than consortium ICA. Across the DMN nodes, the PCC yielded the most reliable measurements (≈ 4% test–retest error, ICC = 0.85), the medial frontal cortex the least reliable (≈ 12%, ICC = 0.82) and the lateral parietal cortices were in between (site ICA). Altogether these findings support usage of harmonized multisite studies of resting-state functional connectivity to characterize longitudinal effects in studies that assess disease progression and treatment response.

Introduction

Functional connectivity, i.e. resting-state activity synchronization, among the constituent nodes of the Default Mode Network (DMN) (Gusnard and Raichle, 2001, Greicius et al., 2003, Fox and Raichle, 2007, Buckner et al., 2008) is sensitive to normal aging and neuropsychiatric disease (Bassett and Bullmore, 2009, Rosazza and Minati, 2011, Anticevic et al., 2012, Damoiseaux, 2012, Castellanos et al., 2013, Pievani et al., 2014). Longitudinal assessment of DMN connectivity is therefore of interest as a potential biomarker of disease prediction/progression and treatment response (Persson et al., 2014). Despite the associated technical and logistical challenges, multicenter longitudinal studies are particularly attractive as they allow the acquisition of large datasets over diverse populations while distributing load across consortium participants (Van Horn and Toga, 2009).

The sensitivity of longitudinal studies is often limited by between-session test–retest reproducibility of the parameter(s) of interest (Atkinson et al., 2001, Castellanos et al., 2013). As recently reviewed, several factors can affect the test–retest reproducibility of DMN connectivity measurements at a single-site level, including demographics, psychophysiological state, scanner hardware, pulse sequence settings, data preprocessing and analysis methods. Nevertheless, single-site studies have indicated that the between-session test–retest reproducibility of the DMN is fair, and that DMN functional connectivity measurements may therefore deserve consideration as a functional biomarker in longitudinal studies (Zuo and Xing, 2014). However, the reproducibility from single sites using different MRI systems, different acquisition protocol details and different analysis methods cannot necessarily be extrapolated to the reproducibility that may be found in a consortium using a harmonized acquisition and analysis protocol.

In fact, until very recently, limited multisite resting-state fMRI data have been available, making it difficult to evaluate the consistency of test–retest reproducibility of DMN connectivity. This is an important shortcoming, because heterogeneous reproducibility can bias and severely limit the power of multisite longitudinal investigations. The Consortium for Reliability and Reproducibility (CoRR: http://fcon_1000.projects.nitrc.org/indi/CoRR/html/index.html) is a very recent effort which aims at addressing these limitations by creating and maintaining a public repository for resting state fMRI reproducibility data (Zuo and Xing, 2014).

Comparisons between identical 3.0 T scanners conducted on healthy participants have not revealed significant differences in temporal signal-to-noise ratio (tSNR), in the default mode and attention networks (Huang et al., 2012), nor in graph-based connectivity parameters (Braun et al., 2012). Unfortunately, such studies do not reflect the fact that multi-site investigations, almost invariably involve multiple scanner configurations (models and vendors) having heterogeneous hardware performance (number of channels, RF noise factor, gradient strength, etc.) and software settings (pulse sequence design, reconstruction and filtering parameters, etc.).

There also remains some controversy around which data analysis method is preferable to measure DMN connectivity in multisite settings. Since its inception in seminal work demonstrating intrinsic functional connectivity in the resting brain (Biswal et al., 1995), seed-based analysis (SBA) has remained a popular choice. The precuneus and posterior cingulate cortex (PCC) play a pivotal role in DMN connectivity, and as such, consideration of the blood-oxygen level-dependent signal (BOLD) average time-course from the PCC robustly characterizes the DMN at a single subject level (Andrews-Hanna et al., 2007, Buckner et al., 2008, Fransson and Marrelec, 2008). An alternative method not involving anatomical priors is independent component analysis (ICA) (Calhoun et al., 2001, Beckmann et al., 2005). While this method is arguably more robust than SBA to physiological and movement-related noise, the choice of the number of spatial components is not trivial and may entail a trade-off between avoiding splitting the DMN over multiple components and avoiding merging of unrelated networks. Diverse implementations of ICA are available and give comparable results in single-site studies conducted mostly on healthy young participants (Shehzad et al., 2009, Meindl et al., 2010, Van Dijk et al., 2010, Zuo et al., 2010, Li et al., 2012), but to our knowledge no data are available regarding the test–retest reproducibility of ICA-derived DMN measurements in multisite studies of elderly subjects.

Predicated on the above, we set out to: i) implement a harmonized international multi-site 3.0 Tesla MRI data acquisition protocol for resting-state fMRI (13 sites in 6 European countries, covering 3 common scanner vendors and 8 different scanner models), ii) acquire across-session test–retest data (at least one week apart) on healthy elderly participants (5 per site), and iii) evaluate the between-session reproducibility of tSNR and DMN functional connectivity measured using ICA and SBA. For ICA, group analysis was performed both at the single-site level (separate decomposition and back-reconstruction for each site) and at the consortium level (pooling all sites together).

Section snippets

Materials and methods

Participant demographics, study design and data preparation steps have been described in recent morphometry (Jovicich et al., 2013) and diffusion (Jovicich et al., 2014) studies from the PharmaCog project, but are repeated here following applicable updates. The test–retest raw data from this study are publicly available (https://neugrid4you.eu/).

Results

The interval between test and retest scans ranged between 7 and 69 days, with a median of 14 days. The following protocol deviations were detected: 1) Site 01 scanned one participant with incorrect TR (2.3 s) in both test and retest sessions, and one participant with different TRs between sessions (test: 2.3 s, retest: 2.7 s); 2) Site 11 scanned two participants (4 and 5) with inconsistent voxel size across (test: 3.5 × 3.5 × 4.0, retest: 3 × 3 × 3 mm3); 3) Site 13 consistently used an in-plane voxel size

Discussion

The main study findings are as follows: 1) Despite careful harmonization of the fMRI acquisition protocol, strong phantom and brain tSNR differences remain across sites; 2) regardless of inhomogeneous tSNR, the DMN is always detected, albeit with some significant differences in regional functional connectivity metrics across sites; 3) regardless of the inhomogeneous tSNR, the relative test–retest reproducibility error of regional functional connectivity metrics is rather consistent across

Conclusions

The test–retest reproducibility of DMN functional connectivity as measured by ICA and PCC-SBA is consistent across sites despite highly heterogeneous tSNR due to hardware and pulse sequence differences; furthermore, site-by-site and consortium ICA give more reliable measurements of DMN functional connectivity than SBA. These findings support consideration of resting-state fMRI as a functional biomarker in multicentric longitudinal studies.

The following are the supplementary related to this

Acknowledgments

PharmaCog is funded by the EU-FP7 for the Innovative Medicine Initiative (grant no. 115009). The authors are grateful to all members and collaborators of the PharmaCog project, and particularly to L. Venturi, G. Borsci, T. Günther and A. Monnet for their contribution in the start-up phase. LM was funded by Scienze Mente-Cervello (Rovereto, Italy).

Conflicts of interest

The authors have no conflicts of interests to declare.

References (57)

  • J. Jovicich et al.

    Multisite longitudinal reliability of tract-based spatial statistics in diffusion tensor imaging of healthy elderly subjects

    Neuroimage

    (2014)
  • X.H. Liao et al.

    Functional brain hubs and their test–retest reliability: a multiband resting-state functional MRI study

    Neuroimage

    (2013)
  • J. Persson et al.

    Longitudinal assessment of default-mode brain function in aging

    Neurobiol. Aging

    (2014)
  • C. Triantafyllou et al.

    Comparison of physiological noise at 1.5 T, 3 T and 7 T and optimization of fMRI acquisition parameters

    Neuroimage

    (2005)
  • C. Triantafyllou et al.

    Effect of spatial smoothing on physiological noise in high-resolution fMRI

    Neuroimage

    (2006)
  • C. Triantafyllou et al.

    Physiological noise and signal-to-noise ratio in fMRI with multi-channel array coils

    Neuroimage

    (2011)
  • M.P. van den Heuvel et al.

    Network hubs in the human brain

    Trends Cogn. Sci.

    (2013)
  • X. Wang et al.

    Investigating univariate temporal patterns for intrinsic connectivity networks based on complexity and low-frequency oscillation: a test–retest reliability study

    Neuroscience

    (2013)
  • X.N. Zuo et al.

    Test–retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: a systems neuroscience perspective

    Neurosci. Biobehav. Rev.

    (2014)
  • X.N. Zuo et al.

    Reliable intrinsic connectivity networks: test–retest evaluation using ICA and dual regression approach

    Neuroimage

    (2010)
  • X.N. Zuo et al.

    Toward reliable characterization of functional homogeneity in the human brain: preprocessing, scan duration, imaging resolution and computational space

    Neuroimage

    (2013)
  • A. Abou-Elseoud et al.

    The effect of model order selection in group PICA

    Hum. Brain Mapp.

    (2010)
  • A.J. Atkinson et al.

    Biomarkers and surrogate endpoints: preferred definitions and conceptual framework

    Clin. Pharmacol. Ther.

    (2001)
  • D.S. Bassett et al.

    Human brain networks in health and disease

    Curr. Opin. Neurol.

    (2009)
  • C.F. Beckmann et al.

    Investigations into resting-state connectivity using independent component analysis

    Philos. Trans. R. Soc. B

    (2005)
  • B. Biswal et al.

    Functional connectivity in the motor cortex of resting human brain using echo-planar MRI

    Magn. Reson. Med.

    (1995)
  • B.B. Biswal et al.

    Toward discovery science of human brain function

    Proc. Natl. Acad. Sci. U. S. A.

    (2010)
  • R.L. Buckner

    The brain's default network: origins and implications for the study of psychosis

    Dialogues Clin. Neurosci.

    (2013)
  • Cited by (0)

    1

    These authors contributed equally to this work.

    View full text