Nonparametric statistical testing of EEG- and MEG-data☆,☆☆
Introduction
The topic of this paper is the statistical analysis of ElectroEncephaloGraphic (EEG) and MagnetoEncephaloGraphic (MEG) data. These data will subsequently be denoted together as MEEG-data. MEEG-data have a spatiotemporal structure: the signal is sampled at multiple sensors and multiple time points (as determined by the sampling frequency). The data are typically collected in different experimental conditions and the experimenter wants to know if there is a difference between the data observed in these conditions. In most studies, the conditions differ with respect to the type of stimulus being presented immediately before or during the registration of the signal. In other studies, the conditions differ with respect to the type of response (e.g., correct or incorrect) that was given.
In the statistical analysis of MEEG-data we have to deal with the multiple comparisons problem (MCP). This problem originates from the fact that the effect of interest is evaluated at an extremely large number of (sensor, time)-pairs. This number is usually in the order of several thousands. The MCP involves that, due to the large number of statistical comparisons (i.e., one per (sensor, time)-pair), it is not possible to control the so-called family-wise error rate (FWER) by means of the standard statistical procedures that operate at the level of single (sensor, time)-pairs. The FWER is the probability under the hypothesis of no effect of falsely concluding that there is a difference between the experimental conditions at one or more (sensor, time)-pairs. A solution of the MCP requires a procedure that controls the FWER at some critical alpha-level (typically, 0.05 or 0.01).
In this paper, we discuss nonparametric statistical testing of MEEG-data. Contrary to the familiar parametric statistical framework, it is straightforward to solve the MCP in the nonparametric framework. Nonparametric tests were first proposed for testing the difference between MEEG-waveforms at a particular sensor (Blair and Karniski, 1993, elaborating on a parametric procedure proposed by Guthrie and Buchwald, 1991), then for MEEG-topographies at a particular time point Achim, 2001, Galán et al., 1997, Karnisky et al., 1994, and finally also for whole spatiotemporal matrices (Maris, 2004). Nonparametric tests have also been used very successfully for frequency domain representations of EEG- and MEG-data Kaiser and Lutzenberger, 2005, Kaiser et al., 2000, Kaiser et al., 2003, Kaiser et al., 2006, Lutzenberger et al., 2002. Recently, nonparametric tests were proposed for distributed inverse solutions obtained by a minimum variance beamformer Chau et al., 2004, Singh et al., 2003 or a minimum norm linear inverse (Pantazis et al., 2005). Finally, nonparametric tests for fMRI-data were proposed by Holmes et al. (1996), Bullmore et al., 1996, Bullmore et al., 1999, Nichols and Holmes (2002), Raz et al. (2003), and Hayasaka and Nichols, 2003, Hayasaka and Nichols, 2004.
The present paper contributes to the literature in several respects: (1) it explains how the sensitivity of the statistical test can be drastically improved by incorporating biophysically motivated constraints in the test statistic, (2) it is written in a tutorial-like fashion, enabling neuroscientists to construct their own statistical test, maximizing the sensitivity to the expected effect, and (3) it explains why the nonparametric test is formally correct, making use of the so-called conditioning rationale, a concept that is both rigorous and intuitive. The paper is written for two audiences: (1) empirical neuroscientists looking for the most appropriate data analysis method, and (2) methodologists interested in the theoretical concepts behind nonparametric statistical tests. With the empirical neuroscientist in mind, we have written Sections 2 Methods, 3 Results in a tutorial-like fashion, and with the methodologist in mind, we have written Section 4 that is sufficiently rigorous.
Section snippets
Methods
We make use of an example data set that was obtained in a study on the semantic processing of sentences (Jensen et al., submitted). This study involved a comparison of two experimental conditions that differed with respect to the semantic congruity of the final word in a sentence with the first part of the sentence. As is the case for most neuroscience studies, the authors are interested in the difference between experimental conditions with respect to the biological data. A central point of
Single-sensor analyses
For well-studied experimental paradigms, one often knows at which sensor the strongest effect can be observed. For instance, previous EEG-studies in which semantic congruity was manipulated (for a review, see Kutas and Federmeier, 2000) have shown the maximum effect over parietal cortex near the midline (Pz and the surrounding electrodes). And previous MEG-studies Simos et al., 1997, Helenius et al., 1998, Helenius et al., 2002, Halgren et al., 2002 have shown a dipolar pattern over left
Justification
Until now, we have deliberately ignored three important issues: (1) the exact specification of the null hypothesis that is tested by the nonparametric statistical test, (2) the proof that this test controls the FA rate, and (3) the issue of how to choose a test statistic. The theory of nonparametric statistical tests is not well documented and not very accessible. Surprisingly, the central argument in this theory (the so-called conditioning rationale, see further) is rather intuitive and can
The permutation test for multiple-subject MEEG studies
In practice, one is often interested in a null hypothesis about a population of subjects, instead of a single subject. In a very similar way as for a single-subject MEEG study, the null hypothesis about a population can be tested by means of a permutation test. To test the null hypothesis at the level of a population, a sample of subjects is drawn from this population. The first step involves taking the average2
Conclusions
We have shown how nonparametric statistical tests can be used to evaluate different effect types that are studied in the MEEG-literature (i.e., single-sensor and multi-sensor evoked responses and time–frequency representations). We have also presented a theory for these nonparametric statistical tests, which demonstrates their validity in a rigorous way. This theory applies to both single-subject and multiple-subject studies. The null hypothesis of a nonparametric statistical test involves that
References (35)
Statistical detection of between-group differences in event-related potentials
Clin Neurophysiol
(2001)- et al.
Improving permutation test power for group analysis of spatially filtered MEG data
NeuroImage
(2004) - et al.
Detecting activations in pet and FMRI: levels of inference and power
NeuroImage
(1996) - et al.
Testing topographic differences between event-related brain potentials by using nonparametric combinations of permutation tests
Electroencephalogr Clin Neurophysiol
(1997) - et al.
N400-like magnetoencephalography responses modulated by semantic context, word frequency, and lexical class in sentences
NeuroImage
(2002) - et al.
Validating cluster size inference: random field and permutation methods
NeuroImage
(2003) - et al.
Combining voxel intensity and cluster extent with permutation test framework
NeuroImage
(2004) - et al.
Gamma-band activity over early sensory areas predicts detection of changes in audiovisual speech stimuli
NeuroImage
(2006) - et al.
Statistical probability mapping reveals high-frequency magnetoencephalographic activity in supplementary motor area during self-paced finger movements
Neurosci Lett
(2000) - et al.
Dynamics of gamma-band activity in human magnetoencephalogram during auditory pattern working memory
NeuroImage
(2003)
Electrophysiology reveals semantic memory use in language comprehension
Trends Cognit Sci
Nonparametric statistical testing of coherence differences
J Neurosci Methods
A comparison of random field theory and permutation methods for the statistical analysis of MEG data
NeuroImage
Statistical tests for fMRI based on experimental randomization
NeuroImage
Source localization of the N400 response in a sentence-reading paradigm using evoked magnetic fields and magnetic resonance imaging
Brain Res
Group imaging of taskrelated changes in cortical synchronization using nonparametric permutation testing
NeuroImage
An alternative method for significance testing of waveform difference potentials
Psychophysiology
Cited by (5444)
Prime saliency in semantic priming with 18-month-olds
2024, CognitionImpulsivity and neural correlates of response inhibition in bipolar disorder and their unaffected relatives: A MEG study
2024, Journal of Affective Disorders
- ☆
The methods described in this paper have been implemented in the Matlab toolbox Fieldtrip, which is available from http://www.ru.nl/fcdonders/fieldtrip.
- ☆☆
The authors would like to thank Ole Jensen for generously sharing his data.