Whole-brain functional correlates of memory formation in mesial temporal lobe epilepsy

Highlights: • Large study of encoding and subsequent memory for words, faces, and scenes.• Less ipsilateral mesial temporal activity in mesial temporal lobe epilepsy (mTLE).• Extra-mTL activity in mTLE only partly relevant for memory formation.• Across materials contralateral mTL decisive to maintain intact memory in mTLE.• Left frontal activation correlates with better verbal memory only in left mTLE.


Introduction
In patients with temporal lobe epilepsy (TLE), memory impairments are frequent because the epileptogenic lesion and epileptic activity often disrupt brain structures involved in memory formation. (Hermann et al., 1997;Jokeit and Schacher, 2004) Some studies still support the notion that impaired verbal memory is most prominent in patients with their epileptic focus in the temporal lobe (TL) of the language-dominant hemisphere, (Hermann et al., 1987;Jeyaraj et al., 2013;Mungas et al., 2009;Willment and Golby, 2013) Knowledge regarding cerebral correlates of memory in TLE patients is crucial with regard to planning surgical interventions and avoiding potential adverse consequences. Moreover, data on memory formation in TLE can inform theories on cerebral correlates of memory and memory plasticity in general. Therefore, memory functions in TLE have been investigated using functional magnetic resonance imaging (fMRI) paradigms. Several studies have focused on encoding-related activity in the mesial temporal lobe (mTL). The majority of those have revealed that mTL activation is more strongly lateralised to the hemisphere contralateral to the epileptogenic focus in both rTLE and lTLE patients regardless of the encoded material (see e.g. Detre et al., 1998;Towgood et al., 2015). This has been reported to be due to reduced activation in the mTL ipsilateral to the epileptogenic focus, (Avila et al., 2006;Cheung et al., 2006;Jokeit et al., 2001;Powell et al., 2007;Sidhu et al., 2013) − but also increased activation in the mTL contralateral to the epileptogenic focus. (Figueiredo et al., 2008;Powell et al., 2007) Thus, studies indicate hypoactivation in the epileptogenic mTL, reorganisation to the contralateral side, or both. The functional relevance of increased contralateral activation is controversial: Some studies support its functionality, (Milian et al., 2015;Richardson et al., 2003;Figueiredo et al., 2008) but others suggest it is inefficient. (Powell et al., 2007;Bigras et al., 2013) In addition, several studies indicate a functional role of the ipsilateral posterior hippocampus to maintain intact, especially postoperative, memory performance. (Sidhu et al., 2013(Sidhu et al., , 2015Bonelli et al., 2010Bonelli et al., , 2013 Besides mTL structures, a network including lateral temporal, frontal, and parietal structures is involved in memory (for review, see Simons and Spiers, 2003). So far, few studies have examined, via wholebrain analysis, whether or to what extent TLE induces distributed changes in encoding-related activation. Extant data indicate more widespread activation in lateral temporal, frontal, and parietal regions during verbal, (Alessio et al., 2013;Dupont et al., 2000Dupont et al., , 2002Maccotta et al., 2007;Sidhu et al., 2013) and non-verbal memory encoding, (Alessio et al., 2013;Limotai et al., 2018;Maccotta et al., 2007;Sidhu et al., 2013) in both lTLE and rTLE patients compared with controls. Overall, additional recruitment of lateral temporal regions appears most consistent. (Sidhu et al., 2013;Alessio et al., 2013) An increase in frontal activation varies both among studies and within individual studies across different encoding materials. Additional activation in frontal areas contralateral to the epileptogenic focus has been inconsistently reported during verbal and non-verbal memory encoding in either lTLE patients, rTLE patients, or both. (Sidhu et al., 2013;Alessio et al., 2013;Maccotta et al., 2007;Limotai et al., 2018) Some inconsistencies might be due to the fact that existing studies had mostly rather small sample sizes and partly lacked statistical comparison with a control group, thus potentially over-interpreting descriptive differences. Furthermore, it remains unclear if reported changes in activation are specific to memory formation or reflect other unrelated cognitive processes, because most extant whole-brain studies only focused on encoding, but did not calculate subsequent memory effects. (Maccotta et al., 2007) The subsequent memory effect is a more specific measure of memory formation. It contrasts fMRI activation during encoding of subsequently remembered items with subsequently forgotten ones. Whereas the more general encoding activation represents a mixture of perceptual, attentional, and memory networks, subsequent memory analyses show activation specific to successful memory formation. Within the mTL, to our knowledge, only three studies have examined this effect in TLE patients. (Richardson et al., 2003;Powell et al., 2007;Bonelli et al., 2010) They have shown divergent effects for words and faces and consistently no difference for line drawings between a control group and lTLE or rTLE patients (for details, see Supplementary Table 1, columns 1 − 3).
Two more recent studies have extended subsequent memory analyses in TLE to the whole brain (see Supplementary Table 1, columns 4 − 5). Sidhu et al. (Sidhu et al., 2013) investigated the encoding of faces and words in 44 mesial temporal lobe epilepsy (mTLE) patients with hippocampal sclerosis and a control group. In controls, the verbal subsequent memory effect was restricted to left temporal and frontal regions, but lTLE patients exhibited right temporal activation. Non-verbal subsequent memory activated right temporal and frontal regions in controls, whereas rTLE patients showed right temporal and predominantly left extra-temporal activation. Regarding extra-temporal regions, subsequent memory analysis and correlations of activation and neuropsychological memory measures underline the critical involvement of the orbitofrontal cortex in both controls and mTLE patients in memory formation, whilst the insula and the anterior cingulum have been found to play a role only in mTLE patients. (Sidhu et al., 2013) Taken together, the findings by Sidhu et al. (Sidhu et al., 2013) are in line with the notion of epilepsy-induced functional reorganisation, but the authors did not report direct between-groups comparisons. Hill et al. (Hill et al., 2020) focused on verbal subsequent memory in a heterogeneous sample of five left, seven right and four bilateral TLE patients, none of whom had hippocampal sclerosis. When comparing across groups − without a statistically significant group difference − they identified the verbal subsequent memory effect in the left inferior frontal gyrus, but not in the mTL. Further, unlike what has been commonly assumed, they found no impact of the epileptogenic hemisphere (Hill et al., 2020). However, it remains unclear to what extent the lack of significant differences might be due to the specific material used or the heterogeneous and small sample.
In summary, data regarding the general encoding network predominantly indicate mTL lateralisation contralateral to the epileptogenic focus in TLE patients. Moreover, TLE patients may have increased lateral temporal activation as well as activation changes in frontal regions, but their functional relevance is unclear. Evidence is scarce for the more specific subsequent memory effect. Only a handful of studies investigated subsequent memory effects in temporal lobe epilepsy with varying results. More comprehensive knowledge about potential changes in the brain's memory encoding system is needed. This comprehension is especially important for TLE patients who are candidates for surgical treatment. For those patients, we not only need to identify the functionality of the to-be-resected mTL area, but also understand potential compensatory mechanisms across the whole brain to better predict postsurgical memory sparing or decline.
To address these issues, we have applied a memory fMRI task of learning words, faces, and scenes. Several studies, (Avila et al., 2006;Bigras et al., 2013;Cheung et al., 2006;Detre et al., 1998;Golby et al., 2002;Limotai et al., 2018;Mechanic-Hamilton et al., 2009) have explored encoding-related activity for complex scenes in TLE, but so far none has investigated specifically the subsequent memory effect for this material. We have also optimised scanning parameters by means of coronal slices aligned with the long axis of the hippocampus to investigate mesio-temporal and extra-temporal activation in one study. This endeavour is challenging: Many studies focusing on the whole brain did not detect significant hippocampal activation. (Alessio et al., 2013;Dupont et al., 2000;Maccotta et al., 2007;Hill et al., 2020) To further engage the hippocampus via emotional modulation, taking advantage of the reciprocal connection of hippocampus and amygdala, (Strange et al., 2014) we have included a mixture of both emotional and neutral stimuli. We have performed statistical comparison of encoding-and subsequent memory-related activation between groups and further correlated the fMRI activation with memory performance in mTLE patients, including independently obtained clinical neuropsychology performance measures, in order to determine the functionality of activation also with more ecologically valid measures. Thereby, we aimed to extend knowledge on the brain basis of memory encoding and, more specifically, subsequent memory formation in mTLE patients.

Participants
We studied 56 patients with mTLE (25 right sided), who were consecutively recruited from the pre-surgical evaluation ward at Mara Hospital, Bielefeld, Germany, and 21 healthy controls (HC). Four patients had incomplete datasets: One right mesial temporal lobe epilepsy (rmTLE) patient had a seizure during the last run (completing only words and scenes) and thus the recognition task could not be performed after scanning. For one left mesial temporal lobe epilepsy (lmTLE) patient, the fMRI data of scenes and faces could not be analysed due to technical issues during the recordings. Further, two lmTLE patients discontinued the fMRI task after the second run (one completed words and faces, the other one completed words and scenes), but both still performed the recognition task.
Patients' epileptic onset zone was ascertained via video-EEG monitoring and structural MRI at 3 T to be unilateral and in the mTL. All patients spoke fluent German and received antiepileptic drugs (AEDs). The individual AED load was estimated as the sum of the ratios of the prescribed daily dose by the estimated average dose prescribed ("defined daily dose") as reported by the World Health Organization (see https://www.who.int/tools/atc-ddd-toolkit/about-ddd) for each prescribed AED. (Deckers et al., 1997) Requirements for control participation were ≥18 years old and absence of known neurological and psychiatric disorders. A detailed description of all participants' demographic and patients' clinical characteristics is given in Table 1.
All participants had normal or corrected-to-normal vision. They gave written informed consent according to the Declaration of Helsinki. The University of Bielefeld's and the Westfalen-Lippe medical association ethics committees approved the study.

Neuropsychological testing
All participants but one healthy control underwent neuropsychological testing including verbal and visual memory assessment. For verbal memory, the German adaption of the Rey Auditory Verbal Learning and Memory Test (Schmidt, 1996) (Verbaler Lern-und Merkfähigkeitstest (Helmstaedter et al., 2001, VLMT) was used and visual memory was assessed via the Diagnosticum for Cerebralschädigung II (Weidlich et al., 2011) (DCS), which were part of the standard diagnostic procedures during patients' stay at the pre-surgical evaluation ward. In the VLMT participants listen to five repetitions of the same list of 15 unrelated words, which have to be freely recalled each time. In the DCS participants are similarly presented 9 geometrical figures for 10 s each and then asked to reproduce the memorised figures with wooden sticks. There are again five trials. The DCS was available for all but two lmTLE patients. For both tests the standardised, agecorrected z-scores of the sum of recalled items across five trials were calculated according to the norms of the respective test.

Memory-encoding paradigm
The encoding paradigm comprised three stimulus conditions, namely scenes, faces, and words. Scenes were coloured complex scenes mostly taken from the International Affective Picture System (IAPS) set, faces were front-view colour photographs of adult faces, and words were single nouns (see Supplementary Table 3 for a list of all stimuli used). Each stimulus condition included 36 negative and 36 neutral randomly selected stimuli. In three randomised consecutive runs, the stimuli of each condition were presented for 3 s each in alternating blocks of four neutral or four negative stimuli, followed by a 12-second baseline condition. Participants were instructed to encode and memorise the stimuli for the out-of-scanner recognition task. In the baseline condition, participants were asked to maintain fixation at a randomly moving dot and to avoid thinking about the preceding stimuli.
The recognition task outside of the scanner immediately followed the fMRI task, about 10 − 15 min after encoding. The three stimulus conditions were presented in the same order as in the fMRI task. In each condition, all 72 previously displayed stimuli and an additional 48 new stimuli, which served as foils, were presented. Participants were asked to press the right arrow key, labelled with "yes", for stimuli shown in the MRI and the left arrow key, labelled with "no", for new stimuli.

Magnetic resonance data acquisition
MRI data were collected on a 3 T Siemens Verio MRI scanner. Highresolution T1-weighted structural data were acquired using a 32-channel head coil, with 192 sagittal slices, a slice thickness of 0.8 mm, a field of view of 15.36×24×24 cm, 0.75×0.75 mm in-plane resolution, echo time of 2.5 ms, and repetition time of 1.9 s. For fMRI, gradient-echo planar images were acquired using a 12-channel head coil, with 37 coronal slices aligned with the long axis of the hippocampus, a slice thickness of 4 mm, a field of view of 19.2×14.8×19.2 cm, 2.4×2.4 mm in-plane resolution, echo time of 33 ms, and repetition time of 3 s. Three additional dummy scans were included to ensure a steady state of magnetisation.

Anatomical and functional data pre-processing
MRI data were pre-processed using fMRIPrep 1.4.1rc4, (Esteban et al., 2019) which is based on Nipype 1.2.0. (Gorgolewski et al., 2011)  A deformation field to correct for susceptibility distortions was estimated based on fMRIPrep's fieldmap-less approach. The deformation field results from co-registering the fMRI reference to the sameparticipant T1-weighted reference with its intensity inverted. (Wang et al., 2017;Huntenburg, 2014) fMRI images were slice-time corrected using 3dTshift from Analysis of Functional NeuroImages (AFNI) 20160207 (Cox and Hyde, 1997) and resampled into MNI152N-Lin2009cAsym standard space. Resampling was performed using ant-sApplyTransforms configured with a one-step Lanczos interpolation by collapsing all the pertinent transformations (i.e. head-motion transform matrices, susceptibility distortion correction, slice-time correction, and co-registrations to MNI space) to minimise the smoothing effects of other kernels. (Lanczos, 1964) 2.6. Statistical analysis 2.6.1. Behavioural data We compared lmTLE and rmTLE patients and controls regarding the percentage of hits and false alarms, recognition accuracy, and response bias separately for each condition using Python 3.6.8 and the SciPy package. (Virtanen et al., 2020) We calculated recognition accuracy (hits − false alarms) and response bias (false alarms/[100 − recognition accuracy]) according to the two-high threshold model. (Snodgrass and Corwin, 1988) Because the data were partly not normally distributed (Shapiro-Wilk p < 0.05), we used the Mann-Whitney U test to compare groups. We compared behavioural data with two-tailed tests and a critical p value of 0.05.

fMRI data
We conducted the first level analyses using Python 3.6.8 and the module Nistats 0.0.1b1 (https://nistats.github.io). The imaging time series were smoothed with a Gaussian kernel of 8 mm full-width at halfmaximum and high-pass filtered with a cut-off at 0.01 Hz. We included the six movement parameters, frame-wise displacement, and the first six components of the anatomical-component-based noise correction (Behzadi et al., 2007) as confounding factors. For the second level analyses, we used the FMRIB Software Library (FSL) 6.0 randomise 2.9 non-parametric permutation (perm.) testing with 10,000 permutations. (Winkler et al., 2014) We added age at evaluation as a nuisance variable for the second level analysis because age slightly differed between groups. We corrected all results for multiple comparisons using threshold-free cluster enhancement (TFCE). (Smith and Nichols, 2009) The significance level was set to p < 0.05 family-wise error rate (FWE) corrected. We additionally display t-values significant at p (FWE) < 0.1 to give a more comprehensive picture of the activation pattern. We used this threshold, instead of an uncorrected p value, in group comparison and correlation analyses to ensure a certain level of sensitivity while still being aware of the expected FWE rate. (Bennett et al., 2009;Roiser et al., 2016) We conducted whole-brain and small volume correction (SVC) analyses for a region of interest (ROI) encompassing the hippocampus and the amygdala as defined in the Harvard-Oxford Atlas because of the often low signal-to-noise ratio in this region. (Powell et al., 2005;Strange et al., 2002;Olman et al., 2009) 2.6.2.1. Blocked analyses: Encoding scenes, faces, and words. The regressors of interest were the three stimulus categories − scenes, faces, and words, each convolved with the canonical double gamma haemodynamic response function. We created first level contrast images for these three conditions each contrasted with the respective baseline condition. Then, on the second level, we used one-sample t-tests to examine statistically significant group-invariant main effects of encoding elicited by each stimulus modality and two sample t-tests to compare mTLE patients with controls within each modality. To further investigate individual lateralisation of memory function in a combined mask with the hippocampus and amygdala during each stimulus condition, we calculated lateralisation-indices as follows: LI = [(L − R)/(L + R)], where L and R indicate the number of voxels with z >1.96 in the left and right mTL.
Additionally, we aimed to explore functionality of neural reorganisation in lmTLE and rmTLE patients by calculating correlations between the fMRI encoding activation of each condition with both, the respective recognition accuracy in the experiment, and either verbal or design learning scores as determined via clinical neuropsychology assessment.
2.6.2.2. Event-related analyses: Subsequent memory effects of scenes, faces and words. Like previous studies, (Richardson et al., 2003;Powell et al., 2007;Bonelli et al., 2010) we also analysed the blocked fMRI task with an event-related approach. In these analyses, our regressors of interest were subsequently remembered items and subsequently forgotten ones, separately for each condition, which were contrasted for the first level analyses. We took these first level contrasts of remembered vs. forgotten items to the second level, first for a one-sample t-test collapsed across groups to assess regions relevant for successful memory formation in each modality and then for two-sample t-tests to evaluate potential reorganisation of regions relevant for successful memory formation in mTLE patients compared with controls.
Due to the wide range of recognition performances across participants, we had to limit the subsequent memory contrast to subgroups (which varied for the stimulus modalities), similarly to previous studies. (Hill et al., 2020;Clark and Wagner, 2003;Park et al., 2013) This guards against spurious estimates due to floor and ceiling effects or guessing, because some participants showed excellent or very poor performance, thus having only a very small number of either subsequently forgotten or remembered items. Again, others showed a poor recognition accuracy suggestive of guessing. All cases result in a very uncertain estimate of the subsequent memory contrast. To deal with this factor and to still achieve a valid subsequent memory contrast, we only included participants with at least 10% (eight items) of subsequently remembered and forgotten items and a recognition accuracy of at least 10%. These criteria resulted for scenes in samples of 12 controls, 25 lmTLE, and 23 rmTLE patients; for faces in samples of 18 controls, 23 lmTLE, and 17 rmTLE patients; and for words in samples of 18 controls, 27 lmTLE, and 22 rmTLE patients.

Data availability
All unthresholded statistical fMRI maps are available on NeuroVault (https://identifiers.org/neurovault.collection:8993). Further data from this study are available from the corresponding author upon reasonable request.

Behavioural data
Hits, false alarms, recognition accuracy, and response bias are detailed in Supplementary Fig. 1

Blocked fMRI analysis: Encoding scenes, faces, and words
As shown in Fig. 1, across groups the blocked fMRI analyses of encoding scenes, faces, and words revealed for each of the three conditions widespread bilateral anterior and posterior mesio-temporal, lateral temporal, frontal, basal ganglia, parietal, and occipital activation. In detail, frontal regions included parts of the frontal and central opercular cortex, the anterior cingulate, paracingulate, precentral and inferior, middle and superior frontal gyrus, the insula, the frontal orbital cortex, and the frontal pole.

mTLE patients versus controls
As shown in the top row of Fig. 2A and detailed in Table 2, there were significant group differences for encoding scenes between lmTLE and rmTLE patients and controls. Both patient groups displayed reduced activation in the anterior hippocampus and the amygdala of the epileptogenic hemisphere. Further, rmTLE patients had reduced right lateral occipital activation. lmTLE patients exhibited increased activation bilaterally in predominantly anterior temporo-lateral regions and in left basal ganglia, insula, and frontal regions including the frontal pole, the frontal orbital cortex and the precentral, superior frontal and middle frontal gyrus. At a threshold of p (FWE) ≤ 0.1 activation was bilateral in the frontal pole and basal ganglia and additionally encompassed the left inferior frontal gyrus. rmTLE patients demonstrated increased activation in the left basal ganglia, postcentral gyrus, superior parietal lobule, middle frontal gyrus and frontal pole, in right anterior temporo-lateral regions, and bilateral insula. These were more widespread and additionally encompassed the left frontal orbital cortex, superior and inferior frontal gyrus and anterior temporo-lateral and temporo-occpital region at a threshold of p (FWE) ≤ 0.1. lmTLE and rmTLE patients also exhibited clusters of more activation in regions, where the overall contrast showed less activation during encoding than during baseline, amounting to less deactivation in patients. These were mostly in regions associated with the default mode and auditory network including the anterior and posterior cingulate gyrus, the precuneus, the angular gyrus, the frontal pole, the posterior superior temporal gyrus, the planum polare and temporale and the Heschl's gyrus.
For face encoding, lmTLE and rmTLE patients had less activity in the epileptogenic mTL encompassing the anterior hippocampus and amygdala. In addition, lmTLE patients exhibited reduced activation in the right occipital fusiform, lateral occipital and lingual gyrus. These effects are visualised in Fig. 2A (middle row) and detailed in Table 3.
The bottom row of Fig. 2A shows that for word encoding, rmTLE patients exhibited increased activation in the left anterior cingulate and paracingulate gyrus. lmTLE patients had increased activation in the right lateral occipital cortex at a threshold of p (FWE) ≤ 0.1. Further, lmTLE patients showed significantly decreased deactivation in similar regions as during scene encoding, namely the posterior cingulate and posterior middle temporal gyrus, and the angular and precuneus cortex. At a lower threshold of p (FWE) ≤ 0.1 this held also for rmTLE patients having decreased deactivation in the posterior cingulate gyrus and precuneus cortex. The extent to which individual regions were differently activated is further described in Table 3.
Lateralisation-indices for mTL activity are depicted in Fig. 2C. In HCs, they indicate slight right lateralisation for scenes and faces, and strong left lateralisation for words. In lmTLE patients, the distribution of individual lateralisation is shifted towards the right mTL during scene encoding. Likewise, there is a shift to the right for most lmTLE patients during face encoding, although there are several outliers. For word encoding, most lmTLE patients show right mTLE lateralisation, but the range is wide and several maintain left lateralisation. By contrast, the distribution of rmTLE patients' lateralisation is shifted relatively homogeneously towards the left mTL during scene, face, and word encoding. Fig. 3 illustrates the subsequent memory effect for the different stimulus classes. Across groups, stronger activation for subsequently remembered compared with subsequently forgotten scenes was apparent in bilateral anterior and posterior mesio-temporal, lateral temporal, basal ganglia, parietal, and occipital regions as well as in left inferior, orbital and precentral frontal regions, the insula, and the frontal pole. Subsequently remembered faces exhibited only bilateral anterior mesio-temporal, lateral temporal and occipital activation. For words, the subsequent memory effect was seen in bilateral anterior mesiotemporal and almost exclusively in left-sided lateral temporal and occipital regions as well as widespread in left frontal regions including the inferior, middle, orbital, and precentral frontal gyrus, and the frontal operculum and pole.

fMRI event-related analyses: Subsequent memory effect of scenes, faces and words
There was a negative subsequent memory effect with increased activation for subsequently forgotten compared with remembered items particularly in regions associated with the default mode network. It was widespread for scenes and more circumscribed for faces and words (see Fig. 3).

mTLE patients versus controls
Only in the subsequent memory effect of scenes were there marginal group differences (p (FWE) ≤ 0.1). lmTLE patients exhibited in a small cluster weaker left amygdala and anterior hippocampus activation for subsequently remembered scenes. rmTLE patients showed increased activation in the right middle frontal gyrus and frontal pole (see Fig. 4A). Note, however, that overall this region was associated with the negative subsequent memory effect (see Fig. 3), implying increased activation for subsequently forgotten scenes. Fig. 4B illustrates the distribution in the individual groups of participants.

Correlation between encoding activation and memory performance in mTLE patients
We further correlated encoding-related fMRI activation in lmTLE and rmTLE patients with the post-scanner recognition accuracy and verbal and design learning scores to identify functional relevance of activation (see Fig. 5 and Table 4 and 5). Given that we had to limit the subsequent memory analysis to a subgroup of patients, we decided to correlate memory performance with the general encoding-related activation (learning vs. baseline).
For scene encoding and lmTLE patients, increased activation in the right posterior hippocampus, parahippocampal gyrus, fusiform cortex, posterior cingulate gyrus, precuneus, lingual gyrus and supracalcarine cortex were associated with better scene recognition accuracy. There was a similar pattern in the left hemisphere at a lower threshold of p (FWE) ≤ 0.1. Comparably increased activation in the right amygdala, the anterior and posterior hippocampus, the parahippocampal gyrus and bilateral pallidum was associated with better design learning scores. Again, this held in tendency also for the left posterior hippocampus, and the right fusiform cortex and frontal pole (p (FWE) ≤ 0.1). In rmTLE patients there was an association of increased left anterior hippocampus activation and better scene recognition accuracy.
For face encoding and lmTLE patients, increased activation in the right amygdala and anterior hippocampus were associated with both better face recognition accuracy and design learning scores. At a threshold of p (FWE, SVC) ≤ 0.1 a more circumscribed activation cluster in the left amygdala and anterior hippocampus also correlated with better face recognition accuracy. Further, there were negative correlations between encoding activation and face recognition accuracy in the preand postcentral gyrus, the supramarginal, angular and posterior cingulate gyrus, the precuneus and lateral occipital cortex. rmTLE patients showed, only at a lower threshold of p (FWE, SVC) ≤ 0.1, similar negative correlations between encoding activation and design learning scores in the supramarginal, angular and posterior middle temporal gyrus.
For word encoding and lmTLE patients, increased activation in the Abbreviations: fMRI, functional magnetic resonance imaging; FWE, family-wise error rate; HC, healthy controls; lmTLE, left mesial temporal lobe epilepsy; rmTLE, right mesial temporal lobe epilepsy.

Impact of clinical characteristics on memory performance in mTLE patients
To identify potential clinical features which impact memory performance, we calculated spearman correlations of the recognition accuracy of each condition, and verbal and design learning scores with epilepsy onset, epilepsy duration, and AED load. In lmTLE patients higher drug load was associated with lower memory performances (scene recognition accuracy r = − 0.50, p = 0.005, design learning r = − 0.34, p = 0.07). Further, memory performance negatively correlated with epilepsy duration (scene recognition accuracy r = − 0.41, p = 0.03, design learning r = − 0.32, p = 0.10) and also somewhat with epilepsy onset (word recognition accuracy r = − 0.32, p = 0.09, verbal learning r = − 0.30, p = 0.11). In rmTLE patients higher drug load was also associated with lower memory performance (design learning r = − 0.39, p = 0.05). All other comparison revealed ps > 0.15.

Discussion
We studied whole-brain neural correlates of memory encoding and subsequent memory in mTLE patients and controls. The current fMRI paradigm of encoding scenes, faces, and words proved well-suited to measure encoding as well as subsequent memory-related activation within and outside of the mTL. Scenes elicited especially strong and bilateral, if slightly right dominant, encoding-related haemodynamic activation and large subsequent memory effects, providing a sound basis for detecting disease-related alterations. Analysis of encoding-related haemodynamic activity revealed decreased activation in the epileptogenic mTL and extra-temporal increases in mTLE patients. However, subsequent memory analyses provided only marginal systematic group differences specifically related to memory formation. Correlations of Fig. 3. Across-group fMRI activation of the subsequent memory effect. Across-group fMRI activation of the positive and negative subsequent memory effect for scenes, faces, and words. The colour code indicates t-values significant at p (perm.) ≤ 0.05 FWE-corrected with small volume correction in the mTL. Activation is displayed in MNI-152 space. Abbreviations: fMRI, functional magnetic resonance imaging; FWE, family-wise error rate. encoding-related activation and memory performance revealed a predominant role of the mTL and, solely for words and lmTLE patients, of left frontal regions. Especially correlations between activation in the contralateral mTL and behaviour may reflect disease-induced reorganisation within the mTL system.

Behavioural results
Compared with controls, both rmTLE and lmTLE patients had reduced recognition accuracies for scenes and words, while only rmTLE patients showed decreased face recognition accuracy. Direct comparison of lmTLE and rmTLE patients revealed a tendency for worse face recognition accuracy in rmTLE patients.
The finding of reduced word recognition irrespective of the side of mTLE does not support theories of a predominant role of the left mTL in verbal memory. (Jeyaraj et al., 2013;Willment and Golby, 2013) Results are instead in line with Sailing (Saling, 2009), who casts doubt on strictly dichotomous material-specific lateralisation in mTLE. This is further supported by a number of previous imaging, (Dupont et al., 2002;Maccotta et al., 2007;Sidhu et al., 2013) and behavioural, (Chang et al., 2019;Postma et al., 2020;Reyes et al., 2019;Sidhu et al., 2015b) studies reporting impaired verbal memory in both lTLE and rTLE patients. Further, our task may have implicitly elicited semantic clustering during encoding, which has been found to be equally impaired in lmTLE and rmTLE patients, (Grewe et al., 2020) potentially contributing to rmTLE patients' verbal memory impairment. Conversely, increased activation in the right mTL might reflect reliance on imaginability and dual-coding strategies during word encoding. (Maguire et al., 2003) Pictorial memory might be reduced in both patient groups because it relies on memory for spatial detail, theoretically more strongly linked to the right hippocampus, and memory for self-generated verbal labels that may depend on the left hippocampus. Due to their structural similarity, faces may be hard to verbalise, causing the more pronounced facememory impairment in rmTLE.

Neural correlates of encoding
Across-group encoding activation (compared with baseline) was reflected in widespread mesial and lateral temporal, frontal, basal ganglia, and occipital regions and most pronounced for scenes. Deactivation occurred mostly in regions typically associated with the default mode and auditory network. Lateralisation-indices show that mTL activation in HC was slightly right-lateralised for scenes and faces, and strongly left-lateralised for words. In mTLE patients, lateralisation was, on average, shifted towards the mTL contralateral to the epileptogenic focus, a finding that is consistent with previous studies. (e.g. Detre et al., 1998;Towgood et al., 2015) This general pattern was further reflected in the strongly decreased activation in the contralateral mTL during non-verbal encoding. This might be expected due to the pathology and broadly replicates earlier findings. (Powell et al., 2007;Sidhu et al., 2013;Avila et al., 2006;Cheung et al., 2006;Jokeit et al., 2001) During verbal encoding, the signal change in the mTL was much lower in all groups, probably yielding insufficient power to detect group differences except in lateralisation-indices. lmTLE patients further demonstrated reduced activation for faces in the right occipital fusiform and lingual gyrus. Fig. 4. Group comparisons of fMRI activation of the subsequent memory contrast. A Group differences in the subsequent memory effect of scenes when comparing lmTLE (left) and rmTLE (right) patients with controls. The colour code indicates t-values significant at p (perm.) ≤ 0.1 FWE-corrected with small volume correction in the mTL. The green outline indicates subsequent memory activation for controls. Activation is shown in MNI-152 space. There was no significant between group difference in the subsequent memory effect of faces and words. B Bar plots with overlaid swarm plots, showing mean percent signal change of subsequent memory activation of controls and lmTLE and rmTLE patients for clusters significant in the group comparisons. Clusters were defined using p (FWE) ≤ 0.1 with small volume correction in the mTL. Whiskers indicate 95% confidence intervals. Abbreviations: fMRI, functional magnetic resonance imaging; FWE, family-wise error rate; HC, healthy controls; lmTLE, left mesial temporal lobe epilepsy; rmTLE, right mesial temporal lobe epilepsy. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) rmTLE patients had less activation for scenes and in tendency also for faces in right lateral occipital regions. These findings agree with theories assuming functional coupling between medial temporal and visual processing regions during stimulus encoding (Ranganath et al., 2005) and demonstrate pathology-specific activation deficits in this system.
Most consistently across conditions and most prominent during scene encoding, mTLE patients exhibited less deactivation than controls in brain regions known to be part of the default mode network, (Raichle, 2015) such as the posterior cingulate gyrus and the precuneus. Further, patients had less deactivation in regions of the auditory network, which has also been established during the resting state. (Raichle, 2015) This region again revealed stronger baseline than task activation in all groups. Sidhu et al. (Sidhu et al., 2013) reported activation changes for mTLE patients in a comparable region. Hill et al. (Hill et al., 2020) and Stevens et al. (Stevens et al., 2008) found activation in this region to be negatively correlated with memory performance and proposed that failure to suppress scanner noise hinders memory formation, a phenomenon that would fit well with our data. We further detected increased activation in predominantly anterior temporo-lateral regions, the insula, the basal ganglia, anterior cingulate gyrus and in the left frontal gyrus, frontal orbital cortex, and frontal pole during encoding of either scenes or words in mTLE patients, thus reflecting widespread extra-mTL recruitment as previously suggested. (Sidhu et al., 2013;Alessio et al., 2013;Dupont et al., 2000Dupont et al., , 2002Maccotta et al., 2007;Limotai et al., 2018) Altogether, these findings suggest that mTLE patients exhibit primarily reduced activation during encoding in lesioned brain regions, which impedes memory formation and may increase subjective task difficulty. Consequently, mTLE patients might either deactivate less areas associated with the baseline condition during the encoding task or, conversely, suppress areas associated with the encoding task less during the baseline condition in order to rehearse more to compensate memory impairments. The increased lateral temporal and frontal activation seen during encoding could reflect more effort, reliance on the semantic system, or strategic control of encoding, (Saling, 2009;Simons and Spiers, 2003;Eichenbaum, 2017) although group-level data suggest it is insufficient to consistently compensate for mTL damage.

Neural correlates of subsequent memory
A critical question we addressed was to what extent patients reorganise to compensate for their pathology by using different brain regions compared with controls for subsequent memory formation. Our analysis revealed no region with increased subsequent memory activation in mTLE patients. In detail, the subsequent memory analysis only revealed that compared with controls, lmTLE patients exhibited in tendency lower left amygdala and anterior hippocampus activation during subsequent memory formation of scenes. Given that lmTLE patients, unlike controls, showed no memory benefit for emotional scenes (negative hit − neutral hit: lmTLE patients: Mdn = 2.8, Min = -19.4, Max = 25.0; controls: Mdn = 15.3, Min = -8.3, Max = 33.3; U = 95, p = 0.08, d = 0.61), the reduced anterior mTL activation might also be a correlate of a memory benefit for negative scenes present in controls but not lmTLE patients. This would appear to be in line with reduced emotional memory in lTLE (Múnera et al., 2015) and the notion that co-activation of an intact amygdala may enhance memory formation. (Strange et al., 2014) rmTLE patients instead only showed increased activation in the right middle frontal gyrus and frontal pole, regions overall associated with the negative subsequent memory effect and thus associated with activation for subsequently forgotten scenes. Overall, at least at the group level, the subsequent memory analysis revealed little evidence of temporal or extra-temporal functional reorganisation in mTLE patients. This could be due to highly individual reorganisation patterns which are therefore not significant at group level testing. Moreover, the subsequent memory contrast might be less sensitive despite being more specific than encoding alone. (Sidhu et al., 2015) The finding that regions in which mTLE patients exhibited more activation than controls during encoding compared with baseline were partly involved with subsequent memory formation, at least at more lenient thresholds, might be indicative thereof.
Given that mTLE patients demonstrated typical impairments in memory performance, it seems plausible that memory formation in most of these patients still largely relies on the same brain regions as in controls. On the other hand, some extra mTL regions with increased activation in the encoding vs. baseline contrast in mTLE patients compared with controls also showed up in the subsequent memory analysis, indicating some extra-temporal functional recruitment, as previously suggested by Sidhu et al. (Sidhu et al., 2013) Two other previous studies instead reported stronger contralateral mTL activation in TLE patients than in controls for subsequently remembered stimuli. (Richardson et al., 2003;Powell et al., 2007) However, unlike our study or the ones by Bonelli et al. (Bonelli et al., 2010) and Hill et al. (Hill et al., 2020), the TLE patients in those studies had preserved memory function. Powell et al. (Powell et al., 2007) further reported an inverse correlation between memory performance and degree of contralateral activation for faces and words, which might also be due to the small group (n = 7) of the on average unimpaired patients. Accordingly, Sidhu et al. (Sidhu et al., 2013) demonstrated for a larger sample of mTLE patients a positive correlation between memory performance and subsequent memory activation in the mTL contralateral to seizure focus for words and faces. Thus, contralateral mTL reorganisation of subsequent memory may be restricted to only some TLE patients, potentially those with intact memory performance.

Correlation between encoding activation and memory performance in mTLE patients
The correlation analysis of encoding activation and memory performance demonstrated an association between increased contralateral mTL activation and better memory performance. This finding was consistent across stimulus categories and held both for performance Table 5 Correlations of face and word encoding fMRI activation with memory performance in lmTLE and rmTLE patients. Anatomical labels are based on the Harvard-Oxford atlas. within the experimental paradigm and when correlating with clinical tests, adding ecological validity. As was to be expected, the association was stronger for within paradigm correlations. It was also more prominent in lmTLE patients. Additionally, but only in smaller clusters, there was a tendency for increased ipsilateral mTL activation to correlate with better memory performance in lmTLE patients. mTL correlations were more posterior and extended to temporo-lateral and parietal regions for scenes and located anterior for faces and words, irrespective of the hemisphere. Further, increased word encoding activation in left frontal regions (inferior and middle frontal and precentral gyrus, frontal orbital cortex and frontal pole) correlated with better verbal memory in lmTLE patients. Clinically, better memory performance correlated with lower AED load and in lmTLE patients also with earlier epilepsy onset, and shorter epilepsy duration, suggesting that these clinical factors may affect mTL plasticity and contribute to the extent to which ipsi-or contralateral activation subserves memory formation. Albeit age significantly correlated with epilepsy onset and duration in lmTLE patients we would not assume the correlations with memory performance to be a pure age effect. This is because uniquely in lmTLE patients higher age was broadly associated with lower memory performance, whereas no impact of age on memory performance was found in rmTLE patients or controls. Thus, we would speculate the correlation of epilepsy onset and duration with memory performance to be due to epilepsy-induced increased effects of aging, which might only be evident in lmTLE patients due to the longer epilepsy duration in this group (lmTLE patients M = 20.2; rmTLE patients M = 14.0). This might also contribute to the finding of stronger correlations in lmTLE patients, suggesting that in patients with longer epilepsy duration functional recruitment of especially the contralateral mTL is more relevant to maintain intact memory. However, given that onset, duration and age are all intercorrelated in this dataset, we cannot determine their individual contribution, although it is plausible that these factors have at least partly unique roles. Further, it should also be noted that the lmTLE group was somewhat larger, yielding more power to detect effects.
Overall, these correlations are in line with previous studies indicating reorganisation to the contralateral mTL being functional in both lTLE, (Richardson et al., 2003;Figueiredo et al., 2008) and rTLE patients (Milian et al., 2015). They further reveal functional ipsilateral mTL activation in some lmTLE patients, which were either anterior or posterior depending on stimulus material. Hence, other than previous studies underlining the role of the posterior ipsilateral hippocampus, (Sidhu et al., 2015a;Sidhu et al., 2013), we found evidence for both the anterior and posterior ipsilateral hippocampus to support memory performance. Further, data support a role of left frontal regions for verbal memory performance, (Sidhu et al., 2015a;Sidhu et al., 2013), at least in lmTLE-patients. The pattern as a whole may reflect reorganisation, compensatory strategies, resilience, or their combination.

Conclusion
In summary, we have extended previous studies by investigating general encoding-related activation as well as subsequent memory activation for scenes, faces, and words in a relatively large and homogenous group of mTLE patients and well-matched HCs, simultaneously assessing both mTL and extra-mTL regions. Encoding-related activity was decreased in epileptogenic mTL and increased in extratemporal regions, which were partly associated with subsequent memory formation but did not correlate with better memory performance. Further, we found decreased deactivation in regions, which have previously been associated with the default mode and auditory network. The findings suggest that successful memory formation in mTLE patients does not systematically rely on different brain regions compared with controls, although we cannot rule out differences on the individual level. The correlations of encoding activation and memory performance in mTLE patients hint towards reorganisation to the contralateral mTL to maintain intact memory and the importance of left frontal regions particularly for verbal memory performance and lmTLE patients.
Although we investigated a relatively large and homogenous group of mTLE patients, an even larger sample would allow us to further specify the individual and joint role of clinical, demographic, and cognitive variables on fMRI activation and functional outcome. This would facilitate more detailed understanding of distinct reorganisation patterns in subgroups. So far, data underline mTL activation as a bottleneck for memory formation with functional reorganisation occurring primarily within the mTL system and further hint towards increased activation in extra-temporal regions in mTLE patients, whose functionality is not entirely clear.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.