Using Pareidolia to Study the Impact of Semantic Processing on Brain Oscillations, Memory Encoding, and Representational Similarity in EEG

Semantic content crucially influences memory formation. However, controlling semantic content in an experimental setting is challenging as humans tend to see meaning even in abstract shapes, a phenomenon called pareidolia. We here made use of this spontaneous semantic processing by presenting random line drawings (“squiggle”). In experiment 1 participants repeatedly rated these squiggles according to their subjectively perceived meaningfulness and subsequently provided a name for each item they had considered meaningful (M+). In experiment 2 we tested the influence of meaningfulness ratings on recognition memory. Meaningfulness ratings and given names across studies were intraindividually consistent, but varied interindividually. M+ items were more likely remembered and elicited stronger decreases in alpha/beta power (~8-20Hz). Alpha/beta decreases also predicted subsequent memory of items. This relation of alpha/beta decrease and memory was mediated by subjective meaningfulness. Concurrent to alpha/beta power decreases, representational similarity analysis revealed more consistent item-specific information across repetitions for M+ items in contrast to Mitems. Additionally, between-item EEG similarity patterns correlated with semantic similarity of the names given to M+ items. These results reveal a direct link of semantic representability and episodic memory and show a potential link of alpha/beta power decreases and the representational format of semantic information.


Introduction
Decades of memory research have demonstrated the influence of semantic and conceptual processing on memory (Craik & Lockhart, 1972;Tulving, 2001). Especially decreases in alpha/beta power have so far been linked to the specific role of semantic processing in memory encoding (Fellner, Bäuml, & Hanslmayr, 2013;Hanslmayr, Staudigl, & Fellner, 2012). The specific role of these brain oscillatory activity changes in memory formation and the representation of semantic information remains unclear. Controlling semantic or conceptual processing in memory experiments is difficult, as humans tend to attribute meaning to even random patterns and non-sense items.
In the current study we used random line drawings, so called squiggles (Groh-Bordin, Zimmer, & Ecker, 2006;Voss, Federmeier, & Paller, 2012), to study how this automatic, subjectively perceived semantic processing influences EEG correlates and memory formation. In a first study EEG was recorded while participants rated repeatedly presented squiggles for their meaningfulness and subsequently provided subjective labels to meaningful items. In this first study the main interest was to test the stability and incidence of M+ ratings and the EEG correlates of this phenomenon. In a second study a recognition test after meaningfulness ratings was added to test for a relationship of M+ ratings, memory formation and alpha/beta power changes.

488
This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0 To furthermore elucidate whether EEG activity represent semantic information we how perceived meaningfulness modulates item specific representations and whether EEG patterns during M+ trials reflect semantic processing. Representational similarity analysis was used to investigate, whether semantic content is represented in the recorded EEG (Kriegeskorte, Mur, & Bandettini, 2008).

EEG Recording & Analysis
In both experiments EEG was recorded using 64 channels (500 Hz sampling rate). EEG data was analyzed using the fieldtrip toolbox and custom Matlab scripts. Raw EEG data was epoched in trials around item onset (-2 to 4 sec), artifacts were excluded using visual inspection and ICA. Data was referenced to average reference and subjected to wavelet analysis (5 cycle wavelets) to calculate time frequency resolved oscillatory power. Power data was z-transformed to the respective mean and standard deviation in each frequency band and electrode. EEG data was source localized using a LCMV beamformer.
To calculate similarity between EEG trials, Spearman correlations were calculated on downsampled and artifact corrected raw EEG trials. As event related EEG data is highly correlated across trials, data was normalized across all trials ("cocktail blank removal" i.e. removing mean activity), then cut in 200 ms windows of data vectorized across all 64 channels and correlations between these vectors were calculated for every time window and trial combination, sliding the 200 ms window with a 90% overlap.
EEG power changes and RSA results were statistically tested using a two stage cluster permutation approach (Maris & Oostenveld, 2007). In a first stage ttest were carried out for each time x frequency x electrode bin or time x time bin, respectively. In a second stage t-value across significant (p<0.05) contiguous bins were summed. A random distribution of t-sums was generated by randomly shuffling condition labels/item labels 1000 times. p corr refers to the rank of empirical clusters with regard to the distribution of surrogate clusters.

Behavioral Results
On average 45% of the squiggles (range 10-74%) in experiment 1 and 35% (range 18-56%) in experiment 2 were rated as meaningful. Meaningfulness ratings were stable across blocks as evident in the high correlations between ratings (Figure 2A) in experiment 1. The rating during the task also predicted naming of the squiggle in the subsequent naming task. ROC analyses discriminating named/unnamed squiggles based on M ratings were above chance in all participants (ROCs curves not included here). M+ items were better remembered than M-items (T(20)=10.38, p<.0001, Figure 2B). Interestingly, for the majority of participants, the hit rates for M+ items were at ceiling, indicating that participants tend to not forget squiggles that were meaningful to them.
A similar pattern of results was found in experiment 2. M+ hits were related to stronger decreases in α/β power (p corr =.0001) and increases in θ power (p corr =.01) than M-hits. Subsequent memory was related to stronger α/β decreases (p corr =.006) and increases in θ power (p corr =.001) in similar regions as in the experiment 1. The relationship of α/β power and subsequent memory was mediated by M+ ratings, showing that α/β memory effects index semantic/conceptual processing (GLM approach, mediation analysis, Sobel test T(20)=3.33, p<.005).

Item specific similarity
EEG activity across all channels was correlated across all trials using a sliding time window approach to extract item specific representations and temporal generalization matrices. Both M+ and M-items showed significant clusters of item-specific correlations (all p corrs <.05). Item specificity was significantly stronger for M+ vs M-items.

EEG similarity and semantic similarity
To estimate semantic similarity between the M+ squiggles, we utilized a web search distance approach (Cilibrasi & Vitanyi, 2007;Falco, Ison, Fried, & Quiroga, 2016). The names typically given to the squiggles are very individual descriptions (e.g. "the yellow pokemon" or "an attentive animal"), which are not part of standard semantic similarity databases. To calculate semantic similarity we thus utilized a websearch based semantic distance measure.
The semantic distance between two items (e.g. "dancing man" and "whale") was defined by the number of web-search hits for the two names combined divided by the hits of each name separately. Semantic similarity correlated significantly with the similarity of EEG activity across items (p corr = .003), showing that EEG patterns represent semantic information related to the subjective attributed meanings of the items. Note that this correlation of EEG and semantic content occurred in a very similar time window where we had observed decreases in alpha/beta power.

Conclusion and Discussion
In two experiments, we first confirmed that the squiggle stimulus set allows studying conceptual/semantic processing via subjective meaningfulness ratings. These M+ ratings were stable within subjects but vary across subjects, enabling an investigation of semantic processing independent of stimulus characteristics.
Using these subjective meaningfulness ratings, we could demonstrate that semantic processing across both studies is related to strong decreases in alpha/beta power, which has been previously connected to memory formation, semantic processing, and the entropy of cortical information in general (Fellner et al, 2013;Hanslmayr et al 2012). Alpha/beta decreases were source localized to typical "semantic" regions and predicted later memory. Additionally, a mediation analysis showed that the connection of alpha/beta decreases to memory formation was mediated by the subjective meaningfulness. This result is in line with models of episodic memory (Tulving, 2001), suggesting that memory formation necessarily involves semantic processing.
The presented analysis only provides an indirect link between alpha/beta decreases and item-specific and semantic representations: Significant clusters of itemspecific and semantic representations co-occurred in overlapping time windows as significant alpha/beta power decreases. Further analysis on this data needs to close this gap and explore the potential relationship of alpha/beta power changes and semantic representations in greater detail.