Abstract
The question whether nonconscious processing could involve higher-level, semantic representations is of broad interest. Here, we demonstrate semantic processing of task-relevant and task-irrelevant features of nonconscious primes within a novel, empirical test bed. In two experiments, musicians were visually primed with musical note triads varying in mode (i.e., major vs minor) and position (i.e., the arrangement of notes within a triad). The task required to discriminate only the mode in the following auditory target chord. In two experimental blocks, primes were either consciously visible or masked, respectively. Response times for auditory discrimination of the modes (relevant dimension) of heard triads were measured. Crucially, the targets also varied with respect to mode and position, creating different grades of congruency with the visual primes. Based on the Theory of Event Coding, we expected and found interactions between relevant and irrelevant semantic characteristics of masked primes, illustrating that even irrelevant prime meaning was processed. Moreover, our results indicated that both task-relevant and task-irrelevant prime characteristics are processed in nonconscious conditions only, and that practice in ignoring uninformative conscious primes can be transferred to a subsequent block. In conclusion, this study demonstrates cross-modal, automatic semantic processing using a novel approach to study such effects.
1 Introduction
1.1 Masked Congruency Effects
Masked congruency effects have long been used to investigate and demonstrate the processing of nonconscious stimuli (i.e., stimuli of which humans are unaware) by their influence on decisions about supraliminal stimuli of which humans are aware (e.g., Dehaene et al., 1998). Visual masking is a method to suppress participants’ awareness of a visual stimulus (here, a prime) by presenting a masking stimulus consisting of similar but uninformative visual material shortly after (i.e., backward masking) or before (i.e., forward masking) the stimulus. Despite rendering masked stimuli invisible, this procedure allows the same stimuli to be processed (Marcel, 1983). (However, it remains debated to which degree processing of such masked stimuli is possible, cf. Kouider & Dehaene, 2007). Typically, in each trial, a masked prime is presented and participants decide whether a subsequent visible target (e.g., a word) belongs to one or another category (Greenwald et al., 1996). Even with masked and, thus, nonconscious primes, congruency effects can be found: Responses are faster in congruent conditions (prime and target belonging to the same category) than in incongruent conditions (prime and target belonging to different categories). This congruency or priming effect is found despite participants’ inability to discriminate primes with above-chance accuracy (e.g., Klotz & Neumann, 1999). Researchers often assume that such priming effects are mediated by higher-level, semantic processing (e.g., Dehaene et al., 1998; Greenwald et al., 1996; Kiefer, 2001; Marcel, 1983) but task dependencies of found congruency effects suggested non-semantic interpretations (e.g., Damian, 2001; Klinger et al., 2000; Kunde et al., 2003; for reviews see Ansorge et al., 2014; Kouider & Dehaene, 2007; Van den Bussche et al., 2009a).
A non-semantic explanation of congruency effects, as proposed by Klinger et al. (2000), assumes direct prime(-stimulus)–response associations as the main causal factor, without semantic processing (of prime–target relations). This kind of response priming might require at least some practice with the particular response-associated stimuli to allow nonconscious response priming (Damian, 2001; Klapp, 2015; see also Kouider & Dupoux, 2004; Ortells et al., 2016). Another example is action-triggering theory: It assumes that a nonconscious prime that is sufficiently similar to the targets can trigger an action that has been prepared for the targets (Kunde et al., 2003). Moreover, the dependence of the congruency effect on task relevance supports the view that response priming could explain nonconscious processing: Masked primes’ congruency effects are usually present for (task-)relevant prime–target relations but absent for (task-)irrelevant prime–target relations. For example, Klinger et al. (2000) presented participants with nonconsciously perceived nonwords or words of differing affect (i.e., positive vs negative) as primes and consciously perceived nonwords or words of differing affect as targets while the task required judging only the lexical status (i.e., word vs nonword) of the target word. They found that, in contrast to task-relevant prime–target relations (i.e., relations regarding the lexical status), task-irrelevant prime–target relations (i.e., relations regarding the affect) did not influence target judgments. This means that primes or prime characteristics (e.g., their semantic category membership) that are not also used as targets or are otherwise not task-relevant (e.g., correspond to target characteristics that are irrelevant for the required target discrimination) do not elicit congruency effects.
In contrast to the classical response priming account, we argue that in reaction-time experiments, the congruency effect might not be always the most sensitive measure to reveal nonconscious processing of irrelevant primes or prime features. Instead, we advocate the simultaneous usage of two prime–target relations – a relevant prime–target relation and an irrelevant prime–target relation – for a more exhaustive measure of processing of subliminal stimuli. This enables us and future researchers to investigate if the two congruency relations interact in masked priming, therefore laying proof of subliminal processing of irrelevant prime–target relations even in cases where traditional congruency effects of these irrelevant characteristics of masked primes cannot be found.
1.2 Nonconscious Cross-Modal Priming
Previous studies of semantic priming have shown that masked primes are also being processed, where, crucially, primes and targets were semantically related but lacked a resemblance in physical characteristics. For example, van den Bussche et al. (2009b) used masked (nonconsciously processed) pictures as primes but words as targets, where semantically related primes facilitated target discrimination. Indeed, nonconscious priming even crosses modalities where the meanings of prime and target in different modalities are related to one another (e.g., Ansorge et al., 2016; Ching et al., 2019; Dehaene & Naccache, 2001; Diependaele et al., 2005; Kouider & Dehaene, 2009; McCauley et al., 1980; Mudrik et al., 2014; Tranquada-Torres et al., 2022).
1.3 The Theory of Event Coding (TEC)
Our predictions are derived from TEC (Hommel, 2004). According to TEC, different sensory features of a stimulus (e.g., target colors, target locations) and motor features associated with the stimulus (e.g., response locations), whose neural representations are distributed over the brain, are integrated within one joint event file or episodic representation. The distribution of neural feature representations but also, for example, the simultaneous exertion of different tasks necessitates feature integration of features belonging to the same event as a unity (Hommel, 1998). Thus, when participants repeatedly respond to stimuli and perform associated actions on them, they create event files for different stimuli or stimulus-associated features and their associated responses. Importantly, these event representations need to be updated over time, as both stimuli and responses can change across time even if they are all part of the same task. This updating process is easiest if all features change or if all features repeat from one event file to the next (e.g., target color, target location, and response location). However, if only some of the features change and other features repeat (e.g., target color and response location change, but target location repeats), the activation of the previously associated feature(s) and the recombination of the repeated feature(s) with the novel changing feature(s) in an updated event file give rise to an un- and rebinding cost which manifests in longer response times (RTs) and/or enhances the error rate (ER, Hommel et al., 2001; Hommel, 2004).
TEC has been formulated in terms of simple sensory and motor features – however, some level of conceptual or semantic abstraction is implied. Previous research has found that the repetition of a distractor alongside with a target from one priming trial to a following probe trial facilitated retrieval of an event-file representation in the probe trial. This occurs not only if, across trials, the distractors were semantically and modality-congruent (e.g., a picture of a frog as a distractor in the prime trial and in the probe trial), but also when the distractors were only semantically congruent but modality- and, thus, sensory-incongruent (e.g., a picture of a frog in the prime trial but the sound of a frog in the probe trial, Frings et al., 2013; see also Horner & Henson, 2011; Wesslein & Frings, 2020).
Examples for different visual primes used in the present study. (a) Major triad in root position. (b) Minor triad in root position. (c) Major triad in first inversion. (d) Minor triad in first inversion.
The present study goes beyond previous research by probing the semantic nature of subliminal priming in the domain of music. Notably, the neural substrates of processing syntax and semantics in music are relatively similar to those involved in language processing (Atherton et al., 2018; Koelsch et al., 2004; Koelsch, 2005): for instance, Broca’s area and the homotope area in the right hemisphere, as well as posterior temporal regions are involved in both syntactic language and syntactic music processing, with language being more pronounced in the left hemisphere and music being more pronounced in the right hemisphere. The similarity of language and music processing is also reflected in event-related brain potential data: structural irregularities in both language and music elicit a P600 component (Patel et al., 1998), and semantic irregularities in both domains elicit an N400 component (Koelsch et al., 2004). While some studies investigated supraliminal priming effects in music (e.g., Sollberger et al., 2003; Steinbeis & Koelsch, 2011), only a single study so far investigated nonconscious semantic cross-modal priming using musical stimuli (Tranquada-Torres et al., 2022). These authors investigated whether the perception of a musical illusion consisting of ambiguous tones is influenced by nonconsciously seen musical notes. Results showed that the nonconscious primes shifted the illusion perception in the congruent direction. This indicates that nonconscious cross-modal priming effects occur with respect to the task-relevant features in the domain of music, too (see, e.g., Ansorge et al., 2016, for similar conclusions regarding the spatial features, or Diependaele et al., 2005, for language).
In the present study, we specifically focus on interactions between task-relevant and task-irrelevant event-file characteristics to test if processing of masked primes as one event interacts with processing of supraliminal targets as a following event. According to TEC, interactions between two prime–target relations might be found even where main congruency effects of nonconscious primes cannot be demonstrated: processing of repetitions and switches of both relevant and irrelevant dimensions from prime to target (i.e., double-congruent and double-incongruent conditions) could be more efficient than processing of switches of only one and repetitions of the other dimension. This is because partial switches could incur an un- and rebinding cost on one prime-activated semantic dimension when progressing from prime to target. Thus, employing all possible combinations between prime–target congruency allows for a more exhaustive test of subliminal processing of task-irrelevant prime characteristics. Specifically, it enables us to test priming effects beyond effects depending on task-relevant dimensions only.
1.4 The Present Study
Here, we employ a case of cross-modal masked priming to study if evidence for processing of irrelevant masked prime characteristics can be found in an interaction between relevant and irrelevant prime–target relations in the domain of music. Visual music notes were used as masked primes and clearly perceivable auditory chords as targets. To explore whether task-irrelevant prime meaning is processed and interacts with the task-relevant prime meaning as predicted by TEC, we used (1) a sufficiently large target set size, so that not each target (and prime) was repeatedly seen and anticipated (cf. Kiesel et al., 2006) and (2) any overlap of physical features between primes and targets was prevented (cf. Naccache & Dehaene, 2001). The latter was ensured by the fact that all primes were visual and all targets auditory (cf. Ansorge et al., 2016). Finally, (3) direct associations between prime meanings and specific responses (cf. Kiefer, 2001) were only allowed for the response-relevant prime dimension, but they were prevented for the response- and task-irrelevant prime dimension. To that end, masked visual primes and unmasked auditory targets varied on two dimensions: their mode (i.e., major or minor) and their position (i.e., root position or first inversion; see Figure 1 for examples). For a detailed explanation of “mode” and “position” in music terminology, see Section 2.1.2. Participants categorized the auditory targets with respect to their mode. This was the task-relevant stimulus dimension, specifically associated with the required mode-discrimination responses.
1.5 Hypotheses
We expected facilitated processing (i.e., faster RTs) where both relevant (mode) and irrelevant (position) prime–target relations were congruent or where both were incongruent, while un- and rebinding costs should slow responding in single-congruent conditions (i.e., relevant-congruent/irrelevant-incongruent or irrelevant-congruent/relevant-incongruent) (Figure 2).
Participants are presented with visual primes and auditory targets, where prime–target congruency can vary on two dimensions (mode and position). Processing of congruency in all or none dimensions may be more efficient than single switches (partial congruency). Green arrows correspond to congruency and red arrows to incongruency.
In addition, congruency effects based on the task-relevant dimension of the masked primes would be possible (cf. Klinger et al., 2000; Kunde et al., 2003). However, the interaction predicted by TEC could possibly be found where a main effect of congruency is missing.
Critically, the used primes were not predictive of the targets: congruent and incongruent conditions were equally likely, both regarding the relevant and regarding the irrelevant dimension. Therefore, the interaction might well be restricted to masked (nonconscious) conditions: previous research has repeatedly shown that participants are able to suppress uninformative primes of which they are conscious (Kinoshita et al., 2011; for the particular case of intermodal priming, see Experiment 3 of Ansorge et al., 2016). Therefore, we also included a control condition with supraliminal (unmasked, conscious) primes to learn more about the influence of prime consciousness on partial unbinding costs in uninformative semantic priming.
2 Experiment 1
2.1 Method
2.1.1 Participants
We tested nine trained German musicians without absolute pitch who currently studied music at a conservatory (one female, one left-handed, M age = 22.6, age range: 19–27 years). Eight participants studied keyboard instruments (i.e., organ, piano, and/or harpsichord), one studied voice. In order to be admitted to their courses, all music students had to pass an entrance exam which included auditory chord discrimination. Furthermore, all participants attended classes in music theory and hearing from the beginning of their studies onwards. Therefore, we assumed that chord-reading and auditory-chord discrimination abilities were sufficiently large for all participants. The mean duration of their University music education was seven semesters, ranging from 0 (just begun) to 12 semesters. Note that all participants had thorough musical training even before being admitted to their studies. All participants had normal or corrected-to-normal vision. Participants were paid 10 €/0.5 h. Informed consent was obtained, and the experiment was conducted in accordance with the Declaration of Helsinki.
2.1.2 Basic Terminology: Mode and Position in Music Theory
In music terminology, a basic chord (i.e., a triad) consists of three notes: the root, the third, and the fifth. The mode refers to the interval between the root and the third of the chord which can consist either of four (i.e., major) or three (i.e., minor) semitone steps. Regardless of the pitch level of the root, the mode of a chord can be acoustically distinguished by trained musicians as being major or minor. Independently, the position of a chord can also be acoustically distinguished. It refers to the inversion of the notes within the chord as the root note is not necessarily the lowest note of the chord. Here, root position (i.e., the root is the lowest note) and first inversion (i.e., the third is the lowest note) are of importance (Figure 1).
2.1.3 Apparatus and Stimuli
One participant at a time was tested in a dimly lit room. The visual stimuli were presented on a 14-in. LCD monitor with a vertical refresh rate of 60 Hz and a resolution of 1,366 × 768 pixels. Auditory stimuli were presented via Sennheiser HD215 headphones. Participants could regulate the volume themselves. Responses were given on a conventional QWERTZ-keyboard. Stimulus presentation and response collection were managed by OpenSesame (Mathôt et al., 2012). The set of stimuli contained visual primes and auditory targets at varying pitches. Primes were triads (i.e., three notes written vertically) either in major or minor mode and in root position or first inversion. To ensure a comparable perceptibility across different primes, only triads not exceeding one ledger line above and one below the stave and holding a maximum of one accidental (flat or sharp) were used as primes. This led to an item pool of 34 prime stimuli. As displayed in Figure 3, the masking stimuli consisted of notes and accidentals depicted in a matrix. Within one trial, the same masking stimulus was used as forward and backward mask. Altogether, a pool of 50 different masks was used in the experiment.
Schematic of a masked (left) and an unmasked (right) trial. Before and after the appearance of the prime, forward and backward masks were shown in the masked trial. The arrows represent time. The reference pitch and the target stimulus are depicted by a speaker symbol.
As catch trials, we presented an upper-case letter instead of the prime in approximately 11% of all trials, where participants had to respond by pressing the space bar to indicate that they had seen it. This ensured that participants paid attention to the visual displays at all. Visual stimuli were presented in black against a light grey background. The fixation cross was presented at screen center, whereas prime and mask were shifted slightly to the left at approximately 3.1 × 6.75° of visual angle to ensure fixation on the relevant part of the stimulus – the chord – and not on the space between the time signature and the chord. The auditory targets were always triads, too (i.e., three notes played simultaneously). To ensure good audibility, the targets ranged from 210 to 1,090 Hz. As participants did not have absolute pitch, triads were built upon quarter tones as well. This enlarged the target set size to 170 items in total and minimized the hearing of chord connections between targets, reducing unwanted effects on mode perception. Targets were generated using the open-source program MuseScore (Version 3.0.5.5992; MuseScore, 2019). The piano sound of the program was used for representation.
2.1.4 Procedure
Each participant first completed 32 masked practice trials. Feedback was given for all responses. The experimental block consisted of 200 masked trials with a break of self-determined length after 100 trials. Feedback was given only for incorrect responses. If RT exceeded 1,500 ms, participants were encouraged to respond faster. All masked experimental trials consisted of a fixation cross accompanied by a reference pitch (440 Hz), a forward mask, a visual prime, a backward mask, and an auditory target (Figure 3). In each trial (except fixation control trials), the task was to discriminate the auditory target with regard to its mode (major vs minor). The prime/target combinations were pseudo-randomized and equally likely to be congruent or incongruent with respect to both dimensions (mode and position). This means they could be congruent regarding i) just the mode, ii) just the position, iii) both, or iv) neither dimension. Participants were instructed to discriminate the mode of the auditory targets and used both their index fingers to press keys Y and M for major and minor (stimulus-response mapping was pseudo randomized across subjects). Participants were further instructed to respond with the space bar if they saw a letter embedded in the masks (catch trial), independent of the mode of the target stimulus. All participants were instructed to respond as fast and accurately as possible.
After the experimental block, a control block with 200 unmasked trials was presented: Participants received the same set of stimuli except that blanks were shown instead of the masks. Under these conditions, primes were clearly visible. Instructions for participants did not change in this block, and they again were offered a break after 100 trials.
Finally, an objective visibility test was conducted to investigate the possibility of prime identification for masked as well as unmasked conditions, with 50 trials each. Trial sequences and stimuli remained unchanged in the objective visibility test, except that participants were instructed to indicate via keypress (again, Y vs M) whether prime and target were of the same or different mode. One experimental session lasted 30 min in total.
2.2 Results
2.2.1 Masked and Unmasked Blocks
All participants entered data analysis, as no participant had a mean RT outside the range of 100–1,250 ms or a mean ER of over 30% in the experiment. For mean correct RTs, we calculated a three-way analysis of variance (ANOVA) for repeated measures, using the within-participant variables mode congruency (i.e., task-relevant; congruent vs incongruent), position congruency (i.e., task-irrelevant; congruent vs incongruent), and prime visibility (masked vs unmasked). Incorrect responses (3.8%) were excluded from analysis. Out of all correct responses, 3.9% were below 100 ms or above 1,250 ms and therefore excluded. Catch trials were excluded and not analyzed as this was irrelevant to our purposes.
Figure 4 provides a summary of the results. Results revealed a significant three-way interaction between mode congruency, position congruency, and prime visibility, F(1, 8) = 13.47, p = 0.006,
Plot of mean correct RTs in masked and unmasked conditions depending on prime–target congruency in mode (task-relevant, x-axis) and position (task-irrelevant, color) of visual and auditory chords. Block order was masked–unmasked (Experiment 1). 95% CIs are depicted.
To investigate the origin of the three-way interaction, planned paired-sample t-tests were conducted: Mode-congruent conditions were compared to mode-incongruent conditions, and position-congruent conditions were compared to position-incongruent conditions, separately for masked and unmasked conditions.
In masked conditions, as expected under TEC (Hommel et al., 2001), we found that RTs in double-congruent conditions (M = 743 ms, SD = 91 ms) were reliably shorter than in single-incongruent conditions, regardless of whether the mode was congruent and the position was incongruent (M = 766 ms, SD = 101 ms; t[8] = 3.16, p = 0.007, d = 1.05) or whether the mode was incongruent and the position was congruent (M = 766 ms, SD = 100 ms; t[8] = 2.33, p = 0.024, d = 0.78). Likewise, RTs in double-incongruent conditions (M = 741 ms, SD = 86 ms) were shorter than in single-congruent conditions, again regardless of whether the mode was congruent and the position was incongruent, t(8) = −2.14, p = 0.033, d = −0.71, or whether the mode was incongruent and the position was congruent, t(8) = −3.09, p = 0.008, d = −1.03. No difference was found between double-congruent and double-incongruent conditions, t(8) = −0.31, p = 0.765, d = 0.15.
In unmasked conditions, no evidence of prime processing was found. If anything, the RTs corresponded to the action-triggering hypothesis, with numerically shorter RTs when both mode and position were congruent (M = 721 ms, SD = 99 ms) relative to conditions where only the position was congruent (M = 728 ms, SD = 98 ms; t(8) = 0.70, p = 0.251, d = 0.23); and longer RTs if both mode and position were incongruent (M = 731 ms, SD = 83 ms), relative to conditions in which the mode, but not the position was congruent (M = 717 ms, SD = 100 ms; t(8) = 1.06, p = 0.160, d = 0.40). RTs of double-congruent conditions were numerically shorter relative to double-incongruent conditions, t(8) = 1.06, p = 0.320, d = 0.35. No differences were found for comparisons of double-congruent conditions to mode-congruent/position-incongruent conditions, t(8) = 0.51, p = 0.625, d = 0.17, and of double-incongruent conditions to mode-incongruent/position-congruent conditions, t(8) = 0.29, p = 0.776, d = 0.10.
We ran analogous analyses on ERs and arc-sine transformed ERs. However, no main effect or interaction was significant for ERs (all Fs < 2.96, all ps > 0.123) or for arc-sine transformed ERs (all Fs < 2.97, all ps > 0.123).
2.2.2 Prime Visibility
To test whether participants failed to discriminate the masked primes but identified the unmasked primes, we computed d′, an index of prime visibility (Reingold & Merikle, 1988). Individual d′ was computed separately for masked and unmasked primes. Congruent trials (with respect to the mode) counted as signals and incongruent trials as noise. Accordingly, correct judgments in mode-congruent trials counted as hits and incorrect judgments in mode-incongruent trials as false alarms. One zero value in the data, resulting from an outcome of exclusively correct judgments by one participant in the unmasked condition, was replaced by a probability of 0.01 for the z-transformation. Participants were unable to discriminate the masked primes with better-than-chance accuracy (d′ = −0.13; SD d′ = 0.42), t(8) = −0.91, p = 0.390. The unmasked primes were discriminated successfully (d′ = 2.24; SD d′ = 0.77), t(8) = 8.70, p < 0.001.
2.3 Discussion
Results indicated that both task-relevant (i.e., mode) and task-irrelevant (i.e., position) prime dimensions contributed to nonconscious processing in an interactive way. In the masked condition only, double-congruent and double-incongruent conditions facilitated performance relative to relevant and irrelevant single-congruent conditions, in line with TEC (Hommel et al., 2001; cf. Hommel, 2004). According to TEC, both sensory and motor features contribute to a joint representation as a single event. Therefore, in a condition that is congruent in one dimension but incongruent in the other, un- and rebinding of only one feature would have created a partial repetition cost in the current study. Thus, results indicated a deeper semantic processing of the masked primes than only with respect to the task-relevant (instructed) category (cf. Ansorge et al., 2014): though participants were instructed to process the mode only, they apparently also processed the position of the primes.
Importantly, no such interaction between task-relevant and -irrelevant prime characteristics was observed in control conditions with visible (unmasked) primes (Figure 4). Thus, first, we can conclude that the interaction seems to rely on the participants’ unawareness of the primes. Second, the lack of effects in unmasked conditions leads to the assumption that participants might have actively suppressed prime processing as the primes were not predictive for the task. We conducted Experiment 2 in order to investigate this assumption.
3 Experiment 2
As stated in Experiment 1, interactions of task-relevant and task-irrelevant prime dimensions were found for masked, but not for unmasked conditions. Based on the findings of Ansorge et al. (2016) and Kinoshita et al. (2011), we believe that in unmasked trials, in which the primes were perceived consciously, participants actively suppressed the primes as they were not informing about the targets. To confirm this hypothesis, we conducted a second experiment, in which we reversed the block order: In the first block, participants were presented with unmasked primes, followed by the second block in which primes were masked. Previous studies indicate that intentions of suppressing conscious visual information can transfer to nonconscious information (cf. Experiment 5 in Ansorge, 2004; van Gaal et al., 2009). As participants registered that the prime was uninformative in the unmasked conditions, it is possible that participants established a task set of ignoring or suppressing the primes. Participants might (inadvertently) transfer this task set to a following masked/subliminal priming block. Therefore, if the unmasked nonpredictive primes in the first block led participants to suppress their processing, we might obtain an inhibition of processing not only of the unmasked primes in the first block but also of the masked primes in the second block. Thus, by reversing the block order, we might be able to eliminate the interaction effect of task-relevant and task-irrelevant prime dimensions shown in Experiment 1.
3.1 Methods
3.1.1 Participants
We tested nine trained Austrian musicians who either currently studied music at a conservatory or were employed as professional musicians (five females, all right-handed, M age = 27.8, age range: 23–42 years). As nine participants were sufficient to achieve a power of 96.73% for the three-way interaction in Experiment 1, in our Experiment 2, we should be able to detect a similarly sized three-way interaction as in Experiment 1. Three participants studied voice, two studied piano, one studied clarinet, one studied cornet, one studied musicology, and one studied composing. Again, chord-reading and auditory chord discrimination abilities were assumed as being sufficient, as all musicians had to pass an entrance exam, participated in music theory and hearing courses throughout their studies, and even had profound musical training before being admitted to their studies. The mean duration of their University music education was 10 semesters, ranging from 6 to 12 semesters. Normal or corrected-to-normal vision and the absence of absolute pitch were ensured. Participants were paid 10 €/0.5 h. Informed consent was obtained, and the experiment was conducted in accordance with the Declaration of Helsinki.
3.1.2 Apparatus and Stimuli
We employed the same experimental setup as in Experiment 1. The only difference was that this time, visual stimuli were presented on a 13-in. LCD monitor with a resolution of 1,920 × 1,080 pixels (as the former computer was no longer in function).
3.1.3 Procedure
The experimental procedure in Experiment 2 was identical to the one in Experiment 1, except that this time, the order of experimental blocks was reversed such that the block with unmasked stimuli was presented before the one with masked stimuli.
3.2 Results
3.2.1 Masked and Unmasked Blocks
One participant was excluded due to an ER of 30.5% in the experiment. Data analysis was identical to the one we performed in Experiment 1, such that we calculated a three-way repeated measures ANOVA on correct RTs, using the within-participant variables mode congruency (i.e., task-relevant; congruent vs incongruent), position congruency (i.e., task-irrelevant; congruent vs incongruent), and prime visibility (masked vs unmasked). Incorrect responses (6.7%) were excluded. Out of all correct responses, 4.4% were below 100 ms or above 1,250 ms and therefore excluded.
The plotted results are shown in Figure 5. In contrast to Experiment 1, results of Experiment 2 neither revealed a three-way interaction between mode congruency, position congruency, and prime visibility, F(1, 7) = 0.01, p = 0.936,
Plot of mean correct RTs in masked and unmasked conditions depending on prime–target congruency in mode (task-relevant, x-axis) and position (task-irrelevant, color) of visual and auditory chords. Block order was unmasked–masked (Experiment 2). 95% CIs are depicted.
Similarly, no significant results were observed in the analogous analysis of ERs (all Fs < 4.26, all ps > 0.078). The same holds true for the analysis of arc-sine transformed ERs.
3.2.2 Prime Visibility
We again computed d′ to investigate whether participants were able versus not able to discriminate masked versus unmasked primes. Hits and false alarm assignments were identical to those in Experiment 1. Participants were unable to discriminate the masked primes with better-than-chance accuracy (d′ = −0.05; SD d′ = 0.34), t(7) = −0.38, p = 0.712, whereas the unmasked primes were discriminated successfully(d′ = 0.97; SD d′ = 1.04), t(7) = 2.64, p = 0.034.
3.3 Discussion
We hypothesized that participants presented with nonpredictive visible primes not only ignore these primes in unmasked/conscious conditions but also transfer this intention to masked/unconscious conditions of the same task. Results of Experiment 2 supported this hypothesis. No facilitation or cost effects comparable to those in Experiment 1 could be observed in Experiment 2, leading to the conclusion that participants did not semantically process the primes in the current experiment.
4 General Discussion
In the present cross-modal priming study, we investigated whether musicians process masked visual music primes and whether the processing of primes is semantic and “automatic” (i.e., concerning task-irrelevant prime features). To do so, we conceptualized a novel test-bed derived from TEC (Hommel et al., 2001; cf. Hommel, 2004). This novel test studies interactions between task-relevant and task-irrelevant prime–target relations. Importantly, prime and target were processed by different modalities. Primes were visual and targets were auditory musical triads, respectively, both varying in mode and position. Targets had to be discriminated with regard to their mode only. Thus, the mode was the relevant task dimension and the position was the irrelevant task dimension. We conducted two experiments in which we tested two different block orders of masked and unmasked stimuli: in the first experiment we presented the masked primes before the unmasked primes, and in the second experiment we presented the unmasked before the masked primes.
In the first experiment, we observed an interactive contribution of both task-relevant (i.e., mode) and task-irrelevant (i.e., position) prime dimensions to nonconscious processing of masked stimuli. In line with TEC, conditions in which both dimensions were congruent as well as conditions in which both dimensions were incongruent facilitated performance relative to conditions in which only one dimension was congruent and the other was incongruent. Such an interaction across different sensory modalities of visual prime and auditory target might appear surprising. However, results from previous studies support the existence of these interactions (Frings et al., 2013; Wesslein et al., 2014). The current evidence for the processing of both the task-relevant and -irrelevant prime meanings is in line with this type of conceptual or semantic abstraction. In addition, known dependencies of semantic processing on sensory representations might play a role (cf. Barsalou, 1999). For example, much in the same way that visual word meaning reactivates basic sensory and motor experience (Glenberg & Kaschak, 2002), the meaning of visual musical priming notes might directly facilitate or interfere with auditory processing of heard chords (cf. Leman & Maes, 2015; Leman, 2007).
Interestingly, in Experiment 1, interactions of relevant and irrelevant congruency were found for masked conditions but not for unmasked conditions. Together with the visibility judgments, these results confirmed a qualitative difference between conscious and nonconscious processing (cf. Dixon, 1971) of the type reported by Kinoshita et al. (2011), who attributed lower prime processing in unmasked than masked conditions to the fact that the nonpredictive nature of the primes was only registered in the visible priming conditions, such that participants actively ignored visible but not nonconscious primes. Most noteworthy, in an earlier cross-modal priming study of auditory target positions by masked visual spatial words, a similar reduction of prime processing in visible compared to nonconscious priming conditions was also observed (see Experiment 3 of Ansorge et al., 2016). Here, it is not only easier for the participants to notice the nonpredictivity of the visible primes than that of the masked primes and, thus, that critical requirements for the strategical filtering out of the primes were granted in unmasked/visible but not in masked conditions (cf. Kinoshita et al., 2011): the fact that the primes always call upon a different sensory modality than the targets might have additionally facilitated the active suppression of the visible primes. In the current study, Experiment 2 shows that a corresponding task set might even transfer from a preceding unmasked priming block to a subsequent masked priming block.
5 Conclusion
The present study showed that interactions between relevant and irrelevant congruency relations can successfully be used as a novel test bed for processing of irrelevant nonconscious prime characteristics. This was achieved by a cross-modal effect of nonconscious visual musical primes on heard auditory targets, and results supported the conclusion that some nonconscious effects are due to semantic prime processing. This demonstration is particularly striking, as both an objective visibility measure and qualitative processing differences confirmed the participants’ unawareness of the primes. Additionally, it was shown that participants actively suppressed uninformative primes and even transferred this task set to a different subsequent experimental block.
Acknowledgements
The authors would like to thank Stuart Klapp and two anonymous reviewers for their helpful and constructive comments on an earlier version of the manuscript.
-
Funding information: This research received no specific grant from any funding agency, commercial or nonprofit sectors.
-
Author contributions: MA and UA contributed to concept and study design. SJE and CB programmed the experiment. MA and SJE collected the data which were analyzed by MA. All authors contributed to the discussion of the data. MA wrote the first draft of the manuscript which was revised and proofread by all authors.
-
Conflict of interest: Authors state no conflict of interest.
-
Data availability statement: The datasets generated during and/or analysed during the current study are available in the Open Science Framework (OSF) repository, https://osf.io/nq48s.
-
Ethics statement: This research did not require ethical approval. All participants gave written informed consent in accordance with the Declaration of Helsinki.
-
Open practices statement: The data and materials necessary to implement the experiment are available at https://osf.io/nq48s/. The experiment was not preregistered.
References
Ansorge, U. (2004). Top–Down contingencies of nonconscious priming revealed by dual–task interference. The Quarterly Journal of Experimental Psychology Section A, 57(6), 1123–1148. doi: 10.1080/02724980343000792.Search in Google Scholar
Ansorge, U., Khalid, S., & Laback, B. (2016). Unconscious cross-modal priming of auditory sound localization by visual words. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(6), 925–937. doi: 10.1037/xlm0000217.Search in Google Scholar
Ansorge, U., Kunde, W., & Kiefer, M. (2014). Unconscious vision and executive control: How unconscious processing and conscious action control interact. Consciousness and Cognition, 27, 268–287. doi: 10.1016/j.concog.2014.05.009.Search in Google Scholar
Atherton, R. P., Chrobak, Q. M., Rauscher, F. H., Karst, A. T., Hanson, M. D., Steinert, S. W., & Bowe, K. L. (2018). Shared processing of language and music: Evidence from a cross-modal interference paradigm. Experimental Psychology, 65(1), 40–48. doi: 10.1027/1618-3169/a000388.Search in Google Scholar
Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577–660. doi: 10.1017/S0140525X99002149.Search in Google Scholar
Ching, A. S. M., Kim, J., & Davis, C. (2019). Auditory–visual integration during nonconscious perception. Cortex, 117, 1–15. doi: 10.1016/j.cortex.2019.02.014.Search in Google Scholar
Damian, M. F. (2001). Congruity effects evoked by subliminally presented primes: Automaticity rather than semantic processing. Journal of Experimental Psychology: Human Perception and Performance, 27, 154–164. doi: 10.1037/0096-1523.27.1.154.Search in Google Scholar
Dehaene, S., & Naccache, L. (2001). Towards a cognitive neuroscience of consciousness: Basic evidence and a workspace framework. Cognition, 79, 1–37. doi: 10.1016/S0010-0277(00)00123-2.Search in Google Scholar
Dehaene, S., Naccache, L., Le Clec’H, G. L., Koechlin, E., Mueller, M., Dehaene-Lambertz, G., Van de Moortele, P. F., & Le Bihan, D. (1998). Imaging unconscious semantic priming. Nature, 395, 597–600. doi: 10.1038/26967.Search in Google Scholar
Diependaele, K., Sandra, D., & Grainger, J. (2005). Masked cross-modal morphological priming: Unravelling morpho-orthographic and morpho-semantic influences in early word recognition. Language and Cognitive Processes, 20(1–2), 75–114. doi: 10.1080/01690960444000197.Search in Google Scholar
Dixon, N. F. (1971). Subliminal perception. The nature of a controversy. McGraw-Hill.Search in Google Scholar
Frings, C., Moeller, B., & Rothermund, K. (2013). Retrieval of event files can be conceptually mediated. Attention, Perception, & Psychophysics, 75, 700–709. doi: 10.3758/s13414-013-0431-3.Search in Google Scholar
Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565. doi: 10.3758/BF03196313.Search in Google Scholar
Greenwald, A. G., Draine, S. C., & Abrams, R. L. (1996, September). Three cognitive markers of unconscious semantic activation. Science, 273(5282), 1699–1702. doi: 10.1126/science.273.5282.1699.Search in Google Scholar
Hommel, B. (1998). Automatic stimulus-response translation in dual-task performance. Journal of Experimental Psychology: Human Perception and Performance, 24(5), 1368–1384. doi: 10.1037//0096-1523.24.5.1368.Search in Google Scholar
Hommel, B. (2004). Event files: Feature binding in and across perception and action. Trends in Cognitive Sciences, 8(11), 494–500. doi: 10.1016/j.tics.2004.08.007.Search in Google Scholar
Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24(5), 849–878. doi: 10.1017/S0140525X01000103.Search in Google Scholar
Horner, A. J., & Henson, R. N. (2011). Stimulus–response bindings code both abstract and specific representations of stimuli: Evidence from a classification priming design that reverses multiple levels of response representation. Memory & Cognition, 39, 1457–1471. doi: 10.3758/s13421-011-0118-8.Search in Google Scholar
Kiefer, M. (2001). Perceptual and semantic sources of category-specific effects: Event-related potentials during picture and word categorization. Memory & Cognition, 29, 100–116. doi: 10.3758/BF03195745.Search in Google Scholar
Kiesel, A., Kunde, W., Pohl, C., & Hoffmann, J. (2006). Priming from novel masked stimuli depends on target set size. Advances in Cognitive Psychology, 2(1), 37–45. doi: 10.2478/v10053-008-0043-y.Search in Google Scholar
Kinoshita, S., Mozer, M. C., & Forster, K. I. (2011). Dynamic adaptation to history of trial difficulty explains the effect of congruency proportion on masked priming. Journal of Experimental Psychology: General, 140, 622–636. doi: 10.1016/j.cognition.2007.11.011.Search in Google Scholar
Klapp, S. T. (2015). One version of direct response priming requires automatization of the relevant associations but not awareness of the prime. Consciousness and Cognition, 34, 163–175. doi: 10.1016/j.concog.2014.08.004.Search in Google Scholar
Klinger, M. R., Burton, P. C., & Pitts, G. S. (2000). Mechanisms of unconscious priming. I. Response competition, not spreading activation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 441–455. doi: 10.1037/0278-7393.26.2.441.Search in Google Scholar
Klotz, W., & Neumann, O. (1999). Motor activation without conscious discrimination in metacontrast masking. Journal of Experimental Psychology: Human Perception and Performance, 25, 976–992. doi: 10.1037/0096-1523.25.4.976.Search in Google Scholar
Koelsch, S. (2005). Neural substrates of processing syntax and semantics in music. Current Opinion in Neurobiology, 15(2), 207–212. doi: 10.1016/j.conb.2005.03.005.Search in Google Scholar
Koelsch, S., Kasper, E., Sammler, D., Schulze, K., Gunter, T., & Friederici, A. D. (2004). Music, language and meaning: Brain signatures of semantic processing. Nature Neuroscience, 7(3), 302–307. doi: 10.1038/nn1197.Search in Google Scholar
Kouider, S., & Dehaene, S. (2007). Levels of processing during non-conscious perception: A critical review of visual masking. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 857–875. doi: 10.1098/rstb.2007.2093.Search in Google Scholar
Kouider, S., & Dehaene, S. (2009). Subliminal number priming within and across the visual and auditory modalities. Experimental Psychology, 56(6), 418–433. doi: 10.1027/1618-3169.56.6.418.Search in Google Scholar
Kouider, S., & Dupoux, E. (2004). Partial awareness creates the “illusion” of subliminal semantic priming. Psychological Science, 15(2), 75–81. doi: 10.1111/j.0963-7214.2004.01502001.x.Search in Google Scholar
Kunde, W., Kiesel, A., & Hoffmann, J. (2003). Conscious control over the content of unconscious cognition. Cognition, 88(2), 223–242. doi: 10.1016/S0010-0277(03)00023-4.Search in Google Scholar
Leman, M. (2007). Embodied music cognition and mediation technology. MIT Press.10.7551/mitpress/7476.001.0001Search in Google Scholar
Leman, M., & Maes, P. J. (2015). The role of embodiment in the perception of music. Empirical Musicology Review, 9(3–4), 236-246. doi: 10.18061/emr.v9i3-4.4498.Search in Google Scholar
Marcel, A. (1983). Conscious and unconscious perception: Experiments on visual masking and word recognition. Cognitive Psychology, 15, 197–237. doi: 10.1016/0010-0285(83)90009-9.Search in Google Scholar
Mathôt, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 44(2), 314–324. doi: 10.3758/s13428-011-0168-7.Search in Google Scholar
McCauley, C., Parmelee, C. M., Sperber, R. D., & Carr, T. H. (1980). Early extraction of meaning from pictures and its relation to conscious identification. Journal of Experimental Psychology: Human Perception and Performance, 6(2), 265–276. doi: 10.1037/0096-1523.6.2.265.Search in Google Scholar
Mudrik, L., Faivre, N., & Koch, C. (2014). Information integration without awareness. Trends in Cognitive Sciences, 18(9), 488–496. doi: 10.1016/j.tics.2016.05.005.Search in Google Scholar
MuseScore 3.0.5.5992 [Computer software]. (2019). https://musescore.org.Search in Google Scholar
Naccache, L., & Dehaene, S. (2001). Unconscious semantic priming extends to novel unseen stimuli. Cognition, 80, 215–229. doi: 10.1016/S0010-0277(00)00139-6.Search in Google Scholar
Ortells, J. J., Kiefer, M., Castillo, A., Megías, M., & Morillas, A. (2016). The semantic origin of unconscious priming: Behavioral and event-related potential evidence during category congruency priming from strongly and weakly related masked words. Cognition, 146, 143–157. doi: 10.1016/j.cognition.2015.09.012.Search in Google Scholar
Patel, A. D., Gibson, E., Ratner, J., Besson, M., & Holcomb, P. J. (1998). Processing syntactic relations in language and music: An event-related potential study. Journal of Cognitive Neuroscience, 10, 717–733. doi: 10.1162/089892998563121.Search in Google Scholar
Reingold, E. M., & Merikle, P. M. (1988). Using direct and indirect measures to study perception without awareness. Perception & Psychophysics, 44, 563–575. doi: 10.3758/BF03207490.Search in Google Scholar
Sollberger, B., Reber, R., & Eckstein, D. (2003). Musical chords as affective priming context in a word-evaluation task. Music Perception, 20(3), 263–282. doi: 10.1525/mp.2003.20.3.263.Search in Google Scholar
Steinbeis, N., & Koelsch, S. (2011). Affective priming effects of musical sounds on the processing of word meaning. Journal of Cognitive Neuroscience, 23(3), 604–621. doi: 10.1162/jocn.2009.21383.Search in Google Scholar
Tranquada-Torres, B., Macleod, L., Lavezzi, G. D., & Cacciamani, L. (2022). Cross-modal masked priming of the tritone paradox. Memory & Cognition, 50(8), 1826–1841. doi: 10.3758/s13421-022-01282-6.Search in Google Scholar
Van den Bussche, E., Van den Noortgate, W., & Reynvoet, B. (2009a). Mechanisms of masked priming: A meta-analysis. Psychological Bulletin, 135(3), 452–477. doi: 10.1037/a0015329.Search in Google Scholar
Van den Bussche, E., Notebaert, K., & Reynvoet, B. (2009b). Masked primes can be genuinely semantically processed: A picture prime study. Experimental Psychology, 56(5), 295–300. doi: 10.1027/1618-3169.56.5.295.Search in Google Scholar
van Gaal, S., Ridderinkhof, K. R., van den Wildenberg, W. P. M., & Lamme, V. A. F. (2009b). Dissociating consciousness from inhibitory control: Evidence for unconsciously triggered response inhibition in the stop-signal task. Journal of Experimental Psychology: Human Perception and Performance, 35(4), 1129–1139. doi: 10.1037/a0013551.Search in Google Scholar
Wesslein, A., & Frings, C. (2020). Identity-based crossmodal negative priming: Aftereffects of ignoring in one sensory modality on responding to another sensory modality. Multisensory Research, 33(7), 703–721. doi: 10.1163/22134808-20201471.Search in Google Scholar
Wesslein, A. K., Spence, C., & Frings, C. (2014). Vision affects tactile target and distractor processing even when space is task-irrelevant. Frontiers in Psychology, 5, Article 84. doi: 10.3389/fpsyg.2014.00084.Search in Google Scholar
© 2023 the author(s), published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.
