More mental rotation time does not imply more mental effort: Pupillary diameters do not change with angular distance

The ability to mentally rotate objects in space is a fundamental cognitive capacity. Previous studies showed that the time to rotate the image of a figure to match another increases progressively with angular disparity. It remains unclear whether this increase in response time with angular disparity could reflect increased processing operations or more cognitive effort instead of a sustained use of a 'rotate' mechanism without a change in workload. We collected response times as well as pupillary responses that index cognitive workload and activity in the brainstem's locus coeruleus, from a sample of 38 young adults performing a chronometric mental rotations task. The results showed the expected increase in response times but no increase in pupil diameters between 60, 120, and 180 degrees of rotation, suggesting no significant changes in arousal levels when rotating figures near and far. This indicates that during mental rotation the load on cognitive resources remains constant irrespective of angular distance.


Introduction
A seminal article by Shepard and Metzler (1971) introduced the concept of mental rotation as well as a paradigmatic set of stimuli for testing this fundamental capacity of cognition: two-dimensional images of three-dimensional cube assemblies. The cubes are drawn as if seen in different positions or viewpoints, while mirrored or chiral versions (i.e., non-overlapping enantiomorphs) constitute the control stimuli. Shepard and Metzler (1971; see also Cooper & Shepard, 1973), showed that the time to correctly match an image of a figure to another increased with greater arcs of rotation between the two; they inferred from this that one 'mentally rotates' internal representations of the shapes before making the decision that they are identical. They also suggested that images 'rotate' within a mental space by passing through intermediate points in small steps. Such rotation within a mental analogue space would predict longer time with angular distance just as in physical space longer distance equates proportionally longer time. Carpenter (1976, 1985), on the other hand, suggested that rotations of geometric figures do not occur as a whole, as it would happen with real physical objects in real space, but according to a parts-based process. Thus, longer response times would indicate the intervention of multiple different mental operations, as adding mental operations in parallel can extend processing time. By monitoring incremental eye gaze shifts during the mental rotation of the Shepard-Metzler figures, they concluded that the transformation process happens sequentially for different portions of the image (for more recent similar account: see Xue, Li, Quan, Lu, Yue & Zhang, 2017). This form of transformation would have the advantage of reducing the computational burden in contrast to a truly analogue, holistic rotation, where a parallel computation of the position of all or most of figures' points is required. Such a gradual processing of a portion at a time could also reduce the amount of distortions during the mental transformation operation (e.g., Kosslyn, 1980). However, rotating figures' parts along longer paths, while also holding information about other parts in working memory (Hyun & Luck, 2007), may render the rotation processing progressively more effortful with angular distance, as indeed proposed by Just, Carpenter and Miyake (2003). According to their theoretical account, larger angular disparities recruit more resources for computing the intermediate orientations as well as for maintaining online the visual representations of the stimuli to compare. Hence, such an account predicts incremental processing with added computations that should be mirrored by increases in cognitive workload.
There is ample evidence indicating that changes in pupil size provide the best available and reliable index of cognitive workload while performing a task, starting in the 1960 ′ s (Hess & Polt, 1964;Kahneman & Beatty, 1966) to recent psychophysiological research (e.g., Laeng, Ørbo, Holmlund, & Miozzo, 2011;Wardhani, Mathot, Boehler & Laeng, 2020). A current account of pupillary changes to mental effort is that the eye pupils' diameters reflect the activity levels among neurons in the brainstem's locus coeruleus (Joshi, Li, Kalwani, & Gold, 2016), a part of the neuromodulatory system for cognitive 'arousal' based on norepinephrine (Costa & Rudebeck, 2016). Locus coeruleus sends noradrenergic projections to almost all brain regions, and importantly, has particularly dense projections to regions indicated in attentional processing (such as parietal cortex or superior colliculus, see Bouret & Sara, 2005;Schneider & Kastner, 2009). More recently, a pupillometry and neuroimaging study (Alnaes, Sneve, Espeseth, Endestad, van de Pavert, & Laeng, 2014) confirmed progressive increases in pupil size with cognitive workload, which were also directly related to activity in the noradrenergic locus coeruleus of the brain.
Hence, in mental rotation tasks, if the monotonically increasing response times with angular disparity reflect an incremental addition of cognitive processes while solving the perceptual problem (as seen generally in mental chronometry studies, e.g., Posner, 1978), the increase in cognitive workload should be also reflected in pupil dilations, since these are the hallmark of increased load on cognitive capacity (Kahneman & Beatty, 1966;Kahneman et al., 1967; see for reviews : Laeng, Sirois, & Gredebäck, 2012;Laeng & Alnaes, 2019). In Kahneman's (1973) words, pupil diameter provides a window on the "intensive aspect" of attention or 'mental effort' (i.e., processing capacity and mental resources; see also Beatty, 1982). Indeed, mental effort has been a central concept in psychology at least since William James (1890) and it was rekindled in contemporary psychology by Daniel Kahneman (1973). In more recent years, within the domain of decision making, mental effort has been viewed as a key process in the exercise of cognitive control (Shenhav, et al., 2017). It relates to the flexible use of executive functions to meet goals and control costs, which was also one aspect of Kahneman's model of attention and effort (1973, p. 47). Cognitive control domains are also expressed in pupillary changes (e.g., linked to the domains of 'updating, switching, and inhibition'; e.g., van der Wel & van Steenbergen, 2018). However, the actual process of mental rotation depends on activity in brain areas (e.g., intraparietal sulcus, medial precentral cortex; see Gauthier, et al., 2002;Zacks, 2008) that appear to differ with the ones involved in cognitive control (e.g., cingulate cortex, dorsolateral prefrontal cortex, nucleus accumbens, etc.; see Botvinick, Huffstetler, & McGuire, 2009;Power & Petersen, 2013;Botvinick & Braver, 2015). Moreover, the mental rotation component per se may be best seen as a form of automatic rather than controlled processing (Corballis, 1986).
Although mental rotation is one of the most investigated cognitive tasks in psychology, very few studies have investigated changes in mental effort while mentally rotating figures. Some studies used the eye tracking method but left changes in pupil diameter unexamined (e.g., Martini, Furtner, & Sachse, 2011;Nazareth, Killick, Dick, & Pruden, 2019;Xue et al., 2017); others have mainly focused on differences in pupillary responses between males and females (Campbell, Toth & Brady, 2018;Toth & Campbell, 2019) and showed inconclusive results about pupillary changes along angular disparity. 1 Hence, in the present study, we used an eye tracker during a classic mental rotation task, while monitoring the participants' pupil diameters, to assess whether changes in angular distance cause changes in mental effort (suggesting recruitment of incremental operations) or whether the level of effort remains unchanged (reflecting a constant rate of transformation within mental space).
Based on the account of Just, Carpenter, and Miyake (2003) we would expect that increases in angular distance would increase both response times and pupil diameter. Indeed, Just and colleagues predicted explicitly that pupillary responses should increase monotonically as angular disparity between the same figure pairs increases. In contrast to their account, previous studies suggested that the specific component of mental rotation does not pose additional demands on cognitive resources. Corballis (1986) had participants performing a mental rotation task either alone or during the retention phase of a short-term memory task. Although the dual task slowed overall the response times, it did not alter the rate of mental rotation and, moreover, memory was not differentially affected by angular disparity in the mental rotation task. The above findings with adults were later replicated with a sample of children (Kail, 1991). Based on this evidence, we would expect that pupil sizes would be unaffected by angular distance and, consequently, dissociate from response times.

Participants
Thirty-eight right-handed adults (18 females, mean age 25.6 ± 5) participated in the study. The original study by Shepard & Metzler (1971) with the same type of stimuli reported a reliable effect of angle on response times with the sample as small as N = 8, hence the present sample was deemed sufficient to replicate the effect. Since no other study examined planned comparisons of pupillary responses as an index of mental effort between different angles of rotation, we decided to collect double the sample size reported in an existing within-participant pupillometry study on mental effort (also with 4 levels within the condition per participant; see Alnaes et al., 2014). All participants in the current study were undergraduate or graduate students from diverse study programs and reported normal or corrected to normal (with glasses or contact lenses) vision.

Design and materials
The study design included a within-participant factor of Condition (Matching figures, Nonmatching figures), a within-participant factor of Degree (0, 60, 120, and 180 degrees of rotation), and three dependent variables: accuracy (proportion correct responses), response times, and pupil sizes.
We randomly selected 7 distinct figures from the mental rotation stimulus library (Peters & Battista, 2008, see Supplementary materials). For each of the figures, we chose 4 matching and 4 nonmatching pairs at four different degrees of rotation relative to each other: 0 degrees (no rotation), 60 degrees, 120 degrees, and 180 degrees (56 pairs in total). In the matching pairs, both figures were identical regardless of the rotation degree, whereas the nonmatching pairs were partial mirror images of each other (see Fig. 1).

Apparatus
The stimuli were shown on a compact Tobii T120 eye-tracker (Tobii Technology, Danderyd, Sweden; www.tobii.com) at a 1280x1024 resolution. All stimuli were presented using E-Prime 2.0 software (Psychology Software Tools Inc., Pittsburgh, USA; www.pstnet.com/ products/e-primewww.pstnet.com/products/e-prime/) and offline responses were collected alongside eye data. The Tobii Eye Trackers output pupil diameters (in millimeters) automatically for each gaze point registered on the screen. Pupil diameters are defined as the actual, internal physical size of the pupil in the Tobii eye model. In the T120 eye-tracker, the camera is built into the rim of a 17-inch TFT monitor and allows for a 300 × 220 × 700 mm freedom of head movements at a tracking distance of 50-80 cm. Participants were seated within that distance range from the screen (~60 cm). Eye data were recorded at a sampling rate of 120 Hz throughout the study.

Procedure
The current study was conducted in compliance with the Regional Committees for Medical and Health Research Ethics (REK) in Norway as a part of the "Spatial Language and Spatial Cognition in Autism Spectrum Disorder" project at the Norwegian University of Science and Technology (reference number 2015/1642; note that only typically developing adults from the project sample were included in the current study). All participants signed the consent form for the study participation and were told they would compare geometric figures and respond with button presses to whether the figures were the same or different.
The study started with a practice session (4 trials) followed by a standard 5-point calibration procedure. Next, each participant saw 56, fully randomized trials (one figure pair per trial). Each trial started with a fixation cross (250 ms fixating time required before proceeding to the next image), followed by a baseline image (1000 ms; matched in luminance to the following image), the figure pair (4000 ms) and a blank screen (until space bar is pressed to proceed). Participants responded as soon as they knew if the figures were same or different; however, each stimulus image remained on the screen for a fixed time of 4000 ms in order to allow for the comparison of pupillary changes within that period. The participants were informed that the images might remain on the screen after the response and they should wait for the blank screen to proceed.
When the figure pair disappeared and there was no response, participants could still respond before proceeding to the next trial. These late responses were included in the analyses of the overall accuracy in the task but excluded from the analyses of response times and pupillary dilation.
Data collection took place in the Language Acquisition and Language Processing Lab at the Norwegian University of Science and Technology (Trondheim, Norway) and in the Cognitive Laboratories at the University of Oslo (Oslo, Norway). At the end of the testing procedure, all participants received a gift card (for cinema or a water park) as a reward for their participation.

Behavioral responses
First, we calculated the overall task accuracy for each participant. We set an accuracy threshold at 65% correct in order to focus on accurate performance and avoid the loss of pupil data. Out of all 38 participants tested, 30 met these accuracy criteria. We proceeded with the analysis of the data from 30 included participants (13 females, mean age 24.8 ± 4.5, range 20-36 years).
We ran a planned linear model with Condition (Matching pairs, Nonmatching pairs) and Degree (0, 60, 120, or 180 degrees of rotation) as fixed factors on accuracy scores. We observed a significant main effect of Degree, F(1, 29) = 67.03, p < .001, and a significant Condition and Degree interaction, F(1, 29) = 31.95, p < .001. Post-hoc comparisons with Holm correction showed significant difference between 0 and 60 degrees (p = .007), 60 and 120 degrees (p < .001), and 120 and 180 degrees (p = .024) in the Matching trials, but not in the Nonmatching trials (all ps > .06), showing that accuracy decreased with increasing angular distance in matching figures (see Fig. 2).
Next, we analyzed response times for the Matching pairs (28 trials per participant, 840 trials in total). We excluded the trials with no responses given within 4000 ms of stimulus presentation (141 trials) and incorrectly answered trials (96 trials). We ran a planned linear model with Degree (0, 60, 120, or 180 degrees of rotation) as a fixed factor on response times. We observed a main effect of Degree, F(1, 29) = 226.25, p = < .001. Post-hoc comparisons with Holm correction showed significant difference between 0 and 60 (p < .001), 120 (p < .001), and 180 degrees (p < .001), as well as between 60 and 120 (p < .001) and 180 degrees (p = .009), but not between 120 and 180 degrees (p = .175; see Fig. 3).

Pupillary responses
The obtained pupil diameters (in millimeters) were regressed and averaged between the left and the right eye pupil using PupillometryR package in R software (Forbes, 2020). We ran a planned general additive model (GAM; see van Rij et al., 2019 for a detailed account on using GAMs in pupillometry) on the averaged data from the Matching pairs with Time and Degree as fixed factors using the following model formula: MeanPupil ~ s(Time) + s(Time, by = Degree). The model explained 52.4% of the deviance (adjusted R 2 = .524) and showed a significant smoothed effect of Time (p < .001), and significant Time and Degree interaction (p < .001). Fig. 4 shows the observed difference in the evolution of pupil dilation between trials with no rotation (0 degrees) and rotated figures (irrespectively of the degree of rotation).
Next, we calculated mean baseline-corrected pupillary changes for the Matching pairs by subtracting (trial by trial) mean pupil sizes at baselines (− 1000 to 0 ms) from the mean pupil sizes at stimulus (0 to 4000 ms). We ran a planned linear model with Degree (0, 60, 120, or 180 degrees of rotation) as a fixed factor on baseline-corrected pupillary changes. We observed a significant main effect of Degree, F(1, 29) = 5.59, p < .001. Post-hoc comparisons with Holm correction showed significant difference between 0 and 60 (p = .04), 120 (p = .002), and 180 degrees (p = .006). However, there was no significant difference between 60 and 120, 60 and 180, or 120 and 180 degrees (all ps > .75; see Fig. 5).
To examine at which time-point the differences emerged, we performed a time window analysis on the baseline-corrected pupillary changes in the Matching pairs. We divided the stimulus presentation time (4000 ms) into 8 time windows (500 ms each) and ran a linear model with Time Window (500, 1000, 1500, 2000, 2500, 3000, 3500, 4000 ms) and Degree (0, 60, 120, 180 degrees) as fixed factors on pupillary changes and observed a main effect of Time Window, F(7, 23) = 158.5, p < .001, a main effect of Degree, F(3, 27) = 17.99, p < .001, and a significant Time Window and Degree interaction, F(21, 926) = 2.61, p < .001. Post-hoc comparisons with Holm correction revealed significant differences between pupillary changes in 0 compared to 60, 120, and 180 degrees from 3000 ms to 4000 ms (time windows 3000-3500 ms and 3500-4000 ms; all ps < .01). However, there were no significant differences between 60, 120, and 180 degrees of rotation in any of the time windows (all ps > .5; see Fig. 6).
To gather further support for the negative finding (no differences in pupillary changes between 60, 120, and 180 degrees of rotation), we ran post-hoc Bayesian analyses of variance with Rotation Degree (60, 120, 180) as a fixed factor on pupillary changes in the Matching pairs (default prior r = 0.5 for fixed effects). The analysis showed an estimated Bayes factor of BF 10 = .261, suggesting a moderate evidence in favor of null hypothesis, and the error of 0.02% indicating a good stability of the algorithm. Further Bayesian pairwise comparisons between pupillary changes in 60, 120, and 180 degrees of rotation also showed estimated Bayes factors in favor of the null hypothesis (all BF 10 ≤ .905). In addition, we ran Bayesian sequential analyses (through pairwise t-tests). The illustration of the analysis in Fig. 7 indicates there was no participant whose results were above the 'threshold' indicative of conclusive evidence for a difference in pupil size between any of the rotation degrees.
Because mental fatigue can arise from sustained mental effort (DeLuca, 2005) and in turn, cause changes in mental effort over time, we also ran similar Bayesian comparisons separately for the first half (first 14 trials) and the second half (last 14 trials) of all Matching trials. The   Fig. 4. The smoothed GAM model of the time course of pupil dilation for all degrees of figure rotation in the Matching pairs in the task (0 degrees -no rotation, 60 degrees, 120 degrees, and 180 degrees). Zero represents the onset of the stimulus preceded by baseline (− 1000 ms to 0 ms). The shaded area represents pointwise 95% confidence intervals. pairwise comparisons of pupillary changes between 60, 120, and 180 degrees of rotation from both the first trials (all BF 10 ≤ .432) and from the last trials (all BF 10 ≤ .425) showed estimated Bayes factors in favor of the null hypothesis.
Finally, to further scrutinize the role of individual performance on pupillary changes, we ran a post-hoc analysis splitting the participants to High and Low Performers groups (following Ahern & Beatty, 1979) based on their overall task accuracy (threshold set at 85% correct responses, High Performers: N = 16, M = .93, SD = .25, Low Performers: N = 14, M = .78, SD = .41). Bayesian comparisons showed a reliably larger pupillary dilation in High Performers (M = .282, SD = .144) compared to Low performers (M = .245, SD = .156, BF 10 = 6.65: see Fig. 8), while there was no reliable difference in response times between the groups (BF 10 = .09).

Discussion
We found evidence for no changes in pupil size with angular disparity, as confirmed by Bayesian analyses, though we observed the expected and highly replicated incremental effect in response times with increasing rotations. The eye pupils only showed a significant dilation between instances of no rotation (i.e., when the figure pairs were identical) and all rotated angles we tested, revealing that performing mental transformation taxes mental resources regardless of distance travelled in mental space. Crucially, across the various degrees of rotation (60, 120, or 180 degrees), there was no effect on pupillary responses, suggesting no significant changes in arousal levels controlled by noradrenergic projections from the locus coeruleus when rotating figures near and far. Just and Carpenter (1993;see also Just, Carpenter & Miyake, 2003) were among the first to point the relevance of studying the oculomotor system for understanding mental rotation and also suggested that the pupillary response was an indicator of 'how intensely the processing system is operating.' In addition, Just, Carpenter and Miyake (2003) suggested that pupil dilations reflect an aggregate measure or summed index of brain activity across separate pools of resources, some of which limited by the lower-level architecture of processing modules, i.e. for spatial and language processing, but also with regard to the processing capacity of higher order systems involved in executive control. Given that pupil dilation is a sensitive measure indexing mental effort (Kahneman & Beatty, 1966), the current results suggest that mental rotation exerts a constant rate of demands on mental resources (Kahneman, 1973) and confirm the idea that translations in mental space happen at a constant rate (Cooper & Shepard, 1973).
The present results seem also at odds with an account based on discrete sampling of the parts of figures, as suggested by studies of eye fixations during mental rotation (Just & Carpenter, 1976, 1985. However, the step-by-step sampling by eye gaze might be also consistent with a continuous rotation process, e.g. as a repeated sequence of steps of search, transformation, and verification during rotation. In this way, rotation process does require more time but not necessarily incremental changes in the allocation of resources with distance. In a MEG study of mental rotation, Michel et al. (1994) found that the larger the arcs of rotation, the longer the MEG-measured brain activity persisted, which can be interpreted as evidence for sustained effort during the whole rotation time (instead of an index of increased demand, as suggested by Just, Carpenter & Miyake, 2003). They also found that the number of activated brain areas increased (from unilateral to bilateral) with disparity, but this happened only when the rotation angle was very large. However, according to a true monotonic relationship, the load on capacity should occur in incremental steps and across intermediate as well as extreme disparities. In fact, we found no evidence for a change in pupil diameter between 60 degrees and 120 degrees, or between these and the largest angular disparity of 180 degrees. Hence, we conclude that, in the present case, longer response times do not imply the incremental addition of cognitive processes. Instead, the present results provide evidence for a process that is sustained in time but running at constant rate. In a metaphor, when driving a car at a constant speed, a steadily running engine will not use more fuel along the different points of the journey, regardless of the traveled distance in physical space. However, for the periods when gearshifts are required (i.e., controlled processing), a different amount of resources might be needed at different timepoints. Similarly, when mentally rotating or transforming figures, we perform these operations continuously, with a constant amount of mental resources, and not incrementally, changing mechanisms and the amounts of resources.
Finally, we also observed that participants who were more accurate in the task (High Performers) showed larger pupil dilation compared to those who were less accurate (Low Performers). This indicates that despite the lack of increase in pupil dilation with increasing arcs of rotation, recruiting more effort overall resulted in a better task performance. In addition, the observed result suggests that high performance in the task might be achieved by allocating more mental resources 'in the moment' (i.e. during the task) rather than being dependent on individual aptitudes or predispositions (see Ahern and Beatty, 1979). To conclude, the present findings show that engaging in mental rotation taxes cognitive resources, as it is accompanied by the increase in pupil diameters reflecting increased cortical activity through the release of norepinephrine from the brainstem's locus coeruleus. However, this activity remains unchanged for various angular disparities, as indicated by no changes in pupil sizes when rotating near and far, pointing to sustained mental effort or unchanging demands on cognitive resources Fig. 6. Mean baseline-corrected pupillary changes separately for all degrees of rotation (0 degrees -no rotation, 60 degrees, 120 degrees, 180 degrees) in 8 time windows (500 ms each) during the stimulus presentation (4000 ms in total). Each dot represents an individual baseline-corrected data point from a single trial. Fig. 7. Bayesian sequential analyses (paired t-tests) of the average pupillary changes per participant in 60 versus 120 degrees, 60 versus 180 degrees, and 120 versus 180 degrees conditions. The grey dots indicate evidence for the default prior, and the black and white dots indicate evidence for the wide and ultrawide priors (respectively) as a part of the robustness check. The graphs were created in jamovi software (The jamovi project, 2020). during mental rotations.

Notes
1. Although Campbell, Toth & Brady (2018) concluded, based on a marginally significant main effect in a standard, frequentist ANOVA (not Bayesian), that pupil size increased with angular disparity, the confidence intervals displayed on the graph in their article suggest the opposite conclusion: that pupil responses did not change along angular disparity. Moreover, Campbell, Toth & Brady (2018) and Toth and Campbell (2019) did not report their measures of effort as average pupillary amplitudes, as it is typically done and recommended (Beatty, 1982) in studies of mental effort, but used instead only the maximal change from baseline or peak dilation. Peak pupil responses are not the optimal way to measure effort from pupil responses since they are based on a single data point (the peak in each trial) and therefore prone to artifacts compared to using all the data samples over a whole epoch. Hence, these previous studies remain inconclusive about the role of mental effort during mental rotation. Fig. 8. Mean pupillary change for High Performers (task accuracy > 85%) and Low Performers (task accuracy ≤ 85%). The dots on the boxplot represent mean pupillary change per participant per trial.