UvA-DARE (Digital Academic Repository) Should online math learning environments be tailored to individuals' cognitive profiles?

Online learning environments are well-suited for tailoring the learning experience of children individually and on a large scale. An environment such as Math Garden allows children to practice exercises adapted to their speciﬁc mathematical ability; this is thought to maximize their mathematical skills. In the current experiment, we investigated whether learning environments should also consider the differential impact of cognitive load on children’s math performance depending on their individual verbal working memory (WM) and inhibitory control (IC) capacity. A total of 39 children (8–11 years old) performed a multiple-choice computerized arithmetic game. Participants were randomly assigned to two conditions where the visibility of time pressure, a key feature in most gamiﬁed learning environments, was manipulated. Results showed that verbal WM was positively associated with arithmetic performance in general but that higher IC predicted better performance only when the time pressure was not visible. This effect was mostly driven by the younger children. Exploratory analyses of eye-tracking data ( N = 36) showed that when time pressure was visible, children attended more often to the question (e.g., 6 (cid:1) 8). In addition, when time pressure was visible, children with lower IC, in particular younger children, attended more often to answer options representing operant confusion (e.g., 9 (cid:1) 4 = 13) and visited more answer options before responding. These ﬁndings


a b s t r a c t
Online learning environments are well-suited for tailoring the learning experience of children individually and on a large scale. An environment such as Math Garden allows children to practice exercises adapted to their specific mathematical ability; this is thought to maximize their mathematical skills. In the current experiment, we investigated whether learning environments should also consider the differential impact of cognitive load on children's math performance depending on their individual verbal working memory (WM) and inhibitory control (IC) capacity. A total of 39 children (8-11 years old) performed a multiple-choice computerized arithmetic game. Participants were randomly assigned to two conditions where the visibility of time pressure, a key feature in most gamified learning environments, was manipulated. Results showed that verbal WM was positively associated with arithmetic performance in general but that higher IC predicted better performance only when the time pressure was not visible. This effect was mostly driven by the younger children. Exploratory analyses of eye-tracking data (N = 36) showed that when time pressure was visible, children attended more often to the question (e.g., 6 Â 8). In addition, when time pressure was visible, children with lower IC, in particular younger children, attended more often to answer options representing operant confusion (e.g., 9 Â 4 = 13) and visited more answer options before responding. These findings https://doi.org/10.1016/j.jecp.2019.104730 0022-0965/Ó 2019 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). and resisting interference (Diamond, 2013). IC skills have been found to predict mathematical performance in typically developing children, particularly in preschool and primary school children (Bull, Johnston, & Roy, 1999;Bull & Scerif, 2001;Espy et al., 2004;St Clair-Thompson & Gathercole, 2006). In online game-based learning environments, task-irrelevant distracting stimuli, such as gamified sounds, flashing objects, and alternative answer options, can trigger typically made errors. Similar to the Simon effect (Simon, 1969), where studies have found that irrelevant sensory stimuli in a task directly influence response selection and increase reaction time, the presence of irrelevant information in an online learning environment could interfere with performance in terms of accuracy and reaction time depending on one's level of IC. Furthermore, Bull et al. (1999) and Rourke (1993) suggested that a lack of IC is also reflected in the type of errors children tend to make, for example, the inability to switch away from addition when multiplication is required (i.e., operant-related error).
Interference and cognitive overload in a learning environment do not always stem from external stimuli but can also be internal in the form of worries about individual performance or about perceived time pressure (Ashcraft & Kirk, 2001;Mendl, 1999). These stressors can either drive people to use more efficient strategies (i.e., the best speed-accuracy trade-off within the constraints of the new situation) or compete with the attention that is normally allocated to the execution of the task (Caviola, Carey, Mammarella, & Szucs, 2017;Starcke & Brand, 2012). The latter is also known as the adverse effect of ''choking under pressure", where individuals perform worse than if there were no pressure (Baumeister, 1984;Beilock & DeCaro, 2007;Lewis & Linder, 1997). Critically, studies have found that people with high WM capacity are more affected by this dual-task environment and suffer more under pressure than those with low WM capacity (Beilock & Carr, 2005;Sattizahn, Moser, & Beilock, 2016;Wang & Shah, 2014). In addition, Sattizahn et al. (2016) found that individuals' variability in attentional control processes influenced the effect of pressure. Those with poor attentional processes suffered decreased performance under pressure, reflecting that some individuals are able to prevent the interfering effect of pressure on their performance, whereas others with poorer attentional control are not. So, although increased WM and IC are generally associated with better math performance and efficient strategy use, many studies have found that this depends on the stressors in the environment. The purpose of the current study was to investigate the impact of stressors in the relatively new context of an online learning environment.
One particular stressor, typical to a lot of online game-based learning environments, is time pressure, which is usually presented in the form of a gamified visual stimulus. For example, in Math Garden there is visual time pressure in the form of coins counting down every second, which is also incorporated into the game's scoring rule for math performance (i.e., ''high speed, high stakes" rule; see Maris & van der Maas, 2012). The advantage of using time pressure is that it provides the opportunity to relate speed of processing to the ability of the child, which is valuable with easy problems (Klinkenberg et al., 2011;van der Maas & Wagenmakers, 2005). In addition, in the case of games (similar to sports), the challenge of acting within a time limit can make the activity more enjoyable (Freedman & Edwards, 1988). Because time pressure itself is invaluable for most game-based learning environments, the current study addresses a different question: Should the visibility of the time pressure (in the form of a countdown) be adapted for individuals depending on whether it negatively affects math performance? Following the interference and overload theory, time pressure in the form of animated visual stimuli could be a distracting component that negatively interferes with solving math problems depending on the child's level of IC and WM. However, the alternative situation, with no visible reminder of time passing by, requires attention to be allocated to time perception, which could result in suboptimal strategies in speed-accuracy trade-off in the main task (Brown & Perreault, 2017;Grondin, 2010;Matthews & Meck, 2016;Zakay, 1993).
The purpose of this study was threefold. First, we investigated the association of individual differences in verbal WM and IC with performance of simple addition and multiplication problems in blocks of single or mixed operations in a game-based environment for primary school children. We expected that both verbal WM and IC would be positively associated with math performance and that higher IC would be associated with a reduced cost of switching between multiplication and addition. Second, we explored whether a particular feature of cognitive load, the visibility of time pressure, would affect arithmetic performance in general and whether this impact was different for various children depending on the level of WM and IC. We did not have a hypothesis regarding whether visibility or invisibility of time pressure would be associated with worse math performance because both features create a dual-task condition. Any effect on math performance was expected to interact with individual differences in WM and/or IC.
Finally, whether learners are attending to or actively inhibiting their attention to irrelevant/distracting stimuli can be studied by looking at eye movements and fixations (i.e., moments when the eyes are relatively stationary and fixed on an object) using eye-tracking technology (Duprez et al., 2016;Wijnen & Ridderinkhof, 2007). In a learning environment, eye tracking can be used to investigate how learners interact with the stimuli and how the order and duration of their attending affect their problem solving. Eye-tracking data can also be used to improve the learning environment based on knowledge of how learners process the materials through their eye movements (Asteriadis, Tzouveli, Karpouzis, & Kollias, 2009;Garcia-Barrios et al., 2004). Using eye tracking, we explored differences in the locus of attention during the arithmetic task depending on whether time pressure was visible or not and on children's levels of WM and IC.
This study included data from a single time point and, therefore, will not inform our understanding of how individual differences and task features affect learning over time. However, a better understanding of how performance in online math tasks may be affected by these factors could allow a tailoring of the environment to individual learners, making sure that the task challenges, and therefore trains, their arithmetic skills rather than loading on other aspects of their cognitive capacity.

Participants
A total of 42 primary school children aged 8 to 11 years were recruited through a local voluntary participant database and word of mouth. Three children were excluded from all analyses because testing sessions were interrupted due to distress or tiredness. The final sample consisted of 39 children (19 male; M age = 9.60 years, SD = 1.02, range = 8.00-11.50). For 3 children, insufficient eye gaze data were collected, leaving 36 children (18 male; M age = 9.67 years, SD = 1.00, range = 8.00-11.50) for the eye-tracking analyses. The study was approved by the departmental ethics committee at the university. Informed consent was given by caregivers, and verbal assent was given by participants.

Procedure
All stimuli were presented in MATLAB (The MathWorks, Natick, MA, USA) using the Psychophysics Toolbox (Brainard, 1997). During the first task, participants performed a math task on a computer (see Fig. 1A) similar in design to Math Garden (Straatemeier, 2014). The study took place in a lab setting, and all measures were completed in a single session lasting about 30 min in total. Before data collection started, condition assignment was randomized for a list of 40 participants using MATLAB. An additional 2 participants were tested to compensate for incomplete or withdrawn participants. There were three randomly ordered blocks comprising 20 multiplication problems, 20 addition problems, and 22 mixed multiplication and addition problems, respectively. All problems involved single-digit numbers between 1 and 9.
For each arithmetic problem, participants were asked to choose one of six answer options, which consisted of the correct answer and the five most frequent errors made by children of similar age on that arithmetic problem, based on Math Garden data previously collected from a large Dutch sample (Fig. 1A). Participants had a maximum of 8 s to click on one of the answers, after which the correct answer was highlighted. In a between-participants manipulation, 19 children were randomly assigned to the visible time pressure condition, where the time limit of 8 s was visible in the form of coins counting down at the bottom right of the screen, similarly to Math Garden (Fig. 1A). The other 20 children needed to respond within the same 8 s, but there were no coins on the screen (no visible time pressure condition). After every trial, direct feedback on performance was given; the correct answer was circled in green, and the incorrect answer was circled in red in the case of an incorrect response. The measure of math performance was calculated with a scoring rule following the equation s ij = (2x ij À 1)(d À t ij ) (adapted from Maris & van der Maas, 2012). This rule imposes a speed-accuracy trade-off, where fast and correct responses result in a high score and incorrect responses result in a negative score. Player j responds x ij on trial i (x ij = 1 in the case of a correct answer and x ij = 0 in the case of an incorrect answer) in time t ij (in seconds; range = 0 to 8) before the time limit d (set to 8 s in this study) and obtains the score s ij (range À8 to 8).
Participants' verbal WM was then assessed with a backward digit span task, where children were asked to repeat, backward, lists of single-digit numbers pronounced by the experimenter. After practice with two numbers, the first level consisted of four lists of three numbers; the child moved one level up (with an additional number) when at least three of the four lists were repeated back successfully. A WM score was computed as the total number of correct answers.
IC was assessed with a computerized spatial incompatibility Simon task (adapted from Duprez et al., 2016;see Fig. 1B). Children were asked to move their mouse to either the top left or top right box depending on the color of the target square while ignoring its location. When the target was blue children needed to move their mouse toward and click into the left box, and when it was orange they needed to move their mouse toward and click into the right box. In half of the trials the location of the Here the problem 6 Â 8 is presented at the top of the display with the six answer options underneath. For half of the participants, a visible time pressure was implemented through coins counting down every second. The current total score is depicted in the right bottom corner. The dotted black lines are drawn to represent the areas of interest for the gaze data. (B) Setup of the Simon task. A cue indicating the correct color-response mapping remained on the screen at all times (here indicating that the left box should be clicked for blue target stimuli and that the right box should be clicked for orange target stimuli). Participants first moved their mouse to the center of the display (small white square). After 500 ms, a blue or orange target square stimulus was presented in either top corner of the display. Participants were asked to move their mouse toward and click into the box corresponding to the color of the target stimulus. On congruent trials, the location of the target matched the response associated with the color of the target (e.g., orange target on the right side). When color and location did not match (e.g., orange target on the left side), the trial was incongruent and a conflict arose. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.) target was congruent with the correct response, and in the other half it was incongruent with the correct response (Fig. 1B). Participants completed 40 trials in a randomized order, which resulted in between 1 and 5 trials of the same type (congruent/incongruent) being repeated in a row. The measure of IC, referred to as the IC interference effect, was computed as the difference between incongruent and congruent trials mean reaction time (RT) divided by congruent trials mean RT, using correct trials only. A high score reflects a slower RT on incongruent trials (i.e., difficulty in inhibiting attention to irrelevant information) than on congruent trials (i.e., baseline processing speed).

Eye tracking
During the math and Simon tasks, children were seated at a distance of 60 cm in front of an eye tracker. Eye movements were recorded using a Tobii TX300 (Tobii, Danderyd, Sweden) at a sampling rate of 120 Hz. The raw data were classified into fixations and saccades using the gazepath package in R (R Core Team, 2013;van Renswoude et al., 2018). Gazepath uses an algorithm to categorize the data into fixations and saccades while accounting for individual differences and data quality. Fixations in the math task were labeled as one of the following three areas of interest (AOIs): (a) the question box, (b) one of the six answer options, or (c) the coins (i.e., the visible countdown of time) (Fig. 1A).

Statistical analyses
Data management and statistical analysis were performed using R software (R Core Team, 2013). For all independent variables, z scores were generated to standardize the scores for further analyses. In a first set of analyses, math performance was averaged over the three blocks (addition, multiplication, and mixed addition and multiplication) and compared between the visible time pressure condition and the no visible time pressure condition, covarying for age and WM score or IC interference effect, using between-participants three-way analyses of covariance (ANCOVAs). With a sample size of 39, the study had 80%, 90%, and 95% power to detect large eta-square (g 2 ) effect sizes of .18, .22, and .26, respectively, when comparing two groups. Eta-square effect sizes have been classified as follows: small g 2 = .02, medium g 2 = .13, and large g 2 = .26 (Cohen, 1988). An additional analysis investigated associations between IC and the cost of needing to switch between operations. We subtracted the average performance of the mixed block trials from the average performance on the trials in the single operation blocks for multiplication and addition problems separately. These cost measures were entered into ANCOVAs including the IC interference effect, visibility of time pressure, and age for multiplication and addition separately. Assumptions of the ANCOVAs were met, with analyses showing homoscedasticity and normality of the residuals.
Eye-tracking analyses (n = 17 in the visible time pressure condition and n = 19 in the no visible time pressure condition) focused on correct trials (excluding 12.7% of trials) and trials where there was at least more than one fixation to ensure high eye-tracking data quality (excluding a further 1.2% of trials). The average number of fixations and the proportional duration of fixation on each AOI were calculated for each participant. An additional metric was the average number of answer option AOIs the participant attended to on a trial. We explored, in three-way ANCOVAs, whether these eye-tracking metrics differed according to the visibility of time pressure and whether this interacted with WM score, IC interference effect, or math performance.
The data were checked for outliers using a criterion of |z score| > 3 for both the dependent and independent variables. No outliers were identified. In the regression analyses, Cook's distance suggested one to three influential points for some behavioral and eye-tracking results. Analyses were repeated excluding these data points, and the results were strengthened except in one case, which is discussed further below.
In addition, Bayesian ANCOVAs were performed post hoc for the results with null effects or p values just under the threshold (p < .05) using JASP (JASP Team, 2019). To quantify uncertainty about effect size and to obtain evidence in favor of a null hypothesis (Wagenmakers et al., 2018), we distinguished between experimental insensitivity (Bayes factor [BF] 10 and BF 01 < 3) and robust support for the alternative hypothesis (BF 10 > 3) or null hypothesis (BF 01 > 3) (Dienes, 2014)

RT and accuracy
We performed two one-sided equivalence tests with alpha = .05 and no assumption of equal variance and found statistical equivalence between the visible and no visible time pressure groups for age, percentage female, verbal WM, and IC (Table 1).
t Tests were run to test whether visibility of time pressure was associated with math performance. We did not find any difference between the groups in mean RT, t(38) = 0.82, p = .42, proportion of correct responses, t(38) = 0.45, p = .66, proportion of no response within the time limit, t(38) = 0.15, p = .88, or mean math score, t(38) = 0.77, p = .45. These comparisons indicate that the visibility of time pressure did not have an effect on math performance. The average overall math performance was 3.34 (SD = 1.34), meaning that the average score was correct and answered within roughly half the time limit (see Method for scoring rule). This measure was used for further analyses.
Mean accuracy in the Simon task was high (M = .99, SD = .05). As expected, RTs differed between congruent and incongruent trials, t(38) = 6.17, p < .001. Participants were on average 150 ms slower in incongruent trials ( Fig. 2A). The individual average IC interference effect was used as a measure of IC for further analyses (Fig. 2B).

Impact of time pressure on math performance depending on level of IC and verbal WM
The first analysis included only age and visibility of time pressure as predictors of math performance. This showed a positive association between age and math performance, F(1, 35) = 32.05, p < .001, g p 2 = .53 but not time pressure (p = .892, g p 2 = .00), nor was there an interaction between age and time pressure (p = .679, g p 2 = .01). The second analysis included WM score as a covariate (Table 2).
WM score was positively associated with math performance, F(1, 31) = 13.15, p = .001, g p 2 = .24 ( Fig. 3A), but there was no interaction with age or time pressure (all ps > .50, g p 2 < .01). The Bayesian ANCOVA showed that a null model with merely main effects for WM score and age was 11.4 times more likely than including any of the above-mentioned interactions or the main effect of time pressure. The third analysis (Table 2) included the IC interference effect as covariate. There was no main effect of IC interference effect (p = .62, g p 2 = .01), but there was a significant two-way interaction between time pressure and IC interference effect, F(1, 31) = 6.59, p = .015, g p 2 = .18, and a three-way interaction between time pressure, age, and IC interference effect on math performance, F(1, 31) = 4.55, p = .041, g p 2 = .13. Significant evidence for both interaction effects was demonstrated through Bayesian analyses (Table 2). To examine the two-and three-way interactions, separate multiple regressions were performed in the visible and no visible time pressure groups. In the group with visible time pressure, the IC inter- ference effect and Age Â IC interference effect interaction terms did not significantly predict variance in math performance (BF 01 = 0.57, i.e., no evidence for either hypothesis) (Fig. 3B). In contrast, the group with no visible time pressure showed a negative association between math performance and IC interference effect, b = À.42, t(16) = 2.77, p = .014 (BF 10 = 6.47, i.e., substantial evidence for including this effect) (Fig. 3B), and an interaction between age and IC interference effect, b = .43, t(16) = 2.629, p = .018 (BF 10 = 8.84). The interaction effect showed that the association between math performance and IC interference effect was mostly driven by younger children (Fig. 3C).

Operation switch cost
To investigate whether switching between operations led to a cost in performance, we compared the mean math scores of single operation versus mixed operation blocks for multiplication and addition problems separately. Paired t tests showed that children's performance on multiplication problems did not differ between the mixed (M = 2.83) and single (M = 2.76) operation multiplication blocks, t(38) = 0.41, p = .341. For addition, children performed less well on the trials in the mixed block (M = 3.67) than on those in the single operation block (M = 4.00), t(38) = 2.51, p = .008. Therefore, children showed a cost of needing to switch between multiplication and addition on addition problems only.
Because the ability to switch between arithmetic operations has been associated with IC in previous studies (Bull et al., 1999;Rourke, 1993), additional analyses explored whether IC predicted the ability to switch between addition and multiplication operations in the mixed operation blocks compared with the single operation blocks (Table 2). For addition problems, the IC interference effect predicted the performance difference between the mixed and single operation blocks, F(1, 31) = 5.06, p = .031, g p 2 = .13. Bayesian ANCOVA showed that a model including IC was 2.68 times more likely than the null model; no interaction with age (p = .302, g p 2 = .03) or time pressure (p = .153, g p 2 = .06) was found.

Eye fixations and patterns
Exploratory analyses investigated whether eye movements during the math task could give some insight into the behavioral findings. Analyses were performed on the mean number of fixations and proportion of total fixation duration on specific AOIs. The latter did not show any significant effect.
The first analyses looked at the fixations on the question box AOI (e.g., 6 Â 8 in Fig. 1A) because other studies have found that looking back and forth at the question is positively associated with attentional and WM load (Droll & Hayhoe, 2007;Orquin & Mueller Loose, 2013). ANCOVAs were run to test for associations with the visibility of time pressure in interaction with individual differences in IC and WM separately while covarying for age and math performance (Table 2). A significant main effect for time pressure, F(1, 28) = 12.02, p = .003, g p 2 = .43 (BF 10 = 30.88, i.e., very strong evidence), showed that there were more fixations on the question box when time pressure was visible (M = 2.69) than when there was no visible time pressure (M = 2.02) (Fig. 4A).
Second, because operation-related errors have been found to be associated with the level of IC (Bull et al., 1999;Rourke, 1993), fixations on the operation-related error answer options were investigated separately for addition and multiplication. ANCOVAs were performed to test for associations with the visibility of time pressure and the IC interference effect while covarying for age and math performance. For the addition problems with multiplication-related errors as answer options, we found no significant predictors (ps > .20, g p 2 < .05) ( Table 2). For multiplication problems with  addition-related errors, a significant interaction between time pressure and IC interference effect, F(1, 28) = 5.34, p = .018, g p 2 = .44 (BF 10 = 5.21, i.e., substantial evidence), showed that the mean number of fixations on the addition-related error increased with increasing IC interference effect (b = .56) only when time pressure was visible. Finally, analyses were performed to investigate the mean number of answer options participants looked at before giving their answer and whether this was related to WM, IC, and the visibility of time pressure. An ANCOVA was performed with the average number of answer options attended to as the dependent variable, visibility of time pressure and IC interference effect or WM as independent variables, and age and math performance as covariates. The analysis with WM as a predictor showed no main or interaction effects, only evidence that a null model was 11 times more likely than including any of the predictors. The analysis with IC as a predictor showed a significant interaction between visibility of time pressure and the IC interference effect, F(1, 31) = 4.60, p = .039, g p 2 = .13. However, Cook's distance highlighted that there was one influential point that drove this interaction. Consistent with this, only anecdotal evidence (BF 10 = 2.90) for including this interaction to the null model was found in the Bayesian regression ( Fig. 4C and Table 2). Follow-up regression analysis showed a trend for a positive association for the IC interference effect when time pressure was visible, b = .54, t(13) = 2.03, p = .063, but little evidence (BF 10 = 1.23) in the Bayesian regression. No association between the IC interference effect and number of answer options visited was found when time pressure was invisible, b = À .21, t(16) = 0.82, p = .423; Bayesian regression showed anecdotal evidence for the null hypothesis (BF 01 = 2.00). The association between math performance and IC interference effect depended on the visibility of the time pressure. (C) Graph illustrating the Age Â IC interference effect interaction on math performance for the no visible time pressure group. A median split was performed for age showing two regression lines for young age (8.0-9.5 years) and old age (9.6-11.5 years), but note that age was treated as a continuous variable in the analyses. (D) The cost of mixing operations on performance of the addition problems (mixed operations block score À single operation block score) was positively predicted by the IC interference effect.

Discussion
This study combined behavioral and eye-tracking measures to test whether individual differences in verbal WM and IC in primary school children could predict their ability to solve arithmetic problems in different online learning environments, where visibility of time pressure was varied. The behavioral results showed that verbal WM was a positive predictor of arithmetic performance in general, in line with previous studies (see Raghubar et al., 2010, for a review), and that this association was independent of the visibility of time pressure. In contrast, individual differences in IC predicted arithmetic performance only when the same time pressure was not visibly illustrated by an animation. In addition, we found that this association with IC was mostly driven by younger children, similar to previous studies (Bull & Scerif, 2001). Eye-tracking results also showed that children fixated on different parts of the stimuli during the math task depending on the visibility of time pressure, their IC level, and age.
Overall, these findings point out that the visibility of time pressure may affect performance of certain individuals in online learning environments and that possible constraints of attentional control (i.e., the amount of interfering information compromising cognitive resources) should be considered. Learning environments with both visible and invisible time pressure can create dual-task environments, leading to less attention to the main task of solving math problems. When time pressure is visible, the user has a constant physical reminder of timing (in this study in the form of an animated visual stimulus). Adding more visual stimuli and time pressure was suggested by previous studies to contribute to loading WM capacity, leading to suboptimal strategies and attention (Barrouillet, Bernardin, Portrat, Vergauwe, & Camos, 2007;Caviola et al., 2017;Terras & Ramsay, 2012). This impact can also be influenced by other individual differences such as math anxiety (Ashcraft & Krause, 2007;Caviola et al., 2017;Kellogg, Hopko, & Ashcraft, 1999), engagement, and attitude to learning (Barkatsas, Kasimatis, & Gialamas, 2009;Kebritchi, Hirumi, & Bai, 2010). Although the visibility of time pressure did not interact with individual differences in verbal WM in terms of math performance, the notion of visible time pressure as an increasing demand on WM resources is reflected in our eyetracking results. Children made more fixations on the question in the visible time pressure condition than in the invisible time pressure condition, suggesting that they may have found it more difficult to keep the question in mind (Orquin & Mueller Loose, 2013). Although previous studies suggested that the impact of extra stressors on math performance depends on the ability to resist distractions (i.e., IC; Sattizahn et al., 2016), we showed that the performance of children was not affected by their level of inhibition when time pressure was visible. The higher number of fixations on answer options and on operation-related errors did suggest that for children with lower IC the task was more demanding in terms of decision difficulty and/or attentional resources (Orquin & Mueller Loose, 2013), but this did not result in lower performance.
Time perception is intensively studied (for an overview of recent reviews, see Block, Grondin, & Gibbon, 2014) and involves diverse perceptual, motor, cognitive, and brain processes (Block & Gruber, 2014). One line of investigation in time perception concerns its bidirectional interference with higher-level executive cognitive processes such as mental arithmetic but also with executive functions (Block, Hancock, & Zakay, 2010;Brown, Collier, & Night, 2013). This interference occurs in a dual-task condition where time perception competes for the same attentional resources as the other task, leading to cognitive load. Because the interference is bidirectional, studies have also shown that lower IC is associated with less accurate time perception (Brown & Perreault, 2017;Meaux & Chelonis, 2005). This closely aligns with our finding that low levels of IC were associated with low arithmetic performance when children also needed to estimate time without a reminder. This could be due to an impairment of time perception, such that these children have trouble in deciding on an optimal speed-accuracy trade-off strategy. Therefore, for children with low IC, visualizing time pressure could reduce cognitive load, whereas children with high IC seem to be able to estimate time in parallel with solving arithmetic problems.
One of the limitations of this study was the small sample size. This was due to the use of an eye tracker, which necessitated a lab setting. The use of a participant volunteer database and testing in a lab setting also likely biased our recruitment toward children from higher socioeconomic backgrounds and with higher cognitive abilities. The next step would be to replicate our findings with a larger heterogeneous sample from online learning environments such as Math Garden to ensure that the behavioral findings are reliable. In addition, we chose a between-participants design that minimizes the effect of learning and testing time, but a within-participants design would have had more power to detect interactions between the time pressure manipulation and individual differences in WM and IC. Future work will investigate whether learning, rather than performance at a single time point, can be improved based on an adapted environment, informed by the results of the current study. Although the purpose of this study was to implement these findings in an online adaptive environment, the arithmetic problems used were standardized to ensure that we could compare arithmetic performance within this sample size. Due to our wide age range (8-11 years), certain arithmetic problems were inevitably less challenging for some children; therefore, all analyses were covaried for age. Note, however, that there are large individual differences within year groups on arithmetic tasks (Straatemeier, 2014), so a more homogeneous sample in terms of age may have still shown considerable variability in arithmetic performance. To further investigate whether the associations among IC, WM, time pressure, and arithmetical outcome change with age, the difficulty level of the arithmetic problems should be adapted to the ability of the child. Finally, whereas we considered the coin countdown to reflect time pressure, it also indicated the potential reward to be gained when correctly solving the problem. Although the reward obtained was shown to both groups of participants when a trial was completed, the group with no visible coin countdown did not have a constant reminder of the potential reward. This reward cue difference between the groups may have led to some of the differences observed between the conditions.
In conclusion, we found that the (in)visibility of time pressure, a key feature that is adaptable in a lot of online game-based learning environments and psychological tasks in general, may create cognitive overload and affect the application of knowledge and skills. Specifically, we showed that this aspect of online game-based learning environments may differentially affect children's arithmetic performance as a function of their cognitive abilities. Measuring the individual levels of cognitive functioning, in particular WM and IC, is essential to allow children to perform and practice tasks at their highest level. In addition, the use of an eye tracker in this context allowed an in-depth exploration of how learners interacted with the different elements in the environment above and beyond accuracy and RT. Future work should focus on developing a broader online adaptive framework for learning mathematical skills and knowledge that adapts not only to children's mathematical skills but also to their more general cognitive strengths and weaknesses.