Perspectives in eye-tracking technology for applications in education

system utility in educational research regarding the mathematics teaching–learning process.


Read online:
Scan this QR code with your smart phone or mobile device to read online.

Introduction
Eye-tracking is a technique that tracks a person's point of gaze or eye movements during a task, which can be helpful to research aspects of human cognition such as attention targets, focus and gaze patterns that suggest the observer's problem-solving strategy (Shayan et al. 2017).The eyetracker device records eye movements and identifies gaze points by projecting an infrared or near-infrared light to the eye.The positioning factors (corneal reflection and the centre of the pupil) are processed by the eye-tracking software that estimates the 'point-of-regard' (point in the space of the gaze trajectory) by trigonometric calculations indicating the eye movements (Hansen & Ji 2009).More than a century ago, scientific research used to use an eye-tracking system to obtain information about eye movement and explore how such data can expand our knowledge of human behaviour (Delabarre 1898).Since the first insights into the characteristics of eye movements in the late 1800s, several studies have investigated the gaze and reading (Erdmann & Dodge 1898;Huey 1898;Javal 1878).Over the years, a tremendous technological advance has improved the accuracy of the eye-tracker devices employed in research (Duchowski & Duchowski 2017).Which has increased the possibility of applications of this tool in research.
The eye-tracker equipment has been applied to estimate where an individual is looking at a given moment, which allows researchers to make assumptions about what the observer deems relevant while observing stimuli.Moreover, eye-tracker can provide the looking time data, which have been influential as a measure in developmental psychology for decades (Lai et al., 2013).For example, Yeung et al. (2016) recorded infants' gaze during surprising, neutral, or unsurprising events, showing that eye movement data can shreds light on infants' reasoning regarding a sampling event's likelihood.Interestingly, the study highlights that eye tracking can extract a more precise measurement than traditional looking measures, i.e., manual looking time coding, and therefore reveal some subtleties of response patterns in an automated way.The eye movement data is helpful to investigate the human cognition, including attention, focus and gaze patterns of the observer's problemsolving strategy (Shayan et al. 2017).
Interest in using eye-tracking in educational research has increased recently.Studies have applied research paradigms to elucidate mathematical thinking by investigating eye movements during controlled experiments (Was, Sansosti & Morris 2017).However, only in the last decades has the improvement of eye-tracking technology enabled more naturalistic tasks, that is, out of the traditional labs (Foulsham 2015;Hayhoe & Ballard 2005).In this sense, there is a trend in eye-tracking research in mathematics education (Lilienthal & Schindler 2019).Regarding the potential of eye-tracking applications in educational research and the growing use of computer resources in schools, it is vital to foster discussions in multidisciplinary research to propose solutions and paradigms that can benefit mathematics teaching learning.An analysis of eye movement studies related to learning would help education practitioners by providing insights into how eye-tracking technology can be helpful and the possibilities for future applications of the eye-tracking system in the educational context.
Our main goals are to provide an overview and different opportunities for educational eye-tracking research to investigate mathematical thinking at schools.Here, we present the usual measures and the trends of this technology that could enable educational practitioners and scientists to apply the eye-tracking system to benefit teaching and learning mathematics in naturalistic research.

Eye fixation and Saccadic movement when solving mathematical problems
Eye-tracker devices usually offer the possibility for the experimenter previously selecting an area of interest (AOI), which are specific regions of the stimulus displayed that the researcher defines before the task and then extracts specific quantitative metrics of the eye movements into the selected locations recorded during the experiment (Figure 2a).Among the main eye movement measures, saccades and fixations stand out as the most popular data analysed in the AOI.Fixations are eye movements that stabilise the retina over an object of interest.It is a measure widely used in research with eye-tracking, especially to investigate engagement and visual attention (Holmqvist et al. 2011).It is well known that participants tend to fixate their eyes when they find interesting information during the search.Moreover, the more significant number of fixations directed to a particular region indicates greater visual attention (Duchowski &  http://www.sajce.co.zaOpen Access

NaturalisƟc research in schools
Duchowski 2017).The fixation count indicates how often a participant fixated the eyes within a relevant AOI.
On the other hand, dwell time (fixation time) measures how long the observer fixated the eyes on the AOI (Figure 2b).Evidence suggests that longer dwell time and higher fixation count indicate that learners require more cognitive capacity for processing information on a specific area (Carpenter & Just 1978;Lin & Lin 2014).For example, Hegarty, Mayer and Green (1992) employed eye-tracking technology to observe undergraduate students' dwell time during arithmetic problem-solving.The study demonstrated that unsuccessful problem-solvers tended to have longer dwell time on the problem area, which is consistent with recent experiments on the geometry domain showing that low-accuracy students had higher fixation count and spent more time fixating their eyes on the problem region (Lin & Lin 2014).
Eye movements between the fixations are called saccades, reflecting the foveas's the moment-to-moment positioning (Gregory 1990).Therefore, saccadic movements and eye fixations have become practical measures of visual attention in cognition (Liversedge & Findlay 2000) and educational research (Kennedy 1992;Rayner 1998;Rayner et al. 2006;Taylor 1937).For instance, a seminal study on reading tests demonstrated that as the text became conceptually more difficult for readers, the duration of fixations increased and the length of saccades decreased (Rayner & Pollatsek 1992).
Regarding the mathematics domain, several studies investigate mathematical reasoning through fixation and saccades behaviour (Andrà et al. 2009(Andrà et al. , 2015;;Hegarty et al. 1992;Suppes 1990;Susac et al. 2014;Verschaffel, De Corte & Pauwels 1992).Eye-tracking research showed that students with mathematical anxiety had more saccade movements and eye fixations count during the arithmetic problemsolving tasks (Hunt, Clark-Carter & Sheffield 2015).The findings suggest that eye movements can indicate difficulties in performance and reduce students' processing efficiency.Andrà et al. (2009) also demonstrated differences between how a novice and an expert student read mathematical representations.The study suggests that low-performance students have difficulties knowing where to direct their eyes to obtain relevant clues and solve the problem.Such information could be relevant for teachers that want to identify how students approach math problems and offer specific instructions.Thus, sacades and eye fixations have been demonstrated to be promising measurements of how students try to solve mathematical problems.

Heatmap and scanpath as an attention map
It is possible to present eye movement data, such as fixations and saccades, as visual representation through heatmaps and scanpaths (Holmqvist et al. 2011).The heatmap indicates the dwell time in the general distribution of eye movements through a colour gradient (Duchowski et al. 2012).Usually, warm colours (red and yellow) indicate the regions with extended dwell time, while cold colours (blue, purple or green) represent the AOIs with less dwell time (Figure 3).It is a method for quick identification of the regions that attract the most attention of the observer (Duchowski et al. 2012).It is possible to compare the Heatmaps of different individuals or even groups of people to analyse the behaviour of specific populations that differ when faced with the same stimulus (Holmqvist et al. 2011).
On the other hand, the scanpaths are a visual representation of the regions where the subject's eyes were directed   http://www.sajce.co.zaOpen Access during a presented stimulus.This gaze map is associated with fixations and saccades, as the circles represent the gaze, the circles' diameter indicates the dwell time, the numbers in the centres of the circles are the fixation count and the lines between the circles are the saccades (Holmqvist et al. 2011).Such an approach can reveal visual attention or how the observer search for some object or information of interest during a task (Bojko 2013;Poole & Ball 2006).
Several studies applied the scanpath to compare the differences and similarities of attention maps of experts and novices (Heminghous & Duchowski 2006;Jarodzka et al. 2010).Jarodzka et al. (2010) reported that the gaze patterns were more heterogeneous between experts than novices during the arithmetic task of fish locomotion animations.Moreover, both groups presented a similar scanpath when the task was easier with less required mental processing.Besides this approach, scanpath has been used to access memory (Hannula et al. 2010)

Pupillometry studies and mental activity
In addition to the extrinsic movements of the eyeball, there is also a variation in the diameter of the pupils that allows more or less light to enter, which ends up being a relevant modulation for vision to occur correctly.Three main factors influence an individual's variation in pupil diameter: (1) the accommodation that occurs because of the focusing process, (2) the pupil light reflex (PLR) and (3) emotional factors that involve attention and engagement (Kardon 2005).Engagement in specific tasks may be associated with the release of norepinephrine.This specific signaling chemical transmits signals across the nervous system.One of the primary brain nuclei involved with the release of this neurotransmitter in the brain region is locus coeruleus.In this sense, the task-evoked pupillary response was and can be used as a surrogate measure for arousal and mental effort (Aston-Jones & Cohen 2005; Bradley et al. 2008;Mathôt 2018).
Pupil dilation is a suitable measure to evaluate the individual's strategy for solving mathematical problems.Hess and Polt's study (1964) showed that when participants try to solve multiplication problems mentally, their pupil diameter gradually increases as the arithmetic challenge becomes more difficult.This finding suggests that pupil dilation can indicate mental activity (i.e., cognitive processing) and has been replicated in mental arithmetic multiplication (Klingner, Tversky & Hanrahan 2011;Schaefer et al. 1968) and addition problems (Jainta & Baccino 2010).Ahern and Beatty (1979) showed that the relationship between math performance and pupil dilation varies with intelligence.Individuals with better scores on intelligence tests had less pupil dilation during the mental calculation test and performed better than individuals with lower intelligence test scores at all task difficulty levels.Thus, the authors concluded that better-performing individuals need less mental effort to do the calculations because they can process information more efficiently.
On the other hand, a more recent study showed the opposite.Individuals with high performance had more significant pupil dilation during reasoning tests (Van der Meer et al. 2010).The difference in the results might be explained by the nature of the tasks used in the studies.In the first study (Ahern & Beatty 1979), the authors investigated mental calculations, which required a skill already learned and well consolidated by high-performing individuals.On the other hand, the reasoning test proposed in the second study was unpredictable for the high-performing individuals, so it required more cognitive effort to complete.Thus, pupil dilation can be an indicator of mental effort and the difficulty level of tasks.

Naturalistic research in schools
Traditionally, researchers have applied eye-tracking technology in laboratory environments with restrictions such as head fixation (Duchowski & Duchowski 2017).However, in the last decades, the device has been changing to a more portable and cost-effective system, making it possible to set up research in the classroom.Advances in camera quality are allowing eyetracking research in mobile devices, such as webcams (Papoutsaki, Laskey & Huang 2017;Robal et al. 2018) or even cell phones (Chaudhuri et al. 2021;Cortina et al. 2015;Prieto et al. 2015).A recent study showed promising eye-tracking results in smartphone cameras using machine learning.This method uses data to automate computer programs to learn and improve without human intervention, to improve the method's accuracy.Interestingly, the device could replicate previous research results of high-resolution devices (Valliappan et al. 2020).Such findings exemplify the perspective of new algorithms allowing eye-tracking research to be increasingly accessible with several options for studies on mobile devices for more ecological validity and massive applications.
Eye-tracking research in schools is more realistic and inviting to students (Bolden et al. 2015).The less formal nature of the school environment turns out to be more comfortable for young students that intend to participate in the experiments (Mason, Pluchino & Tornatora 2016).An eye-tracking study on children reveals the strategies used during visual representations of mathematical problems (Bolden et al. 2015).The students' gaze patterns indicated difficulties in promoting multiplications, which could help teachers identify which instructions should be addressed to students.Following these ideas, other studies investigated eye-tracking usability from the view of professionals with experience in the classroom.First, we recorded students' eye movement data during geometric problem-solving in primary school.
Then applied a questionnaire to teachers from another school to assess their expectations of the AOIs where they suppose that students would look more.The results showed that the overlap regions pointed by the teachers were different from the students' heatmaps, which indicates that they imagined other student behaviours before seeing the eye-tracking video (Figure 4) (Da Silva Soares et al. 2021).Therefore, eyetracking could be valuable for teachers to obtain real-time monitoring access to students' performance.The discussion of the possible applications in this matter is presented in the following subsection, Eye-tracking videos to promote instruction.

Eye-tracking as a complementary educational instrument
It is more beneficial to employ eye-tracking in educational research as a complementary methodology and other approaches, such as the Think-Aloud interview and quantitative survey (Holsanova 2014;Schindler & Lilienthal 2018, 2019;Shayan et al. 2017).For example, Schindler and Lilienthal ( 2019) applied a 'stimulated recall interview' with a student watching the gaze video of himself solving a geometry problem.This creative approach enables the student to verbally express what he was thinking during the task verbally.The results confirmed that the student eye gaze pattern of looking back and forth between the AOIs during the geometry task was related to different mental processes.
Combining eye movement data with complementary information made it possible to reduce the inherent ambiguity of eye movement data.Moreover, it was possible to investigate visual attention to know what students are looking at and explore why they direct their gaze to specific regions during the mathematics task.
Eye movement data are relevant to educational research but do not answer all questions about learning or performance in mathematics.It is not possible to know what the observer is thinking or to guarantee that the results reflect attention only using eye movement data.In addition, there are limitations to the methodology concerning lighting noise, head movement, contact lenses, or even observer' glasses that can interfere with data collection from the eye-tracker (Borah, 2006).In these cases, it is essential to do a good calibration before starting the experiment and to guide the participants not to move around too much during the task (O'Brien et al., 2009).
It should be mentioned that it is challenging to implement the quality control protocol of studies with infants and toddlers as they would not be instructed to avoid head movements or keep looking at the screen during the calibration process.Despite the difficulties, there are studies with young infants and children showing that it is possible to use eye-tracking to understand some aspects of human cognition and neurodevelopment (Katus et al., 2019;Forssman et al., 2017;Guarini et al., 2021).
Eye-tracking technology alone to investigate students' reasoning, attention or engagement while performing mathematics tasks has significant limitations.For example, dissociation is when individuals may fix their eyeball in one AOI but think about something completely different (Posner 1980, Posner et al., 1980).Therefore, it is impossible to know what the observer thinks just by analysing the eye gaze.
Although the long fixation duration within an AOI might indicate that the focused region was attractive to the observer, it might also suggest that the information was confusing or problematic for the student (Andrá et al. 2015;Duchowski & Duchowski 2017).Moreover, visual representations of eye movements such as heatmaps or scanpaths are just spatial distributions of the regions where participants look and nothing else (Bojko 2009).In this matter, it is helpful to combine complementary techniques and data to analyse human behaviour.For example, the think-aloud protocol is a technique that asks participants to verbalize their thoughts during a task (concurrent think-aloud) or after a task (retrospective think-aloud).The retrospective think-aloud method seems to be preferable, as speaking during a task could muddle participants' attention and performance (Russo et al., 1989;Alhadreti et al., 2017).Therefore, such an approach has been combined with eye-tracking to investigate attention based on participant reports and eye movement data.Likewise, we believe eye-tracking and retrospective thinkaloud methods could be helpful in educational environments.
In research, eye-tracking has also been used with other portable devices such as EEG (Rozado & Dunser, 2015;Pavlov et al., 2022) and fNIRS (Da Silva Soares et al., 2022;Yeung et al., 2021).This multimodal approach could enable more accurate and robust real-time monitoring of cognitive processes, such as attention and workload.Therefore, educational practices can benefit from multimodal studies that help elucidate students' cognitive abilities.

Eye-tracking videos to promote instruction
The eye-tracker device also allows recording a subject's eye movements and replaying the scanpath data in videos.
Several studies applied eye-tracking videos as an instructional resource (Jarodzka et al. 2013;Mason et al. 2016;Van Gog et al. 2009).For example, Mason et al. (2016)  http://www.sajce.co.zaOpen Access (EMME).This method consists of recording videos from experts in a given task and then presenting this pattern model of attentional cues to beginners or low-performing students to assist them in the learning process.The study showed that low-performing students training with EMMEs used the experts' gaze pattern as visual instructions to direct their attention, which resulted in better performance in the mathematical task.
The eye-tracking video presents eye movement data that can also be useful to identify student characteristics that are often unnoticed during mathematics tasks.In a recent study, we recorded eye-tracking videos without ID information about the students that solved the multiplechoice mathematics problems.Then, we showed the eyetracking videos to teachers from other schools and asked them to guess if the student chose the right or wrong answer based on the scanpath.Results demonstrated that teachers had great precision in their guesses just by watching the eye-tracking videos.The teachers' prior knowledge and experience helped them interpret the students' scanpath.Moreover, the eye-tracking videos gave hints of why students' eye gaze presented a particular pattern, which was helpful for teachers to identify instructions that would help the students to improve their performance during the mathematical problem-solving task.
Hopefully, the combination of eye-movement data and educational research methodology will provide a better understanding of students' mental processes and behaviour involved in learning and solving mathematical problems, especially at individual levels of instruction.

Multimedia learning, metacognition and selfdirected approach
Another current trend is to integrate technological tools to promote mathematical learning.The pandemic increased the need to integrate technology resources into the teaching process (Iannizzotto et al. 2020) 2018).For example, one research used eye movements as a metacognitive tool to develop their arithmetic skills.After three intervention sessions, the students improved their test scores (Van den Broek 2018).
Moreover, several pieces of research used eye-movement data to enhance students' self-directed learning skills with eye-tracking techniques (Biedert et al. 2010;Buscher, Dengel & Van Elst 2008;Daraghmi et al. 2015;Hannula et al. 2010;Hyrskykari 2006;McDonald & Boud 2003).Most self-directed learning studies focus on reading skills.However, Gauthier et al. ( 2020) used eye movement data as a visual cue in the e-learning environment to train mathematics/Science skills and enhance the inhibitory control of young students.The proposed system provides insights for further innovative approaches using the eye-tracker device to promote evidence-based practices and design new interventions through self-directed learning and metacognition skills that could be helpful in mathematics education.
Eye-tracking as an educational apparatus is an opportunity to evaluate students' strategies and not just how many questions they can solve.For example, a case study with a child with dyscalculia showed a scattered fixation pattern during the multi-digit number processing task, suggesting a use of a dysfunctional strategy that was completely different from what was observed with typically developing children (Van Viersen et al. 2013).Such an approach represents the idea that eye movements could be increasingly used to unveil mathematical thinking to help students, especially those with more difficulties.

Conclusion
The eye-tracking system enables several approaches toward increasingly naturalistic educational research scenarios that allow us to study problem-solving strategies, aspects of visual attention and engagement in realistic classroom situations.Eye movement data can help teachers identify performance characteristics that lead to instructions regarding how a struggling student approaches math problems.Moreover, it is possible to apply eye-tracking as a tool for metacognition development in students and teachers, as it provides performance characteristics that would be unnoticed in daily school life.Such information could offer insights and resources for self-directed learning methods.In this sense, concerning so many possibilities for educational research applications and the continuous development of technological resources of eye-tracking systems and multimedia aimed at learning, we believe that eye-tracking tends to be increasingly used in schools to benefit education.

FIGURE 1 :
FIGURE 1: Graphical abstract showing the eye-tracking functionalities that can be useful in an educational environment.
schematic representation of a math problem in Figure 2a was presented to a student.Three AOIs were specified: 'problem statement' (AOI 1, blue), 'Figure' (AOI 2, orange) and 'multiplechoice' (AOI 3, yellow).The bar graph in Figure 2b represents how long the observer fixated the eyes on the AOI 1 (3.92 s).Area of interest 2 (10.11 s) and AOI 3 (6.73s) during the task.AOI, area of interest.

FIGURE 2 :
FIGURE 2: (a) Area of interest and (b) Fixation time.

FIGURE 3 :
FIGURE 3: Heatmap of a student solving a math problem.

FIGURE 4 :
FIGURE 4: Eye-tracking video replay of a student solving a geometry problem.