Understanding the use of eye-tracking recordings to measure and classify reading ability in elementary children school

The iRead EU Project has released literacy games for Spanish, German, Greek, and English for L1 and L2 acquisition. In order to understand the impact of these games on reading skills for L1 German pupils, the authors employed an eyetracking recording of pupils’ readings on a weekly basis as part of an after-school reading club. This work seeks to first understand how to interpret the eye-tracker data for such a study. Five pupils participated in the project and read short texts over the course of five weeks. The resulting data set was extensive enough to perform preliminary analysis on how to use the eye-tracking data to provide information on skill acquisition looking at pupils’ reading accuracy and speed. Given our set-up, we can show that the eye-tracker is accurate enough to measure relative reading speed between long and short vowels for selected 2-syllable words. As a result, eyetracking data can visualize three different types of beginning readers: memorizers, pattern learners, and those with reading problems.


Introduction
Adaptive literacy games provided by iRead EU Horizon Project (www.ireadproject.eu) are deployed into elementary school classrooms. We employed an eyetracker to understand how to study reading skills in order to use the eye-tracker as an evaluation method in future work. The underlying thesis is that the level of understanding orthography should have an effect on eye movement that can be measured with an eye-tracker (Behrmann & Bub, 1992;Rau, Moll, Snowling, & Landerl, 2015). Age and reading skills are correlated (Blythe & Joseph, 2011). The work presented here forms a further step in understanding how eye-trackers can be used to visualize reading skills for the key orthographic principles of the German language, namely the pervasive long and short vowel grapheme patterns. This case is for beginning readers because they are still decoding the letters. As reading skills increase, changes in the movements of the eye will reflect the decrease in cognitive processing difficulty. We may expect to observe differences between equally old children who vary significantly in reading skills. Processing difficulty is reflected in the reader's eye movement behavior (Blythe & Joseph, 2011). Häikiö, Bertram, Hyönä, and Niemi (2009) found that less able readers aged between eight and ten have slower reading speeds and a smaller perceptual span than more able readers have. Based on the literature, we expect to be able to measure minor differences in reading speed due to vowel length in 2-syllable words, such as 'Hase' vs. 'hasse' in order to determine reading accuracy.

Method
Over a period of five weeks, five pupils at the age of 8-9 attended a weekly reading club with short reading sessions that were recorded with an eye-tracker at a frequency of 120 Hz on a high-performance laptop with a large monitor with a mobile Tobii Pro X3-120 (https://www.tobiipro.com/). The eye-tracker was re-calibrated for each pupil during each visit. The laptop was positioned for optimal lighting and the seat was boosted with pillows for correct positioning of the reader. Figure 1 shows the stimulus example presented to the pupils with invisible Points Of Interest (POI). A given short story was presented across multiple pages to the reader. The following adjustments were important: POIs should not appear at the beginning of a line, and POIs were stretched higher above the text to capture most of the eye movements above the words. Font size, word and line spacing were adjusted to the eye-tracker accuracy, and the font was chosen for beginning readers. Pupils read the text out loud to avoid skipping difficult parts of the text and to control the pupil's reading speed.
The focus of the experiment was the measurement of reading speed for four distinct categories of 2-syllable words: long vowel ('Hase') and short vowel words ('hasse') and, within each of those categories, distinguishing between high frequency and low frequency words. This allows us to study whether the beginning readers are able to pronounce the words correctly by distinguishing vowel length based on orthographic patterns. The frequency dimension allows us to distinguish readers that can generalize to unknown words by applying orthographic patterns vs. those only memorizing high frequency words. Figure 1. The image shows POI marked for retrieving data from the eye-tracker Word frequency was determined through the use of a dictionary provided by Heidelberg University (Schroeder et al., 2014) with word frequency counts for different age groups. Age group for 8-10 years old children was used. The Total Fixation Duration (TFD) measurement from the eye-tracker was used for all calculations. The final data set contained four school visits for five pupils, reading nine texts (1,432 words) over a time span of five weeks as listed in Table 1. The data was cleaned as follows before normalizing for further analysis: • repetitive words were eliminated because they lead to quick memorization for all readers; • first words in lines were not used; • apparent outliers were excluded; • zero readings were excluded; and • misspelled words in the stimuli.
Normalizing the data eliminated overall reading speed in order to focus on relative speed for vowel duration, enabling furthermore direct comparison between readers.

Results
After analyzing the data, we identified three reading profiles: • Pattern recognizer: a good reader having acquired the patterns and able to distinguish between long and short vowels regardless of the frequency of the word. We can assume this reader can generalize to new words and is therefore a strong reader.
• Memorizer: a reader that has memorized frequent words, without understanding the orthographic pattern and will make reading mistakes in rare or new words. Memorizers sound like fluent readers on familiar texts like school books but may not move on easily to unknown texts.
• Problematic reader: these readers may need more time to read the short vowel because they did not understand the orthographic concept of the double consonant. By mistakenly choosing to read the short vowel as a long one, they run into a semantic conflict and this hesitation is visible in a longer time spent on reading this word. 'Sie hassen Eis' (They hate icecream) is read as *Sie hasen Eis (They bunny ice-cream).

Conclusions and future work
The path to the current understanding of how to record and analyze the data was not straightforward. This paper has shed some light on how to set up the eye-tracker correctly and select the optimal measurement, clean the resulting data appropriately, and then use it for analyzing relative reading speed for 2-syllable words. What remains to be done are additional data collections, detailed statistical analyses, and generalization to a larger number of beginning readers (Berkling & Franken, 2019). Our findings will form the basis for measuring reading improvements over time. In addition, we plan to use reading comprehension questions to validate reading profiles.