Mouse Tracking as a Method for Examining the Perception and Cognition of Digital Maps

: This article aims to present the authors’ perspective regarding the challenges and opportunities of mouse-tracking methodology while performing experimental research, particularly related to the map-reading process. We brieﬂy describe existing metrics, visualization techniques and software tools utilized for the qualitative and quantitative analysis of experimental mouse-movement data towards the examination of both perceptual and cognitive issues. Moreover, we concisely report indicative examples of mouse-tracking studies in the ﬁeld of cartography. The article concludes with summarizing mouse-tracking strengths/potential and limitations, compared to eye tracking. In a nutshell, mouse tracking is a straightforward method, particularly suitable for tracking real-life behaviors in interactive maps, providing the valuable opportunity for remote experimentation; even though it is not suitable for tracking the actual free-viewing behavior, it can be concurrently utilized with other state-of-the-art experimental methods.


Introduction
A map constitutes a medium designed to communicate generalized spatial information as well as existing relationships among geographic entities [1]. Cartographic products aim to transfer spatial information from mapmakers (cartographers) to the map readers/users taking also into account the third (i.e., time) dimension. Depending on the nature of the used medium, maps could be either analog or digital. Nowadays, the majority of the maps are distributed as digital products using the World Wide Web; they can be either static, animated, or interactive. In any case, the maps' design is directly connected to the implementation of visual [2], dynamic [3] and/or sound [4] variables which are utilized towards the representation of qualitative and/or quantitative differences that characterize geographical entities and/or phenomena. Compared to other types of spatial representations (e.g., satellite images, orthophotos, etc.), both the effectiveness and the efficiency of the map-reading process are directly influenced by the selections made during the cartographic design process. The effectiveness and the efficiency are connected to the accuracy and the corresponding completion time required for the execution of map-reading tasks, accordingly [5].
The examination of map perception and cognition is mainly based on the performance of experimental research studies in which cartographic products act as visual stimuli and certain research hypotheses are tested under map-reading conditions [6]. More specifically, in cartographic research it is important to rate and/or rank the performance of the design tools (i.e., visual, dynamic and sound variables) utilized towards the visualization of spatiotemporal phenomena, as well as to analyze and model the strategies followed by map readers during the execution of basic or more complex map tasks. Over the last decades, several experimental methods, including questionnaire analysis (e.g., [7]), thinkaloud protocols (e.g., [8]), reaction-time measures (e.g., [9]), eye tracking (e.g., [10]), mouse tracking (e.g., [11]), electroencephalography (EEG) (e.g., [12]), and functional Magnetic Resonance Imaging (fMRI) [13], as well as combinations among them (e.g., [14]), have been employed for the examination of perceptual and cognitive aspects related to the map-reading process. These approaches constitute well-established behavior research methods implemented in several fields related to cartography, such as psychology and neuroscience. In addition, the research outcomes produced by the performance of scientific experimentation have a direct influence on the process of cartographic design, especially for the production of modern multimedia and interactive maps distributed either as standalone applications or through the internet.
The aim of this work is to briefly present the methodology of mouse-tracking and mouse-movement analysis, as well as to highlight both the opportunities and the limitations it provides towards the examination of perceptual and cognitive issues related to the mapreading process. The higher-level goal of this article is to provide the future perspectives and potential directions in cartographic research, based on authors' viewpoints.
In Section 2, mouse tracking is presented as an experimental method providing specific metrics and visualization techniques used for the analysis of the collected experimental data. Moreover, existing software solutions are reported in the same section. Section 3 summarizes recent cartographic studies which are based on the implementation of mousetracking techniques. Section 4 provides an outline of the strengths and future potential on the discussed research field, while the associated limitations are presented in Section 5. Section 6 showcases some concluding remarks of this perspective work.

The Mouse-Tracking Methodology
Mouse-tracking methodology constitutes one of the simplest methods used towards capturing user response during the execution of typical computer tasks (i.e., a task that is performed on a graphical user interface (GUI) presented on a digital display (e.g., mouse clicking/logging on specific linear elements)). Hence, this methodology could be utilized for the examination of both perceptual and cognitive processes related to the executed tasks [15]. Indeed, over the last decades, several applications have appeared in different domains (see, e.g., the examples presented by [15]).

Basic Description of the Method
The methodology of mouse tracking involves the process of recording and analyzing computerized mouse-movement trajectories [16]. In practice, the implementation of mousetracking techniques enables the collection of spatiotemporal mouse-movement data, also including mouse pointer events (i.e., single-and double-clicking). Hence, each mousemovement data record mainly includes both horizontal and vertical coordinates of the mouse cursor position expressed in a display monitor coordinate system (usually referred to as pixel system), as well as the corresponding timestamp (usually expressed in msec). In other words, each record is mainly characterized by two spatial (x,y) and one temporal (t) value. Since the recording frequency of mouse-movement data is mainly influenced by the capabilities of the utilized system, such data can be characterized by high density. Therefore, the nature of mouse-movement data can be considered similar to other types of movement data, such as GPS trajectories and eye-tracking data. As a result, quantitative and qualitative approaches that are implemented in the aforementioned types of data could be adapted to mouse-movement data analysis and visualization (see, e.g., the studies provided by [15,17]).
The trajectories of the captured movements could reveal critical patterns regarding the strategies followed by the users. Moreover, the computation of specific metrics leads to the standardization of the analysis process, especially for the case of scientific experimentation. Besides, the computed indices are usually supported by simple or more sophisticated visualization techniques (see Section 2.2 for further details). Visualization techniques are able to qualitatively express existing patterns of the users' searching behavior.
In order to highlight the potential provided by the method during a 'real-world' scenario, a simple example in the field of cartographic research is presented in Figure 1. In this example, experimental participants are asked to detect a specific (point) symbol on a cartographic background. Using mouse-tracking and analysis software, participants' behavior could be modelled considering either specific metrics or different types of visualizations. Figure 1 involves different aspects of mouse trajectory produced by the MatMouse toolbox [15]. Digital 2023, 3, 127-136 129 techniques are able to qualitatively express existing patterns of the users' searching behavior.
In order to highlight the potential provided by the method during a 'real-world' scenario, a simple example in the field of cartographic research is presented in Figure 1. In this example, experimental participants are asked to detect a specific (point) symbol on a cartographic background. Using mouse-tracking and analysis software, participants' behavior could be modelled considering either specific metrics or different types of visualizations. Figure 1 involves different aspects of mouse trajectory produced by the Mat-Mouse toolbox [15].

Metrics and Visualization Methods
Both the quantitative and the qualitative analyses of mouse-tracking experimental data are based on the computation of specific metrics which mainly aim to describe the

Metrics and Visualization Methods
Both the quantitative and the qualitative analyses of mouse-tracking experimental data are based on the computation of specific metrics which mainly aim to describe the individual (each participant's) mouse-movement trajectories, as well as on the development of specific visualization methods that are used for highlighting either individual or aggregated (i.e., several/all participants') searching strategies. The mouse-movement metrics, visualization techniques and software tools reported in the following sections (Sections 2.2.1, 2.2.2 and 2.3) are indicative, whereas their reporting aims to reveal several opportunities that are offered towards addressing research questions connected to different aspects of visual perception and cognition.

Mouse-Movement Metrics
Mouse-movement trajectories indicate the searching strategies during the execution of a task (or a series of tasks) on a visual stimulus presented on a digital display. The principal aim behind the computation of specific metrics is to describe such strategies in a quantitative way, also providing the capability for statistical comparisons. Mousemovement metrics include the computation of specific values that characterize the mouse trajectories' deviations either temporally or geometrically [18]. More specifically, mousemovement strategies could be described based on the relative reaction times, changes in direction, as well as on the corresponding trajectory velocity and acceleration [16]. The geometry-based description of mouse-movement trajectories includes quantitative measures that are able to reveal the existing differences compared to the optimal trajectory that corresponds to a straight line [15]. The most typical metrics in this category are the Maximum Absolute Deviation (MAD), the Area Under Curve (AUC), the Maximum Deviation (MD), and the Convex Hull (CH) area [15,[19][20][21].
More precisely, such metrics provide fine-grained quantification of the conflict or uncertainty among response options [16]. For instance, MD quantifies the response between two elements of potential choice (e.g., icons in a website) by calculating the furthest point on the actual (mouse) trajectory from the optimal/ideal (straight) trajectory, while AUC calculates the area between the actual and the optimal straight trajectory [20]. Trajectories that approach the straight trajectory tend to reflect less conflict between the two options [22]. In cartographic research, the application of such metrics can significantly support the quantification of the amount of conflict for decisions regarding the elements existing in interactive cartographic products (e.g., map symbols) and/or GUIs (e.g., interactive tools). Aside from these mouse-tracking metrics, there are other, less standardized ones that quantify the temporal development of mouse trajectories (e.g., acceleration and velocity [23], or entropy analyses [24]) in order to assess the way a decision evolves [20,22].

Visualization Techniques
Visualization techniques could be used for supporting the analysis of experimental data in qualitative means. Although representing mouse trajectories as simple lines seems to be the more intuitive method for representing individual behaviors, several techniques have been reported in mouse-tracking studies over the last years. More specifically, considering the similarity between mouse-and eye-movement data in terms of their spatiotemporal nature, the corresponding techniques for data collection and analysis could share the approaches followed towards the visualization of both individual and aggregated experimental data. Hence, aside from the simple representation in horizontal plane as reported above, mouse-tracking data referring to an individual behavior during the execution of a task or a series of tasks on a visual scene could be also illustrated in both horizontal and vertical coordinates over time [15] and/or using Space-Time Cubes (STC) [25]. Furthermore, duration diagrams that depict the stationary situations (as happened with fixation events in eye-movement analysis) during searching, as well as curvature diagrams that highlight the corresponding curvature changes, could be utilized [15]. Additionally, mouse-movement data could be visualized using grayscale statistical heatmaps and isolines [15], as well as contiguous irregular cartograms [26]. These three techniques are also suitable for the visualization of aggregated behavior (i.e., cumulative outcome produced by different users' searching strategies). Among them, grayscale statistical heatmaps could also serve as statistical products expressing the possibility (per each pixel of an image) of having mouse activity during search on a visual stimulus [11].

Software Tools
The practical implementation of mouse-tracking studies requires the utilization of software tools that are capable of supporting the process of experimental design, data analysis and visualization. Table 1 includes a collection of software tools described in scientific articles and delivered to the scientific community since 2006.

Mouse-Tracking Techniques in Cartographic Research
Since in our days the majority of cartographic products are digital and usually interactive, mouse-tracking techniques have been incorporated in cartographic research in order to examine how people interact with such modern products. Moreover, existing studies highlight that the mouse-tracking method is used as an alternative or as a supplementary approach to eye-tracking techniques [8,11,31,32]. Indeed, an experimental study presented by [25] has shown that there is a connection between eye and mouse movements during the execution of map-related tasks.
Previous studies highlight that mouse-tracking and -logging techniques have been utilized for the examination of usability issues in the map-reading process [33], for the interpretation of the existing patterns and the identification of differences between novice and expert users [34]. Mouse events, in conjunction with other inputs that may reveal user behavior (e.g., eye movements), have been employed for the examination of specific GUIs or cartographic interfaces (see, e.g., the work presented by [35]), which are also characterized by interactivity [36].
Mouse movements could also serve as one of the inputs in order to develop specific indices towards the evaluation of the effectiveness of interactive cartographic products (see, e.g., the recent work presented by [37]). Furthermore, mouse tracking has also been used for the examination of preattentive attributes of vision in cartographic symbols [11].
In a nutshell, the aforementioned studies confirm that the method is well accepted by the cartographic community. As it is obvious, the total number of empirical studies which utilize mouse tracking as the main experimental method is not that high, especially after comparing it with the existing applications of eye tracking in cartography and spatial research [38][39][40][41]. However, considering the main strengths and future potential discussed in the following section, mouse tracking could serve as one of the most powerful methods in cartographic research.

Strengths and Future Potential
In order to identify the future potential of mouse-tracking techniques in cartographic research and, more specifically, in the examination of both perceptual and cognitive issues related to map reading, it is important to highlight the main advantages of the discussed method compared to others.
Mouse tracking constitutes a simple method providing quantitative data during the performance of an experimental study. Hence, the analysis of the collected data can be based on the computation of specific metrics, as well as on the implementation of specific visualization methods (as discussed in Section 2.2). The description of participants' behavior with quantitative indices allows the performance of statistical comparisons of the produced results. Therefore, the method appears to be quite suitable for the performance of cartographic experimentation since it aims to examine map-reading procedures under different conditions, including different cartographic designs, environmental conditions, as well as participants with different characteristics (e.g., age or level of expertise). In this way, the mouse-tracking method could substantially contribute towards meeting the clear need of understanding how map users behave when exploring modern interactive cartographic products [42].
The methodology could also be executed remotely through the internet (see, e.g., the technical framework recently provided by [32], as well as the experimental study described in [8]). Although this approach could have some limitations (e.g., different experimental conditions per participant, influence by network connection quality and/or connected device specifications [32]), it is considered much simpler than eye tracking in terms of implementation. This can be confirmed taking into consideration two additional (except for its suitability for remote testing) facts: it does not require any calibration process, and it can provide high-frequency spatiotemporal data using typical computers. Consequently, the method permits the ease of experimental data collection by a high overall number of participants. This advantage is quite important, especially when the examined research hypotheses are connected to several factors (see also the recent study provided by [43] where this issue is analyzed and discussed in the framework of the performance of mapuser studies).
Since mouse tracking is mainly based on mouse events and trajectories, the method is suitable for the examination of task-based processes. Especially when dealing with more complex map GUIs in which the main map display is combined with multiple accompanying data displays [35], or in cases where interaction with the map GUI plays an inextricable role for map reading (through specific 'digital activities' such as zooming, panning, etc.) [36], mouse tracking appears to provide a unique advantage in recording and further quantifying the users' mouse trajectories, as if these trajectories were executed in real-life conditions. Mouse tracking extends the data that are collected by the traditional experimental approach of recording the reaction times-an approach widely implemented in psychological research [44]-by providing spatiotemporal information related to participants' searching strategies. Examining the behavior of map users under different types of tasks could reveal critical outcomes regarding several aspects related to the map-reading process. Typical examples in cartographic research involve the examination of identification, memorability and recognition issues [45], as well as the examination of visual complexity influence [46][47][48][49].
The collection of large-scale experimental data could also result in the distribution of the corresponding datasets to the scientific community. This is a common practice in research studies related to visual attention, including cartography and geographic information science (see, e.g., the eye-tracking datasets distributed by previous research studies [45,46,50]). In a recent study [40], the twofold advantages of this approach are explained: freely distributed datasets are considered as objective ground truths for the analysis and the modeling of searching behavior; at the same time, open science is promoted by providing the collected data to the scientific community for further research.
Finally, as also can be shown by the research studies reported in Section 3, mouse tracking could be easily combined with other experimental methods (i.e., questionnaire analysis, eye tracking, EEG, and fMRI), permitting the multimodal description of participants' reactions. Moreover, the development of the existing tools in common programming language and/or frameworks could support the integrated manipulation of all stages in cartographic experimentation, including designing, analysis and visualization.

Limitations
Although mouse tracking has the potential to serve as a robust technique in cartographic experimentation, it has some limitations. These exact limitations can be particularly identified when mouse tracking is compared with eye tracking. Several studies report the correlation between eye and mouse movements (e.g., [25,[51][52][53][54][55][56][57]). However, mouse movements cannot replace the detection of the so-called 'point of regard'. Hence, even though task-based and interactive procedures can be adequately or even properly supported, mouse tracking is not that suitable in the examination of free-viewing procedures. Even in the cases in which the visual stimulus corresponds to an interactive map which requires mouse cursor panning for navigation, mouse trajectory is not necessarily connected to the salient locations of the visual stimulus. Therefore, mouse tracking cannot be utilized for generating ground truth towards modeling the unconscious reaction of map users.
Mouse tracking constitutes a computer-based method. Therefore, it can be used only for examining digital (cartographic) products or analog products that have been digitized and presented on a digital monitor. In other words, compared to other experimental methods, we are not able to implement this method in a real-world scenario which involves the execution of a typical map task using a traditional paper map, or a map printed in any other type of a physical analog medium.
The method is not quite suitable for capturing the reaction of map users during the utilization of touch-screen displays (e.g., smartphones and tablets). However, although the trajectory of map-user response is not recorded, response events referring to clicks, pan and zoom in/out could be used for higher-level analyses.
Summarizing both strengths/future potential and limitations presented in Sections 4 and 5, a comparison table is provided (Table 2). More specifically, Table 2 summarizes the strengths/potential, as well as the limitations of mouse tracking-mainly compared to eye tracking-with a special focus on experimental procedures in cartographic research.

Concluding Remarks
Capturing and analyzing mouse movements and events constitutes a valuable method towards the examination of both perceptual and cognitive issues related to digital mapreading processes. Despite the existing limitations, the method is simple and straightforward, it appears particularly suitable for tracking real-life behaviors in interactive maps, also providing the valuable opportunity for remote experimentation. At the same time, considering the relatively common spatiotemporal nature between mouse-and eye-movement raw data, similar metrics and visualization techniques can be used in order to examine the response of map users during the execution of task-based procedures on digital cartographic products. This particularity also permits the direct comparison between mouse and eye movements, while the simplicity of the method grants its concurrent utilization with other state-of-the-art experimental methods.