Introduction

Increasing numbers of studies use visually guided reaching movements in humans and non-human primates not only to measure visuomotor behavior but also to examine cognitive states unfolding over time. The analysis of motor output can provide unique insight into cognition because action and cognition are integrated rather than discrete processes (e.g., Song & Nakayama, 2009; Song, 2017). Traditional research in motor control has provided a strong foundation demonstrating how and why motor output can flexibly and rapidly adapt to changing external and internal conditions (e.g., Breener & Smeets, 1997; Chua & Elliott, 1993; Elliott, Binstead, & Heath, 1999; Elliott & Hansen, 2010; Glover, 2004; Scott, 2004; Todorov & Jordan, 2002; Wolpert & Flanagan, 2001). Because sensory-motor areas in the brain involved in planning for reaching movements are also involved in higher order decision-making processes, they can represent several distinct potential movement goals simultaneously (e.g., Song, Rafal, & McPeek, 2011; see Cisek & Kalaska, 2010, for review). As a result, analysis of motor output can reveal underlying cognitive processes such as those involved in making decisions or selecting one target among several competing options.

Because of these characteristics of motor output, analyses of the spatiotemporal characteristics of simple motor movements have provided a valuable tool across a variety of domains in cognitive research. These data allow researchers to track temporal changes of internal processes, going far beyond what more traditional discrete measurements such as button presses have typically provided. Furthermore, in recent years, reach movement tracking has become more accessible and affordable due to technological advances. For instance, researchers examining reach movements in recent years have uncovered new knowledge about attention (e.g., Chapman & Goodale, 2008; Jax & Rosenbaum, 2007; Kerzel & Schönhammer, 2013; Moher, Anderson, & Song, 2015; Neyedli & Welsh, 2012; Song & Nakayama, 2006; Welsh & Elliott, 2004; Welsh, Elliott, & Weeks, 1999), cognitive control (e.g., Erb, Moher, Sobel, & Song, 2016), social cognition (e.g., Freeman & Ambady, 2009; Johnson, Freeman, & Pauker, 2012), cognitive development (e.g., Erb, Moher, Sobel, & Song, 2017a; 2017b), language (e.g., Spivey, Grosjean, & Knoblich, 2005), decision making (e.g., Chapman, Gallivan, Wood, Milne, Culham, & Goodale, 2010; Moher & Song, 2014; Resulaj, Kiani, Wolpert, & Shadlen, 2009), and numerical cognition (e.g., Song & Nakayama, 2008a; Erb et al., 2018), among other topics. In some cases, patterns of behavior relating to cognitive processes even exhibit divergent patterns when goal-directed action is involved relative to simple keypress responses (e.g., Buetti & Kerzel, 2009; Moher et al., 2015; Welsh & Pratt, 2008). Thus, the field of cognitive psychology benefits greatly from an integrated approach to cognition and action.

While these studies all share an approach of examining the spatiotemporal characteristics of movements, the method of measuring movements varies across studies. Three-dimensional (3D) optical or electromagnetic tracking devices provide the most straightforward and natural method to directly measure the positions of hand or finger movements. This type of device that measures natural goal-directed reaching movements in 3D space has an advantage in that it can easily capture reach movements across various populations such as young children, aging seniors, and non-human primates. (e.g., Erb et al., 2017a, 2017b; Pohl, Winstein, & Fisher, 1996; Song, Takahashi, & McPeek, 2008; Song et al., 2011; Song & McPeek, 2010, 2015). This versatility exists because substantial training is unnecessary to use these devices. Other tracking devices are also available for measuring two-dimensional (2D) movements such as a computer-mouse (e.g., Freeman & Ambady, 2010; Hehman, Stolier, & Freeman, 2015), a stylus pen (e.g., Duncan, Chylinski, Mitchell, & Bhandari, 2017; Galantucci, 2005; Grosjean, Zwickel, & Prinz, 2009), or a tablet (e.g., Dotan & Dehaene, 2013; Pinheiro-Chagas, Dotan, Piazza, & Dehaene, 2017). These approaches, compared to 3D tracking devices, provide relatively cost-effective ways to obtain data. However, additional training may be required for participants to map visual targets and appropriate action responses because these devices control a cursor on a screen in which stimulus and action spaces no longer coincide under some circumstances (Song & Nakayama, 2009). Furthermore, movements are constrained in different ways depending on the device used. For example, an observer cannot make a three-dimensional movement with a mouse, and the characteristics of a direct three-dimensional hand movement towards an object in real space are different from those of a two-dimensional indirect mouse movement towards an object on a display.

While the use of these devices is on the rise in cognitive research, relatively little attention has been paid to how movement behavior might vary as a function of the method used to generate and record movement output. Measurements of movement initiation and movement deviation are particularly popular in studies of cognitive processes; for example, almost all of the studies cited earlier in the Introduction as examples of combining cognition and action examine one or both of those metrics. Thus, it would be of particular interest to cognitive researchers to examine how the timing and trajectory of movements that are used to infer internal cognitive processes might differ depending on the method used to generate and record movements. This is particularly important because traditional motor control approaches have demonstrated how the flexibility in motor output varies as a function of the constraints placed on movements (e.g., Desmurget, Jordan, Prablanc, & Jeannerod, 1997; Desmurget, Prablanc, Jordan, & Jeannerod, 1997; Palluel-Germain, Boy, Oliaguet, & Coello, 2004).

In the present study, we compared spatio-temporal characteristics of reaching movements recorded with a 3D reach tracker, a mouse, or a stylus pen in a double-step reach task. The goal of this study was to determine what differences, if any, exist in the timing and trajectory of movements in a simple reaching task as a function of the device used to measure the movement and the constraints placed on the movement by that recording device. In other words, the present study is designed to examine how simple parameters such as movement initiation latency and movement curvature might differ across these devices and their associated movements given the same visual input and task. We used each device based on the way these techniques are typically used in cognitive research. This comparison is relevant within cognitive research in particular, as researchers rely on these parameters to make inferences about cognitive processes. Thus, if the method used to execute and record movements affects these parameters, this would be important knowledge for interpreting effects within and across cognitive studies.

In the task, participants executed a motor movement to a target (or moved a cursor to a target by controlling a mouse or stylus) on each trial. On some trials, the initial target location was displaced to one of the two possible locations after a varied delay. This task is ideal for comparing data gathered from different devices because while the initial movement is elicited by the first target selection, the second movement is triggered by the displacement as the first target disappears and the second target appears; thus, the path of movement reflects the timing and execution of a change in target selection (Becker & Jurgens, 1979; Prablanc & Martin, 1992; Soechting & Lacquaniti, 1983; Turrell, Bard, Fleury, Teasdale, & Martin, 1998; van Sonderen, Denier van der Gon, & Gielen, 1988). As a result, this task enables us to estimate internal events, such as when the new corrective target selection process begins and how efficiently and sensitively each device and accompanying movement can be used to detect the onset of such an internal event. We chose this double-step task specifically because cognitive researchers are frequently interested in trajectories, as they can be interpreted to reflect cognitive processes such as distraction (e.g., Kerzel & Schönhammer, 2013; Moher, Anderson, & Song, 2015) or changes of mind in decision making (e.g., Burk, Ingram, Franklin, Shadlen, & Wolpert, 2014; Moher & Song, 2014; Resulaj, Kiani, Wolpert, & Shadlen, 2009). By externally controlling changes in target selection in the double-step task, here we can examine in a more controlled manner how the constraints of different movement devices might impact these changes in target selection.

Methods

Participants

Ten Brown University students (five female, mean age = 20.1 years) participated in this study for course credit. All participants self-reported being right-handed with normal or corrected-to-normal vision. The protocol was approved by the Brown University Institutional Review Board.

Apparatus

As shown in Fig. 1, stimuli were presented on a back-projected plexiglass screen (1,280 × 1,024 pixels), which was placed on the table at a distance of approximately 48 cm from the seated participant. We used a ViewSonic DLP PJD6221 projector with a refresh rate of 60 Hz. Stimulus presentation and device control were conducted using custom software designed with MATLAB (Mathworks) and Psychtoolbox (Brainard, 1997). Hand position was recorded using a computer-mouse (Dell MO56uoa) or a stylus pen in 2D or an electromagnetic position and orientation recording system (Liberty, Polhemus, Clochester, VT, USA) with a sensor secured near the tip of participant’s right index finger in 3D. 3D reach movements started from a marker on the table in front of the participant, located 27 cm from the screen aligned with the midline of the participants, directed towards the screen. Participants made 2D movements with the mouse on the surface of the table or with the stylus on a transparent tablet touch screen (The Magic Touch add-on, Garland, TX, USA) located on the table. For each centimeter of mouse movement, approximately 133 pixels were traversed by the cursor, corresponding to approximately 3.5 cm of space along the display. The ratio of pixels/cm on in the display space was 37.8 pixels/cm.

Fig. 1
figure 1

A schematic of the experimental setup for the 3D reach-tracking condition. In the mouse and stylus conditions, participants performed the 2D reaching movements with the mouse on the surface of the table or with the stylus on a transparent table touch screen located on the table

Full size image

Stimuli

All stimuli appeared on a black background. The target for reaching movements was a white circle with a diameter of 30 pixels. The starting position for 2D movements was indicated by a black square measuring 30 × 30 pixels. The starting position for the hand or cursor was located at the horizontal center of the display, 256 pixels above the vertical bottom of the display. The target was presented against a black background either at the top (302 pixels from the top of the display, directly above the starting position along the vertical axis), the left (340 pixels left of center along the x axis and 450 pixels from the top of the display along the y axis), or the right position (340 pixels right of center along the x axis and 450 pixels from the top of the display along the y axis).

Procedure

Each session took place in a semi-darkened room. At the beginning of each session, each participant was asked to sequentially touch nine equally spaced points on the screen for calibration of the 3D tracker. This calibration allowed a subsequent conversion of 3D reaching data from the default centimeter unit into corresponding pixel space by affine transformation, providing a direct comparison with the other devices. All participants performed the three device conditions defined by the apparatus that they used for reaching movements: mouse, stylus, and 3D tracker. Each condition included one practice block (30 trials/block) in which participants familiarized themselves with the device, and two experimental blocks (80 trials/block). Thus, participants performed a total of nine blocks. The order of each experimental block was randomized. Each practice block for a given device occurred prior to the first experimental block involving that device. Trials were initiated only after participants moved the cursor or their index finger to the starting position. Following this, there was a waiting period of 500, 750, or 1,000 ms. In the practice block, a single stationary target was then presented at the one of the three locations (top, left, or right) on every trial. In the experimental blocks, there were an equal number of single-step trials, in which a stationary single target was then presented on the top, and double-step trials, in which an initial target at the top location was displaced to one of the two other locations (left or right) after 150- or 250-ms delay with equal probability (Fig. 2). Single- and double-step trials were randomly intermixed within a block. Participants were instructed to reach or move the cursor to the final target as rapidly and accurately as possible. Participants were also asked to keep their cursor or finger at the target location for approximately 200 ms at the end of their reach. There was a minimum of 500 ms of an inter-trial interval before the waiting period for the next trial began.

Fig. 2
figure 2

(A) An example of a single-step trial, in which the participant points or moves the cursor to a stationary target. (B) An example of a double-step trial, in which the target moves after a delay, and the participant is required to point or move the cursor the target’s new location

Full size image

Data analysis

Sampling rates were not equivalent across devices, as the 3D reach movements were sampled at approximately 240 Hz while the mouse and stylus movements were sampled at approximately 125 Hz. To approximately match consistent sampling rates across devices, all data were resampled to 240 HzFootnote 1 at uniformly spaced time intervals using an FIR filter through Matlab.Footnote 2

To directly compare the three device conditions, all device measures including speed were calculated based on position measure on the x and y axes of the screen. For reach movements, these coordinates were calculated based on affine transformations of 3D hand position calibrated to the display at the start of the experiment. For the reach condition, this meant excluding the data from the axis that connects the observer to the screen. These measurements had a one-to-one correspondence between movement in real space and movement along the screen for reach movements, as did movements along the stylus as the pixel space for the stylus was calibrated to the pixel space on the display to have the same correspondence to real space. Mouse movement translation into pixel space was based on the default system settings for mouse control, detailed in the apparatus section above. This approach was used in the present experiment because mouse movements when used by cognitive researchers are typically used according to default computer settings.

We calculated initiation latency, movement time, curvature, and peak speed. Both the onset and the offset of the movement were defined using speed thresholds of 378 pixel/s (equivalent to 10 cm/s in display space). Each individual trial was visually inspected and thresholds were adjusted to more appropriate levels for trials in which default thresholds failed to properly capture movement duration (e.g., Song & Nakayama, 2006, 2007a, 2008a).

Initiation latency was measured as the time elapsed between the onset of the target presentation and the onset of the reaching movement. Movement time was defined as the duration between movement onset and movement offset. Curvature was defined as the maximum point of deviation divided by the length of the line from the start to the end points of the movement (see, e.g., Desmurget, Jordan, Prablanc, & Jeannerod, 1997; Moher & Song, 2013; Song & Nakayama, 2006). Larger values indicate greater curvature. Peak speed was defined as the point during the movement at which speed was greatest.

To examine how quickly participants were able to respond to target displacement, we calculated the lateral velocity using a variation of the extrapolation methodFootnote 3 (Veerman, Brenner, & Smeets, 2008). This method calculates the velocity along the axis of target displacement towards the new target location on double-step trials; in the current study, this is velocity along the x axis, time-locked to the moment at which the target was displaced. Movements to targets on the left side were flipped over the vertical meridian so all velocity could be calculated in the same horizontal direction. We calculated the re-direction latency as the estimated time at which movements were re-directed to the new location following target displacement. This estimation was derived by examining the mean lateral velocity for each individual participant in each experimental condition, and subsequently identifying the points at which the hand reached 25% and 75% of peak mean lateral velocity in that condition. Then, a line was created going through these two points. The point at which this line intersected the x axis, which estimates the point at which lateral velocity would be equivalent to zero, was inferred as the latency of lateral velocity. This calculation was done for each participant separately for lateral velocity profiles calculated independently for each device and double-step delay condition.

In addition to calculating the re-direction latency, we also calculated the slope of the lateral velocity curve. The slope was defined as the slope of the line described in the preceding paragraph connecting the points where lateral velocity reached 25% and 75% of peak velocity. These slopes may reflect variation in movement initiation on a trial-to-trial basis, as more variation would spread out the lateral velocity curve and thus produce shallower slopes. However, slopes could also be affected by acceleration, as faster acceleration would produce steeper velocity curves. Thus, to better interpret slope differences across conditions, we did a secondary analysis in which we re-calculated slopes with each trial’s peak velocity matched to the same point in time. This allowed us to cancel out trial-to-trial variation across devices in the latency to reach peak velocity and isolate potential differences in acceleration. For more details on the method, see Veerman et al. (2008).

The raw data are publicly available at https://osf.io/srmhv/.

Results

Measures across all trial types are included in Table 1. We conducted a 3 × 3 within-subjects ANOVA with factors of device (reach, mouse, stylus) and trial type (single-step, double-step short delay [150 ms], and double-step long delay [250 ms]). All p values and degrees of freedom are Greenhouse-Geiser corrected in cases where the assumption of sphericity was violated, and all post hoc comparisons were conducted using Tukey’s critical values.

Table 1 Performance by device and experimental condition
Full size table

We eliminated trials in which no full movement was recorded, recording errors occurred, or movements were initiated from a location not at the starting position (2.6% of all trials), and subsequently used a recursive trimming procedure to eliminate outliers within each experimental condition across all measures (3.7% of all trials; Van Selst & Joliceour, 1994). We conducted a one-way ANOVA on trial eliminations with device as a factor; there was no main effect of device for trials eliminated using either of these methods, ps > .05, ηp2s < .06. In the following sections, we examine differences across device conditions both in preparing the movement and in executing the movement.

Movement preparation

There was no main effect of trial type on initiation latencies, F(2,18) = 3.41, p > .05, ηp2 = .28. In other words, there was no additional preparation to move that occurred prior to a double-step trial compared to a single-step trial. This suggests that the execution of the updated movement towards the new target location occurred after the initial movement had already begun.

However, there was a main effect of device, F(2,18) = 7.97, p < .01, ηp2 = .47. Initiation latency was longer for the stylus condition (276 ms) compared to both the reach condition (239 ms) and the mouse condition (241 ms), ps < .05 (Tukey’s critical difference = 22 ms). In other words, movements were initiated more slowly with a stylus compared to either a reach or a mouse movement. The difference between the reach and mouse conditions did not reach significance, p > .05. There was no interaction between trial type and device, F(4,36) < 1, ηp2 = .08.

Movement execution

There was a main effect of trial type on movement time, F(1.1,9.9) = 230.32, p < .001, ηp2 = .96. Movement time was shortest in the single-step condition (307 ms), followed by the double-step short-delay (446 ms) and the double-step long-delay (549 ms) conditions. All three conditions differed significantly, ps < .05 (Tukey’s critical difference = 32 ms). This result is not particularly surprising, as movements that are re-directed mid-flight would necessarily take longer than direct movements. There was also a main effect of device on movement time, F(2,18) = 27.70, p < .001, ηp2 = .76. Movement time was shorter on reach trials (383 ms) compared to both mouse (450 ms) and stylus (470 ms) trials, ps < .05 (Tukey’s critical difference = 24 ms). There was no difference between mouse and stylus trials, p > .05. There was no interaction between trial type and device, F(2.6,23.5) < 1, ηp2 = .06.

There was also a main effect of trial type on curvature, F(2,18) = 263.95, p < .001, ηp2 = .97. Not surprisingly, curvature was much smaller in the single-step condition (.031) compared to the short-delay (.459) and long-delay (.649) double-step conditions; differences among all conditions were significant ps < .05 (Tukey’s critical difference = .058). In addition, there was an effect of device on curvature, F(2,18) =11.45, p < .01, ηp2 = .56 (Fig. 3). Curvature was greater in the reach condition (.426) compared to the stylus (.342) and mouse (.373) conditions, p < .05 (Tukey’s critical difference = .037). The difference between and stylus and mouse conditions did not reach significance, p > .05. In addition, there was a significant interaction for curvature, F(4,36) = 11.49, p < .01, ηp2 = .56. The interaction was largely driven by the fact that curvatures in the single-step condition were similarly low across all conditions because the movement to the target was similar to a straight line, whereas curvatures were greater in the reach condition relative to the other devices in the double-step conditions during which movements had to be re-directed (Fig. 3). This pattern was largely consistent across individual participants (Fig. 4).

Fig. 3
figure 3

For presentation purposes for this figure, each movement on double-step trials was re-sampled in space to 101 samples and averaged across participants. Movements to targets on the left were flipped over the vertical meridian. For both short-delay (A) and long-delay (B) trials, curvature was greater and a longer path was taken on 3D reach trials compared to other devices

Full size image
Fig. 4
figure 4

For presentation purposes for this figure, each movement on double-step trials was re-sampled in space to 101 samples for each individual participant, combining across delay type. Movements to targets on the left were flipped over the vertical meridian. For almost all participants, curvature was greater and a longer path was taken on 3D reach trials compared to other devices

Full size image

To further examine these curvature differences, we conducted a follow-up 2 × 3 ANOVA with factors of double-step condition (short vs. long delay) and device. We again found a main effect of device, F(2,18) = 10.71, p < .01, ηp2 = .54, with the difference between reach (.619) and other devices (mouse: .546, stylus: .499) again reaching significance, ps < .05 (Tukey’s critical difference = .055). However, there was no longer an interaction between device and condition, F(2,18) < 1, ηp2 = .05.

There was a main effect of trial type on peak speed, F(1.2,11) = 7.23, p < .05, ηp2 = .45. Peak speed was lower in the double-step short-delay condition (2,548 pixels/s) than in the double-step long-delay condition (2,631 pixels/s) and the single-step condition (2,616 pixels/s), p < .05 (Tukey’s critical difference = 64 pixels/s). There was no difference between the double-step long-delay and single-step conditions, p > .05. There was also a main effect of device on peak speed, F(2,18) =23.25, p < .001, ηp2 = .72. Peak speed was much higher in the reach condition (3,168 pixels/s) compared to both the mouse (2,374 pixels/s) and stylus (2,253 pixels/s) conditions, ps < .05 (Tukey’s critical difference = 306 pixels/s). The difference between mouse and stylus conditions was not significant, p > .05. Finally, there was an interaction between trial type and device, F(4,36) = 3.47, p < .05, ηp2 = .28. In the stylus condition, peak speed in the double-step condition was 147 pixels/s higher than in the single-step condition. Peak speed in the double-step condition was also higher compared to the single-step condition in the other devices, but the magnitude was lower in both the reach condition (38 pixels/s higher) and the stylus condition (72 pixels/s higher). Notably, these differences are quite small in magnitude relative to the overall difference between devices in peak speed. Together, these results demonstrate that three-dimensional reach movements are executed in shorter time, with more curvature, and with greater peak speed relative to mouse and stylus movements.

Response to target displacement

Finally, we examined the redirection latency for the double-step condition. We conducted a 2 × 3 ANOVA with factors of double-step condition (short vs. long delay) and device on measures of re-direction latency and slope.

There was no main effect of trial type on latency, F(1,9) < 1, ηp2 < .01. However, there was a main effect of device on latency, F(2,18) =12.92, p < .001, ηp2 = .59. Latency was shortest in the reach condition (206 ms after target displacement), followed by the mouse (221 ms) and stylus (230 ms) conditions (Fig. 5). All comparisons were significant, ps < .05 (Tukey’s critical difference = 8.24 ms). There was no interaction between trial type and device for latency, F(2,18) = 1.31, p > .05, ηp2 = .13. In other words, participants were able to re-direct their movements in response to updated target information most rapidly in the reach condition relative to other conditions.

Fig. 5
figure 5

Lateral velocity calculated along the axis of target displacement (the x axis) relative to the onset of target displacement for both the 150 ms (A) and 250 ms (B) double-step conditions. Dashed black lines reflect the estimated velocity slope calculated from points at 25% and 75% (each indicated with asterisks) of peak lateral velocity. 3D reach movements were re-directed to the new target location earlier in time relative to other devices

Full size image

There was a main effect of trial type on the estimated slope of the velocity curve, based on the points at which lateral velocity reaches 25% and 75% of peak lateral velocity, F(1,9) =8.66 p < .05, ηp2 = .49, with steeper slopes at the long delay (14.69 pixels/ms) compared to the short delay (12.67 pixels/ms). There was also a main effect of device on slopes, F(2,18) =22.61, p < .001, ηp2 = .72. Slopes were steeper for reach trials (18.31 pixels/ms) compared to mouse (14.12 pixels/ms) and stylus (8.61 pixels/ms) trials. All comparisons were significant, ps < .05 (Tukey’s critical difference = 2.48 pixels/ms). There was also an interaction between trial type and device for slopes, F(2,18) =4.61, p < .05, ηp2 = .34. Specifically, the magnitude of the difference between the short and long delay was much larger in the reach condition (3.38 pixels/cm) compared to the mouse (1.7 pixels/cm) and stylus (1.0 pixels/cm) conditions. Follow-up one-way ANOVAs with a factor of device conducted separately for both the 150-ms and 250-ms delay conditions showed that despite the interaction, the main effect of device on slope was present at both delay conditions, ps < .01.

The slope data, however, can be difficult to interpret, as changes in slope could be attributable either to differences in acceleration or differences in trial-to-trial variation in redirection latency. To correct for trial-to-trial variability, we next aligned the velocity curve for each individual trial so that the peak velocity was matched at the same point in time on all trials, in order to isolate effects of acceleration. After this matching procedure, there was still a main effect of device on slope, F(2,18) =24.35, p < .001, ηp2 = .73. Slopes were steeper for reach trials (20.85 pixels/ms) compared to mouse (13.68 pixels/ms) and stylus (8.72 pixels/ms) trials. As before, all comparisons were significant, ps < .05 (Tukey’s critical difference = 3.00 pixels/ms). Thus, differences in slope due to the device used likely reflect differences in acceleration across the different devices. However, the main effect of trial type and the interaction disappeared, ps > .05, (trial type: ηp2 = .10, interaction: , ηp2 = .20), so it is likely that those particular effects were attributable to differences in variability of movement onset across the different conditions.

To sum up, in response to a displaced target, participants responded at an earlier point in time, and increased their velocity more rapidly, when using 3D reach movements compared to either mouse or stylus movements. These results suggest that the latency to shift a goal-directed action towards a new target goal is shortest when using a reach-tracker measuring 3D reach movements as opposed to a mouse or stylus that measures 2D cursor movements.

Discussion

In this study, we compared movement behaviors across three devices that are frequently used to measure motor output in order to understand internal cognitive processes. We found that initiation latencies were shorter in 3D reach movements and mouse movements compared to stylus movements. Movement times were shorter and reach curvature was greater with 3D reach movements compared to mouse and stylus movements. Finally, the time elapsed between when a target shifts to a new location and when the movement is re-directed towards that new location was shortest in the 3D reach condition. Overall, however, the general pattern of results with respect to experimental conditions was similar across devices.

There are two primary takeaway messages from these data. First, even without any particular training, people were able to complete a simple reach task across all the devices measured here. Though there were several relatively small interactions, at least in the present task, the overall conclusions about movement behavior were largely the same regardless of the measurement device and type of movement used. Therefore, it seems reasonable to use any of these devices to examine movement timing and movement trajectories in order to gain insight into cognition.

However, the second takeaway message is that the spatio-temporal characteristics of movements made across each device differed. Specifically, 3D reach movements exhibited faster movement times, higher curvature, and shorter latency to re-direct movements to a new target location compared to the other two device conditions. This suggests the possibility that this particular technique, of measuring 3D movements with a reach tracker, may be more sensitive to subtle changes in movement that reflect internal processes. The present task produced large overall movement curvature across all devices in the double-step condition, but even subtle changes in movement curvature can reveal important information about internal cognitive processes (e.g., Chapman et al., 2010; Jax & Rosenbaum, 2007; Moher et al., 2015). Furthermore, rapidly initiated movements are more likely to produce high curvature, while slowly initiated movements may produce less curvature and thus provide less insight into target competition (e.g., Song & Nakayama, 2008b). Some methods, such as fixation offsets at the time of stimulus onset, have been used in the past as a way to speed movement initiation (e.g., Song & Nakayama, 2007b). The present results suggest that 3D reaching may provide a similar benefit, and thus be a particularly valuable tool for researchers examining subtle changes in movement output. Alternatively, it may be the case that slowly developing internal processes that take longer to affect motor behavior may be better studied with mouse or stylus cursor movements, which may take longer to execute. This additional time would allow internal cognitive processes more time to unfold as they continue to operate after initial motor plans have been triggered (e.g., Resulaj et al., 2009).

Reach movements were also re-directed to a new target location more rapidly than mouse or stylus movements. In some cases, participants may shift attention to multiple targets in a short period of time, either because of external factors such as in the present task, or because of internal conflict in the target selection process. The present results suggest that 3D reach-tracking might be the ideal method in these cases, as the participant can respond more quickly to shifts in target selection with a 3D reach than with a mouse or stylus movement, and thus these shifts will be easier to detect.

Often, subtle differences in movement timing and movement trajectories are interpreted to reflect important differences in cognitive processing (e.g., Freeman & Ambady, 2009; Moher et al., 2015). The present results are particularly important for addressing potential Type II errors in which researchers fail to reject the null hypothesis (e.g., false negatives), suggesting that certain methods may be more appropriate for detecting subtle but important differences in the timing and trajectory of movements. Moving forward, we would recommend that cognitive researchers account for these factors when interpreting both significant and null results.

Previous research has established that the physical constraints of a movement can influence characteristics of the movement such as the timing and trajectory. For example, participants making a movement to a target displayed on a tabletop initiate movements in less time and exhibit increased movement curvature when those movements are unconstrained (allowed moving off the table in three dimensions) as opposed to constrained (required to stay attached to the table in two dimensions; e.g., Desmurget, Prablanc, Jordan, & Jeannerod, 1997, 1999). In addition, the role of visual feedback has been established as an important factor in motor control. Both the timing and trajectory of goal-directed movements are affected by whether participants can view their movement directly or can only view a projection of their movement (Palluel-Germain, Boy, Orliaguet, & Coello, 2004). However, in the case of controlling a cursor indirectly with a mouse or stylus as in the present study, an important distinction is that the movement is indirect in two ways. As in Palluel-Germain et al. (2004), the visual feedback is indirect as participants are not likely to be looking down at the mouse or stylus, because they must attend the screen in order to complete the task. In addition, at least in the present study (and in the way that many researchers measure mouse movements), the spatial mapping of the movements of the mouse cursor and the movements of the hand that control the mouse cursor were not equivalent. We believe this is a particularly critical point, as mouse-tracking research has increased in popularity recently due to its ease of use and the availability of free data-analysis software (e.g., Freeman & Ambady, 2010). Thus, the present study adds to the literature by providing a direct comparison of these three devices in the manner in which they are typically used in cognitive research.

We note that the differences in behavior across device conditions in the present article are very likely driven in part by different physical demands, such as greater amplitudes for 3D reaches and the differences in gravitational forces across the different types of movement output. Thus, the present results do not necessarily provide insight into aspects of motor control that are attributable specifically to the devices used. Because both the device used for measurement and the type of movement itself differed across conditions (e.g., 3D vs. 2D, direct vs. indirect), we cannot draw direct conclusions about whether differences we observed are due to the device itself or the movement constraints placed on participants due to the way the device was used. However, given that we modeled the design after the ways in which these devices are typically used in cognitive research, the differences observed here do provide insight into how and why the conclusions that cognitive researchers make about cognitive processes based on motor output might be directly affected by the methodological choices those researchers make.

Of course, there are numerous choices for devices including and beyond the ones discussed in the present paper. These devices for measuring movement output may vary along many dimensions including the movement constraints they place on participants, spatial precision, temporal precision, cost, ease of use, and familiarity to participants, to name a few. In some cases, the researcher can select some of these parameters to manipulate; for example, the ratio of mouse movement in physical space to cursor movement on the screen can be adjusted; although most computers default to a non-equivalent ratio, some researchers do use a one-to-one mapping of real space movement to cursor movement (e.g., Elliott, Welsh, Lyons, Hansen, & Wu, 2006). In addition, with mouse movements, the movement of the mouse on the screen can be perturbed relative the participant’s movement in real space in order to study processes such as visuomotor learning (e.g., Song & Bédard, 2015). There are also many choices in how to process and analyze the movement data after acquisition, including how to define the beginning and end of a movement. There are many factors including and beyond the ones listed here that a researcher should consider when designing an experiment examining cognition and action. We consider the current article as a modest first step in examining how choices in the type of movements a researcher examines and the devices used to measure those movements, two choices that are often correlated, might affect research in cognition and action.

In the present study, we compared only data conducted for a single, simple reaching task across devices. Additional data would be needed to determine how comparisons differ across devices and movements for other tasks, including ones where conflict regarding target selection is driven internally, by indecision on the part of the observer, rather than externally, by having the target move to a different location mid-trial. MacKenzie, Sellen, and Buxton (1991) previously found differences in cursor control as a function of both the device used and the type of task required (pointing vs. dragging). These results, together with our results, suggest that motor output can be affected by the device used to measure and carry out the motor movement and the specific type of movement required.

Another important factor in the current study is familiarity. Certainly, the typical person has more experience reaching to objects in the world than they do using a stylus or mouse. The constantly changing demands of technology mean that familiarity and practice across different movement control devices changes over time; for example, the dramatic increase in smartphone use over the past several years (e.g., Perrin, 2017) would suggest that touchscreen interaction familiarity and skill has likely increased as well. Still, hand movements are unlikely to go out of style anytime soon. Thus, 3D reach movements have the advantage of being a natural motor output mechanism across different populations.

An open question in examining movement output patterns across devices is whether the constraints that a particular device places on movements may change the cognitive processes involved in the behavior being measured. In addition to familiarity and expertise as discussed above, the demands of a movement can affect cognitive processes upstream. For example, changes of mind in a simple perceptual decision-making task occur less frequently when the motor demands of a change-of-mind are high (e.g., Burk, Ingram, Franklin, Shadlen, & Wolpert, 2014; Moher & Song, 2014). The trajectories in Fig. 3 would suggest that movement paths differ according to the device used and the movement it requires. Thus, the biomechanical demands of a change of mind would likely also differ depending on the device used and the movement that device requires. As a result, the choice of which device to use may directly impact the cognitive processes that are being studied.

In sum, the present results provide novel data on how different measurement devices and their associated movement constraints may reflect and possibly even affect different patterns of behavior. The overall pattern of results in the task was consistent across all three devices, suggesting that in many cases the choice of device may not materially impact the overall results. However, 3D hand movements were executed more quickly and with greater curvature relative to mouse and stylus movements, and were re-directed at a shorter latency following target displacement. These results have implications for researchers interested in studying the motor system and cognition. For example, it may be important to consider the size of the effect under study and the demands of the motor output involved when choosing which device to use and which type of movement to require when studying cognitive processes. Finally, designers of human-computer interaction systems may benefit from a fuller understanding of how different means of measuring and constraining motor output may reflect and impact user behavior and decision-making. The present results contribute to understanding one part of the overall picture, but more research is needed to better understand how human motor behavior and cognitive processes can be measured and are impacted by the ways in which motor output is executed.