The Simultaneous Assessment of Time and Motion Response during Dual Tasks

Measurement of reaction time in clinical settings is generally employed to assess cognitive abilities by having a subject perform standard simple tests. In this study, a new method of measuring response time (RT) was developed using a system composed of LEDs that emit light stimuli and are equipped with proximity sensors. The RT is measured as the time taken by the subject to turn off the LED target by moving the hand towards the sensor. Through an optoelectronic passive marker system, the associated motion response is assessed. Two tasks of 10 stimuli each were defined: simple reaction time and recognition reaction time tasks. To validate the method implemented to measure RTs, the reproducibility and repeatability of the measurements were estimated, and, to test the method’s applicability, a pilot study was conducted on 10 healthy subjects (6 females and 4 males, age = 25 ± 2 years), reporting, as expected, that the response time was affected by the task’s difficulty. Unlike commonly used tests, the developed method has proven to be adequate for the simultaneous evaluation of the response in terms of time and motion. Furthermore, thanks to the playful nature of the tests, this method could also be used for clinical and pediatric applications to measure the impact of motor and cognitive impairments on RT.


Introduction
The ability to elicit a goal-directed motor response to visual stimuli is a key aspect of our daily life activities: an individual with a short reaction time will be able to respond quickly to surrounding events. However, it could be compromised in subjects with motor and/or cognitive impairments. Specifically, the time taken to perform a task could be affected by different conditions, i.e., old age or the presence of pathologies [1][2][3][4]. Specifically in the pediatric population, different studies have investigated the impact of deficits in cognitive abilities. In children with Down syndrome (DS), motor and cognitive development is delayed with respect to typically developed ones [5]. In the case of autistic spectrum disorders (ASD), there may be difficulties in shifting visual attention [6,7]. As regards hemiplegic cerebral palsy (HCP), the critical role of vision for efficient planning and performing tasks was demonstrated by Surkar et al. [8], highlighting the need for therapeutic interventions focused on improving visuomotor coordination to enhance motor performance.
In clinical settings, clinical scales or standard simple tests are commonly used to evaluate motor and cognitive abilities [9][10][11].

Protocol Development
In this work, we developed a protocol for the simultaneous evaluation of response times and movement kinematics following a light stimulus using a motion capture system (SMART 400 DX-BTS Bioengineering, Garbagnate Milanese, Milan, Italy) composed of 6 infrared cameras (accuracy: <0.3 mm on a volume of 4 × 3 × 3 m; acquisition frequency: 100 Hz). The stimuli were emitted according to a predetermined pattern by WittySEM (Microgate©, Bolzano, Italy), a system typically used to train and measure athletes' reaction time to visual stimuli, composed of 8 semaphores, which are 5 × 7 LEDs matrices, wirelessly connected to a chronometer (time resolution: 4 × 10 −5 s; impulse transmission accuracy: ±0.4 ms). Each matrix is equipped with a sensor that detects the proximity of the hand when it is close to turning off the LED target.
Response time (hereinafter RT) is defined as the time between the appearance of the stimulus and the turn-off of the target, and it is measured as follows: RT =(t turn−off 2 − t turn−off 1 ) − (t turn−on 2 − t turn−off 1 ) (1) where the interval between the time of turn-off ( t turn−off ) of one stimulus and the time of turn-on ( t turn−on ) of the following one is set at 8 s ( Figure 1).
Sensors 2023, 23, x FOR PEER REVIEW 3 of 15 proposed in this pilot study, allowing us to: (i) accurately quantify the subject's response to visual stimuli, (ii) evaluate different subintervals of time in response, and (iii) quantify total-body kinematics while performing tasks of different difficulty levels.

Protocol Development
In this work, we developed a protocol for the simultaneous evaluation of response times and movement kinematics following a light stimulus using a motion capture system (SMART 400 DX-BTS Bioengineering, Garbagnate Milanese, Milan, Italy) composed of 6 infrared cameras (accuracy: <0.3 mm on a volume of 4 × 3 × 3 m; acquisition frequency: 100 Hz). The stimuli were emitted according to a predetermined pattern by WittySEM (Microgate©, Bolzano, Italy), a system typically used to train and measure athletes' reaction time to visual stimuli, composed of 8 semaphores, which are 5 × 7 LEDs matrices, wirelessly connected to a chronometer (time resolution: 4 × 10 −5 s; impulse transmission accuracy: ±0.4 ms). Each matrix is equipped with a sensor that detects the proximity of the hand when it is close to turning off the LED target.
Response time (hereinafter RT) is defined as the time between the appearance of the stimulus and the turn-off of the target, and it is measured as follows: where the interval between the time of turn-off (t turn-off ) of one stimulus and the time of turn-on (t turn-on ) of the following one is set at 8 s ( Figure 1). The semaphores are placed on a grid similar to those realized in previous studies with analogous systems, i.e., the Fitlight Trainer [17] or D2-Dynavision [19]. The subject is asked to stand in front of a grid and to perform different tasks, in which the target changes its position following a predetermined sequence unknown to the subject. The grid is composed of three poles spaced at 30 cm: three semaphores are located on each of the lateral poles and two in the middle one. To ensure the same execution conditions for all subjects, some distances have been established: the distance between the grid and the subject is equal to the upper limb length, and the position of the top and bottom semaphores is such that there is a 20° angle between the extended upper limb and the horizontal line ( Figure 2). This angle was deemed the most appropriate to reach the target without inducing difficulties. The semaphores are placed on a grid similar to those realized in previous studies with analogous systems, i.e., the Fitlight Trainer [17] or D2-Dynavision [19]. The subject is asked to stand in front of a grid and to perform different tasks, in which the target changes its position following a predetermined sequence unknown to the subject.
The grid is composed of three poles spaced at 30 cm: three semaphores are located on each of the lateral poles and two in the middle one. To ensure the same execution conditions for all subjects, some distances have been established: the distance between the grid and the subject is equal to the upper limb length, and the position of the top and bottom semaphores is such that there is a 20 • angle between the extended upper limb and the horizontal line ( Figure 2). This angle was deemed the most appropriate to reach the target without inducing difficulties. Two tasks were considered ( Figure 3): • Simple response time (SRT): for each stimulus, only the target lights up. In this case, the subject is asked to perform the task separately with the right and left upper limbs  -RRT with colour distraction (hereinafter RRT-COL), in which, for each stimulus, other semaphores of different colours are lit together, but only one, of a specific colour, is the target; -RRT with symbol distraction (hereinafter RRT-LET), in which, for each stimulus, all matrices are simultaneously illuminated with the same colour, but with different symbols, that is, letters of the alphabet, one of which is the target. Two tasks were considered ( Figure 3): • Simple response time (SRT): for each stimulus, only the target lights up. In this case, the subject is asked to perform the task separately with the right and left upper limbs (hereinafter SRT-R and SRT-L, respectively).

•
Recognition response time (RRT): for each stimulus, other LEDs light up besides the target, and the subject freely chooses the upper limb to be used. For the RRT task, two levels of difficulty are provided: -RRT with colour distraction (hereinafter RRT-COL), in which, for each stimulus, other semaphores of different colours are lit together, but only one, of a specific colour, is the target; -RRT with symbol distraction (hereinafter RRT-LET), in which, for each stimulus, all matrices are simultaneously illuminated with the same colour, but with different symbols, that is, letters of the alphabet, one of which is the target.  Each task involves 10 stimuli. The first one is considered as the test onset, so each task has 9 measured RTs. Considering that it is not necessary to touch the target but only to reach the threshold for turn-off, when the subject reached for the target, but not close enough, thus recording RTs longer than the real ones, the stimuli were classified as outliers and neglected in the analyses.
To evaluate RT and the motion response, we developed a protocol based on the motion capture of passive markers following the Davis model [21], with the addition of markers on the head (one marker on the front of the head and two on the temporal areas), upper limbs (one on the lateral epicondyle of the elbow and one on the ulnar styloid), and hands (one marker corresponding to the third carpometacarpal joint), as shown in Figure  4. To identify the semaphore position, six markers are also placed on the grid. Each task involves 10 stimuli. The first one is considered as the test onset, so each task has 9 measured RTs. Considering that it is not necessary to touch the target but only to reach the threshold for turn-off, when the subject reached for the target, but not close enough, thus recording RTs longer than the real ones, the stimuli were classified as outliers and neglected in the analyses.
To evaluate RT and the motion response, we developed a protocol based on the motion capture of passive markers following the Davis model [21], with the addition of markers on the head (one marker on the front of the head and two on the temporal areas), upper limbs (one on the lateral epicondyle of the elbow and one on the ulnar styloid), and hands (one marker corresponding to the third carpometacarpal joint), as shown in Figure 4. To identify the semaphore position, six markers are also placed on the grid.
The RT is calculated considering that the instant of turn-off recorded by the proximity sensor coincides with the instant in which the minimum distance between the target and the hand's marker was detected by the optoelectronic system. Figure 5a shows the subject approaching the hand to the target. Figure 5b shows the hand displacement along the anterior-posterior axis; peak values, i.e., when the target is turned off, are shown by a red dot.
The RT is divided into time subintervals, calculated through purposely developed MATLAB (Version: 9.13.0-R2022b) code, summarized in Figure  Bioengineering, Milan, Italy). The MR includes the movement of all body districts with sequential involvement of ( Figure 6): -Head: since the head is the first district of the body to be moved, the response time of the head (hereinafter R-HEAD) coincides with the CR and it is considered as the time between the appearance of the stimulus and the beginning of head rotation; -Trunk: the trunk moves after the head, and its response time (hereinafter R-TRUNK) is considered as the time between the appearance of the stimulus and the beginning of trunk rotation; -Hand: the hand is used to turn off the target, so it moves last. The hand's response time (hereinafter R-HAND) is the time between the appearance of the stimulus and the beginning of hand movement. R-HAND may be considered as the real "reaction time" according to the definition commonly used in the literature i.e., the time between the appearance of the stimulus and the beginning of the effective motor act requested to complete the task. In this sense, MR could be considered as a composition of "pre-motor" response, consisting of head and trunk preparatory movements to localize the target, and "motor response" of the hand to reach the target itself.  The RT is calculated considering that the instant of turn-off recorded by the proximity sensor coincides with the instant in which the minimum distance between the target and the hand's marker was detected by the optoelectronic system. Figure 5a shows the subject approaching the hand to the target. Figure 5b shows the hand displacement along the anterior-posterior axis; peak values, i.e., when the target is turned off, are shown by a red dot.   The RT is calculated considering that the instant of turn-off recorded by the proximity sensor coincides with the instant in which the minimum distance between the target and the hand's marker was detected by the optoelectronic system. Figure 5a shows the subject approaching the hand to the target. Figure 5b shows the hand displacement along the anterior-posterior axis; peak values, i.e., when the target is turned off, are shown by a red dot. the stimulus and the beginning of hand movement. R-HAND may be considered as the real "reaction time" according to the definition commonly used in the literature i.e., the time between the appearance of the stimulus and the beginning of the effective motor act requested to complete the task. In this sense, MR could be considered as a composition of "pre-motor" response, consisting of head and trunk preparatory movements to localize the target, and "motor response" of the hand to reach the target itself. Consequently, the delay in activation of one part of the body with respect to another can be assessed. Specifically, the delays of the trunk with respect to the head (Delay1), of the hand with respect to the trunk (Delay2), and of the hand with respect to the head (Delay3) can be computed as follows: Delay 3 = R HAND − R HEAD = Delay 1 + Delay 2 (4) Figure 6. The temporal scheme of the RT's subintervals.
Consequently, the delay in activation of one part of the body with respect to another can be assessed. Specifically, the delays of the trunk with respect to the head (Delay 1 ), of the hand with respect to the trunk (Delay 2 ), and of the hand with respect to the head (Delay 3 ) can be computed as follows: Delay 3 = R HAND − R HEAD = Delay 1 +Delay 2 (4)

Protocol Application
To validate the protocol's applicability, a pilot study on a group of 10 right-handed healthy subjects (6 females and 4 males, age 25 ± 2 years, height 167.4 ± 8.5 cm) was conducted. The participants underwent the protocol, and the aforementioned temporal and kinematics parameters were calculated during the execution of the four planned tasks.
The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the Politecnico di Milano. All participants were required to sign a written informed consent form, in which the details of the experimental tests were reported.

Protocol Validation
The developed protocol measures RT considering the turn-off as the instant of the minimum distance between the target and the hand. To evaluate the accuracy of the measurements, the RTs recorded by WittySEM and collected on the digital chronometer were compared with the RTs calculated using the protocol for all subjects in each task: The value of the proximity threshold, selected through WittyManager software (Mi-crogate©, Bolzano, Italy), can be set as "close", "medium", or "distant", and indicates the approaching distance necessary to trigger the proximity sensor. In this study, "medium" distance was selected. However, the proximity sensor is influenced by the background light, and therefore the proximity threshold might vary according to the illumination condition.
For each stimulus, the turn-off distance between the hand's marker and the target position at the turn-off instant was recorded, and the following analysis has been conducted: • measurements' repeatability: to evaluate the intra-variability of the measurements, 50 turn-offs were purposely conducted by the same subject, and the mean and standard deviation were calculated in two different lighting conditions of the laboratory environment (illuminated and dark); • measurements' reproducibility: to evaluate the inter-variability of the measurements, the turn-off distance of the tasks conducted by the 10 participants was considered and the related standard deviations were compared.

Protocol Validation
The comparison between the RTs recorded by WittySEM and the RT calculated using the protocol shows that ∆RT, calculated as the difference between the mean of all RTs, is −0.002 ± 0.014 s.
The evaluation of the measurements' repeatability shows that, with the same operator, the standard deviations of the turn-off distance values are ±10 mm in dark conditions (laboratory with dark curtains, no artificial light) and ±17 mm with standard (office) illumination (no curtains, office illumination turned on). Results are summarized in Table 1 with a difference between the means of the two conditions equal to 20 mm. Table 1. Mean and standard deviation of the turn-off distance in dark and illuminated conditions for the same operator.

Lighting Condition Mean [mm] Standard Deviation [mm]
Dark 122 ±10 Illuminated 102 ±17 To evaluate the measurements' reproducibility, the mean and the standard deviations of the turn-off distance for the 10 subjects were calculated. The results are reported in Table 2.

Temporal Parameters
The temporal parameters summarized in Figure 6 were evaluated by comparing the different tasks and considering the position of the target with respect to the laterality of the hand used to turn it off. The RT values of each task are reported in Table 3 and the relative partition in CR and MR subintervals are represented in Figure 7. Table 3. Mean and standard deviation of the RTs for each task. Hereinafter, the simple response time tasks, SRT-R and SRT-L, are depicted in green and red, respectively, while the recognition response time tasks, RRT-COL and RRT-LET, are in yellow and blue, respectively.

SRT-R SRT-L RRT-COL RRT-LET
In Table 4, the RT values obtained for each task are listed, considering male and female subjects separately.   In Table 4, the RT values obtained for each task are listed, considering male and female subjects separately.  To compare the response of the right and left upper limbs, we considered separately the use of the two upper limbs during RRT-COL and RRT-LET. We compared the corresponding RTs (Figure 8) R-HEAD, R-TRUNK, and R-HAND ( Figure 9) and the delays ( Table 5)    To compare the response of the right and left upper limbs, we considered separately the use of the two upper limbs during RRT-COL and RRT-LET. We compared the corresponding RTs ( Figure 8) R-HEAD, R-TRUNK, and R-HAND ( Figure 9) and the delays ( Table 5)    To further investigate the differences between the left and right upper limbs, time parameters for SRT tasks were evaluated according to the position of the target on the grid in front of the subject and are reported in Figure 10.    To further investigate the differences between the left and right upper limbs, time parameters for SRT tasks were evaluated according to the position of the target on the grid in front of the subject and are reported in Figure 10.

Kinematics
The developed protocol allows for assessing the total-body kinematics. The results show that, for the 10 healthy participants, evaluation of the lower limbs was not necessary since the subjects did not need to move them to turn off the targets.
During the MR intervals, the ROM of elbow flexion-extension and the ROM of head and trunk rotations were calculated for stimuli involving the left and right upper limb according to the target position for each task and reported in Tables 6-8.

Kinematics
The developed protocol allows for assessing the total-body kinematics. The results show that, for the 10 healthy participants, evaluation of the lower limbs was not necessary since the subjects did not need to move them to turn off the targets.
During the MR intervals, the ROM of elbow flexion-extension and the ROM of head and trunk rotations were calculated for stimuli involving the left and right upper limb according to the target position for each task and reported in Tables 6-8.  Considering all the tasks, the mean of the elbow's ROM and trunk's ROM have been calculated and reported in Figure 11, analyzing the left and right upper limbs separately, according to the target position. Considering all the tasks, the mean of the elbow's ROM and trunk's ROM have been calculated and reported in Figure 11, analyzing the left and right upper limbs separately, according to the target position.

Discussion
This work aimed to develop a new protocol for the simultaneous evaluation of time and motion response during a dual task. The light stimuli are emitted by WittySEM (Microgate©, Bolzano, Italy), a system typically used in sports, composed of eight semaphores. The evaluation of temporal and kinematic parameters is performed through six infrared cameras while the subject is reaching the hand to turn off the target during the execution of tasks of different difficulty levels.

Protocol Validation
To validate the protocol, RTs recorded by WittySEM were compared with those measured considering the position of the hand's marker: the difference between the means was 0.002 ± 0.014 s. The value is acceptable, considering that, including all the tasks, the measured RTs ranged between 0.51 s and 2.68 s. The evaluation of repeatability, performed on 50 acquisitions purposely made in two different lighting conditions by the same subject, showed that the standard deviations of the turn-off distance are ±10 mm and ±17 mm in dark and illuminated conditions, respectively. Although the intra-variability is minimal compared with the mean thresholds recorded, it may be affected by the light sensitivity of the proximity sensors. Considering the acquisitions made for each of the 10

Discussion
This work aimed to develop a new protocol for the simultaneous evaluation of time and motion response during a dual task. The light stimuli are emitted by WittySEM (Micro-gate©, Bolzano, Italy), a system typically used in sports, composed of eight semaphores. The evaluation of temporal and kinematic parameters is performed through six infrared cameras while the subject is reaching the hand to turn off the target during the execution of tasks of different difficulty levels.

Protocol Validation
To validate the protocol, RTs recorded by WittySEM were compared with those measured considering the position of the hand's marker: the difference between the means was 0.002 ± 0.014 s. The value is acceptable, considering that, including all the tasks, the measured RTs ranged between 0.51 s and 2.68 s. The evaluation of repeatability, performed on 50 acquisitions purposely made in two different lighting conditions by the same subject, showed that the standard deviations of the turn-off distance are ±10 mm and ±17 mm in dark and illuminated conditions, respectively. Although the intra-variability is minimal compared with the mean thresholds recorded, it may be affected by the light sensitivity of the proximity sensors. Considering the acquisitions made for each of the 10 participants, standard deviations ranging from ±4 mm and ±16 mm confirmed a good intra-variability of the turn-off distance, demonstrating that the subjects stop after turning off the target without physically touching it, and this result may be attributable to the sound signal that is emitted by the system at the instant of task completion. On the other hand, the mean values ranged between ±162 mm and ±226 mm, showing lower reproducibility. In this case, the inter-variability of the measurements may be due to both the different lighting conditions on each day on which the tests were conducted and the different orientations of the hand in approaching the target for each subject.
The results have shown that the method is accurate in RT measuring and that the inter-variability found can be reduced by standardizing lighting conditions.

Protocol Application
A pilot study on a group of 10 right-handed healthy age/height-matched subjects was conducted applying the developed protocol. Temporal and kinematic parameters were measured by comparing the different tasks according to the upper limb used to turn off the target and the location of the target relative to the subject placed in front of the grid.

Temporal Parameters
The RTs were computed for each stimulus as the time taken by the subject to turn off the target. The target position changed during the task, ensuring the independence of the learning effect, also known as the "sequential effect", which could indeed affect the RT by reducing it [14,22,23].
The mean RT values obtained for each task showed that the response was affected by the task's difficulty. SRT-R and SRT-L had the same mean value of 0.73 s, which is close to the 0.77 s obtained for RRT-COL. On the other hand, RRT-LET had an RT of almost twice the previous ones. This result confirms what Darbutas et al. [24] and Donders et al. [25] demonstrated in their works: RT worsens in proportion to the increasing complexity of the task. Indeed, we found that distraction generally increases the RT, although colour is strongly helpful in discriminating the target compared with the use of different symbols. Besides, in accordance with other studies [14,[26][27][28][29], the mean values of RTs in female subjects were higher than in males, and this difference increases with the task's difficulty.
The RTs were then divided into different subintervals. The partition in CR and MR shows that the cognitive response, intended as the period of absence of movement after the appearance of the stimulus, is almost equal for the various tasks, indicating that the time required to plan the movement is independent of the type of the stimuli which, on the contrary, does affect the MR. Considering the reaction sequence of the head, trunk, and hand, it is evident that the head begins movement first, to look for the position of the target on the grid. The trunk, and consequently the hand, react with a delay to the head that depends on the task's difficulty. Specifically, in the RRT-LET tasks, these delays are longer than the corresponding ones in the RRT-COL tasks. In turn, RRT-COL delays are longer than those recorded for SRT tasks. These results confirm that in the RRT-LET task, in which there is no colour help, the "pre-motor phase" is longer because the subject rotates the head and then the trunk for a longer time in search of the target before proceeding to turn it off.
Regarding the laterality of the upper limb used, the results show that, although all the participants were right-handed, there are similar values in RT, R-HEAD, R-TRUNK, and R-HAND comparing the turn-offs done with the left and right limb for all the tasks.
Comparing SRT-R and SRT-L, concerning the target position, the right and left upper limbs have specular behaviour, as expected. The RTs are lower in the case of the middle position than in the case of the lateral position, independently of the upper limb used. On the other hand, when the target is on the lateral pole of the grid, the RTs are slightly higher (some ms) when the limb used is contralateral to the target position and vice versa if it is ipsilateral. This behaviour is the same for SRT-L and SRT-R.
The results show that, when the subject can choose the upper limb, there is a tendency to use the preferred laterality. To optimize the execution and compensate for the difficulty of the tasks, the subject preferred the right upper limb. This is demonstrated by the absence of the use of the left limb to turn off the semaphores on the right pole during RRT-LET and RRT-COL. But, despite this preference, the RTs would not be affected by laterality. These findings are independent of the task's difficulty [30].

Kinematics
For all the tasks, the total body kinematic parameters were calculated during the MR subintervals. No foot or lower limb movements were analysed as the subjects were able to reach the target with only upper body movement.
The ROM results highlighted that there is a large variability in angle values with high standard deviations. This can be associated with the different movements performed by the subjects. In addition, the vertical position of the target, not considered in this protocol, can also influence the angular values. The ROM analysis shows that there is a similar behaviour between the left and right upper limbs: as expected, the ROM is higher when the target is in a contralateral position with respect to the upper limb used to turn off the target.

Conclusions
In the present study, a protocol for the simultaneous assessment of time and motion response during different difficulty levels of dual tasks by using WittySEM and an optoelectronic system was developed. The protocol was applied and validated in an exploratory pilot study on a small healthy population. The results confirmed the dependency of the response time on the task's difficulty, while also providing information about kinematics.
With a perspective to apply the protocol in a clinical study to characterize a larger pediatric population, the proposed method overcomes the limits of the commonly adopted protocols: since the analysis of the response is based on the use of an optoelectronic system (and WittySEM is only used to generate visual stimuli), the test is extremely versatile, and results are accurate and independent from the experimental setup, allowing comparison between tests performed in different laboratories. The method may be adapted and applied with other analogous instruments generating different kinds of stimuli, and the playful nature of the tasks makes the protocol suitable for pediatric context. It also seems possible to reasonably vary the tasks to adjust the method to specific pathologies.
Another advantage is the motion analysis that provides a more comprehensive assessment of the response in terms of kinematics also. The evaluation of the total-body kinematics may be redundant on healthy subjects with this specific experimental setup, but it is expected to provide meaningful information in the case of subjects with deficits, for example in detecting compensatory motor strategies. In this sense, quantitative measures of the response may also circumvent the low sensitivity of the clinical scales in view of quantification of motor and cognitive impairments.
Despite the promising results in this pilot study, further work to test the protocol applicability to a pathological pediatric population, and an in-depth analysis to characterize the relative control group, needs to be conducted.