Trial-by-Trial Adaptation of Movements during Mental Practice under Force Field

Human nervous system tries to minimize the effect of any external perturbing force by bringing modifications in the internal model. These modifications affect the subsequent motor commands generated by the nervous system. Adaptive compensation along with the appropriate modifications of internal model helps in reducing human movement errors. In the current study, we studied how motor imagery influences trial-to-trial learning in a robot-based adaptation task. Two groups of subjects performed reaching movements with or without motor imagery in a velocity-dependent force field. The results show that reaching movements performed with motor imagery have relatively a more focused generalization pattern and a higher learning rate in training direction.


Introduction
Mental simulation of various actions can be used as a tool for studying theoretical concepts about cognitive neuroscience. Motor imagery, a subcategory of mental simulation, is an internal reproduction of a specific motor action without any overt motor output and is widely used for improving the motor performance. In relation to it, the underlying neurological mechanisms activated by mentally rehearsing motor actions are quite similar to the ones activated during actual physical movements [1]. There is a high overlap between the active brain regions of subjects undergoing movement execution and the movement imagination [2]. It provides the idea that motor imagery might help the CNS in the learning process and can be used in conjunction with physical training to improve motor performance [2][3][4][5]. As an example, it is used for improving the performance of athletes and sports men [6]; experienced musicians have used motor imagery for improving coordination between complex spatial and timing components of a musical composition [2]. It is also used, for speeding up the recovery process of stroke patients and neurological rehabilitation [7], for motion accuracy, and for adaptation to the changing dynamics and arm kinematics [5,8].
In the current study we consider a task in which subjects make series of reaching movements in the presence of external dynamics, that is, an externally imposed force field from a mechanical robot. The force field introduces significant errors in contrast to the movements that take place in the absence of any external force field. These errors gradually fade out with practice as the nervous system adapts to the newly imposed dynamics; this recovery of performance is "motor adaptation" [9]. The force field is switched off unexpectedly in some trials during adaptation; these trails are termed as "catch trials" and they help in investigating the properties of internal model that human nervous system updates to predict and neutralize the error. The trajectories formed during the catch trials are quite similar in shape but opposite in direction to the trajectories that are observed at the sudden introduction of force field (also termed as "after effects"). This supports the notion that model-based motor commands are generated by central nervous system (CNS). There are predominantly two modes in human motor control mechanism: feedback and feedforward. During the early learning stage, internal model is evolved and learning is achieved through sensorimotor feedback mechanism. After sufficient practice, motor systems adapt with external environment and operate autonomously in feedforward mode; this is called as "late learning" [10].
Human brain formulates internal model in such a way that motor learning in one direction has a positive impact on learning in other adjacent directions. This effect gradually decreases as the difference in directions increases. The ability to apply what has been learned in one context to other contexts is termed as the "generalization" of motor learning. When generalization increases learning in some contexts, it is called as "transfer. " In some contexts, generalization diminishes learning and it is said to be causing an "interference" [11]. It shows that the model evolved by human nervous system learns beyond the boundaries of training data and its output is broadly adapted across the state space of motor commands [9]. In the contexts where generalization is detrimental, it is usually due to the large alteration in the learning problem associated with comparatively small contextual changes. This is relevant to our experiment where a large change in direction, that is, around 135 ∘ to 180 ∘ , has an associated small change in context. This is also true in general; for example, driving car in the reverse direction or counting backward is difficult as compared to normal routine.
The current work is focused on how CNS learns to control and compensate errors in imagined reaching movements and how an error experienced in one direction can affect the reaching movements in other directions, with or without motor imagery. In other words, an investigation is made to answer how mental practice affects the generalization pattern of internal learning model developed by CNS. Up to the best of our knowledge, the relation between generalization and motor imagery in reaching movements has not been studied explicitly. By motor imagery, we mean that the individual subjects imagine the subsequent movement before actually performing it (MI group). The group of subjects without any conscious intent before starting movement or has not mentally rehearsed the upcoming movement constitutes the no motor imagery group (No-MI group).
At this stage we develop our initial hypothesis as follows.
(1) The motor imagery affects the generalization function in such a way that it transfers the learning in nearby directions.
(2) The group of subjects who rehearsed the task mentally prior to their physical action will have a high learning rate in the direction of training and associated directions.
(3) The group of subjects who rehearsed the task mentally prior to the physical action will have a more focused generalization pattern with respect to the No-MI group.
The composition of the remaining paper is as follows. Section 2 describes the related work. Methods and materials are explained in Section 3. Results are outlined in Section 4. The conclusion and future work are included in Section 5.

Related Work
Mussa-Ivaldi and Bizzi studied the possible ways in which the information about force field dynamics was perceived by the CNS. Finding the movement path based on perception of force field is a complex inverse dynamics problem, and brain forms an internal model composed of motor primitives to solve this inverse problem. This internal model is updated regularly to conform with the ever-changing environmental and physical dynamics [12]. Robotic manipulandum systems are widely used to study the underlying dynamics of motor commands issued by CNS [13].
Previous studies suggest that motor imagery has a constructive effect on the human motor performance. It has been argued that the covert mental practice is a cost effective, easily accessible strategy to improve motor performance of affected body parts after stroke [14]. Gentili et al. have studied the associated question of how imagination and mental execution of physical activities can help in learning process. It is found that although subjects with physical training (without imagery) have good learning rate than the subjects undergoing mental training (without any sensorimotor feedback), yet the movement rhythms and adaptation rates were identical. Authors proposed that the internal forward model of human brain provides state estimation to improve motor performance during imagery [5].

Experimental
Setup. We considered a behavioral task for studying the effect of motor imagery on trial-by-trial motor learning. The subjects performed center out reaching movements by using a robotic manipulandum. An external force field was generated by the robotic manipulandum for desired perturbations in a plane during the movements. The subjects, then, had to adapt to the new environment. This helped in studying the adaptive capabilities of human motor system Figure 1(a).
The experimental setup shown in Figure 1(a) was the same as [8]. In this setup the Braccio di Ferro robot (see [15] for details) was used to generate the forces and record the motion paths. The plane of motion was restricted to only two dimensions for the ease of analysis. Fourteen-channel EEG was recorded using gold cup electrodes (g.EEGcap g.tec, Guger Technologies OEG, Graz, Austria). The electrodes were placed at central locations (C3, C1, Cz, C2, and C4), frontal locations (F3, Fz, and F4), parietal locations (P3, Pz, and P4), and temporal locations (T3 and T4) by adapting international 10-20 electrode placement system. Left earlobe and right earlobe were used as reference and ground, respectively. Analogue EEG signals were amplified and band-pass filtered (0.1-100 Hz) by the EEG amplifier (g.BSamp g.tec, Guger Technologies OEG, Austria). The signals were then sampled at 256 Hz (NIDAQ 6040-E) and were stored for later offline analysis. The online feedback was provided by a software application based on BCI2000 [16].  3.1.1. Subjects. Total 12 subjects participated in this experiment. Eleven of the subjects were right handed, while one left handed subject was present. Before undertaking the experiment, a screening process was performed in which EEG patterns of all subjects were analyzed. During this process, each subject was asked to rest for 3 seconds (base line) followed by imagining hand movements for 2 seconds, and a total of 96 trials were conducted in this way. Then, for each subject, we identified the spectral bandwidth and the electrode locations that correlated most with the motor imagery. The most responsive spectral bandwidth and electrode locations were then used for online feedback. We also calculated the "coefficient of determination" for each subject. It acts as a measure to determine the quality of human intention that can be inferred from the EEG signal. It is expressed as a correlation coefficient defined over a bivariate signal composed of EEG signal during motor imagery and a task condition signal that consists of EEG signal during rest period: where 2 value was calculated from each electrode. After screening, the subjects were randomly assigned to two experimental protocols as folows: "with imagery" (6 subjects, 1 M and 5 F, mean age 23±1.5 years) and "no imagery" (6 subjects, 3 M and 3 F, mean age 25 ± 2.8 years).

Experimental
Procedures. The subjects sat on a chair in front of the manipulandum. The height and position of the seat were adjusted so that the arm could be kept horizontally at shoulder level pointing towards the center of the work space. In normal position, the elbow and the shoulder joints were flexed about 90 ∘ and 45 ∘ , respectively. The experimental protocol was displayed to the subjects on a 19 LCD computer screen placed about 1 m away at eye level. The subjects performed 10 cm reaching movements with dominant hand. The targets were displayed on a black background as white circles of 1 cm diameter appeared at one of the eight random locations (0 ∘ , 45 ∘ , 90 ∘ , 135 ∘ , 180 ∘ , 225 ∘ , 270 ∘ , and 315 ∘ ). The current position of the hand along with target was continuously displayed on the computer screen.
The experiment was organized into sets; each set consisted of a sequence of 48 target presentations, with target appeared at 8 different positions, 6 times each. Every set lasted for approximately 7 ∼ 8 minutes, and the subjects were allowed to take rest between sets. Each movement started from the center of the work space. In order to initiate a movement, the subject had to hold the cue at the starting point (initial position of the target). Once the cue is in center of the target, the target changed its color to gray; after 4 s it shifted to one of the eight random outer positions and turn into red. At this point, the "imagery" group subjects were required to "imagine" the hand movement toward the target. EEG signals were continuously recorded and after every 300 ms a spectral estimate in the most responsive frequency band was calculated. This value was compared with the threshold value to detect the presence/absence of eventrelated EEG desynchronisation (ERD). The binary signal was transmitted to the robot and used for changing the color of the target, that is, red to yellow to green. A "go" signal is then generated (target color turning into green), indicating that the actual movement could start. This signal can only be generated if either of the following conditions was fulfilled: (1) the subject successfully generated 5 ERDs or (2) the 3 sec time limit of waiting was reached.
In the "no imagery" experiments, only condition (2) was applied and the subjects had to wait for 1.5 to 3 sec randomly between target appearance and the "go" signal. On "go" signal, the subjects were required to move as fast and as accurate as possible. Subjects were encouraged to keep an approximately constant movement timings and to avoid eye blinking and head movements or throat clearing during the imagery and movement phase. The next trial started as soon the subject placed the cursor inside the target at the central initial position.
Movements were performed under three different conditions: (i) null field (robot generated no force, 5 target sets); (ii) force field (velocity dependent force field was turned on, 5 target sets); (iii) after-effect (no field again, 2 target sets). During force field trials, the robot generated a viscous curl field that perturbed the reaching movements. The force field was perpendicular to the instantaneous hand velocity vector with magnitude proportional to the velocity where, where the viscous coefficient is 12 N ⋅ m −1 ⋅ s −1 . The hand velocity vector (and its subsequent derivatives) was estimated online by means of a numerical differentiation technique. During the field sets, "catch trials" were inserted in which the force field was unexpectedly turned off. The probability of occurrence of one catch trial was set to 1/6, which corresponds to one catch trial per direction per set.

Data Analysis
3.3.1. Screening. During screening phase, the recorded EEG data was arranged into 1 s long epochs and mean was removed. A 20th-order autoregressive model was used for estimating the power spectral density. The spectrum was calculated from 0 Hz to 40 Hz at every 0.2 Hz, and then spectral average was made into 2 Hz bins for 96 hand imagery trials and compared them with the rest period. The averaged spectral change (spectra at rest condition minus spectra during imagery) was also estimated during the screening process. Screening gave an overview about the most responsive electrode and the maximum change in the ERD. This information was the basis of online feedback.

Online
Feedback. EEG signals were recorded in 300 ms blocks, and for each block the software application estimated the power spectral density. The online ERD detection threshold was set at the 80% of the averaged spectral change from the base measurement during the rest period. Thus, for each subject the threshold was different and it was 80% of the maximum spectral change he/she could produce. As a result, a binary signal, that is, 1 (presence of a ERD) or a 0 (no ERD) was generated after every 300 ms and was used to change the color of the target from red to yellow to green.

Familiarization Session.
For each subject, we tested the error measurements for normal distribution using Shapiro-Wilk, Kolmogorov-Smirnov, and Lilliefors tests. It turned out

Adaptation Session.
Hand trajectories were sampled at 100 Hz. The and components were smoothed with a 6thorder Savitzky-Golay filter (window size 270 ms, equivalent cut-off frequency of around 7 Hz). The first three-time derivative was estimated for the following indicators of motor performance Aiming Error. Aiming error provides angular difference between the required target direction and the actual hand movement direction in the early phase of the movement, that is, 300 ms from movement onset. This error provides information about the lateral deviation and is used as a general measure of curvature.
Learning Index. The learning process was quantified by using an indicator similar to that proposed by [17]. This measure is independent of the magnitude of force field and other userspecific parameters such as the net compliance of the arm: where and are the 300 ms aiming errors in the field and catch trials, respectively. Both error measures were adjusted for any bias present in the last null field set. Therefore, errors were always referred to change from errors in the null set.

State Space Modeling.
Internal model developed by brain is composed of a set of primitives that translate desired movement trajectories into required motor commands. In an event of external perturbation, motor commands are issued to minimize its effects. The forces produced as a result can be expressed in terms of desired position and velocity primitive functions [18]: is the experience dependent weighted matrix which is adjusted according to The shape of the primitives in the above equations can be found out by fitting a linear state space model over experimental data. Such a fit is possible as explained in [17,19]. Although, various types of models can be used for dynamic system modeling, we used prediction error estimate method (PEM) to identify a structured linear state space model. PEM algorithm is quite similar to maximum likelihood estimation used in time series analysis [20].
Let us suppose that we have eight dimensional input force field signal denoted by ( ), which triggers maximum speed path error of ( ) with sample number . If we have data upto Nth sample a set of input output pairs can be defined as, Here, the input ( ) is dependent on the trial which may be a force field trial (FF), simple null field trial (NF) or null field catch trial (C) after the removal of force field; ∈ {C, NF, FF} .
Linear state space model can be represented as a predictor model that estimates ( + 1)th output sample: Using an iterative procedure, an estimate of parameter vector +1 is generated for ( + 1)th output.̂+ 1 depends both on samples from 1 ⋅ ⋅ ⋅ and parameter vector . represents the parametrization, and F(⋅) is the function defined on observed data [20]: where is a shift operator and and are the linear time or shift invariant filters which we will specify in a further discussion. State space equations are given bẏ The linear state space model is estimated based on the assumption that the data has been generated according to (11). PEM tries to minimize a weighted norm of estimation error. In our case, where there is only one output, this cost function (⋅) is given by wherê( | ) is the output estimate of model, and PEM produces an output which is optimal in least squares sense. is the number of data values of errors during handreaching experiments. In the cost function estimated output is supposed to be:̂( where ( ) is the input of the model. Here, ( , ) and ( , ) are the matrices that can be described in terms of state space matrices. In turn, they define filters as follows: PEM is a fast algorithm and has similar merits as that of maximum likelihood estimation. However, it requires accurate parameterization and may get stuck in local minima. The initial parameters were estimated using a numerical algorithm for subspace state space system identification that projects both input and output data to find optimal state sequence (N4SID Algorithm by van Overschee and de Moor [21]). These sequences can be interpreted in terms of states of a parallel bank of Kalman filters. By using this interpretation, state space system matrices can be easily determined from the given data with no requirement of providing parameterization for nonzero initial conditions [21]. This algorithm uses QR decomposition and singular value decomposition. Thus, it is numerically stable and always converges to a finite value. With these benefits, we used it for finding an initial estimate of state space matrices of linear model.
In our experiments, the force field magnitude and direction (anticlockwise) were kept constant and its presence or absence was recorded using normalized integers; On each sampled input, a value of −1 indicates the presence of force field while a value of 1 indicates a catch trial or null field. Similar discrete scalar representation of force field magnitude was adopted by Thoroughman and Shadmehr [18] and Smith and Shadmehr [17]. Instead of using coordinate information in maximum errors, we used the relationship between actual arm compliance and the angular error. The details of the derivation can be found in [17]. In general, the two-dimensional compliance matrix is given by [ ] = [ 11 12 21 22 ] [ ] .
This two-dimensional compliance matrix can be transformed to one-dimensional oppositional compliance having a value in each direction of motion. The magnitude of onedimensional compliance matrix depends on direction of force and three parameters 11 , 22 , 21 , and 12 , see [17] for details. Briefly, We parameterized matrix of the state space model with the value of 1 ( ).

Measuring Goodness of Fit.
We also compared the variances of estimated output and the actual errors (see Figure 2) to account for the goodness of fit of our model. We defined our goodness of measure by [19] as follows: where 0 ( ) is a baseline model obtained by setting matrices and to zero. In (11), For our experiments, the model fit was reasonably good in subjects data (with a mean ≅ 82% and standard deviation ≅ 0.078). In comparison, Krakauer et al. in [11] do not report the error numerically, although authors state that model parameters are chosen such that the mean square error between model prediction and actual experimental data is minimized. Thoroughman and Shadmehr have reported 60% model fitness in their experiments related to human motor learning [18]. Donchin et al. have documented percentage deviation in model and actual output to be 77% [19]. Scheidt et al. report the variance accounted for (VAF) of 84% as the measure of error of their model [22]. In the nutshell, our model fit is competitive with the results reported in previous model-based studies. Figure 4 shows the group averaged ERD patterns during the online feedback. The subjects in No-MI group were waiting for the "Go" signal, while the subjects from MI group were imagining upcoming movement. Both groups showed ERDs, however, ERDs in MI group were more prominent.

Results and Discussion
From the model parameters it is found that the directional changes of equal magnitude have nearly same estimated values. For the sake of convenience we reduced the number of free parameters to 5 by averaging the parameters on same directional difference values. Let be the vector in direction for user . Thus, for each direction , we can formulate a matrix D for both MI and No-MI subjects as follows: A new matrix M can be defined over to reduce the free parameters to 5 by averaging the values on similar distance from peak learning rate. Each column M can be defined by vectors where + wraps around in an event of dimension outflow.    rate in one particular direction (the one in which training is performed). Generally, in both groups the trial-to-trial transfer of learning has decreased as the directional difference increase, but the transfer rate is higher in MI group. In case of 90 ∘ directional difference, the averaged generalization pattern shows that the mean and SD for both MI ad No-MI are not significantly different. This can be attributed to the fact that the perpendicular motion is unique and not much difficult to perform. Thus the learning transfer is less as compared to other direction. All the models were found to have a good fit and stable eigenvalues as shown in Figure 8.

Conclusion
In this study we compared the performance of two groups of subjects (MI and No-MI) in a center out-reaching movement task under a force field. The small number of subjects in both groups is a limitation of this study and suggests the need for caution in the interpretation of our results. However, this study helped us to investigate the trial to trial effect of motor imagery on learning. It turned out that our initial three hypotheses were true (see Section 1). MI group has a higher learning rate and transfer of learning as compared to No-MI group and has a more focused generalization pattern. These results show positive influence of motor imagery and suggest that motor learning can be facilitated by mentally rehearsing the upcoming movement and could be used to increase the rate of adaptation.