Pattern Recognition and Characterization of Upper Limb Neuromuscular Dynamics during Driver-Vehicle Interactions

Summary In this work, pattern recognition and characterization of the neuromuscular dynamics of driver upper limb during naturalistic driving were studied. During the human-in-the-loop experiments, two steering tasks, namely, the passive and active steering tasks, were instructed to be completed by the subjects. Furthermore, subjects manipulated the steering wheel with two distinct postures and six different hand positions. The neuromuscular dynamics of subjects' upper limb were measured using electromyogram signals, and the behavioral data, including the steering torque and steering angle, were also collected. Based on the experimental data, patterns of muscle activities during naturalistic driving were investigated. The correlations, amplitudes, and responsiveness of the electromyogram signals, as well as the smoothness and regularity of the steering torque were discussed. The results reveal the mechanisms of neuromuscular dynamics of driver upper limb and provide a theoretical foundation for the design of the future human-machine interface for automated vehicles.


INTRODUCTION
Since the modern vehicle was first invented in the nineteenth century, the modality and the quality of people's lives have been changed dramatically by road mobility (Smith, 1936). However, although automobiles have already been massively deployed for over a hundred years, the mechanisms of driver behaviors are still not fully understood. It is known that human drivers play a critical role during driving in terms of vehicle safety, ride comfort, and energy efficiency. In recent years, automated vehicles have gained increasing attention from both academia and the industrial sector (National Highway Traffic Safety Administration; 2017). Automated vehicles are considered a promising alternative to replace human-driven vehicles, with the aim of reducing casualties, improving traffic efficiency, and lowering human driver workloads (Kyriakidis et al., 2015;Bansal et al., 2016;Lv et al., 2017;Nunes et al., 2018;Xing et al., 2020). Although the capabilities in highly and even fully automated driving have continuously increased, unresolved problems still exist due to strong uncertainties and complex driver-vehicle interactions. In this context, one of the critical issues is to guarantee safe and smooth interactions between automatic functionality and manual driving (Saleh et al., 2013;Erlien et al., 2015;Eriksson and Stanton, 2017). This challenge requires an indepth understanding of driver behavior and the design of human-machine collaboration.
Although some studies about muscle characteristics during vehicle steering have been investigated since the 1970s, the functions and patterns of people's muscles during naturalistic driving, and particularly when interacting with automated driving vehicle, are still unclear, especially from the biomechanical perspective. Driver neuromuscular studies have dominated the development of the collaborative steering system design of automated driving vehicles (Hallé and Chaib-draa, 2005;Fuchs et al., 2007;Nguyen et al., 2017). The neuromuscular dynamics of an upper limb were widely applied to the development of shared steering systems, haptic take-over systems, steering assistance systems, and driving fatigue detection systems (Pick and Cole, 2008;Nash and Cole, 2016;Fan et al., 2019). The lateral driver model that used the queuing network and driver neuromuscular dynamic model significantly improved the vehicle lateral control performance with high vehicle speed (Bi et al., 2015). The co-contraction features of the muscle activity supported an optimal control strategy for the muscle reflex system to assist the steering-and pathfollowing tasks (Pick and Cole, 2006). Furthermore, a neuromuscular model considering co-activation could be represented as a combination of feedforward control and feedback control (Hoult and Cole, 2008). The The analysis of the neuromuscular dynamics of an upper limb is also critical to the development of the shared control and collaborative driving mode for automated driving vehicles (Abbink et al., 2011a(Abbink et al., , 2011b. The knowledge of the muscle activities and responses to the control signals from a machine enables the machine to understand intentions, capabilities, and stabilities with human steering behavior. Existing studies mainly focus on the modeling of lateral control systems such as the lane departure assistant and lane-keeping assistant systems by integrating neuromuscular dynamics such as co-contraction into consideration (Pick and Cole, 2007;Abbink et al., 2011aAbbink et al., , 2011b. However, it is not clear how the neuromuscular dynamics are influenced by the steering task and how the neuromuscular behaviors determine and dominate the steering input. It is essential to answer these questions and to study the mechanisms of neuromuscular dynamics during different steering tasks so that an efficient collaborative driving system can be developed for next-generation automated vehicles.
Considering these challenges, in this work, driver neuromuscular dynamics were studied with consideration of different driving postures and steering objectives. Specifically, we analyzed the different muscle responses to two different steering tasks, namely, the active steering and passive steering. The basic posture of an example participant is shown in Figure S1. The active steering required the drivers to steer the wheel independently and to follow the pre-defined target object. However, the drivers needed to hold the steering wheel steadily to overcome the disturbance steering torque from the machine actuator in the passive steering mode. Moreover, three hand positions and their influences on the neuromuscular dynamics and steering performance were designed. Experiments were conducted in a driving simulator. A total of 42 participants were involved in the experiment. The testing scenarios are shown in Figure 1. Detailed specifications of the measured signals can be found in Tables S1 and S2. In summary, the following four basic but critical aspects of this study were investigated in this study.
1) A correlation analysis between the EMG signals and the steering torque was performed with consideration of the steering directions, steering objectives, and steering postures.
2) The amplitude analysis of the EMG signals was performed to evaluate the steering effort. A larger amplitude indicates a more significant response of an EMG signal to the steering task. Besides, different steering scenarios were considered. Moreover, a comprehensive analysis that integrates the results of correlation and amplitude was conducted to explore the key muscles during naturalistic driving.
3) We studied the time delay characteristics between the EMG signals and steering toque based on the cross-correlation analysis, which contributed to the development of the steering torque prediction system based on the neuromuscular dynamics.

4)
We analyzed the impact of the steering postures on the driving performance. The driving and steering performance was characterized by the smoothness and regularity of the steering torque. The approximate entropy and sliding standard deviation tests were applied for the steering performance evaluation.

RESULTS
In this work, the patterns of the correlation between the upper limb neuromuscular dynamics of humans and the output steering torque were studied, as well as human performance during normal driving tasks. Driving experiments for active and passive steering maneuvers using both-arm and single-arm arrangements with the different hand gripping positions were conducted and analyzed. The following four aspects, namely, (1) the correlation between the EMG signals measured from the drivers' upper limbs and the steering torques, (2) the quantitative evaluation of the strength and importance of the EMG signals for different driving modes, (3) the time delay of the steering torque after the generation of the EMG signals, and (4) the performance and smoothness of human drivers during the various steering experiments were investigated. In the single-arm steering scenarios, 10 different neuromuscular signals denoted as MS1-MS10 were measured from the right upper limb. These 10 signals were the pectoralis major of clavicular portion, the deltoid anterior, the deltoid middle (lateral), the deltoid posterior, the triceps long head, the triceps lateral head exterior, the infraspinatus, the biceps, the pectoralis major, and the teres major.
For the steering experiments with the both-arm arrangement, 10 EMG signals, denoted as MB1-MB10, were detected from both the right and left upper limbs. They were the pectoralis major of the clavicular portion, the deltoid anterior, the deltoid posterior, the triceps long head, and the teres major of the right arm and the left arm. In total, there were 42 participants in the experiments. Among these participants, 20 were randomly assigned to conduct the single-arm experiments, and the rest of the 22 participants were engaged in the both-arm steering experiments. The detailed experimental results are described as follows. The values are presented in the form of the mean G SD.

Correlation between the EMG Signals and the Steering Torque
The correlation between the EMG signals and the steering torque was analyzed for the both-arm and single-arm driving experiments with respect to different hand gripping positions. The motivation for this (B) View from inside the driving simulator. Key components used in the experiment mainly include the EMG sensors, the steering wheel, and the torque and angle sensor. (C), Illustration of the six distinct hand positions. As shown in the subplots of (1), (2), and (3), the driver operates the steering wheel with both arms. The three gripping positions are the 3 o'clock position (denoted as 0300), 10:10 position (denoted as 1010), and 12 o'clock position (denoted as 1200), respectively. Also, as shown in the subplots (4), (5), and (6), the driver operates the steering wheel using the right arm only, and the left hand is held away from the steering wheel. The three gripping positions are the 3 o'clock position (denoted as 0300), 1:30 position (denoted as 0130), and 12 o'clock position (denoted as 1200), respectively. (D) EMG measurement regions on the right arm.
(E) EMG measurement regions on the right arm.
(F) The scenario of the active steering task. (G) The scenario of the passive steering task. See also Figure S1, Tables S1 and S2.
The correlation values of the MB1, MB2, MB6, and MB7 were all larger than 0.75, showing a strong correlation to the corresponding steering activities with hand positions of 3 o'clock and 10:10 o'clock. Similar patterns could be found in the 12 o'clock scenarios, while one outlier was that the MB2 on the right arm was not strongly correlated with the driving activities in both the clockwise (0.683 G 0.143) and counterclockwise (0.654 G 0.139) directions.
However, for the steering experiments with the single-arm (the right arm) arrangement, the observed patterns of the cross-correlation results were different for the three different hand positions, as shown in the bottom two rows in Figure 2. Specifically, for the scenario of active steering with the 3 o'clock hand position, six muscles, namely, the MS1, MS2, MS3, MS5, MS7, and MS10, showed strong correlations (correlation larger than 0.75) to the steering in both the clockwise and counterclockwise directions. The first three muscles showed higher correlations.

Amplitude Analysis of the EMG Signals
This section describes how the amplitude of the EMG signals for different participants was analyzed under each testing scenario. The direction-dependent amplitudes of the EMG signals were statistically analyzed, as illustrated in Figure 3. In general, based on the results, it could be found that for experiments with both single-arm and both-arm arrangements, the pectoralis major of the clavicular portion and the deltoid anterior muscles generated significantly higher responses than the other muscles in terms of the amplitude value. In addition, the amplitude statistics of the EMG signals without consideration of the steering directions are reported in Figure S2. Key results are reported as follows, and detailed statistics can be found in Tables S11-S16 in the supplemental file.
As shown in Figure 3, in the steering experiments with a single right arm in the counterclockwise direction, the amplitudes of the EMG signals were significantly larger than those in the clockwise steering experiments, for both the active and passive steering cases. Moreover, the MS1, i.e., the pectoralis major, and the MS2, the deltoid anterior, of the right arm generated significant larger amplitudes than the other EMG signals did. In contrast, for the both-arm driving experiments with both active and passive steering modes, a consistent pattern for the amplitudes of the EMG signals could be found. We observed that the pectoralis major and the deltoid anterior of the left and right arms had a different response to the steering maneuvers in terms of the correlation values. Specifically, the MB1 and MB2, i.e., the pectoralis major and the deltoid anterior of the right arm, showed significant responses to the counterclockwise steering activities with the hand positions of 3 o'clock and 10:10 o'clock for both active and passive steering. However, the corresponding muscles MB6 and MB7, i.e., the pectoralis major and the deltoid anterior on the left arm, were more relevant to the clockwise steering maneuvers. In addition, it was interesting to see that the amplitudes of MB6 and MB7 during counterclockwise steering were larger than those in the ll OPEN ACCESS iScience 23, 101541, September 25, 2020 5 iScience Article clockwise direction. This was opposite to the patterns of the MB1 and MB2 mentioned above. However, it is also interesting to see that such opposite patterns were not observed with the hand positions of 12 o'clock.
Next, the contribution ratio of each muscle to the corresponding steering activity was further analyzed based on the integration of the correlation and the amplitude of each EMG signal in each direction (for the detailed computation method, refer to the section Amplitude Analysis in the Transparent Methods). The overall contribution statistics are numerically described and visualized in Figure 4. For the both-arm experiments based on the contribution analysis of the data with both active and passive steering, patterns that were consistent with the correlation analysis could be found. It was shown that MB1, MB2, MB6, and MB7 for the left and right arms played the most important roles with the both-arm driving condition.
Although some muscles such as the MB3 and MB5 showed a strong correlation to the steering torque, their amplitudes and the overall contributions were not large enough to be viewed as important muscles. Similar See also Figure S2, Tables S11-S16. iScience Article results could be found in the single-arm steering mode. Specifically, the MS1 and MS2 were the two most important muscles in both the active and passive steering maneuvers. Moreover, according to the statistics, most of the contribution ratios of the measured EMG signals had median values within 0.05 and 0.15. Detailed statistical analysis can be found in Tables S17-S20 in the supplemental file.

Time Delay between the EMG Signals and the Steering Torque
The time delay between the neuromuscular signals and the generated steering torque signal was analyzed in this part. Specifically, the four key muscles (MB1, MB2, MB3, and MB4) for the both-arm steering mode and the two key muscles (MS1 and MS2) for the single-arm steering mode were selected for the analysis. The time delay with the maximum absolute correlation could be measured when the two signals were best aligned. A negative value for the time delay indicated that the activity of the specific muscle led to See also Tables S17-S20.

OPEN ACCESS
iScience 23, 101541, September 25, 2020 7 iScience Article the generation of the steering torque. If the time delay was positive, then the muscle signal lagged behind the steering torque due to the co-contraction mechanism of the muscles. Key results are reported as follows, and detailed statistics can be found in Tables S21-S24.
The statistical results of the time delay between the neuromuscular signals and the steering torque considering different steering tasks and hand positions are shown in Figure 5. Based on the results, for the botharm active steering cases, it could be found that different hand positions could lead to different patterns of the time delay. For the both-arm active steering scenario, the muscle signals of the MB1, MB2, MB6, and MB7 of the both-arm arrangement showed significant lags compared with the steering torque with the three hand positions. Among the four muscles, the MB2 generated the largest time delays with the three hand positions, which were À335 G 97 ms, À412 G 95 ms, and À424 G 221 ms. Considering the three hand

Smoothness Analysis of the Steering Activity
The steering smoothness, which was a key performance factor for driving, was analyzed using the steering torque signal. Two different approaches, namely, approximate entropy and the sliding standard deviation (SSD) test, were applied to evaluate the smoothness. Key results are reported as follows, and detailed statistics can be found in Tables S25-S32.
As shown in Figure 6, similar results were obtained with these two approaches. For the single-arm active steering, at the 1:30 o'clock position, the smoothest results were generated as 0.0084 G 0.0014 by the approximate entropy method and 0.0219 G 0.0027 by the SSD test approach. The worst smoothness performance was generated with the hand on the 12 o'clock position, and the results were 0.0090 G 0.0016 when using the approximate entropy method and 0.0224 G 0.0027 with the sliding SSD test. For the single-arm passive steering, the statistical results that were obtained based on the approximate entropy were similar to the regularity measurements at 3 o'clock and 1:30 o'clock positions. According to the sliding SSD test results, the 1:30 o'clock position led to a smoother control performance than the 3 o'clock position. In terms of the both-arm active steering, the two algorithms showed consistent results in that the performance at the 3 o'clock position was slightly smoother than that at the 10:10 o'clock position. The smoothness and regularity measurement using the approximate entropy method generated the results of 0.0087 G 0.0012 for the 3 o'clock position and 0.0086 G 0.0009 for the 1:30 o'clock position. Similarly, the SSD test-based approach generated the result of 0.0210 G 0.0035 for the 3 o'clock position and 0.0213 G 0.0026 for the 10:10 o'clock position. Then for both-arm passive steering, the hand position of 10:10 led to a slightly smoother performance than the 3 o'clock position. Last, it was interesting to see that for both single-arm and both-arm steering, the 12 o'clock position led to the worst performance in terms of the steering smoothness.

DISCUSSION
For the both-arm steering experiments, based on the correlation analysis, it could be found that in the active mode, the muscle correlation patterns at the 3 o'clock and 10:10 o'clock hand positions were similar. The activities of the pectoralis major of the clavicular portion and the deltoid anterior muscles, i.e., MB1, MB2, MB6, and MB7, showed strong correlations to the steering behavior in both the clockwise and counterclockwise directions. A steering direction-dependent result was observed in the passive steering experiments. Specifically, the left pectoralis major of the clavicular portion and the deltoid anterior muscles, i.e., MB6 and MB7, were strongly correlated with the active steering in the clockwise direction. The right pectoralis major and deltoid anterior muscles, i.e., MB1 and MB2, were strongly correlated with the active steering in the counterclockwise direction. The steering activities at the 12 o'clock position generated different patterns compared with the others. The right deltoid anterior muscle, i.e., MB2, did not show a strong correlation to the active steering that was shown in the 3 o'clock and 10:10 o'clock cases. Based on the both-arm experiment results, it could be found that the mus- iScience Article For the single-arm experiments, in the active steering mode, different correlation results were observed with the three different hand positions. Specifically, at the 3 o'clock position, the first three muscles showed significantly larger correlations than the others did. However, the correlation distributions became complex for the other two positions. For the 1:30 o'clock position, despite the first three muscles, the muscles MS5, MS9, and MS10 also showed strong correlations to the steering torque output. Last, for the tests with the hand position of 12 o'clock, only the MS1, MS4, and MS9 muscles showed strong correlations to the steering torque. Based on the aforementioned results mentioned, it could be concluded that single-arm active steering was very sensitive to the hand positions. Different hand positions during steering activities led to different patterns of muscle responses. The correlation distributions of the single-arm arrangement for passive steering were also direction dependent, which was similar to the patterns of the both-arm passive steering scenarios. The first three key muscles showed strong correlations to the steering in the counterclockwise direction, whereas the resultant correlations during steering in the clockwise direction with the hand positions of 3 o'clock and 1:30 o'clock were very small. For the 12 o'clock position, the top three key muscles, i.e., MS1, MS2, and MS4, also showed significantly high correlations to the steering in counterclockwise direction but low relations to the clockwise steering.
Based on the experiment results of the both-arm and single-arm steering, it could be found that the correlation patterns of the neuromuscular dynamics were consistent in different directions when performing active steering. However, the correlation patterns became direction sensitive when performing passive steering. Among all the measured muscles, according to the amplitude and contribution analysis, the iScience Article pectoralis major of the clavicular portion and the deltoid anterior were the two most important muscles for steering maneuvers. According to the amplitude analysis, the patterns were different for different hand positions when the drivers performed active steering with the single-arm arrangement. Specifically, the hand position of 3 o'clock led to the largest values in terms of amplitude, whereas at the 12 o'clock position, the amplitudes reached their lowest values. Based on this observation, when performing single-arm active steering, the 12 o'clock position was a more energy-efficient position compared with the other position.
Although the results of the passive steering for the 1:30 o'clock hand position showed an inverse trend, there was no significant difference among the three hand positions. For both-arm steering, the lowest average amplitude was found when both hands were in the 10:10 o'clock position. However, there was also no significant difference between these positions in both active and passive steering. In summary, when performing active steering, the most energy-efficient manner that led to the lowest average value of the amplitude was the type of single-arm steering with the 12 o'clock hand position. For active steering using both arms, the most energy-efficient hand position was the 10:10 o'clock position.
According to the time delay analysis, it was found that different muscles showed significantly different delays to the steering torque. In particular, during both-arm active steering, the right pectoralis major of the clavicular portion (MB1), the right deltoid anterior muscle (MB2), the left pectoralis major of the clavicular portion (MB6), and the left deltoid anterior muscle (MB7) showed consistent patterns of a significantly negative time delay to the steering torque. This indicated that the activations of these muscles caused the variation of the steering torque. For the both-arm passive steering experiments, the right deltoid anterior muscle (MB2) and the left deltoid anterior muscle (MB7) showed positive time delay to the steering torque at the 12 o'clock position. For the single-arm active steering, the right pectoralis major of the clavicular portion (MS1) and the deltoid anterior muscle (MS2) showed consistent patterns of a negative time delay to the steering torque at any hand position. Among all the testing cases, single-arm active steering with a hand position of 12 o'clock led to the largest time delay for the steering torque left after the neuromuscular signals.
In addition, based on the analysis of the smoothness of the driving performance, it was observed that when performing a single-arm active steering maneuver, for the 1:30 o'clock hand position, a smoother steering performance could be generated compared with the results at other two positions. When performing botharm active steering, a smoother performance could be achieved at the 3 o'clock position. There was no significant difference found between the 3 o'clock and the 1:30 o'clock (in the single-arm case) positions or the 10:10 o'clock position (in the both-arm case) when performing passive steering. For either the both-arm or the single-arm driving modes, the 12 o'clock position always generated the worst control performance among the three positions in terms of smoothness. Although the single-arm active steering with the hand at the 12 o'clock position could generate the lowest average value of neuromuscular amplitudes, it also led to a larger variation and unstable steering performance compared with the other two postures. Finally, although the mean values of the smooth metrics for the both-arm steering were smaller than those for the single-arm steering, there was no significant difference between the two steering modes. Considering the aforementioned observations in the smoothness analysis, to ensure smooth, stable, and efficient driving, it was recommended to perform steering control using a single arm in the 1:30 o'clock position or using both hands in the 3 o'clock and 10:10 o'clock positions.

Limitations of the Study
Two main limitations of this study are as follows. First, the driving environment and driving tasks in the study were simulated, which were not exactly the same as those in the real-world driving scenarios. The designed driving tasks for the participants may not fully reflect the naturalistic driving behaviors, especially under critical and emergency driving situations. Second, the participants involved in this study were all young male drivers, and their neuromuscular dynamics can be different from elderly and female drivers. Therefore, future works can be carried out from the following two aspects: more participants with diverse ages, gender, and driving habits can be recruited to do the test and real vehicle experiments with driver EMG signal measurement can be designed and conducted to collect real-world data for further analysis and validation.

Resources Availability Lead Contact
Further information and requests should be directed to and will be fulfilled by the Lead Contact: Chen Lv (lyuchen@ntu.edu.sg).

Materials Availability
This study did not generate new unique materials.

Data and Code Availability
The data and code that support all the findings of this study are available from the corresponding authors upon reasonable request.

METHODS
All methods can be found in the accompanying Transparent Methods supplemental file.

ACKNOWLEDGMENTS
This work was supported by JTEKT Corporation.

AUTHOR CONTRIBUTIONS
Y.X. processed and analyzed data, interpreted results, and wrote the paper. C.L. supervised the project, analyzed data, interpreted results, and led the writing of the paper. Y.Z. supervised the project, analyzed data, interpreted results, and wrote the paper. Y.L. supervised the project, designed and performed experiments, and reviewed the paper. D.C. analyzed data and wrote and reviewed the paper. S.K. designed and performed experiments and reviewed the paper.

DECLARATION OF INTERESTS
The authors declare no competing interests.  At beginning of test, the test participants must hold the steering wheel at 3 o' clock position with the right hand. The test participants' arm is slightly bent at the elbow where the forearm and upper arm form an angle about 100-110deg. And the steering wheel is raked so that a projected line along the steering axis is parallel to a line through the shoulder and wrist joint. During passive test or fixed steering wheel test, participants posture should be kept constant as the initial posture. During active steering test, participants should steer mostly in the tangential direction to the wheel. The participants must be familiar with the operation before measurement. Figure 3. The summations of ten EMG signals are first calculated for each participant. Then, the statistics of the overall amplitude for each hand position of the driving mode are determined accordingly. The upper two boxplots are statistic results for the single-arm active and passive mode, while the bottom two boxplots are statistic results for the both-arm active and passive mode, respectively. Based on the single-arm experiment, it can be found that the single-arm active steering with the 3-clock position generates the largest overall amplitude of the ten muscles (0.242±0.092 mv) while the single-arm active steering with hand on the 12-clock position shows the lowest overall amplitude (0.142±0.044 mv). Although the single-arm active steering with the 3-clock position shows the largest muscle amplitude, the corresponding passive steering mode generates the lowest overall amplitude among the three modes (0.121±0.038 mv). At this moment, the single passive steering with hand on 130clock position leads to the highest amplitude (0.136±0.043). Regarding both-arm active steering maneuver, although the test with hands on the 3-clock position generates the largest overall amplitude (0.217±0.076), there is no significant difference exists between the three positions. A similar trend can be found in the both-arm passive steering mode, where the mean amplitude of all the participants is 0.117±0.034 with hands on the 3-clock position, 0.117±0.045 with hands on the 10-10-clock position, and 0.126±0.037 with hands on the 12-clock position.      R  TLH R  TM R  PM L  DA L  DP L  TLH L  TM L  0300  Mean  -232  -207  2  -43  1  -227  -281  -110  -45  7  SD  354  319  414  413  391  347  262  378  420  320  1010  Mean  -101  -257  -28  26  -164  -65  -327  -73  191  -66  SD  419  331  450  432  345  423  338  353  403  407  1200  Mean  -12  15  -88  49  -71  -156  62  -76  -108  54  SD  410  457  370  435  404  380  429  372 432 381  Smoothness analysis of the steering performance

Experiment Design
A human-in-the-loop driving simulator, as shown in Figure 1, was used as a test rig for a range of steering experiments. The test subjects were required to sit on the driver's seat of the driving simulator, which included a driver's cab for a passenger vehicle with a steering system. The reaction torque actuator that was equipped in the simulator could generate steering torque based on the calculation results of the vehicle model in its controller. The basic posture of a test subject is shown in Figure S1. The seat and the steering wheel were adjusted to ensure that the upper limb was slightly bent at the elbow (approximately 110 between the forearm and upper arm). The line along the steering axis was approximately parallel to the line through the shoulder and wrist joints when the right arm was held at the 3 o'clock position and when the left arm was held at the 9 o'clock position. The basic posture of the test subject was approximately the same in the driving environment, and the basic posture was easy to maneuver in the experiment. Two steering tasks were designed, as shown in the subplots (g) and (f) in Figure 1, including Task A, a steering test with a single arm, i.e., the right arm, and Task B, a steering test with the both-arm arrangement. The detailed experiment description and the setup are introduced below. The neuromuscular dynamics were measured by EMG signals in mV, and the steering activities were measured by driver's steering torque in N·m and steering angle in degree, respectively. Detailed specifications of the measured signals can be found in Table S1 and Table S2.

1) Task A: Steering test with a single-arm arrangement
The purpose of Task A was to study the relationship between the EMG signals of the single arm (i.e., the right arm) and the steering torque applied to the hand wheel during naturalistic driving. Two testing scenarios, namely the passive steering and active steering, were designed. The active steering is the major and most common driver-vehicle-interaction manner for conventional and low-level automated driving vehicles. The neuromuscular dynamics of the driving during active steering reflect how drivers use their upper limb muscles to control the steering wheel and steadily control the vehicle. While, the passive steering reflects how driver response to the external disturbance torque, which can be generated by the road situation, vehicle vibration or the driver assistance system. It can also reflect how driver's intent differs from the automation's decision in automated vehicles. Therefore, it is also needed to analyze the driver passive steering maneuver to better understand the driver-automation interactions.
Passive steering with a single arm: The passive steering task was set to mimic the steering maneuver, which restrained the steering angle movement against the external disturbance torque for lane keeping or vehicle stabilization in naturalistic driving. In the experiment, the subject was asked to keep the steering wheel at the neutral position using a single arm, i.e., the right arm, when the external disturbance torque was given by the driving simulator. The external disturbance torque was given by a triangle wave with a constant frequency of 0.025 Hz and an amplitude of 5 Nm, as shown in Figure 1g. The amplitude value was selected with reference to the steering torque value in the normal driving of a vehicle equipped with a power steering system. The frequency value was chosen in a semi-static level in order to research the preliminary activities of a muscle.
Active steering with a single arm: The active steering task was set to mimic the steering maneuver for entering a corner and returning from a corner to straight line driving in naturalistic driving. In this experiment, the subject was asked to hold the steering wheel fixed with the required hand posture in the beginning, and then start to steer the hand wheel with the required angular position using their right arm. The required angular position was a sinusoidal angle input with a constant frequency of 0.25 Hz and an approximate amplitude of 60, as shown in Figure 1f.

2) Task B: Steering test with both-arm arrangement
The purpose of Task B was to study the relation between the EMG signals of both arms and the steering torque applied to the hand wheel during naturalistic driving. The passive steering and the active steering that were used in the single-arm driving test were also adopted.
Passive steering with both-arm: In this experiment, the subject was asked to keep the steering wheel at the neutral position using two arms when the external disturbance torque was given by the driving simulator. The external disturbance torque was given by a triangle wave with a constant frequency of 0.025 Hz and an amplitude of 5 Nm, as shown in Figure 1g. The amplitude value was selected with reference to the steering torque value in the normal driving of a vehicle equipped with a power steering system. The frequency value was chosen at a semi-static level in order to research the preliminary activities of the muscles.
Active steering with both-arm: In this experiment, the subject was asked to hold the steering wheel fixed with the requiring hand posture in the beginning, and then start to steer the hand wheel with the required angular position using both arms. The required angular position was a sinusoidal angle input with a constant frequency of 0.025 Hz and an amplitude of 60, as shown in Figure 1f.

3) Hand postures
Both-arm with three hand positions: In this scenario, the driver used the steering wheel with both arms. There were three gripping positions on the steering wheel, which were the 3 o'clock, 10:10 and 12 o'clock positions. The 3 o'clock position as well as the 12 o'clock position indicated that the test subjects were required to grasp the steering wheel at the locations of the above times using two hands. For the 10:10 position, the left and right hands of the subject were placed at the locations of the hour and ten past the hour on a clock, respectively. A schematic diagram of the above hand positions is illustrated in Figure 1c (1)- (3). Each test subject completed both the active and passive steering tasks for all six hand positions.
Single-arm only with three hand positions: In this scenario, the driver operated the steering wheel using the right arm only, and the left hand is held away from the steering wheel. The three gripping positions were the 3 o'clock position, 12 o'clock position, and 1:30 o'clock position, which correspond to Figure 1c (4), (5), and (6), respectively. The test subjects were required to grasp the steering wheel at the locations of the hour hand, which pointed to the above-mentioned times, in order to complete the driving tasks.

4) Muscle selection and EMG signal measurement
In order to measure a muscle electromyography (EMG) signal (Lv et al., 2018) that could reflect the muscle behavior during a steering maneuver, it was necessary to determine the key muscles that were involved in the steering maneuver to generate steering torque. The study in (Liu et al., 2017) has indicated some key muscles for this, including the anterior deltoid, pectoralis clavicular, pectoralis sternal, posterior deltoid, middle deltoid, and triceps long head. It has also been indicated that the muscles of the shoulder are likely to be important for generating steering torque. These results were used here. Furthermore, not only the extension and flexion of the upper arm that were around the shoulder joint but also abduction, adduction, supination, and pronation were included based on kinesiology. Thus, addition to the muscles described above, the biceps, teres major, and infraspinatus were measured in this experiment. In the single-arm steering scenarios, ten different neuromuscular signals, denoted as MS1-MS10, were measured from the right upper limb. These ten signals were the pectoralis major of the clavicular portion, the deltoid anterior, the deltoid middle (lateral), the deltoid posterior, the triceps long head, the triceps lateral head exterior, the biceps, the infraspinatus, the pectoralis major, and the teres major. For the steering experiments with the both-arm arrangement, ten EMG signals, denoted as MB1-MB10, were detected from both the right and left upper limbs. These signals were the pectoralis major of clavicular portion, the deltoid anterior, the deltoid posterior, the triceps long head, and the teres major of the right arm and the left arm. The electrode placements for the EMG measurement are shown in Figure 1d.

Participants
In total, 42 subjects in the age range of 22-50 (31±7) years were recruited for the experiments. Each subject had a valid driving license. All of the subjects had no previous knowledge of the research topic. Among these participants, 20 were randomly assigned to conduct the single-arm experiments, and the rest of the 22 participants were engaged in the both-arm steering experiments. The study protocol and consent form were approved by the JTEKT Corporation, Japan, and consent was obtained from all subjects.

Correlation Analysis
The cross-correlation analysis was used to measure the correlation between the ten measured neuromuscular signals and the applied steering torque during normal driving. The best match between the two signals could also be determined using cross-correlation. In this study, three levels of the correlation strength, namely, the strong correlation, moderate correlation, and weak correlation, were defined. Additionally, all the correlation statistics were bounded within the range of 0-1. The value of the correlation between 0.75 and 1 indicated a strong correlation. The value of the correlation between 0.3 and 0.75 indicated a moderate correlation, and the correlation value that was positioned between 0.3 and 0 indicated a weak correlation. The cross-correlations for some muscles were direction-dependent, which meant that the correlations between some neuromuscular signals and the steering torque were sensitive to the steering directions (the clockwise and counterclockwise directions). Therefore, the cross-correlations were calculated separately. A moving average filter with a span of five data points was applied in order to smooth the EMG signals before calculating the cross-correlation. The cross-correlation was calculated with the MATLAB function in the statistical analysis toolbox. The correlation and the time delays of the two timeseries signals could then be determined accordingly. To initiate the cross-correlation function, the maximum lag was selected as 3000, and the cross-correlation sequence was normalized.

Amplitude Analysis
This section describes the analysis of the amplitudes of the EMG signals for the ten different muscles. The amplitudes of the EMG signals reflected the signal strengths and to what extent the specific muscles influenced and determined the steering maneuvers. Due to the co-contraction mechanism, it was essential to analyze the muscle activation patterns under different driving scenarios with distinguished hand postures according to the amplitude of the EMG signals. Hence, the amplitudes of the EMG signals were analyzed based on two aspects, which were directional-oriented and non-directional oriented. Lastly, an integrated weighted muscle contribution analysis was proposed based on the joint consideration of the correlation and directional-oriented amplitudes.

1) Directional-Oriented and Non-Directional-Oriented Amplitude Analysis
The aim of the directional-oriented amplitude analysis was to provide a clear visualization of the different patterns of the EMG signals in different steering directions (i.e., the clockwise and counterclockwise directions). Three different analysis approaches were adopted using muscle strength features. First, the muscle strengths were statistically analyzed with respect to different hand postures. The strength of each EMG signal was statistically analyzed with a boxplot and bar chart. The muscle strengths for different participants were identified separately based on the steering direction. Then the statistical visualization, including the median and SD of the directional-oriented muscle amplitude, could be described with the boxplots. Second, the total muscle strength for the single-arm and both-arm steering scenarios was compared. For each participant, the total muscle strength was the summation of the amplitudes of all ten measured muscles with consideration of the directions. Then the mean value of the total muscle strengths for each driving mode for all participants was compared with respect to different hand positions. Third, the mean value of the muscle strength for each muscle was also compared with respect to different steering modes in order to understand the different patterns of the muscle amplitudes for different driving scenarios.
A similar analysis process was proposed for the non-directional oriented analysis. For the no-directionoriented cases, the absolute value of the steering torque without consideration of the steering direction was used. The non-direction-oriented amplitude analysis was the summation of the amplitudes of the measured muscles without consideration of the directions. The analysis generated a direct visualization of the different distributions of the muscle strengths with respect to the different driving modes and muscles.

2) Weighted Muscle Contribution Analysis
A weighted muscle contribution analysis was proposed based on the integration of the correlation and the amplitude of each EMG signal, for which the correlation was used as the weight of the amplitude. The purpose of setting this weighted muscle contribution index was to identify the most relevant muscles while eliminating the less important muscles used in the steering tasks. For example, although the teres major muscle showed a strong correlation based on the cross-correlation analysis, the absolute amplitude of its signal was very weak. This meant that the teres major muscle was not a determinative muscle. The weighted muscle contribution was calculated based on the direction-dependent correlation and amplitude of each EMG signal. Specifically, the correlation and amplitude of each EMG signal with respect to the counterclockwise and clockwise directions were multiplied separately at first, and then the contributed muscle importance was the summation of the generated products of both directions. The contributed energy Ei of each muscle could first be given with Eq. (1): where is the contribution index of the i th muscle, and _ and _ are the correlation values of the i th muscle with respect to the positive steering torque (i.e. in clockwise direction) and the negative steering torque (i.e. in counterclockwise direction), respectively.
_ and _ are their corresponding muscle amplitudes. The overall contribution of the certain muscle among all the measured muscles could then be represented as: Based on the proposed contribution assessment method, the most determinative muscles could be better identified and visualized as the importance, which represented a joint consideration of both the correlations and the absolute strengths of the EMG signals.

Steering Smoothness Analysis
It was important to maintain smooth steering operation in our naturalistic driving in order to ensure the ride comfort and vehicle safety. In this study, the smoothness of each participant's steering action was evaluated based on two different methods, namely, the approximate entropy method and the sliding window method.

1) Approximate Entropy Method
Because the participants were required to steer the hand wheel and to generate a steering torque according to the experimental requirements, the expected steering torque could be seen as a determinative and predictable sequence. Therefore, the regularity and the variation of the output torque could be used to evaluate the smoothness and the stability of the steering performance. The approximate entropy (ApproxEnt), which was an efficient solution for the regularity measurement of the non-linear time-series signals, was used (Pincus, 1991). A delayed reconstruction : with the same length ( points) of the original steering torque sequence with a lag of was generated at first. Then the numbers of the points within the range at each sampling step could be determined as (Pincus, 1991) where • is the indicator function, and is the radius of the similarity.
Then the approximate entropy of the sequence could be calculated as , The smaller the value of the approximate entropy of a certain sequence, the more regular and stable the system is. The approximate entropy of each sequence was calculated with the MATLAB built-in function.
The embedding dimension of the function was two, the reconstruction lag was one, and the radius of similarity was selected as 0.2 , where is the steering torque sequence.

2) Sliding Window Method
Another method that was adopted in order to measure the signal's irregularity and smoothness was the sliding window approach. In this method, a sliding window was used to calculate the standard deviation of the sliding slice. Then the average STD of the signal over the entire time horizon was statistically analyzed based on the SD of these segments. To increase the computational efficiency, a non-overlapped sliding window method was used. The sliding window size in this study was selected as 100 data points, which was a 100 ms segment. The STD index (STDI) of the smoothness of the entire steering period could be represented by the mean STD of the whole sliding slices of each sequence: where is the total number of the sliding slices of the entire sequence, M is the selected window size for each slice, is the m th point within the segment, and is the mean value of the segment.
The size of the sliding window was 100 points, which was small enough to smoothly evaluate the stability of each slice. It should be noted that a sliding window of a very large size could not efficiently measure the local smoothness of the steering torque, which could lead to a coarse statistical result. However, the size of the sliding window could not be too small because a single window could only provide a short period variation, and a fixed single window would be unable to measure the dynamic trends and significant variations among the signals.

Statistical Methods
The statistical analysis was performed in Matlab (R2019b, MathWorks) and Microsoft Excel. The statistical significance was determined using paired t tests at the α = 0.01 threshold level throughout the research. The central tendency was estimated using the mean.