UNI-DIRECTIONAL AND BI-DIRECTIONAL LSTM COMPARISON ON SENSOR BASED SWIMMING DATA

D. Tarasevičius Research Scholar, Department of Electronic Systems,Vilnius Gediminas Technical University,Vilnius. ...................................................................................................................... Manuscript Info Abstract ......................... ........................................................................ Manuscript History Received: 14 March 2020 Final Accepted: 16 April 2020 Published: May 2020

Human activity recognition from sensors has been gaining alot of attention. One of the most exciting applications of human activity recognition systems are in sports and especially in automatic classification of swimming styles. The swimmingstyle recognition system could be valuable for elite swimmers to increase race performance and provide real-time feedback to thecoach, potentially enabling more efficient competitive and quantitative coaching [1]. Furthermore, the system can be beneficial for beginners who are practicing correct swimming style movements and the possibility to provide virtual coach assistance. Swimming styles and specific motion can be registered and collected using an inertial measurement unit (IMU), which consists of a 3-axis MEMS accelerometer, gyroscope, and magnetometer. Some of the devices also include additional sensors such as ambient light sensor barometer and heart ratesensor. One of the early research work using the inertial sensor to analyze swimming kinematics was presented by Ohgi and Yasumura in 2000 [2]. In this work, a wrist-worn accelerometer was used to perform kinematic analysis of a freestyle stroke. In the work of Jensen et al. [3] head-worn sensor was used to classify four main swimming styles breaststroke, backstroke, freestyle, and butterfly as well as turns.
Research work of [4] presents only breaststroke phase identification using inertialsensor worn on the arm and other on the leg. Android phoneworn on the arm was used in [5] to perform stroke recognition.Back worn IMU sensor was used to perform kinematic analysis [6] of a swimmer and tracking of swimming styles [7] as well as classification of them [8]. A chest-worn accelerometer was used in [9] to classify swimming stroke styles. The challenges that researchers face are proper feature extraction from data to reduce computational cost [3]. Moreover, some discussion arises on proper data collection in realistic conditions [10]. Most of the classification algorithms that are used in 736 swimming style and stroke recognition are classic machine learning methods. However, as deep learning is recently getting more attention in human activity recognition, it can be applied to swimming style classification as well. An interesting Deep learning approach for human swimming style recognition and lap counting can be observed in [10] where convolutional neural network (CNN) was used. The authors also provided publicly available swimming data collected using IMU, barometer, and ambient light sensors. To extend the deep learning approach of swimming style recognition a Uni-LSTM and Bi-LSTM were applied and the results were compared.

Data:
Publicly available data of swimming activity was provided by G. Brunner et al. [10]and was recorded using a "Nixon the Mission" smartwatchwith integrated inercial measurement unit (IMU) (accelerometer, gyroscope and magnetometer) as well as ambient light and pressure sensors. The data consisted of 40 swimmers data with 8 classes: Unknown, Null, Freestyle, Butterfly, Breastroke, Backstroke, Kicks and Turns. The class distribution is presented in FIG 1. Initially signals of sensors were sampled at maximum frequency possible wich was sampled at 104 Hz and 6.67 Hz, IMU and pressure as well as ambient light sensors respectively. Data, which was available for download, was provided resampled with cubic splines at 30 Hz and relabled.

Data preparation for comparison:
To compare LSTM models data was segmented into a fixed size 180 vector samples windows with overlap of 150 samples. To reduce the computational cost it was decided to extract specific set of features which are presented in a TABLE 1. Moreover, pressure and ambient light signals were omitted, because it was observed that these sensors do not provide any useful information for classification task.Kick class and unknown classes were omitted and turn and null classes were merged. In total 5 classes remained: null, freestyle, breastroke backstrokeand butterfly.
here f t , f t -Forget gate, i t , i t -Input gate, o t , o t -Output gate, z t , z t -Cell candidate, c t , c t -Cell state), h t , h t -. Cell output,h t−1 , h t+1 and c t−1 , c t+1values from previous block, X t -Vector of features, W ij , W ijweights, b j , b jbias weights, y t -LSTM output concatenation, tanh(x)hyperbolic tangent activation, σ(x)sigmoid activation, ○ -Hadamard product.
Uni-LSTM structure can be observed in a FIG 3. As was mentioned earlier Uni-LSTM can process sequences only in one direction in general cases only in a right direction. The equations (1-6) represent Uni-directional LSTM. LSTM models were trained using the same hyperparameters. It was chosen to use ADAM optimization algorithm. The base learning rate was chosen to be 0.001 and networks were trained for 10 epochs with 1024 mini-batches. Also a dropout was used in each layer for overfitting reduction. First layer had 50% dropout and second20% dropout.

Experimental Investigation: Data Investigation:
The data set contains competitive and non-professional team of men and woman swimmers of ages 25-75 years. All of the swimmers were able to swim a 100 meters in under 2 minutes. To understand key differences between the swimming styles accelerometersignals were plotted (FIG 4 -FIG 5). The key difference between backstroke and other swimming styles is the negative acceleration of z-axis. Freestyle and butterfly share similarities in an x-axis. Comparing the intensity of acceleration the most intensive swimming style is a butterfly stroke.   740 Testing of trained model was performed using the same approach as the authors of the dataset [10]. LSTM networks were trained on whole dataset and one swimmer was excluded on which these network models were tested. So in total 40 trainning and testing cycles were performed. The confusion matrices of Uni-LSTM and Bi-LSTM are  averaged and provided in the TABLE 1 and TABLE 2 respectively.

Butter. Target Class
Uni-LSTM performed with an average F1-score of 87.66%. Bi-LSTM performed with an average F1-score of 90.35%. As can be observed in both tables the most misclassified swimming styles are freestyle and butterfly. This is due to the fact that both classes share similarities when the position of IMU is on the wrist.

Conclusions:-
In this work a Uni-LSTM and Bi-LSTM models and their performance was evaluated on publicly available swimming data using confusion matrix and F1-score as the data was imbalanced. Bi-LSTM performed with 90.35% and Uni-LSTM performed with 87.66%. These results shows that Bi-LSTM performed much better as it can process sequences in both directions.