An Integrated Deep Ensemble-Unscented Kalman Filter for Sideslip Angle Estimation With Sensor Filtering Network

An integration scheme for sideslip angle estimation is proposed where a deep neural network and a simple kinematics-based model are combined in an unscented Kalman filter. The deep neural network contains two modules: a sensor filtering network designed to overcome the limitations of the kinematics-based model and a deep ensemble network to estimate the sideslip angle and its uncertainty. Both networks use recurrent neural networks with long short-term memory to analyze sequential sensor data. The networks were trained using only input signal sets that can be obtained from on- board sensor measurements. The filtering network reduces the noise and bias of the input signals to match the model used for the unscented Kalman filter. Next, the initial estimate and its uncertainty obtained from the deep ensemble network are utilized as a new measurement in the unscented Kalman filter, inducing an adaptive measurement variance. The algorithm was verified through both simulation and experiment, and the results demonstrate the effectiveness of the proposed algorithm.


I. INTRODUCTION
Modern vehicles are equipped with several components to improve their safety and stability [1]. Among these, many functions concern the lateral stability of the vehicle [2]- [5]. In particular, state variables related to vehicle motion cannot be easily measured by on-board sensors and their estimation has been the focus of intensive research in past decades, resulting in many works in the literature. These include a comparative study on the sideslip angle estimation [6], the longitudinal velocity estimation [7], and roll angle estimation [8]. The focus of this work considers the information about lateral state, especially the sideslip angle, β. To obtain a direct measurement of the sideslip angle, expensive and bulky sensors such as optical or RTK-GPS sensors are needed. Therefore, adequate strategies of indirect estimation are required to overcome this problem.
The associate editor coordinating the review of this manuscript and approving it for publication was Francesco Piccialli.
To address this problem, extensive studies have been conducted on methods for estimating the sideslip angle. For conventional methods, there are several model-based Kalman filtering (KF) approaches that utilize kinematics-and dynamics-based models. First, the kinematics-based model considers the movement of a vehicle without considering the forces [9]. The representative model is expressed considering the velocity, lateral acceleration, and yawrate [10]. However, the sideslip angle is in an integrated form in the sensor results. Thus, it is vulnerable to noise and bias, which accumulate over time, resulting in a large estimation error. Therefore, dynamics-based models with numerous degrees of freedom have been considered to enhance model accuracy and estimation performance. For example, two DOF singletrack model and higher-order DOF vehicle models [11]- [15] have been utilized for estimator design. The advantage of using dynamics-based models is that the quality of models improves the estimation quality of the states. However, as the model complexity increases, the number of parameters to estimate also increases. In addition, the uncertainty and VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ non-linearity of the models make the estimation problem very difficult.
Recently, sideslip angle estimation methods [16]- [21] based on artificial neural networks (ANNs) and deep neural networks (DNNs) have seen dramatic progress. Melzi et al. designed a simple ANN using a single hidden layer to verify its performance [16]. Chindamo et al. also used the ANN to verify the estimation performance by means of the CarSim simulator [17]. Boada et al. designed an adaptive neuro-fuzzy inference system (ANFIS) [18]. This method improves the estimation performance in comparison to the simple ANN by combining the advantages of the steady performance of fuzzy logic and the generalization of ANN. Ghosh et al. used a multilayer long short-term memory (LSTM) network [19] to exploit the characteristic that previous inputs can be used to obtain the current state. Graeber et al. proposed a hybrid approach to sideslip angle estimation using recurrent neural network (RNN) and kinematic vehicle models [20]. They used the derivative of the sideslip angle,β, obtained from the kinematic model, as a new input feature of the RNN to improve the performance. Bonfitto et al. employed a classification neural network to identify road conditions, and then designed three regression neural networks for use under different road conditions that were identified by the former network [21].
Nonetheless, there still exist limitations of learning-based models such that the estimation of learning-based methods is always considered to be reliable. Kim et al. considered uncertainty of learning-based method and designed various Kalman filter algorithms based on a dynamic model [22]. But, this approach requires significant computational burden due to the dynamic model. In addition, dynamic models that deal with forces and torques require considerable knowledge of parameters that can change over time.
In this paper, two independent approaches are combined to overcome these limitations. First, a sensor filtering network is proposed to overcome the limitation of the kinematicsbased model which does not require vehicle parameters, tireroad friction coefficients, or a tire model. The sensor filtering network is utilized to match the calculatedβ from the filtered data to the actual value while reducing the noise and bias of the original sensor data. Next, the deep ensemble method is used to obtain a robust estimate and its uncertainty [23]. The information on uncertainty is valuable because the estimate can be less reliable when tested in an unfamiliar environment. To address this, the uncertainty is utilized as a measurement variance value of the adaptive unscented Kalman filer (UKF). This method can improve the reliability of the estimate when uncertainty is low, and otherwise, the model with the filtered sensor inputs is considered to be more reliable. The contributions of this study can be summarized as follows: • A novel sideslip angle estimation method is proposed that combines the DNN and UKF with a simple kinematics-based model.
• The filtering network is designed to overcome the limitations of the kinematic-based model.
• The deep ensemble network with LSTM is utilized to analyze sequential sensor data, in addition to providing a robust estimation and the estimation uncertainty.
• The feasibility of the proposed method was validated under various road surface conditions using the commercial software CarSim, which uses high-dimensional vehicle dynamics. In addition, verification was carried out through real-car experiment in some limited environments. The remainder of this paper is organized as follows. In Section 2, the deep learning models comprising the sensor filtering and deep ensemble networks are described. Next, Section 3 explains the integration of both deep learning and kinematics-based models with the UKF in detail. In Section 4, the proposed algorithm is verified in simulation. Then, the proposed algorithm is verified in experiment in Section 5.

A. PRELIMINARIES
In this study, RNN and deep ensemble methods are used for precise sideslip angle estimation. First, an RNN is utilized to analyze the sequential data. A basic RNN has a vanishing gradient problem; therefore an LSTM network is used instead [24]. Next, a deep ensemble is used in this study as it uses multiple models to derive outcomes. This method improves the estimation performance and makes the estimate more robust. In our work, the deep ensemble network developed by Lakshminarayanan et al. [23] is used to estimate the sideslip angle and its uncertainty. From each network of the entire ensemble model, the estimate (µ) and standard deviation (σ ) of the estimate are obtained given the sensor input (S). Then, the final estimate (µ f ) and standard deviation (σ f ) are calculated as follows: where N is the number of networks and subscript i indicates the index of a single network. As the standard deviation should always have a positive value, the softplus function is applied as follows.
The deep learning models are divided into two modules. The first module filters sensor data to make it available for the kinematics-based model. The filtering network is composed of denoising and debiasing networks as shown in Fig. 1; both have one LSTM layer with a 32-dimensional cell state. The output of each denoising LSTM cell is connected to a fullyconnected layer with 32 hidden units, whereas in the case of the debiasing LSTM, only the end of LSTM is connected to a fully-connected layer. The sensor data input for the filtering network, S SF , includes velocity (v x ), yawrate (ψ) and lateral acceleration (a y ) for 10 steps in the interval of 0.1 s. The noisy and biased measurements at timestamp n are modeled as follows [25]: where bψ n and b a y n are quasi-constant biases and w v x n , wψ n , and w a y n are zero-mean Gaussian noises. The sensor data with noise and bias are input to the denoising LSTM which provides denoised sensor signals, S n , with biases. The output of the debiasing LSTM, V b , is utilized to obtain the final filtered sensor signals,S SF , which are used to calculate the derivative of the sideslip angle using (9) through the kinematics-based model.
To train the sensor filtering network, the loss function is composed of the following three terms: whereβ true andβ indicate the true value of the derivative of the sideslip angle and the estimated value from the sensor filtering network, respectively, and bs is the batch size. c 1 , c 2 , and c 3 are weights for each loss.
The first loss term is used to reduce the difference between the initial unfiltered sensor values and denoised sensor values. This loss prevents the output of the denoising network from deviating significantly from the original value. The second term reduces the difference between the true and calculated value from the sensor signals obtained from the network. The last term restricts the bias value from fluctuating because quasi-constant biases are assumed in (5) and (6). Through these processes, the sensor signals are trained to be modified to match well to the kinematics-based model.
The second module is used to estimate the sideslip angle and its uncertainty. The module consists of fully connected layers and an LSTM. Input features, S DE , are a combination of the steering wheel angle (δ), velocity (v x ), yawrate (ψ) and lateral acceleration (a y ). A total of 10 sequential data points, with an interval of 0.1 s, is used as the input of each fully-connected layer as shown in Fig. 2. Each input feature passes through two fully-connected layers with 128 hidden units, and the result is connected to the one-layer LSTM with a 256-dimensional cell state. Next, by passing the output of the LSTM to a fully-connected layer, an estimate and its uncertainty are obtained. Finally, by incorporating the results from each ensemble network using (1) and (2), the final sideslip angle estimate and standard deviation are calculated.
To train the deep ensemble network, negative log likelihood (NLL) loss is used as follows [26]: where y is the true value, µ is the estimated value and σ is the standard deviation of the estimate. As the NLL loss captures predictive uncertainty, if the unseen input data is given, the deep ensemble provides the estimate with large uncertainty accordingly. Two networks are trained independently and the standard ADAM optimizer [27] is used. For the filtering network, the learning rate is set to 0.0001 with a batch size of 32 and 10000 training iterations. Then, the learning rate is set to 0.001 for the deep ensemble network with a batch size of 256 and 20000 training iterations.

III. INTEGRATION OF DEEP LEARNING-BASED AND MODEL-BASED SIDESLIP ANGLE ESTIMATION A. KINEMATICS-BASED MODEL
A kinematics-based model considers only the motion of vehicle without any information regarding tire-road friction or other dynamic properties of the vehicle. A typical kinematicsbased model can be expressed as a numerical integration of sensor signals comprising yawrate (ψ), vehicle velocity (v x ) and lateral acceleration (a y ) as follows: However, there is a critical drawback of using a kinematicsbased model. As sensor signals are always accompanied by noise or bias, the kinematics-based model cannot be perfectly accurate, resulting in poor estimation performance [9]. This problem is investigated using a sensor filtering network in this study.

B. INTEGRATED METHOD
The extend Kalman filter (EKF) uses a linearization process; thus, the performance worsens when the non-linearity gets stronger. In contrast, the UKF uses a non-linear function without a linearization process. First, the system state, x t , and measurement, z t , are expressed using the state transition function, g, and measurement model, h, as follows: where u t is the input, and w t and t are the process and measurement noise, respectively. The non-linear function g in our work uses the kinematic model in (9 Then, the UKF uses sigma points that are symmetrically distributed near the estimated state. For n-dimensional Gaussian with the mean value (µ) and covariance ( ), the resulting 2n + 1 sigma points (X [i] ) are chosen as follows: (14) where λ = α 2 (n + κ) − n, α and κ are scaling parameters. Then, to recover the mean and covariance of the Gaussian distribution, each sigma point is given a weight as follows: where ζ contains prior information of the distribution. w m is used to compute the mean and w c is used to recover the covariance of the Gaussian. The mean (µ ) and covariance ( ) of the Gaussian distribution can be recovered from the sigma points explained above and are shown in (18) and (19): where  17: return µ t , t 19: end function All processes of the standard UKF are outlined in Lines 6-17 of the Algorithm 1 ifQ t ,z t andS t,SF are changed to Q t , z t and u t , respectively [28].
The deep learning-based and model-based methods are combined as shown in Fig. 3. From the deep ensemble network, the sideslip angle estimate, β DE , and standard deviation, σ β DE , are extracted. This process can be seen in Line 2 of Algorithm 1, where f DE (·) denotes the deep ensemble model. The obtained values are utilized as the measure and noise variance values. The standard deviation, σ β DE , provides valuable information to the model such that the reliability of the new measure can be adaptively considered in the UKF.
Next, the sensor filtering procedure is explained in Line 3, where f SF (·) denotes the sensor filtering network. The output of the network is the input vector for the UKF system. This process overcomes the limitations of the kinematics-based model by filtering out the original biased and noisy sensor data.

IV. SIMULATION
To verify the feasibility of the proposed estimation method, CarSim, MATLAB/Simulink and Tensorflow [29] are utilized. From CarSim, the dataset needed for training the network is obtained by constructing the simulation environment. Then, the UKF algorithm is designed using MATLAB/Simulink. Finally, deep learning models are designed and trained in the Tensorflow environment. The test scenarios include different driving maneuvers or different surface condition compared to the dataset used in the training phase.

A. TRAINING DATASET
Collecting a dataset is an important process. A high-quality dataset can be used to train a model to deal with a wide range of situations. In our training dataset, two road surfaces are included, i.e., a dry asphalt road with tire-road friction (µ) of 0.85, and a slippery road with tire-road friction of 0.3. Sine wave steering, swept steering, and ramp steering maneuvers are considered to construct the dataset. The velocity is set between 20 and 120 kph, and the steering angle range is between −120 • and 120 • . For the sine wave steering maneuver, the frequency of the steering wheel angle is set to 0.2 or 0.5 Hz. In the swept steering scenario, the frequency is set to be varied between 0.2-0.5 Hz.
To emulate sensor data from on-board sensors, Gaussian noise with variances of 0.04 • , 0.03 m/s, 0.04 • /s, and 0.01 m/s 2 are added to steering wheel angle, velocity, yawrate and lateral acceleration, respectively. In addition, biases of 0.1 • /s and 0.2 m/s 2 are added to yawrate and lateral acceleration, respectively.

B. TEST DATASET
The scenarios for the test dataset to verify the performance of the algorithm are listed in Table. 1. The first scenario includes a double lane change (DLC) maneuver at µ = 0.3. In the next two scenarios, sine wave steering is performed at µ = 0.85 and µ = 0.3. In the 2 nd and 3 rd scenarios, the frequency of the steering wheel angle is set to 0.25 Hz, which is different from that in the training dataset. The velocity is accelerated at 1 m/s 2 in both scenarios. The last scenario includes a new surface condition (µ = 0.2), and step steering is applied with a maneuver that is not used in the training phase.

C. RESULTS OF FILTERING NETWORK
Figs. 4 and 5 compare the results ofβ with and without the filtering network in all test scenarios. The plots include realβ value from CarSim (red solid),β estimate without the filtering network (gray dashed) andβ estimate with the filtering network (blue dash-single dotted). The result using the filtering network matches better with the ground truth than  the result without the filtering network, indicating a better accuracy of sensor inputs for the kinematics-based model used in our work. The performance of the filtering network is summarized in Table. 2. Metrics for validation are selected as the root mean square error (RMSE) and mean absolute error (MAE). The results with the proposed network show better performance in all test scenarios.

D. RESULTS OF SIDESLIP ANGLE ESTIMATION
In this subsection, the results of the proposed method are explained in detail. Figs. 6, 7, 8 and 9 show the results of the sideslip angle estimation. In each figure, the top graph shows the real β value obtained from CarSim (red solid), the estimate from the deep ensemble (black dashed), β DE , and the estimate with uncertainty values (green solid), β DE ± σ β DE . The graph in the middle shows the β estimation results without the filtering network (blue dash-single dotted) and with the filtering network (green solid). As can be seen in the bottom graph, the estimation performance improves when the filtering network is utilized and this indicates that the filtering network improves the suitability of sensor inputs for use with the kinematic model. Furthermore, the uncertainty of deep ensemble network is high at about 6 s in Fig. 9. The performance of the proposed method is improved by giving a greater weight to the UKF result. The final estimation results in terms of RMSE and MAE are summarized in Table. 3. The proposed algorithm is also compared in simulation with the other algorithms [17], [19] reported in the related literature. In general, the integrated deep ensemble and UKF provides robust estimation results for the test scenarios than various artificial networks only approaches. In addition, the results using the filtering network show better performance than the others in terms of both metrics.

V. EXPERIMENT
For experimental verification, the test vehicle is equipped with a Correvit S-Motion optical sensor from Kistler as shown in Fig. 10. The optical sensor determines the longitudinal and lateral velocities to calculate the sideslip angle precisely. All the data from the on-board sensors of steering wheel angle, velocity, yawrate and lateral acceleration is logged through the Controller Area Network (CAN) interface.

A. DATASET
For the data collection using the test vehicle, various maneuvers in a velocity range of 40 ∼ 130 kph and a steering wheel angle range of −100 ∼ 100 • are conducted. For the maneuvers, lane change, sine wave steering and random steering in various speed ranges are selected. The training and test datasets consist of different scenarios. The experimental scenarios for test dataset are listed in Table. 4. The first scenario includes the random steering maneuver on dry asphalt road with the velocity range of 40 ∼ 60 kph. In the second scenario, DLC (Double Lange Change) maneuver is performed with steering wheel angle of −60 ∼ 60 • at the speed of 75 ∼ 100 kph. For the last scenario, the slalom maneuver is performed with steering wheel angle between −60 ∼ 60 • at the speed of 40 ∼ 70 kph.

B. RESULTS OF FILTERING NETWORK
The same filtering network architecture which is used in the simulation section is used for training. Fig. 11 compares the  result ofβ with and without the filtering network in test scenarios 1. The result with the filtering network shows better performance than the result without the filtering network. The experimental results of the filtering network are summarized in Table. 5 in terms of RMSE and MAE. The results with the proposed network show better performance in all test scenarios.

C. RESULTS OF SIDESLIP ANGLE ESTIMATION
In this subsection, the experimental result of the proposed method is explained in detail. Fig. 12 shows the result of the sideslip angle estimation. As seen in the bottom enlarged graph, the estimation performance improves when   the filtering network is utilized. This indicates that the filtering network improves the suitability of sensor inputs for use in the kinematic model in the real-car experiment.      UKF, resulting in better performance. The final estimates for the sideslip angle are summarized in Table. 6. The proposed algorithm is also compared in experiment with the other algorithms [17], [19]. The experimental results demonstrate that the proposed approach integrating deep ensemble with UKF provides least errors in the sideslip angle estimation. The estimation performance without the filtering network is the worst in most cases compared to the methods using the deep ensemble or filtering network. This indicates that using the raw sensor data to a simple kinematic model in the UKF provides large error in the estimation result. The proposed method is run with an average computation time of 15 ms on a PC (Intel(R) Core(TM) i7 at 4.00 GHz, 16.00-GB memory).

VI. CONCLUSION
This paper presents a novel framework integrating deep learning-based and model-based methods to estimate the vehicle sideslip angle. A sensor filtering network is designed to ensure that the input sensor signals work well for the kinematics-based model. Then, a deep ensemble network provides a sideslip angle estimate and its uncertainty. The estimate and its uncertainty from the network are utilized as measurement inputs of the UKF. The proposed algorithm is first validated in a simulation by means of CarSim. Then, a real-car experiment is also carried out using an optical sensor for validation purpose. Both simulation and experimental results demonstrate that the proposed approach shows better performance in sideslip angle estimation than deep ensemble only and integrated method without the filtering network. Future work will validate this method under more diverse driving maneuvers and road conditions. GIHOON KIM received the B.S. degree in automotive engineering from Hanyang University, Seoul, Republic of Korea, in 2017, where he is currently pursuing the Ph.D. degree with the Department of Automotive Engineering. His research interests include interaction-aware driving, which is planning the path considering effects to other vehicles from ego vehicle's movement and the development of control system for collision avoidance using aggressive-maneuver driving like drift control.
SEUNGWON CHOI received the B.S. degree in automotive engineering from Hanyang University, Seoul, Republic of Korea, in 2019, where he is currently pursuing the Ph.D. degree with the Department of Automotive Engineering. His research interests include trajectory prediction, motion planning, localization, and deep learning.
KUNSOO HUH (Member, IEEE) received the Ph.D. degree from the University of Michigan, Ann Arbor, MI, USA, in 1992. He is currently a Professor with the Department of Automotive Engineering, Hanyang University, Seoul, South Korea. His research interests include machine monitoring and control, with emphasis on their applications to vehicular systems. His current research interests include sensor-based active safety systems, V2X-based safety systems, autonomous vehicle control, and AI applications in autonomous vehicle. VOLUME 9, 2021