Intelligent PI control based on the ultra-local model and Kalman filter for vehicle yaw-rate control

This paper proposes an intelligent proportional integral (PI) control based on an ultra-local model and a Kalman filter with automatic parameter tuning for vehicle yaw-rate control. A good vehicle yaw-rate controller is essential for autonomous driving. The traditional PI controller proposed in the conventional method cannot realize the desired yaw-rate response because the vehicle velocity yaw-rate characteristics change with vehicle velocity. Although an intelligent model-free PI control can improve the yaw-rate response, it does not account for the measurement noise, so the noise amplified by feedback control is added to the control input. This problem is crucial because a large control input velocity (i.e. a high steering-angle velocity) may cause wheel damage. A Kalman filter was introduced to conventional intelligent PI control to address this problem. In addition, we automatically tune the design parameters of the Kalman filter by Bayesian optimization. The effectiveness of the proposed method was investigated in a vehicle simulator. The experimental results confirm higher control performance of the proposed method than of conventional methods in noisy cases.


Introduction
A good vehicle yaw-rate controller is essential for autonomous driving.The target yaw rate is calculated to decrease the tracking error between the actual and target paths of an automated driving vehicle.The yaw-rate controller calculates the steering angle as a control input signal to decrease the tracking error between the yaw rate and its target value.The desired yaw-rate response enables the desired path planning [1][2][3][4][5][6][7][8][9][10].Among the conventional methods, yaw-rate control based on proportional integral derivative (PID) control has been proposed [8][9][10].PID control has been popularized in the industry owing to its simple structure [11,12]; however, when the vehicle velocity varies, a fixed PID controller cannot realize the desired yaw-rate response [13] because the lateral dynamics of the vehicle system become time dependent [14].In particular, a variable vehicle velocity alters the cut-off frequency and DC gain of the lateral dynamics.These time-variant properties of the vehicle deteriorate the control performance of normal PID control.
In a study on control theory, Fliess et al. proposed a new PID control method based on an ultra-local model.Their method, called intelligent PID control [15][16][17], can realize the desired control response even for controlled objects with strong nonlinearity and timevariant properties.Such an intelligent PID controller is efficiently designed because the number of tunable parameters is extremely low.As a small number of parameters is crucial in industrial systems, intelligent PID control has been widely studied [18][19][20][21][22][23][24].Furthermore, intelligent PID control can be expanded to integral-proportional derivative (I-PD) control [22].I-PD control based on intelligent PID can realize the desired closed-loop response even when the target signal varies rapidly.An intelligent PID control scheme can better realize the desired control performance than normal PID control.In an automated vehicle with intelligent PID, the vehicle yaw-rate controller can realize the desired control response even when the vehicle velocity varies, but the measurement noise is not considered.The output velocity signal of intelligent PID control, which is calculated as a differential, must be fed back for calculating the control input signal.Measurements noise is greatly enhanced in the differentiated output signal, causing a noisy control signal [24].Consequently, the control-input-signal velocity may become extremely large.In an automated vehicle, the control-input-signal velocity should be minimized because it is set as the steering-angle velocity.A large steering-angle velocity risks breakage of the wheel due to road friction.Therefore, intelligent PID controllers cannot be directly applied to automated vehicles.
To address this problem, intelligent PID control was expanded to decrease the effect of measurement noise.The proposed method consists of a PI controller and a time-invariant Kalman filter [25,26] to estimate the un-modelled dynamics.The control input signal can be calculated without a differentiated output signal, and its velocity is extremely small.In the conference paper [23], the parameters for designing the Kalman filter were determined by trial and error, and the parameter settings affected the controller performance.The present paper introduces automatic parameter tuning with a constraint on the steering-angle velocity using Bayesian optimization [27][28][29], which applies to a nonlinear cost function.Bayesian optimization was selected because it is derivative-free and iterates more quickly than other optimization algorithms, such as particle swarm optimization, genetic algorithms and sequential quadratic programming.The effectiveness of the proposed method is confirmed through numerical simulations using a vehicle dynamics blockset [30].With the parameters obtained by the proposed method, the vehicle achieves the desired yaw-rate response within the constraints of the steering-angle velocity.
The remainder of this paper is structured as follows.The problem is formulated in Section 2. The conventional and proposed methods are presented in Sections 3 and 4, respectively, and the parameter tuning scheme is presented in Section 5.The numerical simulation results are presented in Section 6.The study concludes with Section 7.
[Notation] To enhance readability, we denote the output time signal y(t) of G(s) relative to the input time signal u(t) as y = G(s)u and the time-derivative signal ẏ as ẏ = sy.Throughout this paper, we omit the notation "s" from G(s) if the context clarifies that G(s) is a rational function with respect to s.

Problem formulation
The automated vehicle is a front-wheel-steering vehicle, as shown in Figure 1.The control input signal u is the front-steering angle, and the output signal y is the yaw rate of the vehicle obtained by applying u to the vehicle plant.The reference signal r is the target yaw rate, and V is the vehicle velocity.The output signal is measured as the vehicle yaw rate with a measurement noise d (assumed as white noise).The lateral dynamics from the steering-angle u to the yaw rate y are timevariant.When the vehicle velocity is high, the cut-off frequency of the lateral dynamics is small, and the DC gain of the lateral dynamics is high (further details are given in the Appendix).Therefore, this vehicle is a timevarying system.The control system should track the yaw rate y as closely as possible to T d r.As the input constraint, we set the maximum steering-angle velocity u vmax at which the tires are not damaged by road friction.A small steering-angle velocity is required to satisfy | u| < u vmax .

Conventional intelligent PI control scheme
This section describes the conventional intelligent PI control method described in [22].

Basic idea
This section discusses the control performance of intelligent PI control in the absence of measurement noise.The controlled object can be expressed as where f () is an unknown dynamic system.Therefore, the controlled object is modelled as the following ultralocal model [15][16][17]: where n and a are tunable parameters.F represents the un-modelled dynamics of f ().Here, we set a simple controller structure with n = 1.The plant model is represented as The controller structure is set as the PI controller under n = 1.The control law is given as where e is the tracking error calculated as e = r − y, and K p and K i are the PI gains.From Equations ( 3) and ( 4), the control response can be expressed as From Equation ( 5), the closed-loop transfer function can be expressed as Note that the closed-loop transfer function consists only of the PI gains.Therefore, if we can implement the control law given by Equation ( 4), we can realize the desired closed-loop response by tuning the PI gains.However, the control law cannot be implemented because F is unknown.F was estimated using the output/input signal to solve this problem.Rearranging Equation ( 3), F can be calculated as follows: Equation ( 7) cannot be implemented because it involves an algebraic loop.Replacing u with u e , the un-modelled dynamics are estimated as follows: where u e is the shaped control input based on the control input u.Using a low-pass filter, u e can be calculated as where T is a time constant.Replacing F with F e , the control law becomes Here, u e approximates u if the time constant T is extremely small.Therefore, F e is approximately equal to F. The closed-loop transfer function is then expressed as Based on the above discussion, the desired closed-loop response can be realized under the control law given by Equation (10).When the reference model is given as the PI gain can be uniquely determined as where τ and ω are the damping coefficient and a natural angular frequency, respectively.Figure 2 presents a block diagram of the control system.The feedback  controller consists of the PI controller and the estimator computed by Equation ( 8).This control system can eliminate the influence of the un-modelled dynamics F, thereby realizing the desired closed-loop response even when the controlled object has strong nonlinearity and time-variant properties.

In the presence of measurement noise
This section analyses the control performance of intelligent PI control in the presence of measurement noise.Figure 3 is a block diagram of the control system.In the presence of measurement noise d, the output signal can be expressed as where y p denotes the noise-free output of the controlled object.Substituting Equation ( 14) into Equation (8), F e is obtained as Note that the measurement noise is differentiated.High-frequency noise gives rise to a large ḋ and a noisy F e , and consequently, a noisy control signal.Therefore, this method is undesired in vehicle yaw-rate control systems.

Yaw-rate controller design based on intelligent PI control
This section explains our vehicle yaw-rate control based on intelligent PI control [23], on which the present study is based.

Control law
The proposed yaw-rate control system consists of a PI controller and a time-invariant Kalman filter.First, we assume that Ḟ vanishes: The ultra-local model given by Equation ( 3) can then be expanded as the following state-space model: To estimate F e , we designed a time-invariant Kalman filter [25]: with where x e is the estimated state variable, y e is the estimated output signal, and the Kalman gain L can be calculated using the algebraic Riccati equation as follows: Here, Q∈ R 2 × 2 and R∈ R are the process and measurement noise covariance matrices, respectively.They are positive definite matrices set by the user.Using x e , the control law becomes When the Kalman gain is properly tuned, F e approximates F. Therefore, the closed-loop transfer function can be expressed as Equation (11). Figure 4 is a block diagram of the proposed control system.

Analysis of estimation performance
We next analysed the control performance in the presence of measurement noise.Using Equations ( 25) - (27), F e can be expressed as Substituting Equation ( 14) into Equation (32), F e can be expressed as As shown in Equation (33), the low-pass filter eliminates the influence of ḋ.Therefore, the proposed method can eliminate the influence of the measurement noise in the estimation of F e .

Parameter tuning of the Kalman filter
When designing the Kalman filter, we require Q and R. The settings of these parameters affect the controller performance, as shown in the next section.Although the parameters can be set if the controlled object is known, we consider a model-free approach with nondetermined parameters.In the conference paper [23], these parameters were determined by trial and error, and the automatic parameter tuning was constructed as an open problem.Our parameter optimization aims to minimize the cost function where θ = [Q R] and u vmax is given by the designer.We have rewritten the above optimization problem as where P c is a penalty function and w is a weight.We automatically optimize the parameters using the Bayesian optimization approach.Here, the Bayesian optimization is briefly described based on the literature [27][28][29].

Gaussian process regression (GPR)
Let observation z be the performance cost described by an unknown function f : with where GP and N represent the Gaussian and normal distribution processes, respectively, σ 2 n is the variance in the measurement noise, θ and θ are two sets of parameters, m(θ ) is the mean function, and k(θ , θ ) is the covariance function given as Then the predicted mean and variance are respectively given as with Here, m i (θ * ) and σ i (θ * ) are the posterior mean and variance, respectively.The covariance function of the GP can be defined (for instance) as the squared exponential (SE) covariance kernel The hyper-parameters σ 0 and λ, which characterize the SE kernel, can be chosen by maximizing the log marginal likelihood [29].

Bayesian optimization
Bayesian optimization is a black-box optimization method based on Gaussian process regression.Assume that our goal is to minimize the black-box function f (θ ) with noisy observations (38).Figure 5 shows the framework of the parameter optimization.Based on this figure, the Bayesian optimization algorithm was developed as follows: [ Step 0] Initialization Run N in ≥ 1 experiments on N in different (e.g.randomly selected) values of the parameters θ j (j = 1, . . ., N in ).Performing this experiment for each parameter θ j and measuring the observation signal z j which is the output of the cost function J w , construct an initial set D i = {(θ 1 , z 1 ), (θ 2 , z 2 ), . . ., (θ i , z i )} of parameters and their corresponding observations. [ Step 1] Learn GP Using data D i , train a GP to approximate f. [ Step 2] Obtain the optimization parameters Using the acquisition function, obtain the optimization parameters θ i+1 . [ Step 3] Conduct the experiment and obtain the data.This experiment obtains the observation signals z i+1 for the parameters θ i+1 .Update the data D i+1 as D i ∪ {θ i+1 , z i+1 }. [ Step 4] Return to Step 1 and update i as i + 1.If i is N end , the algorithm terminates.
In this paper, the acquisition function α is the wellknown expected improvement (EI) function, given as where f (θ + ) represents the best value of the function at the ith iteration, and In the GP, the EI is evaluated analytically as Here, where ψ and are the probability density function and cumulative density function of the standard normal distribution, respectively.

Remark 1.
Here we summarize the features of the proposed method.The PI gain is determined automatically and uniquely for a user-defined reference model from Equation (13).The design parameters of the Kalman filter are optimized by Bayesian optimization.The designer must tune only the parameter a in the ultra-local model.According to the literature [15,16], a can also be determined intuitively.
Remark 2. The previous study [21] proposed the PI gain and a as tunable parameters but did not optimize the disturbance estimator parameters.In the present paper, the PI gain and a are determined in advance, and the parameters for designing the Kalman filter are optimized.
Remark 3.Although the Kalman gain L is considered to be directly optimized, the Kalman filter may have unstable poles.Thus, we handle Q and R as tunable parameters.

Numerical simulation
This section evaluates the proposed method through numerical simulations of vehicle yaw-rate control.

Simulation models
The proposed method was verified using the vehicle dynamics blockset of MATLAB/Simulink [30], which was used in the literature [31][32][33].As the vehicle dynamics blockset was built by conducting various experiments, it represents the dynamic behaviour of a real vehicle with high fidelity.When developing the vehicle model from the front-steering angle to the yaw rate, we assumed a vehicle body with a three-degreesof-freedom truck block.Table 1 lists the simulation parameters of a passenger car [13].The sampling time was set to 0.01 s, and the measurement noise was set using the band-limited white noise block of MAT-LAB/Simulink.The noise power was set to 2.0 × 10 −9 [23].

Simulation conditions
In the simulation, the desired closed-loop transfer function was set as the following second-order system: Here, the PI gains were determined by Equation ( 13) and were tuned as To prevent road-friction damage to the wheel, the input constraint u vmax of the steering-angle velocity was set to u vmax = 1 rad/s.The tunable parameter a was set as a = 1.The Kalman gain L was determined using the following hyper-parameters: Note that Q can be set as a positive definite matrix in Equation (55).However, the matrix Q was set to a diagonal matrix because an increase in tuning parameters may lead to an increase in the number of optimization iterations.
To validate the proposed methods, the control results of the proposed method were compared with those of the conventional intelligent PI control of Equation (10) and normal PI control.In the conventional intelligent PI control, the time constant T was set to 0.05 s.The normal PI control law is Note that this equation represents the I-P control, which has no derivative controller.The PI gains of the conventional intelligent and normal PI controllers were those used in the proposed method.Tables 2 and 3 show the design parameters and Kalman gains before and after tuning, respectively.

Simulation results and discussion
This section describes the simulation results.The target yaw rate was set as the ramp signal.To verify the robustness of the model against the time-variant properties of the vehicle, the vehicle velocity V was varied as 60 and 5 km/h.The optimization results are shown in Figure 6.The cost function, computed using Equation   (36), reduced as the iterations proceeded.Figure 7 and Figure 8 present the reference signal used in the Bayesian optimization.Figure 7 plots the simulated yaw rate (output), steering signal (input), and steering-angle velocity of a vehicle travelling at 60 km/h.tracking error between y and T d r.Although the conventional intelligent PI control achieved the desired closedloop response, the steering-angle velocity exceeded the maximum steering-angle velocity due to measurement noise.Meanwhile, the proposed method obtained a ied, but the steering-angle velocity exceeded the maximum limit.In contrast, the proposed method achieved the desired closed-loop response while satisfying the constraint condition.As shown in the bottom panel of Figure 8, the steering-angle velocity in the proposed method overlapped that of normal PI control and  remained below the maximum steering-angle velocity.Moreover, the proposed method generated the desired closed-loop response more closely than normal PI control.These results indicate that the proposed method can realize the desired yaw-rate control response of a vehicle with variable velocity.
Next, this study confirmed the effectiveness of the proposed method when the vehicle velocity changes and various references were given.Figure 9 shows the yaw rate (output), tracking error, steering signal (input), and steering-angle velocity, in this case, and Table 6 lists the corresponding control results, including the MSE, maximum absolute values of the steering angle velocity and the constraint satisfaction.The results show that the proposed method achieved high tracking performance while satisfying the constraint.
The performances of conventional intelligent PI control, traditional PI control, and the proposed method were evaluated.The proposed method provided a high tracking performance while satisfying the constraints of the steering-angle velocity.The design parameters of the Kalman filter were optimized within 15 iterations of experiments.Furthermore, the proposed control law based on the simple structure of an ultra-local model delivered high performance in a system with vehicle-speed-sensitive characteristics.Therefore, it can potentially advance the effectiveness of mass-produced products.

Conclusion
A Kalman filter was introduced to the intelligent PI control method, which eliminates the influence of measurement noise and hence realized the desired vehicle yaw-rate control.The proposed method consists of a PI controller and a Kalman filter.The design parameters were obtained through the Bayesian optimization approach.The effectiveness of the proposed method was confirmed in a vehicle simulator.The simulation results demonstrated that the proposed method could realize the desired yaw-rate response even in the presence of measurement noise.Future work will validate the proposed method in an actual automated vehicle.

Figure 2 .
Figure 2. Block diagram of intelligent PI control.

Figure 3 .
Figure 3. Block diagram of intelligent PI control in a noisy case.

Figure 4 .
Figure 4. Block diagram of yaw-rate control systems in the proposed controller.

Figure 7 .
Figure7.Yaw-rate control results of a vehicle travelling at 60 km/h.In the bottom panel, the solid and dotted lines represent the steering-angle velocity and input constraint, respectively.The yaw rates of the desired, conventional, and proposed methods after tuning almost overlap.

Figure 9 .
Figure 9. Yaw-rate control results in the case of changing the vehicle velocity.The yaw rate of the desired and proposed methods almost overlap.

Table 1 .
Simulation parameters of a passenger car.

Table 2 .
Parameter values before and after tuning by Bayesian optimization.

Table 3 .
Kalman gains before and after tuning by Bayesian optimization.

Table 4 .
Control results of a vehicle travelling at 60 km/h (MSE = mean square error).

Table 4
Yaw-rate control results of a vehicle travelling at 5 km/h.The yaw rates of the desired, conventional, and proposed methods after tuning almost overlap.
lists the control results, including the mean square error (MSE), maximum absolute values of steering angle velocity, and constraint satisfaction, of a vehicle travelling at 60 km/h.The MSE was calculated using the Figure 8.

Table 5 .
Control results of a vehicle travelling at 5 km/h.

Table 6 .
Control results in the case of changing the vehicle velocity.