R ESEARCH ON DRIVING BEHAVIOR DECISION MAKING SYSTEM OF AUTONOMOUS DRIVING VEHICLE BASED ON BENEFIT EVALUATION MODEL

: Autonomous driving vehicle could increase driving efficiency, reduce traffic congestion and improve driving safety, it is considered as the solution of current traffic problems. Decision making systems for autonomous driving vehicles have significant effects on driving performance. The performance of decision making system is affected by its framework and decision making model. In real traffic scenarios, the driving condition of autonomous driving vehicle faced is random and time-varying, the performance of current decision making system is unable to meet the full scene autonomous driving requirements. For autonomous driving vehicle, the division between different driving behaviors needs clear boundary conditions. Typically, in lane change scenario, multiple reasonable driving behavior choices cause conflict of driving state. The fundamental cause of conflict lies in overlapping boundary conditions. To design a decision making system for autonomous driving vehicles, firstly, based on the decomposition of human driver operation process, five basic driving behavior modes are constructed, a driving behavior decision making framework for autonomous driving vehicle based on finite state machine is proposed. Then, to achieve lane change decision making for autonomous driving vehicle, lane change behavior characteristics of human driver lane change maneuver are analyzed and extracted. Based on the analysis, multiple attributes such as driving efficiency and safety are considered, all attributes benefits are quantified and the driving behavior benefit evaluation model is established. By evaluating the benefits of all alternative driving behaviors, the optimal driving behavior for current driving scenario is output. Finally, to verify the performances of the proposed decision making model, a series of real vehicle tests are implemented in different scenarios, the real time performance, effectiveness, and feasibility performance of the proposed method is accessed. The results show that the proposed driving behavior decision making model has good feasibility


Introduction
As the development of economy and technology, car ownership is increasing rapidly in worldwide. With the amount of cars grows, traffic congestions and traffic accidents increase significantly. Traffic congestions cause extra fuel consumption and reduce traffic efficiency . Meanwhile, traffic accidents cause serious loss of lives and economy. Research of intelligent transportation system provides a new way to solve the problems above , Czech et al., 2018. With the development of computer technology, sensor technology and vehicle electronic control technology, research of autonomous driving vehicles has become a new hot topic in both academia and industry , Veres et al., 2011. Autonomous driving vehicles could increase the efficiency of road and reduce traffic congestion. Beyond that, autonomous driving vehicles could reduce the proportion of people involved in traffic activities, and reduce or even eliminate traffic accident caused by human factors , Wu et al., 2017. Furthermore, autonomous driving vehicles could complete various driving tasks in extreme environment without human drivers . With the further study of autonomous driving vehicles, decision making systems of autonomous driving vehicles are gaining increasing attention from researchers. Decision making system of autonomous driving vehicles could receive environmental information and outputs driving behavior according to its own decision criteria , Veres et al., 2011. Driving decision making system takes traffic information, environment information and vehicular states as inputs, and outputs stable and effective driving instructions in complex and changeable traffic scenarios. The real time performance, environmental adaptability and the robustness of decision making algorithm are the key indexes to evaluate the performance of decision making system (Galceran et al., 2017, Cunningham et al., 2015. At present, the research of autonomous driving vehicles decision making systems could be broadly divided into decision making model based on logic rules and decision making model based on mathematical statistics . The structure of machine learning algorithms based on mathematical statistics is relatively simple. And it could ex-tract scenario features and decision making attributes automatically. Scholars have done a lot of significant research works in this field. Advanced algorithms such as deep learning, decision tree algorithms and reinforcement learning algorithm have been applied in current studies , Veres et al., 2011. Decision correctness could be improved with the completeness of training data. However, the algorithm model is over-dependent on the quantity and quality of training data. Furthermore, due to the limitation of algorithm, the interpretability of the decision result is weak and model updating is difficult. Therefore, decision-making systems based on learning algorithms are still being improved. As a representative method of rule-based decisionmaking, the finite state machine (FSM) method has been mature and widely used in decision-making system of autonomous driving vehicles. As early as 2007 in the DARPA unmanned ground vehicle challenge, some of the participating vehicles had been equipped with decision-making systems based on finite state machines (Xie et al., 2007), which could satisfy the driving behavior decision of simplified traffic scenarios. A Furda has carried out a great deal of research work on intelligent vehicles' decision making in complex scenario, a decision making system based on Multiple Criteria Decision Making (MCDM) was proposed to select the most appropriate driving maneuver . J Chen has studied decision making method for autonomous driving vehicles in complex traffic scenarios, driving decision making process was decomposed into three steps and a driving decision making approach based on Multiple Attribute Decision Making(MADM) was proposed, both driving safety and driving efficiency were taken into consideration to choose the optimum decision . J Ji established finite state machine based on the logical relationship and state change process of driving behaviors, by combining with a virtual dangerous potential field, the main autonomous driving functions for intelligent vehicles could be realized (Ji et al., 2018). M Du presented a decision making model based on ID3 decision tree for autonomous driving vehicles, to analyze the transformation of driving states, the order of grey entropy relation grade of all condition attributes influence are confirmed, the real time performance and accuracy are improved (Du., 2016). G Xiong proposed a hybrid state system for autonomous vehicle by establishing finite state models of behavior prediction and behavior decision making. Combining with safety rules, decision making of autonomous vehicle in intersection scenario is achieved (Xiong et al., 2015). M Czubenko has proposed an intelligent decision-making model based on human psychology which could output humanized driving behavior in complex critical conditions (Czubenko et al., 2015). In order to take the uncertainties factors into consideration as far as possible, scholars established observable Markov process model which regarded driving as a problem in continuous space and improved the reliability of final decision results (Song., 2016). However, the huge amount of computation limits the application of Markov process model in real vehicle cause complex mathematical statistics decision making model could not satisfy the real-time performance in real traffic scenarios. Most research works presented now mostly aimed at simplify traffic scenario, which could not satisfy the complex real traffic scenario. In real traffic scenarios, the perception information is not accurate enough and the data has a certain delay. Therefore, the decision making model ought to be fault tolerant and adapt complex driving scenarios. Beyond that, the selection of appropriate decision frames and decision rules extraction algorithms to ensure that the decision result is the optimal solution under specific evaluation criteria. In order to understand real-time traffic scenarios, and make decision based on incomplete information, vehicle decision making system needs to consider multi-objective judgments and evaluation criteria. Furthermore, previous works on decision making of autonomous driving vehicle are mostly based on theoretical research and virtual simulation. However, virtual simulation environment is quite different from real traffic environment. Therefore, it is necessary to study and verify the behavior decision making of autonomous driving vehicle in real traffic environment. The real time performance and adaptability will be fully verified in real environment. In this paper, deconstruction and analysis for the behaviors of autonomous driving vehicle in certain traffic scenarios are implemented. Considering the psychological factors of drivers for driving decision making, quantitative analysis of attribute weights of driving behavior decision-making is carried out. Based on the presented analysis, an autonomous driving vehicle decision making model considering driving behavior revenue for is proposed. An autonomous driving system based on finite state machine is designed. In order to carry out real vehicle test, an automatic driving system framework is proposed. The proposed real time control system is built based on MATLAB/Simulink, which consists of perception module, decision planning module and motion control module. The main contributions of this paper could be summarized as: (i) Based on the analysis of the characteristics of structured road driving scenario, the operation process of human drivers is decomposed, and the effective driving behaviors are extracted and classified. Then, a driving behavior decision making framework for autonomous driving vehicle based on finite state machine is proposed.
(ii) To achieve lane change decision making for autonomous driving vehicle, lane change behavior characteristics of human drivers are extracted and analyzed combing the process of human driver lane change maneuver. Based on the analysis, multiple attributes such as driving efficiency and safety are considered, all attributes benefits are quantified and the driving behavior benefit evaluation model is established. (iii) To test the performances of the proposed decision making model, a series of real vehicle tests are implemented in different scenarios, the performance of the proposed method is accessed. The rest of this paper is organized as follows: In Section 2, the characteristics of driving behavior decision making for autonomous driving vehicle are analyzed; In Section 3, decision making characteristic of lane changing behavior is analyzed; In Section 4 and Section 5, a driving behavior decision making framework based on finite state machine and a decision making model based on driving behavior benefit evaluation are proposed; In Section 6, real vehicle tests are implemented to verify the performance of the proposed framework and decision making approach; Finally, conclusions and future work are presented in the last section.

Analysis on driving behavior decision mak-
ing of autonomous driving vehicle At present, the automatic driving system could not replace human drivers completely. It can be predicted that in the coming period of time, artificial driving vehicles and autonomous driving vehicles will coexist, and human drivers will still occupy the dominant position in traffic activities. Therefore, to ensure the traffic safety, autonomous driving vehicles driving behavior should consistent with human expectations. For the driving behavior decision making of autonomous driving vehicle, it is necessary to obtain environment information and vehicular state information at the same time. Based on the current dynamic traffic state and driving behavior criteria, real-time and accurate driving behaviors are generated. However, for the same traffic scenario, different scenario semantic understanding and decision making criteria will lead to different driving behavior even for human drivers (Muslim et al., 2018, AFANASIEVA et al., 2018. Therefore, analysis on driving behavior is an important prerequisite for the decision making of autonomous driving vehicle. The driving maneuver of vehicle driving on structured road has certain rules. It is usually a switch and combination of a few commonly used driving maneuvers. Combining the driving behavior characteristics of human driver, five basic driving behavior modes are constructed for the automatic driving system mentioned in this paper. The logical structure diagram of driving behavior switching is shown as Figure 1. Based on above logic framework, an automatic driving system is developed in MATLAB environment. Figure 2 shows the configuration of automatic driving system. The system is a distributed architecture. It could be divided into perception module, dynamic decision and planning module, motion control module. The dynamic decision and planning module is the core module of automatic driving system. This module obtains traffic and environment information from perception module. Based on the information, an optimal driving behavior and path is generated. Then the control command is sent to each actuator. Analysis shows that, driving behavior of autonomous driving vehicle on structured road are the switching of above mentioned behaviors, which triggered by different traffic scenarios. To extract switching rules, specific traffic scenarios need to be deconstructed according to driver habits, then the correlation of effect factors of switching rules are analyzed.

Analysis on lane change behavior of autonomous driving vehicle
Lane change behavior is affected by many traffic factors , Liu et al., 2019. For the same traffic scenario, sometimes multiple alternative driving behaviors exist. In this case, autonomous driving vehicle decision making system lack a screening mechanism to choose optimally. In the same scenario, if the logical rule-based alternative state is not unique, decision making difficulty or non-optimal decision making will occur in autonomous driving vehicle. Lane changing behavior could be divided into mandatory lane change and free lane change based on the existence of external interference . Mandatory lane change refers to behaviors that have to be taken to comply with traffic rules or to avoid collisions. Decision making of mandatory lane change is relatively simple, it could be generated based on traffic information and environment information. Free lane change is the behavior that driver executes lane changing maneuver without force. Free lane change behavior is usually adopted for greater driving space and higher driving efficiency. Behavior of free lane change is more random and uncertainty. Its decision criteria are not fixed. Scenarios that satisfy free lane change behavior usually contain other alternative behaviors besides lane change behavior. As a result, rationality and interpretability of generated behavior in decision making module are decline. For autonomous driving vehicle, the division between different driving behaviors needs clear boundary conditions. In free lane change scenario, multiple reasonable driving behavior choices cause conflict of driving state. The fundamental cause of conflict lies in overlapping boundary conditions. However, forced division conflict state will result in incoherent driving behavior for autonomous driving vehicle. Beyond that, the optimality of lane change behavior and other alternative driving behaviors should be judged. Therefore, by evaluating the benefits of all alternative driving behaviors and introducing other decision making theories above problems could be solved. Comprehensive scenario information is the basis for lane change decision making. The input information for lane change decision making consists traffic information, environment information and vehicular state information. All the processed information is input into lane change decision making module. The input information of lane change decision making is shown in Figure 3.
Where Q represents the set of all driving behaviors, E represents the set of all input events, represents state transition function, 0 represents the initial state, F represents the set of termination state. For autonomous driving vehicle, the switching of driving behaviors is triggered by external input event set E. The initial state of the vehicle entering automatic driving mode is free driving. In termination state, the vehicle enters failure parking mode or exits automatic driving mode. State transition function represents the transfer rules between driving behaviors. According to the feature analysis of finite state machine model, the input set E and the transition function are the key factors affecting the decision making performance. Therefore, to realize the driving behavior decision making of autonomous driving vehicle based on finite state machine, the triggering conditions and transition function of specific driving behaviors need to be quantitatively analyzed. Internal factors refer to the driving principle and driving expectation of ego vehicle. For human drivers, the purpose of switching driving behavior is to obtain benefits such as driving efficiency improve-ment, driving safety improvement and fuel consumption reduction. The transition condition between different behaviors is essentially the reflection of driving behavior expected benefits. Therefore, the expected benefits could be quantified by assessing different driving behaviors. By establishing driving behavior reward value evaluation model and comparing the rewards values of different driving behaviors, a final decision for autonomous driving vehicle is made. Benefits of lane change behavior could be summarized as following aspects: driving efficiency, driving space, driving safety and driving comfort. In addition, potential negative effects of lane change behavior should also be considered.

Lane change model based on driving behavior reward value evaluation model
represents the benefit value of current driving behavior, 1 … … an represent the benefit values of alternative driving behaviors. The benefit value of current driving behavior and alternative driving behavior could be expressed as follow: Where represent different weight coefficients, the values are calibrated manually based on test experience; represent different benefits. H is regulates constant representing the execution cost of driving behavior switching, it regulates the trigger to ensure the benefit of driving behavior switching is higher than current ones. It avoids the potential effect of frequently switching of driving behavior. Lane change behaviors are affected by multi factors. To analyze the specific factors, a lane change scenario model is established. The model is shown in

a. Driving space benefit
Driving space refers to the longitudinal distance between vehicles, in dynamic traffic scenario, time to collision (TTC) has been proved to be an effective measurement for the safe space between vehicles. The safe space could be expressed with time to collision (TTC) dependent function. Therefore, benefit function of driving space could be expressed as Rspace.  (1), [21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36]2020 Where represents the time to collision of ego vehicle and the preceding vehicle in lane; represents the threshold of the time to collision with preceding vehicle; represents the time to collision of ego vehicle and rear vehicle in the lane; represents the threshold of the time to collision with rear vehicle; 1 represents adjustment factor.

b. Driving safety benefit
For human drivers, free lane change behaviors are implemented to ensure driving space and view area. Research shows that at the same speed, human drivers have stronger desire to implement lane change maneuver when the volume and mass of front vehicle are larger . Therefore, in addition to consider driving space, the feelings of passengers should also be considered, combining the types of preceding vehicle to further optimize driving safety. The safety benefit of ego vehicle could be expressed as: Where 2 represents adjustment factor; is the parameter used to characterize the hazard level of preceding vehicle, in this paper, preceding vehicles are divided into four grades according to the volumes and types.

c. Driving efficiency benefit
Without considering the impact of traffic flow, the driving efficiency of autonomous driving vehicle is related to its velocity. The driving speed of autonomous driving vehicle is significantly affected by preceding vehicle. If there is no vehicle in the lane, the expected speed of ego vehicle could be regarded as the prescribed speed of the lane. For this reason, the speed benefit of ego vehicle could be expressed as: Where represents the expected speed of ego vehicle, represents the prescribed speed of the lane, 3 represents adjustment factor.

d. Fuel economy benefit
In addition to improving driving safety and efficiency, free lane change driving behavior may improve driving economy, so the driving economy benefit could also be considered as decision making factor. Driving economy is related to the change rate of ego vehicle acceleration (Li et al., 2014, Li et al., 2012. The evaluation of economy benefit could be expressed as:

e. Law benefit
For lane driving scenario, in addition to ensure driving safety, traffic rules must be obeyed. For instance, lane change cross solid lines and driving over speed are not allowed. Therefore, the benefit value is used to judge the rationality of driving behavior, it is defined as:

Real vehicle test
In order to verify the effectiveness of the proposed decision making model for autonomous driving vehicle, real vehicle test are carried out. Experimental vehicle is equipped with Differential Global Positioning System (DGPS) and Inertial Measurement Unit (IMU) module. This module achieves real-time positioning, navigation and the lateral speed measurement of experimental vehicle. Lidar, Mobileye Camera and Electronic Scanning Radar (ESR) are equipped to realize the real time perception of road environment. To manipulate the experimental vehicle automatically, an automatic driving system controlled by wire is equipped. The system could realize the automation of steering wheel, throttle and brake. All actuator control information, sensor and vehicular state real time information are interacted by Mi-croAutobox through CAN Bus. The architecture of the experimental vehicle platform is shown in Figure  5.
To Wang, P., Gao, S., Li, L., Cheng, S., Zhao, H., Archives of Transport, 53(1), 21-36, 2020 31 Lane changing behavior will bring higher driving speed, greater driving space and better driving safety, it has higher priority. Finally, experimental vehicle change to the left lane. The experimental results are shown in Figure 8. In Scenario 3, experimental vehicle is driving at constant speed in right lane and an obstacle vehicle is driving in the same lane with lower speed. Static obstacle is set in the left lane. In this case, experimental vehicle determines that the lane change space is insufficient, and outputs automatic car following command. The experimental results are shown in Figure  9. Figure 9 (a) shows the driving trajectory and steering wheel angle of experimental vehicle. Figure 9 (b) shows the speed of ego vehicle and the car-following distance of experimental vehicle in automatic car following mode. Figure 9 Wang, P., Gao, S., Li, L., Cheng, S., Zhao, H., Archives of Transport, 53(1), 21-36, 2020 33 In Scenario 4, experimental vehicle is driving at constant speed in right lane and an obstacle vehicle is driving in the same lane with lower speed. A static obstacle is set in the left lane. The initial distance between experimental vehicle and the static obstacle is greater than that between experimental vehicle and obstacle vehicle. In this case, the lane change space is enough, while lane change behavior will reduce driving efficiency and current driving benefits. Therefore, experimental vehicle keeps the automatic car following mode. The experimental results are shown in Figure 10.  Figure 10 (b) shows the speed of ego vehicle in automatic car following mode. Figure 10 (c) shows the driving mode and decision result. Figure 10 (d) shows the distances between preceding vehicle and ego vehicle, the distance between ego vehicle and static obstacle, respectively. The experiment verifies the feasibility, real time performance and multi choice filtering performance of proposed decision making model in dynamic traffic scenario.

Conclusions
In this paper, a driving behavior decision making framework for autonomous driving vehicle in structured road scenario is proposed. The proposed framework takes multi-sensor information as input, combines human behavior characteristics, then the structured road driving behavior mode switching logic for autonomous driving vehicle is constructed. With the constructed logical relationship of behavior switching, a decision-making framework for autonomous driving vehicle based on finite state machine is established. The proposed decision-making framework could output driving behavior instructions and switch driving behavior mode in real time according to traffic scenario information. It meets the basic automatic driving function in dynamic scenario. To optimize the lane change decision making of autonomous driving vehicle and solve the problem of finite state machine switching, a driving behavior To optimize the lane change decision making of autonomous driving vehicle and solve the prob-lem of finite state machine switching, a driving behavior benefit evaluation model is established by extracting and analyzing the operation characteristics of human driver. The benefit evaluation considers multiple driving objectives, and evaluates the comprehensive benefits of all alternative driving behaviors. Based on the benefit evaluation results of all driving behaviors, the performance of lane change decision making module is further optimized. In order to verify the performance of decision-making module of autonomous driving vehicle, a series of real vehicle tests are carried out. Long term experimental tests results show that the driving behavior decision making module proposed in this paper has good adaptability and usability. The decision-making framework could meet the requirements of automatic driving in real traffic scenario and output correct behaviors instructions according to real-time scenario information. Furthermore, the decisionmaking module has good real time performance and stability, it could stably output expected driving behavior in scenario with multiple choices.
In this paper, driving behavior benefit of single autonomous driving vehicle is analyzed and the optimal driving behavior is generated. The coupled dynamic effects of interacting traffic agents are not considered. However, the impact of traffic flow should be considered in actual traffic activities. In future research, real time traffic flow information will be combined to study the driving behavior decision making for autonomous driving vehicle.