Parameter Matching Analysis of Hydraulic Hybrid Excavators Based on Dynamic Programming Algorithm

In order to meet the energy saving requirement of the excavator, hybrid excavators are becoming the hot spot for researchers. The initial problem is to match the parameter of each component, because the system is tending to be more complicated due to the introduction of the accumulator. In this paper, firstly, a new architecture is presented which is hydraulic hybrid excavator based on common pressure rail combined switched function (HHES). Secondly, the general principle of dynamic programming algorithm (DPA) is explained. Then, the method by using DPA for parameter matching of HHES is described in detail. Furthermore, the DPA is translated into the M language for simulation. Finally, the calculation results are analyzed, and the optimal matching group is obtained.


Introduction
The demand for fuel efficient and low-emission hydraulic excavators has been increased due to the growing energy crisis and environmental deterioration recently. The appearance of hybrid excavator has the immense potential for reducing the fuel consumption, because it can eliminate the throttling loss theoretically and recover the braking or gravitational potential energy. Nevertheless, the system tends to be more complicated by introducing the hydraulic accumulator, which is used as another power source. The power flow is also changed due to the new power source and the recovery energy; hence, different parameters of the system units can result in different fuel consumption rate. It is important for improving the system efficiency and reducing the fuel rate of the hydraulic hybrid excavator by investigating the parameter matching method, which is also a good way to cut down the rated engine power and cost.
The parameter matching of power transmission system makes the parameters of the components in the system adjust to the working conditions by choosing the parameters of the components appropriately in the premise that the system working correctly can guarantee the system in optimal working condition, and then the overall efficiency of the system is improved; the purpose of energy saving is reached [1][2][3][4]. Static matching is the main way in existing matching methods. In this method, the maximum values in the working process of all actuators are used to choose the parameters of components. However, the working characteristics of frequent and large-scale power changes when the excavator works, to some extent, lead to oversize of components. However, the excavator has characteristic that multiple actuators of the excavator act at the same time, so the working conditions and system dynamics under the condition of composite actions have to be considered to make various components work in high efficiency and reduce the fuel consumption of the engine. In the existing optimization algorithms, the Dynamic Programming Algorithm (DPA) can solve the optimizing problems of any complex systems in theory, so it has been widely used, but DPA algorithm is mainly used to solve the optimal trajectory of controlled variables to provide reference for designing suboptimal controller [5][6][7][8][9]. One of the earliest researchers in this regard is Filipi et al. [10], who proposed a design optimization process in 2 Journal of Applied Mathematics two stages for a parallel hybrid medium truck. Then, Cross used this algorithm to extend the application in parameter matching [11]. In this work, the Hydraulic Hybrid Excavator based on CPR combined switched function (HHES) is investigated. CPR means Common Pressure Rail which is similar with the electric grid. It is divided into two lines including high and low pressure pipelines. All of the hydraulic actuators are connected with the two lines in parallel; it means that it is convenient to arrange the hydraulic components. Moreover, it not only eliminates the throttling loss in the theory aspect, but can also recover the braking or gravitational potential energy. Hence, applying this structure on the hydraulic excavator is a promising hydraulic architecture in the aspect of saving energy. However, because HHES is a new system, there are only a few relevant research papers published on parameter matching. In this paper, the optimal control principle based on DPA is first introduced to the parameters optimization matching research of HHES. The minimized engine fuel consumption in typical working condition is treated as the optimization goal. Considering the influence of the factors such as the efficiency of components and system dynamics, the minimum fuel consumption of various components parameters matching mode will be excavated most possibly by choosing a group of optimal parameters, and the method in this paper can guarantee that the fuel consumption of the different components parameters can be compared fairly without considering the influence of control method.

Dynamic Program Algorithm
Principle. DPA algorithm is an effective computing method combined with sorting decision method and optimization principle. In 1953, American mathematician Robert Bellman proposed the optimization principle in his writing "An optimal policy has the property that, whatever the initial state and optimal first decision may be, the remaining decisions constitute an optimal policy with regard to the state resulting from the first decision" [12]. According to this theory, the sorting decision can be applied in a complicated system, and "optimization procedure" is used at each level so as to achieve the overall optimization goal. Now, the basic principle of sorting decision is simply illustrated by Figure 1. For Figure 1, numbers close to the connecting lines between two points are the distance of two points. The red lines in Figure 1 show the trajectory between and : * = where constitutes the initial control and * represents the shortest distance from to . So we can calculate every possible route and compare to get the shortest distance. However, if the number of the points is large, it tends to be impossible to get the suitable result through the calculation process: so we can get * = 18.
The application of optimization algorithm can reduce the number of trajectories to be considered, as shown in Figure 1. Taking the reverse calculation from point as an example, if optimal path passes state point , the optimal path between and is from the above node to (the required time is 2+5 = 7) instead of the path from the below node to (the required time is 6 + 6 = 12), then the minimum cost and optimal path from this point to terminal point are determined. By repeating the calculation process to all stated points, the minimum costs and optimal paths for all state points can be calculated, and the optimal path of the whole process can be obtained until the calculation of point is finished. Because of the iteration method used in DPA, the main application background is for discrete system. For continuous system, it should be converted into discrete system, and the optimal solution can be solved after discretization. For a given system, the system dynamics can be described aṡ= where is the state vector, is the control vector, is the disturbance vector, and the subscript is the time instant. Generally, to simplify the problem, the system dynamics can be described in a discrete domain; in other words, differential equations are replaced by difference equations: Generalizing the principle of optimal control to discrete time systems results in [11], * ( , ) = , +1 ( , , ) + * +1, ( ( + 1)) , (5) where * is the minimum cost of operation from k to N for a specific state ( ) and control ( ). The minimum cost of operation for all combinations of control is calculated from * ( ) = min the two lines. The rotating loads can be controlled by regulating the displacement of hydraulic pump/motor, while the linear loads are actuated by hydraulic transformer because the hydraulic cylinders are hard to change displacement normally [13][14][15][16][17]. Since the system includes secondary components and accumulators, energy can be recovered when the actuator brakes or falls and then is stored in the accumulator. Hence, the excavator possesses two kinds of power source. The low fuel consumption can be obtained by using adopted appropriate control strategy. In this configuration, the former three fixed displacement motors, which are used for swinging and driving, respectively, should be replaced by three hydraulic pump/motors [18,19]. However, the key component is not popular and expensive. We propose a new architecture which uses on-off valves to switch the hydraulic transformer control, and Figure 2 shows the schematic. The reason for this modification is the working condition of excavators, because the travel part and the arm cylinder or the bucket cylinder are not working at the same time. So the fixed displacement motors which are used for traveling in the original nonhybrid excavator can remain. There are travel 1 and arm cylinder in Group 1, and Group 2 includes travel 2 and bucket cylinder. Moreover, two sets of valves, in which there are four on-off valves, are used to switch the hydraulic transformer control motor or cylinder. Hence, not only the energy-saving characteristic is remained, but also the cost can be reduced because of the manipulation of the fixed displacement motor instead of variable displacement pump/motor. Furthermore, it is easier to modify based on the existing manufacture process.

Application of DPA for Hydraulic Hybrid Excavators
The purpose of this paper is to calculate the component parameter configuration that minimizes the fuel consumption in typical working condition of the excavator by DPA algorithm, and a 5 ton LS-control prototype is used as research object, and the existing components in prototype should be changed as less as possible to reduce the reform cost. The main components of the entire hydraulic system include constant pressure variable pump, hydraulic accumulator, and hydraulic transformer, and the actuators contain boom hydraulic cylinder, bucket hydraulic cylinder, arm hydraulic cylinder, swing motor, and travel motors. By using switch control principle, the actuators except for the quantitative swing motor are reserved, and the quantitative swing motor is replaced by variable pump/motor. Because of the limitation of current technical level, the hydraulic transformers have not been applied widely, and the displacement of hydraulic transformer is not a choice. In addition, the main pump of original system also has the function of electronically controlling variables, so it has been in use. Thus, Table 1 shows that the components parameters need to be optimized matching in the entire system. Installation space of a 5-ton excavator is limited, so the optional maximum volume of the hydraulic accumulator is determined as 40 L. The decision of swing motor mainly refers to the existing parameters of the hydraulic pump/motor. We need to know the relevant data of circulatory working condition when using DPA algorithm. In addition, we need to determine the state and controlled variables of the system, and the dynamic state equation also needs to be established.

Working Cycle.
The standard working cycle is used for calculation. This cycle represents an excavator digging a load of dirt, rotating and releasing the load into a truck or onto a pile, and then returning to its initial position. It should be noticed that the travel part is not considered in this paper. This process is divided into four parts. Figure 3 shows the velocity of each actuator, respectively [20]. During the beginning part, the boom cylinder and the swing keep the position basically, but the arm cylinder and bucket cylinder move out to dig. Then, the boom cylinder extends, and the swing rotates to lift the dirt and prepare for dumping. Next, the bucket cylinder retracts to dump the dirt. Finally, the swing rotates back, and the boom cylinder retracts to go back to the initial status.

State Variables and
Controls of the System. The critical state variables of the system can be selected by (7), and Table 2 shows the symbol and the meaning = [ eng , , 1 bm , 2 bm , 1 , 2 , 1 bk , 2 bk , ℎ , According to the DPA principle, if all the state variables in the state matrix we establish are unknown, then it is difficult to realize the optimization process because the calculation amount will increase rapidly [11]. Hence, according to In addition, the critical control of the system is The controls can also be divided into two parts: one is being decided by the working cycle, and the other is the optimizing trajectory. In order to finish the working cycle, the torque and force requirement should be met. For instance, 2 would be decided during each step after the state variable ℎ is confirmed by the next equation: where is the requirement torque of the swing and is the torque loss.
Hence, the free controls are chosen as

Discretization of the System.
After the state and controlled variables are determined, we need to ensure the scope of the state and controlled variables and perform the mesh generation. The rotating speed range of the engine is determined by the inherent curve of the original engine, and the maximum value of the high pressure pipe line is defined by the allowable maximum pressure 350 bar of the components. The interval of the engine rotating speed is 100 rpm, and the interval of the pressure in high pressure pipe line is 5 Mpa, both the range of the controlled variables 1 , 1 being from 0 to 100. The grids are shown in Figure 4 [21]. Generally speaking, the more dense grids, the more accurate results, but the calculated amount will be greatly increased. The purpose of this paper is to obtain minimum fuel consumption in the same cycle. Dynamic performances of the variable displacement mechanism in pump have not been considered, so it is more reasonable to choose the similar time interval with the variable displacement mechanism, since the frequency of the variable displacement mechanism is 5 Hz, and dt is chosen as 0.2 s.
Due to the big difference among the different components, especially for the excavator which is used widely, we consider the cost combined with the object of optimal fuel consumption by using weight factor method, where represents the additional cost for different components.

Engine Dynamics.
The engine dynamics is a complicated process. It is difficult to state the detailed procedure by using mathematical analysis, especially, how to model a model is not the object of this work. Hence, one effective method which is based on the experience data is adopted. It means that the main torque types such as friction torque and loss are obtained from the lookup table which is calculated from the exact speed and torque. For the HHEC, the only load torque of the engine is the torque of the main pump and the friction torquė where = (( ℎ ⋅ 1 )/(2 ⋅ )) 1 is the torque of the main pump, represents the maximum torque for different engine speed, loss is the loss torque which is a lookup table by using the experimental dates, and is the friction torque.
A discrete difference equation is required by using DPA, so the continuous differential equations are approximated as

Pressure of the High Pressure Pipe.
The pressure buildup equation describes the change of pressure in the system with respect to time. Because all of the high pressure sides of components in CPR are connected together, every component flow rate should be considered to calculate the pressure change. In detail, the high pressure pipe contains a main pump, hydraulic accumulator, and the actuators which are depicted in Figure 5. The direction of the flow rate is defined by positive if coming from the component to the high pressure pipe and negative for the opposite direction. Then, the pressure is calculated by the following equation, and it is noticed that again travel motors are omitted in the part: where represents the index of each actuator, such as bucket, arm, and boom cylinders; ∑ 3 =1 is the total capacity which includes each port of the HTs, every cylinder volume of the rod side, and the pipe line volume; the initial volume of the motor/pump is represented as ; sk is the stroke of each cylinder and is the displacement of every cylinder; represents the output flow rate of the main pump; 2 is the flow rate of the rod side of each cylinder; 2 is the flow rate which goes into the motor/pump and is the total flow rate of leakage. HT is the flow rate that goes into the HT, respectively.

6
Journal of Applied Mathematics Also, in the previous equation accu is defined as the capacity of the accumulator which is the function [22] Again, the discrete difference equation is as follows: where 2 is confirmed by the working cycle which equals the velocity times to the area of the rod side for each cylinder. However, the way to calculate HT should be pointed out. The SHT of boom cylinder is chosen to show the process. The method for the other two HTs is the same. The boom cylinder is controlled by regulating the port plate angle of the HT in HHEC. Firstly, we define the transformer ratio, and the next equation is [ where net bm = 1 bm ⋅ 1 bm − 2 bm ⋅ 2 bm means the net force of the boom cylinder because all of the pressure and the area are known according to the cycle data.
Moreover, the flow rate of and can be obtained by After considering the leakage coefficient in total, where 1 bm equals the velocity times to the area of the bore side for boom cylinder, and it is also the known data.
3.6. Programming. Figure 6 shows the whole flow chart of the program [21]. The program can be divided into three loops, in which the inner is the control loop and the middle is the state loop; the outside ones are the district layers which are divided by district time dt. Then, every state in per layer should be calculated by using all of the controls through the dynamic equations. During the calculation, the control values result in the result which exceeds the state domain that should be abandoned, and the calculation should go on by using the next control values. For those accepted controls, the fuel consumption for that state and the controls should be added. After comparing all the controls in that state, the minimum one is stored. The middle loop includes the same cycle for each state. Figure 7 shows the process in detail. represents the step. The calculation begins from the end. In fact, the dynamic programming is one type of iterative algorithms. It begins from the end; hence, the initial value must be given. In this work, the initial value and are set to 0. Some states are unavailable, which are represented by red rectangles. The black cycles represent the minimum fuel consumption values corresponding to those states, respectively. And the blue triangle means the optimal value in the step. All of the fuel consumption values (matrix ) in each step should be used as  the initial value for calculating in the next step. For example, the matrix 1 is used for − 1 step. It should be noticed that when calculating the new state by using the controls, the values may not fit well in the mesh grid. Hence, the bilinear interpolation algorithm is introduced to calculate the fuel consumption for [24]: where ( , −1 ) ( −1 , −1 ), ( , ), and ( −1 , ) are the fuel consumption which are coming from the former results.

Simulation Results
There are 60 combinations of the three parameters in total. Hence, the simulation runs 60 times for each group of parameters. It takes about 5 hours once by using a single core computer. In order to eliminate the influence of the initial state, 5 cycles are input into the simulation, but only the middle three are used to compare the fuel consumption. Figure 8 shows the relationship among 0 , 2 , and 0 . It can be found the general tendency, with the increment of 0 , the fuel consumption decreases. However, the fuel consumption reduces slowly after 0 approaches 40 L. 2 is not independent from the other parameters, but it is coupled with 0 and 0 . In general, the fuel consumption reduces with the increment of 2 , and it shows the similar tendency with 0 ; that is the fuel consumption reduces slowly after 0 approaches 40 L. Furthermore, the precharge pressure is a key variable to impact the fuel consumption. The optimal pressure value locates from 100 bar to 150 bar normally according to the simulation results.
In order to state it in detail, different fuel consumption values corresponding to different precharge pressure values of the 16 L accumulator are plotted in Figure 9, which shows that the minimum fuel consumption appears in 150 bar. All of the simulation results show the similar trend. This is because the energy storage reaches the maximum around this pressure level To get the precharge pressure which results in the maximum energy, the derivation of E is calculated as follows: It means that if the maximum pressure and the parameter are decided, then the optimal precharge pressure can be obtained from (25). Hence, the same accumulator under the optimal precharge pressure can store the maximum energy, and then the fuel consumption can be reduced.
In general, large components have low fuel consumption under the same condition. Because in this algorithm, the minimum engine fuel consumption is taken as the optimizing objective, so every state pursues the highest efficiency. However, for key components, such as the axial piston type component, the efficiency gets lower with the pressure increasing, then when we want to achieve the same torque, large components can reach the purpose of efficiency improvement in smaller pressure conditions; however, we need to take into account the price growth of complete machine. After the comprehensive comparison, a set of parameters we choose are 2 = 40 mL/r, 0 = 15 Mpa, and

Conclusion
Optimal parameter matching results for HHES were analyzed with the aim of reducing the fuel consumption and modification cost. Firstly, a new architecture HHES is presented which not only keeps the advantages of the hydraulic hybrid excavator but also reduces the modification cost. Then, the DPA was applied in the matching process successfully. The results show that the fuel consumption reduces with the increment of the 0 . And the similar tendency is obtained for the swing pump/motor. However, it is coupled with 0 and 0 . The precharge pressure shows the independent relationship for the fuel consumption among other parameters. The optimal value is located around 10∼15 Mpa under the conditions that the maximum pressure is 35 Mpa and is 1.25. By combining the cost factor, the optimal group is obtained which is 2 = 40 mL/r, 0 = 15 Mpa, and 0 = 16 L. The future work will focus on the optimal trajectory of the state variable based on the dynamic programming result firstly. Then, design the suboptimal control strategy according to the optimal trajectory and test it in the real excavator.