Load-In-Load-Out AGV Route Planning in Automatic Container Terminal

Efficient automatic guided vehicle (AGV) scheduling is the key to increase the throughput of automated container terminals. Traditional transport strategies cannot guarantee that AGVs are fully loaded during their traveling between the dock and the container yard, which leads to the insufficient utilization of AGVs. A load-in-load-out AGV route planning mode provides two-way loading between the dock and the container yard and thus improves the efficiency of container terminals. In this paper, a load-in-load-out AGV route planning model is designed with the help of a buffer zone where an AGV can carry at most two containers. A simulated annealing algorithm is used to solve it. Comparisons with the popular generic algorithm and particle swarm algorithm are made. Simulation results show that the proposed algorithm can effectively solve the problem and realize the bi-directional loading of AGVs, which is of great significance to improve the efficiency of the production of automated container terminals.


I. INTRODUCTION
As one of the most basic logistics and transportation modes, sea transportation has been playing an important role in economic trade and cultural communication among worldwide countries. With the development of the current frequent international trade, container terminals need to use advanced technologies to increase their efficiency and throughput. Efficient automatic guided vehicle (AGV) scheduling strategies are the key to ensure the high efficiency of an automatic container terminal, which has been studied by many scholars. They can support the full use of port resources and realize the efficient operations of the quayside and yard.
Studies about AGV scheduling can be divided into three aspects: path planning, real-time scheduling, and resource The associate editor coordinating the review of this manuscript and approving it for publication was Yanbo Chen . allocation optimization. Path planning means scheduling AGV to complete transportation tasks. It can use intelligent optimization algorithms or hybrid algorithms to plan the AGV routes. Mohammad and Saeed propose a mathematical model that is composed of a job shop scheduling problem and a conflict-free routing problem. A two-stage ant colony algorithm is also proposed to solve it [1]. Based on the characteristics of the A * algorithm, Zheng et al. optimize a node search method and use an angle evaluation cost function to find the path with the least inflection point [2]. Jerald and Asokan implement a non-traditional optimization technique called the adaptive genetic algorithm to do simultaneous scheduling [3]. Liu proposes an algorithm based on an immune algorithm and constructs AGV path planning by using a bipartite graph [4]. Chen et al. propose a new navigation method based on a fuzzy neural network by defining safety measures [5]. Cordeau et al. provide a unified tabu search heuristic algorithm for time windows and multi-warehouse vehicle paths [6]. Real-time scheduling is a strategy for avoiding collisions in a planned path. It implements deadlock-free and collision-free transport rules. This will achieve the goal of increasing the efficiency of the container terminals. Ayoub et al. propose a hybrid approach to solve the scheduling and path selection of automatically guided vehicles [7]. Nishi et al. propose a Petri net decomposition method for scheduling and conflict-free routing simultaneously for two-way automatic vehicle systems in dynamic environments [8]. Dimitri et al. propose a new strategy based on time logic to avoid collisions by delaying AGV operations [9]. Shi et al. present a two-stage scheduling strategy for offline shortest path library generation and online optimal scheduling scheme generation [10]. Ho develops a strategy that not only can prevent the collision of vehicles but also can avoid the disadvantage of fixed-zone strategies [11]. Gan et al. aim to minimize the end time of AGV tasks and the time of invalid operations and optimize the unloading operations of automatic container terminals [12]. Luo and Ni propose a method based on ordinary Petri nets to design the programmable logical controller to prevent collisions between vehicles in the automatic vehicle system [13]. Resource allocation focuses on disposing of containers in the shortest time with limited transportation resources. Singh et al. adopt a new scheduling rule and simulate the distribution efficiency and uniformity [14]. Toshiyuki et al. study the scheduling and path planning of AGV with a capacity [15]. Hadjar and Soumis consider a widespread branch-and-price approach to solve the multiple depot vehicle scheduling problems with time windows [16]. Confessore et al. propose an approach for solving the dispatching problem in an AGV system. The problem is modeled through a network by relying on the formulation of a minimum cost flow problem [17]. Kim and Chung present an analytical model to design a tandem AGV system with two-load AGVs [18]. Hassan and Edward define a scheduling problem for automated guided vehicles in container terminals and formulate as a minimum cost flow model [19]. A delay in one job may affect all the subsequent jobs served by the same quay crane. To handle this assignment problem, Song and Huang propose a hybrid metaheuristic method [20]. However, the aforementioned studies cannot guarantee that AGVs are fully loaded during their traveling between the dock and the container yard, which leads to the insufficient utilization of AGVs.
With the increasing throughput of the port, the traditional operation mode will cause a waste of time and resources. This will affect the whole transport link of the quayside bridge, AGV transport zone, and yard bridge. To the best of our knowledge, the existing big container ports cannot guarantee that AGVs are fully loaded during their traveling between the dock and the container yard, which leads to the insufficient utilization of AGVs. Also, in the existing studies few scholars study the considered fully loaded mode. Thus, this paper proposes a load-in load-out route planning for AGVs providing two-way loading between the dock and the container yard. It is designed with the help of a buffer zone. A simulated annealing algorithm is used to solve it. It can undoubtedly improve the efficiency of port transport. In this paper, we consider the strategy that an AGV can accommodate and carry two containers at the same time. In recent years, abundant achievements in various areas including scheduling in slab yard [21], manufacturing [22]- [25], transportation [26]- [28] and cooperative systems [29], [30] have been made with the help of intelligent optimization algorithms. We use a simulated annealing algorithm to solve the model and adopt a repetitive locations combination strategy to improve the efficiency of the algorithm.
The rest of the paper is organized as follows. Section II presents the formulation of the problem where an AGV can carry two containers. Based on a seaside buffer zone, it designs a load-in-load-out AGV scheduling model. Section III proposes a solution algorithm for the model based on a simulated annealing algorithm. Section IV makes a comparative analysis of the results. Finally, Section V concludes this paper and discusses the future work.

II. PROBLEM FORMULATION
The model studied in this paper is the multi-AGV path optimization model for a single cargo ship. Fig. 1 shows the part of seaside transportation area in Qingdao Qianwan Container Terminal. In this figure, there are some quay cranes, a ship on the left of quay cranes, and some AGVs under the quay cranes. At the seaside of the port, the task of AGVs is to transport imported containers from a quay crane to a yard crane, unload the containers, and return to the quay, or to transport exported containers from a yard crane to a quay crane, unload the containers, and return to the yard. We design a load-in-load-out AGV transportation (LILO) mode in this process. After the imported container is unloaded at a yard, the AGV loads another exported container and returns to the quay as shown in Fig. 2. The goal is to complete all containers transfer tasks between the sea-side and the yard in the load-inload-out mode while minimizing the total traveling distance of all AGVs. A quayside buffer can ensure loose coupling between quayside work and trolley path planning, and thus the efficiency of port work can be increased. The buffer at  the quayside bridge is designed in Fig. 3. Taking the loading process as an example, quayside buffers store containers. Containers are allocated from the bridge crane to the carriage, and the carriage can move from side to side between the tracks so that there is no requirement. We give some assumptions about the model: 1) There is one ship, several AGVs, and a similar number of containers that need to be transferred from quayside to the yard and from the yard to the quayside. 2) Each AGV can carry two containers at most, and each task needs to transport containers from the quayside to the yard or from the yard to the quayside.
3) The quayside buffer can automatically realize the container loading and unloading of AGVs.
First, we give the notations that are used in the paper below.
represents a subset of a multiple set consisting of v i , i ∈ N; 6) X : X ∈ {V -{v 0 }} * represents a set of locations where each location in the yard needs to receive a container that is unloaded from the ship; 7) Y : Y ∈ {V -{v 0 }} * represents a set of locations where each location in the yard has one container that needs to be transported to the ship; 8) |S|: represents the number of elements in set S; 9) c: The maximum number of containers for each AGV; 10) m: m ∈ N is the total number of AGVs; 11) k: k ∈ N m represents an index of an AGV; 12) d ij : represents the distance between location v i to v j ; 13) η k : η k = x 1 x 2 . . . x j represents a sequence of locations that the kth AGV passes through during its transportation, and for the convenience we denote x i ∈ η k ; 14) Q k : Q k = i,j d ij represents the distance traveled by the kth AGV; 15) l k i : it represents the number of loading containers at v i in η k , l k i ∈ {0, 1, 2, . . . , c}; 16) u k i : it represents the number of unloading containers at v i in η k , u k i ∈ {0, 1, 2, . . . , c}; 17) n k i : it represents the number of the containers carried by the kth AGV before its next loading or unloading task at v i in η k , n k i ∈ {0, 1, 2, . . . , c}. According to the LILO mode, the objective function is the shortest distance of AGVs that can accomplish all tasks.
The constraints are as follows.
Constraint (2) means that AGV should fully load into and out of the quayside bridge, and this meets the requirements of the LILO mode. (3) means that the maximum carrying capacity of the AGV is no more than c. (4)-(5) restrict the number of loading and unloading containers for each AGV. (6) prohibits the illegal operation of neither loading nor unloading in the process of transportation at a location in the yard. (7) means that each unloading container at a location can be accomplished. (8) means that each loading container at a location can be accomplished. VOLUME 8, 2020

III. PROPOSED ALGORITHM
First, we analyze the scale of the solution space of the problem. When c = 1 and X = Y = n. A basic sequential operation of an AGV is ''unloading → loading'' denoted by UL for short. Both the numbers of solutions for unloading containers in X and loading containers in Y are n!. When we have m AGVs, the solution space is O(n! 2 × C m−1 n−1 ). When c = 2 and X = Y = n. There are two basic sequential operations, i.e., ''unloading → loading → unloading → loading'' and ''unloading → unloading → loading → loading'' that are respectively denoted by (UL) 2 and U 2 L 2 for short. We 2n 4 assume that the number of carriage times by AGV is t, then t = 2n 4 , there are altogether 2 t modes of transport. Both the numbers of solutions for unloading containers in X and loading containers in Y are n!. When we have m AGVs, . It can be seen that the time complexity level of the exhaustive method is O(n!). When the scale of the problem becomes larger, the traditional algorithm cannot solve the problem within an acceptable time. Therefore, simulated annealing algorithms for solving the problem are presented in this paper.
We adopt a two-stage coding scheme. It divides the code into two parts. The first part is an integer sequence representing an order of locations. The second part represents the number of locations traversed by AGV. Fig. 4 shows an example of the two-stage coding scheme. Now, we propose a code decomposition and combination (CDC) strategy. In order to make the algorithm have good convergence and avoid falling into local optimum, we screen the same locations in a scheduling process. At the same time, we analyze the condition that an AGV could carry two containers.

A. SIMULATED ANNEALING ALGORITHM BASED ON LILO
A simulated annealing algorithm was proposed by Metropolis et al. [31] in 1992. The idea comes from the similarity between the annealing process of solid matter in physics and general combinatorial optimization problems. The steps of the simulated annealing algorithm to solve this problem are as follows: a) Constructing an initial solution; b) Generating new solutions from the random perturbations of the old ones; c) Calculating the difference between the new-generated and the old solutions; d) Updating the solution according to a Metropolis rule; e) Going back to b) until reaching the maximum number of iterations.
Initial Solution Generation: Step1: Combining repeated locations. To reduce the size of the initial solution space, we can combine successive operations in the same yard. If c = 1, we select the same locations from X and Y to form C, and remove the locations of X and Y appearing in C to form X and Y . If c = 2, for every two identical locations in X , both locations are screened out from X and put into CX. For every two identical locations in Y , they are screened out from Y and put into CY. By referring to CX and CY, elements in X and Y are removed to generate X and Y , the same elements are screened from X and Y to form C, and the locations that appear in C are removed.

C. ALGORITHM PROCESS
In addition to the symbols defined in the problem model and the initial solution generation process, symbols and formulas in the simulated annealing algorithm are as follows.
Input: locations V , set of unloading locations X , set of loading locations Y , the number of AGVs m, the maximum capacity c, the number of iterations per temperature L, the initial temperature T 0 , the end temperature T end , cooling rate T v .
Output: legally scheduled travel distance Z .
Step1: Code split and set the initial value. Elements in α 1 are composed into unloading sequence α x and loading sequence α y , we initialize l = 1, and α best = α; Step2: Generate new solutions. Two randomly selected elements in α x and α y are exchanged to produce a new solution; Step3: Accept new solutions. We combine α x and α y into a new solution α , let dp = Pathlength(α )-Pathlength(α). If dp < 0, then α = α , otherwise, we accept the α with a probability of -dp/c. If α best > α, α best = α; Step4: If l < L, then l = l + 1 and return Step2, else enter Step 5; Step5: Iteration. T 0 = T 0 * T v , if T 0 < T end , then return Step1, else end the algorithm and map α best back to the sequence of locations F(v) −1 , output the travel distance Z .
The flowchart of the algorithm is shown in Fig. 5. Since the algorithm mutates, crosses, and reverses sequence codes in this algorithm process, there will be illegal solutions after the operation. For example, support c = 1, 1 represents the quayside location, 5 and 4 represent unloading locations, 3 and 6 represent the loading location, the sequence is [1 5 3 1 4 6 1] meaning that an AGV loads a container from the quayside and unload at the location 5 in the yard, then loads another container from the location 3 and returns to the quayside. The operation of locations 4 and 6 is the same as described above. In the mutation operation, it selects the 2 and 3 positions in a sequence and the sequence after mutation is [1 3 5 1 4 6 1]. At this time, the sequence [1 3 5 1] is illegal, because it is illegal to load the container first and then unload the container when the load is full. In order to solve the above problems, we adopt a CDC strategy to fix the operation of each position in the sequence. In fact, in the case of c = 1, the operation of each position in the sequence is fixed, but the location corresponding to this position is uncertain. In other words, position 2 in the example can only be an unloading operation, but the location of position 2 may be 5 or 4. First, the unloading sequence [5 4] and loading sequence [3 6] are separated according to the sequence. Then, mutation and reversal operations were performed on the two sequences respectively. We assume that the two sequences after the operation are [4 5] and [3 6], we can get a sequence [1 4 3 1 5 6 1]. The above operation solves the problem of generating illegal solutions during mutation or reversal.
In the case that c = 2, there are two situations in one carriage: (UL) 2 and U 2 L 2 . At this point, the sequence will also produce a solution that does not conform to LILO when performing cross operations. For example, there are two sequences [1 2 3 4 5 1][1] and [1 2 5 3 4 1] [2]. The first section represents the scheduling sequence of the AGV, while the second section 1 represents the transport strategy of the car in this transportation as U 2 L 2 , 2 means that the transportation strategy is (UL) 2 . 1 represents quayside location, 2 and 3 represent unloading locations in the yard, 4 and 5 represent loading locations. During the crossover operation, it is assumed that we select the 3rd position in the sequence, and the sequence after the crossover is

IV. SIMULATIONS
The parameters of the simulation, including the size of the AGV transportation area, and the number of storage yards according to the Qingdao New Qianwan Container Terminal, China. The tool used in the simulations is MATLAB R2019a. VOLUME 8, 2020 To measure the optimization ability of the model, we set c = 1, 2, and the initial values m = 2, t max = 100, T 0 = 1000, T end = 100, T v = 0.9, and L = 100. The results are shown in Tables 1-4. Within the scope of small-scale locations, an optimization precision of simulated annealing algorithm (SA) can reach 100%. With the increase of problem scale, SA can significantly reduce the time required for solving problems. When the scale of locations continues to increase, the results cannot be obtained by exhaustive search (EA) within an acceptable time. At this time, SA algorithm can find a good solution to realize LILO in a very short time, which reflects the good optimization ability of the algorithm.    According to the LILO mode and algorithm optimization process in this paper, we made a comparison with the other two popular algorithms, i.e., genetic algorithm and mixed particle swarm optimization. When |X |, |Y | = 16 and 20, we set the initial parameters as m = 3, c = 2, T max = 300, T 0 = 2000, T end = 0.001, T v = 0.95, L = 100. After running the algorithm multiple times, we adopt the best of them as a final result. The simulated results are shown in Figs. 6. SA has the best effect in finding the optimal solution although the convergence speed is a little bit slower than that of a genetic algorithm.
In order to measure the influence of the increasing number of task points on the convergence performance of the algorithm, the evolutionary process of the three algorithms is compared and analyzed when |X | and |Y | = 8, 12, 16, and 20. We set the initial value of parameters as: m = 2, c = 2, T max = 300, T 0 = 2000, T end = 0.001, T v = 0.95, L = 100. Simulated results as shown in Tables 5 and 6 and    Fig. 7, the genetic algorithm converges on 12th, 34th, 75th and 124th generations ( Fig. 7 (a)), hybrid particle swarm optimization converges on 63th, 115th, 220th and 203rd generation ( Fig. 7 (b)), simulated annealing algorithm converges on 88th, 212th, 219th and 229th generations (Fig. 7 (c)). It can be seen from the simulations that as the number of locations increases, the convergent number of iterations of the three algorithms also increases, but the algorithm can still solve the problem in a short time, which shows the applicability of the algorithm in this problem.
The scenario in this paper concerns a single ship, so the number of containers is not very large. For small-scale problems, we can get the exact optimal solution; and for medium-scale problems, we have compared with classical intelligent algorithms, and get good solutions.

V. CONCLUSION
To improve the working efficiency of port transportation, we propose a new AGV scheduling strategy -load in load out mode. We construct the mathematical model of the mode of LILO and solve the problem based on the simulated annealing algorithm. According to the simulated results, the improved algorithm is effective in solving the problem. To the best of our knowledge this is the first study a load-in-loadout AGV route planning mode providing two-way loading between the dock and the container yard. It can improve the energy-efficiency of AGVs as well as of efficiency of container terminals. The proposed algorithms can be used in automated container ports. A necessary condition is that the numbers of imported and exported containers are similar. With further exploration of the model, we will consider multiple ships, the specific design of the quayside buffer and use a time window to study the task scheduling process. At the same time, when the problem scale becomes very large, the simulated annealing algorithm may not find the optimal results due to the blind search issue, which is also a problem to be solved in our future work. We can also use some novel heuristic algorithms to solve the model to improve the accuracy of the problem. To improve the applicability of the model, we will also consider detecting path conflicts in AGV scheduling.