Simulation optimization based ant colony algorithm for the uncertain quay crane scheduling problem

Article history: Received September 2


Introduction
Maritime transport, by its possibilities of consolidation, plays a crucial role in the international trade, and has become one of the essential actors of globalization.Today, the global volume of freight carried by sea is estimated at over 90%.A growth that is explained by several factors such as the modernization of logistics, the harmonization of equipment and the large capacity of the world fleet of container ships that has exceeded 17.5 million TEUs in 2014 (UNCTAD, 2014).This staggering growth, place the shipping market in the second global position after the agro-food market with a turnover of more than 1.5 trillion Euros (UNCTAD, 2016).This brings new investments but also creates strong competition between container terminals that belong to the same geographical area.One of the key points for improving the competitiveness of a container terminal is the minimization of the time spent by the vessels at the port, denoted by the vessel turnaround time, which depends mainly on the efficiency of planning methods used by the port operators in the handling, storage and transport operations (Tongzon and Heng, 2005).Thus, port operators are on a continuous search of adequate solutions that can address this challenge.Especially for the seaside operations, where arises three major problems: the Quay Crane Assignment Problem (QCAP), the Berth Allocation Problem (BAP) and the Quay Crane Scheduling Problem (QCSP) (Meisel, 2009).
This work is devoted to the study of the Quay Crane Scheduling Problem (QCSP), which consists of determining the best schedule of the unloading/loading operations of containers by quay cranes assigned to a vessel such as the overall handling time is minimized.In the QCSP, vessels are partitioned longitudinally into bay areas, each of them is carrying a certain number of containers.Containers with the same characteristics; namely: weight, origin, destination and type of operation (loading or unloading), are usually located adjacent to each other and they are considered as a single task to facilitate their handling operations (ExpóSito-Izquierdo et al., 2013).Precedence constraints are defined on tasks in order to respect the stacking plane of the containers, and quay cranes move on the same track therefore, they cannot pass each other and a safe distance must be kept between them to avoid congestion.According to the level of aggregation considered on the definition of a task, we can distinguish in the existing literature between three classes of QCSP: QCSP with container groups where a task involves the handling of a group of containers within a bay, QCSP with complete bays where a task refers to all containers within a bay and the QCSP with bay areas where a task represents all containers within a connected area of bays (See Fig. 4 in Bierwirth & Meisel, 2010 for more details).The QCSP has been the subject of several studies.However, as shown in the next section, most of previous works related to the QCSP study the deterministic version.Even if, in the real-life situations the port operators have to deal with uncertainties in a range of factors such as handling time of tasks and travel time of quay cranes, those perturbations reduce the quality of deterministic solutions and have a significant impact on the overall vessel turnaround time.Therefore, this work is devoted to the study of the uncertain QCSP with groups of containers, in which the time of unloading/loading of containers and the travel time of quay cranes between adjacent bays are supposed uncertain.The problem is solved with a Simulation Optimization approach which takes advantage of the great possibilities offered by the simulation to model the real details of the problem and the capacity of the optimization to find solutions with good quality.An Ant Colony Optimization (ACO) meta-heuristic hybridized with a Variable Neighborhood Descent (VND), used as a local search, is proposed to determine the assignments of the tasks to the quay cranes and the sequences of executions of tasks on each crane.Simulation is used inside the optimization algorithm to generate scenarios in agreement with the probabilities of distributions of the uncertain parameters, thus, we carry out stochastic evaluations of the solutions found by each ant.
The remainder of this paper is structured as follows.The next section presents an overview of existing literature on the QCSP problem.We give more insights about the problem studied in section 3.Then, a Simulation Optimization based Ant Colony approach is proposed to solve the uncertain QCSP in section 4, followed by numerical experiments in section 5. Finally, section 6 summarizes the work.

Related works
Container terminal operations are considered as the most challenging topics in the area of operation research, because of their complexity and their applicability.In particular, the QCSP has received great attention during the last decades.Recent compressive overviews of the problem are presented by Bierwirth andMeisel (2010, 2015), Carlo et al. (2015) and Boysen et al. (2017).QCSP with complete bays was first studied by Daganzo (1989), with the aim of minimizing the total cost of delays.Container ships were assumed to be partitioned into ships-bays, a task was defined as the handling operations of all containers on a bay and only one crane was allowed to work on a bay at a time.Non-interference constraints between quay cranes were not considered and tasks were supposed preemptive.The static and dynamic version of the problem were considered and solved by an exact and a heuristic approach.Peterkofsky and Daganzo (1990), considered the QCSP with the same previous assumptions and proposed a branch and bound method for its resolution.Later on, Lim et al. (2004) studied a more realistic version of the QCSP with complete bays by considering the non-interference constraints.Three approaches were proposed to determine the best schedule of quay cranes; a dynamic programming algorithm was addressed to solve simpler instances, where a probabilistic Tabu Search and a Squeaky Wheel Optimization heuristic were used to tackle the hardest instances.Zhu and Lim (2006), addressed the QCSP to minimize the latest compilation time of all tasks and considered that tasks are non-preemptive.They showed that the problem is NP-complete and provided a branch and bound algorithm and a simulated annealing approach to solve small and large instances of the problem.Lim et al. (2007) proposed a new formulation for the QCSP with complete bays and showed that there is always an optimal solution of the problem among all possible unidirectional schedules of cranes.A simple approximation heuristic and simulated annealing heuristic was designed to solve the problem.Lee et al. (2008) provided another proof of NP-completeness of the QCSP with complete bays, and proposed an efficient genetic algorithm to address the problem.The efficiency of the genetic algorithm was tested on forty random instances with large sizes and the experimental results showed that the Genetic Algorithm is very efficient since deviation to the lower bound was less than 0.9% on all instances.
Few papers in the literature were devoted to the study of the QCSP with bay areas.Steenken et al. (2001) addressed the problem with the aim of minimizing the difference in the use of any two cranes.They showed that for instances of practical size the problem leads to a partitioning problem that can be easily solved by straight-forward enumeration.Lu et al. (2012) proposed an efficient heuristic based on unidirectional movements of cranes for the problem, the heuristic achieved a good trade-off between solution quality and computational time.
The QCSP with group of containers represents the most complex and realistic variant of the QCSP in literature.This variant was introduced by Kim and Park (2004).In this work a task was defined to be a collection of containers located adjacent to each other and that share the same characteristics.Namely, the same port of origin or of destination and the same size.The authors proposed a MIP formulation of the problem, with the aim to minimize the weighted sum of makespan and quay cranes finishing times.This formulation takes into account non-interference constraints between quay cranes, precedence among the handling operations of tasks and the availability date of quay cranes.To avoid collisions between cranes, the non-interference constraints were enforced by a non-simultaneity constraint between tasks located in adjacent bays.A Branch and Bound method and a Greedy Randomized Adaptive Search Procedure (GRASP) were proposed to solve the problem.Results showed that the Branch and Bound method outperforms GRASP in terms of solution quality but fails for largest instances.Moccia et al. (2006) revised the formulation proposed in (Kim & Park, 2004), to avoid some cases where interference cannot be detected.They significantly improve the results found by Kim and Park (2004) using a Branch and Cut algorithm.In (Sammara et al., 2007), the QCSP with group of containers was seen as a combination of a routing and scheduling problems.The routing problem was solved by a Tabu Search heuristic, and a Local Search technique was used for the resolution of the scheduling problem.Experiments showed that the proposed algorithm reduced significantly computation time for the largest instances, compared to the Branch and Cut of Moccia et al. (2006) with a slightly weaker quality of solutions.The MIP formulation developed in Kim and Park (2004) was also improved by Bierwirth and Meisel (2009), the authors introduced a new set of interference constraints and a fast Unidirectional Scheduling (UDS) heuristic based on the Branch and Bound algorithm was proposed to solve the resulting problem.Numerical tests revealed that UDS heuristic outperforms all previous existing algorithms in the literature, in terms of computational time and solution quality.In later work of Chung and Choy (2012) a Genetic Algorithm was proposed to deal with the QCSP.Experiments were executed using Kim and Park benchmarks, and results proved that the Genetic Algorithm is competitive and efficient as compared to the existing algorithms.Monaco and Sammarra (2011) studied the QCSP with group of containers under the hypotheses that cranes can only make unidirectional moves and that their availability is given within a predefined times windows.A Tabu Search heuristic was developed to solve the problem, and its efficiency was tested on known Kim and Park benchmarks and on a real-world application on the Italian Gioia Tauro terminal.In (Wang & Kim, 2011), the QCSP was combined with yard management problem with the aim of minimizing the time spent by vessels in the port and minimizing the difference of workloads between yard blocks.The problem was solved by a GRASP meta-heuristic.New benchmarks for the QCSP were proposed by Meisel and Bierwirth (2011), these instances tests provide 400 sets ranging from 10 to 100 tasks and 2 to 6 cranes.They were widely used in some recent works by (Kaveshgar et al., 2012), (Unsal and Oguz, 2011), (Chen et al., 2014) and (Rouky et al., 2015).In the work of Nguyen et al. (2013), hybrid evolutionary computation approaches based on Genetic Programming and Genetic Algorithm were proposed to address the QCSP.Computational results demonstrated that the proposed methods perform as well as the existing methods, and that they were able to obtain better solutions than the best known ones in many instances.
Although a considerable attention has been paid in literature to the different variants of the QCSP, to our best knowledge, very few publications studied the QCSP with uncertainties.Legato et al. (2010) were the first that addressed the QCSP while taking uncertainties in consideration.They considered uncertainties that arise in handling process and proposed a Simulated Annealing algorithm to solve the QCSP and a Discrete Event Simulation to compute the expected cost of the solutions.However, the authors provided only one instance to evaluate the efficiency of their results.In a recent work, AL-Dhaheri et al. (2016) studied a problem that combine Quay Crane Scheduling with complete bays and Straddle Carriers Routing with the aim of increasing the container terminal throughput.While considering the randomness and dynamics related to containers discharging process.A simulation based Genetic Algorithm was proposed for the resolution.The numerical tests demonstrated the significance of using simulation to obtain more realistic solutions.
Our contribution in this paper is different from previous works of Legato et al. (2010) and AL-Dhaheri et al. (2016); Since, on the one hand, the simulation procedure is used in this paper inside the optimization approach to evaluate every possible solution obtained whiting the optimization algorithm rather than only evaluating the best solution as it was proposed in (Legato et al., 2010).On the other hand, this work is devoted to the study of the uncertain QCSP with group of containers, which is known to be more complex than the QCSP with complete bays studied in (AL-Dhaheri et al., 2016).Moreover, to the best of our knowledge, this is the first time in literature when the integration of an Ant Colony Optimization (ACO) with a Simulation procedure is considered to solve an uncertain QCSP problem.

Problem description
As explained in section 1, the QCSP consists of determining the best schedule of containers handling operations by a set of quay cranes assigned to a vessel.By extension the uncertain QCSP studied in this paper can be defined as a QCSP in which the handling time of tasks and travel time of quay cranes between location of tasks are supposed uncertain and they are given by probability distributions.Formally, in the uncertain QCSP a vessel is divided to a set of locations, 1, . ., | | denoted by bays.Bays are used for the storage of a set of tasks T , . ., , that represents the loading and unloading operations that must be executed by a set of quay cranes Q , . ., assigned to the vessel.Each task has a position , expressed by a bay number.Quay Cranes (QCs) and bays are both supposed indexed in ascending order from left to right.Processing time of tasks and travel time of quay cranes between adjacent bays are assumed to be independent random variables with known distribution; The uncertain process time of task follows a 32-Erlang distribution with an expected value , and the uncertain travel time ̃ of a quay crane between two adjacent bays follows a Triangular distribution with a lower bound of 1 minute, mode of 1.5 minutes, and upper bound of 2.5 minutes.Precedence constraints are defined between tasks to respect the stacking plane of the containers.Since in the QCSP with group of containers, a bay is carrying several tasks, whiten the same bay; tasks that represent unloading operations must be performed before those of loading operations, the unloading operations on the deck of the vessel must precede the unloading on its hold and the loading on the deck can only start after the loading on the hold.We denote by the set of all pairs of tasks linked by precedence relationship, and by the set of all tasks that must be accomplished before executing task .Moreover, quay cranes are mounted on the same track therefore they are not allowed to cross each other and at most one quay crane can operate on a bay at a time.Furthermore, to avoid collisions between cranes some tasks that are located in adjacent bays cannot be processed simultaneously.Let denote by the set of all task pairs that have non-simultaneous relationship and by the set of tasks that cannot be performed simultaneously with task .Finally, for each quay crane , an initial position and an initial ready time are given.

Table 1
An example of input data for the uncertain QCSP with group of containers Precedence relationships T , T , T , T , T , T , T , T , T , T  The QCSP can be viewed as a parallel identical machines scheduling problem, which is known to be NP-Hard (Michael, 1995).Therefore, we proposed in the next section a Simulation-Optimization based Ant Colony algorithm to solve the uncertain QCSP.

Solution methodology
A variety of approaches, such as fuzzy programming, robust optimization and stochastic optimization, has been used in the literature to deal with uncertainty in logistics and production systems, good reviews of these approaches are given in (Sahinidis, 2004) and (Gabrel et al., 2014).However, the level of detail and the accuracy described by those approaches are most of the time very insufficient since perturbations on real-world systems are too complex to be modeled analytically (Figueira & Almada-Lobo, 2014).Simulation provides a good way to model details of the problem, nevertheless, when it is used alone, it can only evaluate the performance of some possible alternative organizations of the system, which is not sufficient from the optimization point of view (He et al., 2013).Thanks to the tremendous development in computer performance in recent decades, combined Simulation Optimization (SO) approaches have received great attention, since they provide an intelligent way to explore simultaneously the great possibilities offered by the simulation to model the real details of the problem and the capacity of the optimization to find solutions with good quality.
SO approaches have been successfully applied to a wide range of problems arising in port logistics (Abourraja et al., 2017;Benghalia et al., 2016), rail transportation (Tréfond et al., 2017), risk management (Better et al., 2008) and production systems (Lim et al., 2006), among others.In particular, they have shown their greatest advantage when applied to highly dynamic and uncertain problems.Where only local information is available, such as the supply chain management under demand uncertainty (Jung et al., 2004), the stochastic location-routing problem (Herazo-Padilla et al., 2015) and the container yard design problem under uncertainty (Zhou et al., 2016).In this section we propose a new Simulation Optimization based ant colony heuristic to solve the uncertain QCSP with the aim to minimize the expected value of the compilation date of the last task in the vessel (makespan).Section 4.1 presents the general framework of the proposed Simulation Optimization approach, while the detailed steps are completely described in sections from 4.2 to 4.5.

General Framework of the Simulation Optimization Approach
As it is shown in Fig. 2, the SO-based ACO approach starts by loading input data from the instance file.
Input data consist of the number of tasks, the number of QCs assigned to the vessel, the location of tasks, type of tasks and their position in the Deck or in the Hold of the vessel.An iteration of the SO approach begins by setting the position of each QC and it ready time to their initial values.An ant procedure is then executed to determine a set of feasible solutions of the QCSP problem, in this procedure we use a set of ants , , . ., , each ant has to find a feasible schedule , which determines an initial assignment of tasks to QCs and the sequence of the execution of tasks by the QCs with respect to the precedence, the non-interference and non-simultaneous constraints, and by using only the nominal values of processing times of tasks and nominal travel times of QCs between adjacent bays.Next, a Variable Neighborhood Decent (VND) algorithm is executed on each initial schedule , obtained by the Ant procedure, and the resulting improved schedule is denoted by , .
Whereas, the simulation procedure is then executed to evaluate each improved schedule on realistic scenarios generated on concordance with the distribution probability of travel times and processing times.At the end of the simulation the expected value of makespan of each improved solution is computed and compared, and a global update of pheromones is executed to increase pheromones over the edges of the best found solution at the iteration.The approach stops when a maximal number of iterations is reached and the best current solution is returned.

Ant Colony procedure
Ant Colony Optimization (ACO) is an algorithm that was initiated by Dorigo and Caro (1999).ACO was inspired by the behavior of ants in real life.Ants start by exploring the area surrounding their nest randomly; then, on their way back to the nest, they lay a substance called pheromone.Pheromones guide other ants towards the target point, since pheromones on the paths used by several ants will be reinforced and these paths will be more interesting to the next ants.ACO algorithm is considered as one of the best choice to solve scheduling problems since the literature about these problems has demonstrated its effectiveness to give good solutions (Rajendran et al., 2004;Hirsch et al., 2012;Thiruvady et al., 2016;Bencheikh et al., 2016;El Khoukhi et al. 2017).

Graphical representation
In each iteration I of our proposed SO approach a ACO Structure that include a set of m ants , , . ., is used.Each ant build a initial schedule , well considering tasks and QCs attributes and the initial values of travel time of QCs and of processing times of tasks.The ants run through a bi-level graph (Fig. 3), vertices on the first level of this graph represents the QCs where vertices on its second level represents tasks.Two dummy vertices S and F are added to this graph to represents the beginning and the end of one ant move.

Choice of a quay crane
Each ant starts building its own schedule from the initial position S. Selects the first quay crane to be used according to the first transition rule defined in Eq. ( 1), where a real number chosen randomly in the interval [0,1], and is a parameter of the algorithm.Thus, depending on the value of q, we favorite either the choice of the quay crane with the lowest available date , or we select a random quay crane  (1) Fig. 3. Proposed Bi-level graph for the displacement of ant

Candidate set
Once a quay crane is selected, the ant creates a set of candidate tasks that can be executed by the current crane.The set of candidate tasks of an ant is denoted by and contains all tasks that are not yet assigned to a quay crane (i.e. that are not in the set of tasks already selected by the ant ) and that their list of predecessor is empty.

Choice of a task
Then, the ant selects from the next task to be affected to the current crane Q according to the second transition rule defined in Eq. (3).
where, τ shows the amount of pheromone concentration in edge , .The pheromone represents the memory of the ACO and it is used to promote displacements on edges that have been selected by a large number of ants.It is initialized at the beginning of the SO approach by a small value τ , and is locally updated in the ACO at each time when a task is selected as shown in the section 4.2.6.A global update of pheromone trail is also performed at the end of each iteration of the SO approach according to the mechanism described in section 4.4.
represents the heuristic information associated with assigning task T to the current crane Q , this information provides worthy information about the problem for guiding the search procedure.Two different strategies are proposed in the section 4.2.5 for the heuristic information.Parameters α and β are introduced to control the search direction, which determine the relative intensity of the pheromone trace and the heuristic information.

Heuristic Information
In order to converge to good schedules in the ACO procedure, we proposed two different strategies for the heuristic information. - The heuristic information associated to the EST strategy is defined by Eq. ( 5) as: 1 1 (5) -Local Work Load strategy (LWL): This strategy is based on the Local Work Load (LWL) of a candidate task ∈ , which is defined in Equation ( 6) as the sum of initial processing time of remaining tasks in the radius of 2 bays from the location of task .
The heuristic information associated to the LWL strategy is defined by Eq.n (7) as: where represents the initial processing time of candidate task , ̂ is the initial travel time of a quay crane between two adjacent bays and thus | | gives the required travel time of quay crane Q from its current position to the bay location of candidate task .

Compilation time, ready time and Local update of pheromone
Once a task is assigned to the current crane , the ant updates its state and moves to the vertices F. The ant adds the selected task to the set of tasks already selected and deletes the task from the predecessor sets of all other tasks such as .Then we assign, using Eq. ( 8) a completion time to the selected task and we set the ready time of the current quay crane to .
When the ant moves to vertices F, a local update of pheromone is performed on the edge c, i of the graph according to the following equation: Steps from 4.2.2 to 4.2.6 are repeated until that all tasks were be selected by the current ant .Then we start the construction of the schedule , of the next ant .The ACO procedure is summarized in Algorithm 1. Iniitalize all predecessor sets e of tasks 4: For each quay crane Q : Set the ready time r to r and the curent postion time l to l 5: , ← ∅ and ← ∅ 6: repeat 8: Assign to parameter q a random value from the interval 0,1 9: if do 10: Q ← argmin

Variable Neighborhood Descent
We use a Variable Neighborhood Decent algorithm (VND) as a general local search to improve schedules obtained by the ACO algorithm.VND is considered as a single solution based meta-heuristic that is known for its ability to escape from local optimum (Hansen et al., 2010).The proposed VND algorithm is executed on each ant schedule, it begins with an initial schedule , and attempts iteratively to improve it by employing a set of three neighborhood structures.The sequence order of exploring the neighborhood structures is randomly generated, at each call of the algorithm.The VND executes the first neighborhood in the generated order as long as an improvement is obtained, and moves to the next one in the sequence when the previous fails to lead to improvement.VND stop when all neighborhood structures were applied.Steps of the VND algorithm are given in Algorithm2 and the details about the neighborhood structures are described bellows.
-Swap1: Randomly select a quay crane and examine all possible swaps between each pair of tasks, only possible swaps are considered.
-Swap2: Randomly select two different quay cranes and examine all possible swaps of tasks between the selected cranes, only possible swaps are considered.
-Relocate: Randomly select a quay crane and examine all possible moves of tasks that are currently assigned to this quay crane to a different position on the same quay crane, only possible moves are considered.

The Simulation procedure
Simulation experiments are performed using Compare-run tool of Anylogic Software (Multimethod Software, 2015).AnyLogic is a multi-approach simulator, equipped with a rich and easy-to-master toolbox, that allows to effectively combine the various simulation techniques (agent based simulation, discrete-event simulation and system dynamics).Anylogic is used in this paper for the analysis, the collection and the comparison of results as well as for extension, visualization and experimentation of simulation, which allows us to considerably reduces the time of developing the simulation model.
The simulation procedure starts by generating a simple path , , … , , each element of this path represents one replication of the simulation experiment.Each replication defines one possible scenario generated on concordance with the distribution probability of the uncertain parameters.Travel time of QCs between adjacent bays are generated according to a Triangular distribution with a lower bound of 1 minute, mode of 1.5 minutes and an upper bound of 2.5 minutes.Process time of tasks follows a 32-Erlang distribution with an expected value .A Discrete Event Simulation (DES) is then used to evaluate the schedules obtained in the optimization procedure and to correct the starting and completing time of tasks according to each generated replication.The DES starts by loading the tasks assignment and operation sequences of QCs from schedule , given by the optimization model, and the values of the uncertain parameters from replication .We use the status of each QC, i.e. either "idle" "busy" or "cannot be selected", and the number of tasks waiting to be executed by each QC as two variables to describe the state of the simulated system.QCs are marked as "busy" until their ready times is reached.Then, the first available QC is selected and moves to the bay position of the first task in its sequence and the handling operations of containers in this task start.After that, the selected QC changes its statue from "idle" to "busy" and we move to select another available crane.In the case where several QCs show equivalent ready times, priority is given to the QC with the lowest index.Before performing any moves of a QC, we check if this move will cause interference, if it is the case, we change the statue of the QC to "cannot be selected now" and we select another idle QC.The statue of a QC change also to "cannot be selected now" when the first task in its sequence cannot be executed simultaneously with one of tasks that are on execution by other QCs.When the handling operations of a task finish, its assigned QC changes its statue to "idle" and the number of tasks waiting to be executed by this QC decrease by 1.The simulation process finishes when the number of tasks waiting in all QCs is equal to 0, and the DES return the completing time of the last task executed.This experiment is performed for all improved schedules , , . . ., , obtained at the end of the optimization procedure, and the expected value of the makespan of each schedule is computed by:

Global Update of Pheromone
At the end of each iteration of the Simulation-Optimization approach, the different schedules are compared and a global update of pheromone trail is performed on arcs of the schedule with the less expected makespan, according to the following formulation: where Δ 1 is the amount of pheromone added to the arc , of the best schedule at the iteration, and is the coefficient of pheromone evaporation.

Numerical experiments
In this section several numerical experiments are executed to evaluate the performance of our developed SO-based ACO approach.First, the selection of the best parameters for both the ACO algorithm and the simulation procedure is investigated.Then, the performance of the proposed Hybrid Ant Colony Optimization (HACO) procedure (i.e. the ACO with the VND algorithm), is tested on the well-known data of Kim and Park under deterministic environment.In the stochastic case, two Gaps are recorded to compare the performance of the SO approach using both the EST and LWL strategies .
All experiments are carried out on an Intel(R) Core (TM) i5-3337U, 1.80 GHz PC with 6.00 GB of RAM.

Ant Colony parameters
In general, efficiency of any meta-heuristic depend on its parameters setting since a good tuning can allow a fast convergence to solutions with high quality well a random choice can cause bed performances.
The IRACE package (López-Ibáñez et al., 2016) is used in this section to select good combination of parameters used in the ACO algorithm.IRACE package performs an automatic algorithm tuning, which starts by generating a finite set of possible configurations.Then, compare their performance on a set of training instances.Elite configurations are then used to generate more good potential configurations of the parameters in next iterations.The procedure is repeated until the tuning budget, which is given by the maximal number of configurations to be examined, is reached.The current best combination of parameters is returned at the end of the procedure.Table 2 presents the results obtained for each parameter used in the ACO algorithm, The tuning budget in the IRACE package was set to 3000 experiments.

Number of replications in the Simulation
Compare-run tool of Anylogic Software provides a Parameter Variation Experiment that stops the Simulation procedure after a minimum number of replications, when the confidence level is reached.If the confidence level is not met, the Parameter Variation Experiment ends when the maximum number of replications has been exceeded.The confidence level was fixed at 95% and the error percentage was set as 0.5.The minimum and maximum number of replications were set as 10 and 500, respectively.

Deterministic Results
In order to test the performance of the proposed optimization problem, we conduct a series of computational tests over the well-known benchmarking data of Kim and Park (2004).This benchmark is commonly used to test the efficiency of algorithms for the Quay Crane Scheduling Problems (QCSP).
The data consists of 43 instances, from k13 to k49, divided on 4 sets.The first set represents the small size problem composed of 2 QCs and 10 tasks, set 2 contains medium size problem with 2 QCs and 15 tasks, where sets 3 and 4 are regarded as large size problem with 3 QCs, 20 tasks and 3 QCs, 25 tasks, respectively.

Performance Analysis
The results of the Hybrid ACO algorithm using both the Earliest Start Time (EST) strategy and the Local Work Load (LWL) strategy are compared to those from Branch-and-Bound (B&B) and Greedy Randomized Adaptive Search Procedure (GRASP) algorithms by Kim and Park (2004), Tabu Search (TS) meta-heuristic by Sammarra et al. (2007), Unidirectional Scheduling (UDS) heuristic by Bierwirth and Meisel (2009), Genetic Algorithm (GA) by Chung and Choy (2012), Hybrid Genetic Algorithm (HGA) and Hybrid Genetic Programming (HGP) by Nguyen et al. (2013).The relative percent deviation (Eq.12) of each algorithm to the Lower Bound (LB) is used to evaluate the performance of the methods.
Relative Percent Deviation (RPD) on an instance i is given by: 100 where is the makespan obtained by heuristic H on instance i and is the Lower Bound obtained by CPLEX solver and reported in (Bierwith & Meisel, 2009).that both proposed strategies, i.e. EST and LWL, used in the HACO are very competitive and effective to solve the QCSP as compared to the existing algorithms in the literature.The results reveal that the Hybrid ACO with the LWL strategy is the only algorithm in literature that is able to achieve 0% relative deviation in both small and medium size problem with a good performance in large instances of set 3 and a slightly weak deviation in the instances of set 4. On the other hand, the Hybrid ACO with the EST strategy also gives good results when it is applied to small and medium instances of set 1 and set 2, since the average RPD is less than 0.06% in set 1 and equals to 0.04% in set 2. In addition, the HACO-EST outperforms all other methods on the large instances of set 3 with an average RPD equals to 0.39%, and shows quite similar performance to the UDS heuristic and HGP in the large instances of set 4.

Computational Time
It is generally very difficult to compare computational times since each method is implemented in different computer configuration.To make a fair comparison, we adopt an approach based on Million Floating Point Calculation per second (Mflops) as reported in (Dongarra, 2014).Table 4 shows the different Mflops values relevant to our study.Column 2 presents the configurations of the computers where each tested algorithm is implemented.In column 3, since some of the computers configurations are not listed in the report of Dongarra (2014), we give the equivalent machine configuration found in the report, and we report the corresponding approximate Mflops value for the particular machine in Column 4. Finally in the last column, We report the conversion factor that will be used to scale the computational times of the algorithms.The conversion factor presents the ratio of the Mflops value of a given computer configuration to the Mflops value of our computer configuration.

Table 3
Mflops values of the different computers configurations relevant to our study.Table 4 presents the results of computational times comparison.The average computational time corresponds to the original computing time of each algorithm in the literature, where the scaled average time is equal to the original computational time multiplied by the conversation rate .Thus, the scaled time shows the computing time of the different algorithms if they had been executed on our computer.We can clearly see from results of Table 4 that our proposed algorithm is very efficient at solving the QCSP, because all instances are solved quickly in less than 0.1 minutes.Our proposed algorithms are advantaged, in terms of computational time, with respect to most existing algorithms.The UDS is the only heuristic that is faster than our proposed algorithm.

Stochastic Results
Previous deterministic results have shown that the hybrid ACO algorithm, with both proposed strategies EST and LWL defined for the selection of task, is very effective and efficient to solve the deterministic QCSP.This performance is very recommended in Simulation Optimization (SO) practice, because the choice of an optimization algorithm always depends on its computational efficiency (Kelly, 2002).In this section we will compare the results of both strategies when they are applied in the Simulation Optimization approach to deal with uncertainties.Then we demonstrate the significance of using Simulation Optimization to obtain more realistic solutions under stochastic assumption rather than employing deterministic approaches.results for the SO approach on all tested instances since the only negative gap was observed in the instance k25 with a weak value of -1.19%.
Therefore, it is obvious from results presented in Fig. 7 that the use of EST strategy within the hybrid ACO procedure is more useful to deal with uncertainties than the LWL strategy, even if both strategies had shown similar performance in the deterministic case.
To demonstrate the significance of using the SO approach to deal with uncertainties we compare its results with those of deterministic solutions when they are put into operation with identical scenarios.
For the SO solutions we use those obtained by applying the EST strategy, and for the deterministic solutions we consider on each instance the best solution over the EST and LWL strategies.Solutions are then simulated on 1000 scenarios which have stochastic quay crane travel time and tasks processing times.The relative improvement that SO solutions yield compared to deterministic solutions is recorded in Fig. 7.The average improvement value was equal to 7.43% and the maximum improvement was as high as 15.6%.Therefore, we can conclude that the use of SO solutions are very useful to achieve a good robustness against uncertainties.

Conclusion
In this paper, we have considered the Quay Crane Scheduling Problem under stochastic assumptions, we have assumed that there is uncertainty in task processing time and quay crane travel time.To solve the resulting problem, we proposed a Simulation Optimization (SO) approach, which takes advantage of the great possibilities offered by the simulation to model the uncertain parameters of the problem and from the capacity of the optimization to find high quality solutions.In the optimization procedure we implemented a Hybrid Ant Colony Optimization algorithm which includes an Ant Colony and Variable Neighborhood Descend and we proposed two different strategies for the selection of tasks, namely; the Earliest Start Time (EST) strategy and the Local Work Load (LWL) strategy.A Discrete Event Simulation (DES) was used to evaluate the obtained solutions under different scenarios generated on concordance with probability distribution of the uncertain parameters.Since it is known that performance of SO approaches depend on computational efficiency of the selected optimization algorithm, our first numerical tests were devoted to the comparison, in deterministic case, between the HACO algorithm using EST and LWL strategies and other existing algorithms in the literature.The results have shown that the HACO algorithm, with both proposed strategies, is very effective and efficient to solve the deterministic QCSP.Then, a series of computational study was carried out on the stochastic case using the proposed SO approach to test the efficiency of both strategies under uncertainty.Tests revealed that the use of the EST strategy within the hybrid ACO procedure is more useful to deal with uncertainty in the SO approach than the LWL strategy.
operation; U: Unloading operation; H: Hold; D: Deck Table1presents the input data for an instance of the uncertain QCSP.In this instance, 2 quay cranes are used to perform the handling of 10 tasks located in a vessel that is divided to 10 bays.Rows from 2 to 5 present the attributes of tasks, and they show respectively for each task; its location in the vessel given by the bay number, the nature of the operations (i.e.Loading (L) or Unloading (U)), if the task is positioned on the Deck (D) of the vessel or in its Hold (H) and the expected processing time .The sets of precedence and non-simultaneously relationships are reported in rows 6 and 7. Quay cranes are supposed to be available from the starting of the planning horizon and they are located in bay1 and bay6, respectively.Fig 1.(a) gives an illustration of this instance, and Fig 1.(b) provides a simple representation of precedence relationships.Three tasks are located in bay2; an unloading operation T4 and two loading operations T6 and T1.Thus, T4 has to be accomplished before the starting of tasks T6 and T1, and task T1 must precede T6 since loading on hold precedes the loading on deck.

Fig. 1 .
Fig. 1.Illustration of an instance of the Uncertain QCSP (a) and a simple representation of precedence relationships (b)

Fig. 2 .
Fig. 2. General structure of the Simulation Optimization based Ant colony algorithm .

Fig. 4 Fig. 4 .A
Fig.4shows the general flowchart of the Discrete Event Simulation:

Fig 5 .
Fig 5. Performance evaluation of optimization procedure versus existing algorithms Table 5 (Appendix A) shows the detailed comparison results, well Fig. 5 summarises these results and reports the average Relative Percent Deviation obtained on each set of instances.The results demonstrate the deviation rate between the maximum makespan value obtained by the LWL strategy and the maximum makespan value obtained by the EST strategy, across all replications of the SO approach (each component of the sample path ). the deviation rate between the expected makespan obtained by the LWL strategy and the expected makespan obtained by the EST strategy, among all replications of the SO approach (each component of the sample path ).

Fig 6 .
Fig 6. Results of maximum deviation rate (a) and expected deviation rate (b) Fig. 6.(a) summarizes Gap1 values and shows that EST strategy outperforms the LWL strategy in regards to worst case scenarios, since the maximum makespan value (i.e.worst possible performance) of the former strategy is higher than these of the latter in most instances (positive gap), except in instances k15 and k19 where negative gaps of -3.12% and -0.86% are respectively observed.Moreover, in regards to the objective function that aims to minimize the expected makespan, the results of Gap2 reported on Fig. 6.(b) show clearly that the use of the EST strategy within the optimization procedure provides good

Fig 7 .
Fig 7. Relative improvement offered by SO solutions compared to deterministic solutions Earliest Start Time strategy (EST): This strategy is based on the Earliest Start Time (EST) of candidate tasks, this time is defined in Equation (4) as the first date on which the current quay crane Q can start the handling of a candidate task ∈ , without violating the non-interference and non-simultaneous constraints.

Table 2
Best combinition of parameters

Table 6 (
Appendix A) presents the detailed results for simulation tests based on the EST and LWL proposed strategies.To compare these strategies and selects the most adequate one for the SO approach, we recorded two gaps: