FRAMEWORK OF AN EVOLUTIONARY MULTI-OBJECTIVE OPTIMISATION METHOD FOR PLANNING A SAFE TRAJECTORY FOR A MARINE AUTONOMOUS SURFACE SHIP

This paper represents the first stage of research into a multi-objective method of planning safe trajectories for marine autonomous surface ships (MASSs) involved in encounter situations. Our method applies an evolutionary multiobjective optimisation (EMO) approach to pursue three objectives: minimisation of the risk of collision, minimisation of fuel consumption due to collision avoidance manoeuvres, and minimisation of the extra time spent on collision avoidance manoeuvres. Until now, a fully multi-objective optimisation has not been applied to the real-time problem of planning safe trajectories; instead, this optimisation problem has usually been reduced to a single aggregated cost function covering all objectives. The aim is to develop a method of planning safe trajectories for MASSs that is able to simultaneously pursue the three abovementioned objectives, make decisions in real time and without interaction with a human operator, handle basic types of encounters (in open or restricted waters, and in good or restricted visibility) and guarantee compliance with the International Regulations for Preventing Collisions at Sea. It should also be mentioned that optimisation of the system based on each criterion may occur at the cost of the others, so a reasonable balance is applied here by means of a configurable trade-off. This is done throughout the EMO process by means of modified Pareto dominance rules and by using a multi-criteria decision-making phase to filter the output Pareto set and choose the final solution


INTRODUCTION
The concepts of the 'green ship' and autonomous vessels are two fundamental issues discussed in the Blue Growth strategy developed by the EU. This study addresses both of these simultaneously. Many solutions related to unmanned or autonomous ships have already been presented by researchers, including basic control and steering [21,22,28] and automatic obstacle detection [9,30]. Some of these are relatively new, e.g. USV-dedicated risk analysis [43,44], human factors in the remote control of maritime autonomous surface ships (MASSs) or unmanned surface vehicles (USV) [23]. There are also a number of issues that are not completely novel but which have been redefined in the context of MASS/USV, such as optimising collision avoidance manoeuvres [28,31] in compliance with International Regulations for Preventing Collisions at Sea (COLREGS) [2,5,7,24,27], which is the topic of the this paper.
The International Maritime Organisation (IMO) specifies four degrees of autonomy for MASS: 1. Ship with automated processes and decision support: Seafarers are on board to operate and control shipboard systems and functions. Some operations may be automated.

Remotely controlled ship with seafarers on board: The ship is
controlled and operated from another location, but seafarers are on board.

Remotely controlled ship without seafarers on board:
The ship is controlled and operated from another location. There are no seafarers on board. 4. Fully autonomous ship: The operating system of the ship is able to make decisions and determine actions by itself.
As can be seen, the role of human operators is expected to be reduced in the development of MASSs. It may also be expected that these ships will combine two or more degrees of autonomy. A common feature of all these degrees of autonomy is that a reduction in the role of a human operator means a larger role for the decision-making system on board, which will replace the older concept of a decision support system. Human navigators heavily rely on experience and an intuitive understanding of the ship's behaviour and environmental conditions; in the case of a MASS, all information must be provided and taken into account in a more direct and formal manner. This gives rise to new needs related to data access and processing, and optimisation methods. The required data include at least: -hydro-meteorological conditions [4], -ship manoeuvrability and stability-related issues, -fuel consumption.
Similarly, in terms of optimisation, it is necessary to focus on: -robust, adaptive and time-efficient methods and algorithms that are adjusted to the needs of MASS, -modelling and taking into account multiple constraints, including: • COLREGS, • the limitations of a waterway.
Of these constraints, COLREGS are of particular note. While there are many publications on this topic, the majority of them deal with a limited subset of COLREGS. The exceptions include research [42] done at Massachusetts Institute of Technology that aims to quantify compliance with COLREGS in terms of collision avoidance. However, these researchers omit the issues of navigation within traffic separation schemes (TSSs) or in restricted visibility [35]. It should also be noted that COLREGS will need to be partly revised for use with MASSs. Since the future form of COLREGS remains unspecified, any related method that is developed should be flexible enough to enable configuration and updating with the implemented COLREGS rules.
In terms of optimisation methods, a single-objective approach is largely used in works on collision avoidance [20,39,48]. Another approach is to aggregate many criteria into a single cost function by applying weight factors. Both of these techniques have certain limitations, and may be insufficient for collision avoidance applications. Time and fuel consumption are important objectives, and the risk of collision needs to be addressed more fully than it has been in the past. Handling the risk of collision as a single optimisation constraint is an outdated approach, resulting in solutions that are not fully acceptable or applicable. Instead, the risk of collision should be minimised, especially if it can be done at a marginal cost in terms of extra time and fuel consumption. A true multi-objective approach is therefore considered here. However, the application of this approach to collision avoidance is a scientific challenge due to: -the longer processing time of this approach compared with single-objective or simplified (aggregated) multiobjective methods, -the high complexity of the optimisation problem itself (discussed in detail in the following sections), -the real-time limitations of the computational process in encounter situations (depending on the particular situation, a safe solution needs to be found within one to three minutes).
In view of the above, the aim of the present research is to design and develop a multi-objective method of planning safe trajectories for a MASS involved in encounter situations at sea. Our method will apply an evolutionary multi-objective optimisation (EMO) [1] algorithm to pursue three objectives: -minimisation of extra time spent on collision avoidance manoeuvres, -minimisation of fuel consumption due to collision avoidance manoeuvres, -minimisation of the risk of collision.
The developed method should be able to: -model a ship's behaviour with a sufficient accuracy for collision avoidance purposes, -carry out optimisation taking into account three objectives simultaneously (minimisation of time spent on manoeuvring, minimisation of fuel consumption and minimisation of the risk of collision), -be fast enough for real-time MASS applications and to handle encounters with multiple targets, -make decisions without interaction with a human operator, -handle all basic types and circumstances related to ship encounters (including open or restricted waters and good or restricted visibility), -be compliant with COLREGS in their current form, -include future updates of COLREGS.

OUTLINE OF THE PROPOSED METHOD
The proposed method will include modelling part necessary for the correct development and verification of the main optimisation method. A set of mathematical models is needed, since the performance of our multi-objective method must be tested in a near-real computer simulation environment. Unrealistic simplifications are undesirable, as these can reduce the method's computational complexity and computational time, making it unrepresentative. In order to avoid an overly optimistic assessment of the performance of the method, it should also cover the full functional scope of the part of the MASS that is responsible for autonomous decision making in encounter situations. Hence, it must cover: 1. Mathematical and simulation models of a ship's motion and fuel consumption, 2. An efficient EMO method including multiple problemdriven enhancements and mechanisms, 3. A set of objective functions, constraint-handling functions and various operators that can boost the performance of the EMO method, 4. An application integrating all of the developed elements of the method as a simulation environment software tool.
The proposed method is summarised in Fig. 1.
It is assumed here that the proposed method will cover a lower layer of the integrated navigation system (INS), according to the IMO resolution [10] on avoiding collisions with targets encountered while following a pre-determined route. The routing for the MASS should be done in such a way that heavy traffic areas are avoided as far as possible, in order to reduce the number of encounters. The MASS should also stick to the traffic flow direction within a TSS. If a TSS cannot be avoided, the MASS should cross it in such a way that the existing traffic flow is not affected (keeping safe distances from ships navigating within traffic lanes). An example of this is shown in Fig. 2, which was generated by our prototype software using EasyMap components (the distances in the latitudinal and longitudinal directions are not equal in Fig. 2, due to the geographical projection used by EasyMap).
The remainder of the paper is structured based on the above scheme ( Fig. 1) and the assumptions made. Sections 3 and 4 are dedicated to models of the ship's behaviour and the EMO method.

MATHEMATICAL AND SIMULATION MODELS OF THE SHIP'S MOTION
The development of the model includes several sub-models that allow for unsteady-state analysis under different sea conditions and in different modes of operation.

MODEL OF THE SHIP'S HULL AND PROPELLER
The functions of the model include determining the longitudinal linear velocity of the ship (surge velocity) with respect to the ship's hull resistance [25], and address the following issues: a) The resistance of the ship's hull in calm water b) The resistance of the ship's hull in rough sea, including added resistance, c) The characteristics of the propeller, including the advance number, torque and thrust coefficients, wake coefficient and thrust deduction factor, d) The ship's propulsion efficiencies, and the effective and required power, e) The speed of the ship (surge).
The model is summarised in Fig. 3.

MODEL OF SHIP'S DIESEL ENGINE, GOVERNOR AND PROPELLER PITCH ADJUSTMENT
The unsteady-state behaviour of a diesel engine is considered, particularly in the case where the influence of the governor has significant impact on the ship's motion, and when its interaction with propeller is considered. This model can be applied if a controllable pitch propeller is included in the propulsion system and also when a detailed description of the engine behaviour is required. The main output of this model is the time variation of the torque generated by the engine in response to changes in the command rotational speed. This includes modelling: a) The dynamics of the diesel engine (engine torque), b) The dynamics of the governor (fuel rate into the engine).
A mathematical model of the diesel engine can be constructed based on a quasi-steady concept, and if necessary can then be improved to take into account the thermodynamics and flow regime characteristics of each cylinder, exhaust gas receiver, turbocharger, inlet air manifold, air and exhaust gas valves or ports, and the mechanical parts and shaft dynamics.
The propeller pitch adjustment mechanism is a hydraulic system that is modelled by a full mathematical model of an electro-hydraulic servomechanism. The lower part of Fig. 3 shows a block diagram of modules 1 and 2, while Fig. 4 shows a detailed block diagram of the diesel engine in combination with the propeller, propeller pitch and ship hull dynamics, and their related interactions. To model the crash-stopping test, the time required to reverse the engine shaft rotation and adjust the governor setting should be also considered, although this is not included in the present model.
To ensure that the mathematical model is applicable to a vessel, it was checked by simulating the behaviour of a merchant ship equipped with a slow-speed diesel engine with fixed-pitch propeller, as the command engine rotational speed is changed from 100% to 80% of the normal continuous rating (NCR). The results are presented in Figs. 5-10. From these results, it is obvious that the propeller and engine need only a few tens of seconds to approach a steady mode, while the ship needs several hundred seconds to come to a steady state. This is due to the very high inertia of the ship in comparison to the engine shaft and propeller. In general, the results seem to be rational and acceptable. For instance, the behaviour of the following pairs: • net generated thrust by propeller and ship's resistance (Fig. 8), • propeller torque and engine torque (Fig. 9), • required power by propeller and generated brake power by engine (Fig. 10), which become equal in the steady-state condition, confirm the correctness of the calculations and permit the use of this model for further investigations.

MODEL OF SHIP MANOEUVRES
The previous two sub-sections discussed a mathematical model for the vessel when the rudder angle is fixed and does not change over time. In the case where the rudder angle changes, a manoeuvring model will be conducted and integrated with the previous sub-models, based on which the sway and yaw variables can be coupled to the surge variables. In general, this part includes the modelling of: a) the ship's dynamics and motion for the coupled surgesway-yaw system, b) the hydrodynamic forces on the ship and their moments, c) motion stability checking of the ship.
The mathematical model of the ship's manoeuvres is nonlinear, and an outline of this model is given in Fig. 11.
The first module of the model in Fig. 10 (upper left) provides a general nonlinear mathematical model of the ship's motion in the xy-plane. The second module (upper right) is the most problematic part, particularly when ship is not in calm water. There are several different empirical methods that can be used to model the ship's manoeuvres in calm water, using the so-called hydrodynamic coefficients. For example, Kijima, Fossen, Ankudinov or Lewandowski coefficients can be used [18]. However, in a rough sea, seakeeping analysis should also be provided, and the drift and heading angles should be included. As a result, the position, course and trajectory of the ship will be determined. Alternatively, the simplified model described in [6,26] can be applied. The model can be verified based on the typical procedures and requirements delivered by ITTC in relation to the manoeuvring tests. The applied mathematical model has the following general form: where u and v represent the surge and sway velocities, respectively, and r is the angular velocity around the z-axis (yaw); m is the total mass of the ship (including added masses); I zz is the mass moment of inertia; x g is the position of the centre of gravity; X and Y are the longitudinal and horizontal hydrodynamic forces; N is the hydrodynamic moment around the z-axis; and the indexes h, r and p denote the hull, rudder and propeller, respectively. The last three variables are functions of the abovementioned velocities and their derivatives (accelerations), as well as the rudder angle д.
These relationships permit to indicate the last but one module in Fig. 11. They are supported and completed using the empirical hydrodynamic coefficients. An approximation is applied when items higher than second order are neglected in the equations. To construct the last module, the forces and moment are non-dimensionalised by dividing by ρL pp Tu/2 and ρL 2 pp Tu 2 /2, respectively, and the mass and moment of inertia are non-dimensionalised by dividing by ρL 2 pp T/2 and ρL 4 pp T/2, respectively [8] (where T is the net thrust, ρ is the water density and L pp is the ship's length between perpendiculars).
The propeller rotational speed varies over time and directly influences the change in the surge speed, and should therefore be considered an additional state variable. In this regard, the differential equations in the model of the ship's motion include also changes in the rotational speed of the propeller, ω p , as follows: where Q e , Q p and Q l stand for the engine torque, propeller torque and equivalent torque losses, respectively, and J p is the moment of inertia of the propeller, shaft and power transmission elements connected to the propeller. Q p can be calculated by the following formula: in which k Q is the propeller torque coefficient (which is given by the manufacturer or determined based on model tests, and depends on the number of blades, advance number, Reynold's number, propeller area coefficient and pitch ratio), ρ is the density of sea water [kg/m 3 ], D is the diameter of the propeller and n(t) is the rotational speed of the propeller [rps]. Q l (t) can be determined in relation to the mechanical efficiency of the elements of the power transmission system, including the bearings, coupling, connecting shafts, clutch and gears, if applied. Q l (t) is typically a function of propeller rotational speed, but in a simplified model can be considered to be constant.
The mathematical model of the engine is added to the whole model. The engine model used in this study is the same as that presented in [45], and is therefore not repeated here.
Again, to ensure the viability and accuracy of the model, it is checked using some conventional manoeuvrability tests. An example is the turning circle test [11]. The validity of the manoeuvring models was checked by taking into account the possibility of correcting the hydrodynamic coefficients. The mathematical model was tested for an offshore supply vessel, and the results confirmed the validation of the model (see Fig. 12). The application of the model will allow the manoeuvrability to be taken into account more accurately for all modes of operation, rather than simply the basic mode that has been used in the past to determine a ship's manoeuvrability [47].

AN EVOLUTIONARY MULTI-OBJECTIVE OPTIMISATION METHOD FOR PLANNING SAFE SHIP TRAJECTORIES
The fully functional, problem-oriented EMO method will include: 1. A backbone EMO method, which must allow for multiple custom enhancements, 2. A module that is responsible for handling various optimisation constraints by: a. Determining the degree to which they are met by a given ship's trajectory, c. Applying specialised operators that amend unacceptable trajectories so that they satisfy the constraints. Both of the elements listed above are briefly described in the following sub-sections.

BACKBONE EMO METHOD
The design of the proposed optimisation is roughly based on the classic Strength Pareto Evolutionary Algorithm 2 (SPEA 2) [49]. To improve the performance of SPEA 2, the proposed method will include a number of modifications and enhancements. Choosing EMO as the base for the optimisation method makes it possible to find a good representation of a Pareto-optimal set [41], including potential non-convexities [12]. This is a significant advantage compared to aggregated objective evolutionary algorithms, which are unable to find solutions from the non-convex parts of a Pareto set, even when multi-started with modified weights. EMO methods have therefore recently been applied to numerous marine optimisation problems [46]. Another of their advantages is that they make it possible to include the decision maker's (DM's) preferences [19], which allows the designers to focus on the most important part of the objective space, thus reducing the number of analysed solutions while still returning a good representation of a Pareto optimal set as the end result. Multiple approaches to applying the DM's preferences are known, including interaction with the DM or specifying a reference solution; however, most of these are impractical in a collision avoidance context. Interaction during the optimisation process is impossible because of time limits, while a reference solution would require a database of collision avoidance scenarios covering all possible situations; this is practically impossible, and even if available would be flawed due to overestimation or underestimation errors. In the present research, we have developed our own approach based on a trade-off that includes the DM's preferences. This approach is loosely based on the one introduced in [3], but is generalised to cover a given number of objectives. The exact values of the trade-off factors used here can be configured to reflect the voyage mission defined by the ship owners. In practice, this trade-off is of particular importance in collision-avoidance situations, since safety and economy are usually contradictory objectives. Optimising one may occur at the cost of the other, so a reasonable balance by means of a configurable trade-off is essential. The trade-off is applied throughout the EMO process by means of modified Pareto dominance rules as well as in the multi-criteria decisionmaking (MCDM) phase to filter the output Pareto set and choose the final solution.
The EMO method developed here will cover the three objectives specified in the earlier sections. Of these, particular emphasis must be put here on safety. Traditionally, safety has been taken into account solely as a constraint; for example, solutions that exceed a certain threshold (which is set for a particular type of threat) cannot be accepted, and are thus eliminated from the process. Alternatively, safety can be considered an optimisation criterion and the risk index can be minimised. In the proposed multi-objective approach, this is handled both as a constraint and as one of three criteria. We search for solutions with the lowest possible risk index as well as for solutions that optimise the other two criteria. However, solutions whose risk index exceeds a certain threshold are eliminated, regardless of the values for the other objectives (e.g. fuel consumption), even if they are not Paretodominated. This involves quantifying the risk of collision with other ships, and a robust method of determining safe manoeuvres for a specified ship safety domain [36] has been proposed in [38]. Using this method, safe combinations of course and speed can be found without affecting the method's computational complexity, resulting in a shortening of the overall computational time. The abovementioned approach can be directly applied in the proposed EMO method.

SHIP TRAJECTORY CODING IN THE EMO METHOD
In this method, each candidate trajectory is represented by an individual in the EMO population. Each individual is defined by a sequence of course changes accompanied by a sequence of times at which the manoeuvre is initiated. This representation allows for a significant reduction in the variable space. Each change in course is a discrete value (in degrees) from the predefined set. In addition to course changes, the current values of the ship's course are also computed (they are needed for the evaluation of solutions). However, course changes (rather than course values) are used for mutation and crossover operators, in order to ensure that each newly created or modified trajectory is in the format used for acceptable course changes. In this way, we can reduce the probability of generating solutions that are of no practical value. Using discrete course changes throughout the EMO process enables us to reduce the crowding problem and avoid multiple solutions that are too close to each other, and allows redundant solutions to be easily identified and eliminated.

HANDLING OPTIMISATION CONSTRAINTS
To make sure that the method returns acceptable solutions, we need to check that the optimisation constraints are satisfied. To ensure reasonable progress of the method, we also need to determine the degree to which they are met by a given trajectory for the ship. The most important constraints here are related to: • avoiding running aground, • avoiding collisions with stationary obstacles, • taking into account other limitations of a waterway in restricted areas, • avoiding collisions with other moving objects, • compliance with COLREGS.
The first three of these constraints are handled by use of electronic navigational charts (ENCs). This has already been done in EMO methods applied to optimisation problems in marine navigation, such as weather routing [16,29,33]. In general, the problem of handling ENC-derived constraints in Evolutionary Algorithms (EA) can be considered solved [40], including in coastal waters [39]. Possible approaches to this include the direct use of ENCs as vector maps or converting them to bitmaps for use in the optimisation process. We have developed a solution based on the second approach and apply this in the proposed EMO method.
For collision avoidance, it has been shown that both radar and AIS data on targets can be integrated with ENC [14]. In autonomous ships, the accuracy of target tracking is essential, and multiple methods of increasing this accuracy have been proposed [15]. In the proposed method, dealing with collision avoidance as a constraint is done in the same way as for safety as an optimisation objective, as described in Section 4.1. A special problem that is related to collision avoidance is compliance with COLREGS. Some of COLREGS rules that need to be taken into account include the majority of rules from Part B (Steering and Sailing), as listed below. As mentioned in the Introduction, COLREGS will be re-written in the next few years to include MASS. While the exact form of the new rules is unknown, it is reasonable to assume that changes will be made to minimise the impact of new rules on conventional vessels (otherwise, all navigators would have to be retrained to comply with the new regulations). Consequently, we can assume that a MASS will be obliged to act in a similar way to conventional vessels when engaged in an encounter; thus, the current COLREGS will be applied in the proposed method, and these can be updated as soon as the new rules for a MASS are introduced. As of now, there are no current COLREGS rules that would regulate the negotiation of collision avoidance manoeuvres between ships. We therefore assume that a MASS will manoeuvre with no direct ship-to-ship communication other than broadcast and received AIS messages. Another issue is the possibility that a MASS may encounter a conventional (manned) ship that is obliged to give way, but does not do so. An unexpected manoeuvre (noncompliant with COLREGS) by a manned ship may drastically change the navigational situation and result in an immediate collision threat. In both cases, the MASS will need to react quickly, within a shorter time than usually allowed for planning and executing evasive action. In such cases, a multi-objective optimisation of the trajectory of the MASS may not be possible, and a simple evasive action may need to be applied that can be determined automatically by the algorithm used in [37].
To speed up convergence to the optimal Pareto set, a vast array of specialised operators designed to eliminate particular types of problems is used. These operators have previously been designed for the purposes of research by the present authors [35] although this involved a single-objective optimisation problem rather than a multi-objective one. The abovementioned extensions to the EMO method include: • operators avoiding collisions with other ships, • operators avoiding grounding, collisions with stationary obstacles and violations of various limitations of a waterway, • operators avoiding violations of selected COLREGS rules (listed above).
The problem of complying with basic COLREGS rules related to collision avoidance (Rules 6 to 9 and 13 to 18 of COLREGS) [17] was addressed by the main author in [34], and the problem of collision avoidance in restricted visibility (Rule 19 of COLREGS) was addressed in [35]. Navigating within traffic separation schemes (Rule 10 of COLREGS) was examined, and the developed method automatically generated traffic patterns for compliance with particular rules (used at the pre-processing stage of the evolutionary process). Other optimisation techniques have included: • semi-deterministic operators dedicated to eliminating specified problems of an evolutionarily planned ship trajectory, • modifications to the traditional scheme of evolutionary operations that can reduce the number of most timeconsuming operations.
All of the above result in much greater progress within each generation, and consequently a much faster convergence of the optimisation process. These elements will also be applied in the proposed method. The expected effect is convergence to an acceptable solution within one minute, and further refining of this solution is possible if the situation allows (i.e. there is no immediate danger). In the case of a dangerous situation that has already developed, as observed in [13], a simplified approach referred to as 'fast reasoning' is recommended. In this case, fast reasoning can be carried out by reducing the optimisation problem to the single-objective one of minimising collision risk. It is worth noting that the approach proposed here is considered sufficient for successful collision avoidance actions of the ship, although it obviously cannot be applied to the multi-objective synchronisation of multiple ships supervised via a Vessel Traffic Service(VTS) centre.

SUMMARY AND CONCLUSIONS
The design of a MASS and MASS-related features is one of the most important and challenging topics in today's marine engineering. It is only a question of time before MASSs are exploited on a large scale, and it is therefore crucial to develop methods that minimise the risk of collision and reduce costs and pollution, which both depend strongly on fuel consumption.
At present, published works on collision avoidance methods (for both manned vessels and MASSs) almost exclusively use a single-objective optimisation approach, making it practically impossible to successfully achieve safety-and economy-related goals at the same time and in parallel. The method presented in this paper represents an attempt to develop such solutions. By applying this method, it is possible to reduce the collision threat as far as possible without a significant increase in the abovementioned costs, which are incurred by the ship-owners and the natural environment alike. The proposed method proves that it is possible and desirable to replace the predominant single-objective optimisation approach to collision avoidance with a truly multi-objective one. Additional research into the modelling of the motion of ships will help in understanding the influence of manoeuvring conditions and sea states on fuel consumption, as well as the interactions induced by the motion of the ship and the environmental variables.