Journal of Robotics and Automation

A smart Remotely Operated Vehicle, ROV LATIS, is a prototype platform to test and validate OceanRINGS-a suit of smart technologies for subsea operations, developed at Mobile & Marine Robotics Research Centre, University of Limerick. It is a next generation smart ROV with unique features, including multiple modes of operation, advanced 2D and 3D displays, intuitive, versatile and easy to use pilot interface, built-in autotuning of low-level controllers, voice navigation and fault-tolerant control system. With integrated state-of-the-art navigation sensors/instruments, the vehicle can achieve precision navigation and subsea positioning. Other features include ROV high precision dynamic positioning (DP) in absolute earth-fixed frame or relative to ship, robust speed/course controller with independent heading control and improved ROV-ship link. System validation and technology demonstration was performed through a series of test trials with different support vessels off the north, south and west coast of Ireland, in Donegal, Bantry Bay, Cork Harbour, Galway Bay, Shannon Estuary and La Spezia, Italy. This paper highlights the main features of the system, presents selected results of test trials and discusses implementation issues and potential benefits of the technology.


Introduction Background
Future deployment, installation & maintenance of ocean energy devices require use of underwater robots and support vessels. However, the same technology will also be used by other offshore industry e.g. off shore oil & gas. These vessels may be very expensive and, moreover, their costs are very volatile, depending on offshore peak demands. Thus, it is important to address the requirements for vessels to be used in ocean energy deployments and how these requirements may be configured to reduce the costs of these vessels and, simultaneously, affect technology development.
Due to the complexity of the underwater environment, the vehicle control system must avoid obstacles and compensate for various external disturbances acting on the vehicle (ocean currents, drag effects of umbilical for ROVs, etc.) in the presence of uncertainties in determination of increasing the survey mission costs. In addition, a non-optimal control allocation of actuators leads to inefficient usage of available energy resources. Future intelligent control systems with high degrees of autonomy should perform well under significant uncertainties for extended periods of time, and they must be able to compensate for certain system failures without external intervention [1].
Dynamic behaviour of the vehicle depends upon the spatial configuration of the onboard sensors and equipment. Some ROV applications require precision navigation and positioning capabilities which can be achieved with Fibre Optic Gyro (FOG) based Inertial Navigation Systems (INS). These sensor suites are expensive and often ocean engineering companies will forego use of such technology on cost grounds and use inferior sensor solutions with reliance on piloting skills. Design of smart ROVs is an open challenge for the underwater robotics community. For ROVs, both with FOG based INS systems and without, operation of the vehicles remains a very skilled task and the industry is very dependent on the skills of ROV pilots. However, with the precision instruments significant levels of automatic control functions can be achieved, including absolute DP (ROV dynamic positioning regardless of ship motion), relative DP (ROV dynamic positioning relative to ship motion), and robust speed/course controller with independent heading control. With added autopilot functionality and advanced real-time visualisation, ROV operations can be made significantly less dependent on the skills of ROV pilots and the range of tasks that can be successfully undertaken is expanded [2]. Interesting application of ROV serving as a docking station for Autonomous Underwater Vehicle (AUV) during Search and Rescue (SAR) missions has been proposed in [3]. The authors showed in simulation that the ROV has been able to cap-ture the AUV in presence of ocean currents and wave disturbances. The approach has been extended to include docking hoop for recovery of autonomous underwater vehicle in [4]. This is continuation of work on modelling and simulation software of under actuated ROV [5].

Research activities in mobile & marine robotics research centre
Over the last ten years, researchers in the Mobile & Marine Robotics Research Centre (MMRRC) at the University of Limerick have been focused on research in a number of areas, including AUV/ROV systems integration, embedded controller development, sensor systems development, modelling and real-time simulation of AUV/ROV dynamics & acoustic payload instruments and Airborne Wind Energy (AWE). Since 2006 the MMRRC research team has developed the OceanRINGS (previously known as MPPT Ring, [6]), a set of components that should be installed on an ROV and inside the Control Cabin, in order to increase the level of automation, to make ROV operations easier and save expensive ship time by 20% or more. In order to test OceanRINGS validity and viability, new flexible, multimode of operation survey class ROV LATIS was designed at UL in period 2006-2009 [7]. To date, the MMRRC team deployed the ROV LATIS ( Figure 1) from the RV Celtic Explorer & Celtic Voyager (Marine Institute, Galway), L.E. Eithne (Irish Navy), Shannon One Multi-Cat (Shannon Port Company) and LEONARDO (NATO).

Outline
The outline of the paper is given in the following. Section 2 describes design requirements and the main features of ROV LATIS. Selected results from field trials are presented in Section 3. Implementation issues are discussed in Section 4. Finally, Section 5 summarises the concluding remarks and potential benefits of the technology. • It should be able to detect, isolate and accommodate thruster faults.
• It should be able to work in full automatic mode, i.e. automatic way-points navigation.
• It should be able to keep desired position and orientation deeply submerged in the water (dynamic positioning), in presence of ocean currents.

Features
• The main features of ROV LATIS are given in the following (for full description see [8]): • Modular design with multiple modes of operation, • Very high positioning accuracy of ROV in deep water, • Semi-Automatic Speed Modes enable robust, stable and accurate ROV Course Following & ROV Dynamic Positioning with simple mouse click, • Fully automatic way points navigation with auto-compensation of ocean currents and umbilical drag effects, • Advanced 2D and 3D real-time visualisation-provid-

Design requirements
A number of design requirements have been identified prior to start of design & development work on testbed platform (ROV LATIS) in 2006.
Key design requirements for the new platform included: • It should be able to support the core survey suite, including multibeam, sidescan, Inertial Navigation System (INS), Doppler Velocity Log (DVL), pressure (depth) sensor and Ultra-Short Base Line (USBL) transponder.
• It should accommodate both subsea survey activities and wide area (surface) survey activities in order to maximise equipment utility.
• It should be deployable using small surface support vessels, such as tugs, trawlers etc. thus reducing operational costs for inshore work, as well as suited to operations from larger survey vessels for off shore work.
• Control design requirements included the following: • It should have built-in auto-tuning features for low-level controllers.

B) Advanced Pilot Interface
The Advanced Pilot Interface ( Figure 3) presents all important control data to the ROV pilot using familiar graphic controls & indicators. The pilot is able to use a combination of touch display, joystick, gamepad, mobile device, mouse or keyboard as input devices to generate commands, switch operating modes and enable/disable low-level controllers.
Set points can be entered numerically (e.g. using numeric control fields) or graphically (e.g. moving instrument pointers by mouse). The pilot can also easily switch between manual mode, semi-automatic modes (Speed & Course Controller, Hold Position and Go To Position) and fully automatic mode (automatic navigation through way points).

C) 2D Topview display
The 2D Topview display ( • Built-in thruster fault tolerance and optimal control allocation for any thruster configuration, • Built-in auto-tuning of low-level controllers, providing optimal controller performance, regardless of changes in ROV configuration between missions, • Voice navigation-voice instructions are generated in real-time assisting ROV pilot to navigate and reach the target, • Modular software architecture and extensive interface library enable easy system adaptation to any ROV-ship combination.

A) Operation modes
ROV LATIS is a vehicle with multiple modes of operation. It can be operated on the surface as a survey platform either towed ( Virtual components (ROV, support vessel, underwater structures) have the same appearance as corresponding real-world components (same dimension, colour, etc.). They are driven by real-time measurements obtained through together on the screen, this intuitive high-contrast display is comfortable for ROV pilot's eyes, enabling long operation hours without eye focusing problem, commonly present on commercial pilot displays with separated displays of numerical values and indicators.

D) 3D Real-Time Augmented Reality Display
The 3D Real-Time Augmented Reality Display (Fig-Figure 4: 2D Topview display. , in contrast to typical ROV simulator displays, whose main design requirement is to make it "as real as possible".

E) Fault-Tolerant control system
In contrast to most common control architectures used in modern ROV industry (where ROV is equipped with basic I/O modules, while control synthesis is per-component interfaces. Waves on the surface are generated from estimated sea state, while the seabed is built from previous bathymetry data. The 3D Real-Time Augmented Reality Display provides better situational awareness for ROV pilot. Since this display is real-time aiding tool for ROV pilot, there is a significant difference when compared with ROV simulators: the main design requirement for 3D Real-Time Augmented Reality Display was to make it "as Performance of Heave (Depth) and Yaw (Heading) LLCs after autotuning is shown in Figure 6. It can be seen that, regardless of cross-coupling present due to simultaneous control of depth and heading, tuned LLCs yield excellent time responses, with minimum overshoot and optimal control allocation.
In fault-free case, the fault-tolerant control system uses thrusters over maximum-sized attainable command set in an optimal way, minimizing the energy consumption, the most suitable criteria for underwater applications ( Figure  7). In case of thruster faults, the control system automatically accommodates faults, keeping full controllability of ROV for low speeds and providing opportunity for safe continuation of a mission in presence of thruster fault(s). Full description of ROV LATIS control system can be found in [10].

F) Technical specifications
Technical specifications of ROV LATIS are given in Table 1.

G) Sensors and survey equipment
The sensor systems integrated on ROV LATIS ( Figure  8 FOG basic principle of work is described in the following: • 2 wave ring interferometer made of a multi-turn fibre coil is mounted for each plane, • Light is divided into two counter-propagating waves, • Waves recombine perfectly in phase after having travelled along the same path in opposite directions, • FOG rotation induces a difference of transit time (relativistic Sagnac effect), • This time is measured by interferometric means, • Pure INS resolves the inertial measurements in the navigation frame and updates the attitude, heading, velocity and position, • Extended Kalman Filter integrates measurements from the pure INS with external aiding sensors and provides corrections to inertial system errors.
formed topside), ROV LATIS is equipped with a full real-time embedded control system, which performs all necessary data processing and synthesis online, aboard the vehicle in real-time. The control system includes fast auto-tuning of low-level controllers, automatic thruster fault detection and accommodation, semi-automatic and fully-automatic control modes, and optimal control allocation of thrusters.
Between successive ROV missions, it is likely that some of the onboard instruments/sensors/equipment will be added/removed/replaced, leading to changes in dynamic properties of the ROV (mass, moments of inertia, drag properties, etc). Controllers optimally tuned for a particular vessel configuration will not give the optimal performance in the case of a change in configuration. Autotuning of Low-Level Controllers (LLC) is an advanced feature of the control system, yielding optimal controller performance, regardless of changes in configuration. Two types of autotuning algorithms have been developed. Velocity controllers are tuned by recording and utilising the force-speed static characteristics. Autotuning of position controllers (Heave (Depth) and Yaw (Heading)) utilises self-oscillations. The autotuning process [9] involves the following steps: (1) Generate self-oscillations; (2) Wait for transient stage to finish; (3) Measure amplitude and period of steady-state oscillations; and (4) Find new values of controller gains using tuning rules. A novel set of tuning rules for underwater applications has been developed, which provides the optimal performance of low-level controllers in the case of configuration changes and the presence of disturbances (waves & sea currents).    ship's anchor pulled in rising winds and the ship started to drift backwards ( Figure 11). The mission had to be aborted and a ROV "race" with ship started. The main objective was to bring the ROV to the safe zone on the starboard side of the ship, avoiding any interference between umbilical and ship's propeller. During the race, the Speed Mode "Go To Position" was activated and the Target Marker was moved on the screen by mouse. At the same time, ROV speed was increased to 1 m/s. Fortunately, the ship stabilised its position after 60 m drift, as can be seen in Figure 12.

RMS Empress of Britain (CE-12-012 Cruise):
Multibeam survey of RMS Empress of Britain, located 110 km west of Rosguill, Donegal at 163 m depth, was successfully completed in September 2012. During this survey horizontal thruster HT4 (aft port) and vertical VT3 (aft starboard) were disabled due to mechanical failures. However, ROV LATIS fault-tolerant control system accommodated faulty thrusters in optimal way, yielding excellent quality of acquired bathymetry data. After initial survey of wreck from

Field Trials
The list of field trials performed with ROV LATIS onboard different research vessels is given in Table 2. The main objectives of these trials included testing of smart technologies in various scenarios, such as high-resolution multibeam/video survey of ship wrecks, inspection of underwater structures, etc. Selected results from these trials are given in the following.

Ship wreck surveys
La Surveillante (CV-11-026 Cruise): Multibeam survey of French frigate La Surveillante, located 2 km NW of Whiddy Island in Bantry Bay (Figure 9), was partially completed. The initial plan included quick survey from the surface to locate the wreck, and detailed submerged survey close to the wreck to acquire high-resolution multibeam bathymetry and video. The first view of the ship wreck, obtained during the surface operations, is shown in Figure 10. Just before the ROV started the dive for near-the-wreck operations, the   high-resolution survey with ROV-mounted multibeam ( Figure 13). Runlines (desired tracks) parallel to shipwreck the ship-mounted multibeam, low-resolution bathymetry image was created and used as a background image for   arrival to the initial position R, ROV was rotated to initial heading (156°) and the target was visible on pilot camera view in approximately the same position as at the beginning of the experiment (Figure 19). The same motion pattern bottle attached to the weight at 11 m depth (Figure 18). Initial ROV position (point R in Figure 17) and heading (156°) were saved in temporary variables. Then, using the Speed Mode "Go To Position" the ROV was commanded to go to remote position P 1 25 m away (with all thrusters on). After arrival to P 1 ROV was ordered to return to initial position (point R), which was restored from temporary variables, but this time thrusters HT 1 was disabled (switched off). Upon ing significant swinging of anchored support vessel (Celtic Voyager) with magnitude ± 30 m on starboard/port side, making ROV control and management of umbilical cable very challenging task. Similar to conclusions of previous repeatability test, this experiment confirmed very high-quality of underlying positioning and fault-tolerant control system. To the best knowledge of authors, these results were not achieved before in real-world environment.

Rock Starfish
was repeated three times (remote points P 2 , P 3 and P 4 ) and each time different combination of enabled/disabled thrusters was used, according to Figure 17, and star-like path segments RP 1 , RP 2 , RP 3 and RP 4 were travelled by ROV in both directions. After each return to initial position the target was highly visible on pilot camera view monitor ( Figure  20). It should be emphasised that strong tidal currents were present on the ocean surface during the experiment, caus- In vertical plane voice instructions include UP (Z R -Z T > ε v ) and DOWN (Z R -Z T < -ε v ) (Figure 21b). ROV Voice Navigation system has been successfully tested during the field trials, as indicated in Figure 22a

ROV voice navigation
Similar to voice navigation used in car GPS nav systems, the ROV Voice Navigation system has been developed to assist ROV control & navigation during different mission stages, including ROV deployment and recovery. The original idea for voice navigation was developed by researcher A. Vasilijevic from LABUST research team, University of Zagreb [11].
Depending on relative position between ROV (R) and target (T), a set of the voice instructions is generated and continuously updated to assist ROV pilot to navigate and

Implementation
OceanRINGS technologies are applicable to the growing international offshore oil & gas industry, for future deployment, monitoring, inspection and maintenance of ocean energy devices, for rescue operations, etc.
It should be emphasized that the OceanRINGS software modules are developed in LabVIEW and can be deployed to various Real-Time (RT) targets including NI CompactRIO, NI myRIO and mini PC with Windows/ Linux operating system. Many research groups in underwater robotics use ROS as middleware to integrate and Figure 22b. While ROV pilot was absent from the Control Cabin, a member of the research team placed the target to a new position & depth using mouse, and switched off the monitors. Then, ROV pilot entered the cabin and started search for target, listening the voice instructions on PC speakers (Figure 22a). Using the voice instructions, ROV pilot was able to determine distance & direction to the target, and its depth, in the first step. Then, modifying the speed and course with SOG/COG controls, the ROV pilot navigated ROV to reach the target (Figure 22b).

New ROVs
Full potential of OceanRINGS smart technologies for subsea operations could be utilised during design & development process of new, next generation ROVs. Main features of new ROVs include: • Universal Architecture, • Ethernet as communication protocol, • Use of off-the-shelf components only, • Normalised control spaces and variables, • Standardisation (connectors, protocols, etc.), various components. Authors have developed interface for fast and reliable interface between OceanRINGS Lab-VIEW modules and ROS environment using UDP protocol for data exchange.
OceanRINGS can be used on any ROV/Support Vessel combination, and extended to other marine platforms. However, full potential is exploited using work-class ROVs, due to their payload capabilities. Different implementation strategies should be utilised depending on ROV target groups: Existing ROVs and New ROVs.

Existing ROVs
In order to enhance operations of existing ROVs and implement smart technologies, the overall OceanRINGS suite is integrated into the ROV Upgrade Kit (Figure 23). The ROV Upgrade Kit consists of Dry Components (installed inside the ROV Control Cabin on the ship deck, see Figure   a