An Inland Shore Control Centre for Monitoring or Controlling Unmanned Inland Cargo Vessels

Abstract

Augmenting the automation level of the inland waterway cargo transport sector, coupled with mechatronic innovation in this sector, could increase its competitiveness. This increase might potentially induce a sustainable paradigm shift in the road-dominated inland cargo transport sector. A key enabler of this envisaged shift may be an inland shore control centre (I-SCC) capable of remotely monitoring and controlling inland vessels. Accordingly, this study investigated the concept and design requirements to achieve an inland I-SCC that provides interaction services when supervising an unmanned surface vessel (USV). This I-SCC can help its operator to develop situational awareness and sensemaking. The conducted experiments offered insights into the performance of both the I-SCC system and its operator, and unlock research on the impact on ship sense and harmony when remotely controlling a USV. The Hull-To-Hull project extends the current I-SCC by providing enhanced motion control. This enhancement enables further performance insights and might improve the future monitoring of USVs. The successful I-SCC construction, the preliminary experiments, and the design-extension demonstrate that the I-SCC can serve as an experimental platform for both mechatronic innovation and human-automation integration research in the inland waterway sector, whilst additionally providing fruitful knowledge for adjacent research domains.

1. Introduction

The further automation of inland cargo vessels, with their potentially unmanned or autonomous operation modes in the near future, in conjunction with increased mechatronic innovation in the inland waterway sector itself, could help to exploit the under-used capacity of the smaller inland waterways. These developments could also pave the way for the integration of unmanned, automated, or autonomous surface vessels (USVs, ASVs, or ASVs) in synchromodal transport networks [1], even on an urban, i.e., city-logistics, scale. Although a spectrum of definitions for automated or autonomous can be found [2], they are often entangled with, or separated by, a level of human interference. This interference might hint at a human mediator in juxtaposition with an increasingly automated and independent cargo fleet as a technologically-feasible inland waterway transport concept in the near future. Consequently, the authors think that one of the key components of this envisaged concept could be an inland shore control centre (I-SCC) capable of remotely monitoring and controlling several USVs or ASVs simultaneously.

Over the last few years, the investigation in autonomous or unmanned vessel concepts has rapidly grown. The Finnish Advanced Autonomous Waterborne Applications (AAWA) project [3] proposed preliminary designs for advanced ship solutions. The first autonomous cargo vessel, the Norwegian Yara Birkeland, is on its way [4], whereas a Belgian scale model of an unmanned inland cargo vessel, the Cogge, already performed preliminary unmanned experiments [5]. Additionally, the European Maritime Unmanned Navigation, through the Intelligence in Networks (MUNIN) project [6], investigated a dry bulk carrier for intercontinental trade. Most of these just-mentioned concepts rely heavily on a SCC that can monitor or steer the vessels. The MUNIN project went into great detail for the technological design of a complete SCC—i.e., they included operators, back-up operators, watch keepers (supervisors, engineers, and captains), planners (voyage and maintenance), and administrative staff in their design [6,7]. Accordingly, they estimated a necessary total staff of 169 people to continuously monitor a fleet of 90 vessels dedicated to one SCC wherein an operator monitors six vessels at once. Such a novel SCC design also gives rise to human factor challenges, where other domains (e.g., aviation, cars, subway systems, etc.) can provide useful insights [8]. Furthermore, the operational context between an onboard bridge and onshore SCC differs significantly; i.e., one can most likely not just move the bridge to shore and expect it to work, given that the user–environment interactions and thus the actual work would change [9,10].

Consequently, this study used an activity-centred design (ACD) approach [11] to construct an I-SCC for five main reasons. Firstly, given that the final businesses model for a USV or ASV for inland cargo transport is not yet clear, the emphasis was put on the activity, i.e., remotely controlling or monitoring an USV or ASV via the I-SCC, which will most likely need to be performed independently of the final concept. Secondly, ref. [10] noted that, when having to monitor and control remote simulated unmanned cargo vessels via an SCC, operators tend to use the familiar navigational and collision avoidance technologies, but in a different way. Therefore, this study offers a flexible and modular combination of different technologies that should enable the activity of monitoring or controlling USVs. Thirdly, the first potential business models for short sea shipping and ferries seem to focus on short, local routes [4], not unlike inland cargo shipping, hence the focus on an inland SCC. Fourthly, one of the concluded next steps in the AAWA project stresses the need for the development and testing of specific technological solutions [3], such as the herein-constructed I-SCC. Finally, most of the existing SCC design research focusses on sea-going vessels and stays rather theoretical [3,4,6,7,12], although some studies have investigated the SCC design by controlling or monitoring simulated vessels [9,10], whereas our developments pave the way to studying real ship-shore interactions. In summary, this study aimed to:

I. 

Use an ACD approach [11] to develop an I-SCC concept which provides its operator with interaction possibilities with a USV, where interaction denotes the remote access of the USV actuation system and overall system settings. Accordingly, this I-SCC aims to serve as a tool to enhance the situation awareness [13] and sensemaking [14] capabilities of the remote operator. Thus, the I-SCC unlocks research on the impacts on ship sense [15] and harmony [16] when remotely monitoring or controlling a USV.

II. 

Translate this I-SCC concept into four design requirements: (r-i.) provide the relevant information groups of [12], (r-ii.) provide interaction with the USV, (r-iii.) select industrial components, and (r iv.) keep the design modular.

III. 

Provide technological details of the merged I-SCC and USV system design in accordance with these four design requirements.

IV. 

Offer preliminary experimental data of an operator remotely controlling a USV via the I-SCC. These experiments investigate the performance of the system and of its operator.

V. 

Illustrate the modular design philosophy by extending the I-SCC design within the Hull-To-Hull (H2H) project [17] which augments the remote motion control of a USV via the I-SCC. The change in operator performance due to this extension was also investigated.

The successful accomplishment of these five research aims will offer insights for potential paradigm shifts in the transport sector on the one hand, and for human factor research within the mechatronic innovations in the inland waterway sector on the other hand. In addition, the modular, industrially-relevant, bottom up approach will enable future cost-benefit analyses (CBAs) of the I-SCC system design by means of an engineering estimate approach [18]. This paper continues as follows: Section 2 sketches the background of unmanned inland shipping in order to explain the context in which the I-SCC will be operated. Afterwards, Section 3 discusses the I-SCC design concept together with its design requirements, which resulted in the I-SCC design of Section 4. Thereupon, Section 5 elaborates on the first in situ experiments, Section 6 discusses their results, and Section 7 concludes this study.

2. Background: Towards Unmanned Inland Shipping

Section 2.1 describes the current motivation for further mechatronic innovation on the inland waterway cargo transport sector. Afterwards, Section 2.2 details the current research vessels within the Intelligent Mobile Platforms (IMP) research group of the KU Leuven, which was/will be used during current and future experiments to verify and validate the design of the I-SCC. Furthermore, this section lists the current line-of-sight shoreside components used to interact with these vessels.

2.1. The European Inland Waterway Transport Sector

Despite generating larger external costs which can be defined as costs caused by one entity but imposed on another [19], road-based freight transport dominates the cargo transport sector over the European hinterland with a modal share of approximately 75% of the transported ton-kilometres, which remained rather constant over the last two decades [20]. Although at present, no scientific consensus exists to determine the exact prices of these external costs [21]—such as accidents, air pollution, climate, noise, and congestion—several studies [22,23,24,25] concluded that inland waterway transport offers a more sustainable alternative than road-based transport. Therefore, the European Commission wants to promote inland waterway transport as a competitive and resource-efficient mode of transportation [26], and to induce a freight transport modal shift from road to rail and waterborne transport [27]. The policy priority of the Flemish Government to increase the utilisation and the expansion of its dense inland waterway infrastructure [28] demonstrates a similar orientation of the regional freight transport policies.

Nevertheless, the European inland waterway network remains under-exploited and it even witnesses an outflow of the smaller CEMT type I-II [29] vessels [30]. A non-exhaustive list of the main reasons for this noted outflow of smaller vessels seems to be: (i) few technological improvements, (ii) a decrease in new skippers, (iii) high crew costs, (iv) inadequate waterway maintenance, and (v) a negative investment climate [21,24,31]. Two recent developments aim to help bridge this gap between the noted outflow of small inland vessels and the desire to use the full capacity of the inland waterways and thus also their narrower hinterland connections. On the one hand, the small pallet shuttle barges from Blue Line Logistics need only one crew member and have an onboard crane for transloading their cargo independent from the shoreside infrastructure [30]. On the other hand, the European Watertruck+ [32] project introduces modular push vessels and passive or active barges in order to decouple sailing and transshipment time.

2.2. Unmanned Inland Cargo Vessels

The IMP group currently has scale models of the two above-mentioned novel vessel types: the Watertruck and the pallet barge, discussed in Section 2.2.1 and Section 2.2.2 respectively. The former will be explained in more detail at the component, and internal and external communication level, in conjunction with its line-of-sight shoreside components in Section 2.2.3, in order to serve as a running example for a remotely controlled USV throughout this paper. Note that within the unmanned or autonomous vessels literature [33], few to no research platforms investigate cargo vessels, though these platforms could generate crucial information for the cost calculations of such a real-size vessel and its potential business models [34], and perform a safety investigation regarding deploying such a vessel [35]. Therefore, the chosen scale models, and their sizes are meant to provide input for these and other research areas regarding inland shipping on the one hand, and to investigate the penetration of small vessels into city logistics, e.g., [36,37,38], on the other hand.

2.2.1. The Cogge-Watertruck${}^{+}$ Scale Model

Figure 1a depicts four real-size barges, pushed by a push boat, produced in the Watertruck${}^{+}$ project [32], whereas Figure 1b depicts the IMP scale model of a self-propelled CEMT type I Watertruck${}^{+}$ barge sailing unmanned in the Yser river. In [5], the authors detail the design, build, and first experiments of this USV which they named the Cogge. Given the chosen geometric scale factor of 8, the Cogge has a length of 4.81 m, a beam of 0.63 m, and a maximal draft of 0.35 m. The self-propelled barges of Watertruck${}^{+}$, and thus the Cogge, have a non-conventional propulsion system consisting of two embedded 360-degrees-steerable actuators: a steering-grid thruster positioned at the bow and a four-channel thruster placed at the stern. Both thruster systems have a propeller that draws water from underneath the vessel hull and a steering mechanism, with the internal control angle ${\theta }_i$ that orients the outflow of the water stream. Figure 2a,b present an abstracted working principle of these thrusters and their angle convention of which more information can be found in [39].

Figure 3 illustrates all the vital inter and intra-communication links between the main onboard components of the Cogge; the part numbers and abbreviations refer to

Table A1. List of extended I-SCC and USV system components: their descriptions, abbreviations or names, and specific types.

Nr.	Description	Abbreviation/Name	Type
1	Programmable logic controller	Onboard PLC	Wago PFC200 750-8207
2	Industrial computer	I-PC	Moxa MC-7200-MP-T
3	Power over Ethernet switch	POE switch	Wago 5-port 1000 Base-T Industrial Eco Switch
4	Industrial Router USV	Quartz Router USV	Siretta,Quartz-W22-LTE, 4G/LTE, WiFi, 2 LAN/2 SIM Port Router
5	Radio receiver	RR	Danfoss MPCAN
6	Antenna LTE (PLC)	PLC antenna	LTE Antenna for PLC
7	Antenna LTE (Quartz)	LTE antenna	LTE Antenna for Quartz LTE
8	Antenna Wifi (Quartz)	Wifi antenna	Wifi Antenna for Quartz LTE
9	Bow thruster motor drive	Bow motor drive	Roboteq MBL1660A
10	Bow angle integrated stepper	Bow angle quickstep	JVL MIS234S
11	Bow thruster motor	Bow motor	Turnigy RotoMAx 150cc
12	Stern thruster motor drive	Stern motor drive	Roboteq MBL1660A
13	Stern angle integrated stepper	Stern angle quickstep	JVL MIS343
14	Stern thruster motor	Stern motor	Turnigy Aerodrive SK3-6364-245KV
15	Stereo cameras (2x)	Stereo cameras	Custom built, UI-5280FA-C-HQ Vision++
16	Inertial measurements unit	IMU	EKINOX2-E-G4A3
17	Navigational GNSS sensor	GNSS	Septentrio AsteRx-U MARINE
18	Laser scanner	LIDAR	Neptec OPAL-1000
19	Emergency stops (4x)	ESTOP	Twist to reset 40 mm Mushroom
20	Bilge pumps (3x)	Bilge pumps	Rule Bilge pump 800
21	Stern light	Stern light	LED white 12–24 V
22	Directional lights bow	Port/Starboard light	Allpa LED 2 colors 8–30 V
23	Battery monitoring system	BMS	Mastervolt-Amperian interface
24	Battery 24 V	24V DC	Navex
25	Battery 24 V (2 × 12v)	2 × 12V DC	Navex
26	Wind Sensor	Wind Sensor	Rugged NMEA0183 Wind Transducer WND100
27	Network cameras (4x)	IP cameras	Panasonic WV-S3531L
28	LTE Router	Pepwave LTE Router	Pepwave MAX Transit MAX TST DUO LTEA W T
29	12 Port Switch	Zyxel Switch	Zyxel 12-port GbE Managed PoE Switch RGS200-12P
30	Wearable remote control	RC	IK3 Danfoss
31	Rugged shore side laptop	R-PC	Dell latitude rugged 7204
32	Monitor stern	Stern displ.	Samsung C27F591FDU Color Display Unit
33	Monitor bow	Bow displ.	Samsung C27F591FDU Color Display Unit
34	Monitor starboard	Stbd displ.	Samsung C27F591FDU Color Display Unit
35	Monitor port	Port displ.	Samsung C27F591FDU Color Display Unit
36	Monitor information	Ctrl displ.	HP LD5512 UHD 4K Conferencing Display
37	Monitoring Computer	SCC PC	Intel i9X based PC
38	Programmable logic controller	Shore PLC	Wago PFC200, 750-8207
39	Azimuth controllers (2x)	AC	Verhaar Omega IVOP-BS-01
40	Motion control touch screen	Touch screen	Wago Touch Panel 600, 762-4103
41	Industrial Router SCC	Quartz Router SCC	Siretta,Quartz-W22-LTE, 4G/LTE, WiFi, 2 LAN/2 SIM Port Router
42	VPN server	VPN server	OpenVPN
43	LTM942 Antenna	Multi Band Antenna	Mobilemark Dual Carrier MIMO Multi-Band Mobile Antenna LTM942
44	MBR Antenna 179	MBR 179 Antenna	Kongsberg Maritime Broadband Radio MBR 179
45	H2H Combined Engine	H2H Engines (combined)	Kongsberg Rack, 2x DPS R+ HMI, 2x DPS 232 R+, Display, Keyboard
46	4G LTE Router	Pepwave Router	Pepwave MAX BR1 Embedded 4G LTE Automatic Failover Router
47	GNSS Antenna shore	NovAtel Antenna	NovAtel GPS-713-GGG-N, GNSS & GPS Antenna
48	Dynamic positioning systems	DPS (2x)	2x Kongsberg DPS 232 R+
49	MBR Antenna 144	MBR 144 Antenna	Kongsberg Maritime Broadband Radio MBR 144

. This figure demonstrates the modular hardware design which can be split into three main subsystems: actuation, sensor, and line-of-sight control. The latter is detailed in Section 2.2.3; the former two can be part of the higher-level motion control and autonomy subsystems respectively. To achieve this topology, the actuation subsystem has to be connected with the PLC which orchestrates the desired actuation system states that it can receive from three points: the RC via radio, a web-based interface via 3G, or the I-PC via Modbus TCP/IP. In addition, in order to operate the Cogge as a USV or ASV, the sensor subsystem should be connected with the I-PC which runs the autonomy software. This modular and cascaded lay-out achieves an increased level of operational redundancy and flexibility, crucial for an experimental research platform.

2.2.2. The Maverick-the Pallet Barge Scale Model

Figure 4a shows a pallet shuttle barge handling palletised cargo with its onboard crane at an inland quay [40]. Figure 4b illustrates the functional scale model of this real-size vessel from IMP named Maverick. This Maverick has a length of 6 m and a width of 2 m, and its flat deck can carry one tonne of, for example, palletised, cargo. The current propulsion system consists of two Torqeedo Cruise 2.0 outboard motors (not shown in Figure 4b) which can be independently controlled in rotational speed and in orientation angle, as drawn in Figure 4c. Both the starboard and port motor orientation angles, i.e., ${\theta }_i^p$ and ${\theta }_i^s$, have a theoretically limited steering range of ${\theta }_i\in [-42,42]$° due to the steering mechanism design (not drawn). Currently, the actuation system design of the Maverick is being updated to mirror the design of the Cogge: a PLC will orchestrate the actuation and thus motion control subsystem of Maverick, hence providing a highly similar connection interface for remotely accessing and controlling the vessel.

2.2.3. Line-Of-Sight Control Subsystem

Figure 5 reveals the three main external devices which were used, either on shore or on a support vessel, during the unmanned operations of the Cogge documented in [5]. A wearable remote control, see Figure 5a, is used to position or manoeuvre the vessel in between missions when desired. This remote has a screen which can access a mobile-friendly version of the PLC web-interface. Both the industrial computer on the Cogge and the shoreside monitoring rugged laptop, see Figure 5b, ran the Mission Oriented Operation Suite (MOOS) [41], allowing the latter to monitor the MOOS processes on the former, which also ran MOOS-IvP [42] processes to provide the vessel with autonomy [5]. Finally, Figure 5c depicts the web-based interface accessible via any internet-connected device to monitor the relevant onboard PLC parameters and to remotely control the vessel if necessary, not unlike the developments in [43,44].

3. Concept and Requirements of an Inland Shore Control Centre

The line-of-sight shore components of Section 2.2.3 unlocked a potential for human monitoring and controlling a USV, and together with the motivation and insights of Section 1, this potential inspired the ACD concept of an I-SCC, handled by Section 3.1. Section 3.2 lists the design requirements to achieve this I-SCC concept and afterwards Section 3.3 discusses the H2H navigation concept [17] which extends this I-SCC by providing augmented motion control of the USV. Finally, Section 3.4 addresses the I-SCC risk and safety analysis.

3.1. Concept of an Inland Shore Control Centre

Section 3.1.1 explains the concepts of ship sense and harmony, and the influence on them when an operator remotely controls a vessel. Afterwards, Section 3.1.2 looks at these concepts from another perspective, and Section 3.1.3 forms the ACD concept of our I-SCC based on Section 3.1.1 and Section 3.1.2.

3.1.1. Ship Sense and Harmony

Conventionally, onboard crew members acquire a certain degree of ship sense [15] with which they handle the ship in order to keep harmony [16] between the ship and its environment. Ideally, this harmony results in a safe and pleasant journey for the vessel, crew, cargo, and passengers [16]. Parts of these ship sense and harmony seeking concepts might be compatible with embodied sensemaking [45], which in addition to the more-studied cognitive and linguistic sensemaking literature [46] also investigates the effects of intuitions, sensations, and emotions on how one interprets and acts in the environment [14,45]. These potential compatibilities can be seen in the three harmony sub-categories: (i) environmental prerequisites (context and situation), (ii) vessel-specific prerequisites (inertia and navigational instrumentation), and (iii) personal requisites (spatial awareness, theoretical knowledge, and experience) [16]. Evidently, an operator in a remote SCC loses a direct ship sense, and thus the subsequent harmony with the environment [47], which complicates the ship handling. Therefore, to study the necessities for adequate situation awareness in a SCC, ref. [12] interviewed six bridge officers and uncovered 165 pieces of necessary information which they bundled into nine groups: (i) voyage (e.g., voyage plan), (ii) sailing (e.g., heading data), (iii) observations (e.g., video cameras), (iv) safety and emergencies (e.g., bilge pumps status), (v) security (e.g., video of ship itself), (vi) cargo stability and strength (e.g., stability system status), (vii) technical (e.g., engine parameters), (viii) shore control centre (e.g., voice communication other vessels), and (ix) administrative (e.g., log books). Furthermore, ref. [9] performed several scenario-based trials with a SCC monitoring virtual vessels to study the aforementioned harmony model and the adequacy of these nine information groups for situation awareness. Two of their main conclusions were the importance of a proper alarm management system and the fact that a SCC should not mimic the bridge layout. This latter conclusion was again emphasised in [10] which noted the tight coupling between the user and the environment that could be taken into account by means of an ecological design approach [48].

3.1.2. Situation Awareness, Sensemaking, and Interaction

At a different level of analysis, these concepts of ship sense and harmony seem to span the areas of situation awareness, sensemaking, and interaction. Situation awareness and its different levels are understood as, “The perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” [13]; note that [49] clarifies some of the common misconceptions or misunderstandings of this definition. Likewise, sensemaking refers to: “The interplay of action and interpretation rather than the influence of evaluation of choice”, as defined by [50], or alternatively: “sensemaking is the process through which people work to understand issues or events that are novel, ambiguous, confusing, or in some other way violate expectations” [14]. Finally, within this study, interaction indicates both the possibility of remote motion control actions from an operator and the option of changing onboard operational settings. These three concepts are not mutually exclusive; on the contrary, they seem to have codependent sub parts all centring around the human operator.

3.1.3. Activity-Centred Design Concept of an Inland Shore Control Centre

The ACD approach of this study focusses on providing a tool, i.e., the I-SCC, to remotely control or monitor an ASV or USV, whereas a human-centred design (HCD) approach would focus more on making the tool invisible [11]. Nevertheless, given that the current users, i.e., the researchers, are also the designers of the I-SCC, user-centred design (UCD) and HCD choices were often implicitly incorporated. The I-SCC should offer the possibility of interaction with the USV in order to help its operator to develop certain levels of situation awareness and sensemaking (see Section 3.1.2). Given that the first experiments and usage (see Section 4.5) of this I-SCC focus on (continuously) remotely controlling a USV, situation awareness is herein understood as a cognitive construct, although this definition can be criticised [51]. When future experiments focus more on the monitoring activity, the I-SCC concept will shift further towards a joint cognitive system (JCS) [52,53]. In this socio-technical system, redefining the situation awareness towards a distributed situation awareness [54] might be more suited. Section 3.2 will translate these concepts of interaction, situation awareness, and sensemaking into design requirements for the I-SCC.

3.2. Design Requirements of an Inland Shore Control Centre

Four main design requirements (r-i.–r-iv.) were judged necessary in order to realise the I-SCC concept of Section 3.1.3. Together, they aim to provide a fully-operational, industrially-relevant experimental set-up enabling both current research on I-SCCs and on mechatronic innovation for the inland waterway transport sector, whilst attempting not to block future potential expansions thereupon:

(r-i.)

Generate and communicate elements of the information groups [12] (ii) sailing, (iii) observations, (iv) safety and emergencies, (v) security, and (vii) technical to the I-SCC. These groups will enhance the situational awareness and sensemaking abilities of the remote operator.

(r-ii.)

Provide interaction possibilities with the USV in order to remotely alter the USV motion and its system configurations. This interaction will also assist the sensemaking and situation awareness capabilities of the operator.

(r-iii.)

Install industrial, marine-grade components for both the extensions on the USV and the I-SCC system design. This requirement makes the overall system more safe, more robust, and fit closer to a potential future reality.

(r-iv.)

Keep the system design modular and flexible, where possible. This flexibility smoothens the likely design iterations and potential future system extensions.

These requirements illustrate the activity-centred, bottom-up design focus for constructing the USV and I-SCC ecosystem. As emphasised by (r-iv), this focus allows for design iterations and provides a foundation for future extended designs. Some of the known design limitations are the disregard of the remaining information groups [12]: (i) voyage, (vi) cargo stability and strength, (viii) shore control centre, and (ix) administrative; during the first experiments a voyage plan, transporting cargo, communicating with other vessels, and overall administration are not deemed crucial. Furthermore, the accountability for the different control modes [55] (e.g., unmanned, automated, remotely controlled, surface vessel) has not yet been implemented: under the assumption that, during the first experiments, the operator is fully aware of the goal of the tests, and that these different modes (their definition and their impact) remain part of the ongoing research. In addition, the current design makes an abstraction of the potential onboard social hierarchy of the crew [6,7,9], as there is only one operator, and of the complexity of managing a fleet [6,7] as only one USV will be controlled. Finally, no explicit design focus has been put on ergonomics. Given the flexible physical system lay-out, this design can be tuned during experiments, fitting it to the human, and not the other way around [56]. Likewise, mental work load [57] information could be fetched during experiments, which could induce design iterations. These potential iterations and the design flexibility, will help to avoid the unadvised mimicking of a bridge lay-out [9,10].

3.3. The Hull-To-Hull Navigation Concept

The H2H project offers a first conceptual extension on the I-SCC concept of Section 3.1.3. This H2H project fits within the European Horizon 2020 program, and aims to provide the hull-to-hull distance between H2H objects, which can be either stationary (e.g., a dock), or dynamic (e.g., a vessel) [58,59], in combination with the uncertainty of this distance and of the positions and orientations of these H2H objects. Figure 6a draws an exemplary uncertainty zone in red around a moving H2H object. This zone, together with the dynamics of the ship and its operation mode, could also be used for cascaded, dynamically-changing uncertainty zones, shown in orange and green [60,61] which could be coupled to triggering certain messages when interactions with other vessels in this area occur. Within this project, each H2H object has an H2H System which normally consists of: (i) position and movement sensors, (ii) 2D or 3D geometry models, (iii) an H2H engine to perform calculations, (iv) a data communication link, and (v) an H2H application or user interface. This overarching H2H System design could facilitate close proximity manoeuvring between H2H objects. Accordingly, the IMP research group will test this H2H design philosophy for inland vessels by two types of experiments: single-handed sailing and single-handed docking. Here, the underlying hypothesis would be that the H2H System would allow a vessel operator to single-handedly perform close proximity manoeuvres when using the H2H Application. Figure 6b illustrates such a docking manoeuvre, where the numbers indicate the sequence of envisaged waypoints. Accordingly, the vessel will arrive near the dock, initiate a starboard docking manoeuvre, continue sailing, turn around, and finally perform a second docking manoeuvre from port side.

Since it is physically impossible to board the Cogge, this USV conducted these experiments while being remotely controlled by a human operator in the I-SCC. Given this unmanned nature of the Cogge and the fact that this USV should only avoid (straight sailing) or encounter (docking) the shoreline, a modification in the H2H system design has been made, depicted by Figure 6c which leverages our modular I-SCC design. Instead of having two H2H engines, i.e., one for the USV and one for the dock, both have been combined into one engine. In order to achieve this, the USV needs to send its H2H-engine-relevant data to this engine. Furthermore, given that the human will use the H2H-application, this information needs to be sent from the H2H-combined engine to our I-SCC where it will be shown on a display. Finally, the motion control and monitoring parts of the I-SCC will be used as discussed in Section 4.1 and demonstrated in Section 4.2.

3.4. Risk and Safety Analysis of an Inland Shore Control Centre

Despite the scarcity of research regarding industrial cargo USVs [33], their potentially complimentary SCCs [6,7,45], and thus their subsequent operational modes or business cases, several studies investigated the potential risks, hazards, and safety concerns for these concepts [35,62,63,64], and their integration in the existing International Maritime Organisation regulations [65]. Nevertheless, the usefulness of having more operational USV and SCC concept data remains a common conclusion in most analyses. Therefore, in line with the suggested engineering estimate approach for the potential future CBAs and the overall activity-based, bottom-up, modular system design, the risk assessment concentrates on the lower technical level (e.g., selecting industrially-robust components, having multiple interaction possibilities with the vessel, having onboard safety stops). Consequently, the I-SCC and its induced experimental data can nurture the just-mentioned risk and safety analyses, for which the systems-theoretic accident modelling and processes (STAMP) [35,66] seems to be a good future methodology as it reviews the entire socio-technical system [35,67,68], which will be of paramount importance given the envisaged human–machine interactions. Finally, note that the Flemish waterway administrator continuously defines, monitors, and optimises a legal framework for testing and demonstration purposes with USVs, generating mutually beneficial discussions regarding USV safety and regulations.

4. Design and Construction of an Inland Shore Control Centre

Section 4.1 summarises the resulting technical I-SCC design based on its concept and requirements, i.e., nurtured by Section 3.1.3 and Section 3.2, and Section 4.2 depicts its physical construction. Similarly, Section 4.3 shows the technical details of the H2H-extended I-SCC and Section 4.4 illustrates its physical construction. Finally, Section 4.5 discusses the design of the performed experiments.

4.1. Technical Design of an Inland Shore Control Centre

Figure 7 represents the full I-SCC design in conjunction with the USV of Section 2.2.1, i.e., the Cogge. The additions to this USV, in order to accomplish the I-SCC design requirements together with the I-SCC design itself are detailed in the following three sections, as summarised by

Table 1. Resulting I-SCC design based on the design requirements of Section 3.2.

	Requirement	Design Result	Discussed in
(r-i.)	Information Groups	Figure 7: sensor and I-SCC subsystem, PLC	Section 4.1.1
(r-ii.)	USV Interaction	Figure 7: I-SCC subsystem	Section 4.1.2
(r-iii.)	marine-grade, industrial components	Table A1	Section 4.1.3
(r-iv.)	modular and flexible design	Figure 7	Section 4.1.3

, with reference to the full components list of Table A1.

4.1.1. Design Results for Requirement One (R-I.)

Table 2. Summary of information group realisations.

Information Group	Design Result
Sailing information	Figure 7: autonomy sensor subsystem
Observation information	Figure 7: autonomy and monitoring sensor subsystems
Safety and Emergency information	Figure 7: PLC
Security information	Figure 7: monitoring sensor subsystem
Technical information	Figure 7: PLC

details the USV and I-SCC components or subsystems that provide the necessary data for the selected information groups, with reference to Figure 7. Comparing this figure with the components of the Cogge as a USV, i.e., Figure 3, demonstrates that some of the already-onboard components, namely, the autonomous part of the sensor subsystem and the PLC (number 1), can now be used for multiple purposes. More precisely, the GNSS and IMU sensors, which normally feed the autonomy software nested on the I-PC, can now stream their data to the I-SCC, thus providing Sailing information. Similarly, the Stereo Cameras and the LIDAR can produce observation information, if desired. Furthermore, the onboard PLC monitors all wanted parameters and alarms to provide both Technical information, and safety and emergency information to the I-SCC. The PLC web interface accumulates this information, similar to Figure 5c. In addition to these reused components, the USV has four new cameras installed: port, bow, starboard, and stern orientation. Their images offer security information on the one hand, and given the small USV size, these can also be used as observation information on the other hand.

Three mobile internet connections can pipe these just-discussed information sources to the I-SCC and its display screens: the PLC transmits its technical, and safety and emergency information via a dedicated 3G connection, (ii) the Quartz router (number 4) is capable of streaming the observation information, and (iii) the Pepwave router (number 28) streams the security information. Note that an offshore VPN server (number 42), in conjunction with nested VPN clients shown on some components, facilities the data routing and provides remote access to all these services.

4.1.2. Design Results for Requirement Two (R-Ii.)

With the accomplishment of (r-i.), the operator should have a degree of situation awareness and sensemaking capabilities, which both can potentially be improved by interaction with the vessel. This interaction in the form of remote motion control, i.e., when remotely steering the vessel, is self-evident: one needs to be able to send desired actuation states to the actuation subsystem of Figure 7. A human–machine interface consisting of two azimuth controllers (ACs) and a touch screen provides this service, shown by Figure 8. The latter depicts the current internal steering angles, ${\theta }_i$, and propeller speeds, n, for both actuators, along with their operator-requested values. Similar to the onboard actuation layout, see Figure 2, the bow AC is placed in front of the stern AC. Both controllers are 360-degrees-steerable and have a lever to control the propeller speed. Consequently, a one-on-one mapping describes the relation between ${\theta }_i$ and the AC-angle for both actuators. Likewise, the maximum position of the lever represents the maximum propeller speeds of the actuators.

The motion control part of the I-SCC subsystem in Figure 7 shows the full design of Figure 8. Accordingly, a shoreside PLC (number 38) orchestrates all the desired actuation system states and has a dedicated 3G mobile internet connection with its onboard counterpart at the heart of the USV. Next to this enabled motion control interaction, the onboard PLC (number 1) web interface allows the enabling or disabling of components or settings. Similarly, processes can be started or stopped on the I-PC via a secure shell connection, further augmenting the spectrum of interaction possibilities with the vessel.

4.1.3. Design Results for Requirement Three and Four (R-Iii.–R-Iv.)

The component enumeration in Table A1 exemplifies their accordance with the industrial-robustness requirement, i.e., (r-iii.). This selection procedure means that these components should also be serviceable on a real-scale vessel in its operational environment. The details in the previous two result sections, which handled (r-i.) and (r-ii.), demonstrate the applied modular system design approach. For instance, the multi-purpose utilisation of the USV autonomy sensors, and the nesting of this full USV system design inside the I-SCC concept in general, see Section 4.1.1, reveal the envisaged scalability and component-interchangeability of this modular design. In addiction, Section 4.1.2 expanded the I-SCC remote motion control on the already-existing cascaded motion control hierarchy nested inside the USV. Finally, Section 3.3 presented an additional propagation of the I-SCC, further demonstrating its modularity and adequacy as an experimental set-up.

4.2. Construction of an Inland Shore Control Centre

Figure 9 displays a picture of the first operational status of the I-SCC, with component numbers referring to Table A1. As shown, the motion control part of the I-SCC subsystem of Figure 7 was fully installed, whereas the stern-view camera (number 27) and complimentary display (number 32) were not yet installed. The other three cameras (number 27) provided a live stream of observation and security data while the USV was under remote motion control. These three IP cameras have a variable data throughput depending on their available bandwidth. They can alter this data rate by changing their configuration parameters, e.g., their quality, resolution, or frames per seconds (fps). Figure 10 illustrates the positions of the USV components, where their numbers refer to Table A1.

4.3. Technical Design Hull-To-Hull-Extended Inland Shore Control Centre

Figure 11 summarises the current H2H-extended I-SCC system design of this paper. Note that the ongoing H2H project does not have a fully finalised design yet. Consequently, the final H2H-extended system design might still have small adaptions compared to this current design.

4.4. Construction of Hull-To-Hull-Extended Inland Shore Control Centre

In order to further clarify Section 3.3 and Section 4.3, Figure 12 depicts some core parts of the current H2H-extended I-SCC experimental configuration. Figure 12a presents the combined H2H engine, from Figure 6c, which receives navigational information from the USV via MBR and has an own GNSS antenna (number 47, not shown) which is placed on a pre-calibrated position which, together with its 3D drawing, represents the quay, i.e., a static H2H object. The middle screen (number 32) on the bottom of Figure 12b visualises the output of the H2H application, which receives information from the combined H2H engine, see Figure 6c. This H2H application ran on a laptop (not listed) for our mobile testing convenience, but we plan to run it on the SCC-PC (number 37) in the future.

4.5. Design of Experiments

In order to test the designs and constructions of Section 4.1, Section 4.2, Section 4.3 and Section 4.4, the experiments of this study aimed to measure the performance of the overall USV and I-SCC system and the situation awareness of its operator. The system performance was measured by conducting successful experiments and thus implicitly stress testing its overall design. In addition, indicative communication data rates and latencies were explicitly measured to provide more insights in the system performance. The situation awareness of the operator was assessed by both performance measurements and process indices, similarly to the situation awareness assessments techniques present in aviation research [69].

4.5.1. First USV and I-SCC System Stress Tests and Experiments

An operator inside the I-SCC was instructed to remotely control the USV in two different locations, a canal and a lake, on two different days. In the first experiment, the USV was positioned in a canal near Leuven, in the vicinity of the I-SCC, although there was no line of sight for the operator. In the second experiment, the USV sailed on a lake in Rotselaar and its operator was positioned in the I-SCC in Leuven, which separates them by approximately 7 km. For both experiments, the focus was put on the activity, i.e., continuously remotely controlling the USV via the I-SCC. During the canal mission, the operator was asked to sail past two bridges, turn around, and sail back. During the lake mission, the operator received full freedom to sail around. Consequently, both experiments mainly focus on stress testing the full USV and I-SCC system design. Additionally, given their loosely defined mission goal, the situation awareness of the operator was not explicitly measured with the just-discussed assessment tools of this study. The results of both missions can be found in Section 5.1.

4.5.2. First USV and H2H-Extended I-SCC System Stress Tests and Experiments

The first experiments with the H2H-extended I-SCC system had two goals: to offer a stress test of the complete extended system, and to provide preliminary data on the impact of the H2H-navigation system on the activity of continuously remotely controlling a USV. Hence, these data can also serve to assess the situation awareness of the operator using the H2H-extended I-SCC system. More precisely, an eye-tracker served as process index and the performance measures framework for unmanned systems [70,71] was used as a guideline for performance measurements. This set of first experiments falls under a larger set of scheduled experiments which aims to test the single-handed sailing and docking hypotheses (see Section 3.3) within the H2H project. In this larger set of experiments, both skippers and students will be asked to single-handedly sail the USV via the I-SCC during both straight sailing and docking manoeuvres, once with and once without the H2H-extended interface.

Five performance metrics were selected: time, energy, accuracy, safety, and reliability, in accordance with the commonly used performance dimensions of effectiveness and efficiency, which respectively denote requirement fulfilment and resource consumption [72]. The former two metrics measure efficiency and the latter three effectiveness. Within these experiments, the time metric denotes the elapsed time of a mission, energy the power consumption of the actuation system, accuracy the deviation from the desired trajectory, safety the avoidance of collision, and reliability the overall effectiveness of the tool, i.e., was the participant able to achieve the envisaged goal [61]. Two main sources of performance measure variance [73] will be used and modelled as continuous variables, coined domain knowledge and gamer experience. The former variable could explain a performance increase, given that a person with skipper experience might better understand the propulsion system of the vessel or make better sense of the vessel movements in its environment. The latter variable might explain a performance increase due to more experience in virtually remotely controlling objects in a gaming universe [74], or a better understanding of the motion control system layout.

In this study, two students, with no former knowledge of the USV and the I-SCC, were asked to sail a straight line, back and forth, in the middle of the canal in Leuven. Each participant was given this task twice: once without the H2H-interface, meaning they had to estimate the middle of the canal based on the camera feedback, and once with the H2H-interface which showed a top view visualisation of the vessel in the canal and the target line in the middle of this canal. The order of the experiments was also changed: the first student started with the H2H-interface, whereas the second student started without this visualisation. Section 5.2 analyses their results.

5. Results

5.1. First USV and I-SCC System Stress Tests and Experiments

Figure 13 and Figure 14 respectively present data of the canal (Leuven) and lake (Rotselaar) experiments which Section 4.5.1 designed. Figure 13a and Figure 14a plot the position of the main GNSS antenna mushroom positioned at the stern of the vessel, hence they indicate the sailed trajectories. The USV course can be seen in Figure 13b and Figure 14b in conjunction with the measured steering angles of both actuators. Be aware that these steering angles should be viewed in combination with their propeller speeds, shown in Figure 13c and Figure 14c, given that these determine the thrust size. As can be seen on Figure 13b, the bow thruster was mainly used for small course corrections whereas both thrusters were used during the more complex manoeuvres. Throughout the less-spatially-restricted lake mission, Figure 14, more complex manoeuvres occurred, often involving rotations of both actuators.

In addition, Figure 15a,b offer the calculated time delays between the USV and I-SCC PLCs for both experiments. This calculation goes as follows: the shoreside PLC transmits its clock-time to the onboard PLC which relays this time to the shoreside PLC. At the reception of this relayed clock time, the shoreside PLC subtracts this time from its current clock time and assumes the half of this difference to be the latency time between both PLCs. Figure 15a,b seem to indicate a similar median latency between the onboard and onshore PLCs which use a 3G connection to communicate. In the former mission, two latency peaks larger than 1 s can be noted. A possible explanation for these peaks is that these tests were performed in the vicinity of two large bridges (one highroad and one railroad bridge), whereas the lake missions had no noticeable nearby obstacles. In addition to these motion control latency measurements, some indicative video stream delays of 300–500 ms were measured manually. Here, a timer was placed in the field of view of the camera and pictures were taken of the captured video stream together with this timer, hence providing the latency. It should also be noted that a new 5G network might be deployed in Belgium in the foreseeable future, which could enhance the streaming capacity and quality.

In order to fully crystallise the working principle of the USV and I-SCC system, Video S1 shows a sailing Cogge whilst a human operator remotely controls it via the I-SCC, with no line of sight. Note that this video was shot on another testing day than the one producing the data of Figure 13, but on the same location. Furthermore, it can be seen that inside the I-SCC the monitor for the stern camera (number 32) was installed but used to visualise the H2H application, which ran on a low frequency during its first live operation, but was not present during the experiments of Figure 13 and Figure 14. Furthermore, a combined data consumption of 6 to 9 Mbps was noted when requesting a video stream with a resolution of $1024\times 720$ at 24 fps with a normal image quality. Given the average upload speed of 11 Mbps in the Belgian mobile network [75], the selected Pepwave router (number 28) can pipe this data rate since it has two modems capable of bandwidth fusion with redundant SIM card slot features. The other observations sensors, i.e., the LIDAR and stereo cameras, were not used. Note that their data could be submitted via the Quartz router (number 4) which has a maximal bandwidth of 150 Mbps.

5.2. First USV and H2H-Extended I-SCC System Stress Tests and Experiments

Figure 16a,b respectively show the first run of participant one with the H2H-application and the second run of this participant without the H2H-application. Similarly, but reversed, Figure 16c,d plot the first run of participant two without the H2H-application and the second run of this participant with the H2H-application. These results show that, with or without the H2H-application, the activity of remotely controlling the USV was achievable. It should be noted that, due to their small quantity, these results serve as indicators of how the future larger sets of experiments (see Section 4.5.2) can be analysed. Only when more experiments will be conducted in the future, can significant conclusions be made. Nevertheless the current experiments already indicate some interesting findings. Bearing this limitation in mind, Figure 16a,d seem to indicate that the H2H-application helped both participants to stay in the middle of the canal as they show a smaller offset with regard to this middle of the canal (line between both plotted waypoints) compared to Figure 16b,c. Note that the canal is approximately 25 m wide in this test region. In addition, the recorded durations of these first trials (participant one, run one ≈8.5 min and run two ≈10 min; participant two, run one ≈7 min and run two ≈6 min) are twice shorter when using the H2H-application.

A more in depth analysis of these two performance metrics (time and accuracy), the three remaining performance metrics (energy, safety, and reliability), and the eye-tracker, falls out of the scope of the current study, but will be performed in the H2H project—although it can already be noted that all the experiments shown concluded without any collisions and the participants were able to reliably achieve their mission goals. In order to further clarify the operational working principle of the H2H-extended I-SCC, the supplementary Video S2 shows a participant single-handedly preparing a docking manoeuvre with and without the H2H-application. A snapshot of this video can be seen in Figure 17a. During these experiments, and the straight-sailing experiments, the participants each wore an eye-tracking device from which a recorded feed can be seen in Video S3. In Video S3, the participant turned the USV during a single-handed straight sailing mission, e.g., Figure 16, and the video shows that the participant made use of the H2H-application to orient the USV. A snapshot of this video can also be seen in Figure 17b.

6. Discussion and Future Work

The first loosely-defined experiments of Section 5.1 offered a thorough stress test of the total USV and I-SCC system. Although the situation awareness of the operator was not explicitly measured, it can be noted that the I-SCC helped the operator to develop a sufficient level of situation awareness and sensemaking during the requested activity of remotely controlling the USV. Hence, these experiments offered a first verification of the ACD concept of the I-SCC from Section 3.1.3. Subsequently, the first USV and H2H-extended I-SCC experiments in Section 5.2 put forward three main findings. Firstly, they stress tested the H2H-extended I-SCC and USV system, further verifying the possibility to continuously remote control a USV via the I-SCC. Secondly, their goal-oriented mission design allowed to assess the situation awareness of the operator by means of performance measurements and process indices. Thirdly, these assessment tools enable the investigation of the performance impact of the H2H-Extension. For example, both participants seemed to have benefited from this H2H-application, given that it decreased their mission time and track error, compared to the normal I-SCC set-up.

These two analysed performance metrics of Section 5.2 seem to indicate an improved level of situation awareness for both participants when using the H2H application, although one needs be careful with such preliminary conclusions: a good performance need not be indicative of a better situation awareness, and vice versa [69]. Additionally, similar caution should be taken when processing the eye-tracker data, given its potential limitation due to the “looked but failed to see” [76] phenomenon [69]. Therefore, in future work, it would be interesting to implement some of the other situation awareness assessment tools of [69], in order to complement the more basic assessment (performance measurements and Process Index) tools used throughout this study. For example, the Freeze-Probe technique, would randomly freeze some displays. Afterwards, the participant could be asked to estimate the current system states, such as position, speed, and orientation, or environmental states, such as objects or infrastructure in the neighbourhood. Evidently, the freeze-probe technique would intrude on the activity of controlling the USV. In comparison, a non-intrusive post-trial self-rating technique such as the situation awareness rating technique [77] could also be chosen.

The cascaded growth of the I-SCC, such as the addition of the H2H-application, further demonstrates that the underlying activities mainly shape the developed tools. Nevertheless, the used ACD approach does also rely heavily on the UCD approach [11]. In this study, this connection was rather straightforward given that the researchers act both as product designers and users, thereby often implicitly incorporating their user requirements into the design, which might give rise to biases. The additions to the I-SCC also emphasise the envisaged modularity of the overall system design, handled by Section 4.1.3. In this regard, the currently implemented components of Section 4.1.1 provide the integrated USV and I-SCC with a first iteration of information groups in order for its operator to develop levels of situation awareness and sensemaking. Evidently, these selected components are indicative and will most likely be modified and extended based on the feedback of future experiments, which might decrease the aforementioned potential design biases. For example, when more, or different, information group data needs to be shared, or processed onboard before sharing, installing additional computation and communication devices might offer more redundancy.

Figure 18 further exemplifies the overall modular design of the subsystems within the I-SCC. This figure depicts the motion control subsystem at the lake in Rotselaar, where it only needs a power supply. The shown laptop was connected to a mobile router enabling access to the video stream of the bow-oriented camera. Considering the operator had clear line of sight, this was not necessary but further tested the overall design flexibility. On the top of the picture, a part of the lake can be seen together with the Cogge, a support vessel, and a few researchers. This follow-up vessel was used during all the discussed experiments and had the wearable RC (number 30) onboard. This vessel can also be seen on the bow and starboard oriented video streams on Figure 9. Furthermore, this motion control subsystem could also be used to remotely control other types of USVs such as Maverick discussed in Section 2.2.2. Given that this vessel will have a similar onboard PLC as the Cogge, this should not demand many adaptations. However, it might be more intuitive to change the position of the azimuth controllers from on-top-of-each-other to next-to-each-other. Given that the propellers of the Maverick can rotate in both directions, and that their outboard engines have a limited steering range, it could also be beneficial to replace the azimuth controllers by different control interfaces which better mimic this configuration in order to avoid mode confusion [78]. Furthermore, when handling vessels with a larger mass, haptic feedback [79] might be favourable.

As described in Section 3.1.3 and exemplified by the conducted experiments of Section 5, the current I-SCC concept focuses mainly on the continuous remote motion control of a USV. Nevertheless, the Cogge can already be deployed as an unmanned vessel, capable of autonomously following waypoints [5]. Future work could thus investigate the impact of this operational mode, i.e., monitoring an autonomous inland USV in its spatially restricted waters [80]. In such a JCS [52,53] the situational awareness will become more distributed [54] and perhaps extensions such as the H2H-application might help in providing a representation of the current system state and its limitations [10,48]. Although it is not trivial to define and measure the performance of such a complex system [81,82], the usability, usefulness, and understandability of such a JCS should be measured [83]. Such future experiments, and the currently scheduled experiments within the H2H project, will allow a further bottom-up investigation of the impact on ship sense and harmony when remotely handling a USV under different operational modes. Although these first data sets will be embryonic, originating from conditioned but real outdoor environments, they hopefully provide essential information for future socio-technical CBAs [18] and STAMP [35,66] analyses, and for the inland waterway transport sector.

7. Conclusions

With the intention of providing a finer level of resolution in the feasibility studies regarding unmanned or automated inland cargo shipping, this study discussed the design and building of an I-SCC in order to remotely control or monitor a USV or ASV. Preliminary experiments verified that the operator had sufficient interaction possibilities with the USV in order to continuously remotely control it. Accordingly, the I-SCC had the potential to support the development of situation awareness and sensemaking of its operator within these first experiments. However, additional situation awareness assessment tools would be advised to further probe the achieved level of situation awareness in the future. Similarly, experiments where the USV is remotely monitored instead of controlled could further validate these findings and help to study the impact on ship sense and harmony when monitoring an autonomous USV. A stepping stone in this direction might be achieved during the discussed H2H navigation experiments which investigate the enhanced remote motion control of a USV. This enhancement could serve as a monitoring tool in the future. Evidently, the technological feasibility of this I-SCC system and its potential design iterations or extensions do not prove its socio-economic viability. For such a viability estimation, future CBAs would be better suited for which this, and the suggested future work aims to provide fruitful input.