iFO (infrared Fish Observation) – An open source low-cost infrared underwater video system

Graphical abstract


a b s t r a c t
Underwater video surveillance is an important data source in marine science, e.g. for behaviour studies. Scientists commonly use water resistant ruggedized monitoring equipment, which is cost-intensive and usually limited to visible light. This has two disadvantages: the observation is limited to space and time where visible light is available or, under artificial illumination, behaviour of marine life is potentially biased. Infrared (IR) video surveillance have been used before to overcome these. It records videos at visible light and under IR-illumination. With today's efficiency of IR-LED and video technology even low-cost systems reach visibility ranges suited for many application scenarios. We describe a low-cost open-source based hardware/software system (iFO). It consists of a single-board computer controlling the camera and lamps (with high power IR-LEDs), printed circuit boards (PCB), the underwater housings and 3D-printable models to mount PCBs in the housings and the housings to standard GoPro mounts. The Linux based software includes webserver, remote control, motion detection, scheduler, video transfer, storage at external hard disk and more. A ready-to-use SD-card image is included. We use rugged underwater housings with 100 m (optional 400 m) depth ratings. Finally, we describe a typical application observing the behaviour of cod in fish pots.

Hardware in context
We present ''infrared Fish Observation" (iFO), an open-source low-cost underwater infrared (IR) video observation system using high power LEDs and low-cost CMOS camera modules. The use of IR-video surveillance at night is very common for onshore applications and therefore hardware became very efficient at low-cost. On the other hand, in marine science, the use of visible light in dark environments is mostly inappropriate to avoid bias of fish behavior. Acoustic cameras have been used for this purpose [7], but those are complex and expensive systems with low resolution compared to optical cameras. Like human eyes, many marine species, such as fish, cannot see IR-light. One major obstacle is the relative high attenuation of IR-light under water compared to visible light [2][3][4]. That limits this method to short-range observations. Nevertheless, IRvideo surveillance have been used for underwater observations in dark environments earlier, e.g. [1,5,6,8,9]. These studies do not give detailed description of the IR systems and therefore are not reusable. Additionally, since the time of their development, technology has made great improvements. With today's effectivity of LED technology, even low-cost CMOS cameras can cover ranges of observation >2 m, suited for many application scenarios. Furthermore, single board computers like Raspberry Pi or BeagleBone are inexpensive and can be operated with freely available open source operating systems (Linux based). They are used in many different application areas and a wide range of open-source software tools are free available. This makes them ideal for quickly realizable developments of highly adaptable cost-effective systems.

Hardware description
As there were no affordable underwater IR systems available, we developed the open-source system iFO (infrared Fish Observation). iFO uses a consumer single-board computer (Raspberry Pi) and standard industry parts. The Raspberry Pi single-board computers have been applied for marine supervision and fish observation earlier, e.g. [20,21,22].
In our application one system consists of one camera and two lights, whereby components cost around 530€ including 100 m depth rated housings. With the reuse of existing open-source software and hardware, that were adapted to our scientific needs, we achieved a sustainable ocean monitoring system at low-cost. The system offers a webserver, a comfortable scheduler, a motion detection unit and can store internally more than one week continuous video data.
We present iFO in a typical application where we observe the behaviour of cod (Gadus morhua, L. 1758) at the entrance of different fish pots with the aim to improve the catch effectivity. It delivers 24/7 underwater video footage in a range up to 2 m at infrared illumination and much greater distances at daylight. Additionally, we use a LTE router (FritzBox 6890) with a 2 GB swappable hard disk to be used with up to four camera systems. This allows video data storage for several weeks and provides full access via VPN and LTE to the whole system in the field. It gives remotely live videos, access to the cameras' webserver for adjustment and setup, for instant download of data and to the cameras' operating system for maintenance.

Spectral sensitivity of the camera
The CMOS camera has an OV5647 [13] sensor chip, which has no IR filter. We use lenses that do not have IR filter either. As typical for colour camera sensors, the photoactive area is evenly distributed in red, green and blue filtered pixels. Its relative spectral quantum efficiency is shown in Fig. 1. Quantum efficiency (QE) is measured over a range of different wavelengths to characterize a device's efficiency at each photon energy level. It shows the fraction of emitted electrons e from a given number of photons illuminating the sensor. That means, e.g. a number of 100 photons of 555 nm wavelength generates 90 electrons (yellow line) on a green filtered pixel and a number of 100 photons of 850 nm wavelength generates 35 electrons (blue line) on the same pixel.
Each colour (blue, green and red) represents a third of the full active sensor area. Therefore, the total quantum efficiency of the active area for a monochromatic light at a certain wavelength can be calculated from the sum of the electrons on each coloured pixel type, e.g. for 555 nm it is the sum of 15e at blue, 90e at green and 8e at red pixel equals to 15e + 90e + 8e = 113e. It can be seen that the quantum efficiency for infrared light at 850 nm (@850 nm: 35e + 35e + 35e = 105e) is very similar to those in the visible range (@555 nm: 15e + 90e + 8e = 113e). Consequently, the camera can be used for visible light and IR-light.
The datasheet also specifies a photometric sensitivity of 680mV lxÁs and a dark current value of 16 mV/s [13] (see Table 1).

Spectrum and intensity of the IR lamps
Each infrared lamp uses six Osram SFH4715AS IR-LED's [15] with a beam angle of 90°. The beam angles can be adapted by optical lenses mounted on the PCB (Table 11: no. 15, 15a). The total electrical power is P el ¼ n Á U Á I ¼ 6 Á 3:15V Á 1A ¼ 18:9W. The centroid wavelength is 850 nm with a radiant power / e ¼ 1:530Wper LED, resulting in / e ¼9.180 W per lamp. Fig. 2 shows the relative spectral intensity.  3. Hardware components

Enclosure
The housings for the IR-server camera and the IR-lamp are taken from the Blue Robotics Inc. 3 00 -series. Technical details and 3D-drawings can be downloaded at [16]. We use the acrylic tube version that is depth rated to 100 m, an aluminium version rated to 400 m is also available. The end caps have through-holes for cable feed-through penetrators, whereas different types of penetrators are available [17]. The cable feed-through is sealed with marine epoxy to build a safe custom made cable confection. A detailed tutorial is given at [18].

GoPro mount for IR-lamp and IR-server
The GoPro mount (see Fig. 3 fits to the outer surface of the acrylic tube. It is tightened by a M5 Â 35 mm screw (hexagon socket ISO10642) with M5 hexagon nut. We have used ABS material for 3D-printing to improve stability of the GoPro-mount under water and cold conditions. The use of standard PLA was insufficient because it became brittle in saltwater after a few weeks.

Server/Camera
In the server/camera enclosure, four printed cicuit boards (PCB) are connected to each other and tied on a 3D printable mount.
1 Raspberry Pi single-board computer [12] 2 PCB with Real time clock 3 PCB with the camera module 4 PCB 'IR-server' with voltage regulator and two output drivers.

Enclosure Server/Camera
For the server/camera a 298 mm acrylic tube described in chapter 3.1 is sectioned in two halfes to get enclosures for two server/cameras. For a set of two cameras the following parts (see Table 2), are needed: Table 3 shows a distributor list for the enclosure. For sealing the cable penetrator we use potting compound (Table 8, Item15). The 100 g package can be used for about 35 cable penetrators, when all are sealed with one compound mix (within ½ an hour).

Raspberry Pi single-board computer
The Raspberry Pi single-board computer has a CSI interface for the camera board and a 40 pin-GPIO interface used here for the PCB RTC and the PCB 'IR-server' with power converter and two output drivers (see Fig. 4).
The assembly of the PCB 'IR-server' is documented in chapter 3.3.7. The Raspberry Pi with additional PCBs and the camera PCB is fixed to a 3D-printable mount that fits into the underwater housing. The mechanical details of this mount are specified in chapter 3.3.6. The bill of material is shown in Table 4.

PCB IR-camera
The camera PCB uses the 5MP OV5647 CMOS sensor IC, which has no IR-filter. It is supplied with CSI interface and cable. The M12 S-mounted lenses are without IR-filter, either, different angles of sight are available. The bill of material is shown in Table 5.

PCB 'IR-server'
The PCB 'IR-server' is mounted via the 2 by 6-pin-socket JP1 to the GPIO pin-header of the Raspberry Pi Table 7. On the topside of the PCB, there are two pin-headers JP2 and JP4. The 4-pin I/O header JP2 is connected to the 24VDC input power and the two output drivers (O13 out and O21 out) that allow a remote control of two independent groups of lamps. In our standard applications we control the lamps by light sensors. For remote controlled lamps some modifications are necessary, which are described in chapter 3.4.5. For the standard setup, the 2-pin header JP4 delivers the 5VDC@2A power for the Raspberry Pi. It is connected via a 2-wire line with pin-sockets to the power pins of the Raspberry Pi. Fig. 5 shows the schematic and the top and bottom view of the PCB 'IR server'. Table 7 the jumper connections and Table 8 the bill of material. 3.3.6. PCB mount (3D-printable) Fig. 6 shows the 3D-printable mount that holds the Raspberry Pi with its additional PCB boards in the acrylic tube and the CMOS camera at the centre in front of the acrylic window. It is attached to the flange with four screws.

Assembly
For the assembly of the server/camera, the PCB of the RTC and the 'IR-server' are mounted to the GPIO header of the Raspberry Pi. The PCB 'IR-server' is connected to the Raspberry Pi GPIO-pins #29 to #40. The PCB of the RTC to the odd pin row from #1 to #9. The power supply for the Raspberry Pi is connected by a two wire cable with pin sockets between JP4 at PCB 'IR-server' and Raspberry Pi GPIO-pins #2 (+5V) and #6 (GND). All connections are marked in red in Fig. 7.
We use Raspberry Pi 3 Model B + but we also tested older versions (Pi 3 Model B and Pi 2 Model B). The camera module is without infrared filter. Different types of optics are available e.g. 160°or 75°degrees angle of view. The operating system with all necessary modules, setups and software components is written to the SD memory card. It is recommended to use 64 GB (or greater) SDXC memory card, to have enough space to locally store videos. The Raspberry Pi with the attached PCBs and the camera module are mounted to the 3D-printed mount that fits to the flange (Fig. 8).  The camera is mounted to the front gap with four flat head tapping screws 2.2 mm Â 6 mm with the cable connector to the bottom. The Raspberry Pi is attached with four M2 Â 12 mm screws (each a 3 mm spacers between the PCB and the mount) and hexagon nuts to the four slot holes on the mount. A CSI-cable connects the camera to the Raspberry Pi computer. It is inserted with the blue mark to the front at the cameras PCB and with the blue mark to the back of the Raspberry Pi (Fig. 9).
The PCB mount is fixed with four M3 Â 8 mm screws to the M3 threads of the front flange. The external power has to be connected to the connector J2 at the PCB 'IR-server'. Finally, the electronics will be mounted inside the acrylic tube (enclosure) with the electrical connections tailored at the backside plate ( Table 9). The underwater sealing procedure of the    through-hole connectors is described at [18]. The underwater cable types and cable configuration depends on the requirements of the application. As power and ethernet cable for the server/camera, we use underwater ethernet cable like [19]. After sealing, the power and data lines are assembled according to Table 9. A simple connection of the Ethernet port of the Rasperry Pi can be made by cutting a standard patch cable, dismantle the sheath and solder the four data lines (Tx+, TX-,Rx + and Rx-) to the corresponding wires of the underwater cable. The power is tranferred according to 10/100BASE-TX with PoE pinout. The four wires are soldered to a two pin socket (2x 24VDC; 2x GND) (see Fig. 10).
On the surface side, we use 9-pin WEIPU SP21 connectors to connect the underwater ethernet cable from the server/camera to the top side power and ethernet device. For easy installation we designed a 3D-printable mounting device that fits to all GoPro compatible mounts. A more detailed description with the mechanical drawings is given in chapter 3.2, Fig. 11 shows an assembled server/camera.
In summary the major assembly steps are:

.1. Enclosure lamp
The housing for the IR lamp is the same as for the server/camera except, that the original tube is sectioned in four quarters of 74 mm length each. For a set of four lamps you need parts according to Table 10.
A distributor list can be found in Table 3.
For sealing the cable penetrator we use potting compound (Table 8, Item15). The 100 g package can be used for appr. 35 cable penetrators, when all are sealed with one compound mix (within ½ an hour).

PCB IR lamp
Two PCBs are required for each IR lamp: 'IR-lampV31 0 with six LEDs and 'IR-lampV32 0 for the LED driver. The schematics and PCBs are shown in Fig. 12 and Fig. 13.
The bill of material is given in Table 11 for four IR lamps. The six LEDs at each PCB 'IR-lampV31 0 are reflow soldered, while all other parts (including the SMD parts) can be also soldered manually. Both PCBs have four through holes to mount the PCBs to the M3-threads in the aluminium cooling mount. The PCB 'IR-lampV32 0 has 4 additional holes to mount it to the M3 threads in the flange of the underwater enclosure. When sufficient ambient light is available, e.g. during the day in shallow water, the IR illumination is not required. Therefore, there are two control options for the lamp: 1. Control by a light sensor 2. Control by the IR server/camera control signal In our standard applications, we control the lamps by ambient light sensors (phototransistor VTT9812FH, Table 11, item 14). For a control by a light sensor, the phototransistors pins are soldered to a 3-pin socket with the anode at pin #1 and the cathode at pin #3, pin #2 is not connected. This 3-pin socket is connected to J2 at the PCB 'IR lampV32 0 (Fig. 13, bottom left). The setup for the control by the camera/server is described in chapter 3.4.5. The bill of material for the IR-lamps is given in Table 11.

Aluminium cooling mount
The two PCBs 'IR-lampV31 0 and 'IR-lampV32 0 are finally mounted to the aluminium cylinder that drains the heat from the LEDs to the aluminium flange and further to the surrounding water. The aluminium mount should have a diameter of 63.5 mm to fit best into the flange. We turned a 65 mm aluminium rod down to 63.5 mm and sawed 16 mm thick pieces from it, alternatively aluminium rods with diameters down to 60 mm can be used. The drilling were done by handcraft. There are four through holes with M3 threads to mount the PCB boards on the two sides of the cooling mount (Fig. 14). Fig. 15 shows the assembly of the PCB boards with the cooling aluminium mount from the top left to the bottom right. First, the PCB 'IR-lampV32 0 is mounted to the aluminium mount with four M3 Â 8 mm screws and 3 mm spacers. Then the long 2-pole-pin socket of the PCB 'IR-lampV31 0 connected to the standard 2-pole pin header at the mounted PCB 'IR-lampV32 0 which is inside the 8 mm hole at the center of the aluminium mount. It is important to observe the correct polarity: LED + at 'IR-lampV32 0 has to be connected to LED + at 'IR-lampV31 0 . The PCB 'IR-lampV31 0 is fixed directly to the aluminium mount using four M3 Â 8 mm screws. To improve heat flow, thermal conductance paste between the bottom side of the PCB and the aluminium mount can be used. Finally, the block with the PCB boards and the aluminium mount is inserted into the flange and fixed with four M3 Â 8 mm screws. Thermal conductance paste between the lateral area of the aluminium and the flange would increase heat flow even further.

Assembly
The backside plate of the enclosure with its the electrical connections can be tailored to specific needs. The underwater sealing procedure of the through-hole connectors is described at [18]. Underwater cable types and cable configuration depend on the requirements of the application. In our application, we use 3-wire water resistant DIN VDE 0276 NYY-J 3x1,5mm 2 cable and 3-pin WEIPU SP21 connectors to connect the lamps to the top side surface unit. After sealing, the power Table 10 Bill of material for the underwater housing (4 pieces).

Item
No. and data lines are assembled according to Table 12. The two power wires are soldered to a 2-pin socket, which is connected to J1 of the PCB 'IR-lampV32 0 with respect to the correct polarity. There are two options to control the lamps: by an ambient light sensor (standard) and remotely from the server/camera (optional) (see Fig. 16). For easy installation we designed a 3D-printable mount for the IR lamp that fits to GoPro compatible clamps. A more detailed description with the mechanical drawings is given in chapter 3.2. Fig. 17 shows an assembled IR-lamp mounted to a frame.
3 Assembling the PCB 'IR-lampV32 0 to the aluminium mount with spacers.   4 Assembling the PCB 'IR-lampV31 0 to the aluminium mount with respect to the polarity of LED+ and LED-connectors, heat conducting paste can be used. 5 Assembling the mount with its devices to the front flange of the enclosure. 6 Tailor the the electrical connection at the backside plate to your needs and connect the power to J1 at the PCBs 'IR-lampV32'.

PCB IR lamp under server/camera control
In our standard applications we control the lamps by ambient light sensors. For a remote control by server/cameras the cable setup is different and another cable type is needed. It either has two more wires for the lamp control, e.g. a underwater data cable with shielded 5 Â twisted pairs (Table 13, Item 1) or one twisted pair with a greater cross-sectional area for power   (Table 13, Item 2). It this case, the power supply is connected via the thicker twisted pair and the remaining twisted pair can be used for the two control lines. For the remote control setup, the four twisted pairs for Ethernet are connected as described in Table 9. The two wires of the remaining twisted cable pair are connected to the lamp control outputs at J2 on PCB 'IRserverV11 0 . Each of these wires can control a group up to six lamps. Therefore, the power-in-wires and the lamp control output wires are soldered to a 4-pin socket ordered as shown for JP2 in Table 7. This 4 pin-socket is connected to J2 on PCB 'IRServerV11 0 in the correct orientation. And, as in the standard setup, the 2-pin header JP4 is connected via a 2-wire line with pin-sockets to the power pins of the Raspberry Pi (GPIO jumper pins #2 (+5V) and #6 (GND)). Fig. 18 shows a wiring diagram.
In the surface unit, the control wires from the IR server/camera cable are connected to the control wire of cable to the lamps they should control. Two groups of lamps ('Lamp control 1 0 or 'Lamp control 2) can be controlled by one server/camera. They are controlled by the GPIO Pins GPIO13 and GPIO21. It can be done by simple bash scripts, which can be accessed in the webservers scheduler or other LinuxOS scheduler (e.g. crontab).
For the remote control setup also the IR lamp wiring is modified. At first, the ambient light sensor is removed. The control signal wire (Table 12) from the underwater cable is soldered to a 3-pin socket at pin #2, while pin #1 and pin #3 are not connected. This 3-pin socket is connected to J2 at the PCB 'IR-lampV32 0 (Fig. 16, right). As in standard configuration, the two power wires are soldered to a 2-pin socket, which is connected to J1 (see Figs. 17 and 18).

Software
The server/camera uses a Linux Debian OS without graphical XServer. The open source software RPi-Cam-Control [10,11], provide a webserver which gives access to settings for camera and motion detection, as well as to basic system settings, like restart, power down or user defined functions (Fig. 19). Additionally, a live video stream and a comfortable scheduler are available, that can also start user defined scripts. A full documentation can be found at [12]. All software is packed as .iso image.

Setup the SD card image and access the server
The software for the server/camera is pre installed on the image file, that can be written to the SD-card using command ''dd" under a LinuxOS or using the software Rufus (https://rufus.ie) under WinOS.
This SD-card has to be inserted into the Raspberry Pi before powering the server/camera. Two network connections are predefined, which can be modified after the first connection: The predefined connections were configured with the network command line interface ''nmcli". Both make it easy to install the cameras immediately on any FritzBox either directly wired or via Wifi, when the Wifi connection of the router is setup as predefined in the server/camera above. For a login the following credentials are valid: Hostname: IRServer3 User: pi Password: irserver Static IP: 192.168.178. 6 The server/camera is remote accessible via three protocols at its IP address: 1. via HTTP through the webserver interface, 2. via SSH connection to the Linux OS or 3. via SCP for file or video transfer from the camera to the remote system.  5. Use case with remote access via surface unit In our specific use case, we wanted to improve fish pots for cod (Gadus morhua, L. 1758), especially the design of the pot entrance. Therefore, we needed a video surveillance system to observe cod behaviour in relation to different entrance designs -during day and night for several months.
The requirements were: observation at day and night minimum observation range: 2 m video data storage for several months fast swappable data disk motion detection remotely accessible webserver with live stream In this use case the fish traps are located in shallow water (below 10 m depth) but we use the system also for applications in deeper waters (see specifications). A combination of two iFO systems is installed in the field (each with one camera and two IR-lamps). Both connected to an LTE router (FritzBox6890) with a swappable network attached storage (NAS) hard disk (Fig. 20).
This allows continuous video data storage for several weeks and delivers full remote access via VPN and LTE to the underwater iFO systems. They are remotely accessible for live video streams, setup and administration, for video file downloads and maintenance (see Figs. 21 and 22).

Summary and outlook
In this document, we present iFO, an open source low-budget underwater infrared video observation system with full documentation and sources to easily reproduce this. We briefly outlined one typical field of application, but we also use this system for other observation tasks, e.g. in fish tanks and off-shore. Of course, the system can be used in other environments and purposes as well: e.g. in harsh environments with the current underwater enclosures or in other environments with adapted ones. As a next step, we will derive a practical method from this work to estimate the underwater range of vision for infrared camera and light systems in general.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.