Particle and salinity sensing for the marine environment via deep learning using a Raspberry Pi

The identification of mixtures of particles in a solution via analysis of scattered light can be a complex task, due to the multiple scattering effects between different sizes and types of particles. Deep learning offers the capability for solving complex problems without the need for a physical understanding of the underlying system, and hence offers an elegant solution. Here, we demonstrate the application of convolutional neural networks for the identification of the concentration of microparticles (silicon dioxide and melamine resin) and the solution salinity, directly from the scattered light. The measurements were carried out in real-time using a Raspberry Pi, light source, camera, and neural network computation, hence demonstrating a portable and low-cost environmental marine sensor.


Introduction
It is estimated that around 35,000 tons of microplastics are present in the world's oceans [1], with trillions of microbeads entering the marine environment daily in aquatic habitats of the United States alone [2]. These microplastics can arise from products such as cosmetics, toothpaste and face scrubs [3,4], which contain microbeads, but can also be formed from the breakdown of larger plastics already discarded into the sea [5,6]. Microplastics can have a direct negative impact on marine organisms [7,8], and impact other marine life through transfer via the food chain [9,10]. They have been found in fish and bivalve organisms, and have been shown to have negative effects on zooplankton and oysters [11][12][13][14][15][16]. Table salt from around the world has been found to contain microplastics [17,18], and the sea salt contamination, by plastic itself, has been shown to be an indicator of plastic pollution. Drinking water can also contain microplastics [19], and the potential impact on humans has been discussed [20,21], with specific negative impacts on epithelial inflammation [22] and muscle cell behaviour [23].
Monitoring such particles is necessary to help determine origin and distribution, whilst also providing data for mitigating the effects of plastic pollution [24]. Since microplastics can reach the marine floor, having been found in sediments at depths down to 5000 m [25], and in deposit feeders that ingest sediment [26], it is therefore important to monitor plastics before they reach the marine floor. Such a sensing device must have the capability to identify the different types of microplastics and natural occurring particles, such as sand, in order for accurate monitoring. However, a significant obstacle in monitoring such a global problem is the lack of a reliable portable and low-cost method for characterisation of the pollution particles. Manta nets [27] can be used to collect plastic particulate matter from the marine environment of sizes down to ∼333 μm, with additional laboratory sieving used to separate out smaller micro particles [28,29], and material characterization carried out subsequently using a range of spectroscopic and imaging techniques [30][31][32][33][34]. Such collection and characterization methods are extremely time-consuming and expensive, and hence alternative methods are required.
A holographic technique that involves analysing the scattered light from particles, has shown the potential for the characterisation of particle contaminants in water [35]. Scattered light from particles is dependent on the Original content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence.
Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. illumination wavelength and on the particle parameters, such as shape, size, refractive index and composition [36,37], therefore such parameters can be inferred from analysis of the scattered light in certain cases, via comparison to theoretical calculations. In the case of light scattering from a single homogenous sphere, a full electric field calculation, for example via Mie theory, can be used to produce a simulated scattering pattern. However, the calculations become rapidly more complex as the number of particles is increased, particularly if the particles are not identical. Crucially, however, the inverse calculation is what is needed, as the particle parameters should be determined directly from the scattering pattern. Although particle monitoring from observation of the scattered light has been demonstrated in [35,38], such methods require simulations and lack the flexibility for identification of non-spherical particles. Ideally, the technique should be able to determine the parameters for many particles simultaneously, whilst also being robust enough to deal with real-world effects such as non-spherical particles, variability in light sources and optics, and variability in the aqueous environment.
Deep learning, which is an approach based on the application of neural networks (NNs) [39][40][41], has already enabled advances in imaging [42,43] and enabled automated classification of objects in images [44,45], such as label-free cell classification [46], as well as object classification through scattering media [47][48][49] and through scattering pattern imaging [50,51]. Using NNs to determine particle size and refractive index from their scattering pattern was proposed by [52] and has been subsequently demonstrated experimentally on colloidal spherical particles [53][54][55][56], showing that NNs can bypass the need to develop complex modelling [57]. Moreover, the ability to update a NN [58], for example to monitor additional particles without the need to physically change a sensor, makes such an approach particularly desirable, especially when implemented on a micro-computer, such as a Raspberry Pi [59,60].
In our previous work, we were able to use a NN to identify single particles of polystyrene and silicon dioxide on a glass substrate, and we demonstrated the ability to use a NN on a desktop computer for the real-time identification of a range of real-world airborne pollution particles (diesel soot, wood ash and pollen) on a glass substrate, directly from their scattering patterns, with each identification taking less than 50 milliseconds [61]. Others have demonstrated the combination of holography and deep learning, for the retrospective (i.e. not realtime) classification of particles in water, using a Raspberry Pi for data collection and with the neural network instead run on a desktop computer [62]. In this work, we demonstrate a Raspberry Pi-based sensor, which runs a neural network, loaded onto it via Wi-Fi, to run in real-time and classify the concentrations in water of 5 μm silicon dioxide (a type of sand) and 8 μm melamine resin (a type of plastic used for tableware [63], which has also been found in fish [64]) microparticles.
Because water salinity can impact on the health of marine life, and is a commonly used ocean parameter to study the effects of climate change [65], the ability to monitor the salinity without the need for additional electrical conductivity devices would be an extra benefit of using the sensing technique documented in this work. We thus also show that it is possible to determine the salt concentration of the water in which 8 μm melamine resin microparticles are present. In addition, we demonstrate the robustness of the NN, by performing a second set of measurements 20 days after the NN was trained, and after deconstructing and rebuilding the experimental setup, hence proving the potential for portable and low-cost sensing.

Sample preparation
Silicon dioxide microspheres of size 5 μm±100 nm (Sigma Aldrich, Product number 44054) and melamine resin microspheres of size 8 μm±200 nm (Sigma Aldrich, Product number 95523), with dimensions measured using a Coulter Multisizer II, were deposited via pipette into deionized water-filled glass cuvettes, each of external size 12.5 mm×12.5 mm×45 mm, with an optical interaction length of 10 mm. To mimic seawater, saline samples that were used for NN training were prepared by adding salt (sodium chloride) to deionised water-filled cuvettes in steps of 10 ppt (parts per thousand by mass, where 1 ppt is approximately 1 psu (practical salinity unit) [66]), from 0 ppt up to 100 ppt, and 0.1 ppt of 8 μm-sized melamine resin microparticles was added to all samples. Additional cuvettes of deionized water were filled with different concentrations of mixtures of 5 μm silicon dioxide and 8 μm melamine resin microparticles, to give concentrations in the sample solution (referred to here as actual solids concentration) in the range of 0 to 0.1 ppt, in steps of 0.0125 ppt.

Experimental setup
The schematic of the experimental setup in figure 1 shows light from a ∼1 mW laser diode operating at 650 nm that has been focused into the cuvette using a 2.5 cm focal length lens. This produced a spot size of approximately 20 μm by 10 μm inside the cuvette. The scattered light from the microparticles was projected onto a white polyester screen 1.5 cm from the cuvette, and subsequently imaged by a CMOS camera (Raspberry Pi Camera Module, CSI-2, 3280×2464 pixels), placed 5 cm away. The total volume of the focal region was therefore considerably larger than the volume of a single sphere, enabling the potential for measuring larger particles from the same sensor design. The camera was connected to a Raspberry Pi 3 Model B+ computer to allow real-time capturing of scattering patterns, every 50 milliseconds, with an exposure time of 20 milliseconds. The images were then transferred to a desktop computer that had an NVIDIA Titan Xp GPU. One hundred scattering patterns were recorded for each of the samples stated in section 2.1 by interchanging the cuvettes in the setup. Each cuvette was shaken by hand prior to placing in the imaging setup. Once the NN was trained on the GPU, the NN was transferred via Wi-Fi to the Raspberry Pi, where it was subsequently used to conduct real-time measurements on scattering images. Here, the NN outputs corresponded to the microparticle mixture concentration, and in the second part of the work, the salinity of the water. The experiments were carried out at room temperature (22°C).

Neural network
A convolutional neural network was used, which is a type of NN designed mainly for image processing [57,67], with a regression output. The regression output enabled the capability for the NN to produce a continuous output within a certain range [68]. The NN framework was Tensorflow [69] and was trained on a desktop computer with an NVIDIA Titan Xp graphics processing unit (GPU). Figure 2 shows a schematic of how the NN was used in this work. Owing to the restriction in random-access memory (RAM) that was available for the NN to compute on the Raspberry Pi (1 GB), the input images were cropped such that only one quadrant of the scattering pattern image was selected, in order to still retain high-frequency scattering information in the image data. After cropping, the camera images were resized to 100×100 pixels and converted to grayscale. Before the images were sent to the NN, each individual image was normalized to have a mean of 0 and a standard deviation of 1. The NN was formed of two convolutional layers, followed by a max pooling, dropout, and fully connected layer, with a regression output. In the case of the mixture determination, the NN had two outputs, corresponding to the concentration of 5 μm diameter silicon dioxide microparticles and 8 μm diameter melamine resin microparticles. For the salinity measurements, the NN had a single output, corresponding to the ppt of salinity.
The input layer (greyscale cropped scattering pattern image of 100×100 pixels) was followed by two stages of convolutional then max pooling layers, whereby the convolutional layers had 64 filters with a kernel of 3×3 and stride of 1, and the max pooling layers had a kernel of size 2×2 and stride of 3 [70]. A dropout rate of 50% was used [71], leading to a fully connected layer of 512 neurons and weight decay of 0.0005 [70]. The learning rate of the NNs was 0.0001, while an adaptive moment estimation optimiser [72] was used to minimise mean square error cost function for regression. Through trial and error, the entire architecture was optimised so that the memory requirement of the NN was appropriate to be executed on a Raspberry Pi.

Identification of microparticle mixture concentration
The NN was designed to produce two numerical outputs from the scattering pattern input, corresponding to the concentration percentage for 5 μm silicon dioxide and 8 μm melamine resin microparticles. The two outputs were independent of each other, and hence could be used to provide absolute concentration values. Figure 3 shows the results for a series of measurements of different mixtures, showing the output for a) 5 μm silicon dioxide and b) 8 μm melamine resin microparticles, where each data point displays the mean and standard deviation for the NN predicted values of the microparticle concentration (predicted solids concentration) from 10 scattering patterns. The NN operated in real-time and entirely on the Raspberry Pi for computation.
One set of test measurements was made on day 1 after the NN was trained, and a second set of test measurements was made 20 days later, following the deconstruction and reconstruction of the experimental setup that occurred after the test measurements on day 1. Whilst the experimental setup was therefore conceptually the same, the exact positioning of the light source, optics, sample and camera was slightly different for the second set of measurements on day 20 (although the cropping window was realigned with the focus of the beam), and hence the fact that the predictive capability of the NN did not decrease illustrates the robustness of this NN approach. The accuracy of the NN could be further improved by increasing the amount of training data,  8 μm melamine resin microparticles, from a mixture of these particles in a solution. Each data point on the figure, which shows the mean and standard deviation, corresponds to the prediction from 10 scattering patterns. The R-squared value for the silicon dioxide data is 0.9902 and the R-squared value for the melamine resin data is 0.9978. which could be achieved by physically taking more measurements, or artificially, via augmentation [73]. Whilst the particle concentrations in this proof-of-principle demonstration are higher than might be expected in the marine environment [74] owing to the need for sufficient signal collection, our previous work [61] has shown the capability for the identification of single particles using an alternative experimental setup.

Salinity identification
As the concentration of salt in a solution increases from 0 ppt to 100 ppt, a change in the refractive index of the water occurs in which the microparticles are present. For example, at a wavelength of 589.3 nm, the refractive index ranges from 1.334 (0 ppt) to 1.343 (50 ppt) and 1.352 (100 ppt) [75]. Such a change in the refractive index therefore causes a change in the scattering pattern produced by the microparticles. Here, a NN was trained on salinity values of 0 ppt, in steps of 10 ppt, up to 100 ppt, with 10 scattering patterns recorded for each salinity. The NN was then trained for 100 epochs. Subsequently, the NN was trialled on a range of other salinities that corresponded to known values of water bodies, such as the Baltic Sea (8 ppt) and average sea water (35 ppt), which were predicted with mean values and standard deviation of 8.06±1.67 ppt and 34.24±4.03 ppt, respectively (see figure 4). Each data point on the figure corresponds to 10 recorded scattering patterns. The accuracy of the salinity measurements clearly shows the capability of the NN approach for detecting very subtle changes in the scattering patterns.

Conclusions
In conclusion, we have shown the simultaneous identification of absolute concentration percentage of 5 μm silicon dioxide and 8 μm melamine resin microparticles, when the particles were present in water. Additionally, we have demonstrated the identification via a NN of salt concentration of water containing 8 μm melamine resin microparticles, for salinities including agriculture irrigation and average sea levels. By running the NN on a Raspberry Pi, we have shown the potential for a portable and low-cost marine environmental sensor. Since the scattering pattern from particles varies depending on the size and material, this proof-of-principle technique, which involves using an NN to classify concentrations of two materials of different sizes and refractive index, could be extended to particles of other sizes and materials, such as polystyrene and polyethylene.