Paper The following article is Open access

Touchless underwater wall-distance sensing via active proprioception of a robotic flapper

, , , , and

Published 2 February 2024 © 2024 The Author(s). Published by IOP Publishing Ltd
, , Citation Kundan Panta et al 2024 Bioinspir. Biomim. 19 026009 DOI 10.1088/1748-3190/ad2114

1748-3190/19/2/026009

Abstract

In this work, we explored a bioinspired method for underwater object sensing based on active proprioception. We investigated whether the fluid flows generated by a robotic flapper, while interacting with an underwater wall, can encode the distance information between the wall and the flapper, and how to decode this information using the proprioception within the flapper. Such touchless wall-distance sensing is enabled by the active motion of a flapping plate, which injects self-generated flow to the fluid environment, thus representing a form of active sensing. Specifically, we trained a long short-term memory (LSTM) neural network to predict the wall distance based on the force and torque measured at the base of the flapping plate. In addition, we varied the Rossby number (Ro, or the aspect ratio of the plate) and the dimensionless flapping amplitude ($A^*$) to investigate how the rotational effects and unsteadiness of self-generated flow respectively affect the accuracy of the wall-distance prediction. Our results show that the median prediction error is within 5% of the plate length for all the wall-distances investigated (up to 40 cm or approximately 2–3 plate lengths depending on the Ro); therefore, confirming that the self-generated flow can enable underwater perception. In addition, we show that stronger rotational effects at lower Ro lead to higher prediction accuracy, while flow unsteadiness ($A^*$) only has moderate effects. Lastly, analysis based on SHapley Additive exPlanations (SHAP) indicate that temporal features that are most prominent at stroke reversals likely promotes the wall-distance prediction.

Export citation and abstract BibTeX RIS

Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

1. Introduction

Underwater vehicles employ a range of sensing systems to perceive their environment and conduct operations, such as using sonar (acoustic), camera (optical) and pressure sensors. However, for small-size Autonomous Underwater Vehicles (AUVs) or bio-inspired underwater robots operating in cluttered shallow water conditions (e.g. in turbulent waters over intertidal zones and rocky river beds and shorelines), conventional acoustic or optical sensing methods [1, 2] suffer a number of disadvantages, such as the inability of working in clouded, murky water environment with low visibility [3, 4], bulkiness in design [5], high energy consumption [6], and adverse effects on ecosystems [7]. Therefore, it is desirable to develop novel sensing methods to complement or expand the current sensing capabilities of AUVs.

Without reliable optical or acoustic sensing, it remains possible to extract rich information about the underwater environment from the fluids per se based on pressure and strain information. The ability to 'feel' the water, including its movement (i.e. the flow) and the tiny, low-frequency vibrations propagating through it (i.e. the pressure waves), is crucial for gaining intelligent behaviors in biological swimmers and their survivability in complex, uncertain hydrodynamic environments [8]. Many animals are highly evolved with such abilities. For example, swimming invertebrates, fishes, and flying insects have specialized sensory units distributed on the surfaces of their fins, wings or bodies to sense the pressure on and strain of these hydrodynamic or aerodynamic surfaces. Examples include lateral lines and neuromasts of fishes [914], sensilla of leeches [15, 16], and campaniform sensilla of hawkmoth [17] and flies [18, 19].

Hypothetically, functions of the organs like the lateral line can be emulated in underwater vehicles or robotic swimmers using a distributed array of pressure sensors with proper bandwidth and sensitivity. However, despite recent progresses, artificial sensors of similar quality and compactness as those found in fishes are not readily available [20]. In addition, although flow quantities that can enable perception in lateral lines, such as pressure and velocity fields, have been investigated [14, 21], the exact mechanisms for how fishes utilize their lateral lines for perception in the presence of self-generated and external hydrodynamic stimuli are still unclear [22]. To date, efforts on underwater pressure sensing remain limited to estimating flow states in well-controlled laboratory settings with proof-of-concept testing (e.g. using MEMS-enabled artificial lateral line systems [2325] or commercially available pressure sensors [21, 2628]).

In this work, we explored a bioinspired alternative to existing underwater perception methods. Our approach was based on indirect sensing of actively-generated fluid flows using proprioception. Specifically, we focused on predicting the distance towards an underwater wall based on the internal forces and torques measured at the base of a flapping plate (figure 1). A flapping plate can be used as an abstraction of the propulsion mechanism found in different flying and swimming animals and the robots they inspire. The underwater wall interacts with the fluid flow generated by the flapping plate, and affects both the hydrodynamic forces acting on the plate and the internal forces and torques at its base, the measurement of which is a form of proprioception. In this setup, the flapping plate is the only source of fluid flow, and thus its proprioception is only excited by the self motion or self-generated fluid flow, making the sensing active.

Figure 1.

Figure 1. Overview of the wall-distance sensing method based on active proprioception. The primary hypothesis we investigated was whether the hydrodynamic pressure and the resulting proprioceptive signals encoded information on wall-distance.

Standard image High-resolution image

Proprioception is a measure of kinematic and kinetic states that are internal to the body [29], unlike the lateral line which senses the external flow. The sense of position and velocity of fins or wings as well as the forces acting on them can be considered as proprioception [30]. While proprioception is internal, it responds to external stimuli and may contain meaningful information about the fluid flow that is external to an underwater robot, and therefore can be used indirectly to perceive the external environment. It thus can be hypothesized that proprioception (for example, the internal forces acting at the base of the wing or fin) can act as an alternative to the lateral line by measuring the resultant effects of the surrounding flow on the body [31].

The effectiveness of biological and artificial sensors in perceiving their surroundings can be enhanced by injecting self-generated energy to the environment, which is a form of 'active sensing' [32]. Active sensing is effective when there is not enough excitation in the system for sensors to obtain useful measurements. While sonar and lidar are well-known active sensors, they suffer from the problems of bulkiness and high energy consumption as mentioned earlier. In fish or fish-inspired underwater robots, the movements of their hydrodynamic surfaces, while primarily used for locomotion, can also be potentially used for exciting the fluid for active sensing [13, 14, 21, 28]. Fluid quantities such as pressure can be measured in this self-generated flow to gain information about the surroundings [33]. Notably, in underwater environment with free-stream velocity, the ambient flow interacts with the active or static objects in environment (e.g. rocks and fishes) and carries relevant information (e.g. pressure gradient) downstream. Active sensing based on self-generated flow can potentially enable perception without the free stream in quiescent water.

In this work, we investigated whether the fluid flows generated by the flapping plates can encode distance information relative to an underwater wall, and explored the methods to decode this information from the proprioception present in a flapping plate. We trained an long short-term memory (LSTM)-based neural network to predict the distance of the flapping plate to a nearby wall based on the force and torque measured at the plate base. Next, we used SHapley Additive exPlanations (SHAP) [34] to identify the phase of a flapping cycle when the sensory inputs become most critical for accurate predictions of wall-distance. Finally, we evaluated how the prediction accuracy was affected by the unsteadiness of the self-generated flow (determined by the dimensionless amplitude of flapping motion) and the three-dimensional rotational effects (determined by Rossby number or the aspect ratio of the plate).

2. Materials and methods

2.1. Experimental setup

The flapping plate is actuated by a 3DoF robotic mechanism (figures 2(a) and (b)) consisted of three servos with encoder resolutions of 360/4096° (XM450-W350-R, Robotis). Only 1 DoF from the topmost servo was used in this study for generating flapping motion. The plate was thus assumed to move only within the horizontal plane. The other two servos remained fixed in the positions shown in figure 2(b), keeping the plate's span horizontal and with an angle of attack of 90°. The servos had integrated PID controllers to reach the desired angles. The PID gains were manually tuned to be 200, 10, 10, respectively, to avoid oscillations around the set point and to reduce jerkiness. The same robotic mechanism was also used in our previous work [3537].

Figure 2.

Figure 2. Experimental setup and procedure. (a) Photo of the 3 DoF flapping-plate robot in the mineral oil tank, pointing towards the wall from which distance is to be predicted. (b) Illustration of the robot's position inside the tank. The plate tip-to-wall distances ($d_{\mathrm{tip}}$) and the alternative plate root-to-wall distances ($d_{\mathrm{root}}$) were used to denote wall-distances. (c) CAD render of robot showing the flapping direction and orientation of the force and torque sensor. Only the stroke axis (shown) is actuated. The plate stays at a constant 90 °C angle of attack while flapping, as shown. (d) Plate shapes corresponding to the three Ro. All lengths are in cm.

Standard image High-resolution image

A six-component force–torque load cell (Nano17-IP65, ATI) was mounted on the output shaft of the bottom servo and attached to the plate, thereby measuring the internal forces and moments between the plate and the robotic mechanism. The sensor had resolutions of 1/320 N for forces and 1/64 N mm for moments. The robot and the load cell were immersed in a mineral oil tank (White Mineral Oil, Tulco, density $\rho = {826}$ kg m−3, viscosity $\nu = {6e-6}$ m s−2 at 22 °C) of size ${81}~\textrm{cm}\times{81}~\textrm{cm}\times{81}~\textrm{cm}$. T-slotted beams installed over the center of the tank allowed the plate to be positioned at different wall-distances as shown in figure 2(c).

The servo control and force–torque data collection were performed using a Speedgoat real-time computer (Performance Real-Time Target Machine, Speedgoat GmbH). Code for the target machine was generated using a Simulink® model. The force–torque readings were collected at 200 Hz, while the reading of the encoder angle and the sending of the angle commands were alternated over serial (RS-480) at 100 Hz each.

2.2. Design of plate shape and motion for varying Rossby number and dimensionless amplitude

The rectangular plates were made of 3 mm thick acrylic and was attached at the mid-chord to the bottom servo's shaft. Three aspect ratios of the plates were used for varying the Rossby number (Ro) (figure 2(d)). Ro measures the degree of three-dimensional effects in the fluid flow, and determines the stability of leading-edge vortices (LEV) observed in flapping wings of birds and insects [38]. For rectangular plates, Ro is equivalent to the aspect ratio, and is defined as [39]:

Equation (1)

where $R = s+o$ is the plate tip radius, s is the span (or plate) length, $o = {5.1}~\textrm{cm}$ is the offset between the plate root and the rotation axis, and c is the chord length. Equation (1) was used to create three sets of s and c for the rectangular plates (figure 2(d), table 1) while keeping the surface area ($S = s\cdot c$) constant at 108 cm2. The designed values of Ro (2, 3.5, 5) fell in the range observed in biological creatures [39].

Table 1. Plate shape and motion parameters for each Ro and $A^*$. s is the span length, c is the chord length, φ0 is the flapping amplitude, and f is the flapping frequency.

Ro $A^*$ s (cm) c (cm) φ0 (°) f (Hz)
2212.48.757.30.116
 3  84.90.077
 4  114.60.058
3.5217.16.332.70.229
 3  49.10.153
 4  65.50.114
5220.85.222.90.348
 3  34.40.232
 4  45.80.174

Dimensionless amplitude of the flapping motion ($A^*$) measures the unsteadiness of the fluid flow generated by the flapping plate, as a lower amplitude at the same average flapping speed (determined by the Reynolds number) leads to more frequent back-and-forth. $A^*$ affects the integrity of the LEV [39]. Although the lower stroke amplitude of mosquito wings has been speculated to lead to better signals in a fluid-based active sensing scenario as compared to bird wings [33], the effects of dimensionless amplitude on perception are yet to be extensively studied. $A^*$ was defined as:

Equation (2)

where φ0 is the flapping amplitude. In addition, the Reynolds number at the radius of gyration ($Re_{\mathrm{g}}$) was defined as [35, 36, 39, 40]:

Equation (3)

where $U_{\mathrm{g}}$ is the average velocity at the radius of gyration $R_{\mathrm{g}} = \sqrt{I/S}$ (with $I = \frac{s^3c}{12}+S (o+\frac{s}{2})^2$ being the second moment of plate area about the rotation axis) and f is the flapping frequency. We designed three values of $A^*$ (2, 3, 4), of order 1 like insect flyers [39], at a fixed $Re_{\mathrm{g}}$ of 800, which were used to determine φ0 and f for the each Ro (table 1) at which the plate flapped sinusoidally.

2.3. Experimental procedure and data collection

The plates were placed at 14 distances from a wall of the tank, along a straight line in the middle of the tank (figure 2(c)). The distance was measured from the plate tip to the wall ($d_{\mathrm{tip}}$), which went from 1 to 40 cm in increments of 3 cm. This range corresponded to 1.9 plate lengths for Ro = 5 and 3.1 plate lengths for Ro = 2. While the distance from the force–torque sensor to the wall in the above distance measure was dependent on the plate length, an alternative wall-distance measure from plate root (or the force–torque sensor) to the wall ($d_{\mathrm{root}}$) was also calculated, which was independent of the plate length. The depth of the center of the plate was set to be 40 cm, which was more than 4 times the chord length, thereby leading to minimal effects of surface waves on the force–torques.

For each of the 3 Ro and at each of the 14 $d_{\mathrm{tip}}$, time series data including the force–torque and the flapping angle were collected in 5 separate experimental trials. In each trial, 14 flapping cycles worth of data was collected at all 3 $A^*$ in the order of $A^*$ = 2, 3, 4. To remove the transient force–torque behavior when switching between $A^*$, the first 3–4 cycles of data after setting the flapping motion for each $A^*$ were discarded, after which the measurements did not appear to vary from cycle to cycle upon visual inspection. Thus, data for a total of 70 flapping cycles was collected at each distance for each combination of Ro and $A^*$.

2.4. Supervised learning for wall-distance prediction using LSTM

We used supervised learning to train a LSTM architecture for wall-distance prediction (figure 3). The inputs to the LSTM included the measured six-component force and torque and the flapping angle within one flapping cycle, i.e. a 7-dimensional time-series data, which were mapped by the LSTM to its output as the predicted wall-distance—a single number. LSTM was chosen because of its ability to process time-series data with long-term dependencies [41]. Multiple LSTM units were stacked in multiple layers (table 2). The output of the last LSTM layer at only the last time-step was propagated into the next-stage hidden and output dense layers. Dropout was also used between all layers for regularization.

Figure 3.

Figure 3. Overview of the machine learning process. One flapping cycle is taken as one input-output sample. The LSTM layers have a tanh activation, the first dense layer has an ELU activation, and the output dense layer has no activation function. The dropout layers that exist between all consecutive layers have the same dropout rate D. L, C, D are specified for each Ro and $A^*$ combination in table 3.

Standard image High-resolution image

Table 2. Hyperparameter options used to train the LSTM models.

HyperparameterOptions
Downsampling rate5, 10, 15
LSTM units per layer64, 128, 192
LSTM layers2, 3
Dropout0.2, 0.5

The force–torque, flapping angle, and wall-distance data were pre-processed (figure 3), to improve the learning performance. These include the following:

  • (i)  
    Dividing into cycles: The neural network was trained to output the wall-distance given the time-series data from one flapping cycle (starting at 0 phase).
  • (ii)  
    Dividing into training/validation/testing sets: To avoid bias towards data from any experimental trial or wall-distance in training or in testing, the 14 input-output samples in each trial at each wall-distance were randomly distributed into the three sets in a 64.3%/14.3%/21.4% split, leading to 630/140/210 samples in the respective sets. Additionally, the training process was repeated 5 times to mitigate biases introduced by the stochasticity in the splitting and in the gradient descent. Thus, our results are based on $5\times210 = 1050$ testing samples.
  • (iii)  
    Downsampling: In a preliminary test and in [42], it was found that the learning performance greatly increased when the temporal resolution of the input force–torque and angle data were reduced by downsampling. Here, downsampling was done by averaging a number of consecutive time-steps into a single time-step.
  • (iv)  
    Normalization: For each Ro and $A^*$, the input time-series and the output wall-distances were normalized by having their means subtracted and then divided by their standard deviations.

The neural network was trained with gradient descent with Keras in TensorFlow. The loss function was the mean absolute error (MAE). Adam was used as the optimizer to update the neural network weights because of its adaptive learning rate [43], which was initially set at 0.0005. Exponential learning rate decay (with a rate of 0.9 and a step of 5000) further reduced the learning rate with the number of training epochs to help the weights converge. The weights were updated with every mini-batch of 32 samples from the training set, and this process was repeated for 5000 epochs. Finally, early stopping was used to combat overfitting—the weights from the epoch that had the lowest validation set loss were restored at the end of training.

The LSTM design parameters and pre-processing parameters were designed separately for each Ro and $A^*$. Based our preliminary tests and also inspired by the approach in [42], parameters that had large impacts on the LSTM model's performance and their possible options were identified (table 2). The downsampling rate affected the smoothing of the time-series and reduced the input's dimensionality, the number of LSTM units and layers affected the model's complexity, and the dropout rate affected the strength of the regularization. All combinations of these parameters were used to train the LSTM model using the process described above, and the combination yielding the lowest validation set loss was chosen (table 3). Regarding the large model sizes in proportion to the number of training samples: the larger models still yielded better validation set losses than the smaller models, and were thus used in this work. For example, only models with 128 and 192 LSTM units per layer were selected for all Ro and $A^*$ cases as they had lower validation losses than the 64-unit models, even though the latter would have resulted in the smallest model size. Nonetheless, the smaller models we used in preliminary work (e.g. 8 LSTM units with only 1 layer) still achieved notable prediction performance.

Table 3. Parameters and hyperparameters found best for each Ro and $A^*$ among the choices in table 2 for the neural network in figure 3. The time steps per cycle is a result of sampling rate, flapping frequency and downsampling rate at each Ro and $A^*$.

Ro $A^*$ LSTM units per layer (C)LSTM layers (L)Dropout (D)Downsampling rateTime steps per cycle
2219220.515114
 312820.215172
 419220.215229
3.5219230.51558
 319230.51587
 419230.515116
5219230.55115
 319230.55172
 419220.21576

2.5. Statistics for effects of Ro and $A^*$

We also evaluated the effects of Ro and $A^*$ on predicting the wall-distances using the above LSTM framework. We compared the 9 combinations of Ro and $A^*$ based on the error statistics of the test set predictions. The medians of the prediction errors for all samples in the test set were used as a measure of the expected prediction error, and the lower (Q1) and upper (Q3) quartiles were used as measures of variability in the prediction quality. To give a more comprehensive picture of the variability in the prediction errors, the expected lower ($W_{\mathrm{L}} = Q_1 - 1.5 IQR$) and upper ($W_{\mathrm{U}} = Q_3 + 1.5 IQR$) extremes of the errors were also obtained, with $IQR = Q_3-Q_1$ being the inter-quartile range. In the sections that follow, the lower and upper whiskers in the box plots represent $W_{\mathrm{L}}$ and $W_{\mathrm{U}}$, the lower and upper limits of the box represent Q1 and Q3, and the line crossing the box represents the median, respectively [44].

2.6. Model explanation using SHAP

SHAP [34] values signify the contribution of each input feature, quantifying how much each feature increases or decreases the model's output relative to a baseline output. Because of its additive properties, SHAP values for individual input features can be calculated separately and then be summed up to explain the deviation of a particular prediction from the baseline. Additionally, SHAP values for individual predictions can be averaged over multiple predictions to explain, in general, what input features are important to the model [45]. Here, SHAP was used to identify the temporal phases of flapping during which the input had the most pronounced influence on the predictions. Since the inputs here are multi-dimensional time-series, a SHAP value was first assigned to every time step of every force–torque component and flapping angle. Then, the SHAP values were summed over the 7 input dimensions, resulting in a one-dimensional time-series containing the contributions of each time step to the LSTM model's output. The absolute value was taken to get only the magnitude of the time step's importance. Finally, the SHAP time-series for all samples at the same wall-distance in the testing set were averaged, yielding a single SHAP time-series for each combination of Ro and $A^*$ at a wall-distance.

3. Results

3.1. Instantaneous proprioceptive force–torque and wall distance predictions

The cycle-averaged proprioceptive forces and torques at all $d_{\mathrm{tip}}$ are plotted in figure 4 for one case (Ro = 5, $A^* = 2$). Overall, the presence of the wall at different distances had subtle, but identifiable effects on force and torque components. The most noticeable changes occurred close to when the flapping plate reversed its direction (i.e. stroke reversals). The changes in the forces and torques were relatively small compared to their respective absolute values, except for components with smaller magnitudes, especially the spanwise torque.

Figure 4.

Figure 4. The force–torque data showing the effect of wall-distance for one case (Ro = 5, $A^* = 2$), with the data at $d_{\mathrm{tip}} = {40}~\textrm{cm}$ subtracted as the baseline. The normal, spanwise, and chordwise directions are shown in figure 2(b). Dotted vertical lines mark the stroke reversals. The time-series shown were averaged at each phase across all flapping cycles at different distances and downsampled 5× (section 2.4, table 3). In the raw data without the baseline subtracted (not shown), the greatest spanwise and chordwise forces were 8% and 6% of the normal force, and the greatest normal and spanwise torques were 6% and 2% of the chordwise torques, respectively. Additionally, the largest standard deviations at any phase across all distances as a percentage of the range of the respective forces and torques were 13% for normal force, 20% for spanwise force, 27% for chordwise force, 27% for normal torque, 24% for spanwise torque, and 13% for chordwise torque.

Standard image High-resolution image

Nevertheless, these relatively small changes in the forces and torques were sufficient for the trained LSTM to predict the wall distance with high accuracy. This can be shown by comparing the predictions and ground truth for the testing cases (figure 5), which showed excellent match for all combinations of Ro and $A^*$. In figure 6(a), even $W_{\mathrm{U}}$—an indicator of the worst expected prediction error—is less than half the plate lengths of each Ro, further confirming the LSTM's ability to successfully decode the wall-distance from the proprioceptive force–torque signals.

Figure 5.

Figure 5. Mean ± 1 standard deviation of the predictions at increasing plate tip-to-wall distances ($d_{\mathrm{tip}}$) at different Ro and $A^*$. The diagonal black line signifies perfect predictions.

Standard image High-resolution image
Figure 6.

Figure 6.  Ro vs. prediction errors at different $A^*$ using (a) plate tip-to-wall distance ($d_{\mathrm{tip}}$) and (b) plate root-to-wall distance ($d_{\mathrm{root}}$). The horizontal white lines, edges of the boxes, and whiskers denote the median, first and third quartiles, and expected extremes of the errors, respectively, as described in section 2.5.

Standard image High-resolution image

3.2. Effects of Ro and $A^*$ on distance predictions

The prediction errors for each Ro and $A^*$ combination, including the predictions at all wall-distances, are summarized in figure 6(a), which shows that the predictions were more accurate with lower Ro or plate aspect ratio. Going from Ro of 2 to 3.5, the median of the errors increased slightly by less than 0.25 cm. However, Q3 and $W_{\mathrm{U}}$ increased more substantially, from 1 to 2 cm and from 3 to 7 cm, respectively. Thus, this increase in Ro increased the possibility of having large prediction errors that were greater than the median errors, even though the medians themselves were comparable between the two Ro at 0.25 cm. Going from Ro of 3.5 to 5, the median errors increased more significantly from 0.25 to 1 cm, while the Q3 and $W_{\mathrm{U}}$ also increased. Despite the increase, the median prediction error in all cases were below 5% of the plate length.

Using $d_{\mathrm{tip}}$ as the distance measure, the lower Ro or aspect ratio plates had their force–torque sensor placed closer to the wall due to the shorter plate length. Alternatively, we also used $d_{\mathrm{root}}$ as the distance measure, which made the force–torque sensor in all Ro cases placed at an almost identical distance from the wall. A separate set of LSTM models were thus trained with only the force–torque data having comparable $d_{\mathrm{root}}$ between the different Ro. The prediction errors using this approach is shown in figure 6(b). Compared to figure 6(a), Q3 and $W_{\mathrm{U}}$ of Ro = 2, $A^*$ = 2 are significantly higher, making its performance almost identical to Ro = 3.5, $A^*$ = 2. This may be expected as the sensor on the Ro = 2 plate starts 12 cm—almost an entire plate length—further from the wall than when using $d_{\mathrm{tip}}$. Except this case, the lower Ro had lower prediction error in all other cases, agreeing with the earlier conclusions based on $d_{\mathrm{tip}}$ as the distance measure.

The prediction accuracy also increased moderately with the flapping motions of higher $A^*$ (figure 6(a)). At Ro = 2, Q3 and $W_{\mathrm{U}}$ are clearly higher for $A^*$ = 2. At Ro = 3.5, Q3 and $W_{\mathrm{U}}$ at $A^*$ = 2 were higher than at $A^*$ = 4. However, $A^*$ = 3 at Ro = 3.5 has only slightly higher errors than $A^*$ = 2, but significantly higher errors than $A^*$ = 4, breaking the trend of better predictions being at higher $A^*$. A similar trend is observed at Ro = 5, where the distribution of prediction errors between the three $A^*$ are almost the same. Thus, while the increase in the prediction quality with $A^*$ was more pronounced at Ro = 2, the impact of $A^*$ at higher Ro was less substantial compared to the effect of Ro.

3.3. Change in prediction performance with wall distance

The prediction accuracy also depended heavily on the distance to the wall (figure 7). The errors were lowest closest to the wall, then increased with distance, and either plateaued or lowered again at the furthest distances. The predictions being best nearest to the wall was expected since the wall would be able to influence the flow generated by the plate more significantly. The lower prediction errors at the furthest distances from the wall may also be explained similarly, but due to the influence of the opposite wall of the oil tank. Because the length of the tank is only between 4 to 7 plate lengths, being further from one wall of the tank meant getting closer to the opposite wall. The decrease in prediction errors at the furthest wall-distances was most pronounced for the Ro = 5 plate, which—having the longest plate length—reached closest to the opposite wall, and possibly had its flow altered more than the shorter plates. Thus, larger alterations in the self-generated flow by both the front and back walls could have improved the wall-distance predictions.

Figure 7.

Figure 7. Prediction errors at different wall-distances ($d_{\mathrm{tip}}$) along the tank at different Ro and $A^*$. The horizontal white lines, edges of the boxes, and whiskers denote the median, first and third quartiles, and expected extremes of the errors, respectively, as described in section 2.5.

Standard image High-resolution image

3.4. Temporal contributions to the wall-distance prediction using SHAP

Figure 8 shows the magnitude of the SHAP at each phase of a flapping cycle for all combinations of Ro and $A^*$ and at all wall-distances. The largest peaks of SHAP values occurred during or after stroke reversals, while the SHAP values during the middle flapping portion were relatively low (except Ro = 2 and $A^*$ = 2). Therefore, these results suggest that the proprioceptive forces and torques near the stroke reversals were the most important for wall-distance predictions, and the unsteadiness in the flow may facilitate the underwater wall perception.

Figure 8.

Figure 8. SHAP (normalized from 0 to 1) at different phases of a flapping cycle at all Ro, $A^*$, and distances. Dotted vertical lines mark the stroke reversals.

Standard image High-resolution image

4. Discussion

4.1. Self-generated flows can enable underwater perception

In this work, we showed that the fluid flows generated by a flapping plate did encode distance information relative to an underwater wall, and it could be successfully decoded by an LSTM receiving the proprioceptive forces and torques measured at the base of the plate. Notably, the perception of the wall was made possible by the plates' active flapping movements, thus making the sensing 'active' [33]. Without the active movements, there would be no flow in the tank and no hydrodynamic force or torque for the sensor to measure. Existing works using potential flow theory have shown that the presence of a wall can alter the pressure gradient near a moving airfoil [46], which could in turn result in changes in the hydrodynamic forces and torques and be reflected in the proprioception. Unlike most methods in the literature, an external flow, such as an incoming freestream [47], was not needed to carry the information about the surroundings to a stationary pressure sensor array. Thus, this study substantiates the potential roles of active sensing using self-generated flows in facilitating underwater perception.

While some of the features extracted from the forces and torques may be specific to our experimental setup due to the minimalistic flapping kinematics and the unavoidable imperfections and uncertainties inherent in it, the overall approach is expected to be applicable to other conditions. The specific decoding methodologies and the design of the neural network will likely need to be tailored to the application at hand, for which the kinematics and morphology may be designed to optimize the thrust, lift, or other metrics [48]. However, as long as the repeatability and resolution of the sensing apparatus is adequate, it should be possible to distinguish the effects of walls or other underwater objects on the proprioception, like it was here as evidenced by the visible changes in the forces and torques with wall-distance (figure 4). Moreover, detection of a wall parallel to or in front of a swimming robot may be more useful practically. Our results suggest that the flapper can also potentially detect the opposite wall that is behind the flapper (or in front of a swimming robot). As we moved the flapper further from the wall that its tip is facing, and towards the opposite wall (figure 7 by the secondary x-axes), the prediction error in most cases decreased. Thus, we expect the prediction of wall-distance in other robotic applications to also be feasible.

It is also worth mentioning that the airflows generated by rotating blades in rotary-wing fliers can also enable detection of walls or ground. For example, the presence of surfaces (of various orientations) within the airflow field generated by quadcopters can alter its pressure distribution, which can be measured directly via an array of pressure sensors, and used for surface detection [33]. The presence of a wall can also indirectly alter the body states of a quadcopter via aerodynamic interaction, which can be measured using an Inertial Measurement Unit (IMU), and decoded for wall detection [49]. Our work differs from the above methods in the sense that the perception is enabled by measuring the internal forces and torques between the plate and the robotic mechanism, i.e. proprioception instead of exteroception.

4.2. Stroke reversals and temporal features promote wall-distance prediction

The most critical temporal phase for wall-distance prediction coincided with the stroke reversal, where the plate experiences largest acceleration or deceleration, likely corresponding with highest unsteadiness in the self-generated flow, including rapid vortex formation and shedding. Thus, this suggests that the unsteady excitation in the fluid flow can facilitate its interaction with the wall and generate rich information regarding the wall distance. The role of active flow generated by reciprocal motion in prompting the underwater perception was also reported in the literature. For example, in the work of Meyer et al [50] where a vibrating sphere was used to stimulate a living fish's lateral line, the response of the fish's neurons during the phase at which the sphere reverses its direction could be used to discern the 2D orientation of its vibration direction. However, active sensing with self-generated flow is possible even without reciprocal motion. For instance, the increase in pressure measurement (generated by the movement of the sensor array on underwater vehicle towards the wall) was used to detect the wall [26], and the stagnation pressure at the nose of blind Mexican cave fish has been observed to increase when moving towards a wall [14]. Interestingly, the rate of collision of these fish with the walls increased when they were beating their tails compared to when they were only gliding [13], suggesting that some self-generated flows may be detrimental to perception. Therefore, further investigation of the unsteady flow features that enable perception is needed, which were promoted by reciprocal motion in our work and by body translation in some other works in literature.

In previous works, it has been shown that spatial patterns in the aerodynamic pressure measured by insects' antenna or pressure sensors installed on robotic fliers can be decoded for touchless wall detection or aerodynamic imaging [33]; in addition, the spatial patterns in the lateral line sensory measurement also play a critical role in the underwater perception [14, 26, 51]. In this work, we showed that temporal patterns in the sensory inputs may also play a role in underwater perception, such as wall-distance estimation. While the forces and torques at the plate root do measure the net effect of the pressures acting on the plate, they are not able to measure the pressure gradient along the plate surface like a fish's lateral line or artificial sensor arrays [13, 21, 47, 51]. Thus, the success in wall-distance prediction using only measurements at one location, without knowing the spatial pressure distribution, demonstrates that wall-distance information is also encoded temporally, which also suggests that high unsteadiness in the flow can prompt underwater perception. Moreover, wall-distance could be predicted using only a single component of force or torque. For example, by utilizing changes with wall-distance at stroke reversal in only the normal force, as can be observed in figure 4, perception could be enabled solely by temporal patterns.

4.3. Stronger rotational effects at lower Ro lead to higher prediction accuracy

The Rossby number represents the ratio of inertial force and Coriolis force in the fluid flow, and therefore indicates the relative strength of rotational effects [38, 39]. A rotating plate has higher rotational effects (with a smaller local Ro) near its center of rotation, compared with those at its distal parts. Our results based on both $d_{\mathrm{tip}}$ and $d_{\mathrm{root}}$ show that lower Ro leads to higher prediction accuracy (figure 6), suggesting that the rotational effects may promote underwater perception. Note that, at the same $d_{\mathrm{root}}$, the tips of smaller Ro (or aspect ratio) plates were further away from the wall compared with those of higher Ro. As the prediction accuracy decreases with the distance within the majority of the distance range tested (figure 7), smaller Ro led to higher prediction accuracy regardless (figure 6(b)). It can be speculated that the stronger rotational effect in the flow may promote its interactions with the wall and generate richer temporal features (e.g. pressure gradient, vortex shedding, etc) that enhance the distance prediction. Additionally, at higher Ro, the diminishing prediction performance may be attributed to the fluid's added mass being positioned further from the transducer, thereby magnifying the inertial effects of the fluid during stroke reversals and causing the subtle variations in force–torque profiles from wall interactions to appear relatively smaller. However, detailed studies on the fluid field are needed to fully understanding the underlying physical processes.

In addition, $A^*$ provides a measure of the overall unsteadiness in the flow $A^*$ [39], and shows some moderate effect on the prediction accuracy. Our results show that higher $A^*$ (lower unsteadiness) led to either no significant change or slightly better prediction (figure 6(a)). This result does not align with our SHAP analysis which suggests that the instantaneous unsteadiness during the stroke reversal can enhance the prediction. As a speculation, the lower $A^*$ with smaller flapping amplitude may have generated more noises rather than meaningful temporal features, thereby, reducing the signal-to-noise ratio for the prediction. This result also seems to contradict the speculation made in [33] that lower stroke amplitudes in mosquitoes ($A^*\approx1.61$ [52]) could lead to better signals as the jets released from the wing could become more concentrated. Unlike in our study, these jets were directed towards the surface to be detected, while our flapping plate kinematics were unlikely to produce a strong concentrated jet towards the wall. While the wall-distance could still be predicted in our work, flow visualizations from Nakata's work [33] suggest possibility for improving prediction accuracy by optimizing the motion profile of the plates and, consequently, the direction of the self-generated flow, even when operating at the same $A^*$. However, as the effects of $A^*$ were not as strong as that of Ro (section 3.2), the effects of $A^*$ in perception could not be conclusively established in this study.

Finally, although we stayed at a low $Re_{\mathrm{g}}$ at which the flow is expected to be laminar, we suspect that turbulence may offer a tradeoff for perception. On one hand, turbulence has the potential of degrading the signal-to-noise ratio for wall-distance perception. In their behavioral and computational fluid dynamics studies, Windsor et al showed that the distance to a wall that can be detected by cave fish stays roughly the same and may even decrease slightly with increasing (body) Reynolds number (ranging from 1000 to 6000) [13, 14]. On the other hand, the dominance of inertial fluid effects over viscosity at higher $Re_{\mathrm{g}}$ may help in transmitting the encoded wall-distance signals to the proprioceptive sensors, rather than letting viscosity dissipate them. Additionally, the stronger flow excitation at high $Re_{\mathrm{g}}$ may possibly increase the signal-to-ratio of wall-distance in the presence of environmental noise, as hypothesized by Windsor et al [14]. The wall-distance prediction performance of this sensing mode in turbulent conditions compared to others, like vision, merits further study.

5. Conclusions and future work

In this work, we successfully achieved a touchless method for sensing underwater wall distance in the absence of vision. The accuracy of the wall-distance predictions varied with wall-distance, Rossby number, and dimensionless stroke amplitude. The predictions were more accurate for plates with the lower Rossby number or aspect ratio, possibly because of stronger rotational accelerations in the fluids leading to richer interactions with the underwater wall, and consequently stronger temporal features encoding the wall-distance. High errors were observed at the higher Rossby numbers, but the median errors were still below 5% of the plate length at all Rossby numbers. Increasing the dimensionless stroke amplitude in the tested ranges (2–4) had a relatively minor impact in increasing prediction accuracy when compared to the effects of the Rossby number (2–5). Additionally, proximity of the flapper to the walls, whether in front or behind, led to increased prediction accuracy, which is expected due to the likely stronger interactions of the self-generated flow with the wall. These results may be useful to inform or inspire the design of robots employing fins and wings for perception and control and also to understand the possibilities and limitations of this sensing method.

This work also demonstrated that information about the surroundings can be encoded in temporal patterns in the senses by proprioceptive information at a single spatial location at the plate root. Moreover, using Shapley Additive Explanations, the neural network was found to use the most information from the temporal patterns at the stroke reversals.

In future work, information contained in spatial arrangements of multiple sensors, such as a lateral-line for hydrodynamic imaging [14], can be utilized in addition to the temporal patterns utilized in this work. Additionally, efforts may be directed towards generalizing the perception performance to a larger space of plate morphology and kinematics that are relevant for thrust or lift production, or even for perception performance itself [48], for specific vehicles in different flow conditions. Further, to apply active proprioceptive sensing to real-world underwater robots, various important practical issues need to be solved along the way; space, compute, algorithmic efficiency, sensitivity and water-tightness being notable constraints for the mechatronics and software. Advancing in these areas could enhance this bioinspired sensing approach, potentially enabling robotic capabilities akin to fish schooling and formation flight, where active movements serve the dual role of locomotion and perception [31].

Acknowledgments

This work was funded by the Army Research Office [ARO Grant No. W911NF-20-1-0226] and the National Science Foundation [NSF Grant No. CBET-1903312]. We extend our gratitude to Yagiz E Bayiz for their assistance in setting up the experiment.

Data availability statement

All data that support the findings of this study are included within the article (and any supplementary files).

Author contributions

K P, B C and H D conceptualized this work; K P, B C and H D designed the experiments; K P performed the experiments; K P and Z Z analyzed data; B C, D H and A P provided guidance and feedback on the analysis; K P and B C drafted the paper; All authors contributed to the writing.

Conflicts of interest

The authors have no conflicts to disclose.

Please wait… references are loading.
10.1088/1748-3190/ad2114