The scale of the whale: using video-tag data to evaluate sea-surface ice concentration from the perspective of individual Antarctic minke whales

Abstract

Background

Advances in biologging technology allow researchers access to previously unobservable behavioral states and movement patterns of marine animals. To relate behaviors with environmental variables, features must be evaluated at scales relevant to the animal or behavior. Remotely sensed environmental data, collected via satellites, often suffers from the effects of cloud cover and lacks the spatial or temporal resolution to adequately link with individual animal behaviors or behavioral bouts. This study establishes a new method for remotely and continuously quantifying surface ice concentration (SIC) at a scale relevant to individual whales using on-animal tag video data.

Results

Motion-sensing and video-recording suction cup tags were deployed on 7 Antarctic minke whales (Balaenoptera bonaerensis) around the Antarctic Peninsula in February and March of 2018. To compare the scale of camera-tag observations with satellite imagery, the area of view was simulated using camera-tag parameters. For expected conditions, we found the visible area maximum to be ~ 100m² which indicates that observations occur at an equivalent or finer scale than a single pixel of high-resolution visible spectrum satellite imagery. SIC was classified into one of six bins (0%, 1–20%, 21–40%, 41–60%, 61–80%, 81–100%) by two independent observers for the initial and final surfacing between dives. In the event of a disagreement, a third independent observer was introduced, and the median of the three observer’s values was used. Initial results (n = 6) show that Antarctic minke whales in the coastal bays of the Antarctic Peninsula spend 52% of their time in open water, and only 15% of their time in water with SIC greater than 20%. Over time, we find significant variation in observed SIC, indicating that Antarctic minke occupy an extremely dynamic environment. Sentinel-2 satellite-based approaches of sea ice assessment were not possible because of persistent cloud cover during the study period.

Conclusion

Tag-video offers a means to evaluate ice concentration at spatial and temporal scales relevant to the individual. Combined with information on underwater behavior, our ability to quantify SIC continuously at the scale of the animal will improve upon current remote sensing methods to understand the link between animal behavior and these dynamic environmental variables.

Background

Advances in animal-borne tag technology enable researchers to record and analyze previously inaccessible in situ behavior and kinematics of animals [1, 2]. Beyond elucidating animal behavior, animal-borne tags have been built to collect oceanographic data during deployments on deep-diving marine mammals [3], and to monitor the spatial overlap between seabirds and fishing vessels at sea [4]. Thus, a precedent has been set for using biologging technology to record information that can be used to evaluate the relationship between an animal, its behavior, and the surrounding environment (both physical and biological). To accurately understand these relationships, data must be collected concurrently in space and time, eliminating offset between behavioral and environmental observations.

Remote sensing can be used in conjunction with biologging data to make inferences about how animal distribution and behavior relate to their environment across a range of spatial and temporal scales. In the Polar Regions, ice is a critical feature of the environment and is commonly observed via satellite. Sea ice concentration can be computed from satellite passive microwave radiometry, producing a single estimate of the concentration over a relatively broad area of ~ 10km² to ~ 2500km²—depending on data and methods used [5, 6]. The resolution of these data provides an excellent platform to measure monthly and annual trends in ice concentration at large spatial scales; however, higher resolution data is required to relate ice concentration to animal behavior at the submesoscale (< 1 km). Visible spectrum satellite imagery offers a more pragmatic resolution for this purpose, with a pixel resolution of up to 10 m (area = 100 m²) [7]. In the visible frequency range, however, clouds often hamper any surface observations in the WAP region, and additionally, any such observations require daylight to resolve features. Clouds and daylight are not the only problems with existing visible spectrum satellite data, but also the regularity with which a high-resolution image is taken at any given location. While coarse-scale passive microwave observations are available at least twice daily, high-resolution optical data at any given location are collected at a much lower cadence, (e.g., Landsat scenes are collected every 16 days, and combining observations from Landsat 8 and Sentinel 2 sensors would achieve at best 4.5 days revisit rate [8]). Synthetic Aperture Radar (SAR) can provide similar fine resolution estimates of ice presence and thickness independent of weather conditions [9]. While the spatial resolution may be appropriate to link ice to animal behavior at submesoscales, the likelihood of timely satellite images coinciding with data collection are low, and in many cases tasking SAR instruments to collect synoptic imagery is expensive (e.g., RADARSAT) and may still not capture appropriate scenes if field operations are out of phase due to local conditions. These challenges necessitate new tools to more accurately determine associations between animals and their surrounding ice environment in polar regions.

In both the Arctic and Antarctic, many species have evolved life histories that are dependent on sea ice. From zooplankton like Euphausiid krill that feed on under-ice algal communities, to penguins and seals that haul out on ice floes to rest, to polar bears that traverse and hunt seals on winter pack ice, sea ice is critical. Antarctic minke whales (AMW) are the largest ice-affiliated krill predator and most numerous baleen whale in the Southern Ocean. Antarctic minke are intimately tied to ice, yet very little is known about their behavior, habitat use, foraging ecology, and movement patterns with respect to their environment [10, 11]. This information is critical not only to define how these whales interact with their environment, but to better understand and forecast the impacts of climate change. The Antarctic Peninsula is warming faster than nearly any other region on the planet, manifesting in decreases in the amount, duration, and extent of winter sea ice [12]. Thus, quantifying the ice-covered habitats that AMW utilize at scales that are relevant to the individual animal will allow for a greater understanding of the ecological relationships between these krill predators, their environment, and the impacts that climate change will have on the amount of available habitat.

Given the gaps in our current ability to measure the relationships between minke whale behavior and ice coverage in their environment, the goal of this paper is to develop a robust method for evaluating the amount of surface ice that AMWs encounter using animal-borne motion-sensing and video-recording tags. Using new biologging technology, this method can be used to accurately and continuously assess the ice concentration in the environment that AMWs occupy. This method, used to quantify the ecological relationships between AMWs and their environment at scales that traditional remote sensing techniques cannot resolve, will fill a critical gap in our understanding of the complex niche of these pagophilic predators in a rapidly changing environment.

Materials and methods

Data in this study were collected using Customized Animal Tracking Solutions (CATS) multisensory suction cup attached archival tags [13, 14] between 1/25/2018–3/6/2018. Animal-born video was recorded during 6 tag deployments with 1280 × 720 pixel resolution cameras, and one deployment (see Table 1: deployment #1) at 1920 × 1080 resolution. All deployments occurred in coastal waters on the Western side of the Antarctic Peninsula, with one deployment occurring in the Penola Strait, two in Andvord Bay, and four in Paradise Bay (Fig. 1).

Table 1 Deployment details

Fig. 1

Areas of deployment. Red dots indicate the location of CATS video-recording and motion-sensing tag deployments on Antarctic minke whales. From southwest to northeast, deployments occurred in the Penola Strait, Paradise Bay, and Andvord Bay on the western side of the Antarctic Peninsula

Tags were deployed via a hand-held 6-m carbon fiber pole from a ridged-hulled inflatable boat (RHIB) or Zodiac inflatable boat. All tags contained accelerometers that sampled at 400 Hz, magnetometers and gyroscopes at 50 Hz, and pressure, light, temperature and GPS at 10 Hz. All data were decimated to 10 Hz, tag orientation on the animal was corrected for, and animal orientation was calculated using custom-written scripts in Matlab 2014 [13, 15]. Animal speed was determined using the amplitude of tag vibrations [16]. Positions of the whales were calculated at 10 Hz over the duration of the deployment from georeferenced pseudo-tracks (i.e., dead reckoned reconstructed tracks [17]), constructed from animal depth, speed, pitch, and heading, and corrected via known positions. Fast acquiring GPS sensors were enabled on all tags to provide accurate georeferencing throughout the tag deployments when animals surfaced. For additional georeferenced positions of the tag we used the tag-on position and tag-off position collected via hand-held or tag GPS, as well as opportunistically collected positions during focal animal follows from a smavideoll inflatable boat using GPS, estimated range, and bearing. The resulting tracks still had multi-hour gaps without additional location verification, so errors accumulated, particularly when the whale traveled below the threshold of detectable speed (~ 1 m/s)[15]. Since the local habitat was an enclosed environment with complex coastlines, this provided an additional opportunity to anchor positions when a whale's track matched the contour of the coastline but was not proximate to the coastline. By anchoring tracks such that the whale's track assumed to be following the coastline was in the correct location, all tracks could be corrected so that they remained in suitable areas (i.e., not on land). This process was used to generate 6 additional anchor points for deployment bb180227-45, and 3 additional anchor points for deployment bb180304-40. Because these tracks are based on estimated anchors from coastline data, they cannot be assumed to be precise and we estimate that they are accurate within ± 1 km of the calculated location.

Satellite imagery

SENTINEL-2 (L2A, visible and infrared spectrum) satellite data was sourced from the Sentinel-Hub EO Browser [7] between January 24th 2018 and March 7th 2018 (24 h before the beginning of the first deployment and 24 h from the end of the final deployment). Images were collected for the full spatial range of all deployments, rendering a search range of 10,530 km². Date, time, % cloud cover, and the specific region of each image was recorded for comparison to tag data. The spatial resolution was dependent on the wavelength of light received with a maximum possible resolution of 10 m (100m²) and a minimum of 60m (3600m²).

Estimated surface area viewed from camera tag data

Area of view simulations were used to compare the scale of on-animal video observations to the pixel size of satellite imagery. Visible surface area (m²) was calculated as a function of typical depth, pitch, visibility and tag angle of view for each tag deployment using a custom MATLAB script. Camera parameters included the video resolution (# of pixels height and width) and in-water camera calibration tests provided the parameters of vertical and horizontal angles of view. The distance from the top of the image to the center of visible surface in the image (Ds) and from center to the edge of visible surface in the image (Dl) were calculated based on initial values including depth (P) the angle from the surface to center (90°—pitch) ($\upbeta$), and the angle of view based on camera calibration tests ($\mathrm{\alpha }$) (Fig. 2). The angle between the surface and the depth line (θ), was assumed to be 90°. Angles γ, δ and ε can be solved for by subtracting θ and the associated bottom angle from 180. Ds and Dl may then be determined as follows:

$$\mathrm{Do}=\mathrm{sin}\left(\beta -\frac{1}{2}\mathrm{\alpha }\right)\times (\frac{P}{\mathrm{sin\gamma }}),$$

(1)

$$Ds+Do=sin\upbeta \times \left(\frac{P}{\mathrm{sin\delta }}\right)$$

(2)

$$\mathrm{Do}+\mathrm{Ds}+\mathrm{Dl}=\mathrm{sin}\left(\upbeta +\frac{1}{2}\mathrm{\alpha }\right)\times \left(\frac{P}{\mathrm{sin\epsilon }}\right),$$

(3)

$$\mathrm{Ds}=\mathrm{results of equation }2-\mathrm{results of eq}. 1$$

$$\mathrm{Dl}=\mathrm{results of equation }3-\mathrm{results of eq}. 2$$

Fig. 2

Measurements for area of view estimates. (above) Lateral view of area estimate measurements where vertical angle of view for the camera is represented by angle a, and angle between the depth line and center of view (as calculated from pitch) is represented as angle b. (right) Overhead view of area estimate measurements where Wl represents the width of the camera field of view at the furthest point of visible surface from the tag, Wc represents the width at the center of the visible surface and Ws represents the width of the closest point of visible surface to the tag (top of screen)

In instances of high pitch where Do < 0, Do is assumed to be 0 as this indicates there is no gap between the nearest point of surface and the area within the FOV. The width of the image at the first row, center, and at the edge of visible surface in the image, was calculated using a series of right triangles with the side furthest from the tag representing ½ of the width at each of the three points of measurement. X,Y and Z represent the hypotenuse of a right triangle between the tag position below the surface, the top, center and edge of the image (respectively) and the edge of the horizontal angle at each point (Fig. 2 overhead view). The inner sides of these triangles are FOV^short, center of view and FOV^long which are not visible in the overhead view (see Fig. 2 vertical view). The lengths of the inner sides were determined using:

$$\mathrm{inner length}=\mathrm{sin\theta }\times \left(\frac{P}{\mathrm{sin}x}\right),$$

where x is the angle (γ, δ, ε) corresponding to the inner side.

Angles κ, λ and μ are calculated by subtracting ν (90° angle from the center to edge of image) and ι (½ the horizontal field of view) from 180 (Fig. 2). With this information the distances can be calculated as:

$$\frac{1}{2}Ws=\mathrm{sin\iota } \times \left(\frac{\mathrm{FOVshort}}{\mathrm{sin\kappa }}\right),$$

$$\frac{1}{2}Wc=\mathrm{sin\iota } \times \left(\frac{\mathrm{center of view}}{\mathrm{sin}\lambda }\right),$$

$$\frac{1}{2}Wl=\mathrm{sin\iota } \times \left(\frac{\mathrm{FOVlong}}{\mathrm{sin\mu }}\right).$$

Results are then multiplied by two to yield the width at each point.

For instances where FOVshort or FOVlong are > visibility, the height (in # of pixels) were truncated to only represent pixels within the visible range. For our simulations, we assumed no camera distortion as all cameras used in this study were designed for in-water film. For other cameras and lenses (such as fisheye, in-air, or VR), distortion should be accounted for as not all pixels may represent a similar portion of the angle of view.

A fitted polynomial regression using the polyfit function in Matlab was implemented to interpolate a smooth curve indicating the distance to the far edge of each pixel for both the vertical and horizontal measurements. The height of pixels in each row (N) was determined as:

$$\mathrm{Pixel height}\left(N\right)= \mathrm{vertical polyfit} (N)-\mathrm{vertical polyfit}(N-1)$$

Width for each pixel in each row was calculated as:

$$\mathrm{Pixel width} \left(N\right)=\frac{\mathrm{Horizontal polyfit} \left(N\right)}{\mathrm{Width }\left(\mathrm{total }\#\mathrm{ of pixels}\right)}$$

Height and width results were multiplied element by element to create a matrix representing the area of each pixel in the photo. The sum of these elements produce an estimated area.

The included simulations represent the area at a still frame assuming the given parameters and value at the x-axis. This information may be extrapolated to moving video using the function:

$$\mathrm{Video area}= \mathrm{Area estimate }+{\sum }_{n=2}^{\#\mathrm{of} \mathrm{frames}}w{l}_{n}\times (\frac{{\mathrm{speed}}_{n}\mathrm{ in} m/s}{\mathrm{frames per secon}d}-({\mathrm{depth}}_{n-1}-\mathrm{dept}{h}_{n})),$$

where n represents individual video frames. With reliable speed estimates this method may produce a more accurate area estimate for the video assessment, particularly in instances of high speed and low pitch.

For our camera parameters, roll was found to be arbitrary in the area estimates (thus all simulations assumed roll = 0). This is to be expected with a forward-facing camera, as the image is centered on the roll axis. However, roll must be accounted for any tag in which the camera is not oriented as such. Consider also that at instances of higher roll in a deployment, the animal’s body may take up more of the frame and reduce the visible surface area. For this reason, this study only includes ice observations where the roll of the tag is <  ± 90˚.

Visibility conditions on the peninsula are known to vary widely with productivity, and anecdotally, visibility has been estimated to be as high as 40 m. During these deployments, the depth at which the surface became visible indicated lower visibility conditions of ~ 5–15 depending on the time and location of the observation. The visibility values tested in our simulations are intended to reflect the conditions occurring during the deployments as opposed to theoretical maximum/minimum conditions reported in the area.

Scoring of ice observation

Animal tag video was viewed with Behavioral Observation Research Interactive Software (BORIS) [18]. Using BORIS, an observer marked the initial and final surfacing of each surface series for evaluation (Fig. 3). Two independent observers evaluated ice concentration as a % value in one of 6 concentration categories: 0%, 1–20%, 21–40%, 41–60%, 61–80%, 81–100% (Fig. 4, Additional File 1). If scores were not in agreement for a given point, an additional evaluation was made by a third observer, and the median value determined as the ice concentration.

Fig. 3

Combining environmental and behavioral data. Ice observations in relation to dive profile and behavioral information from a segment of the second deployment. Ice observations are represented by the colored dots over the dive profile in the upper graph, with pitch roll and heading of the animal represented in the lower graph. Ice observations collected from the same platform as the sensor data allow for temporal precision in linking the overhead ice coverage to animal behavior

Fig. 4

Ice categories. Example of typical ice image for each category from animal-borne tag

Each evaluation took place from the first frame in which the surface is visible upon ascent until the frame in which the camera reaches the surface of the water (or when the animal initiates its descent if the camera does not break the surface). The steepest angle to the visible surface (FOV^short) varied between pitches of 23–75 degrees, with an average pitch of 47 degrees. Continuously assessing video allowed the observers to use movement as a cue for the size and depth of larger chunks of ice, as well as help to identify ice concentration in sunny areas.

The observations included in this study primarily occurred in open water, or within glacial bays along the coast of the West Antarctic Peninsula. During summer in the bays, the ice coverage primarily consists of broken chunks (brash) consisting of glacial and/or marine ice. In this study, “surface ice” refers to any ice observed in the marine environment, though not all of the ice may be of marine origin.

Each ice observation in this study is synchronized with satellite GPS time prior to deployment. This allows for video observations to be accurately aligned with behavioral data provided by other tag sensors (Fig. 3). As a result this methodology provides extremely fine temporal resolution for comparing ice observations with any other tag-derived data.

Results

Satellite imagery

From our search criteria the Sentinel-Hub EO browser yielded 21 satellite scenes with an average cloud cover of 80.47%. 12 of the images in this area overlapped with tag GPS data (regardless of time), with the cloud cover in this subset averaging 79.04% cloud cover. None of the satellite images were taken during periods of tag deployments, though one image was taken within a 24 h window (~ 2 h 34 min prior to the deployment on 02/28/19). Cloud cover for this image was recorded as 91.48%, rendering inadequate view of the surface for ice analysis.

Area of view estimates

Area of view simulations (Fig. 5) show that under reasonable conditions the visible surface area from tag can be expected to represent an area similar or finer than a typical 10 m (100m²) pixel of visible satellite imagery. As depth decreases (Fig. 5a, d) AOV was found to increase as more pixels come within visible range. When all possible pixels contain visible surface, the AOV then decreases as the camera ascends towards the surface (imagine Fig. 2 as the tag travels up the depth line P). From the point at which visibility impacts the tag field of view, our simulation show a linear decline towards an area of 0m² as visibility is reduced (Fig. 5b, e). Moderate pitch was shown to produce the largest area estimates, with reduced area as the tag approached a vertical or horizontal position (Fig. 5c, f). To relate these simulations to our deployments, tag data of the first full view of the surface for each tag position (with tag slips determined by changes in the accelerometer orientation) was recorded for 36 independent tag positions (Table 1).

Fig. 5

Area of view simulations. Simulated area of view for different conditions of depth, pitch, and visibility

For the first fully visible frame of surface, we can assume that FOVlong (Fig. 2) represents the visibility. If we assume the mean depth and pitch (Table 2) and solve for the length of FOVlong, we find a typical visibility of 10.00 m. Under these parameters, the average image contains 43.46m² of visible surface. These findings indicate that the typical area viewed by a camera tag is approximately half the area of a single visible spectrum satellite pixel.

Table 2 Tag depth and orientation data

Ice observation

Across all 7 deployments, observers recorded ice concentration for 863 surfacings (Table 1). The tag positions and ice observations indicated in Table 1 exclude instances where the tag was positioned at a roll angle greater than ± 90 degrees from the dorsal of the animal. For some of the deployments, this means that ice concentration was not recorded for the entirety of the video.

Our results indicate that Antarctic minke whales in the coastal bays around the West Antarctic Peninsula occupy low ice content areas for the majority of the time during which video was recorded, with 84.36% of observations in ≤ 20% ice cover (Table 3). Ice was present in 48.55% of observations, however only 8.22% were recorded as > 40% ice cover.

Table 3 Observed ice concentration

Individual deployments, however, showed variability in observed ice content, depending on the location of the deployment and behavior of the animal. The left side of Fig. 6 shows deployment bb180125-30 from the Penola strait. The pseudo-track reveals that the animal is primarily moving in open water along a fairly straight path, indicating traveling or resting behavior. This contrasts with the right side (Fig. 6) deployment, bb180227-45 from Andvord Bay, in which the animal demonstrates fidelity to locations with high-ice concentration, indicating the occurrence of foraging behavior under ice. By linking ice observations with associated tag-data (Figs. 3, 6), the relationships between location, ice cover and animal behavior can be more accurately investigated.

Fig. 6

Ice concentration comparison. Pseudo-tracks with ice observations (indicated by colored markers) of deployments bb180125-30 (left) and bb180227-45 (right) and accompanied ice concentration distributions

Discussion

Animal-borne video data offer previously inaccessible insight into how marine species, such as the Antarctic minke whale, interact with their physical environment. Previous work in the region has relied on visual sighting surveys that provide information on animal distribution at relatively coarse spatial scales and link these to satellite imagery at scales orders of magnitude greater than what we are able to measure from the animal’s perspective [19, 20]. Similarly, the only published account of Antarctic minke whale tag-derived behavior in relation to surface ice used positions from Argos tags that have error estimates of up to several kilometers, and linked behavioral state (transiting versus area-restricted search) to coarse satellite-derived sea ice concentration data [11]. Our method for describing surface ice concentration at such fine spatiotemporal scales, with continuous reliable information on behavioral state from motion-sensing tags, will now allow greater quantification of how the behavior of this species is affected by ice in the environment. This information is critical for understanding both the ecology of the species and how it will be affected by the changing ice conditions.

One of the advances that our method provides relative to current satellite technology is the spatial and temporal precision in linking environmental observation to fine-scale behavioral data. Surface ice is incredibly dynamic and can change quickly depending on local weather and oceanographic conditions [12]. Thus, any differences in the timing of animal behavior and ice measurements could generate spurious results on how ice influences animal behavior. By combining behavioral observations and ice concentration on the same tag platform, we can eliminate any such offset. This will allow future studies to compare environmental features with animal behaviors ranging from feeding rates on a dive-by-dive basis to specific kinematic strategies of a single event over mere seconds (Fig. 3). Our area of view analysis reveals that the spatial scale of our observations are equivalent to or finer than a single satellite pixel representing 10–60 m (100–3600 m²). These results indicate that our method offers significant increases in temporal accuracy from SAR and visible spectrum ice assessments, and in spatial (and likely temporal depending on the timing of satellite passes) accuracy from microwave radiometry.

In this study, we chose to include sub-pixel (satellite) information in the form of ice concentration bins. This allowed us to further characterize the percentage of ice-covered environment around the animal, but inevitably introduces subjectivity in the assessments. We chose not to adopt a computer vision or machine learning approach to ice quantification due to the challenges presented by the highly variable surface conditions and backlighting. Still, observer subjectivity may be decreased by limiting the number of bins. For areas where remote data are scarcely available, or where sea ice is permanent (e.g., fast-ice), even a binary (ice or no ice) evaluation may provide valuable information that is otherwise unobtainable. Though our method allows for a robust assessment of the environment proximate to the animal, it lacks the ability to accurately map the ice distribution of the greater environment beyond the tag’s visual field of view. To adequately answer questions that require such knowledge, a combination of techniques may be required.

Recent advances in remote sensing drone technology may facilitate the development of an approach to study the dynamics of ice distribution at a temporal scale relevant to animal behavior [21]. As drones can cover area quickly and collect high-resolution imagery below clouds, they can be used to establish a precise estimate of the ice-covered habitat available in a given location either as a frequency distribution of % ice cover or some other metric for the amount of ice cover types and concentrations. This information can then be compared to tag-observed surface ice to determine if the animal utilizes ice cover at similar or different frequencies from what is available. This approach will allow a description of animal habitat usage without having to rely on potentially inaccurate GPS data to locate the animal within a specific photograph of ice distribution.

While the main function of this technology and method is to better understand the ecology of the animals in question, tag video may also be useful in concert with remote sensing to understand sub-pixel variation for methods of ice classification in coarse resolution satellite data. For example, close range tag-derived ice data may be useful for understanding sub-pixel variation in ice types within the ice coverage presented in passive microwave imagery. However, satellite validation at the submesoscale requires greater spatial coverage of animal-borne video and high-quality locational information, especially for emerging high-resolution satellite data.

Conclusion

The ability to precisely and accurately understand the relationships between different animals and their environment is a necessary, and often times lacking, piece of information that must be available in order to effectively understand the impacts of environmental change. This method sets the precedent of using tag video data to evaluate ice conditions, allowing us to link environmental and behavioral observations with spatial and temporal precision that is not currently possible using satellite remote sensing methods. In rapidly changing polar regions, this information is particularly important for the numerous species that have evolved to rely on ice as a substrate for critical life history events. Specifically for Antarctic minke whales, this method will allow us to better understand how and if rapid warming and changes to ice conditions around the Antarctic Peninsula will ultimately make this area unsuitable for them to inhabit.