Automated characterization and quantification of hydrocarbon seeps based on frontal illuminated video observations

Hydrocarbon releases, either natural or due to anthropogenic activities, are of major relevance for the marine environment. In this work we specify our approach to quantify these seeps by subsea imaging utilizing camera systems and frontal illumination setups on board remotely operated vehicles. This work showcases, based on a campaign in the region west of Svalbard, improved methodological guidelines for the seep quantiﬁcation operation together with a novel automated post-mission evaluation. The comparison of automated quantiﬁcation with manual information extraction illustrates the efﬁciency of this new method while processing comparable estimates of seep characteristics.


INTRODUCTION
Oil and gas in the subsea environment can emerge from natural seeps or anthropogenic activities.Especially methane bubbles are of high relevance of their global warming potential of at least 20 times as the same mass of carbon dioxide [1].Although numerous gas seeps have been found worldwide, the amount of gas escaping from the seafloor has only been assessed at a few sites [2]- [4] lacking of rapid tools for quantification of gas volume and bubble numbers.This need for additional estimations of the methane transport process is also underlined by new temperature and salinity gradient studies showing much higher estimations of hydrocarbon flux from natural vents in the Golf of Mexico (GoM) than previously expected [5].Gas seepages in shallow water partially contribute to the emissions into the atmosphere due to the seasonal deep mixing of the water column down to the plumes above active seeps.In fact also seeps in the deep sea can be traced far above the dissolution zone to the surface as well as dissolved methane gas raised to the ocean mixed layer where it can merge with the atmosphere through the equilibrium at the oceans surface [6].Releases of methane are often accompanied by hydrate shell formation and surfactants like oil, both strongly affecting the rise characteristics and fate of the bubbles (e.g.[7]- [9]).Furthermore oil coated bubbles are suggested to be able to transport the methane preferable to the oceans surface preventing them from dissolving underway [10].Additional to methane seeps oil droplets from natural seeps are frequently reported in hydrocarbon rich areas like the GoM [11] eventually reaching the sea surface and forming circular oil slicks (so called pancake slicks) that may even consolidate to larger slicks as observed from remote sensing.
Apart from natural seeps and their need for quantification, anthropogenic sources, such as pipeline leakages, are likely to have significant ecological and economic impact, and it needs in situ detection and rapid quantification to mitigate adverse effects [12].To match these requirements optical as well as acoustical methods have been developed over the last decades addressing the in situ bubble measurement.Instrumentation using imaging optics can be divided into backlight and frontal illumination systems.Backlighted systems can provide high-resolution information with optimized image quality [13] however they are rather bulky, require extensive energy and provide only a limited field of observation.Front light illuminated systems are much more flexible and smaller, sometimes already part of the mission equipment of modern Remotely Operated Vehicles (ROV's).Their application in seep characterization and quantification requires specific mission planning and experienced operators.Routines implemented for automated analyses make intensive use of image algorithms like optical flow, edge detection and dilation [14].They require being adapted to the video material available and appropriate bubble tracking strategies.Validation can be achieved by laboratory experiments and supplementary methods during subsea campaigns, among them acoustics [15], manual feature extraction and volume sampling [16] [4].
In the following an automated algorithm for the quantification of hydrocarbon seeps and the corresponding special mission planning will be presented.To illustrate the operation of the algorithm field data from the region along the continental margin of western Svalbard, will be shown.This study site includes methane seeps with and without hydrate shell formation.The procedure for detection and recognition of the bubbles, their tracking as well as a comparison with manual measurements will be presented.

STUDY SITE AND MATERIALS
The frontal illumination approach was tested on different campaigns in the last years.These differ in study site (deep sea and shallow water), as well as in technology used which was available during the campaigns.These campaigns resulted in a high amount of video material from seep observations (>500 hours).
As an example for the application of an automated quantification an Arctic campaign will be described taking place from August to September 2012 along the west Svalbard continental margin [17] (red circle in Figure 1).The overall objective of this campaign was to quantify the amount of methane that is released as gas bubbles from the seafloor in order to prove the described shift of the gas hydrate stability zone (GHSZ) in this region due to a temperature increase of the bottom water of approximately 1 • C by Westbrook et al [18].
The approach was to combine hydroacoustic mapping using ship board echo-sounders (multibeam EM 710, fish finder EK 60) and ROV based measurements of gas emissions.For this purpose measurements were taken both above (∼ 80m below sea level) and below (∼ 400 m below sea level) the GHSZ for methane outlets.
The ROV was equipped with a horizontally-looking sonar allowing to find individual gas bubble streams at the seafloor.The volume flux of methane was then estimated by quantifying the flux of individual bubble streams using three different methods.Direct measurement with a bubble catcher (a funnel with a measuring cup) which was used at selected seeps to collect bubbles over a certain period of time, typically five minutes.Other methods were visual measured using the ROV video material and acoustically by using the horizontally-looking sonar (Imaginex 881a).
For this cruise the R/V Heincke was available (Cruise No. HE 387) equipped with a mid-size inspection-class ROV "Cherokee" (Sub Atlantic, Figure 2) operated by the Centre of Ma- rine Environmental Science MARUM (University Bremen).The ROV was equipped with a 756 x 480 pixels camera (Typhon PAL, Tritech) with a frame rate of 29 frames per second (mounted in the frontal centre with pan and tilt actuators), and with a small manipulator arm (located on the port side at the front of the ROV) which was able to hold a small scaling plate (0.35 x 0.35 m) next to the seep.Since the plate could not be carried on all dives, the manipulator arm dimensions were measured at marked points in order to fulfill the same function.Furthermore, the use of an artificial homogeneous background was omitted due to the reflections from the scaling plate which were caused by the short distance between camera and plate unlike other campaigns before [19].Therefore the ocean was chosen to be sufficiently homogeneous as a background for image analysis.Comparisons between the images with and without the plate were performed to proof that no disadvantages or influences were caused (results not shown).In contrary, the simple use of the manipulator arm as a scaling aid influences the stream much less than the additional scaling plate does and therefore the camera was able to focus on the bright uprising bubbles instead on the reflections caused by the plate.The video material which was used for the image analysis was created according to a specific operating plan, where different perspectives and video modes were used to determine the best possible settings for observing the seeps to keep a standard routine for later comparison with other datasets.After each dive the ROV operators provide the video material which could then be sighted and cut to the relevant scenes for data reduction followed by a dismantling into single images for the following analysis.Which was done with Adobe Premiere Elements Version 11 where the video footage can be cut and dismantled into single images.

PROCEDURES AND ALGORITHMS
For a quantification of the volume flux of emerging gas from bubble streams, in some cases a funnel with a measuring scale was positioned above the source for a defined time.In order to characterize the individual bubbles, a video was recorded (typically 5 -10 min) in the following the bubble parameters could be extracted manually, such as bubble diameter, rising speed and number of bubbles over a defined period [16].This procedure is very labor intensive, since each individual picture needs to be evaluated.For this reason, only a small fraction of the material can be considered and data is then extrapolated.At this point, our work ties in.As our objective is to provide an algorithm that can quantify and characterize rising bubbles.Based on the available parameters that can be determined from a video frame typical characteristics can be calculated including discharge volume, bubble size distribution, rising speed and rising characteristics.The main advantage of the automated approach is obvious when compared to an extensive manual measurement of about 1000 frames (temporal expense of more than a week).From automatic analysis about 55000 frames can be evaluated within one working day.This corresponds to a video of approx.30 minutes against a few seconds of manual evaluation.
For the recording of gas leaks a special mission plan is needed that ensures comparability of the results.The process begins with the search for potential seeps with the ship based sonar systems.If a seep is detected by the acoustic flare, the ROV will be launched in an attempt to locate the source with the ROV based sonar.Reaching the seep an overall view will be created to be able to described seep expansion.Than video sequences will be recorded with multiple angles (front view, 90 degree turn) and zoom settings (overall scene, single seep, close up).In parallel, the sonar measurement can be conducted, if available.Following the recording the leakage will be quantified with a bubble-catcher.Therefore the funnel will be positioned above the seep for a certain time as described to determine the bulk volume flow.This result can also be later used to validate the automated measurement.[17].

Manual Quantification
As described before there are various methods to manually quantify seeps characteristics for a quick estimate of a source.Video cameras are nowadays standard equipment of ROV missions and the resulting video material can be visualized and analyzed with commercially available tools.This approach can be especially useful to decide on the further investigation of gathered data.The method is not very different from the automated measurement.Individual bubbles of a frame are assessed by an image editing tool and written in a table.Another common technique to estimate the volume flux of a methane seep is to place a funnel with a scaling aid over its outlet.This method is more error prone because the funnel must be set directly over the seep to collect all bubbles which is by itself sometimes difficult due to low visibility.In general manual data analysis can be helpful to validate automated analysis methods.

Automated Quantification
All code development and testing was performed using the MATLAB release 2012b software suite and the image processing toolbox release 2012 (The MathWorks).The automatic quantification can be divided into three areas as shown in the schematic workflow (Figure 3).This is firstly the actual image processing in which the objects were detected (in this case gas or methane bubbles) and thus become traceable.In a second step, the detected objects are assigned by a tracking algorithm and no-bubble objects are automatically filtered out such as small fish or marine snow.The final step is the evaluation, namely the representation of the seep characteristics from the tracked objects.
The automated quantification will be described in detail in the following three sub sections.

Image Processing
The first step in the further analysis of the video material is to reduce the amount of data.Meaning that the image is cut with respect to the relevant sequence.The aim is that solely the bubble stream is visible and no interference effects are present in the image like reflections or other disturbing backgrounds.For the recognition of the objects the color image needs to be converted into a gray scale image with rgb2gray (Matlab, The Mathworks), as the subsequent processes requires this format.
After the first steps of adjustment (as shown in Figure 4 (left)) the image will be processed with the optical flow method from Horn-Schunck.Here, the optical flow is the apparent motion within a scene of the resulting movement of an object relative to a reference background, which is represented by the projected pixel area of a bubble in the gray-scale image of the current frame and the previous frame.In this case every moving pixel is represented by an array of vectors that include both the velocity and the direction (heading).For further information of the optical flow detection see [11,14].The result of this process is a binary image (Figure 4 (middle)), which includes only the objects (white pixel areas within a black image) that have moved between two frames.At the same time, this is also requiring to apply filters (basic morphological operations like dilation and erosion) to identify the relevant data.Objects smaller than a defined minimum size (typically 2 mm diameter) will be removed, assuming with reason that they are not bubbles but e.g.marine snow.Furthermore large objects (>15 mm diameter) such as zooplankton or bubbles outside the depth of field will be removed, which cannot be calculated with the current scaling.Additionally objects touching the edge of the image will be removed, because it is not possible to give an unequivocal statement about their parameters.This step is illustrated in Figure 4 (right) where the detected bubbles are outlined with red edges.
For all resulting objects of this image processing the following parameters can be determined and transferred to individual matrices.Here they will be multiplied with the scaling factor (pre defined value by determine the mm to pixel ratio) to get metric distance units.These are: (a) the object length in the minor axis for the diameter (in this campaign the bubbles moved during the exposure of each single frame, resulting bubbles that are not spherical but rather cylindrical captured.Therefore the minor axis is defined to be the bubble diameter); (b) the area centroid of this object (which is later used to trace the bubble); (c) the object position in the image in X and Y coordinates (for later correlation between all objects tracing the bubbles); (d) the object orientation in minor and major axis (also for correlation); (e) the projected area of an object to sort out unexpected size variations (max.variation is 25%) for later correlation and (f) the number of objects per frame.

Tracking
The information gathered in the image processing are stored in single matrices by their parameters.There they are listed in columns frame based and in rows object based.Thus we achieve for each frame the number of objects with the appropriate information.In a next step, the probability tables will be created.These are describing the probability that a single object of the original frame is identical compared to all objects of the following frame.For this purpose, all collected parameters of the objects are compared and the probability of correspon-dence in percentage is created.These are then sorted by the highest probability, thus resulting in a new arrangement of the objects in the following frames.For the calculation the above mentioned information will be used, together with other criteria to exclude unusual changes or special events.These are as follows: (a) movements in the negative Y direction at a defined level (typically twice the max expected bubble size), such sinking objects will be excluded; (b) leaps in X direction that exceed the natural oscillation (oscillation greater than the bubble size) to exclude objects which move sideways; (c) differences in size to exclude misinterpretations of the tracking; (d) objects that exit the image frame must not be edited again; (e) objects that newly entered the image frame need to be recognized as such; (f) objects that are at the threshold of the minimum/maximum size tend to flicker and require treatment to be traceable again; (g) overtaking and overlapping objects; (h) inhomogeneity in the background, reflections or shadows, especially when using the plate must be excluded from the sorting process.It has to be noted that slight movements of the ROV during the recording can not be considered and therefore require appropriate ROV operations.

Visualization
The last step of the automated processing is the statistical analysis and visualization of results obtained from the preceding steps.In preperation of the analysis some information needs to be aggregated.While creating the probability tables and tracking individual bubbles several matrices were created.This results in a large matrix of all the bubbles which have risen in the time of the recording through the image.As this usually corresponds to about 20-30 frames it means that about 20-30 values for each parameter for a single bubble are available.From these a statistical average and the standard deviation are calculated.This information for the individual bubbles can than be visualized in plots like Figure 5.In the left part of Figure 5 the trajectories of each bubble are shown.Each bubble is one line with varying degrees of oscillation and length, depending on the time where they come into the im-age and whether it has changed size during bubble tracing.As a result of different current during the recording time the bubble trajectories diverge while traveling in Y direction.In the lower left corner the number of frames and bubbles are displayed.In the right part of Figure 5 rise velocity is plotted against bubble diameter.In the bottom left corner of the figure, overall mean values and standard deviations are given.These are calculated on the basis of all bubbles to provide average speed, average bubble size and average flow rate per minute.

ASSESSMENT
In the previous sections it has been shown how rising gas bubbles can be quantified and characterized using an automated measurement system.Thus an enormous performance boost can be achieved.While the manual quantification methods allow 100 to 1000 images to be examined, the automated method analyses up to 55,000 images on a working day.This large capacity of evaluation brings new opportunities for the surveillance of seeps with it.These can now be studied over longer periods for searching influences such as tidal gradients or other pressure variations as caused by the arrival of ROV's.
In this example, from a measurement during the campaign around Spitsbergen 2300 frames were recorded.319 bubbles were counted with an average velocity of 278 ± 57 mm/s and with an average bubble size of 3.7 ± 0.2 mm.The mean volume flow of this seep is 54.7 ml/min.The measurement itself lasted a few seconds and would take approximately two to three days if analyzed by the manual method.For comparison we analyzed a subset of N = 100 frames via manual quantification as described in section 3.1 and found an average velocity of 262 ± 32 mm/s with an average bubble size of 4,2 ± 0,6 mm and a mean volume flow of 40,4 ml/min.The volume flow was additionally sampled with the funnel method resulting in 56,3 ml/min.The individual methods thus provide consistent results taking into account the different timeframes are covered.Results from the Svalbard campaign (HE 387), show that most bubbles are in a velocity range from 0.2 and 0.35 m/s with a size from 2 to 5 mm (spherical equivalent radius) which is a well published range for methane bubbles in the region of west Svalbard [18,4,16].
Results of the automated measurement are also comparable with results from literature.Roemer et al [16], used for their work two manual quantification methods (manual counting of bubbles in images, and for validation of the volume flow a bubble catcher), and comparisons of the example of Figure 5 show that the measured values of the automated method correspond to the published methods.In a direct investigation of the images frames it can be seen (Figure 4) that single bubbles can be detected with high accuracy.There, the bubbles are bordered with a red line to show which part is detected.This is meant solely for presentation and verification of detection and is not used for the actual measurement process.In addition, within the automated method multiple parameters can be derived, such as the orientation of the bubble and the area of the bubble, which can be taken into account in the later assignment and tracking of bubbles.Furthermore, tra-jectories of bubbles are generated and the evaluation of these recorded trajectories opens new possibilities in the assessment of the bubble stream characteristics.Potentially the oscillation of bubbles can be used to recognize a possible gas content as described by [3,15].This can be observed especially for so called "dirty bubbles" explained in the following.Bubbles below the GHSZ typically form a hydrate layer.During the formation of this shell dirt, oil and other particles can be torn with, causing the bubble to get a partially opaque shell.On the basis of the trajectories the gas content of such a bubble can possibly be determined.
But there are also current technical limitations even if modern ROV's are equipped with HD underwater cameras and high performance LED lights, the movements of rising objects cannot be optimally recorded for analysis.This is due to the exposure time required, therefore the movement could not be "frozen" resulting in a blur.Currently within the MARquant project a new camera system is under development using a smart camera synchronized with two short-term LED flashlights, to record videos (series of images) in which the illumination it adapted to the exposure of the camera.Thus, a perfectly sharp image of the object can be generated [20].
Further restrictions exist in the geographical extension of seep structures.Gas and oil seeps in the marine environment can range from small irregular point sources to widespread leakages.Obviously single sensor systems are not likely to address the full range, even though significant advances were achieved with modern imaging technologies and algorithms in characterizing and quantifying these hydrocarbon sources.Therefore a combined mission approach of acoustical and optical methods is required (as outlined in chapter 3).Further application areas of this methodology will include the monitoring of CO2 depositions or inspection of bubble curtains used for acoustic shielding during ram actions.Likely future scenarios will integrate bubble sensors on autonomous platforms improving flexibility and spatio-temporal coverage required to fully assess environmental impacts in the subsea domain.

FIG. 1
FIG. 1 Map of Svalbard.The study site was located along the continental margin of west Svalbard, marked with a red circle.

FIG. 2
FIG. 2 SubAtlantic ROV after recovery at west Svalbard (see text for technical details and instrumentation).

FIG. 3
FIG. 3 Schematic workflow of the automated quantification.

FIG. 5 (
FIG. 5 (left) Trajectory plot for a methane seep at west Svalbard.The lines correspond to the movement in x and y coordinates (mm) of each bubble.(right) Bubble rising velocity over the diameter distribution.The green and the blue cross are representing the first and second standard deviation respectively.In the lower left corner also the mean diameter, mean velocity and mean flux are given for the n = 319 bubbles traced.