High‐Throughput Object Recognition and Sizing in Disperse Systems

Size and shape of dispersed objects defines properties of suspensions, emulsions, and foams, such as stability, texture, and flow. Accordingly, a rational product design requires reliable size distribution analysis. This is particularly challenging in dense foams. An endoscopic setup was optimized for bubble imaging minimizing light reflections, uneven illumination, and foam distortion. A software tool was developed detecting large quantities of foam bubbles at dispersed phase fractions up to 93 % from images with spatially varying contrast within minutes based on the template matching algorithm. Reliability of the method is also illustrated for a bimodal glass bead mixture, anisotropic nanocrystals, and emulsion droplets during freezing.


Introduction
The analysis of the size distribution of objects in disperse systems is of particular interest, e.g., for prediction of dispersion stability, texture, and flow. The size distribution of the particles, gas bubbles or liquid droplets can often be accessed by imaging technics, such as microscopy or endoscopy. Further analysis of size and shape requires fast and reliable detection of high quantities of particles on the images. Endoscopy combined with automated image analysis is well-established for emulsions or bubbles produced in stirred vessels, the dispersed volume fraction in these studies, however, was below 45 % [1][2][3][4].
In foams, endoscopy is also appropriate for bubble size distribution analysis [5,6], because it allows for in-situ observation of fragile foam structures and overcomes drawbacks of bubble observation through the container wall, e.g., capturing bubbles sheared or distorted at the wall. In food or personal care foams, however, the gas volume fraction is often higher than the maximum packing fraction of spheres ( 64 vol %), and spatially varying contrast, light reflections, and bubble overlap on the images challenge automated bubble detection, especially when fast transient phenomena need to be monitored.
Here, an approach is presented to determine the bubble size distribution inside densely packed foams combining endoscopic imaging and template matching image analysis. The endoscopic setup is optimized for imaging foams in backlight illumination with no need of filtering or post-processing if the template matching technique as described by Hofmann [7] and Zabulis [8] is used for particle detection and subsequent size distribution analysis. This technique has been implemented in a customized software tool and consists of the selection of a template inhering similarity with the particles on the image, screening of the image for sections with high similarity to the template (calculated using a zero normalized cross correlation (ZNCC)), and subsequent validation of the matches, as illustrated in Fig. 1.
The feasibility of the method, its reliability, and accuracy is illustrated analyzing the size distribution of two glass bead fractions and their bimodal mixture, selectively determining the size and number of crystalline and amorphous emulsion droplets during freezing, and analyzing the length and width distribution of rod-shaped nanocrystals on microscopic images, respectively. From endoscopic images, finally the time evolution of the average bubble size and size distribution width of a protein foam during drainage and aging are recorded.

Image Capturing
Fractions of glass beads (Spheriglass 3000 and Spheriglass 5000, Potters Industries LLC, Augusta, USA, fractionated in a customized up-current classifier) of 5-20 mm and 10-35 mm in diameter, respectively, as well as a 1:1 mixture of them were dispersed in acetone (20 wt % glass beads) and spread onto cover slips. After acetone evaporation, images of the particles on the cover slip were taken with an inverted bright field light microscope (Axio Observer D1, Carl Zeiss, Oberkochen, Germany) under 40-fold magnification (Objective A-PLAN 40x/0.65, Carl Zeiss). Images of each glass bead fraction (depicting over 1000 beads) as well as an image of their mixture (depicting 440 beads) were analyzed. In parallel, both glass bead fractions and their mixture were diluted 1:10 5 in 0.9 % sodium chloride solution and the size distributions were determined using Coulter counting (Coulter Multisizer II, Beckman Coulter, Brea, USA). Additionally, the size distributions of the particles were detected using laser diffraction (Helos, Sympatek, Clausthal-Zellerfeld, Germany) combined with a dry dispersion unit (Rodos, Sympatek).
An emulsion of 1 wt% hexadecane (Sigma-Aldrich, St. Louis, MO, USA) as dispersed phase, 1 wt % Tween 20 (Carl-Roth, Karlsruhe, Germany), and 98 wt % Milli-Q water as continuous phase was prepared using a tooth-rim dispersing machine (IKA T25 digital, ULTRA-TURRAX , Staufen im Breisgau, Germany) at 2.2 m s -1 tangential speed (3200 rpm, 13 mm rotor diameter) at 28°C for 10 min. Emulsion imaging was performed using a customized polarizing microscope (Eclipse Ci-L, Nikon, Shinagawa, Tokyo, Japan) equipped with a temperature-controlled stage (LTS 420, Linkam Scientific, Tadworth, UK); 25 mL of emulsion was pipetted between two microscope cover slips placed on a tempered microscope object slide using a tempered pipette, then covered with a third cover slip and sealed at 28°C. Droplet clusters were focused to depict as many droplets as possible. The sample was cooled from 28°C to 0°C with a cooling rate of 10 K min -1 . Images of the emulsion were recorded at 20, 12, 9, 7.6, 5, and 0°C and the size of liquid and crystalline drops was analyzed separately. Droplet size distribution was determined at T = 28°C, i.e., above the melting point of hexadecane, by means of a laser diffraction particle size analyzer (HORIBA LA-940, Retsch Technology, Haan, Germany) without further dilution of the emulsion.
Images of foam bubbles were captured with a non-flexible 295 mm long endoscope (TVS80.280.BF6.AD10.2x-Zoom, Visitool, Maulbronn, Germany) covered by an outer metal tube, 300 mm in length and 8.5 mm in diameter, closed with an optical glass disk (custom-made product, Visitool, Maulbronn, Germany). It was inserted into the foam prepared from 40 mL of 1 wt % bovine serum albumin solution which was foamed in a filter funnel (VitraPOR glass filter Por.4, ROBU Glasfilter-Geraete GmbH, Hattert, Germany) purged with nitrogen. The endoscope was connected to a USB camera (Lumenera LU 160, Teledyne Lumenera, Ottawa, Kanada). Illumination was provided by a gooseneck light (KL 1500 LCD, Schott, Mainz, Germany) placed on the outside of the filter funnel wall. Three image sequences with a frame rate of 0.1 s -1 were recorded in freshly prepared foam, and bubble size distributions were analyzed for each image, respectively.

Image Analysis
Images of dispersed objects were evaluated with a software tool written in Matlab (MathWorks , Natick, USA). The scaling factor relating pixel number and dimension was determined manually by drawing a line along a defined distance on the images, i.e., a scaling bar, within the software tool. The ratio between the number of pixels spanned by that line and the defined distance was calculated. A binary, circular pixel pattern with a ring thickness of 10 % of its radius and a filled circle was generated within the software tool serving as screening templates for foam bubbles and emulsion droplets, respectively. Images of glass beads were screened using a similar template with a ring thickness of 20 %. An image of rod-shaped nanocrystals was screened with an image section depicting a single crystal. The single crystal template was rotated in steps of 30°t o cover the different orientations of the crystals on the image.
The size range of the objects on the images was measured manually by drawing a line across the diameter of the biggest and the smallest glass bead, bubble or droplet, respectively, within the software tool. The obtained limits were extended by 10 % to improve reliability. For the rod-shaped nanocrystals the biggest and smallest diameter and length were measured likewise. The number of template sizes used for screening was chosen between 20 and 30 logarithmically distributed within the respective size range. The template was rescaled by the software as many times as chosen. The template of each size (M N pixels) was then centered once at every pixel position (x, y) of the image (P Q pixels) to be screened and the similarity between the pixel pattern of the template and the respective image section was calculated with the normalized cross correlation function: Figure 1. Scheme of template matching comprising detection and subsequent validation. A template is centered at each pixel position on an image and its similarity (zero normalized cross correlation value) stored in a similarity map. Local maxima on the similarity map are detected, filtered with a threshold, and positions and template size are stored in a matrix. This detection step is repeated for n template sizes. The validation step compares the matrices of all detections and filters for highest similarity values.
where Template(i,j) 1) is the grey scale of template at position (i,j) , T is the mean grey scale of the template, Image(x-i, y-j) is the grey scale of image beneath the template position (i,j), and I x;y is the mean grey scale of the image section beneath the template centered at (x,y). The zero mean normalized cross correlation is normalized to the range [-1; 1]. The value 0 corresponds to a complete mismatch, the value 1 to a perfect match, and negative values correspond to an opposed match (negative of the template and image show the same pattern) [9,10].
The similarity value at each template position was stored in a matrix called similarity map. The detection step iteratively repeats with the selected number of templates. As a result, a stack of similarity maps was generated. Each similarity map was screened for local maxima (surpassing a specified threshold) using the LocalMaximaFinder Toolbox (FileExchange, Mathworks, Natick, USA). The positions of these local maxima in the similarity map corresponded to the positions in the image, where the algorithm detected an object.
After the detection step was completed for all template sizes, a validation to filter overlapping detections was done. If the Euclidian distance between the centers of two matches was shorter than 80 % of the sum of their radii, the algorithm retained the location and template size of that showing the higher similarity value [7].
The results were presented as histograms and cumulative distributions of counts (q 0 , Q 0 ), area (q 2 , Q 2 ), and volume (q 3 , Q 3 ), and characteristics as Sauter diameter d 32 , median d 50 , and 3 Results and Discussion Fig. 2 presents results of the size distribution analysis conducted for a mixture of spherical glass beads of two different sizes. The same analysis was conducted separately for both individual bead fractions. The distributions were compared to those obtained using Coulter counting, i.e., measuring objects sizes by changes of electric resistance when passing through a channel. Volume distributions obtained by both methods exhibit similar size ranges, namely, from 5 to 20 mm and from 10 to 35 mm. The corresponding d 50 values differ by max. 15 % (10.67 ± 0.14 mm and 23.21 ± 0.14 mm from Coulter counting, 9.9 ± 0.1 mm and 23.0 ± 0.1 mm from laser diffraction, as well as 9.4 ± 1.6 mm and 19.7 ± 4.2 mm from template matching). The bimodality is clearly visible in the volume distribution of the glass bead mixture with two peaks at 9.4 mm and 17.9 mm. Note, the Coulter counting method required a 1:10 -5 dilution of of the suspension. Fig. 3a presents a transmission electron microscopy image of rod-shaped CdSe nanocrystals [11] which was screened using    template matching with an image section depicting a single nanocrystal (see inset). Red crosses show 850 detections of which only 9 are false. The detected length and width distributions of the crystals are displayed in Fig. 3b. The medians of 5.3 ± 0.2 nm in width and 10.0 ± 0.2 nm in length are in excellent agreement with the values 4.3 ± 0.5 nm and 11.8 ± 1.3 nm, respectively, stated in the image source and obtained via manual sizing. In this case, commonly used laser diffraction [12] or Coulter counting would only yield an equivalent sphere diameter strongly depending on the aspect ratio of the analyzed particles, which is only accessible from direct imaging.

Selective Size Distribution Analysis of Emulsion Droplets during Freezing
In this case, Coulter counting, diffraction or scattering experiments are not suitable because liquid and crystalline droplets cannot be distinguished, and furthermore the required dilution of the emulsion would affect the crystallization process (change in heat transfer and droplet collision frequency). However, in hexadecane/water emulsion, crystalline droplets can be distinguished from liquid droplets by their color in polarized light micrographs, as depicted in Fig. 4a. The crystalline droplets in the emulsion appear as closed black circles after blue filtration of the micrographs and were selectively detected with such a template, whereas the transparent liquid droplets match an empty circle. Red crosses in the blue filtered micrographs mark the detections and show the high selectivity of the template matching method. Fig. 4b displays the fraction of crystalline droplet counts as a function of temperature obtained from the micrographs of a hexadecane/water emulsion during cooling down from 20°C to 0°C. Fig. 4c indicates the median volume-weighted diameter d 50 of liquid droplets and crystalline droplets, selectively, during this cooling process. The median diameter at 20°C obtained from template matching is 20.6 ± 1.5 mm and agrees well with 20.5 ± 0.2 mm from laser diffraction measurements. The median diameter decreases during cooling down from 20°C to 7.6°C. The crystallization starts at 12°C in droplets which are about 10 % bigger than the mean droplet size. Droplets that stayed liquid down to 7.6°C are 10 % smaller than the average. Half of the droplets were crystallized at 8.7°C. The image analysis tool allows for an in-situ monitoring of the crystallization kinetics in the dense emulsions.

Bubble Size Distribution in Foams
In this case, the endoscopic images suffer from spatially varying brightness and contrast. The volume-weighted density distribution q 3 and cumulative share Q 3 of bubbles in protein foams after 10 s and 400 s of drainage are indicated in Fig. 5a. Insets show endoscopic images of the foam which were analyzed manually by three individuals as well as by automated template matching with the depicted template. Red crosses mark the template matches and show high precision and yield of the detection. The distributions of the cumulative volume fraction acquired using template matching agree very well to those obtained manually.
The temporal evolution of the mean Sauter diameter d 32 and the distribution width characterized by the span of the bubble size distribution are displayed in Fig. 5b with a temporal resolution of 0.1 s -1 . The mean and standard deviation were derived from the analysis of three independently acquired image sequences of freshly prepared foam. The Sauter diameter and . Red crosses mark the detection and emphasize that more bubbles were detected with a higher accuracy using template matching without preprocessing of the heterogeneously illuminated image.

Conclusion
Simple ZNCC-based template matching was applied in a software tool for high-throughput size distribution analysis of bubbles, captured with an in-situ endoscopic setup in foams at dispersed volume fractions up to 90 %. It provides fast detection of large quantities of particles, i.e., approx. 500 per minute, with high geometric selectivity without previous image processing. It was demonstrated that the results of the software tool can compete with time-and cost-consuming methods such as manual counting, Coulter counting or laser diffraction and sizes particles, droplets, and bubbles in micro-or endoscopic images with great accuracy. The method does not require dilution and can be used to distinguish between different particle types, and it allows for size detection of anisotropic particles in two dimensions.
The template matching approach is especially useful for analysis of endoscopic images with poor and heterogeneous contrast where thresholding, edge detection, and Hough transformation fails. The high time resolution of the size distribution analysis using template matching is particularly suitable for capturing fast transient phenomena in situ.