High throughput holographic imaging-in-flow for the analysis of a wide plankton size range

: We developed a Digital Holographic Microscope (DHM) working with a partial coherent source specifically adapted to perform high throughput recording of holograms of plankton organisms in-flow, in a size range of 3µm-300µm, which is of importance for this kind of applications. This wide size range is achieved with the same flow cell and with the same microscope magnification. The DHM configuration combines a high magnification with a large field of view and provides high-resolution intensity and quantitative phase images refocusing on high sample flow rate. Specific algorithms were developed to detect and extract automatically the particles and organisms present in the samples in order to build holograms of each one that are used for holographic refocusing and quantitative phase contrast imaging. Experimental results are shown and discussed.

The goal of these techniques is to combine the recording of high-resolution digital optical microscopy images with a high throughput flow rate. The main issue is linked to the very weak depth of field of the microscope lenses [6] that moreover decreases with the numerical aperture, or equivalently with the magnification. However, high magnification is mandatory to achieve enough resolution that concerns many diatoms, dinoflagellates and protozoa [7]. The ideal technology should be able to handle plankton ranging in size from 1 µm up to several hundred micrometers [8].
In order to increase the depth of field of the classical light microscopy, Sieracki et al. developed the FlowCAM instrument [9]. Thanks to a phase plate inserted between the microscope lens and the detector, it increases the depth of field by a factor 4 (for a microscope equipped with a 4X microscope lens), allowing to record images of aquatic organisms and particles in a dynamic flow and to improve the volume analysis rate. This instrument can work with microscope lenses up to magnification 20X and with specific flow cells with a depth adapted to the magnification. The use of higher magnifications would give rise to very low throughput rate and clogging problems. For the recording of all the organisms in the sample, more specific flow cell must be used and with a maximum magnification of the microscope lens limited to 10x.
Other instruments like for example those realized by V. Kachel and J. Wietzorrek [6], the FlowCytobot (IFCB) [7] or the ImageStream of Amnis Corporation [10] perform imaging-inflow with the combination of the flow cytometry and video technologies. The organisms of the sample are kept in focus with respect to the camera sensor thanks to the use of hydrodynamic focusing of the water sample in the flow cell. These technologies face to difficulties with the use of high magnifications, for accurately keeping the organisms position inside the depth of field, which is about 1µm for a typical microscope lens 40x.
The digital holographic microscopy (DHM) brings two new capabilities to optical light microcopy: it dramatically increases the depth of investigation and it can also provide quantitative phase contrast imaging [11][12][13][14][15][16][17][18]. Therefore, those capabilities are of great interest to study particles and organisms in water in flow cells or in a water volume.
The in-line DHM configurations are simple to implement and for the plankton analysis, this technology was mainly developed for submersible instruments that record holograms of an in situ water volume [19][20][21][22][23][24]. However, due to the non-complete recorded phase information, the holographic reconstruction gives rise to the twin image and to some difficulties to accurately compute the phase maps of small transparent organisms as often encountered in water application [23]. The interferometric configurations of the DHM counter those limitations. With those ones, it becomes possible to refocus objects of few micrometers sizes over large defocus distances, and to obtain quantitative phase contrast imaging.
Thanks to those capabilities, the interferometric DHM has been used to investigate phenomena as the dynamics of particle motions in split cells [25], the dynamics of soft objects in different types of flow [26], and to propose automated recognition of species [27][28][29].
An aspect of importance is related to the optical quality of the holographic reconstructions of both phase and intensity images. Indeed, high throughput measuring device need further automated processing on the delivered image with the lowest possible level of noise. As far as the optical quality is improved, any image processing is becoming more efficient. However, in many implementations of DHM, coherent laser beams are used. They generate the coherent optical noise in diffusing samples, which can be critical with water samples. A way to counter the coherent noise is to use light source of partial coherence [12,30].
In this article, we used an off-axis DHM working with partially spatial coherent light source that improves the images quality by removing the problem of the laser speckle noise and the multiple reflections generated by the flow cell walls. We already used a similar instrument to perform nanoplankton species classification [29,31]. However, in these two articles, the configurations were limiting the possible flow rates.
Therefore, we implemented a DHM configuration to perform the analysis of fresh, untreated natural water samples in a flow cell with a much higher flow rate while keeping a high magnification and a high resolution, with microscope lens 40x (NA 0.6). We obtained high quality holograms, with a depth of investigation of about 400 µm thanks to the digital holographic refocusing process. This instrument allows performing high throughput holographic imaging-in-flow of microplankton and nanoplankton ranging in size from around 3µm up to 300µm with the same flow cell.
The objective is to achieve a system able to record the hologram of a flux, to detect every object in the size range of interest, and finally to refocus independently each object. For that purposes, with respect to [29], we extended the detection process to be efficient on both phase and amplitude objects of various sizes and shapes, and we implemented a region-of-interest generator around every detected particle based on mathematical morphology processing.

Optical setup
This section briefly describes the off-axis DHM with a light source of partial spatial coherence light to record the complete holographic information on every video frame. This configuration is shown in Fig. 1. A coherent source (a mono-mode laser diode, λ = 635nm) is transformed into a partially spatial coherent source by focusing the beam, by the lens ML1, close to the plane of a rotating ground glass (GG). The lens L1 collimates the beam that is divided by a beam splitter BS1. The reflected beam, the object one, illuminates the flow cell in transmission. The middle plane of the flow cell is imaged by the couple of lenses ML3-L2 on the CCD camera sensor. The reference beam, which is transmitted by the beam splitter BS1, has a similar optical path, excepted that there is no flow cell. It interferes with the object beam on the CCD sensor. The reference beam is slanted on the sensor by an angle of about 1.3° with respect to the object beam in such a way that a grating-like thin interference pattern is recorded. This is used to implement the Fourier method to compute the complex amplitude of the object beam for every recorded frame [32]. Thanks to the partially coherent illumination, raw holograms directly displayed on the PC screen are fully meaningful for the operator as with an usual microscope. Indeed, with full coherent illumination, the direct image is usually too noisy to be interpreted by a direct viewing.
The microscope lenses ML2 and ML3 are Leica 40x, NA 0.6. By a proper choice of the focal length of L2 and thanks to the implementation of a camera JAI RM-4200 GE, with a large CCD array of 2048 x 2048, pixels with a pixel size of 7.4µm x 7.4µm, the field of view is 370 µm x 370 µm. Those magnification and sensor size allow to reach the constraint to detect objects in a wide size range while keeping a large depth of investigation and a large field of view.
To perform the digital holographic refocusing over a distance d, we use the discrete form of Kirchhoff-Fresnel propagation that is convenient for the near field reconstruction [12]. Let us consider a complex amplitude field ( ) , g s t . By propagation over a distance d, we obtain the complex amplitude ( ) where 1 , l m F + and 1 , l m F − represent respectively the direct and inverse two-dimensional discrete Fourier transformations performed on the variables l, m, 1 j = − , e is the sampling distance, n is the number of sampling points by side, s, t, U and V are integer numbers varying from 0 to n-1.
The full refocusing distance D is roughly given by 2 D L G NA = , where L is the sensor width, G is the magnification given by the ratio of the focal lengths of L2 and ML3, and NA is the numerical aperture of the microscope lens [33]. For our setup, G is 41 and the theoretical depth of investigation is about 310µm with the above formula. In practice, as demonstrated by the results hereinafter, we obtained a depth of investigation that covers the thickness of 400 µm of the flow cell.

Microfluidic setup
The flow cell is a hollow square glass capillary (VitroCom) with internal dimensions of 400µm x 400 µm, glued near its two extremities on a microscope slide. A small conic tip, where the water sample is poured, is connected to one extremity of the capillary. The other extremity of the capillary is connected via a tube to a KDS syringe pump in pull mode.
In order to reduce sedimentation inside the capillary, we placed the DHM in horizontal position to perform vertical sample flux in the capillary. The flow rate during the test was 0.8 ml/h with a time exposure of 1ms. The frame rate is tunable from 2Hz up to 15Hz. It can be adjusted, if necessary, to record the holograms of faster flux. Numerical simulations of the Poiseuille flux in a square channel of 400µm x 400µm show that the maximum speed of particles is 2860 µm/s for a flux of 0.8ml/h. Therefore, the maximum displacement of a particle during an exposure time of 1ms is about 2.86µm, which is acceptable with respect to the size range of the particles and organisms we want to detect. Note that the blurring introduced by the speed of the fastest particles during the recording concerns only few ones and the main part of the detected particles has negligible blurring effect.

Samples
We recorded sequences of holograms on different samples of natural water from a Brussels pond (March-September 2013). We performed our tests on natural water samples without any filtration. The advantage of this configuration is to avoid or to reduce the filtration steps allowing to preserve, in particular, filamentous organisms present in the water sample as e.g. some Cyanobacteria species. In order to evaluate the performances of the DHM setup on small organisms, we also performed tests on a commercial sample of Giardia lamblia cysts (Waterborne Inc.)

Results and discussion
In this section we describe the processing steps to obtain the complex amplitude information of each individual particle and organism flowing inside the capillary in the field of view (FOV) of the DHM. After those processing, the further steps consist to perform the digital holographic reconstruction in order to obtain the refocused particle images and their associated phase map.

Holograms acquisition and permanent defect correction
In the approach we followed, we acquire sequences of N holograms (typically, N = 100). The developed flow-through DHM gives excellent quality holograms of the plankton organisms of the fresh untreated pond water samples, as shown in Fig. 2(a).
In our setup, we analyze all systematically recorded images of the sequences instead of using detection triggering for the following reasons. It makes sense to use detection with a camera triggering if there are few particles in the flow in such a way that a sequence should present numerous holograms without particles. With natural surface water, the particle density is usually high enough to have particles in almost every recorded hologram. In this case, the triggering detection is less crucial as any recorded hologram has to be processed. Anyway, if requested, it is still possible to remove a posteriori the holograms without particle by postprocessing.
The recording of large sequences of holograms allows us to also implement corrections of the defects in the intensities and the phase maps in a self-consistent way for each sequence. It is a crucial step of the processing that makes possible to efficiently remove the permanent defects of the system. The defect correction uses the procedures that are described hereinafter.
Let us consider a set S of complex amplitudes field ( )  (2) This process is efficient to remove the small permanent intensity defects as shown by Fig.  2(b). Instead of performing the correction with the averaged intensity, it could be achieved with the median intensity computed for each pixel on the whole set of intensity images. In our case, the areas covered by the objects are small in comparison with the background area in such a way that the averaged intensity gives very good results with weak fluctuation. However, in case of dense samples, the correction by the median intensity is an interesting alternative.
In a second step, the correction of the phase maps   (3) The phase background is after set, in average, to a fixed phase value. The process is also efficient and is illustrated by Fig. 3(a).

Object detection
The goal of this section is to show how the presence of an object can be detected for further processing.
Several detection schemes are possible for the detection purposes. For instance, in [29], the detection is achieved by computing the phase disturbance introduced by the object. This way to proceed is efficient and demonstrated with phase objects. With surface water, it is necessary to consider a detection scheme for phase or amplitude objects with a wide range of sizes and various shapes. In order to gain processing time on the large set of holograms, it is also better to perform the object detection without having to perform a digital holographic reconstruction.
For that purpose, the availability of well-corrected sequence of complex amplitude fields allows to implement a powerful detection scheme based on a high-pass filtering of the ( ) , ck g s t . Indeed, due to the above-described corrections of the complex amplitude fields, if there is no object present in the FOV,

( )
, ck g s t is almost constant. The major contribution comes from the camera noise that can be assumed low. Therefore, the 2D discrete Fourier transformation of the complex amplitude presents a narrow peak centered on the spatial frequency component (0,0). If we assumed, in a first approximation, that the noise is white, it contributes, in the Fourier plane as a low level equally distributed random noise. Therefore a high-pass filtering process followed by an inverse discrete Fourier transformation and a threshold operation will give a zero complex amplitude distribution when there is no object in the field of view. On the contrary, if there is an object in the field of view, spatial frequency components significantly different to zero appear outside (0,0). Those ones are no more filtered by the high-pass filter and provide contributions where the object is located after the inverse Fourier transformation and the threshold operation.
To avoid border effects on the high-pass filter, we implemented the smooth filter ( ) where ( ) , u v are the discrete spatial frequencies with , 2 ,. . . , 2 1 u v n n − = − , and σ defines the strength of the high-pass filtering. In practice, the detection results are not very sensitive to σ . In our case, we selected 30 σ = . After the inverse Fourier transformation, we apply a threshold to determine what is considered as a significant detection or not. For that purpose, the intensity image is computed and converted in 255 gray levels in such a way that clearly visible objects give rise to bright regions, even with saturation, and that the background regions give intensity of few gray levels (typically less than 10). We apply a threshold level about 40 that allows to detect unfocused tiny objects. The binary detection image associated to

Regions of interest (ROI) around the detected objects
The detection is performed on the objects regardless to their positions with respect to the focused plane by the microscope. As a consequence, largely out of focus small objects will give rise extended diffraction patterns with a small inside detection region where the detection method gives a signal. The objective is to build a ROI around every detected object that has to be large enough to make possible the digital holographic refocusing. For that purpose, it is necessary to extend the detection zone obtained by the previous processing.
The extent of a diffraction zone around an out of focus object is directly related to the defocus distance and to the numerical aperture of the microscope lens. It can be shown that the diameter of the diffraction zone due to an out of focus point source can be approximated by: where ξ is the defocus distance and NA the numerical aperture of the microscope lens. This intuitive result, which can be demonstrated with Fourier Optics formalism, expresses that the diffraction zone size around each object increases as the cone defined by the numerical aperture of the lens. In our specific case, we have NA = 0.6, the maximum value of ξ is equal to 200µm. Taking into account the refractive index of the medium, which is essentially water, we obtain that the largest value max D of D is about 90µm, which is approximately a quarter of the width of the field of view.
Therefore, in order to avoid resolution lost for any defocus distance, we should enlarge every detected region by a surrounding region that should have a width of max 2 D , which correspond to 125 pixels. Due to the fact that the several particles may be simultaneously present in the field of view, such large ROI's could have too large overlaps making it difficult to process individually the particles. In order to avoid that issue, we decide to create the ROI around the detected particles that are significantly smaller by an amount of 40 pixels. Note that, if there is an observed resolution loss, it is possible to start in this way and to retrieve the full resolution, in a second step, by applying the same ROI building on refocused full image.
The processing we apply are the following: there is only the area l whose pixel values are equal to 1. Figure 3(b) shows an example of this processing. Note that, even with the reduced sizestructuring element of the morphological dilation, it cannot be guaranteed that some close particles will be not reconnected. However, in this case it can be assumed that there are only few particles and it is possible to separate them within each region of interest by usual image processing, or by analyzing the refocus distances of the different elements inside the ROI [34].

Complex amplitude fields for each particle
The target is to build individual complex amplitude fields associated to each particle to be used subsequently for digital holographic refocusing. For that purpose, each mask , ckl g s t , we can expect strong diffractive effect created by the border of the mask. Therefore, it is necessary to apply a border processing to reduce, as far as possible, such disturbances. The border processing described in reference [35] is efficient, but time consuming. As the number of holograms to be processed can be huge, the computational time could become critical. For that reason, we implemented a simplified version of the border processing that consists to perform a linear extrapolation of the complex amplitudes, outside the ROI, between pixels located on the border. This processing is applied line-by-line and column-by-column. It results that , ckl g s t can be used for digital holographic refocusing of every particle. Figure 4 shows examples of refocused particles corresponding to the images of Figs. 2 and 3. The detection sensitivity can be adjusted to the organisms and particles size range to be analyzed, by the tuning of the threshold level applied during the high-pass filtering.
With the setting we used for the experiments, we estimate that the detection process allows to detect the particles larger than 3 µm. Usually, the missed particles are debris that are weakly significant for the targets of this study. Figure 5 shows an example of a smallundetected particle.  Fig. 3(b). The reconstructions are performed on separated holograms for each particle. The individual holograms are built by a border processing applied on the complete hologram limited by the individual ROI's shown by Fig. 3(b). The letter labels are corresponding to the ones of Fig. 3(b). The labels I and P identify, respectively the intensity and phase images. The background of the phase images is set to the gray level 32. The reconstruction distances d are indicated below every couple of images corresponding to one particle. Scale bar: 20 µm. In order to show the typical image quality in quantitative phase contrast imaging mode, the phase map of Fig. 4 (e P) is shown in Fig. 6.

Refocusing capabilities
Thanks to the digital holographic refocusing capability of our DHM, all the particles and organisms flowing inside the capillary can be refocused. Indeed, digital refocusing is effective from −200 µm to + 200 µm with respect to the recorded plane with a good resolution and therefore all the capillary depth can be investigated. In comparison, the classical depth of field of the used microscope lens 40X is about 1.4 µm. An example of a refocused plankton organism by a distance of 200µm is shown in Fig. 7. We note that the proposed processing is also efficient to detect small and large particles and to build useable ROI's.
The implementation of the microscope lenses 40x allows to record holograms of small organisms with a good resolution. To assess the performance of our DHM setup on small organisms, we performed tests with a commercial sample of Giardia lamblia cysts that have a typical length of 6µm. Experimentally, the obtained phase and intensity images have good resolution, as shown by Fig. 8, to apply the microorganisms classification method described in [29,31]. An example of detection and refocusing of a large plankton organism is shown by Fig. 9.

Capillary lateral borders disturbance
The Field of view covers almost all the capillary width. Due to the diffraction of the two lateral capillary walls, the particles that are very close to the capillary lateral walls are difficult to process. As described above, we implemented for the hologram processing a common region of interest and the effective width of the processed holograms covers about 82% of the capillary width.
If it is requested to cover the full width of the sample flux, a small hydrofocusing effect in the flow cell can be implemented in order to keep the particles a little away from the two lateral walls.

Conclusion and perspectives
We combined in our DHM working with a partially coherent illumination, high magnification objective lenses (40x, NA = 0.6) and a large CCD sensor. This allows to perform holographic imaging-in-flow of a wide range of planktonic organism sizes (3µm-300µm) with the same flow cell. Thanks to the large sensor size, the digital refocusing distance is large enough to refocus organisms in the full depth of the flow cell.
The processing of the hologram involves the removal of the defects of the experimental cell for both phase and intensity images. It results outstanding phase and intensity image quality. The phase and intensity corrections make possible to implement a fast detection scheme of the objects based on a high-pass filtering process. Simple morphological operations applied on the high-pass filtered images are used to build around every detected particle a region of interest that is exploited to make independent holograms for each particle. Those holograms are used to refocus each particle.
In our experiments, we performed flow rates around 0.8 ml/h, with a time exposure of 1ms. With the possible improvement of the light source and the implementation of a camera with a higher frame rate and a shorter exposure time (typically 0.1ms), we evaluate that the flow rate could be increased up 8 ml/h. Moreover, the microscope lens magnifications can be changed and adapted to other size range of plankton. For example, with the same setup equipped with microscope objectives 20X (NA = 0.4), we can use a 800 µm x 800 µm micro-channel section and analyze organisms in a size range around 10µm-600µm with an estimated flow rate of 64 ml/h. The implementation of microscope objectives 10X (NA = 0.3) will allow to use a 1.6 mm x 1.6 mm micro-channel section and to analyze organisms in a size range around 20µm-1.2 mm, with an evaluated flow rate of around 512 ml/h. For comparison, the FlowCytobot [7] working with a 10x microscope lens (NA = 0.2) obtained a flow rate of 15ml/h and without quantitative phase contrast imaging. The use of larger CCD sensors will improve even more our DHM performances.
The phase information quality obtained by our DHM can be used to perform organism classification as described in [29,31]. In these articles, we showed that very similar-looking nanoplankton organisms can be discriminated thanks to the phase information.