Multiple contrast metrics from the measurements of a digital confocal microscope

We describe various methods to process the data collected with a digital confocal microscope (DCM) in order to get more information than what we could get from a conventional confocal system. Different metrics can be extracted from the data collected with the DCM in order to produce images that reveal different features of the sample. The integrated phase of the scattered field allows for the three-dimensional reconstruction of the refractive index distribution. In a similar way, the integration of the field intensity yields the absorption coefficient distribution. The deflection of the digitally reconstructed focus reveals the sample-induced aberrations and the RMS width of the focus gives an indication on the local scattering coefficient. Finally, in addition to the conventional confocal metric, which consists in integrating the intensity within the pinhole, the DCM allows for the measurement of the phase within the pinhole. This metrics is close to the whole-field integrated phase and thus gives a qualitative image of the refractive index distribution.


Introduction
The original idea of the confocal microscope, proposed by M. Minsky in 1957 [1], included both the transmission and reflection geometries for the imaging of scattering samples, in which the scattering itself was the contrast agent. Since that time, the confocal microscope has been extensively analyzed in the literature [2,3]. Nowadays, the confocal microscope is mostly used in reflection and primarily with one or two-photon fluorescence. This is especially true in biological imaging mainly because the fluorophores can be functionalized to specific biological targets and because the signal from the fluorescence can be easily separated from the excitation light. In general, the backscattered light is weaker than the fluorescence, which is omnidirectional. The transmission geometry [4] is only rarely used for the practical reason that the sample is caught between two objectives, the confocality of which has to be carefully adjusted. People have proposed ideas to improve the transmission mode [5][6][7] but these did not change the basic trend. Recently, we introduced digital confocal microscopy (DCM), which is a scanning holographic technique that can be used both in reflection and in transmission, greatly simplifying the implementation in the latter case [8]. The principle is that a digital hologram of the scattered light is made as the sample is illuminated with a focused spot. The hologram is subsequently processed digitally. We have shown that the dynamic placement of the pinhole in the digital domain can dramatically improve the performance particularly in the transmission geometry. In this paper we focus on another promising feature of DCM: its ability to provide a variety of metrics to obtain contrast that can give us additional information about the object.
The schematic diagram of the DCM in the transmission geometry is shown in Fig. 1(a). The light is focused into the sample by lens MO1 and the object, which is mounted on a translation stage, is scanned in three dimensions. The transmitted and the forward scattered field are collected by lens MO2 the back focal plane of which is imaged by lenses L3 and L4 onto a CCD camera where an off-axis digital hologram is recorded. The extracted optical field is digitally propagated (Fourier-transformed) to focus again in the digital domain. In order to simulate the conventional confocal microscope, we place a digital pinhole at the image position of the illuminating focal point. The image of the object is then formed by assigning a value to each pixel The transmitted light is collimated by lens MO2 (same lens than MO1) the back focal plane of which is imaged by a 4f system (lenses L3 and L4) onto a CCD camera. A reference beam is combined before the camera in order to record an off-axis digital hologram. The optical path of the reference can be adjusted with the delay line (DL). It is possible to reject parasitic reflections from the objective by using a short coherence length laser and adjust the delay line so that only the reflection from the focus interferes with the reference. (b) Modified apparatus to improve the stability for phase imaging. The signal and reference beams pass through the same optical components in order to reduce mechanical variations that would change the relative optical paths. (c) Closer look at the sample in the optical system described in (b). The reference beam is shown with a dashed line and the signal beam probing the sample with a solid line.
proportional to the integrated intensity passing through the digital pinhole.
In our experiment, we scan by moving the sample in order to simplify the implementation of the optical apparatus. It is also possible to scan the beam as in most commercial systems, which allows for faster frame rate. The consequence of scanning the beam in the DCM is that the spatial frequency of the interference fringes of the off-axis hologram will change according to the angle between the signal and the reference. It is possible to compensate digitally for this angle, which corresponds to a shift in the frequency domain. The scanning angle is bounded on the upper side by the spatial resolution of the detector and, on the lower side, by the minimal frequency of the fringes that is required to extract the full bandwidth of the signal. Alternatively, a scanned-beam microscope can be implemented with the modified apparatus shown in Fig. 1(b), where the reference and signal beams pass through the same optical elements. The two beams can be scanned together while the angle between them remains constant. The spectrum of the detected field remains thus fixed in the Fourier domain.
In a confocal image, we would normally interpret dark regions as places in the cell with large absorption, reflectivity or structured scattering. It is not possible to distinguish which is which and, in fact, optical microscopy in general does not give a quantitative measurement of a particular optical property of the object. Things are even more complex for the confocal microscope. If a point on the image is dark then it is because the light was absorbed, scattered, or reflected at the point of focus, but it could also be due to the fact that the beam becomes aberrated (deflected, defocused) as it propagates through the object. Such aberrations can change the shape and the position of the digital focus, which, in turn, can cause the amount of light that goes through the pinhole to change. This introduces additional ambiguity since the brightness of the pixel depends not only on local information in the vicinity of the focused spot.
In this paper we discuss approaches to resolve some of these ambiguities with the flexibility afforded by the DCM. Specifically, we demonstrate how, by extracting different metrics from the same data recorded in the DCM, we can form images that all have the basic appearance of the cell structure but which differ from the classic confocal image. We offer an interpretation for the differences between the images obtained with the different metrics, which provide qualitative information about the 3D properties of the object beyond what is possible with the conventional confocal microscope. As shown below, one of the metrics (the integrated phase) yields a quantitative estimate of the refractive index.

Digital data processing
The field in the back focal plane of the imaging lens is relayed onto a CCD camera on which the digital hologram is actually recorded. The complex optical field is extracted from a digital off-axis hologram using well-known holography techniques [9]. The extracted field thus corresponds to the Fourier transform of the focused probe beam as it appears when it exits the sample. A fast Fourier transform (FFT) is applied to the extracted field in order to calculate the field in the focal plane. The optical field is then calculated in the 3D space around the focus point. The field is propagated from the focal plane over some distance along the optical axis (z coordinate) in the +z and −z direction using a standard Fourier beam propagation method [10], as illustrated in Fig. 2(a). In Figs. 2(b) and 2(c), we show the reconstructed field amplitude near the focus for an ideal beam (unaffected by scattering) and an experimentally measured beam respectively. Note that the physical values of x, y and z in the Fourier-transformed space depend on the focal length of the virtual tube lens of the digital microscope. Because this length is arbitrary, we simply consider that the imaging system has a magnification of one.
The position z opt of the effective focus along the z axis, is where the 2D (x − y) correlation function between the ideal focus and the experimental beam reaches its maximum: where u i is the focus spot at the waist of the ideal probe beam, u m is the measured field and denotes the correlation. In Fig. 2(d), we compare the c(z) function for an ideal beam and a scattered beam. The shift in z, ∆z, is defined by the distance between the maxima of these functions and is equal to z opt by definition. The shift in x and y, ∆x and ∆y, are defined by the position of the center of mass of the experimental focus in the plane z = z opt : We define the width of the experimentally measured focus as: where the integral runs over the entire image plane. For an ideal Gaussian beam, w is equal to the 1/e-radius (in amplitude). The confocal signal through the dynamic pinhole is defined as: p(x, y) = 1, if x 2 + y 2 < a 2 /4 and 0 otherwise,  where p(x, y) is the pinhole aperture function and a is the pinhole diameter. Finally, the phase of the confocal signal (confocal phase) can also be measured. For this, we average the phase of the detected signal within the pinhole:

Results
Figure 3(b) shows the image obtained by plotting the x position of the focused spot with respect to the center of the nominal pinhole (i.e. the pinhole centered on an ideal beam not affected by scattering). Comparing Fig. 3 Fig. 3(a), we can clearly see that the basic shape of the cell is reproduced. Similarly, the y-image obtained by plotting the y position and shown in Fig. 3(c) forms a recognizable shape of the cell. In Fig. 3(d) we show the image corresponding to ∆z. The image in Fig. 3(e) is the d ⊥ -image obtained by plotting the Euclidian distance d ⊥ = ∆x 2 + ∆y 2 . Finally, Fig. 3 We get an insight by comparing the confocal image ( Fig. 3(a)) with the d ⊥ -image. At first glance they are contrast-inverted versions of each other. This can be explained as follows. In confocal microscopy, a region of the image is dark if the sample absorbs a lot, reflects or scatters light. In the cells we are imaging in this experiment, there is relatively little absorption or reflection. Therefore, dark areas are those that have some high spatial frequency content and scatter light. These same areas then are likely to have index gradients that steer the beam away from the detection pinhole and therefore a dark signal is plotted. large deflection from the pinhole translates into a bright pixel on the reconstructed image. On the other hand, a clear sample simply leaves the beam unaffected to pass through the digital pinhole. A strong (bright) signal is recorded on the confocal image and a dark signal in the d ⊥ -image. This is generally true but other things may affect the system and, therefore, it is only approximately true that the d ⊥ -image is the contrast-inverted image of the confocal image. The differences may contain interesting information. In order to explore this possibility, we plotted, in Fig. 4(a), the pixel values of the corresponding points in the d ⊥ versus the confocal image.
In such a plot if the two images were linear contrast inverted versions of each other then the relationship would be:d wheref DC is the pixel value for the dynamic digital confocal normalized over its maximum andd ⊥ the pixel value for the transverse distance normalized over the ideal beam waist w 0 . A linear regression on the data of Fig. 4(a) is used to fit the model expressed by Eq. (7). We get α = −2.13 and β = −α as the line is forced through the point (f DC = 1,d ⊥ = 0). The color-coded image shown in Fig. 4(d) shows the samples from the scatter diagram of Fig. 4(a) plotted back in the x − y coordinates of the image. The scatter diagram has been divided into three groups: The group above the regression line, in red, contains the pixels with a large beam deflection but only a small loss in the dynamic confocal signal. This suggest that, in these locations, the probe beam was only weakly scattered close to the focus but still deflected by larger scale structures. The group below the line, in blue, corresponds to the locations where stronger scattering occurred close to the focus, thus leading to a weaker dynamic confocal signal. The weakly affected pixels (plotted in black in Fig. 4(a)), most of which are located outside the cell, are defined as the pixel for whichf DC > 0.93. In order to display the segregation of the data in a more continuous way, the data can be projected into a coordinate system (U,V ) associated to the regression line. The variable U is the distance along the line, from the point (f DC = 1,d ⊥ = 0) and the variable V is the signed distance from the line (see Fig. 4(a)). Images in the U and V metrics are shown in Figs. 4(b) and 4(c) respectively. The U-image represents the amount of scattering, i.e. the extent to which the beam has been altered through the sample. The V -image gives information about the nature of the scattering, i.e. whether it consists in large scale deflection of the beam (e.g. close to the edge of the cell) or in smaller scale scattering close to the focus.
In Fig. 5, we have also plotted the focus width versus the dynamic confocal signal. It is clear that there these two measures are strongly anti-correlated. This is because the confocal signal naturally decreases as the beam gets wider. For an ideal Gaussian beam of normalized waist, the confocal signal is equal to:f whereŵ = w/w 0 and the diameter of the pinhole has been taken equal to w 0 . The deviation from the model above is related to aberrations in the beam, other than tilt and defocus. Dynamic Confocal Signal (normalized), f DC w/w 0F ig. 5. Width of the beam at the focus (1/e-radius of the beam at z = z opt ) normalized to the minimum width w 0 plotted as a function of the dynamic confocal signal normalized to its maximum. There is a clear and expected correlation between the width and the confocal signal. The solid curve represents the expected confocal signal for an ideal Gaussian beam according to Eq. (7). Any deviation from this model is related to scattering in the sample.

Tomographic interpretation of the data
An important feature of the digital confocal imaging system is that the data recorded is in complex form. We therefore have access to the phase and it is possible to use it to extract information about the object. It is possible to form images by unwrapping and integrating the phase, φ (x, y), of the detected 2D field for each recorded hologram: A 3D image can then be plotted with the values of each voxel corresponding to this integrated phase Φ int , exactly as we do for the digital confocal images we already discussed with the integrated phase being a different metric. The main difference with conventional optical phase tomography [11][12][13][14][15][16] is that the measurement is done with a focused beam scanned in three dimensions rather than with a collimated beam scanned in angle. Dillon and Fainman [17] proposed this concept in 2010 and presented experimental results with an absorptive object (fiber and absorbing stripes on a reflecting surface) [18]. In the DCM, as not only the intensity but also the phase is recorded, we can quantitatively reconstruct the refractive index distribution. We consider that the light rays propagate straight through the sample and accumulate a phase shift and attenuation corresponding to properties of the regions they went through. The region close to the focus is common to all rays whereas regions farther away affect only a restricted subset of the rays, as illustrated in Fig. 6. We reject the contribution of the out-of-focus regions by integrating (or averaging) the phase of the complex field in the detection plane. On average, the contribution from a specific region is reduced by a factor 1/r 2 where r is the distance from the focus. Integration of the amplitude gives the absorption at the focus and the integration of the phase gives the refractive index at the focus. 6. Principle of optical tomography with a focused beam. The phase delay and the absorption at point (x 0 , z 0 ) are common to all rays in the probe beam. Conversely, regions away from (x 0 , z 0 ) will modulate some of the rays but not all of them and the averaged contributions from these regions will be weak. The focused beam naturally performs a back-projection to the point of interest (x 0 , z 0 ). The data can be filtered later on with a classical tomography high-pass ramp filter.
The fact that the integration of the phase can actually produce an image and corresponds to a tomographic measurement can be formalized in mathematical terms. This is done by introducing the X-ray transform (denoted by the operator X ), also known as John transform. It consists in an integral transform similar to the Radon transform [19]. The quantity we want to measure in the sample is described by the function f (x, y, z), which corresponds to the absorption coefficient in optical absorption tomography and to the refractive index in the case of optical phase tomography. The X-ray transform is defined as the integral of f (x, y, z) along rays that all pass through point x = (x, y, z) where the probe beam is focused: where Θ ∈ S 2 is the direction vector of the ray. The signal we detect on the CCD camera in our experimental system is, in fact, equal to X ( f ). The x and y components of the 3D vector Θ have on one-to-one correspondence to the two spatial coordinates on the CCD. In order to obtain an image of the sample from the measurements we need to invert Eq. (10). The Xray transform can be inverted by following a two-step procedure. The first step consists in integrating X ( f )(x, Θ) over all the directions Θ, on the half sphere S 2 . From this integration, we get the function g(x): This can be rewritten from spherical coordinates (r, θ , φ ) as an integration in Cartesian coordinates x = (x, y, z), with r = |x|: The second step in the inversion of the X-ray transform is to invert the above equation in order to express f (x, y, z) as a function of g(x, y, z). It can be shown that this can be done by applying a ramp filter in the Fourier domain [20]: where κ is the spatial frequency and F denotes the Fourier transform. Because each direction Θ in the DCM corresponds to a pixel at some position (x, y) in the detector plane, the integrated phase Φ int corresponds to function g(x) defined by Eq. (13). Using Eq. (14), the refractive index can simply be calculated from Φ int as: This filtering operation corresponds to the classical tomography high-pass ramp filter. As opposed to phase tomography with parallel ray illumination, there is no need to project the data in the Radon space or to perform a filtered back-projection. Intuitively speaking, the backprojection is naturally performed by the focused probe beam itself. The two-step inversion procedure of the X-ray transform is exact provided that the straight ray assumption is fulfilled and that the illumination light cones covers the whole sphere around the sample (2π solid angle on the illumination side and 2π on the detection side). The limited numerical aperture in our experimental system limits the resolution in a manner similar as what would happen for confocal imaging.
The absolute phase of the scattered field is necessary to perform the tomographic reconstruction. It may happen that the phase shift undergone by some part of the field exceeds 2π. In this case, the detected two-dimensional phase function has to be unwrapped. Two-dimensional unwrapping is a difficult and time-consuming process that has to be carried out for each pixel, which is not practical in most experimental conditions. Instead, the phase integration metric can be approximated by the phase at the center the digital confocal pinhole. The principle of using the phase in confocal microscopy has been proposed previously in [21] and [22]. The field in the center of the confocal pinhole u c is given by the Fourier transform of the measured field u = exp[ jφ (x, y)], the amplitude of which we consider as approximately constant. We have: where φ is the phase of u. The second equation holds because we consider the value of the field in the center of the pinhole, where k x = k y = 0. If the phase modulation is weak, i.e. φ (x, y) 1, we can write u ≈ 1 + jφ (x, y) and we obtain: The phase in the center of the pinhole, i.e. the confocal phase, is thus approximately given by Φ int = φ (x, y)dxdy. Images of the confocal phase are shown in Fig. 7(d), where they are compared to the dynamic confocal images ( Fig. 7(a)), the ∆z-images (Fig. 7(b)) and the d ⊥ -images (Fig. 7(c)). The confocal phase images were obtained by first calculating the complex optical field at the position of the virtual pinhole and then by measuring the phase of the pixels that falls within the pinhole, as defined by Eq. (6). In our experiment the pinhole was chosen to have the "optimum" diameter equal to one diffraction-limited spot.
The confocal phase is also subject to phase wrapping which lead to artifacts in the image. This has been solved through the use of a two-dimensional phase-unwrapping algorithm on the final image [23]. The main advantage of the confocal phase metric is that the phase unwrapping can be applied once on the final image, whereas in the integrated phase metric, a two-dimensional phase has to be unwrapped for each pixel.
The confocal phase images also display vertical streaks, which are artifacts due to fluctuations in the relative phase between the reference and the signal beam. In the confocal geometry, all the rays that contribute to the signal beam pass through the focus in the sample. Therefore, it is not possible to discriminate between a variation in the signal and a simple offset in the reference phase due, for example, to mechanical vibrations. The quality of the phase image therefore critically depends on the stability of the optical apparatus. In order to enhance the stability for the acquisition of phase images, the digital confocal microscope was modified as described in Fig. 1(b). In the modified configuration, the reference and signal beams pass through the same optical components thus reducing the relative optical path variations between them. In Fig. 7(d), the vertical streaks are a consequence of the remaining phase instability.

Conclusion
With the digital confocal microscope, for each scanning position, we can capture the entire optical field falling within the numerical aperture of the imaging objective. This represents a lot more information than the signal from a conventional confocal microscope in which the out-of-focus contribution is lost. As we have shown, multiple metrics can be obtained from the DCM data in order to produce images of the sample. Interestingly, all the metrics proposed, i.e. the confocal, the dynamic confocal (amplitude and phase), the focus deviations ∆x, ∆y, ∆z, d ⊥ and d 3D , the focus width and the integration of the phase, yield a recognizable image of the cell with different features emphasized. In particular, the integrated phase represents approximately the local refractive index. A quantitative value can be calculated by applying a classical tomography high-pass ramp filter on the three-dimensional data. In a similar way, the local absorption coefficient can be derived from the integrated intensity. The confocal phase (i.e. the phase within the dynamic pinhole) is closely related to the integrated phase and yields similar images. The reason for this similar appearance is that the integration of the phase acts as an averaging process. The averaging itself is a low-pass filter operation similar to the effect of the pinhole in which the confocal phase is measured.
The position of the image of the focus can be measured in three dimensions. This can be done either by calculating the tilt and defocus Zernike factors on the scattered field or by calculating the three-dimensional field as we have shown above (Fig. 2). The deflection of the focus away from its ideal position is a measure of the overall sample-induced aberration. The width of the focus defined by Eq. (3) is a measure of the scattering properties of the sample in a similar way as the dynamic confocal signal, which is strongly correlated to it (Fig. 5). In Fig. 4, we compare the confocal signal (or equivalently the inverse of the focus width) and the deflection of the focus. This gives an indication on the amount of scattering that occurs close to the focus as opposed to more global refraction effects. The different metrics and the physical properties that they reveal are summarized in Table 1.
All these metrics show some sectioning capability as illustrated in Fig. 7, though some seem to have a better discrimination such as the dynamic confocal. The deflection of the focus (quantified by ∆x, ∆y and ∆z), and the dynamic confocal (or the focus width to which it is well correlated) characterize two distinct types of aberrations undergone by the probe beam as it propagates through the sample. The reduction of the confocal signal without major deflection of the beam is most likely associated with scattering close to the focus. As such in non-absorbing media it is a measure of how scattering the sample is locally. Conversely, large deflections of the beam with minor loss in confocal signal suggest that the beam underwent only weak scattering but was refracted at some interface. This is corroborated by the correspondence between particular zones of the cell image shown in Fig. 4(d) and the different regions of the scatter diagram shown in Fig. 4(a). It is likely that the integrated phase, which corresponds to a tomographic back-projection, provides the most reliable images especially regarding the refractive index distribution within the cell. The refractive index distribution can be quantified with this method. There is a very clear distinction between the cytoplasm and nucleus, which is known to have a slightly larger refractive index. Although the confocal images are more difficult to interpret than the tomographic images, they are affected by a variety of spatial structures like irregularities in the cell membrane, which are known to exist for such type of epithelial cell. On the other hand, tomography may be less sensitive to surface ripples as it relies on the line integral along rays which is not much affected by thin surface ripples that do not yield a substantial phase deviation on the transmitted ray.