Imaging of objects through a thin scattering layer using a spectrally and spatially separated reference

Incoherently illuminated or luminescent objects give rise to a low-contrast speckle-like pattern when observed through a thin diffusive medium, as such a medium effectively convolves their shape with a speckle-like point spread function (PSF). This point spread function can be extracted in the presence of a reference object of known shape. Here it is shown that reference objects that are both spatially and spectrally separated from the object of interest can be used to obtain an approximation of the point spread function. The crucial observation, corroborated by analytical calculations, is that the spectrally shifted point spread function is strongly correlated to a spatially scaled one. With the approximate point spread function thus obtained, the speckle-like pattern is deconvolved to produce a clear and sharp image of the object on a speckle-like background of low intensity.


Introduction
Imaging through scattering media [1][2][3] is a challenge as the wavefront of the incident light is distorted by scattering due to the inhomogeneous distribution of the refractive index of the media. Several methods have been put forward to compensate for the wavefront distortion and facilitate imaging. Wavefront shaping [4][5][6] and optical phase conjugation [7][8][9][10] are widely used to compensate the heavy distortion caused by multiple scattering. Transmission matrix methods [11][12][13] are applied to obtain an image through scattering media. Speckle correlation [14] appears to be a very promising method relying on the so-called memory effect (ME), which is the angular tilt-invariant property of the speckle pattern when the incident light is rotated with a small angle [15][16][17]. Within the small ME range, the speckle pattern can be considered as the convolution of the point spread function (PSF) with the object's response. Methods based on speckle autocorrelation retrieval [18][19][20][21][22][23][24][25] can be exploited to reconstruct objects non-invasively. Coherent speckle-correlation [26,27] methods provide field-based information through thick scattering media. Deconvolution methods [28][29][30][31][32] are also exploited to demonstrate fast, 3D, high-resolution and large field-of-view imaging. Among them, color imaging [28] is realized when R, G, B wavelength components of the object are retrieved from their corresponding PSFs captured by a color camera. Singleshot multi-spectral imaging has also been demonstrated [31] with a monochromatic camera. It was found that an image of an object can be reconstructed by deconvolution with the PSF obtained using different illumination, but only if the illumination spectra overlap. To realize multi-spectral imaging in Ref. [31] several spectral PSFs were recorded in advance, and PSFs at different wavelengths were assumed to be uncorrelated. Another method using a scattering medium and a microlens array with the measurement of the spectral intensity transmission is found to be capable of realizing hyperspectral imaging [32]. We have recently reported a deconvolution method to extend the depth of field (DOF) by scaling the reference PSF [33] according to the focal length of the imaging system [34]. Several other recent imaging techniques [35][36][37] have shown retrieval of depth information or even full 3D images using various methods to obtain the point spread function.
In this paper, we introduce a method to retrieve the PSF from the transmission pattern of a reference object that emits light at a different wavelength. The crucial ingredient is the presence of a strong correlation between the PSF with shifted wavelength, and the PSF with scaled spatial coordinates. The wavelength scaling can be combined with the spatial scaling methods of [34] to image incoherently illuminated or luminescent objects through a thin scattering layer, using a reference object that can be extended in size, have a non-overlapping spectrum and be in a different depth plane than the object being imaged. The PSF can be retrieved from a known, physically extended object, which is more robust and provides a higher SNR than the use of a point object in the case of incoherent illumination. We demonstrate the use of wavelength and depth scaling factors for imaging such separated objects. Furthermore, we make use of a color CCD camera to acquire two spectrally separated objects simultaneously and demonstrate single-shot reconstruction imaging. This method is practical and promising for real time imaging through dynamic scattering media. The principle we demonstrate here may be used to enable fluorescence imaging through turbid layers using reference objects emitting at a different wavelength. The experimental setup to demonstrate image reconstruction using depth correlation and PSF scaling is shown in Fig. 1. Light is emitted by a light emitting diode (LED) source (Daheng Optics,1W). The light is passed through two diffusers (diffuser 1, Newport 10-degree light shaping diffuser; diffuser 2, 5-degree) and the object to image is placed in between them. We make use of the a priori information of a reference object of a known shape for reconstructing the unknown object. The speckle pattern formed with spatially incoherent illumination has a lower contrast than that formed with a coherent source. However, such speckle patterns can be treated as the intensity sum of a series of coherent patterns from many coherence areas [38]. Here, different from our previous demonstration of depth-resolved imaging [34], we consider a more realistic situation in which the incident light is scattered by an additional layer. In this case, the exposure time of the CCD for obtaining the PSF from a point source reference has to be very large, which leads to a low signal to noise ratio (SNR) and is impractical in the case of dynamic scattering media. The use of an extended reference object generates a much brighter speckle pattern under the same illumination condition and permits the use of short exposure times. As shown in Fig. 2

Depth correlation and PSF scaling
Note that the speckle-like PSF can only be used to reconstruct unknown objects that are in the same plane with the reference object. If the unknown object [ Fig. 2 where m z is the scaling factor [34]. Note that m z multiplies the coordinates of the PSF, so ' PSF is obtained by a coordinate stretch of 1/m z . The unknown object is reconstructed by a second deconvolution, In our experiment, the distance d o, from the reference object to diffuser 2 is 117 mm and that from the unknown object plane to diffuser 2, d o '= 107 mm. The CCD is at d i = 22 mm from diffuser 2, leading to 1/m z = 1.015. In Fig. 2 Fig. 3. Thus, once a speckle pattern of reference object is obtained, other objects at different depth can be reconstructed by adjusting the scale of the reference PSF.

Spectral correlation and PSF scaling
In most previous work based on deconvolution [28-30], the PSF and the speckle pattern of the unknown object are obtained under the same illumination source, which means that the spectrum of two speckle patterns is the same. Recently, it was reported that even though the PSF and the speckle pattern are obtained with two different broad band sources, deconvolution still works provided there is some spectral overlap, which was referred to as the cross-talk effect [31]. However, when using spectrally separated narrow-band light sources, objects cannot be reconstructed with the PSF corresponding to a different wavelength. Here we show that even in the spectrally separated case a reconstruction is possible. In this case the spatial coordinates of the reference PSF are scaled to correspond to a peak of the correlation function, 12

12
( / , ) ( , ) ( , ) An analytical calculation of this correlation function is presented in the Appendix. In Fig. 4(a) we show the shape of the correlation function for different wavelength ratios, for parameters that form a good approximation to our experimental conditions. It is seen that for the wavelength shifts of order a few percent that occur in the experiment there is a strong peak in the correlation function. In Fig. 4(b) we show the position of the correlation peak versus the wavelength ratio, for conditions that approach our experimental conditions, and for two different values of the correlation length of the diffuser. For a correlation length of 12.5 µm, which corresponds approximately to our 5-degree diffuser, we find the correlation peak occurs at    Since m scales the coordinates of PSF λ1 , the approximation to PSF λ2 is obtained by a coordinate stretch of PSF λ1 by 1/ m. Therefore, an object with separated spectrum can be reconstructed with the corrected spectral PSF, The experimental setup to demonstrate imaging using spectral correlation and PSF scaling is shown in Fig. 5. Two objects (cutout letters "H" and "G") are illuminated by two LEDs (Daheng Optics, 1W) with different spectral profile [ Fig. 6(a)]. The distance from the objects to diffuser d o1 = d o2 =180 mm and that from the diffuser to the chip of the CCD (d i ) is 58.5 mm. Speckle patterns from these two objects are acquired by the CCD independently and the object is reconstructed by the deconvolution method with the PSF retrieved from the reference speckle pattern. In the second experiment, we introduced two narrow band filters before the LEDs to make sure the spectra of the light sources do not overlap [ Fig. 6(e)]. The central wavelengths of the light sources with narrow band filters are 596nm and 620nm, respectively. Thus, the correlation maximum is expected for our experimental conditions at 1/m = 1.02. The speckle patterns are obtained by inserting the corresponding narrow band filter before the CCD [ Fig. 6(f) and (g)].
If the deconvolution method is used without scaling the PSF, only a blurry image is reconstructed [ Fig.  6(h)]. However, upon scaling the retrieved PSF, a clear image is reconstructed [ Fig. 6(i)]. This confirms the strong correlation between the spectrally shifted and the coordinate-scaled PSF enables us to obtain a clear image of the object.

Spectral speckle pattern separation and single-shot imaging
In Sec. 2, the depth dependent PSF is obtained by scaling the PSF according to the reference depth whereas in Sec. 3, the wavelength dependence of the PSF scaling is depicted. When both depth and spectral differences are considered, the total scaling factor is given by The unknown object "G" in Fig. 5 is moved 20 mm towards the diffuser and the two objects are now at different depths and illuminated by spectrum-separated sources with filters. In this case, 1/m z is 1.03 according to Eq. 3. We use two corresponding narrow band filters to separate the speckle patterns of the reference object and unknown object, as in Sec. 3. The reference PSF is retrieved by deconvolution from the reference speckle pattern. The PSF corresponding to the object plane and wavelength is obtained by the appropriate coordinate stretch.
A very convenient method to obtain reference and object speckle patterns in a real time single-shot measurement is to use a color CCD. Thus the monochromatic CCD in Fig. 5 is replaced by a color one (acA2040-90uc -Basler ace). There is a Bayer filter [40] inside the camera in front of the photosensitive chip, which codes the wavelength response of each pixel. Using the spectral response curve of the CCD, different wavelength speckle patterns can be separated from one mixed speckle pattern. Suppose that there are two illuminating lights with different wavelength and their intensities are I λ1 and I λ2 . For any pixel of the color CCD, R, G and B components of the captured speckle image [ Fig. 7(a)] can be written as, where R 1 , G 1 and B 1 are the response coefficients of the CCD filter to the wavelength λ 1 and R 2 , G 2 and B 2 are the counterparts to the wavelength λ 2 . Note that there are three independent equations and only two unknowns in Eq. 8, so I λ1 and I λ2 can be found using pseudo-inverse methods. For our orange-red wavelength range simply using the first two equations suffices as the response of the B coded pixels is very low. Once I λ1 and I λ2 are obtained, a reference PSF is retrieved from I λ1 (Eq. 1), and the PSF used to deconvolve the unknown object is achieved by rescaling the reference PSF with the total scaling factor m total . The spectral sensitivity of the coded pixels is obtained from the sensor manufacturer. The center wavelengths of the filtered light sources are 1   596 nm and 2   620 nm and the corresponding (normalized) coefficients are R 1 =0.67, G 1 =0.29, B 1 =0.04, R 2 =0.88, G 2 = 0.08, and B 2 =0.04. The speckle patterns are calculated according to Eq. 8 and shown in Fig. 7(b) and (c). The reconstructed result is shown in Fig. 7(e). In order to test the scaling behavior, another object letter "F" [Fig. 7(d)] is used as unknown object and is reconstructed [ Fig. 7(f)] from a different mixed speckle pattern.
We now turn to the effect of the spectral width of the sources on the recovery of the PSF. The sensor response for a spectrally broadened source is: are the average sensitivity coefficients of the CCD pixels weighted with the spectrum of the respective light sources. We measured the spectral response using the filtered LEDs. The R, G, B components of the acquired image can be obtained by adding all the corresponding pixel intensities, respectively. The ratio of the R, G, B components represents the sensor response to the light source and hence the spectrally averaged sensitivity coefficients. We find

Discussion
We have considered two scaling factors to resize the reference PSF for different depths and different wavelengths, and presented a single-shot speckle imaging method. The single-shot method is fast, robust and practical since the reference object can exist simultaneously with the unknown object. In the previous deconvolution-based works, the PSFs are formed by a point source and the unknown object is removed for capturing the PSF. Therefore, those techniques are not suitable to be used in dynamic scattering media as the PSF loses its correlation in time quickly.
We now discuss the field of view (FOV) and depth of field (DOF) attainable with our method. The FOV is limited by the optical memory effect, which is about /  o dL [15][16][17], where L is the thickness of the scattering medium. For a holographic diffuser L is effectively very small, on the order of a wavelength. The FOV can be enlarged by scanning methods [6] or by using an axial lens system to collect speckle patterns [28]. For large differences in depth and wavelength, the peak in the scaled correlation coefficient becomes much smaller than 1, as shown in Fig. 4(a), and as a result the deconvolution becomes imperfect. Unlike autocorrelation-based speckle scanning methods [18], our method is invasive requiring the presence of a reference object. However, it is possible to combine these techniques to realize noninvasive depth and spectral imaging. One way we can think of is to use the speckle-autocorrelation method to reconstruct one of the unknown objects. The reconstructed object can then be used as a reference object and the objects of interest can be retrieved by deconvolution with the modified PSF. In this way the iterative phase retrieval algorithm [41], which requires a threshold SNR to converge [19], is needed only once and subsequent images can be recovered more easily.

Conclusion
We have shown that objects can be imaged through a thin diffusing layer using a reference PSF, which can be extracted form a known object that is located in a different plane and/or emits at a different wavelength, than the object of interest. The key finding is that for a thin diffuser the wavelength-shifted PSF correlates strongly with a coordinate-scaled PSF. Analytical calculation yields the shape of the correlation function and the peak position, which depend on the correlation length of the scattering medium and on the size of the illuminated spot. The use of a spectrally separated reference object allows for single-shot imaging where the reference and object are recorded in a single exposure. This method has been implemented in an especially convenient way using the Bayer filter of a color camera to separate the reference and object patterns. The presented single-shot reconstruction method is fast and robust, which opens up prospects of imaging objects through thin but highly dynamic scattering media.