Speckle correlation resolution enhancement of wide-field fluorescence imaging

HASAN YILMAZ,* ELBERT G. VAN PUTTEN, JACOPO BERTOLOTTI, AD LAGENDIJK, WILLEM L. VOS, AND ALLARD P. MOSK Complex Photonic Systems (COPS), MESA+ Institute for Nanotechnology, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands Currently address: Philips Research Laboratories, 5656 AE Eindhoven, The Netherlands Currently address: Physics and Astronomy Department, University of Exeter, Stocker Road, Exeter EX4 4QL, UK *Corresponding author: h.yilmaz@utwente.nl


INTRODUCTION
A conventional optical microscope produces images with a resolution determined by the numerical aperture (NA) of the imaging lens [1].The NA of an imaging lens is defined by the highest wave vector that is accessible in the transversal direction.Many methods have been introduced that enable optical resolution beyond the resolution limit of a conventional optical microscope.These include the use of evanescent waves with near-field scanning optical microscopy (NSOM) [2]; moiré fringes such as in structured illumination microscopy (SIM) [3,4] or blind structured illumination microscopy (blind-SIM) [5]; nonlinear optical phenomena such as in saturated-SIM [6] or stimulation emission depletion (STED) microscopy [7,8]; and specific photophysical properties of dyes such as in stochastic optical reconstruction microscopy (STORM) [9], photoactivated localization microscopy (PALM) [10], or superresolution optical fluctuation imaging (SOFI) [11].Nevertheless, NSOM has a field of view limited by the scan range of the probe; moreover, a scanning probe usually greatly affects the measurement itself.SIM provides a resolution that is potentially two times greater than a conventional optical microscope [3,4], but requires a precise knowledge of the illuminating intensity patterns on the structure of interest.Recently, a blind-SIM method [5] has been demonstrated which exploits the statistical properties of speckle patterns and does not require precise knowledge of the illuminating intensity patterns.Since STED requires intense laser pulses, it is a question how to use it for delicate samples with a low damage threshold.STED, STORM, PALM, and SOFI require dyes with specific photophysical properties.Although the aforementioned far-field microscopy methods realize an optical resolution beyond the diffraction limit, the resolution remains strongly dependent on the NA of their conventional optics.
It has been shown that a scattering medium enhances spatial resolution for acoustic waves [12], microwaves [13], and light waves [14,15], and turns a high-index substrate into a high-NA solid immersion lens by breaking the translational invariance on the interface of the substrate [16].Coherent light illumination on such a scattering medium generates a speckle pattern of apparently randomly distributed bright and dark regions behind the scattering medium.Correlations of speckle patterns are used to obtain images of astronomical objects through turbulent atmosphere with diffraction-limited resolution [17,18].The concept of exploiting correlations between such speckle patterns to obtain images of microscopic objects has started a new class of optical microscopy [19].Within a speckle pattern there is a correlation effect called the optical memory effect [20,21] that has recently been exploited for optical imaging using a high-NA scattering lens [16] and through scattering media [22][23][24][25][26][27][28].A tilt of the incident beam within an angle of Δθ < λ∕2πd (λ is the wavelength of light and d the thickness of the scattering medium) results in a translation of the speckle pattern behind the medium without a significant change in the pattern.The optical memory effect has been employed to obtain optical images of microscopic objects hidden by a scattering medium with diffraction-limited resolution [26][27][28].Previously, a high-NA gallium phosphide (GaP) scattering lens has been used to image gold nanoparticles with elastically scattered coherent visible light [16].However, the available field of view with speckle correlations is limited to 2 μm × 2 μm due to the finite range of the optical memory effect, and the high-index scattering lens has so far not been applied to incoherent imaging modalities such as fluorescence microscopy.
Here, we introduce and demonstrate speckle correlation resolution enhancement (SCORE) microscopy that combines the high resolution of speckle scanning microscopy with a wide field of view of parallel speckle-scan detection.Figure 1 shows the concept of our method.The main element of the experiment is a scattering lens consisting of a GaP substrate with a thickness of L 400 μm and refractive index of n 3.42, of which one surface has a scattering layer with a thickness of d 2 μm and one surface is polished [16].The polished surface is coated with silicon (Si) with a thickness of 100 nm to reduce internal reflections (see Supplement 1).The Si coating is removed in a 10 μm scale window to place objects directly on the polished GaP surface.A beam of coherent light with a diameter of 0.8 mm and wavelength of λ ill 561 nm is incident onto the scattering surface of the substrate.The scattering layer generates a speckled intensity pattern Sx; y that illuminates a fluorescent object Ox; y.The fluorescence intensity distribution on the object plane is imaged on a camera with a resolution of R λ flu ∕2NA, with λ flu 612 nm and NA 0.95.We raster scan the speckle pattern on the object plane by tilting the incident beam by angles δθ x and δθ y , within the angular range of the optical memory effect (Δθ ≈ 1°) that leads to a speckle-scan range of 2 μm on the object plane.We record fluorescence images at every δθ x , δθ y for a range of angles of incidence, resulting in perpendicular speckle pattern displacements δx ≈ δθ x L∕n and δy ≈ δθ y L∕n in the object plane.This procedure results in a four-dimensional datacube Ix; y; δx; δy, which has sufficient information to reconstruct a wide-field image that has a resolution of the average speckle grain size [26].

HIGH-RESOLUTION SPATIAL INFORMATION RETRIEVAL
In Fig. 2, we show the data analysis procedure.We divide the datacube [Fig.2(a)] into N sub-datacubes by applying N square window functions of W m x; y with a 96% overlap, such that each have a width and a height equal to half of the speckle-scan range (1 μm) and each can be processed in parallel.We construct a standard resolution subimage J m x; y [Fig.2(b)] and a specklescan matrix K m δx; δy [Fig.2(c)] from the corresponding subdatacube as follows: we sum our sub-datacube over δx and δy and obtain the standard resolution subimage J m x; y.In our approach, it is useful to represent J m x; y in the Fourier domain, where its spatial information is given by the magnitude and the phase of the Fourier components [Figs.2(d   where the symbol denotes a convolution product and where in the last step we assumed that the scan range stays within the optical memory effect range.In Figs.2(f) and 2(g), we represent the speckle-scan matrix K m δx; δy in the Fourier domain.We obtain the magnitude of the high-frequency Fourier components of the object from its speckle-scan matrix as follows: where C jF fSgj is the optical transfer function of the GaP scattering lens and F fg denotes a Fourier transform.Here we use the approximation that, within the NA of the GaP scattering lens, the absolute value of the spatial spectrum of the field is constant for a fully developed speckle pattern [29] and therefore the magnitude of the spatial spectrum of S will be approximately a cone-shaped function in the Fourier domain with a cutoff at the highest Fourier component of the GaP scattering lens [30].Equation (2) shows that the magnitude of the high-frequency Fourier components of the object is retained behind the scattering layer [Fig.2(f)].The phase information of the object's Fourier components is lost due to the random and unknown phase of the speckle pattern [Fig.2(g)].Fortunately, it is often possible to infer the lost phase information using an iterative phaseretrieval algorithm [31][32][33][34].In essence, our approach relies on reducing the light-scattering problem to a phase-retrieval problem.

IMAGE RECONSTRUCTION
We have developed a new Gerchberg-Saxton-type algorithm that uniquely retrieves the high-frequency phase information of the Fourier components of our object by using the low-frequency phase information of the Fourier components of the object as a new constraint (see Supplement 1).In general, a Gerchberg-Saxton-type algorithm retrieves the phase of the Fourier components of an image from the magnitude of the Fourier components by using some constraints on the image or the Fourier domain.
Using only the magnitude of the Fourier components gives rise to ambiguities that lead to stagnation problems in the reconstruction [33,35].In ptychography and Fourier ptychography, such ambiguities are discarded by overlapping many wide-field images [36][37][38][39].However, both ptychography and Fourier ptychography algorithms work only for diffractive objects.For nondiffractive objects such as fluorescent objects, these ambiguities are absent since the object is represented with real and positive numbers and the object can be determined up to a trivial flip and translation [26][27][28].In our Gerchberg-Saxton-type algorithm, we employ constraints both in the image domain and the Fourier domain to discard even trivial flip and translation ambiguities.
In the image domain, we use the well-known information that the measured intensity of our fluorescent object is real and positive.In the Fourier domain, we introduce as a new feature the phase of the low-frequency Fourier components.Combining these two types of information, the algorithm converges to a unique solution which gives us the shape, position, and orientation of the object.This is a major improvement over previous approaches [26][27][28] that do not provide position and orientation information.We analyzed the convergence probability of our algorithm on a complex test object composed of six letters of the Latin alphabet (see Supplement 1).We found that when the detection NA (NA det ) is higher than 13% of the illumination NA (NA ill ), our algorithm always converges to a unique solution.
If the NA det is lower than the 13% of the NA ill , the convergence probability drops.For our experimental situation, the NA det is always larger than 30% of the NA ill and convergence is fast.The low rate of convergence at very low NA det ∕NA ill can almost certainly be considerably improved by implementing antistagnation mechanisms [40].Even without these mechanisms, however, we obtain a sufficient convergence rate for our experimental situation.
In Fig. 3, the phase-retrieval procedure of high-frequency Fourier components is shown for a single sub-datacube.First, we Fourier transform both a standard resolution subimage J m x; y and the corresponding speckle-scan matrix K m δx; δy.We discard the magnitude of the Fourier components of J m x; y and the phase of the Fourier components of K m δx; δy.We input the phase information of low-frequency Fourier components of J m x; y and the magnitude information of high-frequency Fourier components of K m δx; δy into our Gerchberg-Saxton-type algorithm.The algorithm outputs the phase information of high-frequency Fourier components.Finally, we combine and inverse Fourier transform all available phase and magnitude information of the Fourier components to obtain the high-resolution subimage.Our algorithm has some conceptual similarity with pattern-illuminated Fourier ptychography [41].In pattern-illuminated Fourier ptychography, the wide-field images are overlapped in the Fourier domain, whereas we select a subarea in the x, y plane and project the corresponding sub-datacube onto the k x , k y plane.We then use phase retrieval to obtain the corresponding high-resolution subimage.Our method has the advantage of robustness against the drifts compared to patternilluminated Fourier ptychography.Our method works as long as the drifts are small compared to the optical memory effect range, whereas pattern-illuminated Fourier ptychography and related methods [5] have much more stringent requirements and only converge for drifts smaller than the speckle grain size.
To acquire a wide-field image, we apply the phase retrieval procedure shown in Fig. 3 to every sub-datacube (see Fig. 2) in parallel.We have approximately N sub-datacubes for a wide-field image with N pixels that corresponds to a 96% overlap of each neighboring sub-datacube.Each reconstructed overlapping high-resolution subimage is windowed by a smooth window function to minimize edge effects.The detailed shape of the window has little to no effect on the reconstruction, as long as it is soft-edged and thus does not introduce artifacts.We tile the reconstructed high-resolution subimages to yield a wide-field image of the complete object.The field of view of the reconstructed image is wider than the speckle-scan range and spans the field of view of the detection optics.Using the optical transfer function of our scattering lens, we inverse-apodized the wide-field SCORE image.In order to have a fair comparison, we also inverse-apodized the wide-field conventional image with the known optical transfer function of the detection objective (see Supplement 1).

RESULTS AND DISCUSSION
To experimentally test our new imaging method, we use a collection of fluorescent nanospheres with a diameter of 100 nm as test objects.Figure 4(a) shows an image of a collection of many fluorescent nanospheres taken with conventional high-NA microscopy in a field of view of 10 μm × 10 μm.The zoom-in in Fig. 4(b) reveals five separate nanospheres.Figure 4(c) shows a cross-section of two nanospheres from Fig. 4(b) that have a full width-half-maximum (FWHM) of about 430 nm.We now turn to the high-resolution SCORE results.Figure 4(d) shows the same area as in Fig. 4(a).In Fig. 4(d), the nanospheres are sharper compared to the image in Fig. 4(a).The zoom-in in Fig. 4(e) shows the same area as in Fig. 4(b): we see that the nanospheres are much sharper compared to Fig. 4(b) and we see six separate nanospheres, whereas less nanospheres were discernible in Fig. 4(b).Notably at the left center, two nanospheres are distinguished that were observed as one blob in Fig. 4(b).Figure 4(f) shows a cross-section of three nanospheres from Fig. 4(e) that have a FWHM of about 135 nm.A clear demonstration of the enhanced resolution is given in Fig. 4(f), where we clearly resolve two nanospheres with a center-to-center distance of 146 nm and an edge-to-edge distance of 46 nm.The fact that two 100 nm spheres with a center-to-center distance of 146 nm are clearly resolved suggests that the effective resolution of our current system (according to Sparrow's criterion) must be well below 146 nm [42].In our SCORE images, nanospheres appear as features with a FWHM of 135 nm.As the features in the SCORE image are the convolution of the object and the point spread function (PSF), we estimate the effective FWHM of the PSF for the image feature as Δ PSF Δ 2 feature − Δ 2 object 1∕2 116 nm (see Supplement 1).A resolution of 116 nm agrees very well with the expected diffraction-limited resolution from the geometry of our scattering lens.Our results demonstrate that speckle correlations enhance the resolution of an optical microscope without any restriction on its field of view.In essence, SCORE with the combination of a GaP scattering lens is a high-resolution total internal reflection fluorescence microscope.The demonstrated resolution of 116 nm can be achieved as long as the object of interest is very close to the substrate of the GaP scattering lens (see Supplement 1).
Previously, a speckle imaging method has been reported where a high-resolution image is retrieved from a set of low-resolution images [14].However, a measurement of the high-resolution speckle pattern is required as a priori information.A high-NA scattering lens could only be used in above-mentioned method with a very challenging precalibration using NSOM [43].In cases where a very high resolution speckle pattern is desired, a high-NA scattering lens is inevitable [16].It is interesting to compare the performance of SCORE to blind-SIM [5], which has conceptual similarities.An information-theoretical analysis shows that the two methods perform differently for different NA ratios.For NA ratio of NA ill ∕NA det ≈ 1, blind-SIM offers a higher resolution and a better signal-to-noise ratio (SNR).At the high NA ratio of our experiment, blind-SIM is not able to provide useful SNR, whereas SCORE can work with acceptable SNR up to very high NA ratios (see Supplement 1).

METHODS
Speckle-scan matrices contain high-resolution information of the imaging object.In order to measure a speckle-scan matrix K m x; y; δx; δy, the speckle pattern has to stay correlated over the resolution R λ flu ∕2NA.This constraint is met when R < λ ill L∕2πnd , where n is the refractive index of the GaP substrate, L is the thickness of the GaP substrate, λ ill is the wavelength of the incident light on the GaP porous layer, and d is the thickness of the GaP porous layer.In our GaP substrate, λ ill L∕2πnd is on the order of 2 μm.Our detection optics has a resolution (R 322 nm) that is high enough to fulfill this condition.The average speckle grain size of a GaP scattering lens is r λ ill ∕2n sintan −1 W ∕2L, where W is the beam width.In our case, an average speckle grain size is r 116 nm.We scan the speckle pattern with steps of 40 nm over a range of 2 μm in two dimensions, requiring N 2500 measurements.For each measurement the full camera image is stored, which allows us to retrieve the object at any position of the captured field of view.The data acquisition takes about 1 h.The time scale of the data acquisition is independent of the field of view, while it follows a square law with respect to the targeted resolution.However 1 h is not a fundamental limit.The data acquisition time can be reduced by an order of magnitude by using a more sensitive camera.

CONCLUSION
In summary, we experimentally demonstrate a new method to obtain high-resolution and wide-field fluorescence images.In combination with a GaP scattering lens, SCORE has the ability to acquire very high-resolution images with a field of view that is much wider than the speckle-scan range.The preparation of a GaP scattering lens is easy and versatile by a standard method [44].SCORE is thus excellently suited to be used for imaging of two-dimensional objects as large as a few hundred micrometers with subcellular resolution.It is not necessary to characterize the scattering lens a priori by methods such as wavefront shaping [16], digital optical phase conjugation [22], or transmission matrix measurement [45,46].If one uses the information of both the detection and the illumination NA as in structured illumination microscopy and pattern-illuminated Fourier ptychography, our method has a higher potential resolution than presented here.Using only the illumination NA of our microscope, we have the advantage of robustness on ambient conditions such as optical aberrations and mechanical drifts.Even using only the illumination NA, our method provides a remarkable resolution under ambient conditions.With a setup optimized for stability, our analysis method can be modified to improve the resolution even further.Our method can provide resolution enhancement in various scenarios where collecting low-resolution, wide-field images is possible.In case one can take pictures using a camera with a low-NA objective and wants pictures with higher resolution than the objective can achieve, the scattering environment can be used as a high-NA lens with our method as follows.The object of interest can be illuminated by low-intensity laser light that generates a speckle pattern reflected from a wall.The generated speckle pattern on the object can be translated by the optical memory effect.Taking several low-resolution pictures for a scanning range on the order of the resolution of the low-NA camera objective is sufficient to reconstruct high-resolution images with the wide field of view of the camera objective.

Fig. 1 .
Fig. 1.Concept of wide-field, high-resolution speckle scanning microscopy.A beam of coherent light illuminates a scattering layer on a gallium phosphide (GaP) substrate.The scattered light generates a speckle pattern that enables high-resolution imaging on the object plane.The incident beam is tilted by angles θ x and θ y , and as a result the speckle pattern is scanned across the sample.Fluorescent nanospheres on the object plane are imaged on a CCD camera.With parallel speckle detection, the field of view is wider than the single speckle-scan range.d denotes the thickness of the scattering layer; L denotes the thickness of the substrate.

Fig. 2 .
Fig. 2. Data analysis procedure on a single sub-datacube.(a) Datacube Ix; y; δx; δy.(b) Standard resolution subimage J m x; y obtained by summing the sub-datacube shown by the square non-grayed-out area over δx and δy.(c) Speckle-scan matrix K m δx; δy obtained by summing the sub-datacube shown by the square non-grayed-out area over x and y.(d) Magnitude of the Fourier components of J m x; y.(e) Phase of the Fourier components of J m x; y.(f) Magnitude of the Fourier components of K m δx; δy.(g) Phase of the Fourier components of K m δx; δy.

Fig. 3 .
Fig. 3. Phase retrieval in the Fourier domain.(a) Phase of the Fourier components of the object.(b) Magnitude of the Fourier components of the object.(c) Gerchberg-Saxton-type algorithm.(d) Retrieved phase of high-frequency Fourier components of the object.Phase data comes from Fig. 2(e) and the magnitude data comes from Fig. 2(f).Color bars are as in Fig. 2.

Fig. 4 .
Fig. 4. Wide-field images of fluorescent nanospheres with diameter of 100 nm.(a) Wide-field image by conventional microscopy.(b) Zoomed image of (a).(c) Cross-section of (b) represented by the white line.(d) Wide-field image by SCORE microscopy.(e) Zoomed image of (d).(f) Cross-section of (e) represented by the white line.In (c), a single nanosphere is apparent while in (f), two smaller nanospheres are apparent with a center-to-center distance of 146 nm.