Wide-field Fourier ptychographic microscopy using laser illumination source

Fourier ptychographic (FP) microscope is a coherent imaging method that can synthesize an image with a higher bandwidth using multiple low-bandwidth images captured at different spatial frequency regions. The method's demand for multiple images drives the need for a brighter illumination scheme and a high-frame-rate camera for a faster acquisition. We report the use of a guided laser beam as an illumination source for an FP microscope. It uses a mirror array and a 2-dimensional scanning Galvo mirror system to provide a sample with plane-wave illuminations at diverse incidence angles. The use of a laser presents speckles in the image capturing process due to reflections between glass surfaces in the system. They appear as slowly varying background fluctuations in the final reconstructed image. We are able to mitigate these artifacts by including a phase image obtained by differential phase contrast (DPC) deconvolution in the FP algorithm. We use a 1-Watt laser configured to provide a collimated beam with 150 mW of power and beam diameter of 1 cm to allow for the total capturing time of 0.96 seconds for 96 raw FP input images in our system, with the camera sensor's frame rate being the bottleneck for speed. We demonstrate a factor of 4.25 resolution improvement using a 0.1 NA objective lens over the full camera field-of-view of 2.7 mm by 1.5 mm.


Introduction
Fourier ptychographic microscopy (FPM) is a recently developed computational imaging system capable of acquiring the complex and quantitative field distribution of a sample [1,2]. Unlike conventional microscopes that can only image the intensity distribution, FPM's complex sample field contains both its amplitude and phase information. FPM achieves this by a simple modification in sample illumination without the need for a separate reference beam or mechanical movement within the system as in other phase imaging systems. It uses a coherent light source to image different components of the sample's Fourier spectrum, and uses a phase retrieval algorithm to synthesize these images into a high-resolution complex field distribution. Effectively, it can linearly improve the numerical aperture of the imaging lens by the illumination NA.
There has been various improvements and applications of FPM. Numerical aperture of over 1 for a conventional microscope, usually only achievable by using some immersion medium between objective lens and sample, was realized with a low NA objective and an arrangement of LEDs allowing for steep illumination angles [3]. The high-resolution and wide field-of-view (FOV) of FPM showed potential applications in white-blood-cell counting [4] and resourcelimited imaging scenarios [5,6]. Multiplexed illumination patterns allowed for high resolution high speed phase imaging of unlabeled in-vitro cells [7]. An iterative algorithm that reconstructs the aberration of the microscope system simultaneously with the sample spectrum allowed for removal of spatially varying aberrations throughout the microscope's field of view [8] and made FPM particularly suitable for imaging samples with uneven surfaces [9]. The characterized aberration function further allowed for removing spatially varying aberrations from fluorescence images for even performance across the field of view [10]. Insights from FPM carried over to incoherent imaging to improve the resolution of fluorescence images [11]. There also have been numerous efforts in improving the Fourier ptychographic (FP) reconstruction by adopting more noise-robust algorithms [12,13,14,15,16]. Alternative FPM modalities involving aperture scanning instead of angular illuminations were demonstrated, which allowed for imaging the complex field of a thick specimen [17] and estimating optical aberrations [18]. Imaging a thick specimen with angular illuminations also became possible by employing the first Born approximation in the image formation process [19,20].
With wider adoption of FPM for imaging and the desire to image fast dynamics, faster capturing speed is desired. There have been several efforts in this respect. Using LEDs, the required number of captured images can be reduced by optimizing LED illumination arrangements [21,22], illuminating multiple LEDs of either the same color [23] or different colors [24,25], or using better ways of image processing [15]. Ref. [26] demonstrated for the first time using a high-power laser beam coupled with a DMD which allowed for shot-noise-limited image capturing process, overcoming the power limitation of LEDs. All these methods address the slow capture issue, but they are not without downsides. By reducing the number of captured images via multiplexing, one increases the shot-noise per individual sub-spectrum of sample. In [26], although the power of illumination is easily scalable by using a stronger laser, the onstate mirrors only constitute a small portion of the entire DMD area. The rest of the area which are in the off-state deflects a large portion of the input laser power to a beam dump or in a certain angle which scatters strongly in the optical path and contributes negatively to captured images. Also, the FOV was limited to around 50 µm by 50 µm for a 0.04 NA 1.25x objective, which is much smaller than the FOV typically offered by such an objective lens. Another feature overlooked by many FPM illumination schemes implemented so far is the efficient usage of the illumination beam to improve capturing speed: an LED's radiation profile typically follows a Lambertian distribution [27] and only a small portion of it actually ends up illuminating the sample; and a DMD only utilizes a tiny fraction of the input laser beam for each sample illumination angle.
Here, we present an FPM setup illuminated by a laser guided by a Galvo mirror and a mirror array. We are able to utilize 15% of the total laser output power for sample illumination with the proposed collimation setup. The utilization ratio can be increased for a higher-beam-quality laser and finer collimation optics. The benefit of using a mirror array over a condenser lens such as done in [26] and suggested by [28] is that the illuminating wavefront does not suffer from additional aberrations induced by lenses, which can negatively impact FP reconstruction if not properly corrected for. Also, the NA of the condenser lens limits the amount of angular scanning possible for angularly varying illumination before the aberration from the lens becomes severe.
We demonstrate that our system can generate a quantitative image of a sample with 4.25 times the resolution. The coherence of laser leads to speckle artifacts that presents a challenge in our reconstruction process, the majority of which originates from the strong unscattered laser beam reflected between glass surfaces in the optical path. We mitigate the speckle's influence by incorporating into the FP algorithm the sample's phase obtained from differential phase contrast (DPC) deconvolution method [29] which is a procedure also employed in [7] during FPM initialization for better reconstruction of low-frequency phase information. We show that low spatial frequency artifacts due to speckles are effectively removed from the reconstructed phase of the sample. Overall, our laser FPM demonstrates wide FOV and high quality image reconstruction at a high capturing speed.

Principle of FPM
Our FPM algorithm operates on the principle that the sample to be imaged is very thin [3]. This essentially turns it into a two dimensional sample, similar to a thin transparent film with an absorption and phase profile on it. When the sample on the stage of a 4f microscope is perpendicularly illuminated by a light source that is coherent both temporally (i.e. monochromatic) and spatially (i.e. plane wave), the light field transmitted through the sample is Fourier transformed when it passes through the objective lens and arrives at the objective's back-focal plane. The field is then Fourier transformed again as it propagates through the microscope's tube lens to be imaged onto a camera sensor or in a microscopist's eyes. The amount of the sample's detail the microscope can capture is defined by the objective's numerical aperture (NA obj ) which physically limits the extent of the sample's Fourier spectrum being transmitted to the camera. Thus, the NA obj acts as a low-pass filter in a 4f imaging system with a coherent illumination source.
In the following, we limit our discussion to a one dimensional case. Extending to two dimensions for a thin sample is direct. Under the illumination of the same light source but at an angle θ with respect to the sample's normal, the field at the sample plane, ψ oblique (x), can be described as: where ψ sample (x) is the sample's complex spatial distribution, x is a one dimensional spatial coordinate, and k 0 is given by 2π/λ where λ is the illumination wavelength. This field is Fourier transformed by the objective lens, becoming: at the objective's back-focal plane, where Ψ oblique and Ψ sample are the Fourier transforms of ψ sample and ψ oblique , respectively, and k is a one dimensional coordinate in k-space. Ψ sample (k) is shown to be laterally shifted at the objective's back-focal plane by k 0 sin θ . Because NA obj is physically fixed, a different sub-region of Ψ sample (k) is relayed down the imaging system. Thus, we are able to acquire more regions of Ψ sample (k) by capturing many images under varying illumination angles than we would by only capturing one image under a normal illumination. Each sub-sampled Fourier spectrum from the objective's back-focal plane is Fourier transformed again by the tube lens, and the field's intensity value is captured by the camera sensor. Due to the loss of phase information in the intensity measurement, the sub-sampled images Modified FP algorithm to include DPC-generated phase into the iteration. The reconstruction begins with the raw image captured with the illumination from the center mirror element as an initial guess of the sample field. The iteration process starts by forming the sample's quantitative phase image with 2NA resolution by DPC deconvolution with the initial guess of the pupil function. The phase of the sample field up to the 2NA resolution extent is updated. Images captured under varying illuminations are used to update the pupil function and the sample's Fourier spectrum, just as in the original FP algorithm. The updated pupil function is used to generate an updated DPC-deconvolved phase image for the update process, and the iteration process repeats until convergence. In the end, we reconstruct the complex field of the sample and the pupil function. cannot be directly combined in the Fourier domain. We use a Fourier ptychographic (FP) algorithm, essentially a phase retrieval algorithm, to reconstruct the phase and amplitude of the expanded Fourier spectrum. The algorithm requires the low-passed images to be captured so that each image contains some overlapping region in the Fourier domain [15]. We allow a 60% overlap between images, and this redundancy allows for the FP algorithm to infer the missing phase information through an iterative method which is described in the following section.

Algorithm
A high-resolution image of a sample to be reconstructed is initialized with the low-resolution image captured under a normal illumination. To further improve the iteration's starting point and at the same time apply robust phase constraints to facilitate algorithm convergence, we adopt the DPC deconvolution [29] to obtain the sample's phase image for an additional initialization step. DPC deconvolution is a partially coherent method to achieve the quantitative phase of a sample. It is based on the assumption that the sample's absorption and phase are small such that the sample's complex transmission function, ψ(x) = exp(−µ(x) + jθ (x)), can be approximated as [30]: ψ(x) ≈ 1 − µ(x) + jθ (x). Under this condition, performing simple arithmetic operations on the images captured under different illumination angles generates multiple-axis DPC images and the transfer function associated with the sample's phase and the DPC images [29]. Deconvolving the transfer function from the DPC images results in the quantitative phase image of the sample with the spatial frequency information extending to 2k 0 NA obj in k-space. Also, the partially coherent nature of DPC method significantly reduces the speckle noise originating from the coherent light source [29] and thus improves the reconstructed image of our algorithm. We update the phase of our initial guess with the DPC-deconvolved quantitative phase as follows: where Ψ(k) is the high-resolution Fourier spectrum of a sample, P 2NA is the low-pass filter with the spatial frequency extent of 2k 0 NA obj in k-space, F is Fourier transform operator, θ DPC is the quantitative phase obtained from DPC deconvolution, and ψ 2NA is the simulated image with its phase updated with θ DPC . Unlike intensity image updates in FP, an update with the phase from DPC deconvolution requires us to use a pupil function extending to 2NA obj instead of just NA obj because the deconvolved phase contains information up to 2NA obj resolution [29].
Intensity images captured at different angles are used to reconstruct the high resolution Fourier spectrum and the pupil function of the microscope as done in the original FP algorithm found in [1,8]. The generation of DPC phase and the update process involving the DPC phase and intensity images constitute one iteration. DPC phase needs to be recalculated at the beginning of each iteration because the pupil function of the microscope changes during pupil function update procedure. The overall algorithm is summarized in Fig. 1. For the reconstruction to convergence, we conduct 25 iterations without updating the pupil function, and 15 with, resulting in 40 iterations in total. In the end, we obtain the high-resolution complex field of the sample and the imaging system's pupil function. Fig. 3. The Fourier spectrum region covered by the angularly varying illumination and the layout of the mirror array to achieve the desired coverage. With the objective NA of 0.1 and one normal plane wave illumination, the spatial frequency acquired by the system is delineated by the black circle in the Fourier domain. With varying illumination angles, we can expand the extent of the captured spatial frequency, as indicated by the red circle with the NA of 0.425. The mirror array is 30 cm wide and is placed 40 cm away from the sample plane. Each circular bandpass in the Fourier domain, with its size defined by NA obj and its location by the illumination angle provided by each mirror element, has 60% overlap with the contiguous one.

Setup
The imaging setup is a 4f system consisting of a 0.1 NA objective lens (Olympus 4x), 200mm-focal-length tube lens (Thorlabs), and a 16bit sCMOS sensor (PCO.edge 5.5). The sensor has a pixel size of 6.5 µm and a maximum frame-rate of 100 Hz at 1920x1080 resolution for a global shutter mode. The sensor size limits the available FOV of a sample to be 2.7 mm by 1.5 mm. On the illumination side, 457 nm 1 W laser beam is pinhole-filtered and collimated. After collimation process, the output beam is 1 cm in diameter and 150 mW in power. A set of mirrors guides the beam such that the central part of its Gaussian profile (about 40% of total output area) is incident on the input of 2D galvo mirror device (GVS 212) for a uniform beam intensity distribution at its output. Galvo then guides the beam to individual mirror elements on the 3D-printed array, as shown in Fig. 2. Each mirror element is a 19mm x 19mm firstsurface mirror attached to a 3D-printed rectangular tower. The tower's top surface is sloped at a certain angle such that the beam from Galvo is reflected towards the sample's location. Thus, the element's spatial location relative to the sample determines the illumination angle of the beam. The mirror array consists of 95 elements arranged to provide illumination angles such that contiguous elements produce 60% overlap of the sample's spectrum in the Fourier domain, as shown in Fig. 3. The total illumination NA corresponds to NA ti = 0.325 with the resulting system NA being NA sys = NA obj + NA ti = 0.1 + 0.325 = 0.425, effectively increasing the microscope's NA by a factor of 4.25.
To achieve the maximum frame rate of the sCMOS sensor in the image capturing process, the exposure time is set to its minimum, at 500 microseconds. The sensor and Galvo are externally triggered every 10 milliseconds, resulting in 0.96 seconds of total capturing time for 95 sample images and 1 dark noise image. Maintaining the same exposure time for all images presents a small challenge: the SNR of the images are drastically different between images captured in the bright-field illumination (NA illum < NA obj ) and the ones in the dark-field (NA illum > NA obj ) because the unscattered laser beam comprises the most of the signal from the sample, especially Fig. 4. USAF target, before and after FP reconstruction. Under a normal illumination provided by the center mirror element, the theoretical resolution corresponds to λ NA obj = 4.57 µm periodicity. Up to group 7 element 6 is observed, which has 4.38 µm periodicity, closely matching the prediction. After FP reconstruction, the theoretical resolution is λ NA sys = 1.08 µm periodicity, closely matching the observed group 9 element 6's periodicity of 1.10 µm.
for natural samples such as neurons [31]. Adjusting the laser intensity for the proper exposure of the bright-field images would result in low signal values in dark field images at the same laser intensity level and camera exposure time. As a result, the dark field images would tend to be more affected by dark noise. To account for this, a neutral density filter is placed on each brightfield illumination mirror elements. This allows for increasing the input laser intensity to obtain higher SNR in dark field images while preventing the bright field images from over-exposure.

Spatial resolution
We image a positive USAF resolution target printed on a microscope slide to validate our system's resolving power. Under a normal illumination by the laser beam reflected from the center mirror element, the maximum resolvable feature is group 7 element 6, which corresponds to 4.38 µm periodicity as shown in Fig. 4. We then capture multiple images under varied illumination angles and apply our FP algorithm as in Fig. 1. The improved image shows features as small as group 9 element 6, which corresponds to 1.1 µm periodicity, as shown in Fig. 4. This is in accordance to the theoretical resolution afforded by NA sys which can be calculated as: periodicity = 457 nm 0.425 = 1.08 µm. In Appendix A, we conduct additional measurements using a Siemens star target for a more accurate quantification of our system's resolution [32].

Quantitative phase
We use a microscope slide with microspheres to demonstrate quantitative phase imaging of our laser FPM. The sample consists of 4.5-µm-diameter polystyrene bead from Polysciences, Inc. (index of refraction n s = 1.6119 @ 457 nm) immersed in oil (index of refraction n o = 1.5269 @ 457 nm). The bead diameters and the indices of refraction for the beads and oil are carefully chosen so that they satisfy the requirement for successful quantitative phase imaging as presented in [2]. The maximum phase gradient generated by the microbead shouldn't exceed the maximum resolvable spatial frequency of our FPM system.
In Fig. 5, we show the importance of including the additional step of DPC-deconvolved phase image update into our algorithm. Fig. 5(b) shows the FP reconstructed phase image before and after the modification in the algorithm. Background noise in the original FP reconstruction is mainly due to the speckle noise originating from the mostly unscattered laser beam, as seen in Fig. 5(a), interfering with itself in the optical setup. The noise contributes to the final reconstructed Fourier spectrum as a slowly varying phase signal. By incorporating the DPC deconvolved quantitative phase image in the update scheme, we are able to remove the background noise. This is because the DPC deconvolution is a partially coherent method and thus is immune to speckle noise [29]. The DPC-deconvolved phase image is free from the influence of the low-fluctuating speckle, as shown in Fig. 5(b).
In Fig. 5(b), one representative micro-bead, as indicated by the dashed line in the updated FP reconstruction phase, is compared with the theoretical bead profile. The reconstructed phase of the bead is unwrapped and converted into the bead thickness using the given refractive indices of oil and beads. The measured bead diameter is 4.24 µm, which is within the 10% tolerance of the theoretical value of 4.5 µm. Without an additional DPC-deconvolved phase update, the reconstructed phase image shows an uneven background. After the modification, the reconstructed phase is free from the background noise and the resulting phase is also quantitative. Fig. 6. Blood smear images, before and after modification in FP algorithm. Without the additional DPC-deconvolved phase update in the reconstruction process, the resulting phase of the sample shows an uneven background signal that also influences the cells' phase amplitude. After the modification, the background is uniform and the red blood cells show similar phase values. Note, the modification has little or no affect on the amplitude image.

Imaging of biological samples
We first image a blood smear sample prepared on a microscope slide stained with Hema 3 stain set (Wright-Giemsa). We notice coherence artifacts, i.e. speckles, in the background of the lowresolution captured images. Without the DPC algorithm, the phase image suffers from uneven background signals due to speckles, as shown in Fig. 6. After the DPC update, we observe the phase clears up significantly.
To demonstrate the wide-field performance of our laser FPM, we capture a full FOV image of an H&E stained histology sample, as shown in Fig. 7. We first segment the entire region into small square tiles (370 µm by 370 µm) to account for the spatially varying aberration of our imaging system. Then we apply our modified FP algorithm to each tile to reconstruct a high resolution image of the entire FOV. In the end, we are able to correct for the spatially varying aberration and obtain a wide-field and high-resolution image, just like in the original FPM with LEDs as the illumination source but at a much higher capturing speed.

Discussion
We demonstrate that an FPM setup involving a laser as the illumination source is capable of providing a both wide FOV and high-resolution image. Although the higher temporal and spatial coherence of laser compared to those of an LED lead to speckle artifacts, appropriately including the additional constraint by the DPC-deconvolved phase in our FP algorithm is able to mitigate the negative influences on the final reconstructed image. However, we acknowledge that this modification is not a complete solution to the speckle noise because 1) DPC deconvolution only works for a weakly absorbing and weakly scattering sample; and 2) the background amplitude fluctuation in the captured low-resolution images due to speckles is not directly addressed by the algorithm. In order to significantly reduce the speckles which predominantly originate from the interference of the unscattered laser beam, all the glass surfaces in the optical system would need to have anti-reflective coatings suitable for the laser's wavelength.
The use of 3D-printed mirror elements allows for intuitive optical setup, but it is not as flexible as other illumination schemes when different objective lenses are used in the imaging system. An entirely new array may be required for different objectives to satisfy the desired resolution gain and appropriate overlap in the Fourier domain. A modular or adjustable design of the array would make the system more flexible in different imaging scenarios.
Use of Galvo mirrors to direct the collimated laser beam allows for the efficient usage of the laser power, and liberates the illumination source from being the bottleneck of FPM's capturing speed. With a faster camera sensor and an easily adjustable illumination arrangement, imaging faster dynamic samples with FPM will be possible. Moreover, the proposed setup can accommodate lasers with any wavelengths that are compatible with the optical elements and the sensor by simply coupling the laser output to the collimation optics. Bright-field spectral imaging can be easily realized by utilizing a multispectral laser as the illumination source. The fast frame rate and the variety of laser wavelengths available make FPM more attractive for a wider usage. Fig. 8. The Siemens star used for quantifying our system's resolution performance. It consists of 36 line pairs extending radially, which are etched on a gold-coated microscope slide by a focused ion beam. The scale bar is 5 µm