Near-field Fourier ptychography: super-resolution phase retrieval via speckle illumination

Achieving high spatial resolution is the goal of many imaging systems. Designing a high-resolution lens with diffraction-limited performance over a large field of view remains a difficult task in imaging system design. On the other hand, creating a complex speckle pattern with wavelength-limited spatial features is effortless and can be implemented via a simple random diffuser. With this observation and inspired by the concept of near-field ptychography, we report a new imaging modality, termed near-field Fourier ptychography, for tackling high-resolution imaging challenges in both microscopic and macroscopic imaging settings. The meaning of 'near-field' is referred to placing the object at a short defocus distance with a large Fresnel number. In our implementations, we project a speckle pattern with fine spatial features on the object instead of directly resolving the spatial features via a high-resolution lens. We then translate the object (or speckle) to different positions and acquire the corresponding images using a low-resolution lens. A ptychographic phase retrieval process is used to recover the complex object, the unknown speckle pattern, and the coherent transfer function at the same time. In a microscopic imaging setup, we use a 0.12 numerical aperture (NA) lens to achieve a NA of 0.85 in the reconstruction process. In a macroscale photographic imaging setup, we achieve ~7-fold resolution gain using a photographic lens. The final achievable resolution is not determined by the collection optics. Instead, it is determined by the feature size of the speckle pattern. The reported imaging modality can be employed in light, coherent X-ray, and transmission electron imaging systems to increase resolution and provide quantitative absorption and phase contrast of the object.


Introduction
Achieving high spatial resolution is the goal of many imaging systems. Designing a high numerical aperture (NA) lens with diffraction-limited performance over a large field of view remains a difficult task in imaging system design. On the other hand, creating a complex speckle pattern with wavelength-limited spatial features is effortless and can be implemented via a simple random diffuser. In the context of super-resolution imaging, speckle illumination has been used to modulate the high-frequency object information into the low-frequency passband [1][2][3][4][5][6][7][8], similar to the idea of structured illumination microscopy [9][10][11][12]. Typical implementations achieve two-fold resolution gain in the linear region. With certain support constraints, 3-fold resolution gain has also been reported [8]. More recently, we have demonstrated the use of a translated unknown speckle pattern [4] to achieve 13-fold resolution gain for fluorescence imaging [13]. Different from previous structured illumination demonstrations, this strategy uses a single dense speckle pattern for resolution improvement and allows us to achieve more than one order of magnitude resolution gain without the direct access to the object plane. Based on the same strategy, we have also improved the resolution of a 0.1-NA objective to the level of 0.4-NA objective [13], allowing us to obtain both large field of view and high resolution for fluorescence microscopy. Along with a different line, speckle illumination has also been demonstrated for phase retrieval and phase imaging. The developments include speckle-field digital holographic microscopy [14], wavefront reconstruction [15], phase retrieval from far-field speckle data [16], super-resolution ptychography [17], near-field ptychography [18][19][20], among others. In the latter two cases, a translated speckle pattern (i.e., the probe beam) is used to illuminate the complex object and the diffraction measurements are used to recover the object via a phase retrieval process.
Inspired by the concepts of super-resolution ptychography [17] and near-field ptychography [18][19][20], we explore a new coherent imaging modality in this work. The proposed imaging approach, termed near-field Fourier ptychography, is able to tackle high-resolution imaging challenges in both microscopic and macroscale photographic imaging settings. The meaning of 'near-field' is referred to placing the object at a short defocus distance for converting phase information into intensity variations [21]. Instead of directly resolving fine spatial features via a high-NA lens, we project a speckle pattern with fine spatial features on the object. We then translate the object (or speckle) to different positions and acquire the corresponding intensity images using a low-NA lens. Our microscopic imaging setup is shown in Fig. 1(a)-(b), where we use a scotch tape as the diffuser and project the speckle pattern via a high-NA condenser lens. Based on the captured images, we jointly recover the complex object, the unknown speckle pattern, and the coherent transfer function (CTF) in a phase retrieval process. The final achievable resolution is not determined by the collection optics. Instead, it is determined by the feature size of the speckle pattern, similar to our recent demonstration in fluorescence imaging settings [13]. In this work, we achieved a ~7-fold resolution gain in both a microscopic and a macroscale photographic imaging setting. Figure 1(c)-(d) show the resolution improvement of the near-field FP in a microscope setup. The experimental setup, where the object is placed at a short defocus distance to convert complex amplitude into intensity variations. A translated spackle pattern is used for sample illumination and the captured images are used for super-resolution phase retrieval. The captured images of an amplitude (c1) and phase (c2) object under uniform illumination. The recovered intensity of the amplitude object (d1) and the recovered phase of the phase object (d2). In Fig. 1(b), we place the object at an out-of-focus position in our near-field Fourier ptychography implementation (~40 µm defocused). This short defocus distance is able to convert the phase information of the object into intensity variation in the captured images. The meaning of 'near-field' is, thus, referred to this short distance propagation in our implementation. This strategy is similar to the propagation-based in-line holography which makes transparent objects visible in the intensity measurements [22][23][24]. It is also similar to the near-field ptychography implementation which has a high Fresnel number [18][19][20].
The proposed near-field Fourier ptychography is closely related to three imaging modalities: 1) near-field ptychography [18][19][20], 2) super-resolution ptychography [17], and 3) Fourier ptychography (FP) [25][26][27][28][29][30][31][32][33][34][35]. Drawing connections and distinctions between the proposed approach and its related imaging modalities helps to clarify its operation. Near-field ptychography uses a translated speckle pattern to illuminate the object over the entire field of view. It then jointly recovers both the complex object and the speckle pattern in a ptychographic phase-retrieval process. Similar to the near-field ptychography, the near-field FP setup also captures multiple images under a translated speckle illumination. Since near-field FP is implemented based on a lens system, it appears as the Fourier counterpart of near-field ptychography, justifying the proposed name. By implementing it via a lens system, near-field FP is able to bypass the resolution limit of the employed objective lens.
Super-resolution ptychography uses speckle illumination to improve the achievable resolution. With super-resolution ptychography, the illumination probe is confined to a limited region in the object space, leading to a large number of image acquisitions. With near-field FP, we illuminate the object over the entire field of view. The use of a lens system also enables its implementation in a macroscale photographic imaging setting which may be impossible for regular ptychography approaches.
FP illuminates the object with angle-varied illumination and recovers the complex object profile. Near-field FP replaces the angle-varied illumination with a translated speckle pattern. Both FP and near-field FP can be implemented in a macroscale photographic imaging setting [26,29]. FP, however, cannot be used for fluorescence imaging because the captured fluorescence images under angle-varied illumination remain identical. The near-field FP setup, on the other hand, is able to improve the resolution of fluorescence microscopy thanks to the use of non-uniform speckle patterns [4,13]. As such, it can be used in both coherent and incoherent imaging settings to improve the achievable resolution.
In coherent X-ray imaging, it is relatively challenging to generate angle-varied illumination to implement FP [36]. Near-field FP, on the other hand, can be implemented in a regular zoneplate-based transmission X-ray microscope platform without hardware modification. It also allows zone-plate optics to be replaced by total-reflection mirrors [37] or Kinoform diffractive lenses [38], even they are limited to lower NAs or with introduced aberrations. The reported near-field FP may be able to improve the imaging performance of current synchrotron beamline setups and table-top transmission X-ray microscopes.
This paper is structured as follows: in Section 2, we discuss the forward modeling and recovery procedures of the reported approach. Section 3 reports our experimental results on a microscopic imaging setup, where we use a 0.12-NA lens to achieve a NA of 0.77 in the reconstruction process. Section 4 reports our experimental results on a macroscale photographic imaging setup, where we achieve 7-fold resolution gain using a photographic lens. Finally, we summarize the results and discuss future directions in Section 5. The open-source code is provided in the Appendix.

Modeling and simulations
The forward imaging model of the captured image in near-field FP can be described as where ( , ) is the j th intensity measurement (j = 1,2,3…, J), ( , ) is the complex object, ( , ) is the unknown speckle pattern, (xj, yj) is the j th positional shift of the specimen (or the speckle pattern), ( , ) is the point spread function (PSF) of the imaging system, and '*' stands for the convolution operation.
Based on all captured images ( , ), the goal of the near-field FP is to recover the complex ( , ), ( , ), and ( , ) . We have already demonstrated a reconstruction scheme for positive ( , ) and ( , ) in an incoherent imaging setting [13]. In the current framework, ( , ) and ( , ) are complex values and a phase retrieval process is needed for recovery. The proposed recovery process is outlined in Figs. 2-3, where the captured intensity images are addressed in a random sequence, ( , ) and ( , ) represent the updated object and speckle using the j th captured image. In this process, we initialize the amplitude of the object by shifting back the captured raw images and averaging them. We then initialize the amplitude of the unknown speckle pattern by averaging all measurements. Finally, we initialize the PSF using the pre-set defocus distance.  In the reconstruction process (Figs. 2-3), we first shift the updated object ( , ) by ( , ) in the j th iteration, and multiply with the speckle pattern ( , ) to form an exit wave ( − , − ). We then perform a Fourier transform of ( − , − ) to obtain ( , ) in the Fourier space. The resulting ( , ) is multiplied with the defocus coherent transfer function (CTF) to obtain ( , ). We then perform an inverse Fourier transform of ( , ) to obtain the complex amplitude ( − , − ) on the image plane and perform Fourier magnitude projection to obtain ( , ). We use the extended ptychographical iterative engine (ePIE) [39] to update the high-resolution Fourier spectrum ( , )and the defocus CTF in the Fourier domain: Based on the updated ( , ), we perform an inverse Fourier transform to obtain ′ ( − , − ). We then use the regularized ptychographical iterative engine (rPIE) [40] to update the object and the speckle pattern in the spatial domain: where and are algorithm weights in rPIE. We add Nesterov momentum to accelerate the convergence speed in our implementation (typically 2-3 times faster for convergence). The detailed implementation and parameter choice are provided in the Appendix (Figs. 11-14). We have performed several simulations to validate the proposed approach. Figure 4 shows the simulation results that demonstrate the super-resolution phase retrieval concept. In this simulation, the object ground truth is shown in Fig. 4(a). The simulated captured images under speckle and uniform illumination are shown in Fig. 4(b). Figure 4(c) shows the recovered results by placing the object at the 40-µm defocus position. As a comparison, Fig. 4(d) shows the recovered results for the in-focus position. We can see that the amplitude and phase are well-resolved for a short defocus distance, justifying the 'near-field' concept of the proposed approach. The Fourier spectrum of the recovered complex object is shown in Fig. 4(e), where the red dashed circle indicates the original CTF boundary of a 0.12-NA objective lens. The resolution improvement is significant as the recovered Fourier spectrum occupies the entire Fourier space in Fig. 4(e). The computational time is ~6 mins for processing 1681 images with 512 pixels by 512 pixels each using a Dell XPS 8930 desktop computer.
In the second simulation, we analyze the performance of the reported approach with different numbers of translated positions and different noise levels. Figures 5(a)-5(b) show the recovered complex object using different numbers of translated positions. In Fig. 5(c)

Near-field FP for microscopy imaging
The experimental setup for microscopic imaging is shown in Fig. 1(a), where we use a 200-mW 532-nm laser diode coupled with a single-mode fiber as the light source. A scotch tape (diffuser) is placed at the back focal plane of a 0.83-NA condenser lens to generate dense speckle pattern on the object. Since the speckle pattern propagates along the axial direction, no focusing of the condenser lens is needed. At the detection path, we use a low-NA objective lens (4X, 0.12 NA) and a monochromatic camera (Pointgrey BFS-U3-200S6) to acquire the object images with a ~0.1 ms exposure time. Two motorized stages (ASI LX-4000) are used to move the object to different x-y positions with 0.25-0.75 µm different and known step size. The acquisition time for ~1000 images is ~3 mins in our experiment. The captured images are then used to recover the complex object, the unknown speckle pattern, and the CTF of the microscope platform. We use a 40-µm defocus distance to effectively convert the complex amplitude information into intensity variations. A shorter distance may not be enough for the conversion. A longer distance, on the other hand, may lead to a reduction of the effective field of view. In our experiment, we first move the sample into the in-focus position using incoherent light illumination. We then move the sample to a 40-µm distance using a motorized stage. If the estimated defocus distance is not off too much, the CTF updating process can recover the distance numerically. Figure 6 shows the resolution gain of the reported platform using a USAF resolution target. Figure 6(a) shows the captured image under uniform illumination and Fig.  6(b) shows the raw near-field FP image under speckle illumination. Figure 6(c) shows the recovered object with different number of translated positions. The reconstruction quality increases as the number of captured raw images increases and we can resolve group 10 element 5 with a 310-nm linewidth. In our implementation, we can also recover the unknown speckle in a calibration experiment. The pre-recovered speckle pattern can then be used to recover other unknown objects. Figure 6(d) shows the recovered object with the pre-recovered speckle pattern. In this case, we can reduce the number of translated positions from ~1000 to ~300. Based on Figs. 6(c1) and 6(d1), we achieve a ~7.1-fold resolution gain in this experiment.  In the second experiment, we use a quantitative phase target (Benchmark QPT) as the object. Figure 7(a) shows the captured image under uniform illumination and Fig. 7(b) shows the near-field FP raw image under speckle illumination. Figure 7(c) shows the recovered phase by placing the object at the 40-μm defocus distance. As a comparison, Fig. 7(d) show the recovered phase for the in-focus position. We also plot the phase profile along the dash lines in 7(c) and 7(d). For the 40-μm defocus case, the recovered phase profile is in a good agreement with the theoretical phase value provided by the manufacturer (the dark dash line). This experiment validates the 'near-field' requirement of the proposed approach.
In the third experiment, we use a blood smear sample as the object. Figures 8(a) and 8(b) show the captured raw image under uniform and speckle illumination. The recovered intensity and phase of the blood smear are shown in Fig. 8(c), where the features of blood cells can be clearly resolved using our approach. In many microscopy applications, it is important to achieve both high-resolution and wide field of view at the same time. We have also performed an experiment to demonstrate such a capability. In this experiment, we use a 2X, 0.1 NA objective lens to acquire images with a large field of view. Figure 9(a) shows the recovered image of the blood smear, with a size of 6.6 mm by 4.5 mm. Figure 9(b) shows the magnified views of Fig. 9(a), and Fig. 9(c) shows the corresponding raw images under uniform illumination.

Near-field FP for long-range photographic imaging
The proposed near-field FP approach can be used for macroscale photographic imaging. The key innovation is to use a large-aperture lens to project speckle pattern with fine spatial features. We note that, aberration is not an issue for the large-aperture projection lens. The speckle feature size is only determined by the aperture size of the projection lens, regardless of the aberrations. Therefore, one can use a large plano-convex lens to project the fine speckle pattern.
Our experiment setup is shown in Fig. 10(a), where we use a large plano-convex lens (Thorlabs LA1050) and a diffuser (DG20-600) for speckle projection. On the detection side, we use a Nikon photographic lens (50 mm, f/1.8) with a circular aperture to acquire the images. The collection NA is ~0.0006 in our implementation. We then mechanically scan the speckle pattern to 41 by 41 different positions. The captured images are used to recover the highresolution object image. Figure 10(b) shows the raw captured object image under uniform illumination. Figure 10(c) shows the raw near-field FP image under speckle illumination. The recovered object image is shown in Fig. 10(d), where we can resolve the 62-µm line width. Based on Figs. 10(b) and 10(d), we achieve ~7-fold resolution gain using this macroscale photographic imaging setup. Fig. 10. Near-field FP for macroscale photographic imaging. (a) The experimental setup, where we use a diffuser and a large plano-convex lens to project the laser speckles onto the object. We use the same image-plane defocus distance as the microscope setup in Fig. 1(a)
LocationY(1,iShift) = ShiftStepSize*(jCol-SpiralRadius); 47. end In step 6, we generate the low-resolution image sequence by scanning the high-resolution object to different positions. We shift the object by multiplying an equivalent phase factor to the spectrum of the object, and the method is defined in the function 'subpixelshift()'. The lowresolution image sequence is obtained by applying the lowpass filter given by the CTF with the defocus aberration. The obtained low-resolution measurements with a dimension of 257 by 257 by 1521 pixels are stored in 'TargetImgs'. The first generated image is shown in Fig. 13 [m,n,num] = size(input_image); 60. output_image = (input_image); 61.