wFLFM: enhancing the resolution of Fourier light-field microscopy using a hybrid wide-field image

We introduce wFLFM, an approach that enhances the resolution of Fourier light-field microscopy (FLFM) through a hybrid wide-field image. The system exploits the intrinsic compatibility of image formation between the on-axis FLFM elemental image and the wide-field image, allowing for minimal instrumental and computational complexity. The numerical and experimental results of wFLFM present a two- to three-fold improvement in the lateral resolution without compromising the 3D imaging capability in comparison with conventional FLFM.

(~λ/2NA) of its wide-field counterpart. In fact, several approaches have been reported to improve the resolution of light-field imaging combining a standard high-resolution image. [16][17][18] However, these developments rely primarily on conventional light-field implementations, and a strategy for FLFM remains unexplored.
Here, we introduce wFLFM, an approach that enhances the FLFM resolution using a hybrid wide-field image. Specifically, as FLFM forms elemental images through each partitioned segmentation of the aperture, the on-axis elemental image contains a consistent orthographic view while at a lower resolution with respect to conventional wide-field microscopy. In this sense, we reason that by replacing this on-axis component with the corresponding highresolution wide-field image, the hybrid elemental images are able to provide enhanced spatial resolving power to FLFM without compromising the capture of angular information and thus the overall volumetric imaging capability.
As illustrated in Fig. 1(a), we constructed the wFLFM system on an epi-illumination widefield microscope (Nikon Eclipse Ti-U) implemented with a 40×, 0.95NA objective lens (Nikon CFI Plan Apo Lambda 40XC) and a 647 nm laser (MPB). The fluorescence emission was collected through a dichroic mirror (T660lpxr, Chroma) and an emission filter (ET700/75, Chroma) and imaged separately from the two portals of the microscope in both wide-field and light-field paths. In the wide-field path, the native image plane (NIP) was recorded using an sCMOS camera (Hamamatsu ORCA-Flash4.0, pixel size P cam = 6.5 μm).
In the light-field path, the NIP was then Fourier transformed in conjugation to the aperture plane of the objective lens using a Fourier lens (f FL = 100 mm). The back focal plane of the Fourier lens was partitioned by an MLA (MLA-S600-f28, RPC Photonics; f ML = 16.8 mm, NA ML = 0.018), forming an array of elemental images on its back focal plane, which was recorded by an sCMOS camera (Andor Zyla 4.2, pixel size P cam = 6.5 μm).
As seen in FLFM, unlike conventional microscopy, the axial depths can be acquired as the variations in the composite spatial frequencies at the Fourier plane of the objective, resulting in lateral displacements of the corresponding images in a radially symmetric manner (except for the on-axis elemental image) [ Fig. 1(b)]. 10,11) Notably, as the aperture is partitioned, the effective NA in the formation of each elemental image is reduced from the objective lens. To overcome the limitation, we took advantage of the wide-field image and implemented the wFLFM approach in four main steps. First, we interpolated the FLFM elemental images to match the effective pixel size (P cam /M = 162.5 nm) of the wide-field image. The corresponding simulated FLFM PSF at the same sampling rate can be generated for deconvolution to obtain a high-sampling reconstruction (termed HS-FLFM). Second, the onaxis elemental image was replaced by a wide-field image of the same field of view that contains in-focus projection of the same axial range with respect to FLFM. Here, we noticed that a better performance can be achieved using a wide-field image after deconvolving the original diffraction-limited wide-field image with its corresponding PSF. Third, the four nearest-neighboring elemental images of the on-axis image were removed to enhance the high spatial frequency components in the reconstruction. The intensity of the remaining elemental images was then scaled down in accordance with the new on-axis component to maintain a feasible weighting in the deconvolution process. Lastly, a hybrid 3D PSF was generated by replacing each axial layer of the on-axis component of the upsampled FLFM PSF by the image of the in-focus wide-field PSF. The intensity of the wide-field PSF was accordingly scaled on each layer to exhibit the same peak intensity of the corresponding FLFM PSF. With these procedures, the volume of the object can be retrieved using the FLFM reconstruction algorithm based on the Richardson-Lucy iteration scheme. 11) To validate wFLFM, we first numerically created a set of objects consisting of 9 rings with the same thickness of 1 pixel and the hollow centers ranging from 2 to 18 pixels at an increasing step of 2 pixels (Fig. 2). Using a pixel size of 162.5 nm, these rings exhibit inner diameters varying between 325 nm and 2.93 μm, i.e. from slightly above the diffraction limit of wide-field microscopy (~358 nm) to above the resolution of FLFM (2.81 μm) [ Fig. 2(a)].
The wide-field image was created by initially convolving the ground truth with the widefield PSF, which result was then deconvolved with the same PSF [Figs. 2(b) and 2(c)]. Next, the light-field images of each object were produced by convolving the ground truth with the PSF of FLFM and then resized to match the sampling of the wide-field image (e.g.  Fig. 3(a)], conventional FLFM can barely resolved the former pair as their separation is close to the FLFM resolution of 2.81 μm [ Fig. 3(g)].
Increasing the sampling rate (i.e. reducing the pixel size from 967 nm to 162.5 nm), HS-FLFM mitigated pixelated depiction of the 1 μm objects with a moderately enhanced resolution of both pairs [ Fig. 3(h)]. In contrast, wFLFM offers a clear visualization of the structures, in consistence with the wide-field image [ Fig. 3(i)].
Lastly, we validated wFLFM using 250 nm dark-red fluorescent beads distributed at two axial depths on an attached glass slide and cover slide, as well as a 6 μm surface-stained dark-red fluorescent microsphere. Here, to verify the usability of the approach on a highresolution FLFM system, we replaced a 100×, 1.45NA objective lens (Nikon CFI Plan Apo Lambda 100×) and a Fourier lens mbi; [(f FL = 75 mm)], which lead to a lateral FLFM resolution of 843 nm. To generate the wide-field image that contains in-focus volumetric projection of the same axial range, we rescaled the intensity of the wide-field focal stack with respect to the corresponding FLFM PSF on each layer, overlaid these wide-field images, and utilized the same procedures as above described to form the hybrid elemental images for reconstruction.
In contrast, the wFLFM measurements showed the corresponding 3D FWHM values of the same objects at 209 ± 28 nm (x), 226 ± 26 nm (y) and 448 ± 28 nm (z) at z = −0.49 μm, and 254 ± 22 nm (x), 250 ± 12 nm (y) and 915 ± 106 nm (z) at z = 2.27 μm. These results, in comparison with conventional FLFM, demonstrated a >2× enhancement in the lateral FWHM values with consistent axial measurements, allowing for an adequate resolution of two nearby beads separated as close as 768 nm located at z = 2.27 μm using wFLFM [Figs. 4(c)-4(i)]. With the 6 μm surface-stained microsphere, the reconstructed images using wFLFM exhibited the hollow spherical structure with a finer surface thickness, in comparison with the results using conventional FLFM [Figs. 4(j)-4(o)]. Measurements of the surface profiles showed the thickness reconstructed by wFLFM at 300-400 nm, a ~2× improvement over the 800-900 nm thickness obtained in the FLFM images. Notably, the reconstructed microsphere exhibited an elongated rugby-like axial pattern, which is consistent with previously reported numerical and experimental FLFM observation using a 40×, 0.95NA objective lens. 11) Finally, it should be mentioned that the wFLFM approach presented such an improvement at no expense in the axial information, mainly because of the fact that only the on-axis elemental image is substituted while the rest of the elemental images that consist of higher spatial frequencies (thus high angular sensitivity) are well maintained in the reconstruction process.
In summary, we have developed wFLFM to provide a 2-3× lateral resolution enhancement for FLFM by combining wide-field images. The principle of wFLFM exploits the nature of the FLFM image formation at the aperture plane, making it readily compatible with many commonly used wide-field modules and thus inducing minimum instrumental and algorithmic complexity. In particular, alternative to the use of focal stacks to form the infocus wide-field image of the volume, the development of various extended depth of focus methods readily permits feasible acquisitions of such a high-resolution wide-field image in a scanningless or inertia-free manner across the DOF of FLFM for reconstruction of volumetric objects. 19) Exploring aperture-partitioning features, further developments are anticipated to achieve resolution enhancement in all three-dimensions, thereby advancing wFLFM as a particularly promising approach for interrogating biology with exquisite spatiotemporal resolving power and 3D capability, as well as at a high scalability, spanning broad molecular, cellular, and tissue systems.  Appl Phys Express. Author manuscript; available in PMC 2021 April 21.