One-step deconvolution for multi-angle TIRF microscopy with enhanced resolution

: Total internal reflection fluorescence microscopy (TIRF microscopy) uses a rapid decay of evanescent waves to excite fluorophores within several hundred nanometers ( nm ) beneath the plasma membrane, which can effectively suppress excitation of fluorescence signals in the deep layers. From image stacks obtained with a plurality of different incident angles, a three-dimensional spatial structure of the observed sample can be reconstructed by a Multi-Angle-TIRF (MA-TIRF) algorithm that provides an axial resolution of ~50 nm . Taking into account the point spread function (PSF) of the TIRF microscopes, we further increase its lateral resolution by introducing a fast deconvolution algorithm into the reconstruction of MA-TIRF data (DMA-TIRF), which is approached in just one step of minimizing the reconstruction function. We also introduce a TV regularization term in the deconvolution algorithm to suppress artifacts induced by the excessive noise. Therefore, based on the hardware of existing MA-TIRF microscopes, the proposed DMA-TIRF algorithm has achieved lateral and axial resolutions of ~200 and ~50 nm , respectively.


Introduction
Fluorescence microscopy is instrumental for the visualization of intricate structures and dynamics in cells and animals. Besides resolution, the contrast of the fluorophore excitation determines the ability of microscopy to resolve densely packed structures. As compared to other types of fluorescence microscopy, total internal reflection fluorescence microscopy (TIRF) only excites labeled fluorescent signal close to the glass coverslip, thus provides images of high signal-noise contrast. However, TIRF microscopy only captures two-dimensional images and provides no axial information. On the other hand, fluorescence microscopy techniques such as wide-field microscopy, confocal microscopy and most of super-resolution (SR) microscopy exhibited axial resolution inferior to the lateral axis. By rapidly switching among multiple incident angles of the TIRF illumination (MA-TIRF), it has been shown that an axial resolution of ~50 nm is possible in live cells [1,2]. However, given that raw images are always convolution of fluorescence emission image with the point spread function (PSF) of the microscopy, the lateral resolution in previous studies did not reach the diffraction limit yet, as neither method deblurred the raw image with a deconvolution step. Although a recent paper tried to combine deconvolution into the reconstruction pipeline [3], the authors separated the lateral deconvolution and the axial reconstruction into two steps. The first step is the initial lateral deconvolution of raw images, and then a structure extraction procedure to generate a binary mask. The second step is the multiplication of the binary mask with the axial reconstruction of MA-TIRF to achieve the final MA-TIRF reconstruction. However, given that the binary mask is generated by thresholding the deconvolved images, the final reconstruction results depends on properly adjusting thresholds during the structure extraction procedure. Therefore, this pipeline may suffer from reconstruction artifacts.
Here we have seamlessly incorporated a deconvolution algorithm into the axial reconstruction by including the PSF information in the optimization of the axial information assignment. Equipped also with a Wiener inverse filter and the related regularization terms in our pipeline, our microscopy is able to process raw images of low signal-to-noise ratio but still achieves lateral and axial resolutions of ~200 nm and ~50 nm with few artifacts, respectively. By maximally exploring the potential of graphic processing unit (GPU) using MATLAB, 3D results can be reconstructed from raw images in several seconds, accelerated 5 times compared to running on the CPU alone.

Optical setup
The schematic of DMA-TIRF microscope is shown in Fig. 1, which was based on a commercial microscope (IX81, Olympus) equipped with a TIRF objective (UAPON 100XOTIRF, Olympus). Laser (Sapphire 488LP, coherent) was used as light source. An acoustic optical tunable filter (AOTF) was used to control the intensity of the laser. A collimating lens was used to couple the laser to the single-mode fiber. The output laser beam was collimated using an off-axis parabolic mirror (L1). Then the beam was reflected from a two-axis MEMS (A1B2.2, Mirrorcle Tech), in which the ring illumination TIRF microscopy was generated by two sinusoidal input voltages with a phase difference of 90 degrees. Next the light beam was expanded by Lens 2 (WHN10X, Olympus) and Lens 3 successively. Lens 2 is an eyepiece that can reduce the field curvature by 2 times compared with achromatic lens. The angular dispersion of the incident angle will be as low as ± 0.3° for the incident angle range. The tube lens (L4) focuses the light at the back focus plane of the objective lens (Apo N 100X/1.7 HI Oil, Olympus). The emitted fluorescence was captured by an electron-magnifying charge coupled device (EMCCD iXon 897U, Andor). The whole system was controlled by the software written in LABVIEW.

Forward model with deconvolution
The intensity of evanescent waves I(z, φ, θ) along the illumination depth z, the azimuth φ, and the incident angle θ decays exponentially with the distance from the interface in TIRF microscopy [4]: where I 0 (0, φ, θ) is the illumination intensity at the depth z = 0 [1,4] where n r is the index ratio n r = n 2 /n 1 , n 1 is the reflection index of glass, and n 2 is the reflection index of medium. The parameter d is denoted as: where λ is the excitation wave length.
The emission distribution E(x, y, θ) results from the integration of an object f ′(x, y, θ) excited by the illumination of the evanescent wave: where ρ is a Gaussian distribution of the incident angle α, taking in account the divergence Ω of the Gaussian beam around the angle θ. I(z, φ, α) is the intensity of evanescent wave, considering that incident angle α is a Gaussian distribution around the angle θ. f'(x, y, z) is the sample density of the fluorophores corresponding to the coordinate (x, y, z). Finally, the observed image g′(x, y, θ) captured by the microscopy [1] is a result of the convolution of the emission distribution E(x, y, θ) with the emission PSF according to the coordinate (x, y) and incident angle θ: where g′(x, y, θ) are images captured by the camera with the different incidence angles θ, psf is the emission point spread function of the TIRF microscopy, and the * is the convolution operator. By capturing TIRF images with m different incidence angles, raw data with different θ can be used to reconstruct axial information [1]. Although fluorophores deeper than 300 nm beneath the coverslip were illuminated in our MA-TIRF microscope, we only reconstructed structures within 300 nm away from the coverslip. Given that full width half maximum (FWHM) of the PSF in axial direction is about 500 nm, fluorescent puncta at different depths within the 300 nm plane would not be significantly different in fluorescence profiles. Therefore, 2D PSF is a proper approximation for the deconvolution proposed here. Then the Eq. (5) is shown below: Equation (6) could be discretized along the z direction as follows: where h z,θ is the reconstruction parameter that is computed as the integral of the For a simplified representation of the above equation in m different incident angles θ, the Eq. (7) was represented as follows: is the set of z f ′ at different depths, and H is the n columns and m rows matrix of the parameter h z,θ .

Reconstruction algorithm
To suppress reconstruction artifacts, we introduced the TV regularization and nonnegative constraint in the object function, defined as follows: To efficiently optimize Eq. (9), we proposed an optimization method based on the variable splitting technique (half quadratic splitting) [6]. By introducing an auxiliary variable w, Eq. (9) can be reformulated as follow: where μ is a penalty parameter that was fixed in iterations for each individual sample in our experiments, but varied between different samples. We also introduced two auxiliary variables v and t to decouple the TV regularization and the nonnegative constraint: To optimize Eq. (12), we defined a Bregman distance p Q B associated to a function Q about the variable v and t [7,8]: and where the superscript k is the iteration number. Equation (12), in which D(t) + ∏(v) is replaced by the function Q(v, t), is reformulated as follow: in which Q(v, t) can be optimized as a series of simpler minimization problems in the form of optimizing Bregman distance p Q B : According to the split Bregman method [9], Eq. (17) is equivalent to: (18) Solved from the above Eq. (18), the global scheme of the reconstruction is presented as following iterative scheme: The minimization of the iterative scheme for Eq. (19.1) to Eq. (19.4) is divided as four steps.
The first step is to minimize the objective function of variable w in Eq. (19.1), which can be solved directly as a Wiener filter: where OTF is the discrete Fourier transform of psf, the optical transfer function of the MA-TIRF microscopy; † is the conjugate operator, fft represents the Fast Fourier transform, ifft is the inverse Fast Fourier transform, and c is the parameter of Wiener filter, which was set empirically [10]. The second step is to minimize the objective function of variable f in Eq. (19.2), which is solved directly as: The third step is to minimize the objective function of variable v in Eq. (19.3), which is solved independently using a shrinkage formula via minimization of following function: of which the minimizer is ( ) The last step is to minimize the objective function about TV regularization term in Eq. (19.4), which is not a necessary procedure in the reconstruction. If the noise is low in the raw images, the reconstruction of Eq. (20) to Eq. (23) by Wiener DMA-TIRF is an effective way to suppress artifacts. Otherwise, the TV DMA-TIRF method, of which the objective function contains an additional variable t in Eq. (19.4), which can be solved as a typical optimization of denoising process with TV penalty [9].
We substitute D(t) = ||t x || 1 + ||t y || 1 into this function, and rewrite it into the equivalent form: We use the split Bregman method to optimize this function by introducing a new variable l to approximate the partial derivatives of the TV penalty, and obtained the following unconstraint problem: where γ is the parameter of penalty function. Using the split Bregman method, Eq. (25) can also be rewritten as follows: where i is the sequence of the iterative minimization of the variable t, and the first order partial derivative ( ) in different axis directions are used to reduce the computational complexity in the iteration and could be described as: Through the Split-Bregman method, t could be directly solved as: where x ∇ and y ∇ is the first order derivations operator matrix in x and y direction and was written as [1,1] x ∇ = − and [1,1] Variables l x and l y represent first order partial derivatives in x and y directions of the variable l in Eq. (26), and can be solved by:

Pseudo-code chart of DMA-TIRF
The implementation procedure of our DMA-TIRF algorithm is summarized below.
In our algorithm, the initial T 1 is the tolerance and N k is the maximum number of iterations in the optimization. The reconstruction took ~3 s by Wiener DMA-TIRF method without TV penalty (Wiener DMA-TIRF) and ~328 s by DMA-TIRF method with TV penalty (TV DMA-TIRF) for a desktop PC equipped with a 3.6-GHz i7 processor and a 1060Ti NVidia GPU to reconstruct an DMA-TIRF image stack of 512 × 512 × 10 pixels using the MATLAB procedures provided here.

Simulation experiment
First, we benchmarked our algorithm with MA-TIRF algorithm using simulation with designed ground-truth structures. For simulation, data were excited by 10 different incident angles, convolved with the actual PSF of our TIRF microscope, and then captured by the camera. We added a background value of 99 a.u. to all pixels, in combination with Gaussian noise of a standard deviation at 50 and 500 a.u. to simulate images with low or high noise levels. These images were then reconstructed without (MA-TIRF) and with deconvolution (DMA-TIRF) to verify the validity of reconstruction algorithm on samples with ground-truth. We simulated two types of structures to test the fidelity of the proposed deconvolution pipeline. The first set included an overlay of vesicles, bars and point structures. We simulated a vesicle with lateral diameter of 360 nm, and only the outer membrane was labeled with fluorescence. As shown in Fig. 2(a), although both algorithms were able to reconstruct images of the vesicle at different planes, images obtained with DMA-TIRF exhibited much improved lateral resolution. The second ROI encircled with the solid square is two points, of which the points were separated in the lateral axis by 183 nm (Fig. 2(b)). These points were blurred along lateral axis in the TIRF image and also were blurred in the reconstruction with MA-TIRF, but became separated by the DMA-TIRF (Fig. 2(b) and 2(c)), highlighting the benefit of improving lateral resolution with deconvolution. Finally, we tested the ability of DMA-TIRF in discriminating axial-spaced structures. Three paralleled bars that were separated laterally by 260 nm (Fig. 2(d)), were repeatedly placed at 30 nm and 70 nm depths, and were also detected axially after DMA-TIRF reconstruction (Fig. 2(e)). Similarly, two points placed axially (50 nm in axial distance, red point in Fig. 2(b)) were reconstruct at their correct depths as compared to the ground truth ( Fig.  2(e)), validating the method in correctly reconstructing axial information. In the second experiment, we simulated a curved structure in the three-dimensional space, which mimicked the actin or tubulin filaments in cells (Fig. 3(a)). To simulate the noise in biologic sample, Gaussian noises of a standard deviation at 50 and 500 a.u. were added to the simulation data. Again, DMA-TIRF reconstruction generated an image with better lateral resolution than that reconstructed with the MA-TIRF method (Fig. 3(b) and 3(c)). DMA-TIRF method without TV penalty was able to generate high quality reconstructions in raw image with high SNR (the 2nd column, Fig. 3(b)), but produced a lot of artifacts in reconstructing the raw image of low SNR (iii in Fig. 3(d)). The previous two-step method (deconvolution and structure extraction procedure followed by the axial reconstruction with TV penalty) [3] could provide a similar extension in lateral resolution (Fig. 3(c)). However, in reconstructing raw images with low SNR, it was prone to reconstruction artifacts and exhibited a slightly lower structural similarity index (SSIM) (ii in Fig. 3(d), and Fig. 3(e)). In contrast, using a TV penalty in the DMA-TIRF reconstruction algorithm could suppress the artifacts and improve the reconstruction quality (Fig. 3(d) and 3(e)).

Experiments with biological samples
Actin filaments in live HUVECs were labeled with Lifeact-EGFP, imaged by TIRF microscope of multiple incident angles (Fig. 1), and reconstructed by MA-TIRF and DMA-TIRF methods without penalty. Because actin is mainly distributed in the cytosol, we used a refractive index of 1.36 in the reconstruction [11]. Similar to the simulation, TIRF images were blurred due to convolve with the PSF and structures overlaid in axial direction ( Fig. 4(a)). MA-TIRF reconstruction provided an axial resolution as compared to the TIRF image, however, significant increase in lateral resolution manifested only in the image reconstructed with DMA-TIRF, in which two laterally intersecting actin filaments were clearly separated ( Fig.  4(b) and 4(c)). Reconstruction without deconvolution (MA-TIRF) yielded an axial resolution similar to that obtained with DMA-TIRF, as shown by the images in which different depths (from 0 to 300 nm) were coded with different colors (Fig. 4(d) and 4(e)). Next, we examined clathrin coated pits (CCPs) labeled by clathrin-EGFP in live insulin-secreting INS-1 cells [12]. Due to the excessive noise in the raw image, reconstruction using the MA-TIRF method was corrupted with artifacts ( Fig. 5(a)), which could be relieved by the incorporation of TV penalty (Fig. 5(b)). However, overlapped two CCPs could not be clearly resolved by MA-TIRF related methods. On the other hand, deconvolution without TV penalty (Fig. 5(c)) and with TV penalty (Fig. 5(d)) significantly improved the lateral resolution from ~291 nm to ~195 nm (N = 8) (Fig. 5(e) and 5(g)). Although all methods provided a ~52 nm (N = 10) axial resolution (Fig. 5(f) and 5(g)), only DMA-TIRF method with TV penalty generated high-resolution 3D reconstruction with minimal artifacts.

Method
The HUVECs were isolated and cultured in M199 medium (Thermo Fisher Scientific, 31100035) supplemented with fibroblast growth factor, heparin, and 20% fetal bovine serum (FBS) or in ECM medium containing endothelial cell growth supplement (ECGS) and 10% FBS. The cells were infected with a retrovirus system to express Lifeact-EGFP. The transfected cells were cultured for 24 h, detached using trypsin-EDTA, seeded onto poly-L-lysine-coated coverslips, and cultured in an incubator at 37°C with 5% CO 2 for an additional 20-28 h before the experiments.
INS-1 cells were cultured as described previously [12] and transfected with Clathrin-EGFP using the Lipofectamine 2000 reagent (Thermo Fisher Scientific, 11668019) according to the manufacturer's instructions. After transfection, the cells were detached using trypsin-EDTA, seeded onto poly-L-lysine-coated coverslips, and cultured in an incubator at 37°C with 5% CO 2 for an additional 20-28 h before the experiments.

Discussion
TIRF microscopes can only contain the information of biological sample without axial information because the TIRF raw data is an integration of fluorophore at different depths. By introducing the MA-TIRF microscopy and corresponding reconstruction algorithm, the axial information of the biological sample can be reconstructed. Furthermore, we find that MA-TIRF algorithm sometimes slightly enhance the lateral resolution due to the separation of fluorescent emitters along axial axis. For example, a hollow 3D spherical structure with a 360 nm lateral diameter, which simulated a biological sample labeled with an outer membrane, can only be observed as a solid 2D disc structure by TIRF microscopy (the 1st column of Fig. 2(a)). However, a blurred ring structure manifested in the reconstructed images by MA-TIRF (the 3 rd column 3 rd row of Fig. 2(a)). By incorporating the deconvolution procedure, our DMA-TIRF method could further improve their lateral resolution (the 4 th column of Fig. 2(a)).
Noise seriously affect the fidelity of reconstruction ( Fig. 5(a)). At larger incident angle, the relative signal to noise ratio decreases, which caused reconstructions with many artifacts (Fig.  5(a)). Under such circumstance, we recommended to adjust the parameter c in the Wiener inverse filter in Eq. (20) and incorporate with the TV penalty to remove some of the noise (Fig.  4(e) and Fig. 5(d)).
DMA-TIRF require no hardware changes to the existing MA-TIRF setups [1,13]. Because the imaging speed of the TIRF microscopy can be further improved [14], 100 Hz volumetric imaging shall be possible in the future. With the reduction of exposure time, the signal-to-noise-ratio of the raw images may degrade further. The revised penalty term could be used to suppress the artifacts generated due to excessive noise. The Hessian penalty, a second-order partial derivative penalty, uses a piecewise-approximation of boundaries between regions of different intensities and thus enables globally smooth transitions. In the future, the NLTV [15,16] or deep learning method [17] could also be used to suppress the reconstruction artifacts.

Conclusion
We proposed a one-step reconstruction algorithm DMA-TIRF by taking into account the PSF of the microscopy. The reconstruction of DMA-TIRF provide a higher lateral resolution compared with the state-of-the-art MA-TIRF algorithm [1], which could rapidly reconstruct actin filaments with a 200 nm lateral resolution with minimal artifacts. By including the TV penalty into the objective function to suppress the artifacts generated due to noise, we achieved ~200 nm lateral resolution and ~50 nm axial resolution in reconstructing low signal-to-noise ratio images containing the CCPs.