Improving the space-bandwidth product of structured illumination microscopy using a transillumination configuration

The space bandwidth product (SBP) is a measure to characterize an imaging system, providing the total number of significant samples in the image (pixels) required to represent an image from that system [22]. It can be defined as

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm SBP}=\frac{{\rm FOV}}{\left (0.5 \delta \right)^2} \nonumber \end{align} \tag{ 1 }$$

with $\delta$ as the system's resolution, and the factor 0.5 stemming from the Nyquist–Shannon sampling theorem. For a conventional widefield epifluorescent system, the maximum achievable resolution is

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\delta_{\rm wf}}=\frac{\lambda}{2 {\rm NA}}. \nonumber \end{align} \tag{ 2 }$$

SIM achieves a resolution improvement corresponding to a shift of the OTF in the frequency space as illustrated in figure 1. In conventional SIM, where this shift is limited by the resolution of the generated illumination pattern, and the NA is the same for illumination and collection, theoretically the resolution $\delta_{\rm convSIM}$ is half of $\delta_{wf}$. Considering a constant FOV in the imaging process and the fact that $\mathrm{SBP} \propto 1/\delta^2$ this yields

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm SBP}_{\rm convSIM}=4 \times {\rm SBP}_{\rm wf}. \nonumber \end{align} \tag{ 3 }$$

The spatial frequency of an illumination pattern generated by a conventional imaging objective lens is limited by the objective's NA. For imaging a large FOV it is desirable to use a low magnification and low NA objective lens, e.g. 10$\times$ 0.25 NA, rather than using high-magnification and high NA, e.g. 60-100× 1.4 NA. In the proposed transmission setup as outlined later in section 3.2, we have experimentally obtained illumination fringes with an effective NA of 0.8 over centimetre scale. Thus, when comparing this with an imaging objective lens of 10× 0.25 NA the proposed set-up allows generation of an illumination pattern with spatial frequencies three times higher than this imaging objective lens (10×, 0.25 NA) In such a case, the shift of the OTF and thus the reach towards higher spatial frequencies will be three times as large as well. The final support in the Fourier domain is given by how far the OTF is shifted and its size. Thus shifting the OTF e.g. three times in one direction by its radius (size) will reach to four times its original extent (once for each shift plus once for its size). For this example, the SBP yields:

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{my_first_eqn} \mathrm{SBP}_\mathrm{tSIM} = 4 \times \mathrm{SBP}_\mathrm{convSIM}= 16 \times \mathrm{SBP}_\mathrm{wf}. \nonumber \end{align} \tag{ 4 }$$

Thus the SBP can be increased to four times as compared to a conventional SIM approach if the generated illumination pattern corresponds to three times the NA of the imaging objective. This is 16 times the SBP of conventional wide-field imaging. Both the resolution ($\delta$) and the SBP of the proposed transmission SIM are functions of the imaging objective lens' NA and the highest interference angle (equivalent to the highest NA of the illumination side) formed at the sample stage. Thus, for a given experimental setup, i.e. fixed $\lambda$ and imaging objective lens, the proposed transmission SIM approach supports scalable resolution and SBP simply by altering the interference angles of the multi-mirror setup. However, when it comes to amount of information required to fill the space in the Fourier domain, the larger the shift in the OTF the more orientations will be necessary, as indicated in figure 1. The increase in SBP or resolution can thus be seen as a trade-off with the additional number of necessary images.

A schematic of the proposed experimental setup is presented in figure 2. It consists of a diode-pumped solid-state laser source (BlueMode, TOPTICA photonics, Germany) emitting at a vacuum wavelength of 405 nm. The spatially filtered/cleaned beam is collimated and widened using a telescope. A region with an approximately flat beam profile is selected with a diaphragm and subsequently projected onto a reflective phase only spatial light modulator (Holoeye LETO). The light is divided into the zeroth and two first diffraction orders by a grating pattern displayed on the spatial light modulator (SLM). Figure 2(a) displays how the zeroth order beam is blocked, while the two 1st order beams are reflected at the mirror mounts crossing at the sample plane. By rotating the pattern on the SLM, each pair of opposing mirrors can generate a sinusoidal illumination pattern according to their orientation. The period of the interference pattern $\tau$ depends on the wavelength and angle of mirror pair analogue to the resolution: $\tau=\frac{\lambda}{2\sin\theta}$, with $\theta =90-\alpha$. By changing the angle of mirrors $\alpha$ the period can be manipulated. Here three angles of interference corresponding to three different numerical apertures (NA₁, NA₂, and NA₃) are proposed and studied. Further, by introducing a phase shift to the pattern on the SLM the sinusoidal interference pattern obtains a phase shift (lateral shift) as well. This way the illumination pattern's orientation, periodicity, and phase are controlled.

Zoom In Zoom Out Reset image size

To study the illumination pattern achieved with the presented system, interference patterns are generated and recorded photolithographically so that they can be characterized in a scanning electron microscope (SEM). A glass substrate is spin-coated with a layer of about 1 $\mu$m photoresist (AZ 1505, Microchemicals) and placed at the sample plane, where the interference pattern generated by the mirror mount is inscribed. The diameter where the interference fringes are formed and thus ultimately the FOV of this approach can be increased by adjusting the beam diameter. Three different mirror angles, 23°, 36° and 54°, corresponding to NA₁  =  0.39, NA₂  =  0.59, NA₃  =  0.8, respectively, are utilised for different samples. By rotating the beam, three pattern orientations are also inscribed at different photoresist samples. After laser exposure, the pattern is developed and analyzed under an SEM. The diameter of over 4 mm where the sinusoidal pattern was generated is seen in figure 3(a). Figures 3(b)–(d) display zoomed in sections from the upper left corner, the central region and the lower right corner of figure 3(a) respectively, suggesting a homogeneous pattern. The Fourier spectrum of SEM images as in figures 3(b)–(d) is utilized to determine the pattern period. In figure 4(a), the exemplary Fourier spectrum of such a SEM image is shown. The delta peaks closest to the DC peak in the center stem from the sinusoidal pattern, with their distance to the DC peak indicating the grating period. This analysis rendered values of 510, 330 and 250 nm for NA₁ to NA₃, respectively. Figure 4(b) depicts the results for different NAs and orientations at different regions of the samples. The results show homogeneity of the pattern with minor variations which may be attributed to the photoresist coating. These variations are likely the cause for the higher order peaks seen at figure 4(a). Despite providing a limited comparison with no quantitative information to the visibility of fluorescent fringes, since the exposure time in the photolithographic application is fairly large compared to fluorescence imaging (several seconds versus 10–100 s of milliseconds), the sharpness of the patterns in the photoresist suggests a high temporal stability of the fringes. Further, the sinusoidal pattern is demonstrated without paying attention to polarization. For optimization of the modulation depth, the polarization of a beam could be controlled by, e.g. a waveplate and a Pockels cell [23]. An alternative would be to provide elliptical polarization, which would ensure 50% modulation depth [24].

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Phase shifting of the structured illumination is necessary for the targeted SIM implementation, in order to gather complete information of the sample area. The ability to maintain a stable phase is demonstrated by the fact that the pattern can actually be recorded in photoresist. Since repeated imaging using a shifted pattern is required, the control over the phase shift using the SLM is characterized. For this purpose an interference pattern is generated a low NA and directly projected on a CMOS camera. By introducing phase shift on the SLM, an equivalent phase shift of the pattern on the camera is achieved as presented in figure 5. In figure 5, four phase steps and the intensity profiles are shown, demonstrating the capability of phase control over the extent of the modulation frequency.

Zoom In Zoom Out Reset image size

Following the notation of Wicker et al [25, 26], the image formation in two-dimensional widefield fluorescence microscopy using a sinusoidal illumination pattern can be formulated as

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq:ImageFormation} D(\mathbf{r})=[S(\mathbf{r})I(\mathbf{r})]\otimes h(\mathbf{r}). \nonumber \end{align} \tag{ 5 }$$

The acquired image D is the dye distribution of the fluorescence labeled sample S multiplied with the illumination intensity I and convolved ($\otimes$) with the microscopes point spread function or PSF (h); $\mathbf{r}$ refers to the spatial coordinate. In Fourier space this expression becomes

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \tilde{D}(\mathbf{k})=\left[\tilde{I}(\mathbf{k})\otimes\tilde{S}(\mathbf{k})\right]\tilde{h}(\mathbf{k}). \nonumber \end{align} \tag{ 6 }$$

Tilde (∼) indicates the Fourier transform, and $\mathbf{k}$ is the Fourier space coordinate or spatial frequency. The extend of the optical transfer function (OTF) $\tilde{h}$ is limited to a cutoff frequency ${\|\mathbf{k}\|_2\leqslant k_\mathrm{max}}$. This is a basic property of the imaging objective and results in a limited resolution in the image. If the fluorescence in the sample is excited using a sinusoidal illumination pattern

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq:PatternFormation} I(\mathbf{r})=\sum_{m=-1}^{1} a_m \exp [im(2\pi \mathbf{p} \mathbf{r} + \phi)] \nonumber \end{align} \tag{ 7 }$$

with a modulation depth a_m, a wave vector $\mathbf{p}$ and a phase $\phi$, the Fourier transform of an acquired image can be written as

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \tilde{D}(\mathbf{k}) = \sum_{m=-1}^{1} \exp(im\phi) \underbrace{{a_m\tilde{S}(\mathbf{k}-m\mathbf{p}) \tilde{h}(\mathbf{k})}}_{\tilde{C}_m(\mathbf{k})}. \nonumber \end{align} \tag{ 8 }$$

The Fourier transforms of the acquired images $\tilde{D}(\mathbf{k})$ contain bands $\tilde{S}_m(\mathbf{k}) = \tilde{S}(\mathbf{k}-m\mathbf{p})$ that are shifted by $m\mathbf{p}$ with respect to their original position in Fourier space before they are multiplied with the OTF. This way frequency components of the sample that actually lie outside the OTF support are now transmitted into image space. Taking N images of the sample with different phases $\phi_n$ with $n=1,\cdots,N$, the resulting Fourier transforms of the images $\tilde{D}_n(\mathbf{k})$ can be written as

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \tilde{\mathbf{D}}(\mathbf{k}) = \mathbf{M} \tilde{\mathbf{C}}(\mathbf{k}), \nonumber \end{align} \tag{ 9 }$$

with the acquired images in the vector $\tilde{\mathbf{D}} = [\tilde{D}_{1}, \cdots, \tilde{D}_{N}]^\intercal$, the matrix $\newcommand{\e}{{\rm e}} \mathbf{M}_{nm} = {\exp(im\phi_n)}$ and a vector $\tilde{\mathbf{C}}(\mathbf{k}) = [\tilde{C}_{-1}, \tilde{C}_0, \tilde{C}_+1]^\intercal$. If the inverse of $\mathbf{M}$ ($\mathbf{M}^{-1}$) exists, the different components can be separated by

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq:SepCmp} \tilde{\mathbf{C}}(\mathbf{k}) = \mathbf{M}^{-1} \tilde{\mathbf{D}}(\mathbf{k}). \nonumber \end{align} \tag{ 10 }$$

The final image is the inverse Fourier transform of $\hat{\tilde{S}}(\mathbf{k})$, the final estimate in the Fourier domain. It is obtained by shifting each band to its original position and recombining them using a generalized Wiener filter

$$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle \label{eq:WnrFtr} \hat{\tilde{S}}(\mathbf{k}) = \frac{\sum\nolimits_{m,d} a_m \tilde{C}_{m,d}(\mathbf{k}+m\mathbf{p}_d)\tilde{h}^\ast(\mathbf{k}+m\mathbf{p}_d)}{\sum\nolimits_{m,d} \left| a_{m} \tilde{h} (\mathbf{k}+m\mathbf{p}_{d}) \right|^2 + w} {A}(\mathbf{k}). \nonumber \end{align} \tag{ 11 }$$

The Wiener filter reduces the degrading influence of the OTF and weights the bands in regions where they overlap according to their expected SNR. The Wiener parameter w is determined empirically, $A(\mathbf{k})$ is an apodization function decreasing linearly from unity at the center to zero near the end of the extended OTF support, shaping the overall spectrum in order to prevent ringing artifacts in the final image, and the asterisk ($\ast$) indicates the complex conjugate. Since the resolution improvement only takes place in the direction of $\mathbf{p}$, the process of image acquisition and band separation is repeated for different orientations d to obtain isotropic resolution enhancement.

For the presented simulations, an imaging objective with a numerical aperture of ${\rm NA}=0.25$ is assumed. The excitation and emission wavelength is set to be equal $\lambda=10$ px in the sample space. In figure 6(a), the sample (Siemens star) for the simulations is introduced. Figure 6(b) shows a simulation of the widefield deconvolution and figure 6(c) shows the corresponding Fourier spectrum. The simulated result for plain illumination is generated in the same way as results for structured illumination, just with an interference angle of $0^{\circ}$ generating plane illumination. For the simulation of the raw data for SIM, three interference angles of 23°, 36° and 54° (referred to as NA₁, NA₂, and NA₃) for the generation of the illumination pattern can be used. These interference angles determine the fringe spacing of the sinusoidal illumination pattern. For the orientations, three or six evenly distributed angles with a random overall offset are used. For each direction, a set of three evenly distributed phases, also with a random overall offset, is generated. The raw data images are then simulated according to equations (5) and (7) using a modulation depth of a_m  =  1. The PSF is simulated using a 2D distribution based on the Bessel function of first kind and first order [27]. The reconstruction results as presented in figures 7–9 show the resolution improvement that is to be expected in comparison to figure 6.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Different NAs and orientations for the illumination pattern result in different pattern frequencies and wave vectors $\mathbf{p}$ in the description above. Since the data is acquired consecutively, the different frequency components can be separated one at a time and joined as described in equation (11) with additional terms d in the sum. In order to generate raw data for the reconstructions in figures 8(a) and (b), simulated data only based on NA₂ is used. In figures 8(c) and (d) only NA₃ and in figures 8(e) and (f) only NA₂ and NA₃ are used. This leaves a gap between the low and the high frequency components. This gap results in artifacts in the reconstruction and is clearly visible in the figure. Furthermore, figure 7 shows the reconstructions using only three pattern orientations instead of six in order to create the raw data. This leads to an anisotropic filling of the Fourier space of the reconstruction. Figure 9 shows the expected reconstruction results using only NA₁ (figures 9(a) and (b)), NA₁ and NA₂ (figures 9(c) and (d)) and finally NA₁, NA₂, and NA₃ (figures 9(e) and (f)). The Fourier space of the images is expanded successively and the resolution improves accordingly. There are no gaps in the Fourier space, neither is the Fourier space filled anisotropically.