The wide-field optical sectioning of microlens array and structured illumination-based plane-projection multiphoton microscopy

We present a theoretical investigation of an optical microscope design that achieves wide-field, multiphoton fluorescence microscopy with finer axial resolution than confocal microscopy. Our technique creates a thin plane of excitation light at the sample using height-staggered microlens arrays (HSMAs), wherein the height staggering of microlenses generate temporal focusing to suppress out-of-focus excitation, and the dense spacing of microlenses enables the implementation of structured illumination technique to eliminate residual out-of-focus signal. We use physical opticsbased numerical simulations to demonstrate that our proposed technique can achieve diffraction-limited three-dimensional imaging through a simple optical design. © 2013 Optical Society of America OCIS codes: (180.6900) Three-dimensional microscopy; (170.0110) Imaging systems; (180.4315) Nonlinear microscopy; (260.1960) Diffraction theory. References and links 1. J. B. Pawley,Handbook of Biological Confocal Microscopy, 3rd ed. (Springer, 2006). 2. W. Denk, J. Strickler, and W. Webb, “2-photon laser scanning fluorescence microscopy,” Science 248, 73–76 (1990). 3. J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. Stelzer, “Optical sectioning deep inside live embryos by selective plane illumination microscopy,” Science 305, 1007–1009 (2004). 4. D. Oron, E. Tal, and Y. Silberberg, “Scanningless depth-resolved microscopy,” Opt. Express 13, 1468–1476 (2005). 5. T. Wilson,Confocal Microscopy (Academic Press, 1990). 6. M. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22, 1905–1907 (1997). 7. B. Masters, P. So, C. Buehler, N. Barry, J. Sutin, W. Mantulin, and E. Gratton, “Mitigating thermal mechanical damage potential during two-photon dermal imaging,” J. Biomed. Opt. 9, 1265–1270 (2004). 8. I. Akira, T. Takeo, I. Katsumi, S. Yumiko, K. Yasuhito, M. Kenta, A. Michio, and U. Isao, “High-speed confocal fluorescence microscopy using a nipkow scanner with microlenses for 3-d imaging of single fluorescent molecule in real time,” Bioimaging4, 57–62 (1996-06). 9. J. Bewersdorf, R. Pick, and S. Hell, “Multifocal multiphoton microscopy,” Opt. Lett. 23, 655–657 (1998). #178762 $15.00 USD Received 31 Oct 2012; revised 23 Dec 2012; accepted 26 Dec 2012; published 18 Jan 2013 (C) 2013 OSA 28 January 2013 / Vol. 21, No. 2 / OPTICS EXPRESS 2097 10. V. Andresen, A. Egner, and S. Hell, “Time-multiplexed multifocal multiphoton microscope,” Opt. Lett. 26, 75–77 (2001). 11. P. J. Keller, A. D. Schmidt, A. Santella, K. Khairy, Z. Bao, J. Wittbrodt, and E. H. K. Stelzer, “Fast, highcontrast imaging of animal development with scanned light sheet-based structured-illumination microscopy,” Nat. Methods7, 637–U55 (2010). 12. J.-Y. Yu, C.-H. Kuo, D. B. Holland, Y. Chen, M. Ouyang, G. A. Blake, R. Zadoyan, and C.-L. Guo, “Widefield optical sectioning for live-tissue imaging by plane-projection multiphoton microscopy,” J. Biomed. Opt. 16, 116009 (2011). 13. C. Ventalon and J. Mertz, “Quasi-confocal fluorescence sectioning with dynamic speckle illumination,” Opt. Lett. 30, 3350–3352 (2005). 14. A. Egner and S. Hell, “Time multiplexing and parallelization in multifocal multiphoton microscopy,” J. Opt. Soc. Am. A 17, 1192–1201 (2000). 15. J. Jahns and K.-H. Brenner, Microoptics: from Technology to Applications, vol. v. 97 (Springer, 2004). 16. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light, 6th ed. (Cambridge University Press, 1997). 17. D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniformillumination microscopy,” Opt. Lett. 33, 1819–21 (2008). 18. D. Lim, T. N. Ford, K. K. Chu, and J. Mertz, “Optically sectioned in vivo imaging with speckle illumination hilo microscopy,” J. Biomed. Opt. 16, 016014 (2011). 19. R. Heintzmann and P. A. Benedetti, “High-resolution image reconstruction in fluorescence microscopy with patterned excitation,” Appl. Opt. 45, 5037–5045 (2006). 20. E. Hecht,Optics, 4th ed. (Addison-Wesley, 2002).


Introduction
Fluorescence is one of the most important optical contrasts in biological imaging, for it offers high molecular sensitivity and specificity.The general desire for obtaining three-dimensional (3D) information of dynamic biological systems has spurred the development of several fluorescence microscopies [1][2][3][4] with the ability to selectively image thin sections of an intact sample.This ability of optical sectioning can be categorized into two general regimes -intrinsic and extrinsic.Intrinsic optical sectioning utilizes an axially peaked excitation profile along the imaging path to obtain optical sectioning; examples include two-photon excitation microscopy (2PE) [2] and selective plane illumination microscopy (SPIM) [3].Extrinsic optical sectioning is performed post-excitation and primarily relies on a far-field imaging principle: the higher spatial-frequency components of the images decay more rapidly with defocusing [5].As a result, one can create a high-spatial-frequency excitation pattern and use appropriate spatial filters to selectively extract the in-focus signal.Confocal microscopy (CFM) [5] and structured illumination microscopy (SIM) are two examples that use physical and numerical spatial filters, respectively, to achieve equivalent optical sectioning [6].
Present imaging techniques, however, have important drawbacks.For CFM and 2PE, phototoxicity and photothermal damages are noted [1,7], and are exacerbated if video-rate acquisition is required.To reduce these effects, spatially/temporally parallelized CFM/2PE systems, such as spinning disk confocal [8], multifocal multiphoton microscopy (MMM) [9], and timemultiplexed multifocal multiphoton microscopy (TM-MMM) [10] were developed.In these techniques, multiple foci are created as independent, parallel channels for excitation in and detection from the sample.To reduce crosstalk among parallel channels, however, the foci spacing has to be sufficiently sparse (typically <1 foci per 10 2 -focus area, see Appendix A, B), which limits the degree of parallelization.SIM and SPIM do not suffer from parallelization issues; however both techniques have their own limitations.SIM can lead to significant photobleaching and low signal-to-noise ratio (SNR) in the processed images, because it excites fluorophores and receives fluorescence over a wide axial range at each acquisition.SPIM requires a compromise between axial resolution and the size of the field of view.Moreover, SPIM raises design issues and challenges in sample handling and manipulation for its separate excitation and imaging objectives in close proximity.In principle, intrinsic and extrinsic optical sectioning can be combined to take the advantages of both.For example, it has been shown that a SPIM system can incorporate SIM to improve its optical sectioning when a reasonably large field of view is desired [11].Here, we propose a hybrid optical microscope design (Fig. 1) incorporating SIM and temporal focusing [4], a widefield, intrinsic sectioning technique.Our design utilizes height-staggered microlens arrays (HS-MAs) having periodic patterns (Fig. 1(b)) to precisely control the time staggering for temporal focusing, and to provide an excitation pattern for SIM to further suppress residual out-of-focus signal.We herein refer to this approach as Microlens Array and Structured Illumination-based Plane-Projection Multiphoton Microscopy (MASI-PPMP).

MASI-PPMP: integrating (intrinsic) temporal focusing and (extrinsic) SIM
We have previously demonstrated a new technique, plane-projection multiphoton microscopy (PPMP) [12], in which temporal focusing with randomized excitation patterns is achieved through the randomly staggered light pulses (in the time domain) created by the surface roughness of a ground-glass optical diffuser.The randomized excitation patterns, as suggested by a previous report [13], could be used for SIM.However, mitigating the excitation patterns in the processed images requires considerable effort.For simplicity, periodic excitation patterns are commonly used [6], and chosen for the present study.Further, the design control afforded in the HSMA allows for optimization of the intrinsic sectioning.
The combination of the microlens arrays and temporal masks, as used in TM-MMM [14], can provide equivalent functions of the proposed HSMAs, although additional precision alignment and positioning are required.Notably, the step sizes of time staggering ∆t in TM-MMM and MASI-PPMP are different.In TM-MMM, ∆t is set to be much larger than the pulse duration τ 0 to reduce temporal inter-foci interferences [14], by which individual foci can be considered as independent, parallel channels.In contrast, MASI-PPMP allows ∆t < τ 0 for practical reasons detailed below and in Supplementary Materials.
We begin the design of MASI-PPMP with consideration of the physical limitations of HS-MAs.The dimensions of HSMAs are limited by fabrication technologies as well as diffractive losses.Existing techniques cannot straightforwardly engineer micro-optics of depth variation >1 mm [15].In addition, when we consider the HSMA as an array of time-delay channels, a light pulse propagating in one channel can leak into its neighboring channels due to the nature where d is the aperture diameter of a single time-delay channel, λ 0 is the central wavelength of the excitation light (here ∼ 800 nm), and ∆h max is the height staggering between the longest and shortest time-delay channels.Accordingly, we restrict ∆h max to ∼ 300 µm and 36 µm ≥ d ≥ 18 µm, via Eq.( 1).These dimensions make the HSMA fabrication feasible through existing techniques, and can create up to ∼ 10 6 foci within a 2-inch aperture, thereby providing the periodic, high-spatial-frequency pattern for SIM.For a simple analysis, we assume that the total amount of time delay ∆t tot is separated equally into N t distinct time-delay steps (with step size ∆t, Eq. ( 2)).These time-delay steps are then arranged in a prescribed periodic pattern in the HSMA (Fig. 1(b)).Considering the propagation speed of light in a material, we have where c and n are the speed of light in vacuum and the refractive index of the material of the HSMAs (set to be 1.5), respectively.Because of the limitation of ∆h max , the estimated ∆t in Eq. ( 2) can be around or shorter than the pulse duration of conventional ultrafast oscillators or amplifiers above certain values of N t .In such cases, we should take into account the temporal interferences among light pulses of different time delays.Notably, temporal masks with much larger ∆t tot have been proposed to avoid temporal interferences and to achieve scanningless TM-MMM [14].However, Eqs. ( 1) and ( 2) indicate that the required ∆h max falls far beyond the limits of existing fabrication techniques, and the aperture sizes of these temporal masks could be too large for standard biomedical microscopes (see Appendix B).
To have fabrication-feasible HSMAs and to improve the SNR of the SIM post-processed images, we turn to optimizing the parameters of HSMAs.In the following sections, we first construct a model that considers the temporal interferences ( §3); through this model we investigate how intrinsic optical sectioning depends on the choice of N t (and ∆t) for a given ∆t tot ( §4), and the arrangement of time delays in the HSMA (Appendix C).Then, we couple HSMAs with SIM to enhance optical sectioning ( §5).

Construct a physical optics-based model taking into account temporal interferences
Given the temporal focusing effect in the proposed technique, and with the non-negligible temporal interferences among the light pulses, a time-independent model such as that previously used for TM-MMM [14] is no longer sufficient for analyzing the performances of our setup.Thus, we develop a new model taking into account the time-dependent optical phase of multiple spectral components in an ultrafast pulse.For simplicity, we use a Gaussian-pulse approach, i.e., in the excited area, the electric field E(r, z,t) at position (r, z) (z = 0 at the specimen plane) and time t is approximated as the Gaussian-weighted sum of a series of constant-interval (in k−space), in-phase light waves, where k 0 is the central wavenumber of the pulse spectrum, and E k j is the scalar field of the light wave of wavenumber k j .To approximate the ultrafast pulse train generated by the amplified (C) 2013 OSA laser system we used previously [12], we set k 0 ≈ 7.85 × 10 4 cm −1 and a pulse duration τ 0 of ≈ 30 fs (by using an appropriate σ k ).We then employ the amplitude point spread function (PSF) derived previously for high numerical-aperture (NA) lenses [14,16], E k j , as where α is the maximal focusing angle θ of the objective lens, and J 0 is the zero-order Bessel function of the first kind.The objective lens used in all the simulations presented here is a 60X oil-immersion lens of NA 1.42.Through Eqs. ( 3) and ( 4) we can numerically evaluate the time-dependent amplitude PSF of an ultrafast pulse focused by a well-corrected objective lens, E PSF (r, z,t).Having solved E PSF numerically, we estimate the electric field near the specimen plane, E SP (r, z,t), as the linear superposition of the E PSF from the individual microlenses, where r m and ∆t m are the central position and time delay of the ultrafast pulse going though the m-th HSMA channel, respectively.An example of numerically simulated E SP (r, z,t) is shown in (Media 1), which also reveals the dynamic process of temporal focusing (see Appendix D).Notably, to fulfill the wide-field illumination condition and to simplify the simulations, our model assumes that a 'unit' microlens array is infinitely replicated in the transverse coordinates, as shown in Fig. 1(b).Under such a periodic condition, the physical optics properties in the projected region of one unit microlens array is sufficient to represent the entire system (the validity and details of this model are given in Appendix E).Through Eq. ( 5), the excitation intensity profiles I SP (r, z) can be derived by integrating the excitation intensity over time, as where n p is the number of photons required in single excitation event (here n p = 2).

Optimize intrinsic optical sectioning through tuning N t and ∆t
To quantify the intrinsic optical sectioning created by a particular design of HSMA, we calculate the total fluorescence signal at a depth z, S(z), through integration of I SP (r, z) in Eq. ( 6) over the transverse coordinates, as Experimentally, S(z) corresponds to the detected fluorescence signal from a thin fluorescent film placed at a depth z.
To determine the choice of (N t , ∆t) that produces the most efficient optical sectioning, we evaluate S(z) and the ratio of out-of-focus and in-focus signal, S out /S in , for various sets of N t and ∆t (constrained by Eq. ( 2)) with two inter-foci spacings, d f oci (≡ d/M, where M is the magnification of the microscopy system) ≈ 0.4 and 0.8 λ 0 , corresponding to ∼ 56 and ∼ 14 foci per 10 2 -focus area, respectively (Fig. 2).Because of the square geometry of our HSMAs, we examine N t = 2 2 , 3 2 , 4 2 , ... and 9 2 .Analyzing S out /S in reveals that the decay of the out-offocus excitation significantly slows down between N t = 25 and 64, corresponding to ∆t ≈ 20-8 fs (i.e., 2/3 τ 0 -1/4 τ 0 ) (Fig. 2(c)).We observe similar trends of S out /S in for various geometrical arrangements of the distinct time-delay steps (see Appendix C).This result suggests that the intrinsic optical sectioning for a fixed ∆t tot is optimized when ∆t is slightly smaller than τ 0 .Further reducing ∆t (equivalent to increasing N t (Eq.( 2))) can complicate the fabrication of the HSMAs without major improvement of the intrinsic optical sectioning.In addition, increasing d f oci leads to weaker out-of-focus excitation and a less complex axial excitation profile S(z) (Fig. 2(b)).

Obtain extrinsic optical sectioning via implementing SIM
Next, we implement SIM to extrinsically remove the residual out-of-focus signal of the optimized HSMA.SIM relies on the post processing of multiple images, each noted as I IM .Because the emitted fluorescence (wavelength assumed to be ∼ 0.56 λ 0 ) from the specimen is generally incoherent, we can estimate I IM from a convolution of the excitation intensity profile I SP and the intensity PSF of the microscopy system, I SY S [5], as where f is the concentration distribution of the fluorophore in the specimen.To quantify the optical-sectioning effect, we assume that the specimen is an ideal thin fluorescent film placed at z = z f , i.e., f (r, z) = δ (z − z f ).Eq. ( 8) then becomes Conventional SIM takes 3 shifts of the 1-dimension periodic pattern; each step is 1/3 of the period of the pattern [6].In our case, the periodicity of I SP is 2-dimensional.We thus use 3by-3 shifts (I IM1 , I IM2 , ..., I IM9 denote the obtained images) and apply SIM post-processing to extract the optically sectioned images, We then substitute I SP in Eq. ( 7) with I SIM to evaluate the overall strength of optical sectioning (Fig. 3).
The results show that MASI-PPMP can achieve a better optical sectioning than CFM.Such results are reasonable because the fundamental mechanisms of intrinsic and extrinsic optical sectioning are independent from each other and can be combined.We further simulate the images of a virtual 3D object obtained by the different imaging techniques (Fig. 4) to illustrate the optical sectioning of MASI-PPMP.Compared with conventional epifluorescence microscopy (Fig. 4(b)), the images obtained before SIM post-processing (denoted as MA-PPMP) shows the ability of intrinsic optical sectioning.In the log-scale intensity plot (Fig. 4(c)), MASI-PPMP has the highest signal contrast between fluorescent and non-fluorescent areas, consistent with the results in Fig. 3.Moreover, the reconstructed 3D views show that MASI-PPMP successfully reproduces the details of the object (Fig. 4(d)).
We should note that the SIM process introduced here, i.e., the 9-frame imaging procedure and Eq. ( 10), is not the only way to remove out-of-focus signal from the obtained images I IM .Other methods using a high-spatial-frequency illumination pattern to distinguish in-focus and out-of-focus signals, such as HiLo microscopy [17,18], may also be applied to the proposed optical setup.These methods use different imaging procedures and post-processing algorithms, which may lead to different imaging properties such as speed and spatial resolutions, such that different methods may be most suitable for different imaging applications.For example, the HiLo method, whose algorithms are more complicated than Eq. 10, requires only 2 frames to retrieve in-focus signal, one frame each with uniform and non-uniform illumination; the reduced number of frames may therefore shorten the imaging time.However, such an advantage exists only when the frame rate of the camera is much lower than the repetition rate of the ultrafast pulse train.If these two rates fall in the same order of magnitude, one will need to increase the exposure time to shift the structural illumination pattern around the sample so as to mimic the effect of uniform illumination, and the overall imaging time will eventually be similar to SIM.

Conclusion
In conclusion, our numerical modeling shows that MASI-PPMP can efficiently provide intrinsic optical sectioning via temporal focusing effect generated by the HSMAs, and achieve an axial resolution finer than CFM via the implementation of extrinsic optical-sectioning technique.Our analysis provides the design guide of HSMAs with small time-delay increments; the dimensions of the proposed HSMAs are compatible with standard biomedical microscopes, and feasible for existing fabrication methods.In contrast to conventional SIM, the intrinsic sectioning of our method can reduce photobleaching and increase the SNR of the processed images.
Compared with parallelized CFM and 2PE, MASI-PPMP has a denser spacing of foci (> 10 foci per 10 2 -focus area), and thus has greater potential in high-speed imaging.In addition to the finer axial resolution than CFM, the lateral resolution can be enhanced via utilization of the periodic excitation structures and Fourier analysis of the obtained images [19].Moreover, MASI-PPMP can use the pulse train generated by ultrafast amplifiers, thereby yielding a sufficient signal level and moderate impact on the specimen at high acquisition rates, as demonstrated in our previous PPMP setup [12].Taken together, we have shown that MASI-PPMP has the potential to achieve (better-than-conventional) diffraction-limited, high-frame-rate 3D imaging with a simple, wide-field optical design.d 0 ) 2 ≈ 99.6%, equivalent to ∼0.4 foci per 10 2 -focus area.To cover the un-illuminated area, Egner et al. [14] proposed to use foci that are largely separated in time, so that the interferences among these foci is negligible even though they partially overlap in space.Consequently, the number of distinct time-delay steps required to cover the un-illuminated area, N t , can be estimated as Given N t , and using the propagation speed of light in the HSMA material, we can estimate the difference of height between the longest and shortest time channels, ∆h max , as: where ∆t, c and n are the step size of the distinct time-delay steps, speed of light in vacuum and the refractive index of the material of the HSMAs, respectively.The appropriate values of ∆t for negligible temporal interferences, as suggested in the previous study of TM-MMM [14], are equal to or larger than twice the pulse duration of the light pulse, τ 0 , i.e., ∆t ≥ 2 τ 0 (where τ 0 was set as ≈ 100 fs for conventional ultrafast oscillators) [14].Accordingly, we find that ∆h ≈ 33 mm and d ≈ 0.15 mm using Eqs.( 15)-( 17), wherein λ and n are assumed to be ∼ 800 nm and 1.5, respectively.As a result, if one needs 1000-by-1000 foci in the field of view (FOV), the typical aperture of the entire HSMA will be as large as 1000×d ≈ 150 mm, which is considerably larger than the optical elements of a standard biomedical microscope.Furthermore, when an ultrafast pulse propagates in a long, narrow glass cylinder, both group velocity dispersion and modal dispersion broaden the pulse width, and thus reduce the axial resolution as well as excitation efficiency.From Eqs. ( 15), (17) and the requirement of ∆t ≥ 2τ 0 , we can derive that ∆h ∝ τ 0 and d ∝ √ τ 0 , indicating that ∆h and d can be scaled down via using small τ 0 .However, commonly available ultrafast systems typically achieve τ 0 ≥ 10 fs, such that the resultant ∆h and d remain considerably large.Moreover, the ultrafast pulses of smaller τ 0 (and thus greater spectral bandwidth) will suffer more from group velocity dispersion and modal dispersion while propagating in the HSMA materials.(C) 2013 OSA the physical microscopy system, we can also use r in f to determine the region wherein the inf-HSMA assumption is valid, as shown in Fig. 8(c).For conventional biomedical microscopes using M = 60X objective lenses, the diameter of the full FOV is typically lager than 300 µm.Thus, the inf-HSMA model is valid in the central region of diameter larger than 200 µm.At the image plane, this region corresponds to a disk of diameter ∼ 12 mm (200 µm×M) or larger, which is able to cover most conventional imaging devices.

#(Fig. 1 .
Fig. 1.(a) Setup of MASI-PPMP.L 1 is the microscope objective lens.L 2 and L 3 are tube lenses of focal length f T .XY stage performs the lateral translations for SIM.The specimen plane is defined as the focal plane of the objective lens L 1 .(b) Illustration of a spiral HSMA used in this study.

#(Fig. 2 .
Fig. 2. (a) Left panel: the intensity distribution I SP at several depths from the focal plane.d f oci is the distance between adjacent foci.The intensity profiles indicated by the yellow line segments are plotted in the right panel.The full widths at half maximum of the intensity peaks are similar to those of conventional 2PE (≈ 0.36λ 0 /NA).(b) Fluorescence signal S(z) of various (N t , ∆t) sets.(c) Upper panel: definition of in-focus signal (S in ) and outof-focus signal (S out ). z HM is the position where half-maximum excitation occurs in 2PE (≈ 0.375 λ 0 ); Lower panel: S out /S in as a function of N t .

#(Fig. 3 .
Fig.3.S(z) of MA-PPMP (i.e., without SIM post-processing, red), CFM (green) and MASI-PPMP (blue) in linear and log scales.The S(z) of confocal microscopy is simulated as the diameter of the confocal pinhole equal to 1 Airy unit×M, where M is the magnification of the microscopy system.The log-scale plot shows that the optical sectioning of MASI-PPMP is better than CFM.The wavelengths of both the emitted fluorescence and the excitation light of CFM are set to be ∼ 0.56 λ 0 .

#Fig. 4 .
Fig. 4. Image analysis of conventional epifluorescence microscopy (Epi), CFM, MA-PPMP (i.e., without SIM post-processing), and MASI-PPMP.(a) The object.(b) The sectioned images obtained by various techniques at the corresponding depth of the virtual slice.The intensity profiles indicated by the yellow line segments are plotted in (c).(d) 3D-view reconstructed from the z-stacked images of Epi and MASI-PPMP.

Fig. 5 .
Fig. 5.A model of two adjacent microlenses.d: aperture of the microlenses.f : focal length of the microlens.d f : the diameter of the focal spot.∆h: height difference between two microlenses.

Appendix C:Figure 6
Figure6shows the simulation results of the ratio between out-of-focus and in-focus signal, S out /S in , using various geometrical arrangements of distinct time-delay steps.As concluded in the main article, these S out /S in curves show that the decays of the out-of-focus excitation slow down significantly when N t = 25-64, (∆t ≈ 2/3 τ 0 -1/4 τ 0 ), although the curves from different patterns of time delays show slight differences in the values of S out /S in .
A simple estimation of the appropriate dimensions of HSMAs is given in Appendix A. In short, to have negligible inter-channel light leakage, the spatial parameters of the HSMAs should satisfy d ≥ λ 0 ∆h max ,