Fast volumetric imaging with patterned illumination via digital micro-mirror device-based temporal focusing multiphoton microscopy

Temporal focusing multiphoton microscopy (TFMPM) has the advantage of area excitation in an axial confinement of only a few microns; hence, it can offer fast three-dimensional (3D) multiphoton imaging. Herein, fast volumetric imaging via a developed digital micromirror device (DMD)-based TFMPM has been realized through the synchronization of an electron multiplying charge-coupled device (EMCCD) with a dynamic piezoelectric stage for axial scanning. The volumetric imaging rate can achieve 30 volumes per second according to the EMCCD frame rate of more than 400 frames per second, which allows for the 3D Brownian motion of one-micron fluorescent beads to be spatially observed. Furthermore, it is demonstrated that the dynamic HiLo structural multiphoton microscope can reject background noise by way of the fast volumetric imaging with high-speed DMD patterned illumination.


Introduction
Two-photon excited fluorescence (TPEF) microscopy is a powerful and widely utilized tool for three-dimensional (3D) biological imaging, microsurgery, and microstructure fabrication [1][2][3]. Furthermore, the second harmonic generation (SHG) signal can reveal information of non-centrosymmetry directly in specimens without any fluorescent staining, such as in collagen and myosin [4,5]. Through the advantages of natural optical sectioning capability, minimum invasiveness, lower photobleaching and deeper penetration depth, this type of microscopy is suitable when imaging thick tissues for in vivo studies [6]. Both TPEF and SHG are nonlinear optical phenomena that only occur with a high excitation photon density achieved with a high numerical aperture (NA) objective lens to spatially and temporally focus the ultrafast laser beam to generate a strong electromagnetic field. Although only a small region at the focal point of the objective lens where the TPEF and SHG are excited, 3D images can be rendered by using point-by-point lateral and axial scanning. To accelerate the lateral scanning rate, the x-y scanner can be replaced by a quickly-spinning polygon to achieve 30 Hz video rate [7] or via acousto-optic deflectors (AOD) to dynamically control the scanning area to only the regions of interest [8]. In addition to modifications to the scanning mechanism, the multifocal multiphoton microscope separates the laser beam to excite and detect different points simultaneously, thereby reducing the overall scanning time [9]. Axial scanning can be further improved by multiplexing the excitation beam to temporally separate it at different excitation locations [10]. By rapidly modulating the focal point, such as in an electrical lens [11], acoustic lens [12,13], and chirped frequency modulated AOD [8,14], the system could achieve fast z-axis scanning to increase the overall volumetric imaging rate.
Nevertheless, the high frame rate requirement for real-time biological applications is still limited by the point-by-point scanning mechanism [15]. However, the above throughput improvements for nonlinear optical microscopy notwithstanding, temporal focusing multiphoton microscopy (TFMPM) has the advantage of area excitation at a few-micron axial confinement, which is a breakthrough approach with an even higher degree of parallelization (by a factor of 10 2 -10 4 ) [15][16][17][18][19][20]. The diffraction element (e.g., grating) in the system separates the different spectral components of the ultrafast laser pulses into different angles and the spatial dispersion broadens the pulse width. The spectral components overlap in phase only at the focal plane of the objective lens; as such, the pulse width is the shortest for effective multiphoton excitation. It should be noted that the excitation area depends on the initial laser beam size, the diffraction element, and the overall system magnification.
This system allows for high throughput illumination and detection capability, which could be used for widefield fluorescence lifetime imaging [21], large area multiphoton-induced ablation [22], and capturing dynamic events at high-frame rate [19]. Furthermore, by modulating the illumination pattern on the diffraction grating, arbitrary patterned excitation on the specimen can be performed. And, by placing an optical mask at the conjugated plane of the excitation plane, passive lithographic fabrication is possible [23]. A spatial light modulator (SLM) is commonly adopted as the phase mask and can be calculated from predefined intensity patterns at the Fourier plane by the iterative Fourier transform algorithm [18,24]. Alternatively, a phase contrast filter at the confocal plane of the lenses between the SLM and grating can be used to transfer the phase mask to an intensity pattern on the grating [25]. Yet another approach adopts a digital micromirror device (DMD) as the active graylevel pattern generator to directly modulate and select the number of laser pulses at different pixels [26]. Although the DMD diffracts illuminating light frequencies for temporal focusing, which is similar to a grating, it can also generate arbitrary patterns [27]. Since the DMD is placed on the image-conjugate plane of the objective lens' focal plane, the multiphoton excitation pattern can be precisely projected on the focal plane. To demonstrate the improvements to the lateral and axial resolution, nonlinear structured-illumination microscopy is performed via the DMD-based TFMPM [28].
Due to the widefield excitation of TFMPM, additional high-speed axial-scanning and area-detection are needed to enable fast volumetric imaging for directly monitoring 3D biological activities in vivo. Instead of moving the stage axially step-by-step after capturing each frame, group velocity-dispersion manipulation [29] and conjugate image-plane shifting via a dual-prism grating [30] can be adopted for fast remote axial scanning. However, while the excitation plane shifts from the focal plane of the objective, the imaging optical path might need adjustment during scanning [31]. To circumvent this issue, an electrical lens could be used to perform non-mechanical fast axial scanning without correcting the imaging optical path since the emission light and excitation beam pass through the electrical lens [32]. A highspeed piezo-stage could be synchronized with a scientific complementary metal-oxide semiconductor (sCMOS) camera triggered when every z-step position is reached during scanning [33]. Although the sCMOS camera provides a fast frame rate, the quantum efficiency (QE) is less sensitive compared to an electron multiplying charge-coupled device (EMCCD) camera, which has over 90% QE and an electron-multiplying (EM) gain-assisted signal-to-noise ratio (SNR) improvement. In addition, an image intensifier could be further adopted to massively amplify the coming fluorescence signal before entering the camera [33]. The EMCCD camera must acquire images continuously and be able to recognize the current axial position, which pushes the EMCCD frame rate to the limit. In this study, we synchronize the EMCCD acquisition with the focusing piezo-stage to perform fast volumetric imaging. In doing so, we adopt an EMCCD-generated clock, which indicates when the acquisition of every frame starts. Acting as the synchronization clock, the EMCCD-generated clock drives the focusing piezo-stage with a modified sinusoidal waveform based on a field-programmable gate array (FPGA). Furthermore, the DMD is synchronized with the volumetric imaging via the TFMPM to verify the fast pattering capability. Finally, a fast HiLo microscope [34] with structural illumination via the DMD-based TFMPM is integrated to suppress background scattering noise. In this manner, the proposed DMD-based TFMPM can be applied for fast 3D structural illumination and functional excitation applications. To demonstrate the performance, the 3D Brownian motion of one-micron fluorescent beads is shown at 30 volume/sec video rate. Figure 1 shows a schematic diagram of the DMD-based TFMPM. The laser source is a Ti:sapphire ultrafast regenerative amplifier (Spitfire Pro., Spectra-Physics, USA) integrated with a Ti:sapphire ultrafast oscillator (Tsunami, Spectra-Physics, USA) as the seed beam, the center wavelength of which is 800 nm. The regenerative amplifier has a repetition rate of 10 kHz and can provide a peak power of 400 μJ/pulse with a 90 fs pulse width, which is sufficient to excite a TPEF signal with an excitation area larger than 200 × 200 μm 2 [19]. A half-wave plate (HWP) and a polarizing beam splitter (PBS) are used to adjust the laser power while maintaining the horizontal polarization state. To avoid photobleaching the specimen, a fast mechanical shutter (VS14S-2-ZM-0-R3, Uniblitz, USA) blocks the laser beam when acquisition is not in process. The DMD (DLP7000, Texas Instrument, USA) performs the diffraction component of temporal focusing, and has a 0.7-inch illumination area and a 1024 × 768 diagonal micromirror array with a pitch of 13.68 μm. Every micromirror has a fixed mechanical tilt angle of ± 12°, either reflecting (ON) or blocking (OFF) the laser beam, and is equivalent to the blazed angle of the blazed grating. Moreover, the 10th order diffraction light, which has the maximum efficiency based on the diffraction equation, is adopted as the system beam [27]. The DMD diffraction efficiency is equivalent to the 1st order diffraction of 517 lines/mm grating. Furthermore, in addition to uniform TPEF excitation with all DMD pixels "ON", we can easily change the DMD pattern and project to the temporal-focusing excitation plane for structural excitation without inserting any passive optical masks [28]. In other words, the single DMD can diffract illuminating light frequencies for temporal focusing while simultaneously generating arbitrary patterns.

Overall system setup
Together, the L 3 and objective (UPlanSApo60XW/NA 1.2, Olympus, Japan) form a 4f setup and achieve temporal focusing excitation at the focal plane of the objective in an upright optical microscope (Axio imager 2, Carl Zeiss, Germany). The overall system magnification and the diffracted beam size at the back aperture of the objective are adjusted via L 1 , L 2 , and L 3 . By filtering the collected signal through a dichroic mirror and a short-pass filter, only the nonlinear excited optical signal is imaged thorough L im onto the high-sensitivity EMCCD camera (iXon Ultra 888, Andor, UK). The camera has 1024 × 1024 active pixels with a pixel size of 13 × 13 μm 2 and is able to thermal-electric cooling down to −90 °C at a 30 MHz pixel readout rate. By controlling the motorized stage (H101A ProScan, PriorScientific, UK) with a 3-axis encoder or the fast focusing piezo-stage (NanoScanZ 200, PriorScientific, UK) with a maximum of 200 μm travel range, sequential sectional images at different depths can be obtained to reconstruct 3D images. All peripheral instrument communication and control are operated via a high-speed data acquisition (DAQ) card (PCIe-7842R, National Instruments, USA) with a Virtex-5 LX50 FPGA featuring a custom-made LabVIEW program.

System performance and volumetric imaging synchronization design
The system axial resolution is examined by scanning the TPEF fluorescence intensity profile of a poly(methyl methacrylate) thin film doped with Rhodamine 6G dye (< 200 nm thick). Figure 2 shows the measured points and their fitted curve. The estimated axial size of the excitation volume is 2.8 μm at full width at half maximum (FWHM) [29]. The circular excitation area at the focal plane of the objective is currently around 40 μm in diameter. To achieve a volumetric video rate up to 30 volumes per second (vps), a 15 Hz sinusoidal waveform is applied to drive the piezo-stage with multiphoton images acquired at both the rising and declining directions. The EMCCD camera is set in optically-centered crop mode to enable a frame rate of more than 400 frames per sec (fps) with 256 × 256 pixels and 2 × 2 binning. The exposure time is merely 1.96 ms per frame. Besides the frames near the top and bottom of the 15 Hz sinusoidal waveform, more than 10 frames are acquired at the near-linear region of both the rising and declining portions of the sinusoidal waveform. Since the system's axial resolution is better than 2.8 μm, we drive the piezo-stage to have a 30 μm scanning range total, and so the acquired images are 3 μm apart in depth. To achieve fast volumetric imaging, the EMCCD camera is set at its fastest frame rate under no external trigger mode. Further, the piezo-stage is driven by the FPGA-generated sinusoidal waveform for continuous axial scanning instead of step-by-step driving. The driving waveform can be expressed as: where A is the amplitude, f is the waveform frequency, f s is the FPGA sampling frequency used to generate the sinusoidal waveform, p is the phase offset, m is the period of a single volumetric scan, and u[n] is the unit step function. In our system, f is 15 Hz and f s is 50 kHz. The output clock signal, which indicates when the EMCCD camera is exposed, is fed into the FPGA to record the piezo-stage position of every single frame. When 29 images are acquired for a single period sinusoidal piezo-stage scan, the FPGA will reset the driving signal for the next scanning loop, after which the value of m in Eq. (1) can be determined. When adjusting the sinusoidal waveform phase offset, every acquired frame of the rising part of the sinusoidal waveform at the depth position can be expected to be the same as the corresponding frame of the declining part. Continuous image data are received from the EMCCD circular buffer during volumetric imaging, while piezo-stage driving and component synchronization are achieved via the FPGA with its high-precision timing and stability. Currently, the spatial sampling resolution and the volumetric imaging speed are mainly restricted by the EMCCD maximum frame rate with proper pixel numbers.

Volumetric imaging verification
To verify the volumetric imaging stability and repeatability, a 10 μm fluorescent bead (F-8836, Thermo Fisher Scientific, USA) was fixed in agarose gel as the test sample. Since the size of the fluorescent bead is known and the location is fixed, we can show that the axial average fluorescence intensity profile indicates the measured bead size and analyze the scanned images to ensure all acquired image positions are stable during overall volumetric imaging. The fixed 10 μm fluorescent bead was scanned for around 10 sec at 30 vps. Ten frames were acquired at both the rising and declining portions of the sinusoidal waveform via the piezo-stage driving. Figure 3(a) shows the axial average intensity profile after averaging all 244 volumes, with the first 20 volumes excluded for stability. The solid curve in Fig. 3(a) is the fitted Gaussian curve from which the estimated size of the fluorescent bead is 10.4 μm at FWHM. Figure 3(b) plots the directional-acquired frame positions from both directions averaged by 252 volumes. Blue and red circles represent the rising and falling portions, respectively. The z-position standard deviation (std) of all 252 volumes at the frame center is only around 0.02 μm while the maximum std value at the frame edge is 0.23 μm, which is still small compared with the system axial resolution. Moreover, the volumetric imaging mechanism is stable during the 252 volumes for 10 sec. A side view (y-z plane) video of the reconstructed 3D 10 μm fluorescent bead at the 30 volume/sec rate is shown in Visualization 1.The z-position array of every frame at different depths is recorded as a z-axis weighting function when conducting 3D analysis.

3D Brownian motion of fluorescent microbeads
As demonstrated, the high volumetric imaging rate is able to monitor true 3D Brownian motion of a small particle. Random movements in solution due to particle collisions and thermal perturbation can be described by the diffusion equation based on Einstein's theory, the diffusion coefficient of which is defined by the Stokes-Einstein equation [35,36]. For a spherical particle, the diffusion coefficient D is: where k B is Boltzmann's constant, T is the absolute temperature, η is the medium viscosity, and r is the particle radius. The 3D mean square displacement (MSD) at time t can be defined as: One-micron fluorescent beads (F-8888, Thermo Fisher Scientific, USA) were homogenously mixed in deionized (DI) water and sealed in a concave glass slide. According to the onemicron fluorescent bead size at 20 °C DI water, the theoretical diffusion coefficient based on Eq.
(2) was calculated to be 0.49 μm 2 /sec. With the volumetric imaging rate of 30 vps, the theoretical MSD according to Eq. (3) is 313 nm, which is less than the bead radius of 500 nm. Accordingly, the volumetric imaging rate is sufficiently fast to record the 3D Brownian motion of the fluorescent beads without aliasing. The laser excitation power on the specimen is around 4 mW and the exposure time is 1.96 ms per frame. Further, the EM gain was set as 1500/4095. Figures 4(a) and 4(b) show the front-(x-y plane) and side-view (y-z plane) reconstructed images, respectively. Please note that the lateral and axial resolutions of the TFMPM are correspondingly less than 0.4 μm and 2.8 μm; consequently, the one-micron spherical fluorescent beads appear ellipsoidal. The 3D reconstruction movie is shown in Visualization 2. A single fluorescent bead is traced by its center of gravity, the trajectory of which in the x-y-z space with time is given in Fig. 4(c). Figure 4(d) plots the MSD of the trajectory with time (circles) and the fitting line, based on Eq. (3), indicates a diffusion coefficient of 0.69 μm 2 /sec, which is larger than the theoretical value of 0.49 μm 2 /sec. Two reasons might have caused this discrepancy. One is that the theoretical diffusion coefficient is calculated based on the ideal parameters (ex. DI water purity, local temperature in the sample...) while the measured values indicate the real situation during imaging (ex. higher local temperature than expected). The other reason is that the volumetric imaging system operates at only 30 volumes/sec. This means that every volume now consists of 10 slices, which is the limit of the current EMCCD acquisition speed. Accordingly, if a faster acquisition rate was used to reconstruct the 3D Brownian motion of a one-micron bead, the zaxis tracking would be more accurate for analysis. Although the volumetric rate of the TFMPM is sufficient for 3D Brownian motion analysis of one-micron particles, the center of the small particle tracking on the z-axis is problematic due to the axial resolution limit. Less exposure time might be considered as one possible solution.

Fast optical patterning integration
In addition to being used as a diffraction grating in the TFMPM system, the DMD also generates arbitrary patterns for structural illumination. The DMD driver is connected to a PC via a high definition multimedia interface (HDMI). To increase the pattern refresh rate of the DMD, we created a DMD pattern display window based on LabVIEW to increase the imageupdate speed on the display window. To synchronize the DMD with the volumetric imaging mechanism, a trigger from the FPGA is sent out at the beginning of each excitation volume, after which the image on the display window is updated to switch the illumination pattern on the specimen. Since the HDMI timing protocol cannot be controlled in the current setup, a half-speed of the 30 Hz sinusoidal piezo-stage is adopted to switch the illumination pattern. Hence, the volumetric patterning illumination operates at 15 vps. To verify the fast pattering capability, the four different patterns shown in Fig. 5(a) were applied to the DMD iteratively at every excitation volume and excited an auto-fluorescent plastic slide to monitor the illumination pattern changing with time. The blue circles in the top portion of Fig. 5(b) show the average fluorescence intensity inside the blue-dashed square in the bottom-left corner of the first image of Fig. 5(a) at different volume numbers. The red circles in the bottom portion of Fig. 5(b) are the average fluorescence intensity inside the red dashed square in the upper-right corner of the same image in Fig. 5(a). The fast patterning shows no crosstalk between volumes and could be applied for any arbitrary pattern sequence. However, the light intensity decreases with time due to photobleaching induced by the long-term continuous excitation.
It should be noted that the pattering speed is limited by the DMD model and driver used in the current TFMPM setup. Today's state-of-art DMDs can provide higher resolution and faster pattern rates (e.g., DLP LightCrafter 9000, Texas Instruments, USA) and will soon be integrated into the system. Thereafter, 3D real-time functional excitation and a volumetric imaging technique via the fast DMD-based TFMPM will be realized. Such capability offers the potential for neuron network study, high throughput microfabrication and microsurgery, and dynamic structural illumination [28,37,38]. HiLo microscopy provides out-of-focus noise rejection thereby improving the widefield fluorescence image contrast [34]. This technique requires two images for reconstruction. The first image with uniform excitation is used to obtain the high-frequency components containing the in-focus information via a high-pass filter. The low-frequency components can be analyzed from the second image with structural illumination. According to both the lowand high-frequency components, the in-focus image can be reconstructed without the out-offocus background noise. Herein, uniform and sinusoidal excitation patterns were applied to the DMD during the volumetric imaging for fast patterning excitation at 15 vps. The spatial frequency of the sinusoidal pattern on the specimen is 0.76 μm −1 , which equals 20 pixels on the DMD. Subsequently, the HiLo images could be analyzed by Matlab. The specimen is haematoxylin and eosin (H&E) stained back skin from the pathological section of a New Zealand white rabbit. The excitation power is 3.4 mW on the specimen with all DMD pixels "ON" and the exposure time is 1.96 ms per frame. The EM gain was set as 3000/4095 to ensure better SNR. A sinusoidal pattern illuminated on the fluorescent thin film confirms the uniformity, as shown in Fig. 6(a); in comparison, Fig. 6(b) shows the sinusoidal pattern illuminated on the specimen. As can be seen, only the parts on the focal plane are clearly modulated by the pattern while the scattering parts are not. Figures 6(c) and 6(d) show the raw fluorescence image and the reconstructed HiLo image, respectively. Figure 6(e) displays the intensity profiles of the red-dashed lines in Figs. 6(c) and 6(d), which are normalized with respect to the raw fluorescence image (blue curve) and HiLo image (red curve), respectively. As can be seen, the out-of-focus background noise of the specimen's volumetric image is eliminated via the fast DMD-based TFMPM with HiLo microscopy. The rendered 3D images without (left)/with (right) HiLo are shown in Visualization 3.

Conclusions
In this study, a developed DMD-based TFMPM was synchronized with a fast focusing piezostage and EMCCD camera to perform real-time volumetric imaging at 30 vps. With this setup, the 3D Brownian motion of one-micron fluorescent beads can be observed directly without artifactitious aliasing. A DMD was adopted not only as a diffraction grating to achieve temporal focusing, but also as an active pattern generator to modulate the illumination pattern on the specimen. The volumetric imaging mechanism was also synchronized with the DMD to enable dynamic patterning scanning, the rate of which was shown at 15 vps. Moreover, the HiLo technique can be integrated into the DMD-based TFMPM for fast volumetric imaging with less background noise.