Ultrafast confocal fluorescence microscopy beyond the fluorescence lifetime limit: supplementary material

each bone by inserting a needle (26 gauge) with a sterile syringe filled with PBS/2% Fetal Bovine Serum (FBS) into one side of the bone. After removing red blood cells by lysis buffer (Sigma-Aldrich), white blood cells were stained with biotinylated anti-Ly6G antibody (RB6-8C5, BioLegend). The cells were secondary-stained with V500-conjugated Streptavidin (BD Biosciences) and Pacific Blue-conjugated anti-CD3ɛ (145-2C11), -CD4 (RM4-5), -CD8α (53-6.7), -B220 (RA3-6B2), -NK1.1 (PK136) antibody (BioLegend). Neutrophils and lymphocytes were sorted by FACSAria IIµ (BD Biosciences) as V500 and Pacific Blue single positive cells, respectively, which had no overlap in excitation and emission spectra on the analysis of our imaging flow cytometry. Neutrophils and lymphocytes were suspended in FBS-free RPMI 1640 culture medium and stained with 2 µM of nucleic acid stain SYTO16 (Thermo Fisher Scientific). The cells were incubated at 37°C for 45 minutes, washed, and re-suspended in FBS-free RPMI 1640. The sample was filtered with a 35-µm mesh strainer (BD Falcon) prior to the imaging flow cytometry experiments. A sheath flow of normal saline solution (0.9% w/v NaCl solution) was used for hydrodynamic focusing.


FLUORESCENCE LIFETIME OF FLUOROPHORES
A plot of the fluorescence lifetime of fluorophores listed in Table 4, Table  5, and Table 7 in Ref. [1], which covers most of commercially available fluorophores, is shown in Fig. S1. The red line represents 0.68 ns, which corresponds to the speed limit of the FDM confocal fluorescence microscope for fluorescence detection with the 400-MHz signal bandwidth (used in our imaging experiments). This plot indicates that the speed of the FDM confocal fluorescence microscope surpasses the fluorescence lifetime of most exogenous fluorophores available in the market.

DETAILED EXPERIMENTAL SETUP
A complete schematic of the experimental setup is shown in Fig. S2, which includes the generation of a QAM-SDC beam, the beam exposure to a sample with scans, and the three-channel detection of transmission/fluorescence signals.

IMAGING SPEED OF CONVENTIONAL LASER-SCANNING CONFOCAL FLUOERESCENCE MICROSCOPES
We theoretically evaluate the fluorescence-lifetime-limited imaging speed of conventional laser-scanning confocal fluorescence microscopes. In general, the point-spread function (PSF) of the laserscanning confocal fluorescence microscope is expressed as where ℎ ex ( , , ), ℎ em ( , , ), and ( , ) represent the intensity PSF of the excitation light, the intensity PSF of the emission, and the aperture function of a pinhole, slit, or no aperture, respectively, , , and correspond to the fast-axis scanning direction, slow-axis scanning direction, and axial direction, respectively, and ⨂ denotes 2D convolution [2,3]. If the fluorescence lifetime of fluorophores is comparable to the pixel dwell time of the microscope, the PSF needs to be modified so that past fluorescence excitation is taken into account, where , ( ), and denote the laser scanning speed (in the x direction), the step function (0 for < 0 and 1 for ≥ 0 ), and the fluorescence lifetime, respectively. By simplifying Eq. (S2), we obtain where ( , , ), ( , , ), and ( , ) represent the OTF of the excitation light, the OTF of the emission, and the Fourier transform of the aperture function ( , ), respectively. , , and represent the spatial frequency in the , , and directions, respectively. Meanwhile, the spatial frequency response function as a function of the fluorescence lifetime is given by which is derived from the rate equation of the fluorescence emission process [4]. Eq. (S4) and Eq. (S5) indicate that ( ) degrades the overall OTF along the axis while it does not change the cut-off spatial frequency (denoted as cut ). Here we define the fluorescence-lifetimelimited scanning speed as limit so that | ( )/ (0)| = 1/2 at the maximum spatial frequency that substantially contributes to the image contrast (e.g., half the cut-off spatial frequency cut ) max . Therefore, The highest temporal frequency component of the obtained fluorescence signal is given by which corresponds to the available signal bandwidth in the FDM confocal fluorescence microscopy at the fluorescence-lifetime-limited speed. Assuming sampling at the Nyquist limit, the pixel dwell time at the fluorescence lifetime limit is found to be dwell = 1 2 limit = √3 ≈ .
(S8) Figure S3 shows numerically calculated OTFs [absolute values of Eq. (S4)] under several conditions of the scanning speed, which illustrates the image degradation in the signal level and the spatial resolution in the x direction due to the fluorescence lifetime. We assumed the identical OTFs for the excitation and emission, max /2 as max , uniform light intensity in a circular pupil, and a pinhole aperture having the Airy disk diameter. The horizontal and vertical axes were normalized by cut and (0) at = 0, respectively.

SENSITIVITY
In the FDM confocal fluorescence microscope, each SDC comb line (excitation beam spot) has a longer pixel dwell time by a factor of than the conventional single-point laser-scanning confocal fluorescence microscope having the same line scanning rate. This improves the signal strength by a factor of based on the assumption that each excitation beam spot in the FDM confocal fluorescence microscope has the same average power as that of the conventional single-point laser-scanning confocal fluorescence microscope, which is a reasonable assumption because the power of a single excitation beam spot is usually limited by the saturation of fluorophores to be ~1 mW while visible-wavelength lasers having much higher power such as >1 W are commercially available. On the other hand, since the shot noise (the noise caused by the stochastic nature of fluorescence photon emission) of each pixel is shared by all other multiplexed pixels, the net shot noise level increases. Therefore, the degree of the SNR improvement depends on the spatial distribution of target fluorophores. For example, if target fluorophores are distributed uniformly in the x direction, the shot noise increases by a factor of , resulting in SNR improvement by (√ ) 2 ⁄ =1 (no SNR improvement). If target fluorophores are localized in the x direction, SNR improvement is obtained by lowering the shot noise level. For example, if fluorophores are localized at a single pixel, the SNR improvement is √ ⁄ = √ . Moreover, if the detector noise is non-negligible, further SNR improvement is obtained. Therefore, overall, the SNR is expected to be improved by FDM. The only case where the FDM is unfavorable in terms of SNR is that weakly fluorescent objects of interest lie in the same y position as strongly fluorescent objects within the FOV of the microscope. In this case, the shot noise caused by the strongly fluorescent objects degrades the image of the weakly fluorescent objects.

SIGNAL PROCESSING FOR IMAGE CONSTRUCTION
The procedures for image reconstruction in the FDM confocal fluorescence microscope are shown in Fig. S4. In the standard procedure shown in Fig. S4(a), we first perform Fourier transformation on the photodetector signal waveform to obtain its frequency spectrum. Next, we extract sub-bands, that is, the frequency band from − /2 to + /2, where and represent the th comb frequency of the SDC beam and the comb spacing, respectively. After shifting the center frequency of each sub-band ( ) to DC, we perform inverse Fourier transformation on the sub-band signal to obtain the complex line profile of each intensity-modulated comb line, which is a mixture of the inphase and quadrature components. In order to specify the phase of the in-phase (zero-phase) and quadrature (90-degree-phase) components of the line profiles, we extract phase values from the phase reference signal, which is simultaneously obtained with the fluorescence signal waveform. The phase values are used as that of each comb line of the phase reference signal. Additionally, we use a phase offset dataset ( ) that represents the phase difference between the in-phase components of the fluorescence signal waveform and the phase reference signal. This dataset is obtained by measuring the phase difference of the fluorescence signal waveform with respect to the in-phase-only SDC beam and the phase reference signal prior to the imaging experiments. This phase offset is mainly caused by the phase delay due to the optical or electrical signal propagation time and the phase delay due to the limited fluorescence lifetime. After all, the th complex line profile is divided by ( + ) so that the real and imaginary parts become inphase and quadrature components, respectively. Finally, by combining the real line profiles, we obtain a fluorescence image.
Alternatively, we developed a phase-reference-free procedure. The procedure and its principles are shown in Fig. S4(b) and Fig. S5, respectively. In this procedure, we assume that the in-phase and quadrature components of each SDC comb line undergoes similar modulation by the fluorophores. This assumption is reasonable if the inphase and quadrature excitation beam spots of the th comb line are located very close to each other, as shown in the inset of Fig. 1(a) as an example, and if expected fluorescence images have a significant number of low spatial frequency components, which holds true in most cases of fluorescence imaging. Under this assumption, the complex line profile consists of the in-phase and quadrature components having nearly identical average intensity, indicating that the average of the complex line profile should have an argument of 45 degrees with respect to the in-phase component as shown in Fig. S5. Therefore, we determine the argument of the average (or sum) of the complex line profile ave and obtain the in-phase and quadrature components as a projection of the profile to ave ∓ 45°, respectively.
The above procedures are so-called digital lock-in detection. For FDM signals, we can also demodulate signals using short-time Fourier transformation (STFT) [5], whose procedure is shown in Fig. S4(c). In this case, we can detect local change in phase values. Therefore, this procedure is suitable for imaging of samples that contain fluorophores with non-uniform fluorescence lifetimes.

CROSS-TALKS IN SDC BEAM GENERATION
The SDC beam is accompanied by cross-talks due to the unwanted interference between deflected beams from the AODs, potentially degrading image quality if the interference is significant. Assuming that the AOD-generated comb lines are evenly spaced in the frequency domain and hence in the spatial domain, typical cross-talks occur as shown in Fig. S6(a) and Fig. S6(b). Specifically, Fig. S6(a) shows crosstalk components caused by the interference between neighboring beam spots from the two different AODs. This type of cross-talks, which we call as the first-order cross-talks, appears as "virtual beam spots" between the desired excitation beam spots and have frequency components in between them. These cross-talk components are mixed with the SDC and degrade FDM microscope images. Next, Fig. S6(b) shows cross-talk components due to the interference between beam spots separated by two comb lines. This type of cross-talks, which we call as the second-order cross-talks, is coherently added to the desired frequency components of the SDC beam. The comb lines of the SDC beam are non-uniform in amplitude, depending on the relative phase between the SDC comb lines and cross-talk components. In both cases [ Fig. S6(a), Fig. S6(b)], there are two cross-talk components that influence each SDC comb line and cross-talk comb line. Higher-order cross-talk components caused by the interference between beam spots separated by comb lines ( > 2) appear in a similar manner to the first-order ( : odd) or second-order ( : even) cross-talks.
We can circumvent the adverse effects of the cross-talks by carefully engineering the phase of the driving signals to the AODs. Under the assumption that all of the AOD-deflected beams have identical power, the odd-order cross-talks can be canceled by satisfying the condition where and ′ are the electric-field amplitude of the th deflected beam from the first and second AODs, respectively. Eq. (S9) indicates that two cross-talk components having the same frequency cancel out due to the destructive interference between them. On the other hand, the even-order cross-talks do not influence image quality if the relative phase between the SDC comb lines and cross-talk comb lines is identical for all of the comb lines. Specifically, to meet the above conditions, we used the phases, (S11) where and ′ are the relative phase of the frequency components in the AOD-driving signals to the two AODs that produce the th SDC comb line. Additionally, under these conditions, the driving signals to the AODs and the photodetector signals have a chirped pulse waveform due to the quadratically incremental phase, which is beneficial in the dynamic range of both the AOD-driving signals and the photodetector signals by properly choosing a value for [6]. In our experiments, we chose 0 = 0 ′ = 0 and = 3.6° for the production of the SDC beam so that the dynamic range of the signal detection was close to the maximum value, /√ , where and are the dynamic range of the photodetector and the multiplexing factor, respectively [7]. As a result, the ratio in signal power between the desired SDC comb lines and the cross-talk comb lines was suppressed down to less than 1% (Fig.  S7).
Alternatively, we can generate the SDC beam by harnessing the odd-order cross-talks as shown in Fig. S6(c). In this case, in order for two cross-talk components to constructively interfere, and ′ need to satisfy the conditions, (S12) (S13) We experimentally employed this SDC beam generation method for evaluating the spatial resolution of the FDM confocal fluorescence microscope.

A. Numerical simulation
The theoretical spatial resolution of our FDM confocal fluorescence microscope is identical to that of conventional laser-scanning confocal fluorescence microscopes with a slit aperture [2,3,8]. In fact, the spatial profile of each excitation beam spot in the microscope (i.e., excitation PSF) is the same as that of conventional laser-scanning confocal fluorescence microscopes because the spot is created as the interference between two focused laser beams having an identical beam profile. The detection PSF in the microscope is also the same as that of    S7. Enlarged cross-talk components of the SDC beam (indicated by the red arrows). This spectrum is the same as the in-phase component shown in Fig. 1(e).
conventional laser-scanning confocal fluorescence microscopes. Theoretical PSFs and OTFs, which are given by the Fourier transforms of PSFs, in the microscope compared with conventional laser-scanning confocal fluorescence microscopes with a pinhole aperture and widefield fluorescence microscopes are shown in Fig. S8, where we assumed identical wavelengths and numerical apertures for all the microscopes and an infinitely small aperture size for the pinhole for conventional laser-scanning confocal fluorescence microscopes and the slit aperture for our microscope. As known in literature [2,3,8], the spatial resolution in the x and y directions in our microscope is comparable to those of wide-field fluorescence microscopes and conventional laser-scanning confocal fluorescence microscopes, respectively. Additionally, the OTF in the z axis in our microscope has a nonzero value just as a conventional laser-scanning confocal fluorescence microscope with a pinhole aperture does, meaning that our microscope has the 3D imaging capability. Note that spatial resolution depends on the type of FDM confocal fluorescence microscopy. For example, FDM confocal fluorescence microscopy in Ref. [7] has lower spatial resolution in the x direction due to its FDM beam generation principles, namely, the interference between localized beam spots and a line focus beam (having a non-localized spatial distribution in the x direction).

B. Evaluation of spatial resolution
To evaluate the PSF of the FDM confocal fluorescence microscope, we obtained 3D fluorescence images of 0.75-μm fluorescent beads. Since the expected resolution and the size of the beads are comparable, we deconvolved the obtained 3D fluorescence image of the beads with an ideal image of the bead, that is, a sphere with 0.75-μm diameter so that we excluded the size effect of the beads on the PSF. By using the Wiener filter as a deconvolution method, we evaluated the spatial resolution as the full width at the half maximum (FWHM) of Gaussian functions used for fitting the PSF profiles along the x, y, and z directions. Experimental results with 20x (NA = 0.75) and 40x (NA = 0.95) objective lenses are shown in Fig. S9. Assuming that the spatial resolution is uniform in the x position, we determined the spatial resolution by averaging images of five different beads at arbitrary positions in the field of view, each of which was obtained by averaging multiple images for noise reduction. This assumption is considered to be reasonable because all the components of the excitation beam and fluorescence beam are equally truncated by the optical components such as the objective lens and galvanometric scanner so that the excitation PSFs and the detection PSFs are uniform in the x direction. As a result, the spatial resolution with the 20x objective lens was found to be 0.69 μm (x), 0.46 μm (y), and 5.8 μm (z), whereas the spatial resolution with the 40x objective lens was found to be 0.45 μm (x), 0.44 μm (y), and 3.5 μm (z).

A. Microfluidic device
The microfluidic device was fabricated via standard photolithographic techniques and molding methods. The mold was fabricated on a silicon wafer using photolithography. A silicone elastomer (Sylgaurd) was poured onto the mold, cured, and baked at 80°C for 4 hours. The baked elastomer was removed from the mold. Holes for both inlets and outlets were bored. The mold and a glass coverslip (Matsunami) were activated with air plasma and brought into contact to bond the materials. The microchannels were heated on a hot plate at 110°C for 15 minutes. Tubing was inserted into the inlets and outlets for the imaging flow cytometry experiments. The microchannel has rectangular dimensions of 48 µm × 80 µm for hydrodynamic focusing in the lateral direction with sheath flow. This microchannel was used for the experiments with lymphocytes and neutrophils. A commercially available glass microfluidic channel with dimensions of 400 µm × 250 µm (Hamamatsu) was used for the experiments with Euglena gracilis. This microchannel is capable of hydrodynamic focusing in both the lateral and depth directions. Suspended cells were introduced into the channel by using a syringe pump at a fixed volumetric flow rate.

B. Preparation of MCF-7 cells
MCF-7 was obtained from DS Pharma Biomedical (EC86012803-F0) and cultured in Dulbecco's Modified Eagle Medium (DMEM) with 10% fetal bovine serum, 1% penicillin streptomycin, 1% non-essential amino acids at 37°C, and 5% CO2. The cells were placed on chamber slides  (Matsunami) and allowed to spread for 3 days. They were stained with 5-µM calcein-AM (Thermo Fisher Scientific) in culture media at 37°C for 45 minutes. Imaging was performed after washing the cells with PBS.

C. Preparation of Euglena gracilis cells
Euglena gracilis NIES-48 was obtained from the Microbial Culture Collection at the National Institute for Environmental Studies [9]. It was cultured in culture flasks (working volume: 20 ml) using an AF-6 culture medium [10]. The culture was maintained at 25°C and illuminated in a 14:10-hour light:dark pattern (approximately 120 μmol photon m-2 s-1). To capture the image in Fig. 2(c), cells were stained with 5 µM of SYTO 16 (Thermo Fisher Scientific) in culture medium followed by incubation without light for 45 minutes. To capture the 3D movie in Fig.  3, cells were stained with 2 µM of SYTO 9 (Thermo Fisher Scientific) in culture medium followed by incubation without light for 30 minutes. In the imaging flow cytometry experiment, a part of Euglena gracilis cells were pre-cultured in a modified AF-6 medium where a nitrogen nutrient was omitted. The cells were cultured in the nitrogen-deficient medium for 5 days and were regarded as cells in the "nitrogen-deficient condition." For the observation of intracellular lipid bodies, a stock solution of 1-mmol BODIPY 505/515 (Thermo Fisher Scientific, USA) in dimethyl sulfoxide containing 1% ethanol was prepared. Both the nitrogen-sufficient and nitrogen-deficient Euglena gracilis cells (~10 6 cells/mL) were stained with 10 µM of BODIPY 505/515 in de-ionized water, incubated without light for 30 minutes, washed, suspended in deionized water, and immediately used for imaging.