Analog multiplexing of a laser clock and computational photon counting for fast fluorescence lifetime imaging microscopy

The dynamic range and fluctuations of fluorescence intensities and lifetimes in biological samples are large, demanding fast, precise, and versatile techniques. Among the high-speed fluorescence lifetime imaging microscopy (FLIM) techniques, directly sampling the output of analog single-photon detectors at GHz rates combined with computational photon counting can handle a larger range of photon rates. Traditionally, the laser clock is not sampled explicitly in fast FLIM; rather the detection is synchronized to the laser clock so that the excitation pulse train can be inferred from the cumulative photon statistics of several pixels. This has two disadvantages for sparse or weakly fluorescent samples: inconsistencies in inferring the laser clock within a frame and inaccuracies in aligning the decay curves from different frames for averaging. The data throughput is also very inefficient in systems with repetition rates much larger than the fluorescence lifetime due to significant silent regions where no photons are expected. We present a method for registering the photon arrival times to the excitation using time-domain multiplexing for fast FLIM. The laser clock is multiplexed with photocurrents into the silent region. Our technique does not add to the existing data bottleneck, has the sub-nanosecond dead time of computational photon counting based fast FLIM, works with various detectors, lasers, and electronics, and eliminates the errors in lifetime estimation in photon-starved conditions. We demonstrate this concept on two multiphoton setups of different laser repetition rates for single and multichannel FLIM multiplexed into a single digitizer channel for real-time imaging of biological samples.


Introduction
The local environment of a fluorophore changes the spectrotemporal characteristics of fluorescence emission.Fluorescence lifetime imaging microscopy (FLIM) is an essential tool in biological sciences that measures the lifetime of fluorophores by time-resolved detection of fluorescence emission, where the arrival times of emitted photons are recorded to construct a decay curve of photon arrival times (in time-domain detection), whose time constant is the fluorescence lifetime.Fast and accurate quantitative imaging is particularly useful for characterizing the dynamics of autofluorophores in the biological environment.For instance, imaging the cellular autofluorescence intensity and lifetime from metabolic co-factors such as reduced nicotinamide adenine dinucleotide (NADH) and its phosphorylated form (NADPH) or flavin adenine dinucleotide (FAD) can report on the redox state of the biological samples [1][2][3][4][5][6].Despite being significantly weaker than exogenous contrast agents, the dynamic range of autofluorescence concentrations and, consequently, the fluorescence intensities are larger.For instance, the quantum efficiency of free NADH (0.02) is 20× smaller than bound NADH (0.45) [7].The concentration of NADH has been reported to range from 50-900 µM in mammalian cells [8][9][10][11] and up to 2000 µM in fungi [12].Consequently, assuming that the fluorescence intensity is proportional to the product of fluorophore concentration and the quantum efficiency, the brightest and dimmest regions could differ by a factor of 405× (= [900 µM × 0.45] ÷ [50 µM × 0.02]).Additionally, the metabolic dynamics of a cell happen on a timescale of a few seconds, for instance during apoptosis or electrical activity, demanding faster imaging techniques [13,14].Furthermore, a single autofluorophore lacks the specificity to accurately infer the metabolic state of the sample.Since the various autofluorophores differ in their emission spectra, multichannel detection could capture the dynamics of multiple fluorophores simultaneously for enhanced interpretation of the metabolic states of the samples [15].Therefore, the ideal setup for FLIM must be fast and be able to capture a range of fluorescence intensities and lifetimes in multiple spectral channels.
Traditionally, the time delay between excitation and emission is measured using time-correlated single-photon counting (TCSPC) [16], which is slow due to the electronic dead time after a single record of a photon count on a photomultiplier tube (PMT) or a hybrid photodetector (HPD).FLIM techniques such as analog mean delay FLIM [17,18], frequency-domain FLIM [19,20], and direct pulse-sampling FLIM [13,21] have avoided this problem by analog detection electronics or by sampling continuously at gigahertz speeds.The direct-pulse sampling and analog mean delay techniques do not have a linear readout of the fluorescence intensity.In direct pulse-sampling fast FLIM, we had previously described a computational photon counting method called single-and-multi-photon peak event detection (SPEED), which extracted up to five simultaneously incident photon counts with 640-ps dead-time for a digitizer operated at 3.2 GHz and 400 ps for a 5-GHz digitizer [22,23].Though cost-effective for single-photon imaging, frequency-domain FLIM is not convenient for adaptation to multiphoton microscopy with ultrashort pulsed lasers.Fast FLIM with direct pulse sampling and SPEED has the lowest dead-time among photon-counting FLIM techniques, can resolve up to five simultaneously arriving photons on an HPD, and can estimate the fluorescence lifetime of each pixel in real-time on a parallel processor [22].
Unlike TCSPC, fast FLIM does not explicitly sample the laser pulses; instead, the base clock of the digitizer is synchronized to the laser clock through a phase-locked loop (PLL).For an 80-MHz titanium-sapphire laser, we had previously used a frequency divider module [21][22][23][24] to divide the laser clock by a factor of 8, which was provided as an external 10 MHz reference signal to the digitizer.The instant of excitation is inferred from cumulatively adding the photon arrival times from every laser period across multiple excitation pulses (within a pixel or a whole line within a frame) and assuming that the excitation pulse is concurrent with the instance of maximum fluorescence emission (Fig. 1).However, artifacts are induced in the fluorescence lifetime estimation if a pixel/line does not have enough photons to statistically infer the instance of excitation accurately.Consequently, the accuracy of fluorescence lifetime estimation could differ between different parts of a single frame.For instance, in photon-starved regimes, the fluorescence lifetime values are underestimated by SPEED due to the bias in selecting the instance of excitation to a lower value (earlier) where the photon density is maximum (Fig. S1, S2, S3, Supplement 1, Table S1).Furthermore, compared to TCSPC, the GHz digitizers used in fast FLIM and SPEED are relatively expensive and need considerable real-time computing and fast data storage resources.
range of intensity and lifetime values.In addition to these advantages, in a low-repetition-rate multiphoton microscope (5 MHz laser), STAMP enables multispectral fast FLIM within a single channel of a digitizer for a similar cost as a single spectral channel by multiplexing the laser clocks and photocurrents from HPDs and PMTs into the silent regions following an excitation pulse.STAMP improves the accuracy of SPEED for FLIM, particularly in low photon regimes, using both high and low-repetition rate lasers.STAMP enables fast and accurate fluorescence lifetime characterization across a high dynamic range of fluorescence intensities and lifetimes.The sub-nanosecond dead time of SPEED is optimal for probing typical fluorophores and autofluorophores whose mean lifetimes range between 0.1-8 ns [25] since multiple emitted photons from a single excitation pulse could be captured distinctly with minimal dead time between photon detection events.This range of lifetime values also allows minimal "silent" time between two laser pulses for an 80-MHz laser.However, the applications of lower repetition rate lasers have grown in the last few years due to their higher peak energy capable of inducing higher-order optical nonlinearity processes [26].We have previously demonstrated a series of setups called simultaneous label-free autofluorescence and multi-harmonic generation (SLAM) microscopy, which use a pulse-shaped supercontinuum sourced by a 1030-nm (Yb: YAG or Yb: fiber) laser operating at 5-20 MHz for probing autofluorophores and harmonophores in the tissue microenvironment in several spectral channels between 350 and 650 nm [6,[26][27][28].While fast FLIM with SPEED is advantageous for capturing the multiple emitted photons in each spectral window distinctly from a single high-peak-energy excitation pulse, there is a prolonged period of silence between two laser pulses where no photons are expected, which either requires the acquisition digitizer to operate under sub-optimal data packet sizes in the transfer buffer for continuous imaging or the operation of the system with inefficient data throughput, where non-trivial photocurrents are recorded for less than 2% of the data.
We present SPEED with Time-domain Analog-Multiplexing of excitation Pulses (STAMP), which offers the low dead time and multi-photon detection capabilities of fast FLIM with SPEED and the explicit synchronization of photon arrival to the laser clock like TCSPC.In a high-repetition-rate multiphoton microscope (80 MHz laser), the laser clock is multiplexed with a hybrid photodetector without interfering with the fluorescence decay profiles over a large range of intensity and lifetime values.In addition to these advantages, in a low-repetition-rate multiphoton microscope (5 MHz laser), STAMP enables multispectral fast FLIM within a single channel of a digitizer for a similar cost as a single spectral channel by multiplexing the laser clocks and photocurrents from HPDs and PMTs into the silent regions following an excitation pulse.STAMP improves the accuracy of SPEED for FLIM, particularly in low photon regimes, using both high and low-repetition rate lasers.STAMP enables fast and accurate fluorescence lifetime characterization across a high dynamic range of fluorescence intensities and lifetimes.

Optical and electronic setup
Two multiphoton microscopes, adapted from previously described systems [22,24,26], were used for this study.The first setup used a titanium-sapphire laser (InSight X3+, Spectra-Physics), whose tunable output at 80 MHz was operated between 750-900 nm with 170 fs pulses at the sample plane.The beam was scanned with a pair of galvanometer-driven mirrors in a telecentric configuration with 100-1000 laser pulses per pixel.The emitted fluorescence was separated with a 665-nm dichroic mirror (FF665-Di02-25 × 36, Semrock) and focused on an HPD (R10467U-40, Hamamatsu).The detection band spanned between 417-600 nm.The photocurrents were amplified with a 2.5-GHz-bandwidth transimpedance amplifier (HSA-X-2-20, Femto) and collected with a 5-GHz digitizer (ADQ7WB, Teledyne SP Devices).
The second microscope utilized a photonic crystal fiber-generated supercontinuum pumped with a 1045-nm laser (Satsuma, Amplitude Systemes) whose pulse-picked output was operated at 5 MHz (picked from a 40 MHz oscillator).A part of the supercontinuum (1050 ± 60 nm) was compressed using a Fourier-transform pulse shaper (FemtoJock 640) for <80 fs pulse width.A 1.5-kHz resonant galvo mirror was used to scan the fast axis supported by another galvo mirror for the orthogonal direction operated in a telecentric configuration, resulting in one laser pulse incident per pixel during each scan.The two-and three-photon emitted fluorescence was detected across three fluorescence channels: near-UV (320-350 nm), blue (417-477 nm), and orange (593-643 nm) with analog PMTs (H10721-210, Hamamatsu or H7422P-40, Hamamatsu) or with HPDs (R10467U-40, Hamamatsu), amplified by a 1.5-GHz transimpedance amplifier (C5594, Hamamatsu) and collected with a digitizer (ATS9373, AlazarTech) operated at 2 GHz or 3.2 GHz.
For both microscopes, the digitizers and the data acquisition device (PCIe 6353, NI) used for generating the line triggers, frame triggers, and the galvo motion control were synchronized to the laser by dividing the generated laser clock to 10 MHz by phase-locked loop.Data was acquired using a custom LabVIEW program, where a state-machine-based program was used to asynchronously acquire, a process in real-time on a graphical processing unit (Titan X, NVIDIA Corporation), and save the datasets into PCIe-based M.2 drives with large queue buffers for each task for continuous imaging.

Principle of STAMP
SPEED relies on finding the local peaks by comparing each digitized sample to its nearest neighbors on either side and comparing it to photon thresholds (five thresholds for HPD, two threshold values for PMT).If both conditions are satisfied, N photons (depending on the threshold) are assumed to have arrived at that sampling instant.After this, the photons within each laser period are coherently aligned and summed for all laser pulses in a pixel (for the 80 MHz setup with hundreds of pulses per pixel) or a single line in a frame (for the 5 MHz setup).The laser pulse is assumed to have occurred at the peak of this summed decay profile.All photons are aligned to this inferred laser pulse digitally and cumulated across the response of all laser pulses for the pixel and across frames to build the histogram for fluorescence decay.The lifetime values can be estimated using curve fitting to an exponential or using phasor analysis [29].The latter was used throughout this study.
In STAMP, the laser clock, either available at the laser controller inherently from a fast photodiode at the output of the laser or sampled using a fast external photodiode (with a response time of ∼ 1 ns) is multiplexed with the photocurrents using a high-directivity gain block amplifier (PE15A86000, Pasternack) and an analog summer (ZFRSC-183-S+, Mini-Circuits) (Fig. S4).The high-directivity amplifier at the PMT/ HPD prevents any reflections into the transimpedance amplifier and the photodetector that could distort the pulse profile for the next analysis steps.All impedances were matched to 50 Ω to prevent electrical reflections at any terminal.To place the laser clock exactly into the silent region of the fluorescence decay, a delay line consisting of a length of co-axial cable was used between the laser clock photodetector and the analog summer.The selection of the silent region is illustrated in Fig. S5.For the 80-MHz setup, a value of 9 ns was chosen as the delay, which had less than 10% of the fluorescence photons expected beyond this chosen value for a fluorophore with 4-ns lifetime.Indeed, for a 5-MHz setup, less than 0.5% of photons are expected beyond 40 ns, even for a fluorophore with 8-ns lifetime.Instead of inferring the laser clock, the peak-detection algorithm was utilized to detect and localize the laser clock.The delay between the detected laser clock and the actual instance of excitation was calibrated using a strong fluorescent sample (10 mM NADH solution).Subsequently, all photons are aligned to the laser clock shifted by the calibrated value.All photons are aligned to this instant and cumulated across the response of all laser pulses for the pixel and across frames to build the histogram for fluorescence decay (Fig. 1).
Figure 2 shows two configurations of multiplexed signals.For a 5-MHz setup, the laser clock (positive going pulses from the internal photodiode) could be distinctly separated from the emission (negative going pulses from a PMT) since fluorescence photons are only expected in the first few nanoseconds.In this case, the laser pulses were delayed by 106 ns from the instant of excitation (Fig. 2(a)).For an 80-MHz setup, where there are only 12.5 ns between two laser pulses, the laser clock (negative-going pulses from an external photodiode) was delayed by 9 ns from the HPD photocurrent (positive-going pulses).In each case, the opposing polarities enabled easy separation between the two laser pulses (Fig. 2(b)).The typical outputs shown in blue, which were acquired in the other digitizer channel without multiplexing directly from the PMT/HPD, confirm that the multiplexing does not induce any artifacts to these images.The electronic configurations for the various FLIM setups are shown in Fig. S6.
We imaged fluorophore solutions with various fluorescence lifetimes at various concentrations with both setups (Fig. 2(c)-(d), Fig. S7).For all concentrations of NADH solution, both SPEED and STAMP estimate similar lifetime values.Whereas SPEED underestimates the fluorescence lifetime values slightly, and the mean estimated lifetime was slightly different with different concentrations, STAMP consistently measures the lifetime across all concentrations.For fluorophores with various average lifetime values, the fluorescence decays estimated using STAMP are closer to the universal circle of the phasor plots expected of pure fluorophores compared to STAMP in both the 80-MHz and the 5-MHz setups (Fig. 2(c)-(d)).It is also apparent that both SPEED and STAMP estimate the mean lifetime values similar to previously reported values for the same fluorophores [7,30,31].Table S2 shows the estimated mean and standard deviations of the lifetimes with different amounts of spatial binning for the standard fluorophores to mimic different signal levels.When more photons are used in lifetime analysis, the standard deviation of the lifetime decreases consistently, as expected.There is also a noticeable change to the mean lifetime value, and the estimated mean lifetime using STAMP is consistently closer to the expected value.The differences between the fluorescence lifetimes of Rhodamine B using STAMP from the two setups could be from the formation of aggregates of the fluorophore in aqueous solutions, per previously reported research [32].
Indeed, for a 5-MHz setup, less than 0.5% of photons are expected beyond 40 ns, even for a fluorophore with 8-ns lifetime.Instead of inferring the laser clock, the peak-detection algorithm was utilized to detect and localize the laser clock.The delay between the detected laser clock and the actual instance of excitation was calibrated using a strong fluorescent sample (10 mM NADH solution).Subsequently, all photons are aligned to the laser clock shifted by the calibrated value.All photons are aligned to this instant and cumulated across the response of all laser pulses for the pixel and across frames to build the histogram for fluorescence decay (Fig. 1).The reason for SPEED underestimating the fluorescence lifetime is due to assigning more photons to the t = 0 ns bin from the way the clock is inferred as the instance of maximum occurrence of photons.This is akin to having another species in the sample with a lifetime of t = 0 ns.This explains the deviation of the phasor plots from SPEED into the unit circle rather than lying on the unit circle, and the underestimated mean fluorescence lifetime values.The advantage of STAMP is apparent in imaging 0.4 µM Fluorescein solution in ethanol (Fig. 2(e)).Whereas the decays from STAMP are more typical of expected fluorescence decays (single exponential function), the error in photon registration from the low photon counts (<1 Mcps) in SPEED causes the fluorescence decays to appear atypical.

Biological samples
Mice and rats were euthanized by CO 2 asphyxiation and tissues were surgically resected and placed in an imaging dish with a clear cover-glass bottom containing approximately 100 µL of freshly prepared phosphate-buffered saline.The dishes were placed on ice and the tissues were imaged within a few hours of extraction.All animal procedures were conducted in accordance with a protocol approved by the Institutional Animal Care and Use Committee at the University of Illinois Urbana-Champaign.

FLIM of fluorescent beads
The advantages of STAMP lie in imaging spatially sparse samples with low intensities.We first compare the performance of STAMP with SPEED in both optical setups.The beads had a mean diameter of 100 nm and were stained with four different dyes (Blue-Green-Orange-Dark red, TetraSpeck microspheres, T7279, ThermoFisher Scientific).We expect a mixture of fluorescent lifetime values corresponding to which of the fluorophores were excited for the bead.With the 80-MHz setup, the beads were mounted onto a clean cover glass after sonification with an acetone medium and dried overnight (Fig. 3(a)).Figure 3(b) shows a grid of beads imaged with the setup and processed with STAMP and SPEED considering the photons captured in a single frame and all photons captured during 5 serial frames.The beads are shown in ascending order of their fluorescence lifetime values estimated by STAMP.First, the beads were segmented using centroid detection followed by size exclusion for larger blobs.All photons collected from a bead, either in a single frame or across 5 frames, were cumulated to calculate a single mean fluorescence lifetime for each bead and overlayed on a binary mask of the beads.SPEED consistently underestimates the fluorescence lifetime values compared to STAMP through visual observation (orange box).This disparity is apparent in the histograms of the single frame and the 5-frame average shown in Fig. 3(c).The tendency of SPEED to underestimate the fluorescence lifetime is also apparent in Fig. S7.This is a consequence of SPEED's bias towards lower fluorescence lifetime due to computational inference of laser clock instances.It is also notable that the mean squared error between the single-frame lifetime (200 photons per bead on average) and 5-frame lifetime (1000 photons per bead on average) is lower in STAMP (181.7 ps) than SPEED (241.5 ps) (Fig. 3(d)), showing that STAMP performs better in photon-starved regimes.
We also tested the capabilities of the two techniques on dynamic and sparse samples by suspending the beads in water and capturing the lifetime with the 5-MHz setup.Figure 3(e) shows the sum of intensities of 10 frames captured with the setup, with an average of 62 photons per bead.The segmented beads appear elongated due to motion within the frames Fig. 3(f).While the difference between the estimated fluorescence lifetime is not as dramatic as the 80-MHz setup (Fig. 3(h)), the fluorescence decay profiles shown from STAMP (orange, R 2 value to an exponential fit of 0.988) appear more exponential-like than the decay profiles estimated from SPEED (blue, R 2 value of 0.861) (Fig. 3(g)).These results highlight that STAMP can image sparse samples more accurately than SPEED, has more similar performance between photon-starved and photon-rich regimes, and overcomes SPEED's tendency to underestimate the fluorescence lifetime.

Biological imaging on the 80-MHz setup using STAMP
We tested the performance of STAMP for imaging biological tissues with high dynamic range with the 80-MHz setup by imaging rat muscle (1-10% photon rates) and rat tail tissue (50-300% photon rates) at different depths (Fig. 4).In collagen-rich (type 1) tissue such as the rat tail (Fig. 4(a)), the expected lifetime is close 0 ns due to the dominant second harmonic generation and the relatively weaker collagen autofluorescence, which was not spectrally filtered in our acquisition.However, due to the nature of the inferred laser clock, SPEED consistently estimates the lifetime values as ∼250 ps.However, by explicit registration of photons to the laser clock in STAMP, the lifetime values are lower (150 ps) (Fig. 4(b)).This was observed to be consistent across all depths (up to 50 µm) with minimal changes to the mean fluorescence intensity.It should be noted that the 50-300% photon rate (corresponding to 40-240 Mcps photon count rate) is beyond the detection capabilities of all single-detector TCSPC setups due to the considerable red, TetraSpeck microspheres, T7279, ThermoFisher Scientific).We expect a mixture of fluorescent lifetime values corresponding to which of the fluorophores were excited for the bead.With the 80-MHz setup, the beads were mounted onto a clean cover glass after sonification with an acetone medium and dried overnight (Fig. 3a).Fig. 3b shows a grid of beads imaged with the setup and processed with STAMP and SPEED considering the photons captured in a single frame and all photons captured during 5 serial frames.The beads are shown in ascending order of their fluorescence lifetime values estimated by STAMP.First, the beads were segmented using centroid detection followed by size exclusion for larger blobs.All photons collected from a bead, either in a single frame or across 5 frames, were cumulated to calculate a single mean fluorescence lifetime for each bead and overlayed on a binary mask of the beads.SPEED consistently underestimates the fluorescence lifetime values compared to STAMP through visual observation (orange box).This disparity is apparent in the histograms of the single frame and the 5-frame average shown in Fig. 3c.The tendency of SPEED to underestimate the fluorescence lifetime is also apparent in Fig. S7.This is a consequence of  dead time.STAMP consolidates the advantages of both fast FLIM with SPEED and TCSPC, offering accurate lifetime estimates at high photon flux.
We had previously shown that due to loss in the signal-to-noise ratio in the deeper regions of weakly fluorescent samples, fast FLIM tends to underestimate the fluorescent lifetime values [24].We observe this in the images of the rat muscle (1-10% photon rate) (Fig. 4(c)).Closer to the surface where the photon flux is higher, SPEED and STAMP estimate similar lifetime values.However, STAMP estimates lifetime values closer to that of the surface from the deeper regions (Fig. 4(d)).There are also more locations (yellow arrows) in the STAMP-derived FLIM images where the fluorescence lifetime values are greater than 800 ps, which is characteristic of muscle tissues [24].The phasor plots for these results are shown in Fig. S8.
These results highlight the accuracy of STAMP over SPEED both in high and low photon flux imaging conditions.Additionally, despite the short silent region in the 80-MHz setup, there are no differences in the number of photons estimated using the setup with STAMP or SPEED or any observed artifacts induced by multiplexing.

Multichannel FLIM using STAMP on the 5-MHz setup
The key innovation in STAMP is using the silent regions of fluorescence lifetime measurements more effectively.In lower repetition rate lasers, one could multiplex more than the laser clock into the silent region between two laser pulses where practically no photons are expected.Figure 5 shows an example of the photocurrents from two HPDs and a PMT (H10721-210) multiplexed SPEED's bias towards lower fluorescence lifetime due to computational inference of laser clock instances.It is also notable that the mean squared error between the single-frame lifetime (200 photons per bead on average) and 5-frame lifetime (1000 photons per bead on average) is lower in STAMP (181.7 ps) than SPEED (241.5 ps) (Fig. 3d), showing that STAMP performs better in photon-starved regimes.
We also tested the capabilities of the two techniques on dynamic and sparse samples by suspending the beads in water and capturing the lifetime with the 5-MHz setup.Fig. 3e shows the sum of intensities of 10 frames captured with the setup, with an average of 62 photons per bead.The segmented beads appear elongated due to motion within the frames Fig. 3f.While the difference between the estimated fluorescence lifetime is not as dramatic as the 80-MHz setup (Fig. 3h), the fluorescence decay profiles shown from STAMP (orange, R 2 value to an exponential fit of 0.988) appear more exponential-like than the decay profiles estimated from SPEED (blue, R 2 value of 0.861) (Fig. 3g).These results highlight that STAMP can image sparse samples more accurately than SPEED, has more similar performance between photonstarved and photon-rich regimes, and overcomes SPEED's tendency to underestimate the fluorescence lifetime.

Biological imaging on the 80-MHz setup using STAMP.
We tested the performance of STAMP for imaging biological tissues with high dynamic range with the 80-MHz setup by imaging rat muscle (1-10% photon rates) and rat tail tissue (50-300% photon rates) at different depths (Fig. 4).In collagen-rich (type 1) tissue such as the rat tail (Fig. 4a), the expected lifetime is close 0 ns due to the dominant second harmonic generation and the relatively weaker collagen autofluorescence, which was not spectrally filtered in our   We demonstrated this concept by imaging mouse kidney and liver samples in the three channels spanning near-UV to orange regions of the electromagnetic spectrum (Fig. 6).Upon first observation of the fluorescence lifetime images (Fig. 6a,f), the near-UV channel has the lowest fluorescence lifetime which was expected since this spectral window would be dominated by third harmonic generation (THG) from 1050±60-nm excitation.The blue channel has the highest fluorescence lifetime values (1600 ps on average), which corresponds to the mean fluorescence lifetime of a mixture of free and bound NAD(P)H with peak emission at 450 nm upon three-photon excitation.This channel is also expected to capture autofluorescence from lipids (425-550 nm), which is apparent in the liver samples.This also accounts for the comparatively higher fluorescence intensity in this channel in the liver sample.The orange channel is dominated by two-photon autofluorescence from flavoproteins, primarily FAD, which has a bound fluorescence lifetime of <100 ps in its bound form.Demultiplexing the signals does not induce any crosstalk between these channels.
The fluorescence lifetime in the THG-dominated channel is expected to be low, where the performance between STAMP and SPEED are similar and consistent (Fig. 6c,h).The calculated histograms, which only consider the fluorescence lifetime values of binned pixels with more photons than the number of frame averages considered, appear symmetrical and the median fluorescence lifetime curves are coincident.Like the trends in Fig. S1-S3, the number of photons available dictates the variance estimated by the fluorescence lifetime values, with the lower bound dictated by the sample content.For the blue channel in both images that are in the photon-starved regime, STAMP consistently avoids underestimating the fluorescence lifetime values (Fig. 6d,i) unlike SPEED.The same trends are observed in the weaker yellow (Fig. 6e) channel in the kidney, but not in the photon-rich regime of the liver samples (Fig. 6j).Notably, the stripe artifacts that appear on SPEED due to incorrect registrations of the photons within a line are not apparent in the FLIM images reconstructed with STAMP (white arrows).See Fig. S10 for all histograms.These results highlight that multichannel STAMP has better data throughput, shows fewer artifacts, and is more accurate in photon-starved regimes compared to SPEED with the same dead time and minimal additional cost.Fig. 5. Multichannel STAMP on a 5-MHz system.Plots of the average decays of all pixels in 50 arbitrary lines from an image of cactus leaf showing the peaks from the laser clock (grey, at 0 ns), the PMT at the near-UV wavelengths (pink, at 39.0625 ns), the HPD at blue wavelengths (blue, at 81.25 ns), and the HPD at the orange wavelengths (yellow, at 118.125 ns (or -81.875 ns).The images on the right show the photocurrents averaged across the specified periods.The photocurrents for the yellow, pink, and blue regions are shown after subtracting the "background," indicated as the average photocurrent of the entire period.
with the laser clock in a single digitizer channel.We had previously described a setup where the two channels of the ATS9373 digitizer were operated at 2 GHz each (maximum for dual-channel operation) for fast FLIM using PMTs on two simultaneous channels [26].Multiplexing all signals to a single channel allows for operating the digitizer at 3.2 GHz matching the Nyquist rate for the amplifier bandwidth.All photodetectors were amplified using a C5594 transimpedance amplifier (1.5 GHz) and the laser clock was derived from the internal photodiode at the output of the laser.Since all components were impedance matched to 50 Ω, the gain block amplifier was neglected from this setup.The channels were multiplexed using a 4-way resistive power splitter (ZFRSC-4-842-S+, Mini-Circuits) with a 20 dB attenuator on the laser clock and the PMT's amplified photocurrent to have nearly equal amplitudes for each signal.Without the laser clock amplifier, there are a few reflections of electronic signals through the setup, which causes the average of the "gray" area in the plot to have artifacts that resemble the sample structure but do not correspond to any single channel.This artifact is also apparent when the average photocurrent of the entire range is visualized for the entire 200-ns period in Fig. 5.The residual noise, when no photons are incident on the detectors, appears to be consistent for each laser pulse, indicating a systematic origin (Fig. S9).Therefore, this average value for each pulse was subtracted from each sample in the yellow, pink, and blue regions, which corrects for these background signals.New thresholds for SPEED and STAMP for photocurrent to photon count conversions were calibrated on solutions after this background correction for the following results.After correction, the distinct information in each channel could be separated.Once the delays were calibrated from bright samples with known decay profiles (fluorophore solutions), the separation of these spectral components was performed on a GPU in real-time.This also reduces the data size from 640 samples (corresponding to 200 ns) to 144 samples (15 ns × 3) for saving and further processing.The gap between the two signals was kept at approximately 40 ns using a co-axial cable with a known velocity of transmission (0.83× speed of light, provided by the manufacturer) of 10 m (∼40 ns) or 20 m (∼80 ns) lengths (LMR-240, Amphenol Times Microwave systems).
We demonstrated this concept by imaging mouse kidney and liver samples in the three channels spanning near-UV to orange regions of the electromagnetic spectrum (Fig. 6).Upon first observation of the fluorescence lifetime images (Fig. 6(a),(f)), the near-UV channel has the lowest fluorescence lifetime which was expected since this spectral window would be dominated by third harmonic generation (THG) from 1050 ± 60-nm excitation.The blue channel has the highest fluorescence lifetime values (1600 ps on average), which corresponds to the mean fluorescence lifetime of a mixture of free and bound NAD(P)H with peak emission at 450 nm upon three-photon excitation.This channel is also expected to capture autofluorescence from lipids (425-550 nm), which is apparent in the liver samples.This also accounts for the comparatively higher fluorescence intensity in this channel in the liver sample.The orange channel is dominated by two-photon autofluorescence from flavoproteins, primarily FAD, which has a bound fluorescence lifetime of <100 ps in its bound form.Demultiplexing the signals does not induce any crosstalk between these channels.
The fluorescence lifetime in the THG-dominated channel is expected to be low, where the performance between STAMP and SPEED are similar and consistent (Fig. 6(c),(h)).The calculated histograms, which only consider the fluorescence lifetime values of binned pixels with more photons than the number of frame averages considered, appear symmetrical and the median fluorescence lifetime curves are coincident.Like the trends in Fig. S1-S3, the number of photons available dictates the variance estimated by the fluorescence lifetime values, with the lower bound dictated by the sample content.For the blue channel in both images that are in the photon-starved regime, STAMP consistently avoids underestimating the fluorescence lifetime values (Fig. 6(d),(i)) unlike SPEED.The same trends are observed in the weaker yellow (Fig. 6(e)) channel in the kidney, but not in the photon-rich regime of the liver samples (Fig. 6(j)).Notably, the stripe artifacts that appear on SPEED due to incorrect registrations of the photons within a line are not apparent in the FLIM images reconstructed with STAMP (white arrows).See Fig. S10 for all histograms.These results highlight that multichannel STAMP has better data throughput, shows fewer artifacts, and is more accurate in photon-starved regimes compared to SPEED with the same dead time and minimal additional cost.

Discussions and conclusions
The results presented here show that STAMP has advantages of both TCSPC and fast FLIM with SPEED with the explicit photon registration to the laser clock and the low detection dead time with high maximum photon rate, respectively.The cost to implement STAMP into an existing SPEED setup (Fig. 2-4) was less than $1100; multichannel STAMP was implemented onto an existing SPEED setup for less than $500 (See Table S3 for a cost-breakdown of all parts).We have shown that STAMP works even with the 80-MHz laser with very short silent windows for fluorophores with longer and shorter lifetimes.One restriction of fast FLIM has been regarding the 10 MHz reference clock that must be derived from the laser clock to be used to synchronize with the digitizers and DAQs.Both lasers used in this study had repetition rates that were quadrisectable or quinquesectable multiples of 10 MHz and could, therefore, be synchronized to the electronics.However, there are popular commercial lasers such as the Fidelity-2 (Coherent, 70 MHz) or Flint (Light Conversion, 11/76 MHz) that do not meet this criterion.STAMP may offer a pathway to implement fast FLIM on these lasers without the need for PLL-based synchronization.We intend to explore this and the ability of STAMP to handle laser jitter in the future.Additionally, to improve the dead-times of TCSPC-based FLIM, photodetectors with multiple active detection elements, such as SPAD arrays, have been used.The multichannel STAMP implementation described in this setup could also be used for such configurations where several active detection elements could be multiplexed into a single analog channel.Considering the digitizers used in this project have dual input channels, up to 7 channels (and one clock) channel could be multiplexed into the digitizer in the 5-MHz setup simultaneously in future studies.This could increase the photon count rate significantly.In each implementation of STAMP in this study, all the components, including the cables, amplifiers, and digitizer inputs were impedance matched to minimize reflections.However, there was no active noise cancellation element present here.For devices like the HPDs with high bias and source voltages, any current reflections back into the

Discussions and conclusions
The results presented here show that STAMP has advantages of both TCSPC and fast FLIM with SPEED with the explicit photon registration to the laser clock and the low detection dead time with high maximum photon rate, respectively.The cost to implement STAMP into an existing SPEED setup (Fig. 2-4) was less than $1100; multichannel STAMP was implemented device could cause damage to the device.In the single-channel configuration, the high directivity amplifier coupled with the addition of the delay line to the laser clock, rather than the HPD, helped minimize any reflections into the HPD/ PMT.In the multi-channel configuration, to avoid the added noise from additional electronic elements and having no choice but to add the delay lines to the output of the transimpedance amplifier created additional noise in the analog detection.This was corrected computationally in this study using background subtraction.Future studies will explore statistically custom electronics to combine the transimpedance and high directivity amplifiers into a single unit to minimize this noise.Additionally, the laser clock in the 5-MHz setup was derived directly from the laser controller, for which we had no control over the noise characteristics.Using a low-noise high-bandwidth external photodiode could further minimize the systematic noise (Fig. S9) in future studies.
The multichannel fast FLIM with STAMP has unique scalability compared to existing multichannel FLIM setups for low-to-medium repetition rate lasers (where the pulse period is >50 ns).Multichannel acquisition with SPEED requires multiple digitizer channels that increase the overall data sizes streamed from the digitizer.Multichannel TCSPC needs the purchase of additional modules.Conversely, multichannel STAMP not only needs minimal electronics but also improves the information efficiency of the overall setup.The additional computational resources for multichannel FLIM are also minimal.However, this is all contingent upon the presence of a silent region where negligible photons are expected.For the 80-MHz setup, for any fluorophore with a lifetime of over 4-ns, the percentage of photons in the silent region is greater than 10% (Fig. S5).While this range of lifetime values is typical for autofluorescence of NAD(P)H or FAD in biological samples, it restricts the applications of this setup to imaging fluorophores with longer lifetimes.However, the same argument could be made for any 80-MHz setup, since any fluorophore with a lifetime of greater than 6-ns will have more than 10% of photons beyond a single laser period.In this study, the photons are registered to the closest laser pulse preceding it.Future studies will consider statistically driven registration algorithms to consider several preceding laser pulses as appropriate registration candidates per photon based on prior knowledge of fluorophore lifetime ranges.The 40-ns delays used in this study for the 5-MHz setup are sufficiently long for most fluorophores (Fig. S5).However, some applications may need fluorophores with long fluorescence lifetimes (>10 ns), for instance, to study the steady-state anisotropy of molecules [33].Although the delay between the different channels is relatively easy to control with coaxial cable connections, we will explore electronic methods for programmable delay of laser clocks and photocurrents in future studies using STAMP.
In this paper, we presented STAMP as a new fast FLIM method with explicit registration to the laser clock and sub-nanosecond dead times that can measure a large range of fluorescence lifetime values across a large range of photon emission rates quickly and in real time.We demonstrated its utility in autofluorescence measurements with a high dynamic range of intensities and lifetimes.STAMP has minimal additional costs compared to SPEED, increases the accuracy of SPEED, and improves the overall information throughput efficiency in multichannel configuration.STAMP is versatile, as evidenced by its compatibility with different lasers, photodetectors, amplifiers, and digitizers.STAMP represents an advancement in fast FLIM that drastically improves its adaptability for commercial applications.Acknowledgments.The authors would like to thank Carlos A. Renteria and Eric J. Chaney for helping with the animal studies in this paper and Kayvan F. Tehrani and Alejandro de la Cadena for their assistance with the photonic crystal fiber.Additional information can be found at: https://biophotonics.illinois.edu.
Disclosures.JES, RRI, and SAB have a provisional patent application on the SPEED algorithm through the University of Illinois at Urbana-Champaign related to the SLAM technology.SAB and HT have disclosed and patented intellectual property through the University of Illinois Urbana-Champaign related to the SLAM technology.The authors declare no other conflicts of interest.

Fig. 1 .
Fig. 1.Principles of fast FLIM with SPEED and STAMP.The processing steps involved in estimating the fluorescence lifetime for each pixel from SPEED (left) and STAMP (right) by collecting and tagging the emitted photons from excitation pulses consequently or across frames.

Fig. 1 .
Fig. 1.Principles of fast FLIM with SPEED and STAMP.The processing steps involved in estimating the fluorescence lifetime for each pixel from SPEED (left) and STAMP (right) by collecting and tagging the emitted photons from excitation pulses consequently or across frames.

Fig. 2 .
Fig. 2. Demonstration of STAMP on two different setups.a-b.Multiplexed (orange) and original photocurrents (blue) for a 5-MHz setup (a) and an 80-MHz setup (b) imaged simultaneously on the two channels of the digitizer.The shaded regions indicate the times where the plots are zoomed in corresponding to 1-4 laser periods.c-d.Phasor plots of standard fluorophores imaged using the 5-MHz (c) and the 80-MHz (d) setup, with the contours from STAMP in solid lines and from SPEED in dotted lines after 3x3 spatial binning.The solutions and their previously reported fluorescence lifetimes measured are also listed [7,30,31].e. Fluorescence decay profiles from imaging 0.4 μM Fluorescein using 850-nm excitation pulses on the 80-MHz setup with and without multiplexing of the laser clock.

Fig. 2 .
Fig. 2. Demonstration of STAMP on two different setups.a-b.Multiplexed (orange) and original photocurrents (blue) for a 5-MHz setup (a) and an 80-MHz setup (b) imaged simultaneously on the two channels of the digitizer.The shaded regions indicate the times where the plots are zoomed in corresponding to 1-4 laser periods.c-d.Phasor plots of standard fluorophores imaged using the 5-MHz (c) and the 80-MHz (d) setup, with the contours from STAMP in solid lines and from SPEED in dotted lines after 3 × 3 spatial binning.The solutions and their previously reported fluorescence lifetimes measured are also listed[7,30,31].e. Fluorescence decay profiles from imaging 0.4 µM Fluorescein using 850-nm excitation pulses on the 80-MHz setup with and without multiplexing of the laser clock.

Fig. 3 .
Fig. 3. Imaging the lifetime of fluorescent beads with STAMP and SPEED.a. Multiphoton image showing the average of 5 frames (estimated with STAMP) using the 80-MHz setup.b.Grid of cropped bead images from (a) showing the fluorescence lifetimes of the beads overlaid on the binary multiphoton intensity mask estimated using STAMP (top) and SPEED (bottom) calculated using all the photons in a single frame (right) and across 5 frames (left).c.Histograms of the lifetimes shown in (b).d.Histogram showing the mean squared error for the differences between the fluorescence lifetime values for a 5-frame average and from a single frame.e. Multiphoton image showing the sum of 10 frames (estimated with STAMP) using the 5-MHz setup.f.Grid of cropped bead images from (a) showing the fluorescence lifetimes of the beads overlaid on the binary multiphoton intensity mask estimated using STAMP (top) and SPEED (bottom).g.Overlay of the decay profiles shown in (f) using SPEED (bottom) and STAMP (orange) h.Histograms of the lifetimes in (f).

Fig. 3 .
Fig. 3. Imaging the lifetime of fluorescent beads with STAMP and SPEED.a. Multiphoton image showing the average of 5 frames (estimated with STAMP) using the 80-MHz setup.b.Grid of cropped bead images from (a) showing the fluorescence lifetimes of the beads overlaid on the binary multiphoton intensity mask estimated using STAMP (top) and SPEED (bottom) calculated using all the photons in a single frame (right) and across 5 frames (left).c.Histograms of the lifetimes shown in (b).d.Histogram showing the mean squared error for the differences between the fluorescence lifetime values for a 5-frame average and from a single frame.e. Multiphoton image showing the sum of 10 frames (estimated with STAMP) using the 5-MHz setup.f.Grid of cropped bead images from (a) showing the fluorescence lifetimes of the beads overlaid on the binary multiphoton intensity mask estimated using STAMP (top) and SPEED (bottom).g.Overlay of the decay profiles shown in (f) using SPEED (bottom) and STAMP (orange) h.Histograms of the lifetimes in (f).

Fig. 4 .
Fig. 4. Imaging biological samples of varying fluorescent photon rates using STAMP.a,c.Normalized multiphoton intensity (grayscale) and FLIM images of 8 frame averages estimated using SPEED and STAMP of rat tail (a) and rat muscle (b) tissue samples at various depths from the surface using the 80-MHz setup with 256 incident laser pulses per pixel.b-d.Histograms of lifetime values for the images in (a) and (c).The black dotted lines in (d) indicate the peak location of the lifetime values from SPEED.

Fig. 4 .
Fig. 4. Imaging biological samples of varying fluorescent photon rates using STAMP.a,c.Normalized multiphoton intensity (grayscale) and FLIM images of 8 frame averages estimated using SPEED and STAMP of rat tail (a) and rat muscle (b) tissue samples at various depths from the surface using the 80-MHz setup with 256 incident laser pulses per pixel.b-d.Histograms of lifetime values for the images in (a) and (c).The black dotted lines in (d) indicate the peak location of the lifetime values from SPEED.

Fig. 5 .
Fig.5.Multichannel STAMP on a 5-MHz system.Plots of the average decays of all pixels in 50 arbitrary lines from an image of cactus leaf showing the peaks from the laser clock (grey, at 0 ns), the PMT at the near-UV wavelengths (pink, at 39.0625 ns), the HPD at blue wavelengths (blue, at 81.25 ns), and the HPD at the orange wavelengths (yellow, at 118 125 ns (or -81.875 ns).The images on the right show the photocurrents averaged across the specified periods.The photocurrents for the yellow, pink, and blue regions are shown after subtracting the "background," indicated as the average photocurrent of the entire period.

Fig. 6 .
Fig. 6.Imaging mouse kidney and liver with multichannel STAMP.a,f.Multichannel multiphoton image of a mouse kidney (a) and (b) along with the estimated fluorescence lifetime from the 50-frame average and 4×4 binning, masked by pixels whose fluorescence lifetime values were above the 50-photon threshold in that channel.The white arrows indicate regions where the artifacts apparent in SPEED are corrected by STAMP.b,g.Cumulative median fluorescence intensity values in each channel (solid line) with the interquartile ranges (dotted line) during frame averaging for the data in (a) and (f), respectively.c-e, h-j.Two-sided violin plots showing the histogram of the lifetime values estimated as a function of frame averaging, where the left side of each violin indicates the histogram of mean lifetimes estimated with SPEED, and the right side indicates the lifetime values estimated with STAMP.The blue and orange box plots indicate the interquartile ranges for SPEED and STAMP, respectively.The grey and black lines indicate the median fluorescence lifetime from SPEED and STAMP, respectively.(c,h) correspond to the near-UV channel, (d, i) correspond to the blue channel, and (e,j) correspond to the orange channel from the kidney (c-e) and liver (h-j).

Fig. 6 .
Fig. 6.Imaging mouse kidney and liver with multichannel STAMP.a,f.Multichannel multiphoton image of a mouse kidney (a) and (b) along with the estimated fluorescence lifetime from the 50-frame average and 4 × 4 binning, masked by pixels whose fluorescence lifetime values were above the 50-photon threshold in that channel.The white arrows indicate regions where the artifacts apparent in SPEED are corrected by STAMP.b,g.Cumulative median fluorescence intensity values in each channel (solid line) with the interquartile ranges (dotted line) during frame averaging for the data in (a) and (f), respectively.c-e, h-j.Two-sided violin plots showing the histogram of the lifetime values estimated as a function of frame averaging, where the left side of each violin indicates the histogram of mean lifetimes estimated with SPEED, and the right side indicates the lifetime values estimated with STAMP.The blue and orange box plots indicate the interquartile ranges for SPEED and STAMP, respectively.The grey and black lines indicate the median fluorescence lifetime from SPEED and STAMP, respectively.(c,h) correspond to the near-UV channel,(d, i) correspond to the blue channel, and (e,j) correspond to the orange channel from the kidney (c-e) and liver (h-j).