Towards unsupervised fluorescence lifetime imaging using low dimensional variable projection

Analyzing large fluorescence lifetime imaging (FLIM) data is becoming overwhelming; the latest FLIM systems easily produce massive amounts of data, making an efficient analysis more challenging than ever. In this paper we propose the combination of a custom-fit variable projection method, with a Laguerre expansion based deconvolution, to analyze bi-exponential data obtained from time-domain FLIM systems. Unlike nonlinear least squares methods, which require a suitable initial guess from an experienced researcher, the new method is free from manual interventions and hence can support automated analysis. Monte Carlo simulations are carried out on synthesized FLIM data to demonstrate the performance compared to other approaches. The performance is also illustrated on real-life FLIM data obtained from the study of autofluorescence of daisy pollen and the endocytosis of gold nanorods (GNRs) in living cells. In the latter, the fluorescence lifetimes of the GNRs are much shorter than the full width at half maximum of the instrument response function. Overall, our proposed method contains simple steps and shows great promise in realising automated FLIM analysis of large data sets. Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. OCIS codes: (030.5260) Photon counting; (110.2960) Image analysis; (110.0180) Microscopy; (100.1830) Deconvolution; (170.3650) Lifetime-based sensing; (170.6920) Time-resolved imaging. References and links 1. A. Shivalingam, M. A. Izquierdo, A. Le Marois, A. Vyšniauskas, K. Suhling, M. K. Kuimova, and R. Vilar, “The interactions between a small molecule and G-quadruplexes are visualized by fluorescence lifetime imaging microscopy,” Nat. Commun. 6, 8178 (2015). 2. D. Fixler, “Fluorescence Imaging for Biomedical Analysis,” in The Optics Encyclopedia (Wiley-VCH Verlag GmbH & Co., 2015). 3. G. O. Fruhwirth, L. P. Fernandes, G. Weitsman, G. Patel, M. Kelleher, K. Lawler, A. Brock, S. P. Poland, D. R. Matthews, G. Kéri, P. R. Barber, B. Vojnovic, S. M. Ameer-Beg, A. C. C. Coolen, F. Fraternali, and T. Ng, “How Förster resonance energy transfer imaging improves the understanding of protein interaction networks in cancer biology,” ChemPhysChem 12(3), 442–461 (2011). 4. K. Okabe, N. Inada, C. Gota, Y. Harada, T. Funatsu, and S. Uchiyama, “Intracellular temperature mapping with a fluorescent polymeric thermometer and fluorescence lifetime imaging microscopy,” Nat. Commun. 3, 705 (2012). 5. M. Nobis, E. J. McGhee, J. P. Morton, J. P. Schwarz, S. A. Karim, J. Quinn, M. Edward, A. D. Campbell, L. C. McGarry, T. R. J. Evans, V. G. Brunton, M. C. Frame, N. O. Carragher, Y. Wang, O. J. Sansom, P. Timpson, and K. I. Anderson, “Intravital FLIM-FRET imaging reveals dasatinib-induced spatial control of src in pancreatic cancer,” Cancer Res. 73(15), 4674–4686 (2013). Vol. 24, No. 23 | 14 Nov 2016 | OPTICS EXPRESS 26777 #275434 Journal © 2016 http://dx.doi.org/10.1364/OE.24.026777 Received 7 Sep 2016; revised 19 Oct 2016; accepted 19 Oct 2016; published 10 Nov 2016 6. Y. Zhang, G. Wei, J. Yu, D. J. S. Birch, and Y. Chen, “Surface plasmon enhanced energy transfer between gold nanorods and fluorophores: application to endocytosis study and RNA detection,” Faraday Discuss. 178, 383– 394 (2015). 7. S. Coda, A. J. Thompson, G. T. Kennedy, K. L. Roche, L. Ayaru, D. S. Bansi, G. W. Stamp, A. V. Thillainayagam, P. M. W. French, and C. Dunsby, “Fluorescence lifetime spectroscopy of tissue autofluorescence in normal and diseased colon measured ex vivo using a fiber-optic probe,” Biomed. Opt. Express 5(2), 515–538 (2014). 8. T. S. Blacker, Z. F. Mann, J. E. Gale, M. Ziegler, A. J. Bain, G. Szabadkai, and M. R. Duchen, “Separating NADH and NADPH fluorescence in live cells and tissues using FLIM,” Nat. Commun. 5, 3936 (2014). 9. D. M. Kavanagh, A. M. Smyth, K. J. Martin, A. Dun, E. R. Brown, S. Gordon, K. J. Smillie, L. H. Chamberlain, R. S. Wilson, L. Yang, W. Lu, M. A. Cousin, C. Rickman, and R. R. Duncan, “A molecular toggle after exocytosis sequesters the presynaptic syntaxin1a molecules involved in prior vesicle fusion,” Nat. Commun. 5, 5774 (2014). 10. Y. Sun, J. E. Phipps, J. Meier, N. Hatami, B. Poirier, D. S. Elson, D. G. Farwell, and L. Marcu, “Endoscopic fluorescence lifetime imaging for in vivo intraoperative diagnosis of oral carcinoma,” Microsc. Microanal. 19(4), 791–798 (2013). 11. S. R. Stürzenbaum, M. Höckner, A. Panneerselvam, J. Levitt, J. S. Bouillard, S. Taniguchi, L. A. Dailey, R. Ahmad Khanbeigi, E. V. Rosca, M. Thanou, K. Suhling, A. V. Zayats, and M. Green, “Biosynthesis of luminescent quantum dots in an earthworm,” Nat. Nanotechnol. 8(1), 57–60 (2012). 12. N. A. Hosny, G. Mohamedi, P. Rademeyer, J. Owen, Y. Wu, M.-X. Tang, R. J. Eckersley, E. Stride, and M. K. Kuimova, “Mapping microbubble viscosity using fluorescence lifetime imaging of molecular rotors,” Proc. Natl. Acad. Sci. U.S.A. 110(23), 9225–9230 (2013). 13. E. Gratton, S. Breusegem, J. Sutin, Q. Ruan, and N. Barry, “Fluorescence lifetime imaging for the two-photon microscope: time-domain and frequency-domain methods,” J. Biomed. Opt. 8(3), 381–390 (2003). 14. Y. Zhang, A. A. Khan, G. D. Vigil, and S. S. Howard, “Super-sensitivity multiphoton frequency-domain fluorescence lifetime imaging microscopy,” Opt. Express 24(18), 20862–20867 (2016). 15. H. Chen and E. Gratton, “A practical implementation of multifrequency widefield frequency-domain fluorescence lifetime imaging microscopy,” Microsc. Res. Tech. 76(3), 282–289 (2013). 16. G. Yahav, A. Hirshberg, O. Salomon, N. Amariglio, L. Trakhtenbrot, and D. Fixler, “Fluorescence lifetime imaging of DAPI-stained nuclei as a novel diagnostic tool for the detection and classification of B-cell chronic lymphocytic leukemia,” Cytometry A 89(7), 644–652 (2016). 17. H. A. R. Homulle, F. Powolny, P. L. Stegehuis, J. Dijkstra, D. U. Li, K. Homicsko, D. Rimoldi, K. Muehlethaler, J. O. Prior, R. Sinisi, E. Dubikovskaya, E. Charbon, and C. Bruschini, “Compact solid-state CMOS singlephoton detector array for in vivo NIR fluorescence lifetime oncology measurements,” Biomed. Opt. Express 7(5), 1797–1814 (2016). 18. C. J. de Grauw and H. C. Gerritsen, “Multiple time-gate module for fluorescence lifetime imaging,” Appl. Spectrosc. 55(6), 670–678 (2001). 19. A. V. Agronskaia, L. Tertoolen, and H. C. Gerritsen, “Fast fluorescence lifetime imaging of calcium in living cells,” J. Biomed. Opt. 9(6), 1230–1237 (2004). 20. D. D. U. Li, J. Arlt, D. Tyndall, R. Walker, J. Richardson, D. Stoppa, E. Charbon, and R. K. Henderson, “Videorate fluorescence lifetime imaging camera with CMOS single-photon avalanche diode arrays and high-speed imaging algorithm,” J. Biomed. Opt. 16(9), 096012 (2011). 21. J. L. Rinnenthal, C. Börnchen, H. Radbruch, V. Andresen, A. Mossakowski, V. Siffrin, T. Seelemann, H. Spiecker, I. Moll, J. Herz, A. E. Hauser, F. Zipp, M. J. Behne, and R. Niesner, “Parallelized TCSPC for dynamic intravital fluorescence lifetime imaging: quantifying neuronal dysfunction in neuroinflammation,” PLoS One 8(4), e60100 (2013). 22. R. M. Field, S. Realov, and K. L. Shepard, “A 100 fps, time-correlated single-photon-counting-based fluorescence-lifetime imager in 130 nm CMOS,” IEEE J. Solid-St. Circulation 49, 867–880 (2014). 23. M. G. Giacomelli, Y. Sheikine, H. Vardeh, J. L. Connolly, and J. G. Fujimoto, “Rapid imaging of surgical breast excisions using direct temporal sampling two photon fluorescent lifetime imaging,” Biomed. Opt. Express 6(11), 4317–4325 (2015). 24. S. P. Poland, N. Krstajić, J. Monypenny, S. Coelho, D. Tyndall, R. J. Walker, V. Devauges, J. Richardson, N. Dutton, P. Barber, D. D.-U. Li, K. Suhling, T. Ng, R. K. Henderson, and S. M. Ameer-Beg, “A high speed multifocal multiphoton fluorescence lifetime imaging microscope for live-cell FRET imaging,” Biomed. Opt. Express 6(2), 277–296 (2015). 25. K. Levenberg, “A method for the solution of certain non-linear problems in least squares,” Q. Appl. Math. 2, 164–168 (1944). 26. D. Marquardt, “An Algorithm for least-squares estimation of nonlinear parameters,” SIAM J. Appl. Math. 11(2), 431–441 (1963). 27. C. Y. Fu, B. K. Ng, and S. G. Razul, “Fluorescence lifetime discrimination using expectation-maximization algorithm with joint deconvolution,” J. Biomed. Opt. 14(6), 064009 (2009). 28. D.-U. Li, J. Arlt, J. Richardson, R. Walker, A. Buts, D. Stoppa, E. Charbon, and R. Henderson, “Real-time fluorescence lifetime imaging system with a 32 x 32 0.13 μm CMOS low dark-count single-photon avalanche diode array,” Opt. Express 18(10), 10257–10269 (2010). Vol. 24, No. 23 | 14 Nov 2016 | OPTICS EXPRESS 26778


Introduction
Fluorescence lifetime imaging (FLIM) techniques are much more informative than the intensity based counterparts.Not only can FLIM microscopy sense the fluorescence intensity of fluorescent molecules (fluorophores), but it also can measure the decay rate (or lifetime) of the fluorescence.As fluorescence lifetimes of fluorophores are sensitive to the cellular microenvironments, FLIM is suitable for detecting physiological or electrochemical parameters such as Ca 2+ , pH, O 2 , or temperature.When combined with Förster resonance energy transfer (called FLIM-FRET), it is a powerful tool to study protein interaction networks or molecular metabolisms.FLIM or FLIM-FRET has been used for studying molecular oncology [1][2][3], assessing the efficacy of cancer therapies [4,5], the diagnosis of diseases or cellular imaging [6][7][8], understanding brain functions [9], image-guided surgeries [10], and characterizing fluorescent proteins or contrast agents [11,12].
There are mainly frequency domain (FD) and time domain (TD) FLIM systems.Readers interested in FD systems, can check previously published FD systems for single-exponential [13,14] or multi-exponential FLIM imaging [15,16].Typically 2~20 phase images are taken depending on applications, and usually FD FLIM software tools are iterative least squares method (LSM) based.TD time-correlated single-photon counting (TCSPC) based systems have superior signal-to-noise performance and timing resolution (down to 4ps for state-ofthe-art systems) and are therefore widely considered gold-standard for FLIM-FRET applications.A typical TCSPC FLIM system contains a pulsed laser, a single-photon avalanche diode (SPAD) or a photomultiplier tube (PMT), and a TCPSC module for measuring the time delay between the detected photon and the laser pulses.In this paper, we focus on TD TCSPC based systems to develop methods of processing FLIM data.The developed methods are also applicable to time-gated systems [17,18].
Despite the potential and significant impact of FLIM, primarily in the biological sciences, accurate estimation of fluorescence lifetimes remains a significant challenge.For live cell FLIM, it is necessary to acquire images at a frame rate fast enough to avoid motion artefacts and to resolve temporal cellular dynamics.Fortunately, recent advances in image sensors and microscopy techniques have radically boosted gold standard TCSPC-FLIM acquisition from a frame every few minutes to several frames per second [19][20][21][22][23][24].Such significant progress makes FLIM promising for live cell imaging.However, the massive data throughput, generated by these fast imaging systems, poses a greater challenge to image analysis.For TCSPC-FLIM experiments, the time delays of detected photons are measured repeatedly, and a histogram of time delays is accumulated from which the lifetimes are to be extracted.Commercial FLIM software packages usually use LSM, such as Levenberg-Marquardt algorithms [25,26], to calculate the true fluorescence density from the measured histogram in a pixel.This is an iterative procedure that requires users to provide an initial guess.Choosing the initial guess, however, may be challenging and a LSM can easily fail to converge to a true minimum when the field of view has a wide range of lifetime contribution [27].This explains why commercial FLIM software tools still require manual interventions from experienced users with knowledge in fluorophores and mathematical modelling to deliver a robust analysis.With analyses carried out in such supervised environments, image analysis becomes very time-consuming.A strategy that supports unsupervised analysis is really desirable.
Recently, many non-fitting algorithms have been proposed to provide fast singleexponential [28][29][30] or bi-exponential analysis [31][32][33][34], but they all use tail-fitting (fitting from the peak of the histogram), which assumes that the full-width at half maximum (FWHM) of the instrument response function (IRF) is negligible.They work well when the lifetime components are large, but can easily be biased when the lifetimes are comparable or even less than the FWHM of the IRF.In order to use these non-fitting algorithms robustly, high-precision (timing jitter < 50ps) detectors are required, which can significantly increase the system cost.In many applications, where the analysis speed does not matter or the fluorescence lifetimes are short, biologists still prefer fitting approaches with deconvolution that can guarantee a better performance.So far, numerous deconvolution techniques for FLIM analysis have been developed [27,[35][36][37][38][39][40][41].Among them, the least squares deconvolution based on Laguerre expansion (LSD-LE) provides a fast single-exponential contrast and superior sensitivity in disease detection [38][39][40][41].To effectively apply LSD-LE to diagnosis or parameter identification [41,42], previous studies [40] concluded that the Laguerre basis functions (LBF) should be mutually orthogonal within the observation window (T).This requirement means T should be much larger than the largest lifetime, and it requires shining pulsed lasers with a low duty-cycle, reducing the photon collection speed.In order to avoid this constraint, we examine the LSD-LE techniques and propose to use another optimization procedure [43].Our proposed method allows using lasers with a higher duty-cycle.Moreover, we extend LSD-LE, to our knowledge for the first time, to study bi-exponential two-photon FLIM images of the uptake of gold nanoparticles in living cells [44].Optimized Laguerre dimension L and optimized scale α are found for bi-exponential analysis covering a wide range of lifetime distributions, supporting automated analysis.
To further guarantee unsupervised analysis, we present a simple lifetime extraction method based on a classical algorithm called the variable projection method (VPM) [43,45].VPM is a very useful tool for solving nonlinear least-squares problems in which a number of the parameters are linear.The VP methods have already been adopted and integrated with the global analysis method (GA) for the analysis of FLIM data [46][47][48].VPM was introduced to reduce the stringent requirements on memory incurred by GA.Different from these earlier works, our approach is inspired by recent work of Barral et al. for the computation of singleexponential decays for MRI T1 relaxation imaging applications [49].We generalize this approach to the identification of bi-exponential decays and use it to facilitate our study on the endocytosis of gold nanoparticles in living cells [6,44,50].Indeed, both the single-and biexponential formulas are special cases of the more general VPM [45].Nevertheless, by focusing on the bi-exponential model in particular, we derive an explicit form for the linear parameters involved in the modelling.Thus, for bi-exponential models, implementing our method is much simpler than the general VPM, which is highly desirable in a hardware implementation.We compare our simplified VPM to LSM: Our method is not only faster but also has much better photon efficiency and reliability.
The rest of this paper is organized as follows.In Section II we present our method.In Section III, our simplified VPM and LSM are tested and compared on synthesized FLIM data using tail-fitting [49] and deconvolution.Similar comparisons are carried out on various experimental FLIM data in Section IV.

Variable projection methods
We assume that the fluorescence intensity is where K is the amplitude, a the proportion, τ 1 and τ 2 (τ 1 < τ 2 ) are the fluorescence lifetimes, and T is the observation window.Usually a TD FLIM experiment obtains a fluorescence histogram y j , j = 1,…, N, where y j is the measured photon count in the j-th time bin and N is the number of time bins in a TCSPC system.Here we neglect the IRF to illustrate how VPM works (in Section 2.2 we consider the IRF).The goal of the analysis is to minimize the error , , , .
Here we focus on a bi-exponential model.For simplicity, we denote A = Ka and B = K(1a).Instead of computing the gradient of S with respect to the unknown parameters A, B, τ 1 and τ 2 , as in a traditional LSM, the error S(A, B, τ 1 , τ 2 ) can be rewritten as A simple verification of the equivalence of ( 2) and (1) shows that the expressions for γ(τ 1 ), It is easy to prove that γ(τ 1 ) and δ(τ 1, τ 2 ) are positive.Hence minimizing S(A,B,τ 1, τ 2 ) for given y j is equivalent to The advantage of these formulas, as that of those presented by Barral et al. for singleexponential decays [49], is that explicit expressions are given for the parameters A and B (or K and a) that appear linearly in the model.Compared with a nonlinear LSM, the likelihood of finding a global minimizer rather than a local one is much better.The VP method eliminates the linear unknowns from the least squares problem and optimizes the remaining lessdimensional least squares criterion.At the same time the problem becomes better conditioned.The eliminated linear coefficients are computed from the least squares interpolation problem after solving the optimization problem.The solution is sought from a 2-D plane within τ MIN < τ 1 , τ 2 < τ MAX (τ MAX can be up to T) and users do not even need to know the exact boundaries (choosing 0 < τ 1 , τ 2 < Τ also works well).This minimizes manual interventions and allows unsupervised large FLIM data analysis.Compared with GA [51], VPM is in general faster and able to distinguish the lifetime differences for the pixels in the same segment.GA, on the other hand, is able to reduce the fitting error caused by poor SNR in a pixel.The current method and GA can be combined to provide a better SNR performance.

Calibration techniques for instrumental response functions
In a TCSPC FLIM experiment, a measured fluorescence decay y(t) is the convolution of the fluorescence intensity f(t) and the IRF I(t): where ε(t) is Poisson noise [52] and Eq. ( 5) can be discretized to Then the analysis when I(j) is known is to minimize the error , , , .
We can solve this curving-fitting problem directly by using the MATLAB nonlinear leastsquares subroutine [25].To facilitate our discussions, we call it C-LSM for involving convolution computations.This is the most straightforward LSM approach [51].However, the disadvantages of C-LSM are also well-known: 1) it needs to compute convolutions for each iteration [51] and 2) it requires optimizing four parameters (K, a, τ 1 , τ 2 ) and so expects four starting values.It is time-consuming and the analysis results are subject to the initial values [44].To avoid the first problem above, fast deconvolutions can be used to simplify the cost function [27,38,40,41,44,53], Eq. ( 7).With the deconvolution, the IRF is calibrated and an estimated ( ) f t , and an estimated K  are obtained.The normalized ( ) f t is similar to a filtered histogram leaving only (a, τ 1 , τ 2 ) to be solved [44].

Simulations on synthesized decays
We compared the proposed VPM and LSM using tail-fitting (TF-VPM and TF-LSM) and the improved Laguerre expansion based deconvolution methods (DE-VPM and DE-LSM) in terms of 1) the normalized bias = Δg/g, g can be a, τ 1 , or τ 2 and 2) the F-value, a normalized precision defined as F = (N C ) 0.5 •σg/g (originally introduced for single-exponential analysis [54], but here we extended for bi-exponential models.The decovolutions were performed using CLSD-LE with the Laguerre dimension L = 16 and the Laguerre scale α = 0.912 derived by our recent studies [44].N C is the total count within the observation window.F = 1 for the ideal case, and F > 1 or F >> 1 for realistic FLIM analysis).Monte Carlo simulations were carried out for K = 1000 and 50, N = 256, T = 10ns, τ 2 = 2.8ns, a = 0.1, 0.37, 0.63, and 0.9, and τ 1 = 0.1, 0.37, 0.63, and 0.9ns.The IRF is assumed to have a Gaussian profile with an FWHM = 300ps.Figures 1(a), 1(c), and 1(e) show the normalized bias plots, whereas Figs.1(b), 1(d), and 1(f) show the F plots, for a, τ 1 , and τ 2 respectively.Figures 1(a) and 1(c) indicate that tail-fitting actually causes biased estimations of a and τ 1 (apart from τ 2 ) when τ 1 is small.In applications where the information of a or τ 1 is essential, using tail-fitting (TF-LSM or TF-VPM) can lead to misinterpretation of data.However, if the information of τ 2 is sought after, then the TF-VPM analysis can provide acceptable bias performances (Δτ 2 /τ 2 < 10%) and precision performances that are comparable to the DE-LSM analysis.In comparison with DE-LSM and DE-VPM, C-LSM has comparable or better bias performances only when the initial values are well chosen.Figure 1 shows DE-VPM's superior performances in photon efficiency.Apart from the extreme case a = 0.1 and τ 1 = 0.1ns, the normalized bias is in general less than 10% and the F-value less than 10, showing that it has the best photon efficiency among the five approaches.Note that for synthesized FLIM data it is easy to set the initial values for LSM, as a, τ 1 , and τ 2 are already known.In cases where the field of view has a wide range of variations, LSM is not able to converge robustly to the true minima and its output depends on the given starting values.Compared with DE-VPM, the dynamic range of DE-LSM is very limited, and it only provides comparable performances when τ 2 >> τ 1 .
Figures 2(a)-2(f) show the bias and F-value plots using the same settings as above but with K = 50.The purpose of this analysis is to demonstrate the performance of the proposed approach when the photon count is low.This happens in some live-cell imaging or highthroughput screening applications where the acquisition has to be short [10,55].Again, the TF analysis produces more biased estimations when τ 1 is small.At a low count, it is difficult to use any method to estimate a accurately when a is small, as shown in Fig. 2  In terms of the analysis speed, Table 1 shows the average fitting time of 16 decays for different a and τ 1 (a = 0.1, 0.37, 0.63, and 0.9, τ 1 = 0.1, 0.37, 0.63, and 0.9ns) for different methods.Simulations were carried out using MATLAB® R2015b on a Dell PrecisionM2800 (Intel Core i7-4810MQ CPU @ 2.80GHz, 16.0 GB memory; OS: Windows 7 Enterprise Service Pack 1 64-bit).The table shows that our VPM requires less time than LSM.The simulation time of C-LSM varies significantly depending on the initial values.In realistic scenarios, it is difficult to obtain accurate initial guesses.On the other hand, our VPM does not suffer this problem.It converges robustly at a faster speed, regardless of the initial values and therefore promising for automated analysis.It is surprising that TF-LSM takes more time than DE-LSM, but it is reasonable as more iterations are required for TF-LSM to obtain convergent results when the IRF is neglected.In some applications, such as estimating the efficiency of FRET transfer, the amplitudeaveraged lifetime τ ave = aτ 1 + (1-a)τ 2 is to be estimated [37]. Figure 3 shows the normalized bias and F plots of τ ave using K = 1000 and K = 50.Figure 3(a) shows that TF-LSM and TF-VPM produce comparable estimations, but it also indicates the need to consider the IRF to avoid biased estimations.In general, DE-VPM performs better in the bias (Δτ ave /τ ave < 2%) and photon efficiency (F-value).DE-LSM only provides comparable precision when τ 1 is small.Similarly, for K = 50, Fig. 3(c) indicates that the IRF needs to be considered in lifetime estimations.Without deconvolution, the TF analysis produces the most biased estimations when τ 1 = 0.1ns.Considering both the bias and the precision for different K, DE-VPM does offer a wider dynamic range and superior photon efficiency.

Daisy pollen
The first example is to study the fluorescence lifetimes of the autofluorescence emitted from daisy pollen.Autofluorescence of biological samples as well as the fluorescence lifetime can be very useful.For example, cellular autofluorescence is used as a good label-free indicator for studying cytotoxicity [56].The FLIM data are obtained using the MicroTime 200 (PicoQuant), equipped with the standard piezo scanner from Physik Instrumente (100x100µm scan range) and a Hybrid-PMT (PMA Hybrid-40).The TCSPC system used for the acquisition is the HydraHarp 400 with the bin width set to 8ps and each histogram contains 6253 time bins (the equivalent full range = 50ns).Other parameters include: excitation wavelength = 485nm, the laser repetition rate = 20 MHz (LDH-D-C-485 laser head controlled by the PDL 828 "Sepia II" laser driver) and the detection band = 520/35.
Figure 4 shows the measured IRF, measured and fitted histograms (using DE-VPM) at the brightest pixel, and the summed histogram of all pixels.The FWHM of the IRF is around 300ps; it is much larger than τ 1 .Traditional tail-fitting practices can easily overestimate τ 1 and even τ 2 [57].Therefore, deconvolutions of the IRF should be performed to accurately determine lifetimes.[57,58].To compare the differences between LSM and VPM as well as the impact of the IRF on the lifetime estimations, we also perform the analysis using TF-LSM, TF-VPM, DE-LSM, and DE-VPM.Figures 5(e)-5(h) show the a, τ 2 , τ 1 (note that this is in log scale), and τ ave histograms, respectively.The figures indicate that neglecting the IRF for the TF analysis does bias all parameters and make the estimations sensitive to the observation window.Usually for tail-fitting the observation window starts from the peak of the histogram or a certain distance away from the peak [57].However, the peak position is noise inflicted and difficult to be determined precisely when the photon count is low.Tail-fitting approaches (both TF-LSM and TF-VPM) underestimate a and τ 1 and overestimate τ 2 and τ ave .TF-LSM and TF-VPM show similar analyses except that TF-VPM is superior in photon efficiency and it is able to resolve low-count pixels showing τ 1 > 0.1ns.The figures also show the advantages of the proposed DE-VPM.Instead of searching the optimized solution in a 3-D (a, τ 1 , τ 2 ) space as traditional LSM approaches, the VPM identifies it in a 2-D (τ 1 , τ 2 ) space.As mentioned earlier, our VPM does not need a precise initial guess and a rough range [8ps, 10ns] suffices to guarantee consistent analysis results.For DE-LSM, on the other hand, it usually requires a good guess (from an experienced user) for a satisfactory analysis.In this figure, a starting value (a = 0.9, τ 1 = 0.05ns, τ 2 = 1ns; pink open square curve) close to the real distribution and a randomly chosen one (a = 0.1, τ 1 = 0.5ns, τ 2 = 2ns; green open square curve) were given to illustrate this issue.Figure 5(f) shows that an ill-chosen initial guess (a = 0.1, τ 1 = 0.5ns, τ 2 = 2ns) prompts the search process for a significant amount of pixels to converge to a local minimum and therefore obtain biased estimations τ 2 < 0.1ns.When the starting value is well chosen, DE-LSM can obtain results coinciding with those of DE-VPM apart from some biased estimations at pixels with a lower photon count (pointed to by the pink arrow).As discussed earlier, Fig. 5(g) shows that the TF approaches obtain biased τ 1, even much smaller than the timing resolution of the TCSPC (8ps) at most pixels.For the DE analysis, many pixels fail to converge for the DE-LSM analysis with the initial values a = 0.1, τ 1 = 0.5ns, τ 2 = 2ns.Figure 5(h) shows that the ill-chosen initial values produce a slightly different τ ave histogram from the DE-VPM and the DE-LSM results with a well-chosen initial guess.Again, TF-LSM and TF-VPM produce similar results, but they both overestimate τ ave .
Figures 2(a)-2(f) show the bias and F-value plots using the same settings as above but with K = 50.The purpose of this analysis is to demonstrate the performance of the proposed approach when the photon count is low.This happens in some live-cell imaging or highthroughput screening applications where the acquisition has to be short[10,55].Again, the TF analysis produces more biased estimations when τ 1 is small.At a low count, it is difficult to use any method to estimate a accurately when a is small, as shown in Fig.2(a).Among C-LSM, DE-LSM, and DE-VPM, C-LSM has better performances in Δτ 1 /τ 1 , whereas DE-LSM and DE-VPM have better performances in Δτ 2 /τ 2 .Figures 2(b), 2(d), and 2(f) show similar F plots to Figs. 1(b), 1(d), and 1(f).The F-values of TF-LSM, TF-VPM, C-LSM, and DE-LSM are comparable, and they are in general larger than that of DE-VPM (F DE-VPM < 10).

Figures 5 (
Figures 5(a)-5(d) show the fluorescence intensity, τ ave = aτ 1 + (1-a)τ 2 , a and τ 2 images, respectively, obtained by DE-VPM.They clearly show the differences between the intensity and lifetime maps.The fraction, a, within the pollen is close to 1, showing that there is near single-exponential fast autofluorescence (smaller τ ave ).The fraction is smaller on the spikes, but it is in general larger than 0.5, see the blue open-circle curve in Fig.5(e).The results are very different from those obtained by tail-fitting reported in the literature[57,58].To compare the differences between LSM and VPM as well as the impact of the IRF on the lifetime estimations, we also perform the analysis using TF-LSM, TF-VPM, DE-LSM, and DE-VPM.Figures5(e)-5(h) show the a, τ 2 , τ 1 (note that this is in log scale), and τ ave histograms, respectively.The figures indicate that neglecting the IRF for the TF analysis does bias all parameters and make the estimations sensitive to the observation window.Usually for tail-fitting the observation window starts from the peak of the histogram or a certain distance away from the peak[57].However, the peak position is noise inflicted and difficult to be determined precisely when the photon count is low.Tail-fitting approaches (both TF-LSM and TF-VPM) underestimate a and τ 1 and overestimate τ 2 and τ ave .TF-LSM and TF-VPM show similar analyses except that TF-VPM is superior in photon efficiency and it is able to resolve low-count pixels showing τ 1 > 0.1ns.The figures also show the advantages of the proposed DE-VPM.Instead of searching the optimized solution in a 3-D (a, τ 1 , τ 2 ) space as traditional LSM approaches, the VPM identifies it in a 2-D (τ 1 , τ 2 ) space.As mentioned earlier, our VPM does not need a precise initial guess and a rough range [8ps, 10ns] suffices to guarantee consistent analysis results.For DE-LSM, on the other hand, it usually requires a good guess (from an experienced user) for a satisfactory analysis.In this figure, a starting value (a = 0.9, τ 1 = 0.05ns, τ 2 = 1ns; pink open square curve) close to the real distribution and a randomly chosen one (a = 0.1, τ 1 = 0.5ns, τ 2 = 2ns; green open square curve) were given to illustrate this issue.Figure5(f)shows that an ill-chosen initial guess (a = 0.1, τ 1 = 0.5ns, τ 2 = 2ns) prompts the search process for a significant amount of pixels to converge to a local minimum and therefore obtain biased estimations τ 2 < 0.1ns.When the starting value is well chosen, DE-LSM can obtain results coinciding with those of DE-VPM apart from some biased estimations at pixels with a lower photon count (pointed to by the pink arrow).As discussed earlier, Fig.5(g) shows that the TF approaches obtain biased τ 1, even much smaller than the timing resolution of the TCSPC (8ps) at most pixels.For the DE analysis, many pixels fail to converge for the DE-LSM analysis with the initial values a = 0.1, τ 1 = 0.5ns, τ 2 = 2ns.Figure5(h) shows that the ill-chosen initial values produce a slightly different τ ave histogram from the DE-VPM and the DE-LSM results with a well-chosen initial guess.Again, TF-LSM and TF-VPM produce similar results, but they both overestimate τ ave .