Sub-μs time resolution in wide-field time-correlated single photon counting microscopy obtained from the photon event phosphor decay

Fast frame rate complementary metal–oxide–semiconductor cameras in combination with photon counting image intensifiers can be used for microsecond resolution wide-field fluorescence lifetime imaging with single photon sensitivity, but the time resolution is limited by the camera exposure time. We show here how the image intensifier's P20 phosphor afterglow can be exploited for accurate timing of photon arrival well below the camera exposure time. By taking ratios of the intensity of the photon events in two subsequent frames, photon arrival times were determined with 300 ns precision with 18.5 μs frame exposure time (54 kHz camera frame rate). Decays of ruthenium and iridium-containing compounds with around 1 μs lifetimes were mapped with this technique, including in living HeLa cells, using excitation powers below 0.5 μW. Details of the implementation to calculate the arrival time from the photon event intensity ratio are discussed, and we speculate that by using an image intensifier with a faster phosphor decay to match a higher camera frame rate, photon arrival time measurements on the nanosecond time scale could be possible.


Introduction
Single photon detection and timing capabilities are important in a number of fields such as fluorescence spectroscopy and microscopy, lidar, optical tomography and quantum cryptography, as has been reviewed recently [1][2][3][4]. Time-correlated single photon counting (TCSPC), in particular, is a precise, reliable and mature technique to time photon arrival. Its popularity and widespread use are due to its advantages afforded by its digital nature, and include a high dynamic range, high sensitivity, linearity, well-defined Poisson statistics and easy visualization of photon arrival time data [5,6].
Recent developments in sensor technology have allowed complementary metal-oxide-semiconductor (CMOS) cameras to reach MHz frame rates. Unlike the pixels in charge-coupled device (CCD) cameras, CMOS pixels have their own amplification, digitization and read-out circuitry. No charge transfer takes place, and all pixels can be read out simultaneously and at very high speeds. However, direct detection of individual photons is not possible with these particular cameras. In this context, we note CMOS single photon avalanche diode (SPAD) array image sensors have been developed, incorporating picosecond timing circuitry in each pixel or chip [7][8][9].
Wide-field TCSPC in the microsecond time range can be performed with a fast frame rate CCD or CMOS camera in combination with an image intensifier operated with saturated gain, which allows single photons to be detected [10,11]. After each excitation pulse, a sequence of frames is acquired during the decay time of the figure 1(a). With a three-stage intensifier, up to 10 7 photons can be created by an incoming single photon. This allows single photon events to be imaged with a CCD or a CMOS camera. Photon counting image intensifiers were originally developed because of their sensitivity, and the Hubble space telescopeʼs faint object camera, for example, employed a photon counting cascade image intensifier [43,44]. The European space agencyʼs x-ray multi-mirror satellite was fitted with a photon counting imaging optical monitor [45], and several other astronomical observatories employ photon counting imaging techniques, too.
The phosphor screen decay usually follows a multi-exponential function, where the decay time components depend on the type of phosphor and can range from nanosecond to millisecond time scale [37][38][39]. Although the statistical nature of the signal amplification process in the MCPs leads to a broad pulse height distribution (i.e. variation in the brightness of the photon events), variations in the phosphor decay function are small, as the gain voltage is fixed [46]. The transit time through the MCPs is short leading to a short transit time spread, so the photon arrival time information is preserved [47,48]. Instrument response functions (IRFs) shorter than 20 ps have been reported when using MCP detectors for timing [49]. If the phosphor decay time and the camera frame rate are matched such that the photon events can be detected in multiple frames, the photon arrival time within the frame exposure time can be determined from the relative intensities of the photon events in subsequent frames [40][41][42]. For a photon arriving at the beginning of the frame exposure time, the intensity of the event in the first frame is larger than in the second frame. For photons arriving later during the exposure time the intensity in the first frame decreases, and the intensity ratio between the second and the first frame where the photon is detected increases, as illustrated in figure 1 The mathematical framework to determine the photon arrival time from the phosphor decay has been discussed in-depth previously [40]. Briefly, assuming that the phosphor decay follows a multi-exponential decay function with amplitudes a j and time constants τ j , and that there is additionally a dead time during which no light is detected (due to camera readout) in the time interval zT (0, ) occupying a fraction z of the full period T between two frames, for a photon arriving at time xT after the start of the exposure, the intensity in the mth frame is given by where β = − + m T zT xT max (( 1) , ) 1 , β = mT xT max ( , ) 2 , and ⩾ m 1 [40]. See figure 2 for a detection timing diagram.
If the exact phosphor decay function and dead time of the camera are known, the arrival time x can be found using a fitting routine that minimises the difference between the measured intensities in frames = m n 1 ... and the theoretical intensities I n 1 ... for these frames calculated from the phosphor decay function using equation (2) Figure 1. (a) Schematic diagram of an image intensifier. An incoming photon ( ν h ) hits the photocathode creating a photoelectron ( − e ), which is accelerated into a microchannel plate (MCP) by a high voltage. Its impact creates more electrons which are multiplied by travelling along the channels of the MCP, and the resulting electron cloud is converted back to photons when it reaches the phosphor screen. (b) Determining the photon arrival time from the phosphor decay: as the photon arrival time within the frame exposure time increases, the intensity ratio between the 2nd and the 1st frame where the photon is detected increases. Δt indicates the camera dead time. [40]. In this work the exact phosphor decay function and camera dead-time are not known, and the ratio-totime conversion function is determined experimentally by measuring intensity ratio of the first two frames where the photon is detected I I 2 1 for known arrival times x.

Data acquisition
A diagram of the experimental setup is shown in figure 3. A Photron Fastcam SA1.1 CMOS camera (Photron, CA) was used for image acquisition with 54 kHz frame rate and 320 × 256 pixels image size. Although the exposure time can be varied, we used the maximum possible value of 1/54 kHz. A pulsed 467 nm diode laser with 90 ps pulse width (PLP-10, Hamamatsu, Japan) was used for illuminating the sample. The laser was triggered from the camera with a TTL pulse, also at 54 kHz. The delay for the trigger pulse was set to 5 μs after the start of the frame exposure for decay measurements or varied from zero to exposure time for the ratio-to-time calibration measurements. A 40 mm diameter dual proximity-focused, three-MCP image intensifier (Photek, UK) operating in photon counting mode [50,51] was mounted on the side port of an inverted microscope (Eclipse TE2000-U, Nikon, Japan). The laser was focused into the sample either through a 4× 0.13NA air objective (Nikon, Japan; for solutions of the ruthenium compound and the iridium-containing beads) or a 100× 1.4NA oil immersion objective (Leica, Germany; for cell samples), and the detected light was collected with the same objective. A green filter cube (Nikon, Japan; ex:480/30 nm, DM:505 nm, em:515LP) was used for separating the excitation and emission light; for reflection measurements a mirror was placed on the sample  The image intensifier was mounted on the side port of an inverted microscope, and the phosphor screen of the intensifier was imaged onto the camera with two relay lenses. The excitation laser was triggered from the camera with an adjustable delay after the frame exposure start time. stage and the emission filter was removed. The phosphor screen of the intensifier was imaged onto the camera using two 50 mm focal length photographic lenses (F = 1/1.2, Canon, Japan and F = 1/1.4, Nikon, Japan).

Data processing
The acquired frames saved in 16 bit raw data format containing single photon events were processed with inhouse software written in C. The frames were first thresholded, and the background was subtracted. Areas that had between eight and 300 adjacent pixels above the threshold were counted as photon events. The brightest pixel of this area was recorded as the event centre pixel, and the total area and intensity of the event were determined. The software then checked whether an event had been detected at the same position in the previous frame. If the event was detected in at least three consecutive frames with the ratio of the third and the second detection in the expected range consistent with a decaying photon event rather than a new one, the ratio of the second and the first detection was recorded in a histogram. The decays were fitted with TRI2 software [52,53] using a Levenberg-Marquardt fitting routine.

Single photon events
The single photon events on the phosphor screen of the image intensifier recorded with the camera vary in size and brightness, but are approximately round [56] and stand out from the flat, nearly zero background, as shown in figures 4(a) and (b). Figure 4(c) shows enlarged areas of three photon events arriving at different points during the exposure time of the 2nd frame, with the relative intensities of the first two frames where the photon is detected clearly varying depending on the photon arrival time. This data set corresponds to the schematic shown in figure 1(b). All three events can be detected in many successive frames due to the long afterglow of the phosphor. Due to the statistical nature of the electron multiplication process inside each of the three MCPs of the intensifier, the photon event intensities vary with a broad pulse height distribution, see figure 4(d). Importantly, as the phosphor decay varies with gain voltage (i.e. electron energy) but not with the number of electrons creating the photon event [46], variations in the phosphor decay function with photon event intensity are negligible.
The intensifier used in this work has a P20 phosphor screen with a decay time of 250 μs to 1/10 of the peak value quoted by the manufacturer. The decay is a complex exponential with a fast initial component and a long lasting afterglow. The decay function, figure 4(e), was estimated from the reflection measurements with variable time delay between the frame exposure start time and the laser trigger pulse. However, a detailed knowledge of the phosphor decay is not required if an experimental calibration is performed to convert the photon event intensity ratio in two subsequent frames into an arrival time.

Ratio-to-time conversion
The ratio-to-time conversion function was measured by varying the time delay between the frame exposure start time and the laser trigger pulse, and detecting the photons from reflected laser pulse. For each delay, a frequency histogram of the ratio between the 2nd and the 1st detection was plotted, as shown in figure 5(a). A Gaussian fit revealed the peak positions which were plotted against the laser trigger time delays, and fitted with a tripleexponential function, as shown in figures 5(b) and (c). This function is used for converting the measured photon intensity ratio to arrival time, figures 5(b) and (c) is thus a calibration plot for ratio-to-time conversion. For the following lifetime measurements, a 5 μs delay was chosen for the excitation pulse to avoid timing inaccuracies for photons arriving during or immediately after the dead time (see lower part of figure 5(c)). in glycerol mixtures-the lifetime increases with viscosity due to slower oxygen diffusion rate, as expected [19]. The data set consists of 140 000 images and 370 000 photons with total data acquisition time of 2.6 s. On average, 2.7 new photons were detected in each frame, with mean count rate of 144 000 photons s −1 .

Polymer beads
The lifetimes of the phosphorescent complexes Ir(ppy) 3 , Ir(BMes 2 ) 2 acac, Ir(fppy) 3 and Pd(OEP) infused into polystyrene beads were measured. Beads containing BPEA, which has a fluorescence lifetime of a few nanoseconds were also measured for comparison. The samples were first placed on separate coverslips,   inspected visually through the eyepiece, and their lifetimes measured individually (see table 1). The beads were then mixed and the mixture placed on a coverslip for imaging ( figure 7).
The different compounds are distinguishable to some extent by colour, intensity and lifetime (figures 7(a)-(c), respectively). The lifetimes of the Ir(BMes 2 ) 2 acac and Ir(fppy) 3 compounds are very similar, and they cannot be easily distinguished by lifetime or intensity alone, although their emission spectra are different. Conversely, the green BPEA fluorescence and Ir(ppy) 3 compound look identical in both colour and intensity images, but show very good contrast in the lifetime image (see figures 7(a)-(c), bottom left corner). The decays for these two beads are shown in figure 7(d), where monoexponential fit yields lifetimes 0.32 and 0.97 μs for the green fluorescent beads containing BPEA and the phosphorescent Ir(ppy) 3 , respectively.
The data set consists of 280 000 images and 220 000 photons with an average of 0.8 new photons/frame. The total data acquisition time was 5.1 s with a mean count rate of 43 000 photons s −1 .

Ruthenium in living cells
The lifetime of Ru(dpp) + 3 2 was measured in living HeLa cells. With overnight incubation, the cellular uptake of the ruthenium complex was good and the compound was found to be distributed in the cytoplasm avoiding the nucleus (figure 8). Due to the cellular autofluorescence, the decay has a fast component and was fitted with a double-exponential function. The average lifetime of the complex was found to be 2.7 μs, with a fast component of 0.1 μs attributable to the cellular autofluorescence. The lifetime of Ru(dpp) + 3 2 is longer than in water, in  agreement with previous work [11] and indicating partial protection from quenching by molecular oxygen. The total data acquisition time was 1.3 s, during which 70 000 frames and 57 000 photons were collected, with an average of 0.8 new photons/frame and mean count rate of 44 000 photons s −1 . The excitation power was below 0.5 μW.

Discussion
Our results show that the phosphor afterglow in photon counting image intensifiers does not necessarily have to be an undesirable nuisance, but, on the contrary, can be exploited to obtain photon arrival time information within the camera exposure time. This is conceptually similar to a double exposure technique proposed for time-of-flight studies in velocity mapping [41,42]. However, rather than measuring the time of flight of ion fragments, here the photon arrival time was measured. It allowed us to obtain the decay of luminescent probe Ru (dpp) + 3 2 and Iridium-based compounds in each pixel of an image, thus allowing single-photon sensitive lifetime mapping with wide-field TCSPC with time-resolution well below the camera exposure time. The image size is larger than in microsecond resolution wide-field TCSPC with a P47 short decay time phosphor where the photon events are imaged directly [10,11] (as a slower frame rate allows a larger image size), and the method is single photon sensitive which is an advantage over gating approaches, where signal outside the gate is lost. A low excitation power in the order of 0.5 μW was used to obtain clear lifetime contrast in a few seconds acquisition time.
The microsecond resolution wide-field TCSPC imaging approach described here is especially well suited for time-resolved imaging of transition metal complexes with typical lifetimes in the microsecond region [20], including time-resolved luminescence anisotropy imaging [25]. It combines fast image acquisition within a few seconds with a low excitation power-well below 1 μW, i.e. more than 1000 times lower than reported previously with a sequential time-gating approach [21,22]. Due to the digital nature of photon counting and its associated advantages-e.g. Poisson statistics, a large dynamic range, a high time resolution, easy visualization of decays and the ability to perform meaningful multi-exponential decay analysis-it also has a better signal-tonoise ratio than frequency modulation techniques at low signal levels [27], essentially because the signal intensity has to be high enough so its modulation is practical. Moreover, frequency modulation techniques have been reported to suffer from aliasing [33,35], bleaching [32] and calibration [34] artefacts and a limited dynamic range [57]. However, for bright samples and high signal levels, frequency modulation techniques perform reasonably well and are preferable to time gating where a single gate is moved over the decay, as no photons are lost. We note here that the latest cameras for frequency-domain FLIM detection dispense with an image intensifier and use an all solid state CCD/CMOS camera [58,59], thus allowing phosphor-free frequency domain FLIM, avoiding artefacts associated with the phosphor decay [36].
The performance of a FLIM technique not only depends on its general operating principle, but also on the technical details of its particular implementation. TCSPC-based wide-field FLIM can be implemented using intensifiers with a position-sensitive read-out or SPAD arrays. Wide-field TCSPC FLIM with picosecond time resolution can be performed with MCP-based photon counting image intensifiers [48,60]. Instead of using a phosphor as in gated or modulated image intensifiers, an electronic read-out is employed. Various architectures exist, such as crossed delay line anodes, wedge and strip anodes or quadrant anodes. Quadrant anode [61][62][63][64][65] and delay line anode [66,67] detectors have successfully been combined with picosecond timing and fluorescence microscopy. They typically detect one photon per pulse for the whole field of view, but advanced read-out architectures allowing multi-photon hits to be detected have been designed [68]. Still, the collection of microsecond decays, while certainly possible, would take a long time due to the limited parallel photon detection capabilities of these detectors.
In the last decade a number of SPAD array image sensors have emerged, which simultaneously deliver single photon sensitivity, megapixel spatial resolution and picosecond timing resolution [7][8][9]. These SPAD arrays hold great promise for the advancement of time-resolved fluorescence microscopy, but they currently have a low fill factor and much higher noise levels than MCP-based detectors. The current low fill factor of well below 10% makes their use as cameras for direct FLIM imaging inefficient unless this drawback is overcome, for example with microlenses or multifocal multibeam scanning approaches [69]. The dark noise performance of SPAD arrays, typically hundreds of counts per pixel SPAD, depending on the operating voltage and temperature [8], can be improved to tens of counts per pixel (25 Hz has been quoted [70]) but it is still many orders of magnitude higher than MCP-based intensifiers, for which 0.02 events s −1 cm −2 have been quoted [71]. This consideration gains in importance as the detection window and lifetime to be measured increases, and the count rate drops. It is also an important point to consider when enlarging the size of the light sensitive area in SPADs, as the dark noise increases with the area [8]. To the best of our knowledge, SPAD arrays have not yet been used to image microsecond decays. We note here that intensifiers, as vacuum devices, can also be used in direct ion velocity mapping and time-of-flight mass spectrometry [12], whereas SPAD arrays cannot directly detect ion fragments.
In this present work, two subsequent frames were used for the arrival time determination. Although more frames can be taken into account and a minimisation routine applied to fit the phosphor decay to the measured event intensities [40], it was found that at 54 kHz frame rate and a P20 phosphor the ratio of the first two or three detections provided the most useful information. Although the P20 has a long decay component that allows the events to be detected in up to tens of subsequent frames [38], the intensity is low and the decay so slow that after first three frames the ratio gets very inaccurate due to noise and digitisation effects [72].
The maximum count rate achieved was 144 000 photons s −1 for the Ru(dpp) + 3 2 measurement in water/ glycerol solutions, where the photons were distributed approximately equally over the field of view. Due to the long phosphor afterglow, the photon event pixels were occupied for up to dozens of frames after the event. A phosphor screen with a shorter afterglow would allow the event pixels to be freed quicker for new events. Improvement of the image processing software to better separate or take into account partially overlapping events can also improve the count rate-here care was taken to avoid any overlap of photon events.
If the phosphor decay function and the camera dead-time are known, the ratio-to-time conversion can be calculated using equation (2). In this work, the conversion function was obtained experimentally, because neither the phosphor decay function nor the exact camera dead-time were known. According to the camera manufacturer the Photron SA1.1 dead-time is ∼0.6 μs and does not vary with the frame rate or image size. However, it was found that at 54 kHz frame rate and 320 × 256 pixels image size, measurements with less than ∼3 μs delay between the frame exposure start time and the laser pulse yielded results which did not fit the conversion function (see figure 5(c)). For decay measurements, a 5 μs delay was therefore chosen for the excitation pulse. The calibration and the decay measurements have to be carried out under identical experimental conditions, in particular the intensifier gain settings: a change in the intensifier gain can affect the phosphor decay function [46]. We note here that illumination or emission intensity variations in principle do not affect the decay measurement as TCSPC is independent of this-a change in intensity only changes the number of detected photons, but not their arrival time.
In these experiments phosphorescence lifetimes were measured in the timescale of around ∼0.3 to 5 μs. A result of ∼0.3 μs was obtained for a fluorescent material BPEA where the actual lifetime is a few nanoseconds. This represents the IRF and is a limitation of the current setup with this phosphor and frame rate. 5 μs lifetimes are on the upper end of the measurable scale with this excitation repetition rate, to avoid re-excitation of the sample before it is fully decayed.
By converting the ratio to time, the reflected pulses at different delays after the frame exposure start time can be used as an IRF and to estimate the timing accuracy, as shown in figure 9(a). For the 5 μs delay, the full width at half maximum (FWHM) of the IRF is around 1.5 μs. Although this is quite large, even <1 μs decays can be measured accurately as long as the IRF is recorded before each measurement, and deconvolved from the decay data [6]. The IRF FWHM gets narrower with increasing time and reaches a minimum of about 0.3 μs at around 14 μs delay where the ratio of the 2nd and the 1st detection is one before starting to broaden again, see figure 9(b). Importantly, the number of detected photons remained constant regardless of the photon arrival time, as shown in figure 9(c).
The current time resolution of around 300 ns IRF FWHM is based on a frame rate of 54 kHz and the decay of a P20 phosphor. In order to improve this number, both a faster frame rate and a faster phosphor are required. Frame rates of 2 MHz are possible, albeit only with small images, which allows 500 ns between exposures. P46 phosphors, which advantageously emit in the green where the CMOS sensors are usually most sensitive, have a decay time of around 300 ns [39]. If the excitation pulse is placed near the end of the exposure time, and the decay extends into the second frame (across the dead time) at frames rates of say 500 kHz, then it may well be possible to achieve timing resolutions of the order of tens of nanoseconds. The signal-to-noise ratio will depend on the dead time, the phosphor decay function, the length of the exposure time, and the delay, i.e. the position of the IRF in the frame exposure time. Due to this variety of factors influencing the time resolution, for a given phosphor and frame rate, it is best determined experimentally. While the timing accuracy may allow the measurement and imaging of nanosecond fluorescence decays, thus performing true nanosecond FLIM, the overall count rate may be limited by gain depletion to 10 MHz or so [73].

Conclusion
We have demonstrated that the long-lived, invariant decay of the phosphor screen of the image intensifier can be exploited to find the photon arrival time within the frame exposure time. Whereas our previous approach using a fast camera and a short phosphor decay intensifier (P47) also offered single photon sensitivity and high speed  [11], and could in principle also benefit applications where microsecond timing is not required, the present approach is specifically designed to achieve photon arrival timing. This is relevant for general time-of-flight measurements, e.g. in lidar [74,75], ion velocity mapping [12,41,42] or photon correlation techniques [76], but we have employed this approach for lifetime mapping.
Combined with an ultrafast camera, enough photons can be collected for a lifetime image in a matter of seconds. Sub-microsecond luminescence decays of transition metal compounds were measured with 54 kHz camera frame rate (18.5 μs frame exposure time), including a ruthenium complex Ru(dpp) + 3 2 in living HeLa cells. This complex can be used as a molecular probe for the local oxygen concentration that can be imaged in the presence of strong auto-fluorescence. This technique combines the unique advantages of single photon sensitivity and accuracy with wide-field data collection, enabling the collection of many photons per excitation cycle. Unlike time-gated imaging techniques, every photon is collected, reducing the required illumination intensity and acquisition time. We have used an excitation power more than 1000 times lower than previously reported with time gating [21,22], and acquired images in a few seconds.
The advantages of sensitive wide-field TCSPC imaging as presented here can be applied to imaging modalities such as simultaneous tracking and time decay measurements, total internal reflection fluorescence, supercritical angle fluorescence or light-sheet microscopy, which are difficult or impossible to implement with scanning techniques. By using a phosphor with a faster decay time to match a higher camera frame rate, the measurement of nanosecond decays could be possible with this technique.