A handheld wide-field fluorescence lifetime imaging system based on a distally mounted SPAD array

.


25
Fluorescence imaging is a powerful tool for the analysis of materials, particularly in the context 26 of biological applications, as many biomolecules exhibit auto-fluorescence upon illumination.

27
This emitted light may be used as a fingerprint of the materials present, as well as of their local 28 environment. Fluorescent Lifetime IMaging (FLIM) differs from traditional fluorescence 29 imaging in that rather than just the intensity or spectra of the emitted light, time resolved 30 detection systems are used to obtain the characteristic fluorescent lifetime. This can have 31 applications in fields as diverse as biomedicine [1][2][3][4][5][6][7][8][9][10], plant science [11][12][13], or chemical 32 sensing [14]. The primary benefit of FLIM over traditional fluorescence intensity imaging is 33 that lifetime is largely independent of the density of fluorescent chromophores and excitation 34 power, giving consistent contrast between regions with differing molecular makeup. A 35 particularly promising avenue for FLIM applications is for surgical guidance and endoscopy 36 [2][3][4][5][6][7][8], where FLIM can provide label free contrast between tissue types which may not be 37 apparent when using white light imaging, or fluorescence intensity alone. This is particularly 38 useful when looking at cancer margins, as cancerous tissue has been shown to have a different 39 characteristic fluorescence lifetime compared to surrounding healthy tissue [2][3][4]10]. Several 40 different biomedical FLIM systems have been demonstrated in the literature, generally all 41 employ a pulsed laser source to induce fluorescence (either endogenous or from labels) which 42 is then collected and relayed down a fibre to an image sensor at the proximal end. The spatial 43 resolution is then either achieved using scanning optics at the proximal end of the system [7], the use of fibre imaging bundles [1,15], or by raster-scanning a fibre acting as a point probe 45 over the object being imaged [6,10,16,17].

46
FLIM depends upon being able to temporally resolve the fluorescence signal. There are 47 several methods for achieving the necessary time resolution required to perform FLIM, such as 48 time gated optical intensifiers [2] and high speed digitisers [17,18] but one of the most robust 49 and elegant approaches is the use of SPADs where timing electronics for the imaging pixel are 50 integrated at a chip level [8]. Once SPADs are combined into arrays they may become an even 51 more powerful tool. SPAD array line sensors are very well suited to spectrally resolved 52 measurements, allowing for FLIM to be carried out at multiple spectral bands simultaneously 53 [8,19], while 2D SPAD arrays can effectively act as time resolved cameras capable of rapidly 54 performing wide-field FLIM [20,21].

55
This group has previously demonstrated Endocam, a novel SPAD array specifically 56 designed to perform FLIM in a chip-on-tip fashion, i.e. the SPAD array itself will sit on the 57 distal end of the system with images relayed back to the control unit via a wired data connector 58 [22,23]. This stands in contrast to the proximally mounted sensor of all the FLIM systems 59 described previously and allows a vast simplification of the opto-mechanics required to 60 reconstruct the image. Such systems do not face the same limitations with regards bending 61 radius as fibre systems, and due to the inherent scalability of electronics versus optics may be 62 a lower cost solution [24,25]. Although fluorescence endoscopy systems with an image sensor 63 on the distal end of the probe do exist, these only provide steady state fluorescence intensity 64 rather than FLIM [26][27][28].

65
For surgical guidance and other diagnostic or analytic applications, it is highly valuable for 66 the FLIM system to be flexible and mobile enough such that it can be operated in a handheld 67 fashion e.g. to image a patient undergoing surgery from different directions without having to 68 move and disturb them. For endoscopy applications, any chip-on-tip FLIM system has to be 69 able to operate at a distance from its control unit. In this work, both of these goals are achieved, 70 clearly demonstrating the potential to integrate time gated CMOS SPAD arrays into an 71 endoscopy system.

72
To the best of the authors' knowledge, the system presented here based on the Endocam 73 chip is also the first example of a time resolved SPAD array on the distal end of a handheld

89
A photograph of the handheld system is shown in Fig. 1

103
The intensity image of a ruler in Fig. 1(d) shows the field of view for the system, ~2.4 cm at a 104 working distance of approximately 6 cm. Also note that one of the corners of the image is 105 corrupted (bottom right corner of Fig. 1(d)). This is due to a hardware issue which does not  Briefly, in its current iteration the Endocam die is mounted in a CPGA68 chip carrier 109 package, though the Endocam die itself is only < 2 mm 2 , which will allow much smaller chip 110 carrier packages to be used in future versions. The chip consists of 120 x 128 pixels each 111 comprising of an individual SPAD, SPAD front-end circuitry, and a 14-bit photon counter. The 112 chip also features a micro-controller unit (MCU) and two 16 bit static random access memory 113 (SRAM) blocks to allow successive frames generated from up to 65535 exposure cycles to be added together on chip (a process we will hereafter refer to as frame additions). Although this 115 increases the footprint of the chip somewhat, the SRAM blocks take up less area per bit than 116 the on-pixel photon counters and also cuts down the need for time consuming data transfer off 117 the chip. Thus they allow for a high bit depth without overly compromising the form factor or 118 frame rate of the system [23]. The chip requires only five wires to run-1) 18.5 V supply for the 119 SPAD bias, 2) 2.8 V supply for the on-chip power generation network, 3) Ground, 4) Data I/O,   the chip, such that the chip could not boot correctly. Various other cables were explored, but 143 eventually shielded multicore cable (Alphawire 6305 SL005), which was less susceptible to 144 external electrical interference, was found to be effective and allowed the length of the cable to 145 be increased to ~ 1 m. In this state the chip was still only running at 10 MHz, which was deemed 146 too low a clock rate for practical applications. This then required multiple rounds of firmware 147 revision to optimise the form of the clock signal such that it could be delivered at a higher 148 frequency while maintaining fidelity. Although a clock rate of 37.5 MHz is achievable with the 149 chip mounted directly on its motherboard [23], the 20 MHz presented here was ultimately the 150 maximum which we could achieve in this configuration. Although lowering the clock rate has 151 a corresponding impact on the frame rate, 20 MHz was deemed suitable for this study as it 152 avoided significant fluorescence wrap around from the longest lived chromophores.

153
Additionally, these firmware upgrades provided the opportunity to add some other additional 154 capabilities to the system as it is described in refs. [22,23], namely the generation of the 155 necessary voltage levels for chip operation on the motherboard, freeing the system from 156 requiring a bulky external benchtop power supply unit, and the ability to use an externally 157 generated TTL or NIM clock signal (for example from a laser driver) as the master clock for 158 the system. A clock rate of 20 MHz is commonly used as the fixed repetition rate of many 159 super-continuum lasers, which may be employed in future versions of the system, so the ability 160 to run at 20 MHz and use an external master clock was deemed a useful addition.

189
Orange and green fluorescent targets were 3D printed in the shape of the letter "E" using reels

223
However, the lifetime histogram in Fig. 2

272
Images of a barley plant undergoing an aphid infestation were obtained and Fig. 3(d) shows the 273 intensity images for the auto-fluorescence from two leaves from this plant -the upper leaf had 274 been substantially damaged whereas the lower leaf was not. The corresponding intensity image 275 shows differences between these two leaves, with the lower, undamaged leaf demonstrating a 276 much more homogenous intensity response than the upper, damaged leaf which has bright and 277 dark regions. However, it is possible this difference in intensity could simply be due to e.g. un-278 even illumination due to shadows. Fig. 3(e) shows the corresponding lifetime image. Again,

286
The images shown in Figs. 2 and 3 were obtained with acquisition times of 10 to 20 seconds.

287
This shows the capability of the system in achieving high resolution "snapshots" of static 288 objects, which may be of some value in e.g. diagnostic imaging. However, for other 289 applications such as endoscopy, higher frame rates are more important. Fig. 4

305
We include further videos obtained using this gating scheme in the supplementary material,

306
one of a selection of 3D printed targets imaged with 65534 exposure cycles and 1 frame 307 addition, and one of two leaves picked from an evergreen shrub with 65534 exposure cycles 308 and 10 frame additions, where we achieve frame rates of > 2 Hz and 1.7 Hz respectively. In 309 these videos we can clearly see the fluorescent lifetime contrast between the differing 3D 310 printed targets, and more importantly, the lifetime contrast in the auto-fluorescence of the

339
In this work a novel system which employs a purpose designed miniaturised SPAD array for 340 flexible chip-on-tip fluorescence lifetime imaging is demonstrated. This system is capable of 341 frame rates > 1 Hz, has a form factor small enough that it may be held between the thumb and 342 forefinger of an experimentalist or clinician, and operates over ~1 m of cable and optical fibre 343 to allow targets to be imaged from different angles or distances without the need to move the 344 patient or object under investigation. Initial demonstrations on plant and animal tissues provide 345 high resolution stills and > 1 Hz frame rate videos. FLIM contrast from this system is capable 346 of showing the difference between different tissue types and damaged and healthy tissues. The 347 authors believe this is the first example of a handheld FLIM system with a distally mounted 348 image sensor, and has achieved the operating range and sensitivity required of a biomedical

349
imaging system. This is thus the first step in developing a miniaturised chip-on-tip endoscopy 350 system capable of in-vivo imaging.

355
For the purpose of open access, the author has applied a Creative Commons Attribution (CC 356 BY) licence to any Author Accepted Manuscript version arising from this submission.

357
Innes Centre, Norwich, UK) for providing seeds of the Barley variety "Digger". SB was the 362 recipient of an EastBio PhD Studentship.

365
The authors declare no conflicts of interest.

367
Data availability. Data underlying the results presented in this paper are not publicly available 368 at this time but may be obtained from the authors upon reasonable request.

370
See Supplement 1 for supporting content.