HyU: Hybrid Unmixing for longitudinal in vivo imaging of low signal to noise uorescence

: The expanded application of fluorescence imaging in biomedical and biological research towards more complex systems and geometries requires tools that can analyze a multitude of components at widely varying time- and length-scales. The major challenge in such complex imaging experiments is to cleanly separate multiple fluorescent labels with overlapping spectra from one another and background autofluorescence, without perturbing the sample with high levels of light. Thus, there is a requirement for efficient and robust analysis tools capable of 20 quantitatively separating these signals. In response, we have combined multispectral fluorescence microscopy with hyperspectral phasors and linear unmixing to create Hybrid Unmixing (HyU). Here we demonstrate its capabilities in the dynamic imaging of multiple fluorescent labels in live, developing zebrafish embryos. HyU is more sensitive to low light levels of fluorescence compared to conventional 25 linear unmixing approaches, permitting better multiplexed volumetric imaging over time, with less bleaching. HyU can also simultaneously image both bright exogenous and dim endogenous labels because of its high dynamic range. This allows studies of cellular behaviors, tagged components, and cell metabolism within the same specimen, offering a powerful window into the orchestrated complexity of biological systems. Hybrid Unmixing offers enhanced imaging of multiplexed fluorescence labels, enabling longitudinal imaging of multiple fluorescent signals with reduced 35 illumination intensities. pixels, each with a relatively similar spectrum ( E ). Summing these spectra effectively averages the spectra for that phasor position. This denoising results in cleaner average spectrum for this set of pixels, which are ideally suited for analytical decomposition through unmixing algorithms ( F ). (G) Unmixing results in images that separated into spectral components. Here, linear unmixing (LU) is used for unmixing, but HyU is compatible with any unmixing 5 algorithm. surrounding live


Introduction
In recent years, high-content imaging approaches have been refined for decoding the complex and dynamical orchestration of biological processes. 1,2,3 Fluorescence, with its high contrast, 5 high specificity and multiple parameters, has become the reference technique for imaging. 4,5 Continuous improvements in fluorescent microscopes [6][7][8][9] and the ever-expanding palette of genetically-encoded and synthesized fluorophores have enabled the labeling and observation of a large number of molecular species 10, 11 . This offers the potential of using multiplexed imaging to follow multiple labels simultaneously in the same specimen, but the technologies for this have 10 fallen short of their fully imagined capabilities. Standard fluorescence microscopes collect multiple images sequentially, employing different excitation and detection bandpass filters for each label. Recently developed techniques allow for massive multiplexing by utilizing sequential labeling of fixed samples but are not suitable for in vivo imaging. 12,13 Unfortunately, these approaches are ill-suited to separating overlapping fluorescence emission signals, and the 15 narrow bandpass optical filters used to increase selectivity, decrease the photon efficiency of the imaging. (Figs. S1, S2) These limitations have restricted the number of imaged fluorophores per sample (usually [3][4] and risks exposing the specimen to damaging levels of exciting light. This has been a significant obstacle for the dynamic imaging, and has prevented in vivo imaging from reaching its full potential. 20 Hyperspectral Fluorescent Imaging (HFI) potentially overcomes the limitations of overlapping emissions by expanding signal detection into the spectral domain. 14 HFI captures a spectral profile from each pixel, resulting in a hyperspectral cube (x,y, wavelength) of data, that can be processed to deduce the labels present in that pixel. Linear unmixing (LU) has been widely utilized to analyze HFI data, and has performed well with bright samples emitting strong signals 25 from fully-characterized, extrinsic fluorophores such as fluorescent proteins and dyes 15-17 . However, in vivo fluorescence microscopy is almost always limited in the number of photons collected per pixel (due to the expression levels, the bio-physical fluorescent properties, and the sensitivity of the detection system), which reduces the quality of the spectra acquired.
A further challenge which affects quality of spectra is the presence of multiple forms of noise in 30 the imaging of the sample. Two examples of instrumental noise are photon noise and read noise. Photon noise, also known as Poisson noise, is an inherent property related to the statistical variation of photons emission from a source and of detection. Poisson noise is inevitable when imaging fluorescent dyes and is more pronounced in the low-photon regime. It poses challenges especially in live and time lapse imaging, where the power of the exciting laser is reduced to 35 avoid photo-damage to the sample, decreasing the amount of fluorescent signal. Read noise arises from voltage fluctuations in microscopes operating in analog mode, during the conversion from photon to digital levels intensity and commonly affects fluorescence imaging acquisition. Most biological samples used for in vivo microscopy are labelled using extrinsic signals from fluorescent proteins or probes but often include intrinsic signals (autofluorescence). 40 Autofluorescence contributes photons that are undesired, difficult to identify and to account for in LU. The cumulative presence of noise inevitably leads to a degradation of acquired spectra during imaging. As a result, the spectral separation by LU is often compromised, and the Signal to Noise ratio (SNR) of the final unmixing is often reduced by the weakest of the signals detected. 16 Increasing the amount of laser excitation can partially overcome these challenges, but the higher energy deposition in the sample causes photo-bleaching and -damage, affecting both the integrity of the live sample and the duration of the observation. Traditional unmixing strategies such as LU are computationally demanding, requiring long analyses and often slowing the experiment. Combined, these compromises have reduced both the overall multiplexing 5 capability and the adoption of HFI multiplexing technologies.
We have developed Hybrid Unmixing (HyU) as an answer to the challenges that have limited the wider acceptance of HFI for in vivo imaging. HyU employs the phasor approach 18 merged with traditional unmixing algorithms to more rapidly and more accurately untangle the fluorescent signals from multiple exogenous and endogenous labels. The phasor approach 18 , a popular 10 dimensionality reduction approach for the analysis of both fluorescence lifetime and spectral image analysis [19][20][21] provides key advantages to HyU, including spectral compression, denoising, and computational reduction. HyU pairs phasor processing with unmixing algorithms, such as LU, to provide unsupervised analysis of HFI data, removing user subjectivity. Our results show that HyU offers three key advantages: (1) improved unmixing over conventional LU, especially 15 for low intensity images, down to 5 photons per spectra; (2) simplified identification of independent spectral components; (3) dramatically faster processing of large datasets, overcoming the typical unmixing bottleneck for in vivo fluorescence microscopy. The phasor acts as an encoder, where each histogram-bin corresponds to a number n of pixels, each with a relatively similar spectrum (E). Summing these spectra effectively averages the spectra for that phasor position. This denoising results in cleaner average spectrum for this set of pixels, which are ideally suited for analytical decomposition through unmixing algorithms (F). (G) Unmixing results in images that separated into spectral components. Here, linear unmixing (LU) is used for unmixing, but HyU is compatible with any unmixing 5 algorithm.
Note that HyU offers a major reduction in data size and complexity of the LU (or any other unmixing) computation, because the calculation is applied to the 10 4 histogram bins (D), rather the the ~10 7 voxels in the specimen (A). This reduces the number of calculations required for LU dramatically. 10

Results
HyU combines the best features of hyperspectral phasor analysis and linear unmixing (LU), resulting in faster computation speeds and more reliable results, especially at low light levels. Phasor approaches reduce the computational load because they are compressive, reducing the 32 15 channels of an HFI spectral plot into a position on a 2D-histogram, representing the real and imaginary Fourier components of the spectrum (Fig. 1A,B). Different 32 channel spectra are represented as different positions on the 2D phasor plot, and mixtures of the two spectra will be rendered at a position along a line connecting the pure spectra. Because the spectral content of an entire 2D or 3D image set is rendered on a single phasor plot, there is a dramatic data 20 compression -from a spectrum for each voxel in an image set (up to or even beyond Gigavoxels) to a histogram value on the phasor plot (Megapixels). In addition, because each "bin" on the phasor plot histogram corresponds to multiple voxels with highly similar spectral profiles, the binning itself represents spectral averaging, which reduces the Poisson and instrumental noise ( Fig. 1C-E). Poisson noise in the collected light is unavoidable in HFI unless the excitation is 25 turned so high that the statistics of collected fluorescence creates hundreds or thousands of photons per spectral bin. The clear separation of the spectral phasor plot and its referenced imaging data, permits denoising algorithms to be applied to phasor plot with minimal degradation of the image resolution. LU or other unmixing approaches can be applied to the spectra on the phasor plot, offering a dramatic reduction in computational burden for large image 30 data sets (Fig. 1D). To understand this saving, consider the conventional approach of LU applied to image data at the voxel level ( Fig. 1A,F). A timelapse volumetric dataset of 512x768x17 (x, y, z) pixels, over 6 timepoints, (Sup. table 1), would require 40 million operations. HyU's requires only ~18 thousand operations to unmix each bin on the phasor plot, representing more than a thousand-fold saving (Fig. 1F,G). 35 To quantitatively assess the relative performance of LU and HyU, we analyzed them on synthetic hyperspectral fluorescent datasets, created by computationally modelling the biophysics of fluorescence spectral emission and microscope performance (Fig 2 A, B, figs. S3-S5). We used this synthetic dataset to evaluate LU and HyU algorithm performance quantitatively by using 40 metrics such as Mean Square Error (MSE) and unmixing residual (see Fig. S6, Methods; for both metrics, a lower value indicates better performance). In addition to the computational efficiency mentioned above, HyU analysis shows better ability to capture spatial features over a wide dynamic range of intensities, when compared with standard LU, in large part due to the denoising created by processing in phasor space ( Fig. 2 A, B). The improved accuracy is 45 demonstrated by a lower MSE, in comparing the results of LU and HyU to the image ground truth. The absolute MSE for HyU is consistently up to 2x lower than that of LU, especially at low and ultra-low fluorescence levels (Fig. 2C). MSE can be further decreased by the use of denoising filters on the phasor plot, resulting in superiority of HyU relative to LU for HFI at low (5-20 photons/spectrum) and ultralow (2-5 photons/spectrum) levels ( Fig. 2D). To better characterize the performance in the experimental data without ground truth, we also define the unmixing residual as the difference between the original multichannel hyperspectral images and 5 their unmixed results. Residuals provide a measure of how closely the unmixed results reconstruct the original signal (Fig. S3, Methods). Unmixing residuals are inversely proportional to the performance of the algorithm, with low residuals indicating high similarity between the unmixed and the original signals. Analysis of unmixing residuals in the synthetic data highlights an improved interpretation of the spectral information in HyU with an average unmixing residual 10 reduction of 21% compared to the standard (Fig. S5C). The reduction in both MSE and average unmixing residual for synthetic data demonstrates the superior performance of HyU, and provides a baseline comparison when demonstrating performance improvements for experimental data.
We support the enhanced performance of HyU with analysis of experimental data, which reveals 15 comparatively lower unmixing residuals and a higher dynamic range as compared to LU. Data was acquired from a quadra-transgenic zebrafish embryo Tg(ubiq:Lifeact-mRuby);Gt(cltcacitrine);Tg(ubiq:lyn-tdTomato);Tg(fli1:mKO2), labelling actin, clathrin, plasma membrane, and pan-endothelial cells, respectively (Figs. 2E-L, 3, S7-S9, Supplementary Movie 1). HyU unmixing of the data shows minimal signal cross-talk between channels while LU presents 20 noticeable bleed-through ( Fig. 2M-P). Consistently with synthetic data, we utilize the unmixing residual as the main indicator for quality of the analysis in experimental data, owing to the absence of a ground truth. The residual images (Fig. 2F, G) depict a striking difference in performance between HyU and LU. The average relative residual of HyU denotes a 7-fold improvement compared to LU (Fig. 2H) in disentangling the fluorescent spectra. We visualize  improvements in the unmixing results. HyU has an unmixing residual of 6.6% ± 5.3% compared to LU's 58% ± 17%. The average amount of residual is 9-fold lower in HyU with narrower variance of residual.
Applying HyU to another HFI dataset further highlights HyU's improvements in noise reduction and reconstitution of spatial features for low-photon unmixing. (Figs. 3, S8). In the zoomed-in 15 image of a single slice of the embryo skin surface, acquired in the trunk region, the HyU image correctly does not display pan-endothelial (magenta) signal in the periderm, an area which should be devoid of endothelial cells and mKO2 signal (Fig. 3C). In contrast, the result from LU shows visually distinctive pan-endothelial signal throughout the tissue plane (Fig. 3D). This incorrect estimation of the relative contribution of mKO2 fluorescence for LU is possibly due to 20 the presence of noise, corrupting the spectral profiles. This is further delineated in the intensity profiles of the mKO2 signal between HyU and LU with much higher individual peaks from noise demonstrated for LU (Fig. 3G, lower left). Intensity profiles for both magnified cross-sections of the volume (Fig. 3C-F) provide a striking visualization of the improvements of HyU. The line intensity profiles in HyU present reduced noise and represent more closely the expected 25 distribution of signals (Fig. 3G,H). The visible micro patterns of actin on the membrane of the periderm suggest that the improvements quantified with synthetic data are maintained in live samples' signals and geometrical patterns of microridges 22 . By contrast, noise corruption and the presence of misplaced signals are characterized in the results from LU, with high frequency intensity variations that mis-match both the labeling and biological patterns. 30 HyU is more accurate and results in more reliable unmixing results across the depth of sample with greatly reduced unmixing residuals. The average residual for HyU is 9-fold lower than that of LU with a 3-fold narrower variance. (Figs. 3I, S8). This reduction in the residual is consistent with increasing z-depth where HyU unmixing results stably maintain both lower residuals and 35 variance on average. These reduced residuals correspond both to a mathematically more precise and more uniform decomposition of signals as illustrated by the distribution of residuals versus photons (Figs. S8E,F, S14). We utilized HyU's increased sensitivity to overcome common challenges of multiplexed imaging such as poor photon yield and spectral cross-talk and were able to visualize dynamics in a developing zebrafish embryo. We used a triple-transgenic zebrafish embryo with labeled pan-15 endothelial cells, vasculature, and clathrin-coated pits (Tg(fli1:mKO2); Tg(kdrl:mCherry); Gt(cltca-Citrine)). Multiplexing these spectrally close fluorescent proteins is enabled by HyU's increased sensitivity at lower photon counts. The increased performance at lower SNR allowed us to maintain high quality results (  (Fig. 4B) 23,24 .
HyU provides the ability to combine the information from intrinsic and extrinsic signals during live imaging of samples, at both single (Fig. 5) and multiple time points (Fig. 6). The graphical 5 representation of phasors allows identification of unexpected intrinsic fluorescence signatures in a quadra-transgenic zebrafish embryo Gt(cltca-citrine);Tg(ubiq:lyn-tdTomato;ubiq:Lifeact-mRuby;fli1:mKO2), imaged with single photon (488 and 561nm excitation) (Fig. 5A-D). The elongated distribution on the phasor (Fig. 5C) highlights the presence of an additional, unexpected spectral signature, related to strong sample autofluorescence (Fig. 5D blue). HyU 10 analysis of the sample, inclusive of this additional signal, provides separation of the contributions of 5 different fluorescent spectra with residual 3.9%±0.3%. HyU allows for reduced energy load, tiled imaging of the entire embryo without perturbing its development or depleting its fluorescence signal (Fig. 5A). The higher speed, lower power imaging allows for subsequent re-imaging of the same sample, as we report in the zoomed high-resolution 15 acquisitions of the head section (Fig. 5B,E).
With the ability to unmix low photon signals, HyU enables imaging and decoding of intrinsic signals, which are inherently low light. Two photon lasers are ideal for exciting and imaging blue-shifted intrinsic fluorescence from samples 25,26 . Here, the same quadra-transgenic sample is imaged using 740 nm excitation to access both intrinsic and extrinsic signals (Fig 5 E-G, sup 20 Note 2). HyU enables unmixing of at least 9 intrinsic and transgenic fluorescent signals (Fig. 5), recovering fluorescent intensities from labels illuminated at a sub-optimal excitation wavelength (Fig. 5E). The spectra for intrinsic fluorescence were obtained from in vitro measurements and values reported in literature (Methods). For this sample the intrinsic signals arise from events related mainly with metabolic activity (NADH and Retinoids) 27-31 , tissue structure (elastin) 32 , 25 and illumination (laser reflection) (Fig. 5E). These results confirm our conclusion that HyU is a powerful tool for allowing the imaging and analysis of endogenous labels. HyU's increased sensitivity provides a simple solution for the challenging task of imaging timelapse data at 6 time points (125 mins) for both intrinsic signals and extrinsic signals of a quadra-transgenic zebrafish: Tg((cltca-Citrine);(ubiq:lyn-tdTomato);(ubiq:Lifeact-mRuby);(fli1:mKO2)). (A) -(F) Volumetric renderings of HyU results for time points acquired at 25 min intervals reveal the high-contrast and -multiplexed labels of NADH bound (red), NADH free (yellow), retinoid (magenta), retinoic acid (cyan), mKO2 (green), and autofluorescence from blood cells 10 (blue) when excited @740nm. Further extrinsic signals for mKO2 (yellow), tdTomato (magenta), mRuby (cyan), Citrine (green) and blood cells autofluorescence (blue) are also readily unmixed using HyU when exciting the sample @ 488/561nm. HyU provides the capacity to simultaneously multiplex 9 signals in a live sample over long periods of time, a previously unexplored task. Scale bar: 50 µm.

15
Finally, we exploited the HyU capabilities to multiplex volumetric timelapse of extrinsic and intrinsic signals by imaging the tail region of the same quadra-transgenic zebrafish embryo. We excite extrinsic labels at 488/561 nm and the intrinsic signals with 740 nm two photon, collecting 6 tiled volumes over 125 mins (Figs. 6, S9-S11, Supplementary Movie 3). HyU unmixing in this sample allows for distinction of 9 signals, separating their contributions with sufficiently low requirements to allow repeated imaging of notoriously low SNR intrinsic fluorescence.

5
Our results reveal the advantages of Hybrid Unmixing (HyU) over more conventional Linear Unmixing (LU) in performing complex multiplexing experiments. HyU overcomes the significant challenges of separating multiple fluorescent and autofluorescent labels with overlapping spectra while minimally perturbing the sample with excitation light.
The chief advantage of HyU is its multiplexing capability when imaging in the presence of 10 biological and instrumental noise, especially at low signal levels. HyU increased sensitivity improves multiplexing in photon limited applications (Fig. 2F-L), in deeper volumetric acquisitions (Fig. 3I) and in signal starved imaging of autofluorescence (Fig. 5E, Fig. 6). Our simulation results (Fig. 2) demonstrate that HyU improves unmixing of spatially and spectrally overlapping fluorophores excited simultaneously. The increased robustness at low photon 15 imaging conditions reduces the imaging requirements for excitation levels and detector integration time, allowing for imaging with reduced photo-toxicity. Live imaging on multi-color samples performed at high sampling frequency enables improved tiling to increase the field-ofview (Fig. 3, 4) while maximizing the usage of the finite fluorescent signals over time. Twophoton imaging of intrinsic and extrinsic signals suggests the ability of HyU to multiplex signals 20 with large dynamic range differences (Fig. 5) extending multiplexed volumetric imaging into the time dimension (Fig. 6). Although improved, images with particularly low signal still present corruption (Fig. S4), setting a reasonable range of utilization above 8 photons/spectrum. The simplicity of this approach is especially helpful in live imaging where identifying independent spectral components remains an open challenge, owing to the presence of intrinsic signals (Fig. S12, Supplementary Note 1). High-SNR reference spectra can be derived from other experimental data or identified directly on the phasor. Selection of portions on the phasor 35 plot allows for visualization of the corresponding spectra in the wavelength domain ( Fig  5C,D,F,G). This intuitive versatility allows for identification of both the number of unexpected signatures and their spectra, a task previously difficult to perform due to noise and lack of global visualization tools. In single photon imaging (Fig. 5A-D), HyU phasor allowed identification of a fifth distinct spectral component arising from general autofluorescent background, thereby 40 improving the unmixed results. In two photon imaging, HyU enabled identification and multiplexing of 8 highly overlapping signals possessing a wide dynamic range of intensities, between intrinsic and extrinsic markers (Fig. 5F,G). Combination of single and two photon imaging increased the number of multiplexed fluorophores to 9 (Fig. 6), considering some of the extrinsic labels being excited at two photons. Multiplexing of signals may be further improved by implementing HyU on fluorescent dyes.
HyU performs better than standard algorithms both in the presence and absence of phasor noise reduction filters 33 . Compared with LU, the unmixing enhancement when such filters 33 are applied is demonstrated by a decrease of the MSE of up to 21% (Fig 2C), with a reduction of the 5 average amount of residuals by 7-fold. Even in the absence of phasor denoising filters, HyU performs up to 7.3% better than the standard (Fig. 2D) based on Mean Squared Error of synthetic data unmixing. This base improvement is due to the averaging of similarly shaped spectra in each phasor histogram bin, which reduces the statistical variability within the spectra used for the unmixing calculations (Fig. 1E). This averaging strategy works well for general fluorescence 10 spectra owing to their broad and mostly unique spectral shape.
In the absence of noise, for example in the ground truth simulations, LU produces an MSE 6-fold lower than HyU (Fig. S5, B, C, S6G). In these noiseless conditions, the binning and averaging of spectra in the phasor histogram, without denoising, provides statistically indifferent values of error respect to LU, suggesting results of similar quality.

15
HyU can interface with different unmixing algorithms, adapting to existing experimental pipelines. We successfully tested hybridization with iterative approaches such as non-negative matrix factorization 34 , fully constrained and non-negative least-squares 35 (Methods). Speed tests with iterative fitting unmixing algorithms demonstrate a speed increase of up to 500-fold when 20 the HyU compressive strategy is applied. (Fig. S13, Supplementary Note 3). Due to the initial computational overhead for encoding spectra in phasors, there is a 2-fold speed reduction for HyU in comparison to standard LU. However, this may be improved with further optimizations of the HyU implementation.
One restriction of HyU derives from the mathematics of linear unmixing, where linear equations 25 representing the unmixed channels need to be solved for the unknown contributions of each analyzed fluorophore. To obtain a unique solution from these equations and to avoid an underdetermined equation system, the maximum number of spectra for unmixing may not exceed the number of channels acquired 36 , generally 32 for commercial microscopes. This number could be increased; however, due to the broad and photon-starved nature of fluorescence 30 spectra, acquisition of a larger number of channels could negatively affect the sample, imaging time and intensities. Depending on the number of labels in the specimen of interest, extending the number of labels to simultaneously unmix beyond 32 will likely require spectral resolution upsampling strategies.
HyU improvement is related to the presence of various types of noise in microscopy images, 35 such as Gaussian, Poisson and digital as well as unidentified sources of spectral signatures (Fig.  S5B,C, S6G). In the multiplexing of fluorescent signals, HyU offers improved performance, quality-and speed-wise in the low-signal regime. HyU is poised to be used in the context of in vivo imaging, harvesting information from samples labeled at endogenous-level. 40 In conclusion, the results presented in this paper quantitatively show that HyU, a phasor based, computational unmixing framework, is well suited for tackling the many challenges present in live imaging of multiple fluorescence labels. HyU's reduced requirements in amount of fluorescent signal permit a reduction of laser excitation load and imaging time. These factors enable multiplexed imaging of biological events with longer duration, higher speed and lower photo-toxicity while providing access to information-rich imaging across different spatiotemporal scales. The reduced requirements of HyU make it fully compatible with any commercial and common microscopes capable of spectral detection, facilitating access to the 5 technology. Our analysis demonstrates HyU's robustness, simplicity and improvement in identifying both new and known spectral signatures, and vastly improved unmixing outputs, providing a much-needed tool for delving into the many questions still surrounding studies with live imaging.