Sparse decomposition light-field microscopy for high speed imaging of neuronal activity

One of the major challenges in large scale optical imaging of neuronal activity is to simultaneously achieve sufﬁcient temporal and spatial resolution across a large volume. Here, we introduce sparse decomposition light-ﬁeld microscopy (SDLFM), a computational imaging technique based on light-ﬁeld microscopy (LFM) that takes algorithmic advantage of the high temporal resolution of LFM and the inherent temporal sparsity of spikes to improve effective spatial resolution and signal-to-noise ratios (SNRs). With increased effective spatial resolution and SNRs, neuronal activity at the single-cell level can be recovered over a large volume. We demonstrate the single-cell imaging capability of SDLFM with in vivo imaging of neuronal activity of whole brains of larval zebraﬁsh with estimated lateral and axial resolutions of ∼ 3.5 µ m and ∼ 7.4 µ m, respectively, acquired at volumetric imaging rates up to 50 Hz. We also show that SDLFM increases the quality of neural imaging in adult fruit ﬂies. © 2020 Optical


INTRODUCTION
Optical imaging of neuronal activity using fluorescent indicators [1][2][3][4][5][6] has become a popular method of recording the population activity of neurons in order to study neural circuit dynamics, owing to the low invasiveness, high spatial resolution, and potential scalability of such strategies.However, imaging activity in vivo at high speed still remains a challenging task in part due to the fact that most optical imaging systems are only suitable for imaging a single plane at a time, whereas the brain is three-dimensional (3D).In other words, most fluorescence microscopy methods developed for volumetric imaging are based on time multiplexing (e.g., sequential plane-by-plane imaging), and therefore have inherently low temporal resolution-usually slower than two-dimensional (2D) imaging by a factor equal to, or larger than, the number of planes to be imaged [4,7].Neuronal activity can occur at rates of 50-100 Hz [8], and beyond, and resolving the temporal ordering of events of different neurons-important for understanding how information is processed throughout brain circuits-requires even higher temporal resolution.Furthermore, recently developed genetically encoded voltage indicators are opening up the possibility of direct measurement of action potentials across large brain volumes 2334-2536/20/101457-12 Journal © 2020 Optical Society of America [9].This leaves substantial need for further optical technology development [10,11].
Fluorescence microscopy has a fundamental limit in terms of the number of photons that can be collected in a given time interval, which manifests itself as a trade-off among resolution, speed, and noise [12], which in turn necessitates an optical system design with an optimal trade-off between these variables.Light-field microscopy (LFM) is well suited for this purpose, as it images a large 3D volume at the speed of the camera frame rate, it continuously excites all fluorophores in the imaging volume with the minimum instantaneous peak power, and it makes excellent use of digitization bandwidth, but at the cost of reduced spatial resolution [13][14][15][16].In a recent collaboration, LFM was able to image neural activity throughout entire C. elegans and entire larval zebrafish brains [15], but the spatial resolution obtained from zebrafish was below single-cell resolution, requiring independent component analysis (ICA) to segment the neurons and to extract the activities of neurons.More recently, extended field of view LFM (XLFM) [17] was developed; an advanced form of LFM that allows simultaneous optimization of imaging volume and spatial resolution and also avoids square-shaped artifacts near the focal plane by placing the microlens array on the pupil plane of the system.This technique has demonstrated its potential by imaging whole-brain neural activity of freely behaving larval zebrafish, with both improved spatial resolution and imaging volume coverage compared to earlier light-field methods, but higher resolution is still desired to minimize the crosstalk between the signals from neurons and, in particular, to be able to record neural activity at single-cell resolution across densely labeled entire brains.Multiple methods were developed to extract neuronal activity from lightfield images with below-single-cell resolution in densely labeled brains [16,18] by computationally demixing the optically mixed signals using time-domain information.Notably, compressive LFM [18] decomposed the raw data into multiple sub-videos using ICA so that each sub-video exhibited spatial sparsity and then extracted the location and activity of each neuron with higher accuracy with a compressive sensing approach.Technically, these demixing approaches formulate linear inverse problems to infer neuronal activity directly from raw light-field recordings instead of obtaining volumes first and then extracting activity from the volumes.While these methods offer powerful ways to improve the accuracy of neuronal location and activity, they require multiple assumptions to be met.First, all of the active neurons in the field of view must be accurately detected during the pre-processing steps, which is non-trivial even with diffraction-limited resolution.Second, time-varying components in the images that are not due to neuronal activity (e.g., illumination artifacts, blood flow) may appear as artifacts in the extracted signal, as such components are not modeled by the linear inverse problem.

SPARSE DECOMPOSITION LIGHT-FIELD MICROSCOPY
Here, we introduce a method that further improves the ability of XLFM [Fig.1(a)] to resolve neuronal activity by taking advantage of the inherent temporal sparsity of spikes for volume reconstruction of light-field images.Instead of attempting to directly extract neuronal activity, our method first performs high resolution volume reconstruction, which allows us not only to gauge image quality, which is important for microscopy-oriented experiments, but also to observe things beyond neuronal activity and apply conventional image processing techniques as necessary.The idea of improving spatial resolution by employing the temporal sparsity of fluorescent signals to introduce spatial sparsity in images has been widely explored and implemented by many super-resolution optical microscopy methods [19][20][21], where they turn fluorescence molecules on and off with various means and leave each molecule turned off for most of the time so that only a small subset of fluorescent molecules is imaged at a time (i.e., spatial sparsity is gained from temporal sparsity), which then can be localized with an accuracy beyond the optical system's resolution.Here, we take a similar approach and exploit the natural activity-dependent fluorescence intensity change of each neuron, instead of turning each molecule on and off, as the origin of spatial sparsity.Unlike the fluorescent molecules used for super-resolution microscopy that can be completely turned off, genetically encoded calcium indicators have strong baseline fluorescence, and oftentimes there are other sources of fluorescence that contribute to the background of the image, which causes the image to be not sparse, even when all neurons are silent.To circumvent this, we decompose the image into a non-negative low rank component and a non-negative sparse component by applying a decomposition algorithm to the raw data [Fig.1(b)].After the decomposition, volume reconstruction can be applied to the sparse component rather than the raw images [Figs.1(c) and 1(d)], which can improve the effective spatial resolution (or localization accuracy) for two reasons.First, the volume reconstruction in XLFM from light-field images is essentially limited-angle computed tomography, which performs better when the object is sparse [17].Second, since the solution to the inverse problem for each sparse component is sparse, regularization techniques can be applied for the volume reconstruction to take advantage of the sparsity and further improve the accuracy of the solution.
The pre-reconstruction sparse decomposition is innovative in that it resolves a key challenge in light-field volume reconstruction, known as the 'long-object' problem in tomographic imaging.Different from conventional neural imaging techniques, a 3D LFM image is recovered by decoding and integrating the information in a 2D image.This inverse problem can be accurately solved when the sample is spatially sparse (i.e., the signals in the 2D image are not strongly entangled).However, when it comes to imaging a densely packed sample (e.g., a pan-neuronally labeled larval zebrafish brain), conventional reconstruction algorithms fail to faithfully restore the 3D information that is highly mixed in the 2D image, leading to serious resolution and signal-to-noise ratio (SNR) decline.This problem has been observed in various light-field papers [15,17].The core idea of sparse decomposition LFM (SDLFM) is that we can exploit the temporal sparsity of neural activity and convert that temporal sparsity into spatial sparsity of the 2D images, which allows us to achieve the level of resolution expected for sparse samples, in densely packed ones.On the contrary, although post-reconstruction background-reducing methods can generate 'cleaned-up' data, they do not help alleviate the 'long-object' problem in the reconstruction and, therefore, cannot improve the effective resolution of LFM images.

A. Performance Verification via Simulation
Although neural spikes are inherently temporally sparse, the sparsity is not perfect.Neural circuits are known to show temporal synchronicity in their activities [22] that would impede efficient translation of temporal sparsity of spikes into spatial sparsity in images, especially with the slow kinetics of calcium indicators.In order to quantitatively assess the benefit of SDLFM in the presence of such synchronicity, we ran optical simulations of XLFM using a synthetic volume with a size, shape, and neuron density matched to that of a larval zebrafish brain and assessed XLFM versus SDLFM with different levels of synchronicity, defined as the proportion of neurons that are simultaneously active (i.e., calcium concentration above baseline).There were 80,000 neurons simulated, and the rise time and decay time constants of GCaMP6f, 45 ms and 142 ms, respectively [1], were used.With an average firing rate of 0.3 Hz, 10% of the neurons were simultaneously active within an exposure time of 50 ms [Fig.2(a)].Simulated acquired light-field images were volume reconstructed with conventional XLFM reconstruction as well as SDLFM reconstruction, and then the signals from all neurons were extracted.The extracted signals were compared to the ground truth, and the resulting correlation coefficients were evaluated.The signals from SDLFM showed more stable baselines due to less signal mixing [Fig.2(a)], and, as the result, ∼100% of the neurons [Fig.2(b)], over 95% of the neurons [Fig.2(c)], and over 80% of the neurons [Fig.2(d)] were reconstructed with correlation coefficients above 0.9, when 6.3%, 12.5%, and 25% of the neurons were simultaneously active, respectively, whereas the correlation coefficients of less than 10% of the neurons were above 0.9 with XLFM in the 6.3% case [Fig.2(e)].The Pearson correlation coefficients among the simulated neuronal traces, a numerical measure of the synchronicity of the simulated activity, were 0.056, 0.119, and 0.132 when 6.3%, 12.5%, and 25% of the neurons were simultaneously active, respectively.We note that we deliberately kept the noise levels low (Table S1 of Supplement 1), by a factor of 10 lower compared to practical noise levels attainable with current technologies, in order to clearly demonstrate the correlation coefficient improvement; otherwise, the correlation coefficients become noise-limited and not resolution-limited (i.e., our simulation is performed in the resolution-limited regime to assess the level of signal mixing).While noise is currently a common issue in calcium imaging, it has to be distinguished from the correlation coefficient limitation due to signal mixing as a result of spatial resolution limitations.Deterministic signal mixing introduces artifactual correlations among the activities of neurons, which impedes proper analysis of functional connections, whereas noise is stochastic and does not introduce artifactual correlations.Simulation results from the noise-limited regime showed that the performance of SDLFM approaches that of XLFM at noise levels where the brightness level contributed by the neuronal activity is similar to that contributed by the noise of the background (Fig. S1 of Supplement 1).In such a case, the decomposed sparse component and its reconstruction were no longer sparse, which means the source of resolution improvement disappeared.
What is the worst-case scenario with SDLFM, e.g., as neural synchrony approaches the (physiologically unrealistic) extreme value of 100%?With XLFM, the signals were accurately reconstructed when the spatial density of the neurons was low [Fig.2(f )], and SDLFM achieved higher performance than XLFM by further reducing the effective density of the neurons at plausible synchrony levels [Fig.2(g)], since the volume reconstruction was performed only for the neurons that were active during the exposure time of a given frame.In particular, for realistic values of synchrony, the correlation coefficients could be very close to perfection [Fig.2(g)], but, in the worst case-the physiologically unrealistic value of 100%-the SDLFM algorithm still achieves correlations equal to those of XLFM.Thus, the worst case simply reverts to the earlier best case.In other words, SDLFM "fails gracefully" as neural The intensity profiles measured from the volumes showed that every peak is clearly resolved with SDLFM [Fig.2(n)].In addition to the effective resolution improvement, the SDLFM reconstruction was free from the haze [Fig.2(j)] above and below the dense volume that appeared in the XLFM reconstruction [Fig.2(i)], which originated from the fact that the light-field imaging system captured information from only limited angles and therefore failed to reject it (i.e., XLFM reconstruction is ill-posed for dense volumes).

B. Imaging Whole Brains of Zebrafish Larvae
To demonstrate the ability of SDLFM to image neuronal activity at high speed, we performed calcium imaging in live zebrafish larvae brains expressing pan-neuronal GCaMP6f [1] and nuclear localized GCaMP6s at 4-6 days post fertilization.A larval zebrafish brain was ideal for testing the performance of SDLFM, as it has densely packed neurons, and the neurons occasionally show synchrony across large populations [4,15], both of which pose a challenge to SDLFM, while its optical transparency potentially allows one to approach the theoretical limit of spatial resolution.
We imaged the spontaneous neural activity of fish at volume rates of 2-50 Hz with a volumetric field of view of ∼700 µm × ∼ 400 µm× ∼ 300 µm.Reconstructed volumes acquired using the conventional XLFM reconstruction method allowed neuronal activity imaging at near-cellular resolution for zebrafish with nuclear localized GCaMP6s, as demonstrated in [17] [Fig.3(a), Visualization 1], although the volumes from imaging fish brains with GCaMP6f in the cytosol were not as clear due to signals from axonal and dendritic processes (Fig. S2a  ).In addition, x and z slices and their zoom-ins further confirmed our system's capability even in densely packed regions [Figs.3(c) and 3(d)] over a large volume.Furthermore, to assess and compare the spatial resolutions of XLFM and SDLFM within their fields of view, we performed 2D Fourier-domain analysis with the z and x slices of densely labeled zebrafish brain images using the method presented in Ref. [23] (Fig. S5 of Supplement 1).The lateral and axial resolutions of XLFM and SDLFM were estimated for different axial and lateral positions [Figs.3(e) and 3(f )], which demonstrated that (1) XLFM and SDLFM maintained generally uniform resolutions within a field of view no smaller than ∼ ∅500 µm 2 × 250 µm, and, (2) compared with XLFM, SDLFM improved the lateral and axial resolutions by 2.2 µm and 4 µm, respectively.We also examined the cell-cell distances in the SDLFM image shown in Fig. 3(b) and found separable neuron pairs that were 3-4 µm apart (Fig. S4 of Supplement 1) within the fish brain.The average cell-cell distance was 9.26 µm (Fig. S4), which was a reasonable value compared with the average neuron diameter (∼7 µm) of larval zebrafish [4] given that SDLFM only reconstructed the active cells in a brain.These results mean that SDLFM can approach the theoretical resolutions-3.25 µm laterally and 5.05 µm axially-determined by the numerical apertures (NA) of the objective lens and the microlenses in densely labeled samples.We note that in our experimental results, the resolution degradation due to scattering in zebrafish was minimal, in part because our spatial resolution was already lower than the resolution limit set by scattering for transparent animals such as zebrafish, which has been confirmed by earlier studies that performed wholebrain imaging of a larval zebrafish with higher resolution than us [4,24,25].
In line with the simulation results [Figs.To further evaluate the capabilities of SDLFM to resolve neural signals from a densely labeled brain, we implemented a hybrid optical system [Fig.4(a)] for rapidly alternating imaging of larval zebrafish with light-sheet microscopy (LSM) and SDLFM.In this experiment, a fish expressing pan-neuronal nuclear localized GCaMP6s was imaged alternatively with the LSM mode and the SDLFM mode by consecutively switching between light-sheet and wide-field excitation [Fig.4

C. Imaging Whole Brains of Adult Drosophila
We next imaged brains of adult Drosophila expressing GCaMP6s and nuclear localized GCaMP6m pan-neuronally.We imaged spontaneous neuronal activity in brains of head-fixed flies at volume rates of 2-10 Hz with a volumetric field of view of ∼500 µm× ∼ 350 µm× ∼ 400 µm.With XLFM reconstruction, we were able to identify tens of neurons in fly brains expressing nuclear localized GCaMP6m, although the neurons were not clearly resolved [Fig.5(a)].With SDLFM reconstruction, nearly single-cell resolution was achieved for fly brains expressing nuclear localized GCaMP6m [Fig.5(b), Visualization 9] with the measured mean lateral and axial full widths at half-maximum (FWHM) of 2.9 µm (±0.4 µm, standard deviation) and 6.3 µm (±0.8 µm) [Fig.5(c)] for 10 nuclei.While a brain of an adult Drosophila scatters light significantly more than that of a larval zebrafish, the use of a nuclear localized calcium indicator largely reduces the issue of scattering, as the neuronal cell bodies of a fly brain are located at the surface of the brain.For fly brains with GCaMP6s in the cytosol, we observed well-localized components, which correspond to activities from cell bodies, as well as larger components that correspond to activities from neuronal processes and clusters of neurons as previously demonstrated with multiphoton microscopy (Fig. S10 of Supplement 1, Visualization 10, By directly segmenting the temporal maximum intensity projection volume (Fig. S11 of Supplement 1), neuronal activities from ∼300 neurons were extracted from imaging nuclear localized GCaMP6m flies [Figs.5(d) and 5(e)].About 200 spatial footprints and the corresponding neuronal activities were detected from imaging GCaMP6s flies using constrained non-negative matrix factorization (NMF) (Fig. S12 of Supplement 1) [27].
On a side note, we experimentally verified the performance of our system when the sample is scattering and sparsely labeled with XLFM, which provides an estimate of achievable resolution with SDLFM, by imaging a 500-µm-thick mouse brain slice of cortex expressing yellow fluorescent protein (YFP, Thy1-YFP) (Fig. S13 of Supplement 1).We were able to confirm that our system can resolve putative dendritic processes that are separated by 8 µm at 100 µm depth, measured from the surface of the brain slice as inferred by the location of the first neuronal cell body.In addition, among the ∼200 neurons, we found clearly separated cell pairs that were 15.8 µm apart at 230 µm depth and 12.6 µm apart at 52 µm depth.The real resolutions at these depths might be higher than these estimations, as the sparse expression, and the large cell size, in these specimens might make these neurons too distant to assess the ultimate limits of resolution (Fig. S13 of Supplement 1).Furthermore, neuronal cell bodies as deep as 300 µm were resolved, which is similar to the previously reported performance of LFM [16].

SUMMARY
In summary, we implemented SDLFM that achieves higher spatial resolution than conventional LFM methods without sacrificing temporal resolution and applied it to neuronal activity imaging of whole brains of larval zebrafish and adult Drosophila.By decomposing the raw light-field recording into a low rank component and a sparse component, the solution to the inverse problem can be recovered more accurately.It is worth noting that while SDLFM generally requires the sample to be immobilized or head-fixed, we found that the eye movement of larval zebrafish was faithfully captured in the reconstructed volumes without artifacts, which indicates that being able to decompose into two components does not require the immobilization in a strict sense (Visualization 13).In addition, not only other imaging methods such as compressive light-field microscopy [18] or seeded iterative demixing (SID) [16], but also the prevalent post-processing methods for calcium imaging data based on NMF and ICA, require the same level of immobilization to be applied, which have been assisted by the development of head-fixed protocols for neuroscience studies [28].Furthermore, SID has been successfully demonstrated on freely moving animals [29], which indicates that a brain's position relative to the optical system can remain stationary even for behaving animals.
We believe that SDLFM may be particularly well suited for imaging neuronal activity with fluorescent voltage indicators [9] in 3D, as not only does LFM have superior temporal resolution, but also the high temporal sparsity of voltage signals can be translated to extremely high spatial sparsity and hence excellent spatial resolution.Furthermore, due to the nature of single-exposure imaging, SDLFM has a much longer effective pixel dwell time (i.e., time duration that each pixel accumulates the incident photons) than that of scanning microscopy (e.g., LSM, confocal microscopy) at a given frame rate, which means that it will have a lower chance of missing spikes.
In addition, while our demonstration was focused on imaging neuronal activity with a specific optical system (i.e., XLFM), we believe our strategy can be applied to a wide range of imaging applications that have dominant and static backgrounds (e.g., autofluorescence) and spatiotemporally sparse dynamics of interest.For example, we found that SDLFM can clearly capture the flow of the blood cells even without knowing its existence in the raw image data.In terms of the hardware, almost any computational imaging method (e.g., wavefront encoding for depth-of-field enhancement) can benefit from our approach, as sparsity is an excellent prior that can be easily exploited for the reconstruction.

A. XLFM Implementation
We built three XLFMs introduced in Ref. [17] as the hardware of SDLFM [Figs.1(a), S14].Here, we denote them as XLFM1, XLFM2, and XLFM3.For XLFM1, we used a 16× 0.8 NA water dipping objective lens (CFI75 LWD 16 × W, Nikon) for imaging, a blue laser (λ = 473 nm, 100 mW, OBIS, Coherent) for excitation, and a standard green fluorescent protein (GFP) filter set (FF495-Di03-25 × 36, FF03-525/50-25, Semrock).A customdesigned microlens array was assembled and then mounted on the scientific complementary metal-oxide-semiconductor (sCMOS) camera (Zyla 5.5 sCMOS, Andor) using a continuous rotation lens mount (CLR1, Thorlabs), with the camera sensor at the back focal plane of the microlenses.The sCMOS camera was mounted on a 3D translational platform (PT3, Thorlabs) for fine 3D positioning to ensure that the microlens array was accurately conjugated to the back pupil plane of the objective lens through a 4 f relay system ( f 1 = 180 mm, AC508-180-A-ML, Thorlabs; f 2 = 125 mm, PAC074, Newport) (Table S1 of Supplement 1).XLFM2 was implemented by switching the microlens array of XLFM1: microlenses with different focal lengths were used for XLFM2 in order to extend the axial field of view while maintaining the same magnification for each sub-image.This modification allowed us to reduce the computational cost for the volume reconstruction by 50%.For XLFM3, we used a blue LED (M470L3, Thorlabs) as the light source, a different sCMOS camera (ORCA-Flash4.0V3, Hamamatsu), and a microlens array that supports a larger field of view, while the rest of the parts remained the same as in XLFM1 and XLFM2.

B. Hybrid LSM-SDLFM Implementation
For rapidly alternating LSM and SDLFM of larval zebrafish brain, we built a hybrid LSM-SDLFM by adding LSM capability to XLFM2 [Fig.4(a), Table S1 of Supplement 1].A stationary light sheet was created at the focal plane of the detection objective lens by focusing an expanded laser beam (OBIS 473 nm LX 75 mW, Coherent) using a cylindrical lens ( f = 75 mm, ACY254-075-A, Thorlabs).A blue LED (470 nm, M470L4, Thorlabs) was used for whole-brain excitation.The illumination was altered between the laser light-sheet mode and the LED wide-field mode in each frame for LSM imaging and SDLFM imaging, respectively.In the detection path, a 50:50 beam splitter (BSW10R, Thorlabs) was inserted between the tube lens ( f = 180 mm, AC508-180-A-ML, Thorlabs) and the image plane to evenly split the emission light.The transmitted light was used for SDLFM imaging, while the reflected light formed LSM images on a sCMOS camera (ORCA-Flash4.0V3, Hamamatsu).Digital triggers were generated using a peripheral component interconnect express (PCIe) input/output (I/O) card (PCIe-6323, National Instruments) to synchronize the cameras and the light sources.S 1 is the L-1 norm of S [30,31].We used the alternating direction method of multipliers (ADMM) for the optimization as in Ref. [30] with a modification to ensure non-negative components by enforcing non-negativity of L and S at the end of each iteration.Solving this optimization problem gives a low rank matrix L, which corresponds to the stationary part of the images, and a sparse matrix S, which corresponds to the neuronal activity.After the decomposition, volume reconstruction was applied to S rather than Y using Richardson-Lucy iterations with regularization to introduce weak preference to sparse solutions [32,33]: where x i , y , A{•}, A T {•}, and λ S are the solutions at the ith iteration, the decomposed light-field image, the forward projection function, the backward projection function, and the regularization parameter, respectively.sign{x i } = x i > ε th was used instead of sign{x i } = x i > 0 for relaxation of the sparsity (i.e., seeking a solution where the majority of the entities are close to zero rather than being exactly zero).Both decomposition and volume reconstruction were performed partially on a workstation with a 16-core Intel Xeon processor, Nvidia Tesla K40c GPU and 128 GB of RAM and partially on a medium size cluster with multiple GPUs.The sparse decomposition typically took about 12 h on a single machine for a video with 1000 frames and 100 iterations.The volume reconstruction of a single frame typically took about 3 mi on a single GPU with 30 iterations.The SDLFM software for sparse decomposition and volume reconstruction is written in MATLAB and requires a workstation with MATLAB2019a or a later version, a GPU with at least 8 GB of memory, and at least 64 GB of RAM (Code 1, Ref. [34]).

D. Zebrafish Preparation for Imaging
All procedures involving animals at the Massachusetts Institute of Technology (MIT) were conducted in accordance with the US National Institutes of Health Guide for the Care and Use of Laboratory Animals and approved by the MIT Committee on Animal Care.For zebrafish experiments, pan-neuronal GCaMP6f expressing Tg(HuC:GCaMP6f) j f 1 and nuclear localized GCaMP6s expressing Tg(HuC:H2B:GCaMP6s) j f 5 larvae were imaged at 4-6 days post fertilization.The transgenic larvae were kept at 28 • C and paralyzed in standard fish water containing 0.25 mg/ml of pancuronium bromide (Sigma-Aldrich) for 2 min prior to imaging.The larvae were embedded in agar with 0.5% agarose (SeaKem GTG) and 1% low melting point agarose (Sigma-Aldrich) for immobilization in Petri dishes (Fig. S15 of Supplement 1).Upon solidification of agarose, fish water was added to the dishes.

E. Drosophila Preparation for Imaging
For fly experiments, adult flies with pan-neuronal GCaMP6s and nuclear confined GCaMP6m expression were imaged.Flies with pan-neuronal GCaMP6s expression were obtained by crossing the UAS-GCaMP6s line and nSyb-GAL4 line, both from the Bloomington Drosophila Stock Center.Flies with pan-neuronal nuclear localized GCaMP6m (UAS-nls-GCaMP6m/CyO; R57C10-Gal4(VK00020), R57C10-Gal4(VK00040)/TM6B) were generously sent from Janelia Research Campus.A fly was transferred to a small vial and anesthetized by leaving the vial on ice for a few minutes.The fly was mounted on a custom-designed fly mount (Fig. S16 of Supplement 1), similar to previously reported methods [35,36], made by creating a hole on a piece of aluminum shim using laser ablation.Silicon adhesive (Kwik-sil, World Precision Instruments) was applied to the gaps between the fly and the mount for immobilization.After the adhesive was cured, saline solution [36,37] (103 mM NaCl, 3 mM KCl, 5 mM TES, 8 mM trehalose, 10 mM glucose, 26 mM NaHCO 3 , 1 mM NaH 2 PO 4 , 4 mM MgCl 2 , and 1.5 mM CaCl 2 ) was added to the fly head.The cuticle on the head was first cut using a tungsten dissection needle (RS-6064, Roboz Surgical Store), and then the cuticle, fat bodies, air sacks, and muscle 16 were removed using sharpened forceps (Dumont #5, Fine Science Tools).New saline solution was applied to remove remaining tissue residues in the solution.

F. PSF Measurement
We measured the empirical point spread function (PSF) of the microscopes by imaging a 1-µm-diameter green fluorescent bead located at the center of the field of view with an axial step size of 2.5 µm for XLFM1 and XLFM2 and 4 µm for XLFM3 (Fig. S17 of Supplement 1).We typically took 200 images for each PSF stack, which covered 500 µm, 500 µm, and 800 µm axial fields of view for the three microscopes, respectively.After taking the stack, 10 background images were taken after shifting the bead sample laterally to move it away from the field of view.The averaged background image was subtracted from the measured raw PSF to remove the background component that comes from the ambient light, the camera offset and the reflection of excitation light.Due to the magnification ratio disparity of sub-images introduced by the axial displacement of the microlenses in XLFM1, we manually reorganized the PSF into two complementary parts for reconstruction, as described in Ref. [17].

G. Data Analysis
For neuronal activity extraction from zebrafish data, the temporal MIP of the four-dimensional (4D) time series volumes was first estimated, and then direct cell segmentation based on 3D matched filtering [38] was applied to the volume to obtain spatial filters that correspond to the neurons.The spatial filters were applied to the 4D time series volumes to extract the time courses, and then the extracted time courses were de-noised using deconvolution [27].The same spatial filters were applied to the volume reconstructed low rank component to estimate the baseline fluorescence intensity of each neuron.The SNR was measured as the maximum fluorescence change observed divided by the standard deviation of the baseline, where the baseline was determined as the 80th percentile of the fluorescence, assuming that each neuron was active for less than 20%.
For comparison of LSM and SDLFM images, we first reconstructed a volume from an XLFM image that was taken with the light-sheet excitation mode.A spatial filter was created by blurring and thresholding the XLFM volume and was used for extracting SDLFM volume sections that corresponded to the LSM images.A rigid 2D image registration was performed from the axial MIP of the XLFM volume to the corresponding LSM image.The registration parameters were stored and used for image registration of the spatially filtered SDLFM images to the LSM images.The temporal MIP was taken from the SDLFM images and the LSM images, and direct intensity-based segmentation was applied to obtain spatial footprints for the neurons.The overlapped spatial footprints were applied to extract the time courses from both LSM and SDLFM, and then the extracted time courses were compared.For better comparison, we pre-processed the LSM images using the same sparse decomposition algorithms in SDLFM and normalized the SDLFM signals to the ranges of the LSM traces.
For Drosophila with nuclear localization signal (NLS)-GCaMP6m, the same procedure as analyzing zebrafish data was applied to the data.For the pan-neuronal GCaMP6s strain, the whole 4D time series of volumes was fed into constrained NMF [27] to extract both spatial filters and temporal signal traces.The algorithm first detected the neuron candidates based on their shape and then extracted the signal trace for each neuron.The extracted spatial filters were applied to the volume reconstructed low rank component to estimate the baseline fluorescence intensity of each neuron.The correlation coefficients and the SNRs were calculated following the same procedure as the zebrafish data analysis.

H. SDLFM Simulation
For simulations of SDLFM versus XLFM, we first generated a synthetic volume with a size, shape, and neuron density matched to that of a larval zebrafish brain (Table S2 of Supplement 1).Specifically, each sphere was randomly generated within the volume predefined by a fish-brain-shaped mask.The sphere was accepted as a neuron if it does not spatially overlap with any existing neurons, and this process was repeated until the target number of neurons was reached.In order to test the robustness of SDLFM to increases in synchronicity of neuronal activity, the simulation was performed with a different proportion of neurons that were simultaneously active (i.e., calcium concentration above the baseline) at each frame.In the simulations, the average firing rates of 20 Hz, 1.75 Hz, 0.8 Hz, 0.4 Hz, and 0.2 Hz corresponded to the proportion of neurons that are simultaneously active within the exposure time of each frame of 100%, 50%, 25%, 12.5%, and 6.3%, respectively.By convolving the volumes with the empirical PSF, a time series of light-field images was generated, and low levels of readout noise and shot noise were added to the image to avoid overfitting and to ensure numerical stability (Table S2 of Supplement 1).Then, volume reconstruction was performed on the images with both conventional XLFM reconstruction and SDLFM reconstruction.The signal for each neuron was extracted by applying the spatial filter to the volume for each neuron.The extracted signals were compared to the ground truth and the correlation coefficients were evaluated.

[µm] spatial distribution of neurons
A sphere was randomly generated within the volume predefined by a fish brain shaped mask.The sphere was accepted as a neuron if it does not spatially overlap with any existing neuron.This process was repeated until the target number of neurons was reached.Media S13.SDLFM whole brain functional imaging of a zebrafish larva expressing nuclear localized GCaMP6s pan-neuronally.Maximum intensity projection with a fixed view.The data was acquired at a volume rate of 10 Hz.Temporal MIP is shown at the beginning.

Fig. 1 .
Fig. 1.Sparse decomposition light-field microscopy (SDLFM) of neuronal activity.(a) Schematic of the extended field of view light-field microscope (XLFM) used as the hardware for SDLFM.Unlike a conventional light-field microscope, the microlens array is placed on the conjugated pupil plane instead of the image plane of the system.(b) Schematic of the sparse decomposition algorithm.A light-field video recording is decomposed into additive non-negative components: a low rank non-negative component that corresponds to the static part of the recording and a sparse non-negative component that corresponds to the neuronal activity.(c) Conventional volume reconstruction from a light-field image.Each frame is independently deconvolved using the point spread function (PSF) of the microscope.(d) Volume reconstruction with SDLFM.The decomposed sparse component from (b) is deconvolved with regularization using the PSF of the microscope.

Fig. 2 .
Fig. 2. SDLFM simulation.(a) Simulation results showing the comparison of the ground truth and the extracted activity waveforms.The activities of eight representative neurons are shown.There were 80,000 neurons simulated with an average firing rate of 0.3 Hz and the rise time and decay time constants of GCaMP6f of 45 ms and 142 ms, respectively.On average, 10% of the neurons were simultaneously active in each frame where the exposure time was 50 ms.Black, ground truth; blue, XLFM; red, SDLFM.(b) Normalized (with respect to the total number of neurons) histogram of the correlation coefficients obtained by comparing the ground truth and the extracted activity from simulated SDLFM when 6.3% of the neurons are simultaneously active in each frame.(c) As in (b), but when 12.5% of the neurons are simultaneously active in each frame.(d) As in (b), but when 25% of the neurons are simultaneously active in each frame.(e) As in (b), but for XLFM, and when 6.3% of the neurons are simultaneously active.(f ) Median values of correlation coefficients obtained by comparing the ground truth and the extracted activity from simulated XLFM.d = 1 indicates that the neurons are as dense as that of a larval zebrafish brain.(g) As in (f), but for SDLFM.(h) Temporal maximum intensity projection (MIP) of ground truth input volumes.Top, axial projection.Bottom, lateral projection.Scale bar, 50 µm.(i) Temporal MIP of reconstructed volumes from simulating XLFM.Top, axial projection.Bottom, lateral projection.Scale bar, 50 µm.(j) Temporal MIP of reconstructed volumes from simulating SDLFM.In each frame, 6.3% of the neurons were simultaneously active.Top, axial projection.Bottom, lateral projection.Scale bar, 50 µm.(k) z (top) and x (bottom) slices from the temporal MIP of ground truth input volumes used for simulation.Z indicates the imaging depth from the top surface of the brain.X represents the position along the rostral-caudal axis of the fish, measured as the distance to the rostral surface of the brain.The intensity profile in (n) is measured over the yellow dashed line.Scale bar, 50 µm.The inset in the upper right zooms in on the small box at the center.Scale bar, 10 µm.(l) As in (k), but for XLFM.(m) As in (k), but for SDLFM.(n) Intensity profiles measured over the yellow dashed lines in (k), (l), and (m) along lateral (top) and axial (bottom) directions.A.U., arbitrary units.Black, ground truth; blue, XLFM; red, SDLFM.

of Supplement 1 ,
Visualization 2).With SDLFM reconstruction, neuronal cell bodies were clearly visible in both nuclear localized GCaMP [Figs.3(b)-3(d), S3, S4, Visualization 3, Visualization 4] and cytosolic GCaMP strains (Fig. S2(b) of Supplement 1, Visualization 5, Visualization 6, Visualization 7 2(i) and 2(j)], the haze above and below the brain in the XLFM reconstruction [Figs.3(a), S2(a)] was removed with the SDLFM reconstruction [Figs.3(b), S2(b)].In addition, blood flow through the vessels were visible (Visualization 5, Visualization 6, Visualization 7), which might otherwise be mistaken as neural activity.We speculate that the bright spots circulating through the vessels are inter-blood cell spaces.As blood cells strongly absorb visible light, including GCaMP fluorescence signals from nearby, they effectively cast shadows at their locations, which can be thought of as sparse negative components.However, due to the non-negativity enforced by the decomposition algorithm, the images are decomposed in a way such that absences of blood cells (i.e., inter-blood cell spaces) appear positive.Neuronal activities from ∼8000 neurons were extracted by directly segmenting the temporal MIP volume [Figs.3(g) and 3(h), S7, S8], and the results from SDLFM showed more stable baselines than those from XFLM, similar to the simulation results [Fig.2(a)].The average Pearson correlation coefficients among ∼8000 neurons was 0.059, which falls within the simulated range.The average SNRs, defined as the maximum fluorescence change observed divided by the standard deviation of the baseline, were 74.3 and 9.1 for SDLFM and XLFM reconstruction, respectively.
(b)].Each mode lasted for 100 ms, resulting in an effective frame rate of 5 Hz.LSM images and the corresponding planes from SDLFM images were registered and compared [Figs.4(c), S9, Visualization 8].Despite the intensity non-uniformity presumably caused by distinct illumination strategies, single cells are clearly resolved in the LSM image, which are also visible in the SDLFM image [Figs.4(c), S9].Using direct intensity-based segmentation, we found 280 out of 538 neuron footprints in the SDLFM images overlapped with their counterparts in the LSM images [Fig.4(c)].Well-matched neural activity traces were extracted from these overlaid segmentations, with an average correlation coefficient of 0.864 [Fig.4(d)].

Fig. 3 .
Fig. 3. Imaging whole brains of zebrafish larvae using SDLFM.(a) Reconstructed XLFM image of larval zebrafish brain expressing nuclear localized GCaMP6s.Top, axial MIP.Bottom, lateral MIP.Scale bar, 100 µm.(b) Temporal MIP of reconstructed SDLFM images of larval zebrafish brain expressing nuclear localized GCaMP6s.The temporal MIP shows the accumulated neuronal activity from 200 s of imaging.Top, axial MIP.Bottom, lateral MIP.Scale bar, 100 µm.From the same raw data as in (a).(c) Single z slice (horizontal section, z = 140 µm, counted upwards from top to bottom, z = 0 µm indicates the top surface of the brain) from the temporal MIP of reconstructed SDLFM images of larval zebrafish brain expressing nuclear localized GCaMP6s (left).From the same raw data as in (b).Scale bar, 100 µm.Magnified view of the yellow boxed region (right).Scale bar, 10 µm.(d) Single y slice (sagittal section, y = 440 µm, counted upward from the right side to the left side, y = 0 µm indicates the right surface of the brain) from the temporal MIP of reconstructed SDLFM images of larval zebrafish brain expressing nuclear localized GCaMP6s (left).From the same raw data as in (b).Scale bar, 100 µm.Magnified view of the yellow boxed region (right).Scale bar, 10 µm.(e) Measured lateral resolution of XLFM and SDLFM along the z axis.(f ) Measured axial resolution of XLFM and SDLFM along the x axis (the rostral-caudal axis of the brain).(g) Extracted neuronal activities from imaging larval zebrafish brain expressing GCaMP6f with SDLFM shown as a heat map.(h) Signal traces of the extracted neuronal activities from randomly selected neurons with XLFM (red) and SDLFM (black).

Fig. 4 .
Fig. 4. Hybrid LSM-SDLFM imaging.(a) Schematic of LSM-SDLFM used for rapidly alternating LSM and SDLFM of larval zebrafish brain.(b) Illumination time line.The illumination was altered between the laser light-sheet mode and the LED wide-field mode for single-slice LSM and wholebrain SDLFM snapshots.Each mode lasted for 100 ms.(c) Temporal MIPs of LSM images (left) and the corresponding SDLFM volume sections (center), and segmentation results (right, LSM, magenta; SDLFM, green; overlap, white).Scale bar, 100 µm.The white and yellow boxed regions are zoomed-in and shown in the insets with the corresponding colors, respectively.Scale bar, 20 µm.The light-sheet is 70 µm deep underneath the brain surface.(d) Extracted neuronal activities from the 280 overlapped footprints in (c) are shown as a heat map (left) as well as in signal traces of representative neurons (right, LSM, red; SDLFM, black).The average correlation coefficient between LSM and SDLFM across all 280 cells is 0.864.

Fig. 5 .
Fig. 5. Imaging whole brains of adult Drosophila using SDLFM.(a) Reconstructed XLFM image of adult Drosophila brain expressing NLS-GCaMP6m.Top left, axial projection.Top right, lateral projection.Bottom, rostral-caudal projection.Scale bar, 100 µm.(b) Temporal MIP of reconstructed SDLFM images of adult Drosophila brain expressing NLS-GCaMP6m.The temporal MIP shows the accumulated neuronal activity from 100 s of imaging.The low rank component and the sparse component are shown as blue and green, respectively.The yellow boxed region was used to measure the intensity profile in (c) From the same raw data as in (a).Top left, axial projection.Top right, lateral projection.Bottom, rostral-caudal projection.Scale bar, 100 µm.(c) Magnified spatial MIP views of the yellow boxed region in (b) from a single frame (rather than the temporal MIP) and the lateral and axial intensity profiles across the yellow dashed lines.Within the MIP images on the left side, top left, axial projection, top right, lateral projection, bottom, rostral-caudal projection.Scale bar, 10 µm.(d) Extracted neuronal activities from imaging adult Drosophila brain expressing NLS-GCaMP6m with SDLFM shown as a heat map.(e) Signal traces of the extracted neuronal activities with SDLFM from randomly selected neurons.

C
. Sparse Decomposition and Volume Reconstruction SDLFM first decomposes the raw light-field recording Y into two additive matrices L(stationary component) and S(sparse component) [Fig.1(b)] by solving the optimization problem min L,S L * + λ S 1 subject to Y = L + S, L ≥ 0, S ≥ 0, where each column of Y is the light-field image at each time point (i.e., Y = [y 1 , y 2 , • • • , y N ]), L * is the nuclear norm of L, and

Funding.
Howard Hughes Medical Institute; HHMI-Simons Faculty Scholars Program; National Science Foundation (1848029); National Institutes of Health (. Army Research Laboratory and the U. S. Army Research Office (W911NF1510548); Jeremy and Joyce Wertheimer; NIH Director's Pioneer Award (1DP1NS087724); MIT Media Lab; Open Philanthropy Project; Lisa Yang; John Doerr; Samsung Scholarship; KAIST (G04180057, N10190006); National Research Foundation of Korea (2019R1F1A1058498, 2020R1C1C1009869); Teaching Affairs Office of USTC; Singapore National Science Scholarship; National Key Research and Development Program of China (2017YFA0700500); Chinese Academy of Sciences ("Strategic Priority Research Program" XDB32030200, "International Partnership Program" 153D31KYSB20170059); Shanghai Municipal Science and Technology Major Project (2018SHZDZX05, 18JC1410100); National Natural Science Foundation (NSFC 31871086); Alana Fellowship.
Fig. zebrafis the rec method assigned rostral-(b) His nearest calculat that we below t caudal p S4. (a) Spatial sh brain shown i constructed volu ds such as nonne d to each spatia caudal projection stogram of the d t neighbor cells ted using 3-D Gau ere less than 4 the brain surface projection.Bottom l filters obtained in Fig. 3b with S ume without r egative matrix fac al filter.Top lef n.Bottom, lateral distances betwe in a. Sub-pixel ussian fitting.(c) μm apart from e. Top left, axial m, lateral project d from imaging SDLFM by directl relying on sign ctorization.Rand ft, axial projectio l projection.Scale een individual ce l-precision cell MIPs of example each other at va projection.Top tion.Scale bar, 10 of the larval ly segmenting nal extraction dom colors are on.Top right, e bar, 100 μm.ells and their centers were e neuron pairs arious depths right, rostral-0 μm.

6 .Fig. 3 ,
25%, 12.5%, 25%, 50%, 100% average effective firing rate 0.2 Hz, 0.4 Hz, 0.8 Hz, 1.75 Hz, 20 Hz frame rate 20 [Hz] read noise (s.t.d.level relative to the brightest pixel value) 0.1% shot noise (number of the expected shot noise, √ , at the brightest pixel) 100 e -optical configuration Objective lens: 16× 0.8NA Microlens array focal length = 36.1mmRelay lens 1 focal length = 180mm Relay lens 2 focal length = 125mm Pixel size = 6.5 µm Number of pixels = 2160 × 2560 (embedded in the point spread function by using an empirical PSF of XLFM1) Fig. S1, Fig. S4, Fig. S5, Fig. S6, Fig. S7, Fig. S16, Media S1, Media S2, Media S4, Media S5 XLFM2 Fig. S2, Fig. S9, Media S3, Media S6, Media S7 XLFM3 Fig. 5, Fig. S10, Fig. S11, Fig. S12, Media S9, Media S10, Media S11, Media S12, Media S13 hybrid LSM-SDLFM Fig. 4, Fig. S8, Media S8Media S1.XLFM whole brain functional imaging of a zebrafish larva expressing nuclear localized GCaMP6s pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 10 Hz.Media S2.XLFM whole brain functional imaging of a zebrafish larva expressing GCaMP6f pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 10 Hz.Media S3.SDLFM whole brain functional imaging of a zebrafish larva expressing nuclear localized GCaMP6s pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 2Hz.Temporal MIP is shown at the beginning.Media S4.SDLFM whole brain functional imaging of a zebrafish larva expressing nuclear localized GCaMP6s pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 5 Hz.Temporal MIP is shown at the beginning.Media S5.SDLFM whole brain functional imaging of a zebrafish larva expressing GCaMP6f pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 10 Hz.Temporal MIP is shown at the beginning.Media S6.SDLFM whole brain functional imaging of a zebrafish larva expressing GCaMP6f pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 50 Hz.Temporal MIP is shown at the beginning.Media S7.SDLFM whole brain functional imaging of a zebrafish larva expressing GCaMP6f pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 50 Hz.Temporal MIP is shown at the beginning.Media S8.Rapidly alternating whole brain functional imaging of a zebrafish larva expressing nuclear localized GCaMP6s pan-neuronally using LSM (left) with a laser light sheet illumination and SDLFM (right) with a LED wide-field illumination to demonstrate the achievable resolution.The data was acquired at 5 Hz.Media S9.SDLFM whole brain in vivo functional imaging of an adultDrosophila expressing NLS-GCaMP6m pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 5 Hz.SDLFM reconstruction (green) is overlaid on the volume-reconstructed low rank component (blue).Temporal MIP is shown at the beginning.Media S10.SDLFM whole brain functional imaging of an adult Drosophila expressing GCaMP6s pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 2 Hz.SDLFM reconstruction (green) is overlaid on the volumereconstructed low rank component (blue).Media S11.SDLFM whole brain functional imaging of an adult Drosophila expressing GCaMP6s pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 10 Hz.SDLFM reconstruction (green) is overlaid on the volumereconstructed low rank component (blue).Media S12.SDLFM whole brain functional imaging of an adult Drosophila expressing GCaMP6s pan-neuronally.Maximum intensity projection with rotating view.The data was acquired at a volume rate of 5 Hz.SDLFM reconstruction (green) is overlaid on the volumereconstructed low rank component (blue).