Extended field-of-view and increased-signal 3 D holographic illumination with time-division multiplexing

Phase spatial light modulators (SLMs) are widely used for generating multifocal three-dimensional (3D) illumination patterns, but these are limited to a field of view constrained by the pixel count or size of the SLM. Further, with two-photon SLM-based excitation, increasing the number of focal spots penalizes the total signal linearly—requiring more laser power than is available or can be tolerated by the sample. Here we analyze and demonstrate a method of using galvanometer mirrors to timesequentially reposition multiple 3D holograms, both extending the field of view and increasing the total time-averaged two-photon signal. We apply our approach to 3D two-photon in vivo neuronal calcium imaging. ©2015 Optical Society of America OCIS codes: (090.0090) Holography; (230.6120) Spatial light modulators; (180.6900) Threedimensional microscopy; (180.5810) Scanning microscopy. References and links 1. J. E. Curtis, B. A. Koss, and D. G. Grier, “Dynamic holographic optical tweezers,” Opt. Commun. 207(1), 169– 175 (2002). 2. A. Linnenberger, D. Peterka, S. Quirin, and R. Yuste, “The Pocketscope: a spatial light modulator based epifluorescence microscope for optogenetics,” Proc. SPIE 9164, 91640Y (2014). 3. V. Nikolenko, B. O. Watson, R. Araya, A. Woodruff, D. S. Peterka, and R. Yuste, “SLM microscopy: scanless two-photon imaging and photostimulation with spatial light modulators,” Front. Neural Circuits 2, 5 (2008). 4. S. Quirin, D. S. Peterka, and R. Yuste, “Instantaneous three-dimensional sensing using spatial light modulator illumination with extended depth of field imaging,” Opt. Express 21(13), 16007–16021 (2013). 5. S. Quirin, J. Jackson, D. S. Peterka, and R. Yuste, “Simultaneous imaging of neural activity in three dimensions,” Front. Neural Circuits 8, 29 (2014). 6. A. M. Packer, D. S. Peterka, J. J. Hirtz, R. Prakash, K. Deisseroth, and R. Yuste, “Two-photon optogenetics of dendritic spines and neural circuits,” Nat. Methods 9(12), 1202–1205 (2012). 7. S. Paluch-Siegler, T. Mayblum, H. Dana, I. Brosh, I. Gefen, and S. Shoham, “All-optical bidirectional neural interfacing using hybrid multiphoton holographic optogenetic stimulation,” Neurophotonics 2(3), 031208 (2015). 8. A. M. Packer, L. E. Russell, H. W. Dalgleish, and M. Häusser, “Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo,” Nat. Methods 12(2), 140–146 (2014). 9. L. Golan, I. Reutsky, N. Farah, and S. Shoham, “Design and characteristics of holographic neural photostimulation systems,” J. Neural Eng. 6(6), 066004 (2009). 10. G. Thalhammer, R. W. Bowman, G. D. Love, M. J. Padgett, and M. Ritsch-Marte, “Speeding up liquid crystal SLMs using overdrive with phase change reduction,” Opt. Express 21(2), 1779–1797 (2013). 11. Meadowlark Optics, “Overdrive Plus,” http://www.meadowlark.com/overdrive-plus-odp-p-125. 12. T. W. Chen, T. J. Wardill, Y. Sun, S. R. Pulver, S. L. Renninger, A. Baohan, E. R. Schreiter, R. A. Kerr, M. B. Orger, V. Jayaraman, L. L. Looger, K. Svoboda, and D. S. Kim, “Ultrasensitive fluorescent proteins for imaging neuronal activity,” Nature 499(7458), 295–300 (2013). 13. J. W. Goodman, Introduction to Fourier optics (Roberts and Company Publishers, 2005). 14. G. Sinclair, J. Leach, P. Jordan, G. Gibson, E. Yao, Z. Laczik, M. Padgett, and J. Courtial, “Interactive application in holographic optical tweezers of a multi-plane Gerchberg-Saxton algorithm for three-dimensional light shaping,” Opt. Express 12(8), 1665–1670 (2004). #252162 Received 19 Oct 2015; revised 29 Nov 2015; accepted 1 Dec 2015; published 9 Dec 2015 © 2015 OSA 14 Dec 2015 | Vol. 23, No. 25 | DOI:10.1364/OE.23.032573 | OPTICS EXPRESS 32573 15. W. J. Alford, R. D. Vanderneut, and V. J. Zaleckas, “Laser scanning microscopy,” Proc. IEEE 70(6), 641–651 (1982). 16. R. D. Hanes, M. C. Jenkins, and S. U. Egelhaaf, “Combined holographic-mechanical optical tweezers: construction, optimization, and calibration,” Rev. Sci. Instrum. 80(8), 083703 (2009). 17. M. Kummer, K. Kirmse, O. W. Witte, J. Haueisen, and K. Holthoff, “Method to quantify accuracy of position feedback signals of a three-dimensional two-photon laser-scanning microscope,” Biomed. Opt. Express 6(10), 3678–3693 (2015). 18. C. Xu and W. W. Webb, “Measurement of two-photon excitation cross sections of molecular fluorophores with data from 690 to 1050 nm,” J. Opt. Soc. Am. B 13(3), 481–491 (1996). 19. M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, and M. Levoy, “Wave optics theory and 3-D deconvolution for the light field microscope,” Opt. Express 21(21), 25418–25439 (2013). 20. A. Holtmaat, T. Bonhoeffer, D. K. Chow, J. Chuckowree, V. De Paola, S. B. Hofer, M. Hübener, T. Keck, G. Knott, W. C. Lee, R. Mostany, T. D. Mrsic-Flogel, E. Nedivi, C. Portera-Cailliau, K. Svoboda, J. T. Trachtenberg, and L. Wilbrecht, “Long-term, high-resolution imaging in the mouse neocortex through a chronic cranial window,” Nat. Protoc. 4(8), 1128–1144 (2009). 21. E. Ronzitti, M. Guillon, V. de Sars, and V. Emiliani, “LCoS nematic SLM characterization and modeling for diffraction efficiency optimization, zero and ghost orders suppression,” Opt. Express 20(16), 17843–17855 (2012). 22. G. Katona, G. Szalay, P. Maák, A. Kaszás, M. Veress, D. Hillier, B. Chiovini, E. S. Vizi, B. Roska, and B. Rózsa, “Fast two-photon in vivo imaging with three-dimensional random-access scanning in large tissue volumes,” Nat. Methods 9(2), 201–208 (2012). 23. N. Ji, J. C. Magee, and E. Betzig, “High-speed, low-photodamage nonlinear imaging using passive pulse splitters,” Nat. Methods 5(2), 197–202 (2008). 24. G. Donnert, C. Eggeling, and S. W. Hell, “Major signal increase in fluorescence microscopy through dark-state relaxation,” Nat. Methods 4(1), 81–86 (2007).


Introduction
Many applications requiring three-dimensional (3D) patterned illumination use laser light shaped by phase-only spatial light modulators (SLMs), ranging from optical tweezing [1] to optogenetics [2].In particular, SLMs are increasingly used with infrared femtosecond pulsed lasers to implement patterned two-photon excitation for either multisite neuronal imaging [3][4][5] or multisite photoactivation [6][7][8], to either record or manipulate neural activity, using calcium indicators or light sensitive opsins, respectively.These two-photon approaches afford deeper tissue penetration, crucial for in vivo applications, but are often laser power limited since the signal at each simultaneous site decreases quadratically instead of linearly with the total number of sites.In addition, the accessible field of view over which the illumination can be steered is limited by the pixel count or size of the SLM [9], which has improved only modestly in comparison to recent developments enabling 8x faster SLM switching speeds [10,11].
Here, we simultaneously address both of these limitations by leveraging the higher switching speed of SLMs with a pair of conjugated galvanometer mirrors to time-division multiplex the smaller SLM field of view over a larger area.We use multiplexing to refer to the time-sequential illumination of different tiled spatial regions to cover a larger total spatial area.Although the sites are no longer illuminated truly simultaneously, for two-photon excitation, our multiplexing approach also increases the total time-averaged signal.We posit that for many applications where the requirement for strict simultaneity might be relaxed, the combination of our approach with faster SLMs [10,11] will enable large increases in both signal and field of view.In particular, we consider neuronal calcium imaging applications, where sites need only be sampled at approximately the Nyquist rate for a given calcium sensor [12], rather than simultaneously.
Sections 2 and 3 describe the theory behind tiling the SLM-accessible field of view with galvanometer mirrors and the expected two-photon signal gain, respectively.Section 4 details the experimental setup for dye characterization experiments and in vivo imaging.In Section 5 we experimentally quantify both the extension in field of view and increase in signal, and apply our approach to 3D two-photon in vivo calcium imaging using a technique from [5].Finally in Section 6, we summarize results and conclude with an outlook to the future.

Extending the SLM field of view with conjugated scanning mirrors
In a coherent illumination system, a desired object space intensity distribution, I(x,y) = |{H(u,v)}| 2 [13] can be attained by controlling the complex electric field at the pupil plane, where .
In holographic illumination systems with phase-only modulation, A(u,v) is the fixed laser amplitude, typically uniform or Gaussian.For a phase-only SLM with 0-2π modulation at wavelength λ at each of the N × N pixels, the φ(u,v) = φ SLM (u,v) for a given desired 2D or 3D intensity pattern can be computed [14].
However, the area over which light can be steered, termed field of view (FOV) by [9], is fundamentally constrained by the pixel count of the SLM to 2x max × 2y max where and NA is the numerical aperture of the objective lens.We assume the image of the SLM is optically magnified such that the width of the SLM matches the diameter of the objective pupil (to jointly maximize spatial resolution and light transmission).Often, the objective field of view is larger than the SLM-accessible field of view, as illustrated in Fig. 1(a).To address this, we use a pair of galvanometer scanning mirrors, conjugated to the SLM plane, to apply a lateral shift of (Δx, Δy) by superimposing an additional linear phase term, ( , ; , ) ( ), where c depends on the wavelength and lens parameters [15].The total phase modulation is the sum of the two [16], ( , ) ( , ) ( , ; , ).
Rather than use the mirrors to enable smoother, continuous translation of a single hologram defining multiple optical traps [16], we use the time-sequential repositioning of the galvanometer mirrors and multiple different holograms to achieve a larger effective FOV as shown in Fig. 1(b).
In a typical multiphoton microscope, the galvo positioning time is less than ~1 ms [17], so the area within the objective field of view tiled by our approach in a given exposure time is determined by the SLM transition time, illustrated in Fig. 2.

Increasing two-photon signal with sequential illumination
In two-photon excitation, signal rate depends quadratically on the average laser power P [18] so the time-averaged signal integrated over a total period of exposure T is 2 .S P T ∝ (5) For an illumination pattern that divides the average laser power equally to n excitation sites with equal focal volumes, the total time-averaged signal from the n volumes is which is a factor n times smaller than that in Eq. ( 5).However, if we relax the requirement for simultaneity of the illumination, and allow for M sets of n/M sites to be sequentially excited over the duration of the exposure time T, the total time-averaged signal is 2 ( ) , where t SLM is the time it takes the SLM to change the illumination pattern between each set of spots.This is greater than that in Eq. ( 6) by a factor of ( ; ) 1 .
For a given t SLM , Eq. ( 8) takes a maximum value of M/2 when M = T/(2t SLM ).Hence if the laser power is held fixed and the SLM transition time t SLM is known, we can determine the optimum number of subsets to scan in order to maximize the total two-photon excited signal for a given total experimental exposure time.

Experimental setup
We implemented our time-division multiplexing illumination approach on a microscope equipped with a SLM-based illumination system and galvanometer scanning mirrors, as illustrated in Fig. 3.We additionally incorporated a single-snapshot 3D imaging system (light field microscope) and a single-beam laser scanning two-photon imaging system to enable quantification of two-photon excitation efficiency and 3D in vivo calcium imaging.Fig. 3. Optical layout.The SLM, illuminated by a 920 nm femtosecond laser through a beam expander (BE), is imaged through lenses (L1, L2) and zero-order beam block (ZB) to the midpoint between two galvanometer scanning mirrors, and then to the objective pupil plane through a scan lens (SL), tube lens (TL) and short-pass dichroic mirror (D).Excited fluorescence filtered by an emission filter (F) is captured by either a photomultiplier tube (PMT) or light field microscope (LFM) with sCMOS camera, enabling single-snapshot 3D visualization of the excited fluorescence.
We used 3D holograms computed with the multi-plane Gerchberg-Saxton method [14].For calibration and characterization experiments, we used a green fluorescent dye (Sharpie highlighter) in a cuvette (Thorlabs CV10Q3500) capped with a 0.17 mm coverslip.Our amplifier compressor stage was adjusted to compensate for pulse dispersion from the optics.

Calibration
We determined our SLM transition time to be t SLM = 7.5 ms by steering a single focal spot between two photodiodes placed just after ZB in Fig. 3  With the geometric calibration procedure in [4], we determined the 2D affine transforms between each z-depth of the light field microscope and SLM field of view, for each of 9 galvanometer mirror positions.For each mirror position at two z-depths, the 2D transform was measured by illuminating multiple focal spots in fluorescent dye and recording the resulting 3D light field coordinates; the transforms for the other z-depths were linearly interpolated.

Modifications for 3D two-photon in vivo calcium imaging with time-division multiplexing
To enable 3D in vivo calcium imaging with a modified version of [5] the following changes were made.Two flip mirrors (Newport 8892-K) enabled switching between the 80 MHz and 250 kHz lasers, and between a photomultiplier tube (Hamamatsu H10770P A-40) and the sCMOS camera/microlens array, enabling single-beam 2D laser-scanning two-photon microscopy [15].We extended the geometric calibration procedure from Section 4.2 to include this third coordinate system by registering the corners of the 2D scanned field of view in dye with the light field microscope coordinates at each z-depth.
We used a 512 × 512 resolution, but slower (t SLM = 10 ms), SLM (Meadowlark HVHSPDM512-532) and to match the larger 19.2 mm SLM size to the objective pupil, we adjusted the beam expansion (BE) accordingly and used f 1 = 750 mm and f 2 = 150 mm.
We imaged 6 months after a chronic cranial window was implanted [20].Briefly, mice were sedated with isoflurane and a 5 mm diameter, #1 thickness cover slip (Warner) was implanted over barrel cortex after injection of 1000 μL of AAVdj-Camk2a-GCaMP6m virus [12].In addition, a plate for head-fixation during imaging was affixed to the skull using dental cement (Parkell).The Stanford University Institutional Animal Care and Use Committee approved all experimental protocols.

Extending field of view
We analyzed 3D-reconstructed volumes [19] of a fluorescent dye solution illuminated with and without an extended field of view.In each case, the same 324 focal points in a 420 μm × 420 μm × 200 μm volume were illuminated within a single camera exposure of T = 100 ms using 36 mW (at the sample) to avoid saturation.In Fig. 4(a), we illuminated all focal points simultaneously with the galvanometer mirrors in a fixed position.The lateral extent of the xy maximum projection after applying a 5% intensity threshold was 140 μm, matching the theoretical value of 140 μm from [9] as described by Eq. ( 2) above.
In Fig. 4(b) we used time-division multiplexing of 9 SLM fields of view with 9 galvanometer positions to extend the field of view to 380 μm, an increase in area of 7.4 times.In addition, it should be possible to address the spatially varying diffraction efficiency [21] visible in Fig. 4(a) by choosing the time-sequentially scanned fields of view to overlap further.

Increasing two-photon signal
To quantify the two-photon signal gain from time-division multiplexing empirically, we restricted the illumination in a fluorescent dye solution to the same 33 focal points within the same single SLM field of view (as in Fig. 1(a)), to control for excitation and emission path vignetting effects, and measured the total detected counts on the image sensor as a function of the multiplexing factor, M, again with fixed laser power and camera exposure time of T = 100 ms.A pockels cell shuttered the laser during the SLM transition periods in Fig. 2(b).Measurements for each M were normalized with M = 1, and compared with Eq. ( 8) in Fig. 5.
For t SLM = 7.5 ms and t SLM = 10 ms, the maximum signal occurs at M = 7 and M = 5, respectively, compared with M = 6.7 and M = 5 predicted by Eq. ( 8).We approximated the slower t SLM = 10 ms SLM, by keeping the pockels cell shuttered for this longer duration.The fact that the experimentally measured traces are slightly lower than the theoretically predicted curves might be explained by the limited precision with which we can synchronize our hardware, including the SLM, pockels cell and camera.
Finally, Eq. ( 8) plotted in Fig. 5 suggests that recent SLMs [10,11] with even shorter transition times may approach the limit where t SLM tends toward 0 ms, where the increase in signal should scale exactly linearly with the number of multiplexed subsets.

Two-photon 3D in vivo calcium imaging with time-division multiplexing
To demonstrate the utility of our extended field of view and increased two-photon signal, we applied our approach to 3D in vivo neuronal calcium imaging using the method in [5], except with the microlens array in our light field microscope [19] in place of the cubic phase mask [5].Briefly, fluorescence representing calcium activity of the neurons [12] in an awake, headfixed mouse, shown in Fig. 6(a), is imaged through the cranial window using the following approach.In the first step, lasting several minutes, single-beam 2D laser scanning two-photon imaging [15] yields high-spatial resolution images identifying the locations of neurons at each of 51 z-depths defined by an SLM-implementation of a quadratic phase.We then select for high-speed recording 104 neurons of interest spanning 600 μm × 600 μm × 200 μm from this 3D image stack and grouped them into M = 5 subsets, color-coded in Fig. 6(b).Next, we use our time-division multiplexing approach with 250 kHz laser with 71 mW (at the sample) to excite fluorescence at all sites, which is recorded through the microlens array and in a single T = 100 ms camera exposure.From each camera image we extract the fluorescence contribution from each of the illuminated sites (more below), and by recording an image sequence, we can sample the underlying calcium activity at each site at a rate of 10 Hz, for a total duration of 50 seconds, as shown in Fig. 6(c).
In total, the M = 5 illuminated sub-regions span an area about 4 times larger than the original SLM field of view of 300 μm × 300 μm, and Fig. 6(c) demonstrates that we observe spontaneous neural activity in sites within every region.Additionally, the signal increase from multiplexing, in combination with other efficiency improvements, enabled in vivo mouse cortex recording using only 71 mW.Taken together, these represent a significant improvement in field of view and requisite laser power over the work reported in Fig. 3 and 4 of [5], where ~1200 mW was required to record from 107 sites in a mouse hippocampus slice in vitro, across a smaller 250 μm × 250 μm × 50 μm volume.Importantly, the limited kinetics of the GCaMP6m calcium sensor [12] enabled us to time-division multiplex our sampling of the sites with T = 100 ms without sampling any less information than if we had sampled all sites simultaneously as in [5].
For completeness, the following describes how the calcium measurements shown in Fig. 6(c) were extracted from raw camera images of the illuminated sites.Our light field microscope projects fluorescence from each site in 3D to a unique image sensor diffraction pattern [19], which is either determined analytically [19] or, in this work, calibrated empirically by recoding an image of each site illuminated in isolation.Because each raw camera image represents a linear superposition of the known, possibly overlapping, diffraction patterns of each of the illuminated sites, we can as in [19], but for a much smaller inverse problem, use Richardson-Lucy deconvolution to solve for the fluorescence at each site.The ΔF/F, a normalized measure of activity, of each site plotted in Fig. 6(c) this computed fluorescence across all camera frames divided by the baseline fluorescence, estimated using the 25th percentile [12].

Discussion and conclusion
We have addressed the field of view and two-photon signal limitations of holographic SLMbased illumination approaches by using a time-division multiplexing strategy to timesequentially tile a larger field of view.We demonstrate that for applications where strictly simultaneous illumination is not required, such as 3D in vivo neuronal calcium imaging, a larger accessible field of view and additional signal can be achieved with little additional cost.
Though the peak power at each focal point in our approach is lower than that of single point-scanning approaches [22], it is higher than that required for simultaneously illuminating all sites, which may lead to fluorophore saturation or photodamage [23,24].Although we showed signal gains for the two-photon excitation case, this approach should work for higher order multiphoton signal generation as well.
Lastly, at present, our approach only extends the lateral, and not axial field of view, but new hardware developments, including other phase modulators such as tunable lenses and deformable mirrors could also be combined to extend the axial field of view as well.

Fig. 1 .
Fig. 1.Illustration of time-division multiplexing strategy for laterally extending the SLM field of view (FOV) from a conventional holographic illumination system (a) to our proposed approach (b), where a pair of galvanometer mirrors at a conjugate pupil plane enable lateral time-sequential scanning of one of 9 different holograms to each of 9 regions.Drawings are to scale for the Nikon 16x 0.8NA objective lens (assuming 20mm field number) and 256 × 256 SLM used in our experiments.

Fig. 2 .
Fig. 2. Timing considerations for SLM time-division multiplexing.(a) Our particular SLM has an empirically measured 0 to ~97% transition time of 7.5 ms.(b) Timing diagram indicating SLM state throughout a given exposure time, T, for M sequentially illuminated subsets of focal spots.Each subset is laterally repositioned by galvanometer mirrors (y-galvo not shown).
and measuring power as a function of time, plotted as the solid line in Fig. where the surrounding shaded region illustrates one standard deviation from three measurements.

Fig. 4 .
Fig. 4. Experimentally measured increase in SLM field of view during T = 100 exposure.Measured 3D illumination patterns in a fluorescent dye solution (maximum intensity projections shown) illustrate the field of view increase, denoted by dashed lines, from (a) 140 μm × 140 μm without, to (b) 380 μm × 380 μm with, time-sequential galvanometer scanning.

Fig. 5 .
Fig. 5. Theoretically computed and experimentally measured signal increase with SLM timedivision multiplexing, for average laser power, total number of illuminated sites and exposure time (T = 100 ms was used).SLMs with faster transition times will enable larger increases in total time-averaged two-photon signal with a greater number of subsets of focal sites illuminated sequentially.Error bars indicate one standard deviation.

Fig. 6 .
Fig. 6. 3D two-photon in vivo calcium imaging.(a) Neuronal activity in barrel cortex of an awake, head-fixed mouse is recorded.(b) 104 recording sites, centered on neurons, divided into M = 5 subsets (annotated by number and subset) and spanning 600 μm × 600 μm × 200 μm, are selected from a single-beam raster-scanned two-photon image stack, and then illuminated using our time-division multiplexing approach, enabling the sampling of (c) calcium signals across all sites at a rate of 10 Hz (T = 100 ms).(d) Top 10 largest magnitude responses from (c).