Two-Photon Lensless Micro-endoscopy with in-situ Wavefront Correction

Multi-core fiber-bundle endoscopes provide a minimally-invasive solution for deep tissue imaging and opto-genetic stimulation, at depths beyond the reach of conventional microscopes. Recently, wavefront-shaping has enabled lensless bundle-based micro-endoscopy by correcting the wavefront distortions induced by core-to-core inhomogeneities. However, current wavefront-shaping solutions require access to the fiber distal end for determining the bend-sensitive wavefront-correction. Here, we show that it is possible to determine the wavefront correction in-situ, without any distal access. Exploiting the nonlinearity of two-photon excited fluorescence, we adaptively determine the wavefront correction in-situ, using only proximal detection of epi-detected fluorescence. We experimentally demonstrate diffraction-limited, three-dimensional, two-photon lensless microendoscopy with commercially-available ordered- and disordered multi-core fiber bundles.


Introduction
Flexible optical endoscopes are an important tool for a variety of applications from clinical procedures to biomedical investigations. A common use of such endoscopes is macroscopic imaging inside hollow organs. Another important application is in micro-endoscopy, where micron-scale structures such as single neurons are imaged or optically excited at depths beyond the reach of conventional microscopes. The latter are limited in their penetration depth due to tissue scattering and absorption [1]. In recent years, various solutions for small diameter micro-endoscopes have been developed [2,3]: Micro-endoscopes that are based on single-mode fibers are bend-insensitive, but require distal optical elements such as scanners and lenses [2], or spectral dispersers [4,5] to produce an image. Such distal elements may significantly enlarge the endoscope's diameter, increasing tissue damage, and consequently limiting its use for deep-tissue imaging. Developing a flexible, lensless micro-endoscope with a minimal diameter is thus a sought after goal for minimally-invasive deep-tissue imaging.
Currently, the solutions for constructing lensless endoscopes are based either on imaging fiber bundles [2] or wavefront-shaping [6][7][8][9][10]. Imaging fiber bundles consist of thousands of single-mode cores packed together, where each core functions as a single pixel. While common and straightforward to use, conventional lensless bundles-based endoscopes suffer from limited resolution, pixelation, poor axial sectioning, and a small and fixed working distance.
While axial-sectioning can be obtained in fiber bundles by addition of confocal scanning [2] or structured illumination [11], the working distance is fixed to the fiber facet or its image, and does not allow three-dimensional (3D) imaging.
In recent years, a number of works demonstrated the use of wavefront-shaping for microendoscopy [6,[8][9][10][12][13][14][15]. In wavefront shaping, a computer-controlled spatial light modulator (SLM) is used to compensate the phase randomization and mode-mixing in fiber bundles or multi-mode fibers, in a principle similar to the decades-old works in holography [16,17].
However, since the phase distortions in long fibers are sensitive to fiber bending, even with the state-of-the-art digital wavefront control a direct feedback from the fiber distal end or precise knowledge of the fiber shape [15] are still required to determine the wavefront correction. These requirements make the application of wavefront-shaping techniques for flexible endoscopes very challenging in most practical imaging scenarios. Recently, speckle correlations in the optical transmission-matrices of fiber bundles, known as 'memory effect' correlations, have been exploited for computationally reconstructing distal images from proximal measurements, without wavefront correction [18,19]. However, these approaches currently allow only two-dimensional imaging, and provide high-fidelity reconstruction only for simple objects.
Here, we present a lensless two-photon microendoscope based on wavefront-shaping, which does not require any a-priori knowledge of the fiber transmission-matrix or access to the distal end, even after fiber bending. In our approach the wavefront correction is performed in-situ, by iterative optimization of two-photon fluorescence (2PF) signals detected at the proximal end. Our approach is based on the fact that iterative optimization of a nonlinear signal leads to diffraction limited focusing [20], even when the signal is collected by a spatially-integrating detector having no spatial resolution. This principle has been recently exploited for focusing light through scattering samples [20] or multi-mode fibers [21], and here we show its use for endoscopic imaging. Our imaging technique is composed of two steps: the first is focusing of the excitation beam on the target object by iterative wavefront optimization. After the wavefront-correction has been found, two-photon imaging is performed by scanning the formed focus in three-dimensions using the single wavefront correction, exploiting the 'memory-effect' of imaging fiber bundles [7,18,19].

Methods
The experimental setup is illustrated in Fig.1. A beam from a Ti:Sapphire laser oscillator (Mai-Tai DeepSee eHP, Spectra-Physics) producing 85 fs pulses at 785nm wavelength, is reflected off a galvanometric mirror (GVS012, Thorlabs), The galvanometric mirror is imaged on a phase-only SLM (X13138-02, Hamamatsu Photonics) by a 4-f telescope (L 1 = 35mm, L 2 = 500mm). The SLM is imaged on the proximal facet of a fiber bundle (either Schott 1533385 in Figs. 2-4, or Fujikura FIGH-03-215S in Figure 5d,i) using an additional telescope (L 3 = 250[mm], Obj 1 ; 20X Plan Achromat, Olympus). Two-photon fluorescent (2PF) target objects were placed simultaneously at several distances of 0.8mm-3mm from the fiber distal end. The excited 2PF signals are collected by the same bundle, propagate back to the proximal end, separated from the laser wavelength by a dichroic mirror (FF605-Di02-25x36, Semrock), and focused on a detector (either an sCMOS camera (Zyla 4.2 Plus, Andor) or a photomultiplier tube, PMT (X13138-02, Hamamtasu)), after additional filtering (FF01-510/84-25 bandpass, FF01-650/SP-25 short-pass filter, Semrock). 2PF imaging is performed in-situ in two steps: first, the SLM wavefront-correction for focusing is found by iteratively optimizing the total 2PF signal collected by the detector. This step produces a sharp focused beam on the target object [20]. After the wavefront correction is found, the focused spot is raster-scanned in 3D by adding a parabolic phase on the SLM for axial scanning, and a wavefront tilt with the SLM and/or Galvo for lateral scanning. The 2PF signal detected at the proximal end is used to generate a 3D image of the target objects, as in conventional 2PF microscopy [20]. For inspecting the focusing performance, a reference camera (Mako U-130B, Allied Vision) is used to directly image the object plane, using a microscope objective (Obj 2 ; 10X Plan Achromat, Olympus). Importantly, the reference camera is used only for results inspection, and is not required for successful focusing or imaging.  The SLM is imaged on the proximal facet of a fiber-bundle. Two-photon fluorescent (2PF) targets are placed at short distances from the distal end of the fiber. The excited 2PF is collected by the same fiber and detected at the proximal end using a PMT or an sCMOS camera. The detected 2PF signal is used as a feedback for an iterative wavefront-shaping optimization process, aimed at maximizing the total detected 2PF, forming a sharp focus on the target. 3D two-photon imaging is achieved by scanning the formed focus with the SLM and galvanometric mirror. A reference camera is used only for results inspection.

In-situ Focusing
As a first demonstration, 2PF particles of Coumarin 307 were placed at distances of 2mm − 3mm from the bundle's distal facet, at two axial planes. Initially, the light at the object plane, as recorded by the reference camera, was a random speckle pattern (Fig.2a). After 2,500 iterations of an optimization algorithm maximizing the total collected 2PF signal, I 2PF , a sharp focal spot was formed on the object closest to the fiber facet (Fig.2c). The obtained focus size is of the speckle-grain (diffraction-limited) dimensions (see Fig.4), and has a peak to background ratio (PBR) of ∼ 310 (Fig.2c). For optimizing the 2PF signal we have used an iterative partitioning algorithm [22,23]. In this algorithm, the SLM was divided into 128X160 equally sized square segments. In each iteration, a phase, φ n of zero to 2π is added to a random subset of these segments in N = 4 steps. The 2PF signal as a function of the added phase is fitted to a cosine: I 2PF (φ n ) = A + B · cos(φ n − φ), using a fast Fourier transform, and the phase φ that maximizes the 2PF signal is added to the chosen segments. The obtained 2PF signal as a function of the iteration is plotted in Figure 2d. The total optimization time was limited by the refresh rate of the liquid-crystal SLM ( 5Hz) to tens of minutes. However, the optimization time could be significantly reduced since the integration times were of the order of 0.1[ms] even in the first iterations of the optimization (see Discussion).

3D two-photon imaging
Following the adaptive focusing, 3D imaging was performed by raster scanning the focus. Fast scanning, with a pixel dwell-time significantly shorter than the SLM refresh rate, was achieved by using a galvanometric mirror for fast scanning in one lateral dimension. Scanning in the other lateral dimension was implemented by addition of a linear phase ramps on the SLM wavefront correction. Scanning in the axial dimension was achieved by adding parabolic phase patterns to the SLM wavefront correction, effectively transforming the fiber bundle to an adaptive lens. c. d.  Fig.3(a-d) shows the 2PF images obtained with the focus scanning for two axial planes, where the fluorescent objects were present. Thanks to the inherent axial sectioning of 2PF focused excitation, at each plane, only the objects residing in this plane are visible (Fig.3b,d), whereas the parts of the object residing at different axial planes do not contribute to a substantial background halo, as is visible in the conventional bright-field imaging performed with the reference camera (Fig.3a,c.). Fig.3(e) shows additional two-photon images obtained at axial planes close to the plane of Fig.3(a-b).

Characterization of the formed focus
In wavefront shaping, the focusing resolution is expected to be diffraction-limited, as dictated by the dimensions of a single speckle grain [24] To characterize the axial resolution, the 2PF signal in the image stack presented in Fig.3(e) was plotted as a function of the axial distance. Fig.4a displays the trace of the maximal 2PF intensity obtained at each depth. The axial resolution, defined by the FWHM of a fit to a Gaussian, is δz = 165µm ± 10. The axial resolution, δz in 2PF microscopy is expected to be 0.64 times the FWHM of the focused beam, which is twice its  e. object was removed, and the focus was imaged by the reference camera. The measured transverse (x-y) focal spot size was measured to have a FWHM of 5.4µm ± 0.2 ( Fig.4(b)). A value which is in accordance with the measured axial resolution, and also the expected diffraction-limited spot size dictated by the measurement geometry [18]: where D bundle ≈ 0.45mm is the fiber bundle's diameter, Z ≈ 2.2mm is the distance between the fiber facet and the object in this experiment, and N A ≈ 0.35 is the numerical aperture.

Comparison of ordered vs. disordered bundles for wavefront-shaping based endoscopy
As is visible in the focusing results of Fig.2(c), the obtained focal spot is surrounded by six visible side-lobes, positioned in a hexagonal pattern. These side-lobes are the result of the hexagonal periodicity of the cores in the fiber bundle. Fig.5a shows an image of the fiber facet used in the  above experiments, displaying the hexagonal lattice order. The side-lobes in the formed focus are predicted by the Fourier transform of the cores arrangement (Fig.5b,c) [25, 26]. To suppress the side-lobes, we tested a commercially-available fiber with a less ordered arrangement of cores (Fujikura FIGH-03-215S). Indeed, using such a disordered fiber the side-lobes are effectively suppressed (Fig.5d-f).
The main advantage of using non-ordered fiber bundles is the absence of diffraction side-lobes. When used for focus scanning based imaging, such side-lobes produce replicas of the object parts that are scanned by them. Thus, the side-lobes limit the transverse extent, i.e. field of view (FoV), of the imaged objects to be smaller than x ≈ z · λ/d , where d is the core-to-core spacing [25]. Fig5h,i displays a 2PF image obtained with our approach using the disordered fiber, showing accurate diffraction-limited 2PF image of the target objects (fluorescent beads).
While we have expected the disordered fiber to provide a considerably larger FoV, we have noticed that this was not the case, as is quantified by measuring the reduction in focus intensity as a function of the scan angle (Fig.5g). This narrower 'memory-effect' angular range of this specific fiber is not an inherent limitation of disordered fibers. We attribute the smaller FoV to light propagation between the cores of this fiber. This can be observed in the lower cores-to-background contrast in Fig.5d, compared to Fig.5a. Another effect that this imperfect light guidance is causing is a narrower spectral speckle correlation bandwidth that we have measured for this fiber (not shown). This narrower spectral correlation bandwidth leads to a lower speckle contrast [27] that is measured when the 12nm-wide femtosecond pulsed illumination is used. This in turn, lowers the focus intensity enhancement obtained by wavefront shaping [24]. To increase the initial speckle contrast and focusing PBR using this fiber, a narrow BPF (LL01-785-25, Semrock, 3nm FWHM) was used in the illumination path in the experiments involving this fiber.

Discussion
We have demonstrated an in-situ wavefront-correction approach for two-photon microendoscopy. Since the wavefront correction is sensitive to the bending of the fiber, for each fiber orientation a different correction needs to be found. However, this can be continuously performed during in-vivo experiments. In our experimental implementation, the limiting factor on the time required for determining the wavefront-correction was the refresh rate of the specific liquid-crystal SLM used (<5 Hz). This yielded optimization times of tens of minutes in our experiments. This can be significantly shortened by using faster SLMs, since fundamentally the optimization time is limited by the 2PF signal level. In our experiments, integration times of 0.  the first iterations of the optimization process. Significantly lower integration times are required in the following iterations, when the signal grows. Thus, finding the wavefront correction using SLMs with higher refresh rates, such as Digital Mirror Devices (DMDs) [28], or galvanometric mirrors based approaches [30] can decrease the optimization time by more than three orders of magnitudes. Using more advanced algorithms such as genetic algorithms [29] can also reduce the number of iterations required for optimization. These are expected to yield an optimization time of the order of seconds or less even using fluorescent markers, as was recently demonstrated using galvanometric-mirrors for 2PF microscopy through scattering tissue [30]. In general, any similar approaches for wavefront correction using nonlinear feedback that were originally developed for scattering media are directly applicable for bundle-based endoscopy, since a fiber bundle can be considered as a thin scattering layer [18]. After the initial focusing, image acquisition speed is limited by the galvanometric scanners speed. Faster scanners based on acousto-optic scanners, or resonant galvanometric mirrors can be utilized for faster scanning. An inherent limitation of wavefront-shaping based correction is the sensitivity to fiber bending. While our proximal-only detection approach does not require initial measurements prior to the insertion of the endoscope, or any knowledge of the fiber parameters or shape, any movement of the fiber after the optimization process will hamper the correction and decrease the focus intensity. For a static fiber left untouched, we have experimentally measured decorrelation times of more than 12 hours. However, small bending and shifts of a few millimeters caused complete decorrelation of the speckle patterns and focusing intensity in both fibers used. To overcome this restriction, continuous adaptive focusing may be used to adaptively compensate for the fiber movement during imaging, as small movements of the bundle translates to small shifts and small decorrelation of the focus [8]. Another possible approach is to deploy the endoscope inside a rigid cannula [31], which will significantly improve stability at the price of an increase in the overall endoscope diameter, and loss of mechanical flexibility.
The field of view using fiber bundles with ordered cores is limited by the diffraction side-lobes of the periodic arrangement of cores [18], and the fiber 'memory-effect' angular range. The maximal angular FoV will be attained by using a disordered bundle having single-mode fibers, with no light propagation between the cores. Unlike the memory-effect in scattering media, the memory-effect range in fiber bundles is not limited by the fiber length, and for an ideal bundle having single-mode cores, with no coupling, the memory-effect FOV spans the entire NA of the fiber [18]

Conclusion
We have demonstrated a minimally-invasive, lensless, two-photon micro-endoscope with in-situ wavefront correction. In contrast to prior works, our technique does not require any distal access or prior characterization, making it an interesting potential solution for imaging or optogenetic stimulation [32]. We used the non-linearity of the two-photon excitation process to generate a focal spot. Other nonlinear mechanisms such as three-photon fluorescence, second harmonic generation, or stimulated Raman scattering may also be used. Using an SLM with faster refresh-rate may allow to use our approach for freely behaving animal studies.