Diffuser-based fiber endoscopy for single-shot 3D fluorescence imaging

. Minimally invasive endoscopy using coherent fiber bundles shows great potential for numerous applications in biomedical imaging. With a di ff user on the distal side of the fiber bundle and computational image recovery, single-shot 3D imaging is possible by encoding the image volume into 2D speckle patterns. In comparison to equivalent lens systems, a higher space-bandwidth product can be achieved. However, decoding the image with iterative algorithms is time-consuming. Thus, we propose utilizing a neural network for fast 2D and 3D image reconstruction at video rate. In this work, single-shot 3D fluorescence imaging with an ultra-thin endoscope is demonstrated, enabling applications like calcium imaging for in vivo brain diagnostics at cellular resolution.


Introduction
Minimally invasive in vivo imaging in hardly accessible locations is crucial for many biomedical applications [1].Moreover, there is a great research interest to investigate structural and functional processes in freely moving organisms at cellular resolution [2].Potential in vivo imaging systems need to be robust towards movement and ultrathin to reduce damage to delicate tissue or vessels.Furthermore, a high spatiotemporal resolution, large Fields of View (FoV) and 3D imaging capabilities are desirable for applications like calcium imaging to investigate brain activity [3].
State-of-the-art lens-based coherent fiber bundle (CFB) endoscopes only allow 2D imaging in the focal plane of the lens.Multimode fibers as well as CFB enable lensless endoscoping imaging [4,5].Replacing the lens with a coded aperture, such as a diffuser, for intensity imaging allows both 3D imaging as well as higher SBP than equivalent lens-based endoscopes in 2D-imaging scenarios [6][7][8].Instead of time-consuming iterative approaches, neural networks offer the possibility of video rate computational image reconstruction [9].
In this work, we demonstrate an ultra-thin CFB-based diffuser endoscope with a probe tip diameter of 700 µm and single-shot 3D live fluorescence imaging capabilities.The setup can be illuminated through the optical setup from the proximal end of the CFB, enabling biomedical applications for in vivo fluorescence imaging at cellular resolution like calcium imaging.Illuminated 3D objects are encoded by the diffuser into 2D speckle patterns, that are transmitted by the CFB onto a camera.The 3D object * e-mail: tom.glosemeyer@tu-dresden.de is then computationally reconstructed by neural networks in 20 ms.The setup is shown in Fig. 1.

Results
The approach was first tested in silico for 2D and 3D imaging.The system was simulated using Fourier optics to generate pairs of ground truth and camera images as a training dataset comprised of augmented MNIST digits.A neural network architecture consisting of a combination of Single Layer Perceptron (SLP) and U-Net offered the best reconstruction quality and generalization capabilities, see Fig. 1c.For the 2D simulation, a peak signal-to-noise ratio (PSNR) of 25 dB was achieved on the test data.Next, the 3D capability was tested.After extending the simulation and reconstruction network to a 3D object volume, a PSNR of 29 dB could be achieved.
For the validation of these simulation experiments, a fluorescence imaging setup was used.A digital light projector was employed to project images from different axial distances into the system to train reconstruction networks with a 3D output.After the training, we tested the setup for imaging of fluorescent particles in 2D and 3D by illuminating from the proximal side through the CFB.In both cases, we were able to show that the generalization ability of the networks were sufficient to locate few particles in the FoV.However, the reconstruction is limited to sparse objects.To validate the depth perception, an object was projected into the setup at varying axial positions and reconstructed by the neural network.For both the 2D and the 3D case, the neural networks are able to reconstruct the image in under 20 ms, which offers a reconstruction rate of up to 50 frames per second.

Discussion
The diffuser endoscope can be used for single-shot 3D fluorescence imaging with proximal illumination through the CFB, potentially enabling applications like time-resolved calcium imaging or cancer diagnostics.By using incoherent illumination, only the intensity carries useful information while the phase can be omitted.That makes the imaging system robust to bending of the fiber bundle as well as optical path length differences between fiber cores.Moreover, the diffuser-based endoscope can offer a higher SBP than an equivalent lens-based endoscope.Furthermore, the reconstruction approach with neural networks is fast enough to do live imaging, making real time in vivo applications possible.One limitation of the neural network reconstruction is the sparsity of the object scene to reconstruct.Less sparse input to the optical system leads to smaller contrast in the speckle patterns that are transferred to the camera by the CFB.In the future, the diffuser and the neural network could be optimized further.

Figure 1 .
Figure 1.(a) Comparison of the ultra-thin probe tip with a diameter of 700 µm to the size of a coin.This enables minimally invasive biomedical applications.(b) Imaging Principle.An illuminated object (2D/3D) is encoded into 2D speckle patterns by a diffuser.The intensity pattern is transmitted by a coherent fiber bundle (CFB) onto a camera.(c) Neural Network Architecture for image reconstruction.The input measurement by the camera is fed into a Single Layer Perceptron (SLP) which is trained with Augmented MNIST digits so that the learned weights represent the inverse of the optical transfer function.The intermediate reconstruction is then denoised by a U-Net to retrieve the final output.The networks are trained separately.The training input of the U-Net are the predictions of the training data by the SLP.