Miniaturized omnidirectional flexible side-view endoscope for rapid monitoring of thin tubular biostructures

: Endoscopic imaging allows longitudinal observation of epithelial pathologies in tubular organs throughout the body. However, the imaging and optical diagnosis of tubular biostructures such as small animal models and small pediatric organs require appropriately miniaturized devices. A miniaturized catadioptric flexible side-view endoscope is proposed with omnidirectional field of view (FOV) in the transverse direction and sub-mm-scale feature resolution. The FOV in the longitudinal direction is 50°. Images are unwrapped and stitched together to form composite images of the target by two different algorithms, revealing a composite FOV of more than 3.5 cm × 360°. The endoscope is well suited for minimally invasive rapid monitoring of thin tubular organs in pediatric patients, as well as for imaging of small animal disease models at near-cellular resolution.


Introduction
Catadioptric devices, consisting of a combination of reflective and refractive elements, are uniquely suited for applications requiring instantaneous imaging of a panoramic field of view with a single sensor, and as a result they have found applications in fields as diverse as computer vision and navigation [1], industrial oil pipe inspection [2], and wide-angle surveillance [3,4], as well as in gastrointestinal imaging [5][6][7]. More recently, several teams have investigated side-viewing but non-catadioptric endoscopes for biological applications, such as esophageal endoscopy [8], and brain imaging [9,10].
In the biological sciences, as in industry, there is high demand for miniaturized imaging technology. A large community of researchers is involved in the development of small animal models of cancer in tubular epithelia such as the oral cavity [11], esophagus [12,13], and the colon [14][15][16][17][18]. There is also significant interest in micro-endoscopic imaging of the cardiac organs and the brain [19,20]. Small animal models allow quick and precise tailoring of genetic variants to reveal chemical signaling pathways, reduce the time and cost of experiments, and allow preclinical evaluation of potential therapeutic compounds [21]. Transfection of fluorescence-inducing viral vectors has allowed the fluorescent tagging of tumor cells for orthotopic injection into the colon wall [14][15][16][17][18]. Fluorescent pharmaceutical compounds have also been under development [22]. For the imaging of these targets in vivo and the full longitudinal monitoring of the tumor development cycle, miniaturized side-view endoscopes are necessary. Miniaturized endoscopes developed for small-animal applications may later be adapted for pediatric and minimally-invasive clinical applications.
Catadioptric endoscope probes allow side-view imaging behind epithelial folds, which are not accessible to surgeons when standard or wide-angle endoscopes are used [23,24], potentially enabling the rapid detection of early-formation lesions. At present, state-of-the-art side-view imaging is performed by rotating a mirrored micro-endoscope probe to stitch

Device design and experimental setup
In the application to side-view biological imaging ( Fig. 1(a)), epithelial tissues such as the colon, esophagus, trachea and even cerebral and vascular tissues will surround an endoscopic probe such that the side-view probe is located at the central position of the tubular organs [6,[29][30][31], thereby maintaining a known working distance. To miniaturize the probe and thereby minimize tissue damage, it was decided to focus on a single-fold catadioptric geometry. In this way, not only could the probe be miniaturized, but aberrations introduced by the mirror processing could be minimized, and the mirror assembly could be designed without an additional interior aperture. The standard optical model for catadioptric devices is the single-view-point design [30], which results in an undistorted mapping between the world frame and the camera image, provided that mirror geometries are selected from the conic sections [32]. Compared to other strategies, mapping the catadioptric device field of view to a cylindrical projection of the world frame, trades a loss of overall image pixels and resolution for a wider field of view and simultaneous imaging in multiple directions [5]. For miniaturized catadioptric devices, where sensor area is at a premium (the commercially available sensors used in this study had a sensor width of 1.2 mm on a 3 mm wide package), device resolution is primarily limited by the field of view (FOV) of the lens. The first goal of lens selection or design is thus to maximize the image of the mirror in the sensor FOV at the design working distance. Careful mirror selection will then minimize the area of the image sensor which captures the reflection of the sensor itself in the mirror.
Three narrow-diameter, commercially available lenses were identified which met the required specifications of low aberration, a diameter of less than 3 mm, and a working distance of less than 5 mm. These were the SELFOC gradient index microlens, and the Sumita SEL 110 and the Sumita SEL 120 miniaturized lens assemblies.
The mirror was designed to minimize the field of view which reflected the lens, and for this purpose a parabolic geometry was found to be superior to spherical or hyperbolic geometries. The mirrors shown in Table 1 were fabricated of unprotected aluminum, and image quality was determined qualitatively, based on direct comparison of resolved features from identical printed cylindrical targets, as shown in Fig. 2. Conical mirror geometries show high astigmatism, due to their previously observed large birefringence and high field distortion [29]. For spherical and parabolic mirrors, a lower radius of curvature results in higher-resolution images, but also reduces the field of view. Best image quality was achieved by pairing a SEL 120 lens assembly (Sumita, Japan) with a parabolic mirror of focus 0.172 mm and diameter 2 mm (Table 1 and Fig. 2).
Mirrors were fabricated of unprotected aluminum by an ultraprecise machining process (K-Bio Health, Osong, Korea). Tolerances for mirror manufacture were ± 0.003 mm.
Lens and mirror mounts which were fabricated by additive manufacturing were used to immobilize lenses and mirrors relative to a transparent quartz tube of fixed length, as well as relative to the image sensor, as shown in Fig. 1(b). Representative mirror mount and lens mount schematics are shown in Fig. 1(d) and 1(e), respectively. Mirror mounts had diameter matching that of the quartz or acryl tube to be used (OD = 5.00 mm), with an extrusion which fit snugly within the tube (ID = 3.00 mm) and around the mirror (D m in Table 1), with tolerances of t = 0.10 mm. Lens mounts (length 28.00 mm) contained a hollow chamber for the sensor and integrated circuit (D IC = 5.00 mm), a hole which fit snugly around the lens assembly (D L = 1.00 mm for SELFOC, 1.2 mm for SEL110, and 2.2 mm for SEL120), and extrusions to fit snugly around and within the tube (OD and ID, respectively). Six additional holes of radius 0.5 mm allowed for the draining of fluids during manufacture. Lens mounts were fabricated on a Formlabs Form2 stereolithographic (SLA) printer using white photopolymer resin (FLGPWH03) and grey resin (RS-F2-PRGR-01), followed by an isopropyl alcohol rinse to achieve a print resolution of 0.05 × 0.05 × 0.025 mm. Mirror mounts were fabricated on a ProJet 3500 SD printer using translucent VisiJet M3 Crystal resin at 375 × mounts in the ProJet were c rinsing in an standard were GC0309 sens VGA (  Figure 3 s was simulated (Fig. 3(a)  The exper Cylindrical sl 1000 dpi (38 around cylind MTF was calc the device suit mm and a diam onoscopy prob low as 3.5 mm hich is also a su shows the optic d in Zemax wit to obtain the demonstrate de (about 65°). birefringence a 5 microns for r), and a maxim ray at +57° (+3 n is optimal at ulation transfer . 3(c)), and a ge sensor (   Fig. 3

Results and discussion
For the clinical application of catadioptric imagers, it is desirable to create cylindrical projections of the initial images and to composite those images together into cylindrical composites of the target tubular biosystem. Figure 4 shows a demonstration of two different versions of this unwrapping algorithm and image composition process. As the probe is moved down a cylindrical surface, images are captured, as shown schematically in Fig. 4(a). In the first process, these images are unwrapped to a cylinder individually using a polar coordinate transformation. The unwrapped images are then composited to obtain a larger image using a panorama stitching algorithm [35]. Figure 4(b) shows the unwrapped grid template images, while in Fig. 4(c), the grids are combined into a composite image. This technique is advantageous when the movement of the probe is not smooth, but it is limited by the requirement that features for stitching be comparable between images. A second method for image unwrapping allows for the automatic retrieval of a cylindrical image from video. A single line of the composite image is interpolated from a circular region of each frame of an endoscopy video, giving a very quick and feature-independent unwrapped cylinder. Figure 4(d) shows this counterclockwise interpolation along a circular region in each frame to give a composite unwrapped cylindrical image (Fig. 4(e)).
Several distortions observed in actual images show difficulties which are not encountered in simulated images, namely that the aspect ratio of features in the image depends on the distance from the feature to the mirror. This can result in distortion when using non-radial (or improperly centered) imaging targets. also be seen tha mager. This is l ce of the objec mage quality wa When imaging nce during asse it was also dif ng assembly, c lly from the q mage sensor.     The fabricated endoscope was further tested by insertion into a pediatric vascular 3Dprinted model mimicking a real congenital heart disease patient. A continuous video was captured while the probe was removed from the cardiac phantom. Following the method from Fig. 4(e) and 4(f), a circular section (red dashes) was interpolated from each frame into a cylindrical image, as shown in Fig. 7(a). The probe clearly captured diverging arteries (blue arrows) in the original video ( Fig. 7(b)), as well as in the unwrapped image ( Fig. 7(c)). The well-focused texture of the vascular phantom can be seen in the unwrapped image. The obtained image can be re-wrapped around a 3D model for a fly-through examination of the cardiac phantom, as shown in Fig. 7(d). This is the first realistic phantom image captured by a miniaturized catadioptric endoscope probe.

Conclusion
We report on the realization of a miniaturized flexible catadioptric sensor capable of panoramic view and composite image generation, with applications to pediatric diagnosis and small animal model endoscopy. The device performance was evaluated theoretically and experimentally, showing efficient performance compared to previously published larger prototypes, and significant performance given the miniaturized device type. Following simulations, MTF was calculated to be above 0.2 at 25 lp/mm. Field of view was 50° by 360°, which is exceptional for a side-view device. The device prototype is demonstrated in a pediatric cardiac phantom to give acceptable images in a packaged diameter of 5 mm, and may be further miniaturized by suitable packaging. In short, this family of miniaturized single-fold catadioptric devices is expected to be highly useful for the rapid diagnosis and treatment of thin tubular biostructures.