A four-dimensional snapshot hyperspectral video-endoscope for bio-imaging applications

Hyperspectral imaging has proven significance in bio-imaging applications and it has the ability to capture up to several hundred images of different wavelengths offering relevant spectral signatures. To use hyperspectral imaging for in vivo monitoring and diagnosis of the internal body cavities, a snapshot hyperspectral video-endoscope is required. However, such reported systems provide only about 50 wavelengths. We have developed a four-dimensional snapshot hyperspectral video-endoscope with a spectral range of 400–1000 nm, which can detect 756 wavelengths for imaging, significantly more than such systems. Capturing the three-dimensional datacube sequentially gives the fourth dimension. All these are achieved through a flexible two-dimensional to one-dimensional fiber bundle. The potential of this custom designed and fabricated compact biomedical probe is demonstrated by imaging phantom tissue samples in reflectance and fluorescence imaging modalities. It is envisaged that this novel concept and developed probe will contribute significantly towards diagnostic in vivo biomedical imaging in the near future.

(i) Reflectance imaging using chicken breast tissue with blood clot.
In this experiment, a chicken breast tissue with a blood clot was used as the sample ( Supplementary   Fig. 1a) and imaged. The sample is divided into Regions B1, B2 and B3. Region B1 is the chicken breast tissue. Region B2 is a thin layer of blood clot on the chicken breast tissue. It can be observed from Supplementary Fig. 1b that the chicken breast tissue is still partially visible in Region B2.
Region B3 is the blood clot. The sample was manually moved using a mechanical stage towards the right of the 2-D end of the fiber bundle during data acquisition. With respect to the sample, the 2-D end of the fiber bundle was moving downwards (arrow in Supplementary Fig. 1b). A total of 80 frames were taken at a rate of ~6.16 Hz. Region B1 is the chicken breast tissue. Region B2 is a thin layer of blood clot on the chicken breast tissue which is still partially visible. Region B3 is the blood clot. The fiber bundle in (b) shows its initial position and the arrow indicates its relative movement with respective to the sample during data S2 acquisition. (a) and (b) are images of the same sample but appear to have different colors due to the different illuminations and cameras used.
Supplementary Fig. 2 shows the reflectance mappings of nine datacubes at 600 nm. Supplementary Videos 3 and 4 are made up of 80 frames each, using reflectance mappings of 450 nm and 600 nm, respectively. By looking at the frames in Supplementary Fig. 2

sequentially and Supplementary
Videos 3 and 4, it can be further confirmed that the proposed system was able to perform HS reflectance imaging in a snapshot configuration. The different reflectance between Regions B1, B2 and B3 can be differentiated from each another. The 2-D end of the fiber bundle was initially imaging Region B1 of high reflectance. Then it moved downwards with respect to the sample and started to image Region B2 of moderate reflectance on its left. Following this path, it started to image Region B3 of low reflectance and proceeded to image Region B1 again before data acquisition stopped. These depict the actual relative motion between them during data acquisition ( Supplementary Fig. 1b). Close to the center of the blood clot, there was a small area of Region B1 in between Regions B2 and B3.
This area can be seen in Frame 33 of Supplementary Fig. 2 which represented its size and shape correctly. The spectra in Supplementary Fig. 3 show that the 4-D HS imaging probe could capture the detailed reflectance spectra of Regions B1, B2 and B3 while there was a relative motion between the sample and the 2-D end of the fiber bundle. It can be observed that Region B1 (chicken breast tissue) had the highest reflectance, while Region B3 (blood clot) had the lowest. The reflectance spectrum of Region B2 is between the spectra of Regions B1 and B3. This could be due to Region B2 having the thin layer of blood clot causing the chicken breast tissue underneath it to be still partially visible. The average standard deviations of the reflectance spectra of Regions B1, B2 and B3 were about ±1.31%, ±1.37% and ±0.98% respectively.

(ii) Data acquisition-Details
Data acquisition was done using the dedicated software of the detector camera (SOLIS, Andor). The selected ROI was 1004×756 pixel 2 (spatial×spectral) which corresponds to the spectral range of interest from 400-1000 nm. Although the exposure time was set to 0.1 s, the software set the kinetic cycle time to 0.16221 s. Therefore the images were acquired at a rate of ~6.16 Hz. The electronmultiplying gain of the detector camera was turned off for reflectance imaging, but set to 100 for fluorescence imaging. During the experiment, the detector camera captured a series of 1004×756 pixel 2 images at a rate of ~6.16 Hz until the number of images taken matched the pre-determined number of images to capture. Each image was named in sequence and saved as separate file after the experiment.

S5 (iii) Data processing and visualization-Details
Data processing was done offline using MATLAB®. In reflectance imaging, Sample data was acquired from the sample. The Sample data was corrected using dark reference (Dark) and white reference (White) using equation (1) to get the Reflectance data.
, , (1) Dark data was acquired when the broadband light source was turned off and the forelens covered. It represents the image with dark current noise where the reflectance was 0%. White data was acquired by imaging the 99% reflectance standard where the reflectance was 99%. A set of ten images were taken and averaged to give the Dark and White data. x and λ refer to the column and the calibrated spectral band allocated to the row of the sensor array's selected ROI respectively. Frame refers to the number of images taken for "sample data", and in this case it was 80. Smooth is the 9-point moving average in the spectral direction for spectrum smoothing.
In fluorescence imaging, Sample data was acquired from the sample and 160 images were taken. The Sample data was corrected using a dark reference (Dark) and the quantum efficiency of the detector camera (QE) using equation (2) to get the Fluorescence data.
, , Dark data was acquired when the laser was turned off and the forelens covered. It represents the image with dark current noise and without any fluorescence. QE took into account the varying sensitivities the detector camera had with different wavelengths. A set of ten images were taken and averaged to give the Dark data. Norm was to normalize the entire data set to one.
After applying the calculations, the reflectance and fluorescence measurements were having a spatialspectral-frame data of 1004×756×frame. Using the spatial calibration done on the 1-D end of the fiber bundle, the spectrum for each fiberlet was extracted from the relevant spatial positions to form a fiberspectral-frame data of 100×756×frame. Since Fiberlet 4 was inactive, its spectrum was assigned to be zero. Using the data from each frame, there was a digital reconstruction step to remap the spectrum of each fiberlet back to the respective position on the 2-D end of the fiber bundle. In order to get a correct visualization of the imaged sample, the data was flipped horizontally in the spatial direction as S6 the left side of the 2-D end of the fiber bundle was used to image the right side of the sample, and vice versa.

Supplementary Video
The relevant videos demonstrating the proposed concepts and the related data are as per below.
Supplementary Video 1. A reflectance video with 80 frames of 575 nm at ~6.16 Hz acquired during the imaging of the simulated tissue sample.