Integrated multimodal photoacoustic microscopy with OCT- guided dynamic focusing

: Combining different contrast mechanisms to achieve simultaneous multimodal imaging is always desirable but is challenging due to the various optical and hardware requirements for different imaging systems. We developed a multimodal microscopic optical imaging system with the capability of providing comprehensive structural, functional and molecular information of living tissues. This imaging system integrated photoacoustic microscopy (PAM), optical coherence tomography (OCT), optical Doppler tomography (ODT) and confocal fluorescence microscopy in one platform. By taking advantage of the depth resolving capability of OCT, we developed a novel OCT-guided surface contour scanning methodology for dynamic focusing adjustment. We have conducted phantom, in vivo , and ex vivo tests to demonstrate the capability of the multimodal imaging system for providing comprehensive microscopic information of biological tissues. Integrating all the aforementioned imaging modalities with OCT-guided dynamic focusing for simultaneous multimodal imaging has promising potential for preclinical research and clinical practice in the future.


Introduction
Over the past years, various optical microscopy techniques, such as optical coherence tomography (OCT) [1,2], confocal fluorescence microscopy (CFM) [3][4][5], photoacoustic microscopy (PAM) [6][7][8], and optical Doppler tomography (ODT) [9][10][11], have been established and broadly applied in biomedical research and preclinical/clinical studies. All these optical imaging techniques have been shown to be a powerful tool for revealing structural, functional and molecular features of biological specimens noninvasively [12][13][14][15][16][17]. Based on low-coherence interferometry, the imaging contrast of OCT is mainly provided by boundaries formed by regions with different optical properties. OCT is capable to offer structural information of biological tissues with no need for exogenous contrast agents. ODT is a branch of OCT, which is used to characterize the functional information, i.e. the blood flow velocity (mainly the flow velocity of the red blood cells), of biological samples with high resolution [10,11]. Blood flow is an important functional parameter for biological tissues, for example, various ocular diseases are associated with alterations in blood flow, such as glaucoma, age-related macular degeneration and diabetic retinopathy [18][19][20].
Photoacoustic microscopy (PAM) is an emerging microscopic imaging technology, which is capable of extracting optical absorption information in biological samples based on the photoacoustic effect. In PAM, the sample is irradiated by a pulsed laser with a pulse width down to a few nanoseconds, and a broadband ultrasonic wave is produced as result of rapid temperature rise as well as thermo-elastic expansion/relaxation process of the tissue [21]. The photoacoustic (PA) waves are detected by an ultrasound (US) transducer, and an optical absorption-based PA image can be constructed. Optical-resolution photoacoustic microscopy (OR-PAM), a well-known branch of PAM, relies on the optical focus to provide high lateral resolution at the cost of imaging depth [22]. OR-PAM has successfully demonstrated its great potential for structural and functional imaging in living animal models, such as vessel morphology, brain activity, and oxygen metabolism [8,[23][24][25][26].
CFM is an established technology for imaging the molecular features of biological specimen by mapping the distribution of fluorophores with known fluorescence spectrum [27]. Confocal fluorescence microscope employs a pinhole at the detector site to block the light coming from out-of-focus planes [28]. As a result, high resolution biological imaging is achieved with autofluorescence or exogenous fluorophore. CFM has been explored as an analytical instrument for studying various biochemical reactions in intact living cells [29][30][31].
Each individual imaging technology has its limitations due to the limited specific contrast mechanism used for imaging, and thus each imaging technology alone cannot provide all the information needed for biomedical research and disease diagnosis. For example, conventional OCT in the near infrared spectrum cannot offer molecular contrasts; PAM and CFM are capable of providing optical absorption and molecular information of a biological specimen in cellular level, however, they cannot provide contrasts of tissue structure as OCT can offer and they do not provide blood flow information. To overcome their limitations, multimodal optical microscopy imaging systems are developed to gather more comprehensive information from a biological specimen that was previously unavailable by using a single modal microscope [7,[32][33][34][35]. Many efforts have been made in developing various multimodal imaging systems to provide simultaneous imaging of the different properties of biological samples. For instance, a dual-modal imaging system, combining OCT and CFM, was built for simultaneous visualization of biochemical as well as morphological features of biological tissue [36][37][38][39]. A tri-modal imaging microscope, integrating PAM, two-photon, and second harmonic generation microscopy, has been successfully developed for simultaneous multimodal imaging of mouse ear. In another study, a hybrid optical system has been developed by combining optical-resolution PAM, two-photon excitation fluorescence (TPEF), second harmonic generation (SHG) and third harmonic generation (THG) microscopy for label-free imaging of a mouse ear and zebrafish larva ex-vivo [34,40].
For microscopic imaging, the lateral resolution and signal intensity highly rely on focusing of the objective lens. Dynamic focusing is necessary when the contour of the sample surface is uneven and has a range of depth-variation that exceeds the depth of focus of the objective lens. Several groups have reported dynamic focusing for PAM imaging by using contour scan with PAM [41] or ultrasound [42]. In comparison, our multimodal imaging system provides an unprecedented advantage that the OCT system can accomplish the contour scan with potentially much higher imaging speed and better depth resolution.
In this paper, we present a quad-modality microscopy imaging system that combined OCT, CFM, PAM and ODT to achieve simultaneous multimodal imaging together with dynamic focusing enabled by the integrated OCT system. The system can gather more comprehensive information of biological samples by imaging the complementary contrasts, including optical back-scattering, optical absorption, blood flow speed and fluorescence. The imaging system was tested on a phantom as well as biological tissues both in vivo and ex vivo.

Imaging system
A schematic of the experimental setup is shown in Fig. 1. The system employs two light sources, in the near infrared (NIR) and visible spectrum, for OCT/ODT and PAM/CFM, respectively. OCT/ODT works with a single broadband NIR light source (T-840 Broadlighter; Superlum Diodes Ltd, center wavelength 840 nm, FWHM bandwidth 100 nm). In PAM/CFM, a single Q-switched frequency-doubled Nd:YAG laser (SPOT-10-200-532, Elforlight Ltd, UK; output wavelength: 532 nm; maximum pulse energy: 20 μJ; pulse duration: 2ns; maximum pulse repetition rate: 30 kHz) is used as the illuminating light source. The NIR light power was measured to be 570 µW. The laser pulse energy was measured to be 300 nJ on the sample during the experiments, respectively. The pulsed laser light first passed through a neutral density filter (ND, optical density: 0.04), the surface reflection from which was detected by a photodiode (PD) whose output was used to trigger a digital delay generator for synchronization of the different imaging modalities. The laser light was attenuated by a set of neutral density filters before coupled into a single-mode optical fiber (SMF). After exiting the SMF, the light was collimated into a beam diameter of 2 mm, combined with the OCT probe light by a dichroic mirror (DM1, DMSP650 -Ø1" Shortpass, 650 nm Cutoff, Thorlabs), and focused onto the sample with an objective lens (Microspot focusing objectives, LMH-5X-532, EFL = 40mm, NA = 0.13, Thorlabs). For PAM signal detection, an unfocused needle ultrasonic transducer (40 MHz, bandwidth 80%, active element diameter 1.3 mm) was immersed in a water tank filled with distilled water. A window was opened at the bottom of the water tank and was sealed with a transparent polyethylene membrane. For imaging, a sample is placed under the window between the polyethylene membrane and a glass cube, and acoustically coupled with ultrasonic coupling gel. The induced PAM signals were amplified by a 54 dB preamplifier, and then digitized and directed to the PC by a digitizer (PCI-5154, National Instruments) with a sampling rate of 200 MS/s. A total of 1000 points corresponding to a sampling time of 5 µs were acquired with no signal averaging at each scanning position on the sample.
In the CFM subsystem, the back traveling fluorescence light excited by the pulsed laser in the sample passed through the dichroic mirror combining the OCT light and the 532 nm laser (DM1), reflected by another dichroic mirror (DM2, Short-pass, cutoff wavelength: 550 nm, Edmund Optics), passed through a short-pass filter (F1, FES0750, cutoff wavelength: 750 nm, Thorlabs) for blocking the back-reflected NIR light, and a long-pass filter (F2, FES0550, cutoff wavelength: 550 nm, Thorlabs) together with a laser rejection filter (F3,532 nm BP, OMEGA). It was then focused into a 25 μm pinhole by an achromatic doublet with a focal length of 30 mm (L3) and detected by a PMT module (PMM02, Thorlabs). The output signal of the PMT was digitized into the computer by another channel of the same digitizer. The sampling rate and the record length were the same as for the PAM subsystem.
In the OCT/ODT subsystem, the light from the broadband light source was coupled into a 3dB 2 × 2 fiber coupler and was split into the sample and reference arms. In the sample arm, the light exiting the fiber was collimated and combined with the PAM laser with the dichroic mirror DM1. The combined reflected light from the sample and reference arms was detected by a spectrometer. The spectrometer consisted of a 1200 lines/mm transmission grating, a multi-element imaging lens (f = 150 mm), and a line scan CCD camera (Aviiva-SM2-CL-2014, 2048 pixels with 14 μm pixel size operating in 12-bit mode, e2V) with an exposure time of 36µs.
Step-motor driven platforms (UniSlide, Velmex, Inc.) were used for scanning the sample and adjusting the focus of the objective lens. The sample together with the water tank was scanned in the X-and Y-directions by using a 2-axis platform. Each image was set to have 256 × 256 or 512 × 512 steps (X × Y) covering either 1.6 mm × 1.6 mm or 6.5 mm × 6.5 mm area, a step size of 6.4 µm or 12.8 µm. The entire image acquisition took about 20 or 80 minutes, respectively. The focus of the objective lens was adjusted by the Z-scan platform, which was controlled with the guidance of the OCT contour scan.
All the imaging subsystems were synchronized by the TTL output of the X-Y motor controller at each scanning step, which triggered the pulsed laser. The output of the photodiode (PD in Fig. 1) in response to each laser pulse triggered a multi-channel digital delay generator (DG645, Stanford Research Systems). One output channel of the delay generator triggered the digitizer for acquisition of the PAM and CFM signals. Another output channel triggered a function generator (AFG3021B, Tektronix), the output of which was used to trigger the image acquisition board for the CCD camera of the spectrometer. The function generator worked in a burst mode: with each trigger received it generated four pulses to drive the OCT system to acquire four depth scans with an A-line rate of 24 KHz. These four depth scans were used to calculate the phase shift caused by the blood flow, thus generated the ODT images.
A microscope formed by an imaging lens (L2 in Fig. 1, f = 200 mm) and the objective lens was used to guide the alignment of the sample for imaging. The microscope was coupled in the optical path by a pellicle beam splitter and the image was detected with a CMOS camera.

Imaging system performance
The axial resolution of the OCT subsystem was mainly determined by the center wavelength and bandwidth of the broadband light source. When imaging biological tissues, we only used 50 nm bandwidth and the measured depth resolution was 7.5 μm in air, which was a bit worse than the theoretical value of 6 μm. The axial resolution of the PAM subsystem was quantified by measuring the FWHM of the averaged PA pulse echoes envelope generated from a black tape. The PAM depth resolution was measured to be 31 μm.
Although the lateral resolution of the OCT and PAM is determined by the numerical aperture of the objective lens and the wavelength (theoretically: 2.5 μm and 3.8 μm for PAM and OCT, respectively), the lateral resolution of the current multimodal imaging system is limited by the step size of the mechanical scanner which is 6.4 μm for both OCT and PAM. The performance of the ODT has also been validated before with phantoms [43].

3 OCT-guided dynamic focusing
To achieve dynamic focusing, the relationship of the optical focus of the objective lens versus depth is established by imaging a black tape as the sample. The focus of the objective lens corresponds to the highest photoacoustic signal. When the highest photoacoustic signal is detected, the surface of the tape measured by OCT is defined as the depth location of the focus of the objective lens. This procedure resets the working condition of the imaging system. When imaging a biological subject, we first adjust the sample to get the highest photoacoustic signal at a reference point, which is selected with the guidance of the real-time video image measured by adjustments f difference of t follow the con Before im rapid OCT Bthe subject alo LabVIEW pro

4 Sample
The laser and g circuits. orescence orescence. uantitative spectrum of the 532 nm laser light before coupled into and after exiting the single-mode optical fiber. When the fiber length was 4.5 m, we observed that the peak wavelength was shifted from 532 nm ( Fig. 2(a)) to 545 nm ( Fig. 2(b)) due to Raman scattering in the single mode fiber. One solution to this issue is to add a 532 nm laser line filter at the exit of the optical fiber to filter out the shifted longer wavelengths. However, this lead to a drop in laser energy, in the 4.5 m fiber case from about 600 nJ to 200 nJ. Another solution is to shorten the optical fiber. When we shortened the fiber length from 4.5 m to 0.5 m, the nonlinear effect is significantly reduced. As shown in Fig. 2(c), in the spectrum of the laser light exiting the 0.5 m fiber, the wavelength shift can be neglected.

OCT-guided dynamic focusing: phantom study
The rectangular capillary tube was fixed on a 20 degree angled plate. To get the contour of the sample surface, the phantom was first scanned at each Y location with the OCT system along the X-axis, which consists of 256 A-lines covering a range of 1.6 mm. Figure 3(a) shows one OCT B-scan at the location marked with a yellow line in Fig. 3(e). The contour of the phantom was automatically generated by extracting the maximum of each OCT A-line in the B-scan image. The depth difference was calculated at each point on the X-axis in relative to the predefined depth of the maximum PAM signal. The depth difference was used to adjust the focus of the objective lens point by point to achieve dynamic focusing when the multimodal images were acquired. Figure 3 shows the effect of dynamic focusing on imaging the phantom. Figure 3 (c) and (d) show the PAM and CFM images before application of dynamic focusing. Figure 3 (e) and (f) shows the PAM and CFM images with dynamic focusing. We can see that after application of dynamic focusing, both the signal intensity and the sharpness of the PAM and CFM images are improved across the field of view. To further illustrate the improvement of image quality by the OCT-guided dynamic focusing, we draw the signal intensities of the PAM and CFM (Fig. 3 (g) and (h)) along the line shown in Fig. 3(c), (d), (e) and (f). In the PAM and CFM intensity curves the signal intensities at all the imaged points are significantly improved after the application of dynamic focusing except the points already in focus (the points in the dashed box).

OCT-guided dynamic focusing: ex vivo simultaneous PAM, CFM and OCT imaging
After successful test on the phantom, we imaged the retinal pigment epithelium (RPE) of a human cadaver eye with the multimodal imaging system to test the performance of OCTguided dynamic focusing on biological samples. Before data acquisition, the real-time OCT signal was used to guide moving the sample to the focal plane. The microscope was used to locate the region of interest (ROI). Before image acquisition, the same procedure was followed to extract the contour of the sample surface by using the OCT scan. Dynamic focusing was then applied, point by point, during image acquisition. The performance of the OCT-guided dynamic focusing is demonstrated in Fig. 4. The simultaneously acquired PAM and CFM images were composed of 512 × 512 scanning points covering an area of 6.5 mm × 6.5 mm. The PAM images shown are a maximum-amplitudeprojection (MAP) of the acquired 3D data. In the CFM images each pixel is an average of the acquired 1000 samples at each scanning point. The imaging time is about 80 minutes. Figure  4(a) and (b) show the PAM and CFM images of the sample without dynamic focusing. Figure  4(c) and (d) show the PAM and CFM images of the same area with dynamic focusing. Figure  4(f) and (g) show the intensity of the PAM and CFM signals along the line marked in the corresponding images. The unevenly distributed melanin and lipofuscin granules in the RPE cells give rise to the spike appearance of the PAM and CFM signals. We can clearly see the improvement of the signal intensity and the sharpness of both the PAM and the CFM images by applying dynamic focusing. Figure 4 demonstrates the complementary molecular contrasts provided by the multimodal imaging system. The signals in PAM and CFM were generated by melanin and lipofuscin in the RPE, respectively, which can be verified by the OCT cross-sectional image shown in Fig.  4(e), the location of which is marked in Fig. 4(a) by the yellow solid line. In the OCT image, we can clearly see that the dark regions in the PAM and CFM images correspond to the area deprived of RPE layer. Since both PAM and CFM were generated with the same group of photons, both images are intrinsically registered. The images of the two modalities look similar because both melanin and lipofuscin granules are all contained in the RPE cells and the resolution of the current system is not good enough to separate them.

In vivo simultaneous PAM, CFM, OCT and ODT imaging with OCT-guided dynamic focusing
We imaged a mouse ear in vivo to demonstrate the full potential of the multimodal imaging system with dynamic focusing. After sample preparation, the mouse was restrained in a homemade 20 degree angled plate, which was fixed on the Z-axis platform. By placing the ear at an angle, the blood vessels were not perpendicular to the OCT probe light thus guaranteed that the Doppler angle would not be 90°. The mouse ear was then attached to the imaging window of the water tank. Using the Z-axis stage, the sample was adjusted to the focal plane by monitoring the OCT signal in real time. We could visually identify the ROI by using the microscope camera. The procedure for extracting the contour of the sample by using the OCT B-scans and adjusting the focus of the objective lens to achieve dynamic focusing was the same as in the phantom and ex vivo experiments. The entire image acquisition took about 20 minutes covering an area of 1.6 mm × 1.6 mm. The size of the imaging area was limited by the time limit of the anesthesia method. Figure 5 shows the simultaneously acquired PAM, CFM, OCT and ODT images of the sample to demonstrate the full potential of the multimodal imaging technique with OCTguided dynamic focusing. Each imaging subsystem reveals significantly different features of the mouse ear due to their different contrast mechanisms. The PAM image in Fig. 5(a) clearly imaged the blood vessels because hemoglobin in the red blood cells in the vessels has high optical absorption at 532 nm. Figure 5(b) shows the OCT B-scan image at the location marked in Fig. 5(e), which revealed the tissue structures, including the epidermis, dermis and the articular cartilage. Figure 5(c) shows a cross-sectional ODT image indicating directional blood flows. Red and blue colors represent direction of blood flow corresponding to arteries and veins, respectively. Two vessels, one artery and one vein were captured. The artery and vein were identified due to the fact that veins have typically larger lumen with thinner wall, transporting more blood volume, while arteries are smaller in diameter with thicker wall in proportion to their lumen. In addition, arteries and veins typically appear as a pair in tissue. Depending on the imaging configuration and the scanning direction on the animal, we can also distinguish arteries from veins according to the direction of the blood flow: arteries transport high  Figure 5(d) shows the fluorescence of the sebaceous glands in the skin. Several different molecules, such as porphyrins, keratin and tryptophan, are the possible sources for autofluorescence in sebaceous glands [47,48]. Figure 5(e), shows a projection of the 3D OCT data on the X-Y plane [49]. The sebaceous glands appear dark in the projected OCT images due to the shadowing effect as shown in the OCT B-scans. Figure 5(f) illustrates the fused image merging simultaneously acquired PAM, CFM and OCT images in a single shot. In PAM images, the blood vessels appear as bright color due to the high optical absorption of hemoglobin at wavelength of 532 nm in blood generating strong ultrasonic signals. In contrast, the blood vessels appear as dark color in the projected OCT image due to the shadowing effect caused by the highly scattering properties of the blood cells.

Discussion
We have successfully implemented a novel OCT-guided dynamic focusing technique in an integrated multimodal imaging system. The system was tested by imaging phantom, human eye ex vivo, and mouse ear in vivo. Taking advantage of high depth resolution of OCT, the depth information of each scanning point, in reference to the maximum amplitude of PAM signal, was extracted and utilized for dynamic focusing by using a motorized Z-stage. Although the improvement of SNR and image sharpness may not be the same for PAM and CFM when the highest PA signal may not be located in the same plane as that of the highest fluorescence signal, dynamic focusing will make the SNR more uniform across the imaging area. The adjustments of the objective lens in the Z direction should be recorded and saved to a file, which will be needed for reconstructing the 3D CFM image when optical sectioning is performed. Upon development of this novel method, which maintains the sample at focus through the entire scanning, images with higher SNR can be acquired. This novel technique is suitable for imaging different samples with uneven surfaces.
Currently, the major limitation to the multimodal imaging system is the imaging speed, which is limited by the scanning speed of the step motors. To increase the imaging speed, a faster motor or different scanning mechanisms is needed, which is one of our focus in developing the system into more practical applications. Limited imaging depth of the OCT system, 2 mm for the current OCT system, may be another limitation for the dynamic focusing applications.
The digital delay generator plays a key role in synchronizing the imaging systems. The Aline rate of the OCT system is determined by the scanning motor while the A-line rate for ODT calculations is determined separately by the pulse repetition rate of the function generator. This arrangement assured that at each scanning position multiple depth-scans can be acquired, which decoupled the A-line rate for ODT from the scanning speed of the motor and made the ODT function possible. During the in vivo experiments, we found that the blood flow imaging was not always successful. We hypothesize that the pressure of the water tank on the sample may slow down the blood flow to the extent that exceeds the limit of the ODT measurement. We did not see capillaries in the PAM and ODT images of the mouse ear because ODT is not sensitive enough for capillary imaging due to the slow blood flow in capillaries. Since we imaged the lower part of the ear, there were possibly less capillaries to be captured around the arteries and veins in PAM image. This could also be due to the relatively low sensitivity of the unfocused needle transducer. A better transducer with higher sensitivity will be considered for the future work.
In the current CFM imaging the fluorophores providing the contrasts were lipofuscin in the RPE cells and porphyrins in the sebaceous glands, both are endogenous fluorophores. Lipofuscin is a complex lipid/protein aggregate of nondegradable end products from phagocytosis of shed photoreceptor outer segments in the RPE. It is the major source of fundus autofluorescence and it accumulates with age. The accumulation of lipofuscin in the RPE is believed to contribute to the pathogenesis of retinal diseases such as age-related macular degeneration (AMD) and Stargardt diseases [50][51][52]. Porphyrins in the sebaceous glands in the mouse ear are attached to individual hair follicles containing viscous fluid called sebum, which is responsible for moisture balance in the mouse skin [47]. By comparing the images of the sebaceous glands, we can see that they are highly optical absorbers in the NIR in comparison with the surrounding tissues because they cast shadows in the OCT crosssectional image and appear dark in the OCT projection images. Most of the absorbed photons by the sebaceous glands in the visible spectrum, however, may not be converted to heat because they do not contribute to photoacoustic signals. Since they showed up in the CFM image, we can conclude that the absorbed photons in the visible spectrum were mainly converted to fluorescence. These are just examples for demonstrating the potentials of the multimodal imaging system. For many biological applications, such as understanding the spatial heterogeneity of disease or monitoring of developmental process, the ability to simultaneously capture cellular and whole-organism level processes as well as their mutual interactions is necessary [53]. The proposed imaging system has potential to be used for in vivo retinal imaging, where the contrasts of all the different imaging modalities are provided by endogenous agents such as, melanin, lipofuscin, scattering, and blood flow. We currently only target the autofluorescence with endogenous fluorophores. The system is also suitable for imaging exogenous fluorophores when the fluorescence probe has correct excitation and emission spectrum. There is also a possibility to replace the CFM with multiphoton microscopy by using an ultrafast Ti:sapphire laser as the excitation light source. In this case, the OCT and multiphoton microscopy can share the same light source. This will be our next step for improving the capabilities of the imaging system. More applications will also be explored as the technologies are being further developed and matured.

Conclusion
In conclusion, we have successfully developed a multimodal imaging system with OCTguided dynamic focusing, which can simultaneously acquire OCT, ODT, PAM and CFM images with complementary contrasts. The novel OCT-guided dynamic focusing enabled imaging samples of uneven surface with improved SNR across the whole imaged area. The imaging system was validated by imaging phantom and biological samples ex vivo and in vivo. It is expected that further improvement of this optical microscopic instrument will find even broader applications for tissue imaging.

Disclosures
The authors declare that there are no conflicts of interest related to this article.