Towards real-time multispectral endoscopic imaging for cardiac lesion quality assessment

: Atrial fibrillation (Afib) can lead to life threatening conditions such as heart failure and stroke. During Afib treatment, clinicians aim to repress unusual electrical activity by electrically isolating the pulmonary veins (PV) from the left atrium (LA) using radiofrequency ablation. However, current clinical tools are limited in reliably assessing transmurality of the ablation lesions and detecting the presence of gaps within ablation lines, which can warrant repeat procedures. In this study, we developed an endoscopic multispectral reflectance imaging (eMSI) system for enhanced discrimination of tissue treatment at the PV junction. The system enables direct visualization of cardiac lesions through an endoscope at acquisition rates up to 25 Hz. Five narrowband, high-power LEDs were used to illuminate the sample (450, 530, 625, 810 and 940nm) and combinatory parameters were calculated based on their relative reflectance. A stitching algorithm was employed to generate large field-of-view, multispectral mosaics of the ablated PV junction from individual eMSI images. A total of 79 lesions from 15 swine hearts were imaged, ex vivo . Statistical analysis of the acquired five spectral data sets and ratiometric maps revealed significant differences between transmural lesions, non-transmural lesions around the venoatrial junctions, unablated posterior wall of left atrium tissue, and pulmonary vein (p < 0.0001). A pixel-based quadratic discriminant analysis classifier was applied to distinguish four tissue types: PV, untreated LA, non-transmural and transmural lesions. We demonstrate tissue type classification accuracies of 80.2% and 92.1% for non-transmural and transmural lesions, and 95.0% and 92.8% for PV and untreated LA sites, respectively. These findings showcase the potential of eMSI for lesion validation and may help to improve AFib treatment efficacy.

and depth of lesions. They indirectly monitor formation of ablation lesions through, drop in impedance, contact force, ablation time using force time integral from ablation catheter [16]. These measurements vary from procedure to procedure. Hence, direct assessment of lesion quality through endoscopic images may help improve distinguishing incomplete lesions and assessing ablation line continuity.
Several research groups have previously evaluated ablated cardiac tissue using experimental optical methods. Fiber based optical coherence tomography (OCT) enables visualization up to 1-mm imaging depth of treated myocardial tissue [17,18]. OCT with and without polarization contrast reliably distinguished ablation lesions and untreated tissue [18][19][20][21]. Recently, hyperspectral autofluorescence imaging was applied to analyze the reflected light from the endocardial and epicardial surface of left and right atrium [22][23][24][25]. Lesion contours were outlined using spectral changes under UV illumination. In Ahmed et al., visually guided RF ablation catheters directly showed discoloration of target tissue in vivo in patients with drug resistant CTI-dependent atrial flutter discoloration of the tissue [26]. These initial experiments laid the foundation of adapting endoscopic imaging for use during endocardial ablation procedures, in vivo. Other methods for evaluation of cardiac ablation transmurality include MRI, intracardiac ultrasound and endoscopic visualization. Several groups demonstrated real-time tracking of catheters using MRI [27][28][29]. This allows postprocedural visualization of lesion formation. Also, high frequency ultrasound was integrated into a catheter for real time analysis [30]. Alternatively, diffuse reflectance spectroscopy was studied to evaluate lesion quality [31][32][33]. They compare relative reflection spectra from changes in tissue composition. Since longer wavelengths penetrate deeper into the tissue, near-infrared spectroscopy can overcome depth limitations seen in OCT. Our group has previously proposed a method of real-time processing of wavelengths near the infrared range to reveal differences in ablated and unablated tissue, with estimated lesion depths up to 4 mm [32,33]. lipid . . a obtained from [34]; . b obtained from [35]; . c obtained from [36]. ; . d obtained from [37].
Optical spectroscopy has been widely utilized for characterization of pathological tissue on the basis that useful physiological information is reflected within tissue optical properties [34]. Figure 1 shows chromophore spectra of seven major cardiac chromophores from [34][35][36]. Absorption spectra of tissue can be expressed in a weighted sum [36,38,39]. The weighted sum of absorption spectra shifts after RF treatment since it changes tissue morphology and distribution of chromophore concentrations. Through inverse Monte Carlo simulations, absorption and scattering effects can be separated to reveal the composition of cardiac tissue [33,36,39]. RF-induced absorption changes have been linked to hemoglobin and myoglobin transforming into hemichromes, methemoglobin (metHb) and metmyoglobin (MMb) [40,41]. Reduced scattering spectra in ablated tissue is also elevated along the entire visible and near infrared range. Demos et al. analyzed spectral information of scattered light to show that ratio of spectral intensity at 910 nm over that at 710 nm monotonically changes with depth in bovine tissue [31]. While these methods have shown promise in lesion assessment, an endoscopic method may be better suited for identification of non-transmural lesions and gaps in linear ablation lines, a well-reported culprit for arrhythmia recurrence [16]. In this work, we present the development of an endoscopic multispectral reflectance imaging system (eMSI) for discriminating fully transmural lesions from normal and nontransmural ablation of the left atria porcine tissues. A pixel-based classification model is trained and validated in fresh swine samples ablated near the PV-left atrial wall junction.

System design
The eMSI system mainly consists of a camera, endoscope, LED sources and a microcontroller. It uses a CMOS Camera (Hamamatsu Flash4.0LT, Hamamatsu City, Japan) and a USB3.0 interface board connection. A flexible fiber endoscope (Myriad Fiber Imaging Tech Inc, Dudley, Massachusetts) with 10,000 fibers and a viewing angle of 70° is connected to the camera. The camera can acquire images up to 2048 x 2048 pixels. Our data sets were binned by four to increase signal to noise ratio. When the camera is connected to the viewing port of the endoscope, the effective number of pixels are reduced to 172 x 172 pixels. The software instructs the camera to capture through USB connection. When the camera is directed to integrate, TTL exposure integration time signal is generated. This controls the timing for each image acquisition. The LEDs are controlled by the TTL signal sent to the microcontroller (Arduino UNO). At the falling edge of TTL signal, the microcontroller switches the LED to the next in the sequence.
LEDs are illuminated through the lighting port using a custom designed lens assembly. A plano-convex lens (LA1131-A, Thorlabs, NJ) was placed in front of collimated beam from the light source. The refracted beam is collimated back into the port of endoscope by a biconvex lens (LB1494-A, Thorlabs, NJ) at 4.3cm away from the plano-convex lens. This lens assembly was designed through Zemax (Zemax LLC, Kirkland, WA) to maximize the throughput beam into the port of endoscope. Cagecubes with dichroic shortpass filters are used to align the beam paths to enable multispectral illumination. LED strobe pattern, TTL sequencing and image acquisition is controlled by a MATLAB graphical user interface. Fig. 2. A schematic diagram of endoscopic multispectral imaging system. The system is optically enclosed to block light interference during the experiment. A) shows reference data set of spectrally flat reflectance surface. B) displays raw data of ablated tissue with three lesions taken with the system. Two lesions are shown with light concentrated on the center but are hard to locate exactly all three lesions. In C), normalized image of swine left atrial tissue with three lesions is shown. D) shows distortion corrected image. Since it is difficult to see the correction, a sample grid is shown in E) and F). Image magnification decreases with distance from the center.
Strong differences in biomolecular composition and ultrastructure has been shown in ablated myocardium [36,42]. Therefore, we selected wavelengths that showed largest variations in absorption while reducing scattering. For our system, wavelengths at 450 nm, 530 nm, 625 nm, 810 nm, and 940 nm were chosen. From Flock et al., we estimated penetration depth of each wavelengths selected in our system [43]. Approximate optical properties of a μ and s μ' were found from Swartling et al. s μ' at 940 nm was estimated by interpolating and fitting the graph to a polynomial. a μ at 940 nm was approximated by linearly decreased value from 900nm. 450nm was omitted since penetration depth is too small. As seen in Table 1, the longest wavelength penetrates approximately 3.42mm. Since the sampling depth of 450 nm is limited, the 450 nm channel is primary used to generate RGB images. Swartling et al. showed that the largest absorption coefficient ( a μ ) differences between treated and untreated tissue was 40% and occurred near 630 nm and 490 nm bands. Additionally, the second largest a μ difference was 20% centered at around 520 nm. Based on these differences, wavelength channels at 625 nm and 530 nm were selected to yield contrast between treated and untreated tissue, while constituting the red and green channels of the RGB image. Furthermore, the difference in reduced scattering coefficients show a constant difference in near-infrared range [36,42]. Also, in Fig. 1, 520 nm shows peaks of dominant chromophores, such as 2 2 , , , HbO MbO Mb Hb and 630 nm exhibits second peak of MMb . Importance of these parameters are discussed in the introduction. Furthermore, approximately near 810nm is an isosbestic point for globin derivatives 2 2 ( , , , HbO MbO Mb Hb ) and was selected to correspond roughly to sum globin contributions. Finally, 940 nm was selected due to a scattering dominated measurement as an alternative and additional contrast for distinguishing ablated tissue. From the acquired data, we analyzed the revealed features by comparing these major cardiac chromophores in Fig. 1.

Sample preparation
Total of 15 swine hearts from Green Village Packing Company (Green Village, New Jersey) were utilized. All the experiments were completed within 24 hours of sacrifice. In addition, a human heart was received from National Disease Research Interchange (NDRI): a donor with no known cardiovascular disease (66, Female). The heart was received within 24 hours of donor's death and imaged within 48 hours of donor's death. The purpose of the human heart is to show a proof-of-concept demonstration to showcase a potential for future clinical translation.
The dissected left atrium (LA) with PV intact is placed in a temperature-maintained phosphate buffered saline (PBS) bath at 37 s°C with a pump to generate circulating and pulsatile flow. Each pulmonary vein was transected longitudinally and flattened to expose the veno-atrial junction. A set of non-irrigated lesions were delivered on the endocardial wall near the vein junction using a commercial 3.5mm radio-frequency catheter (Celsius Thermocool D curve uni-directional TC) and generator (Stockert 70, Biosense Webster, Diamond Bar, CA). To generate non-transmural and transmural lesions, the energy delivery duration was set between 15 and 60 seconds with constant target power set at 15W. A linear set of lesions with intentional gaps were created along the pulmonary vein. Bioelectrical impedance and delivered power were recorded for each lesion. For human heart sample, left atrium was ablated near the pulmonary vein using an irrigated catheter with normal saline under the same power setting as swine samples. The flow rate was at 30 ml/min.
The sample was placed 3.9 cm from the tip of the endoscope, as shown in Fig. 3. At this working distance, the diameter of field of view is 16.5 mm. After data acquisition, the center of each lesion was dissected in half. For swine hearts, one half was submerged in 1% triphenyltetrazolium chloride (TTC) vital stain for 40 minutes. TTC stains normal tissue in bright red and reveals damaged tissue, shown in Fig. 3(D). For histopathology assessment, the counter half of swine and all human samples were preserved in 10% formalin for 24 hours then transferred to 70% ethanol. They were stained with Hematoxylin and eosin and Masson's Trichrome technique on adjacent 5 m μ sections. The samples were digitized using a digital microscope (Leica, Microsystems) and reviewed by a pathologist.

Data acquisition and processing
A reference datacube was taken on a spectrally flat 99% reflectance standard (AS-01161, Labsphere, NH) to account for system response and spatial distribution of light. Using the reflectance surface, exposure settings for each illumination is determined. The LED power is also tuned to avoid saturations. The peak quantum efficiency of CMOS camera is 83% at 600 nm and continues to decrease as wavelength increases. At 940 nm, quantum efficiency is 14%. Exposures are set to collect maximum reflectance signal without saturating the reference surface. Each image was predominantly aberrated by barrel distortion. The correction was applied to the distorted images as seen in Fig. 2(E) and 2(F) using digital image processing [44]. As magnification decreases with distance from the optical axis in barrel distortion, normalized scale is applied for distance away from center. Tissue samples were submerged in phosphate buffered saline (PBS) to reduce surface reflection. Although the system can acquire data up to 25Hz, the acquisition time was set at 6Hz for all experiments to maximize signal collection. After barrel distortion correction, images were overlaid and stitched together to create large field-of-view mosaics of the ablated PV region. First, circular endoscopic images are squarely cropped. Then, the translated lesion location X and Y points are located and blended together using a previously published multiband blending algorithm [45,46]. Multiband blending is a technique that divides the original image into multiple bands with different weights. Overlapping bands are smoothed by linear combination of different weights. A motorized stage was placed underneath the sample container. The stage was translated in 5mm increments in both directions. The surface area of the stitched samples ranged from 75 to 375 2 cm .The stitched images are again checked for any existing saturations from surface reflectance. Saturated pixels were omitted during data processing.

Classification of tissue types
From five spectral channels, our goal was to categorize tissue types and detect undertreated sites, including lesion gaps. In this work, quadratic discriminant analysis (QDA) was used to classify each pixel values into four classes: normal tissue, pulmonary vein, non-transmural and transmural lesions. Since the relative reflectance may have low variances between classes, quadratic decision boundary allowed less strict feature covariance matrices. The performance of the classifier was assessed using leave-one-out cross-validation (LOOCV) within MATLAB.
To label classes, RGB colored images were created by compositing 3 wavelength channels: 450 nm, 530 nm, 625 nm. Transmural and non-transmural ablated left atrial tissue, non-ablated left atrial tissue and pulmonary vein were manually segmented using the RGB reference image. Lesions were segmented in elliptical shapes since most ablations performed with radio-frequency catheter are oval shaped [47]. Normal tissue and pulmonary veins are parted in similar size to create consistent sample sets. Areas that were not part of atrial tissue were manually segmented out. The main data processing procedure is shown in Fig. 4.

Statistica
Analysis of va detect differe significant. T California).

Statistical analysis of various tissue types and classification using quadratic discriminant analysis
Statistical comparisons of transmural (T), non-transmural (NT), normal (N) tissue and pulmonary vein (PV) in each wavelength channels are shown in Fig. 6. To reduce data redundancy, the mean values of segmented area were used to compare tissue types. In 450nm channel, PV showed significant differences between T, NT and N. In 530nm, only N, NT and T, NT pairs weren't significant. 625 nm channel had the fewest statistically significant pairs. Both 810 nm and 940 nm channels showed identical significant pairs (T, PV and T, N pairs). In all wavelength channels, statistical differences in transmural and non-transmural data are not shown. The depth of treated tissue depends on myocardium thickness. Same depth may be transmural in one sample, but non-transmural in a thicker sample. Therefore, statistical analysis hints raw spectral data may be insufficient to differentiate between non-transmural and transmural. The purp possibility of identify the d accuracy of n wavelength re that at 940 nm in Fig. 7  LOI is shown in our group's previous work, where we established lesion optical indices ( LOI ) to define spectral differences ablated and unablated tissue and have shown real-time tissue spectra classification and lesion size estimation [33]. In Singh-Moon et al., LOI 1 is described as the quotient given by spectral intensity at 964 nm divided by that at 616 nm [33]. Computing the ratio is a common practice. Demos et al. reported that spectral ratio method led to a parameter that monotonically changed with depth of RF ablation lesions on bovine hearts. In Fig. 7, ratiometric maps of 1 LOI reveal all lesions along with pulmonary vein. Transmural lesions generally showed a distinctive darkened core surrounded by a bright halo, whereas non-transmural lesions are uniformly enhanced. After training the QDA classifier with five spectral data along with two lesion optical indices, the classifier distinguished normal tissue (92.8%), pulmonary vein (95.9%), transmural (92.0%), and non-transmural (80.2%) with higher accuracy seen in Table 3. Most importantly, we saw significant improvements in non-transmural lesion classification accuracy by 22%. Also, misclassifications of non-transmural lesions to transmural lesions and normal tissue were reduced by 9% and 17%, respectively.  In Fig. 8, labeled tissue types of two LA samples are reconstructed. The algorithm detects gaps between two lesions in Fig. 8(G). Lesion gaps can potentially provide pathways for abnormal electrical activities. Therefore, identifying gaps is crucial for improving procedure success rate. Probability distributions of PV and N class in Fig. 8(B) and Fig. 8(C) display high confidence in the classified area. Also, normal tissue around the ablated regions have lower probability shown in Fig. 8(C). Transmural and non-transmural lesions are clearly identified in Fig. 8(D) and 8(E), respectively. Probability distribution of non-transmural lesions shows that lesion peripheries have a high likelihood of being non-transmural probability. This was confirmed through histological assessment.
An LA sample with transmural and non-transmural lesions is shown in Fig. 8(G). Figures  8(H-L) highlights the differences in non-transmural and transmural lesions. Generally, it was observed that permanently injured tissue exhibits a distinct core. However, non-transmural lesion also shows similar attributes in Fig. 8(G). This particular sample had a thicker myocardium layer, which was shown in histology. Although myocardium thickness varies in each sample, our classifier successfully identified this location as non-transmural lesion based on the provided spectral information.
In this work, all swine lesions were ablated using non-irrigated catheters. However, most RF ablations are now performed using irrigated catheters. As a preliminary data, a human heart from National Disease Research Interchange (NDRI) was imaged under our system. In Fig. 9, a heart of donor with no previous heart disease was imaged. The left atrium was ablated near the pulmonary vein using saline irrigated catheters. In Fig. 9(A), lesions are difficult to locate using only a camera image because of thick endocardium within the human left atrium. The contrast between ablated and unablated is low compared to swine samples. Spectral reflectance images of the PV region is shown in Fig. 10(B-D). In Fig. 10(G), 1 LOI clearly unveiled the location of irrigated lesions, which were hard to visualize in the camera image ( Fig. 9(A)). In the human sample, there is less distinction between the thick endocardium and myocardium as previously seen in swine samples in 2 LOI . Fig. 9. A) shows a human left atrium captured with digital camera (Nikon). Three irrigated transmural lesions are generated using RF catheter with saline port. B) represents histology of transmural lesion. Ablated areas are more purple relative to normal tissue. C) shows histology of normal tissue. Thicker endocardium is shown in human sample. Fig. 10. A) is an RGB composite image of human LA tissue sample with three irrigated lesions. B-F are the five spectral channels collected through our optical system. In all wavelength channels, irrigated lesions are difficult to locate without LOIs. In G), 1 LOI parameter highlights the three irrigated lesions that weren't visible within individual wavelength channels shown in B-F. Also, gap between two lesions is clearly viewed. In H), 2 LOI map bears little to no lesion contrast.

Discussion
In this work, we show that direct visualization of cardiac lesions using an endoscope can effectively distinguish lesions, normal tissue and pulmonary vein with high accuracy (Table  3). By choosing specific wavelength channels for our system, collected reflectance spectra displayed significant spectral features, which classified transmural lesions with accuracy of 93.67%.
Previously, ex vivo rat heart samples were excited with UV illumination and imaged through hyperspectral benchtop system [22][23][24]. The sampling depth of UV light is limiting since lower wavelengths cannot penetrate more than a millimeter into the tissue. Therefore, deep lesions are difficult to analyze using surface reflectance. By using both visible and nearinfrared wavelength channels, we demonstrated that distinctive features were revealed in different tissue types. From our relative reflectance measurements, the features are extracted from complex interaction between absorption and scattering properties. Both absorbing chromophores and increasing size of scattering structures influence the data collected through our optical system.
Reflectance collected by our system depends on the interplay between scattering, absorption, and our optical configuration. Prior studies report both an increase in tissue scattering as well as absorption in response to thermal treatment. In Celik et al, an iron assay was used to measure the concentration of ferric and ferrous iron in the lesion core and in untreated cardiac samples [40]. The authors detected a substantial increase in ferric iron which they attributed to the formation of MetHb and MMb species. Optically, this transition would cause an increase in spectral regions where Met derivatives are more absorbing then their ferrous counterparts. Because this change is more expressed within the core and less on the periphery, this leads to a darker appearance in the center of transmural lesions. In nontransmural lesions, this effect was similarly observed in some cases and not observed in more superficially treated cases. In Fig. 5, peripheral regions surrounding ablated tissue increased in reflectance compared to normal tissue in all five spectral channels. Shorter wavelength maps in Fig. 5B and 5C showed decreased reflectance in the center of ablated tissue. However, wavelengths near the infrared showed subtle contrast between the lesion core and lesion periphery as scattering remains the dominant transport parameter. In both Table 2 and  Table 3, the classifier parted normal, pulmonary vein, and transmural lesions with high precision. However, with the addition of two ratiometric indices (LOIs), classification accuracy of non-transmural lesions increased significantly. 1 LOI maps (Fig. 7) help to differentiate lesions from other tissue types. On the other hand, 2 LOI reduces lesion contrast and highlights complex myocardial and endocardial features in the left atrium. 1 LOI ratiometric maps show contrast between ablated and normal tissue. In these maps, heavily treated tissues generally displayed lower values in the core. This phenomenon may be attributed to increased absorption in the ablated tissue divided by increased scattering in both ablated and normal tissue. Number of groups have observed changes in the underlying physiological and molecular properties during ablation therapy. Biomolecular properties, such as absorption and scattering coefficients, are complex parameters that interact with light. In Swartling et al., absorption spectra of ablated tissue was reported higher in 500~600 nm range and also in at 635 nm [36]. This phenomenon can be explained through increase in MMb concentration. Extracted concentration of MMb in normal tissue increased from 32 3 μmol dm − to 137 3 μmol dm − after ablation [36]. It was also found that there was an increase of Fe(III) concentration in ablated tissue compared to normal tissue, which is related to Mb and MMb presence [40]. In our setup, scattering is likely to increase reflectance proportionally. In Thomsen et al, reduced scattering increased in heated tissue compared to normal tissue under yellow (594 nm) and red (634 nm) light. It was also reported that the reduced scattering of ablated tissue was higher than non-ablated tissue throughout the visible and near infrared regions [36,38,49].
Given the small sample size of non-transmural lesions (n = 15) and the variation in reflectance observed within the normal tissue class (transparent vs opaque endocardium), classification accuracy may further improve by increasing the number of non-transmural lesions. Additionally, since we generalize ablated area as non-transmural or transmural based on a single lesion cross-section, histology of multiple lesion cross-sections may help to more accurately assess treatment severity.
By limiting the number of wavelengths, our system can acquire spectral datacubes at 6 Hz and can potentially acquire up to 25 Hz with lower exposure settings. Although a full bandwidth of spectral data may provide more information, the added acquisition time is not feasible for in vivo implementation. Since heart rate can vary widely, quick acquisition is crucial to capture motionless data sets in vivo. Yet, the current data acquisition speed is prone to severe motion artifacts in vivo. One way to improve acquisition speed closer to real-time operation is to use a faster microcontroller. To accomplish this, an optimization of exposure time and illumination power will need to be conducted to ensure adequate signal intensity.
Currently, the majority of data were collected from swine heart samples. Although normal swine samples show similar anatomy to human hearts, left atrial samples from human donors with prior cardiovascular disease show complex fibrosis and increased endocardial thickness [20,46]. Also, relative reflectance measurements from swine samples have different values compared to those from human. When human samples are classified using the same classifier trained on swine samples, most normal tissue sites were thus labeled as pulmonary vein, likely due to the thicker endocardium. Trichrome histology of the human sample also confirmed a thicker endocardium compared to swine samples. Nonetheless, in Fig. 10G, 1 LOI showed promise in distinguishing lesions from the rest in human atrial sample. Although human and porcine hearts are inherently different, the same lesion optical index displayed significant contrast by decreasing intensity values in treated areas. Further studies are aimed at collecting more human data to characterize the thick endocardium, fibrosis, and adipose sites. In vivo translation will require displacement of blood between the tissue surface and endoscopic fibers. The technical feasibilities for these modifications have been demonstrated in endoscopic systems designed to guide laser ablation therapy utilizing a balloon for blood displacement [10,50]. A model based on ex vivo human samples will help build a platform for initial in vivo deployment.

Conclusion
In closing, an optical system and methods to discern endocardial lesions were presented based on endoscopic multispectral imaging. A QDA classifier was developed which showed strong accuracies in distinguishing between four cardiac tissue types (normal tissue, PV, nontransmural and transmural lesions) given inputs of wavelength-specific reflectance and ratiometric maps. Accurate assessment of transmural lesions could aid in uncovering areas that require further treatment or targeting discontinuities in ablation lines to improve the procedural efficacy of cardiac ablation therapy.

Disclosures
The authors declare that there are no conflicts of interest related to this article.