Digital histological staining of tissue slide images from optical coherence microscopy

The convergence of staining-free optical imaging and digital staining technologies has become a central focus in digital pathology, presenting significant advantages in streamlining specimen preparation and expediting the rapid acquisition of histopathological information. Despite the inherent merits of optical coherence microscopy (OCM) as a staining-free technique, its widespread application in observing histopathological slides has been constrained. This study introduces a novel approach by combining wide-field OCM with digital staining technology for the imaging of histopathological slides. Through the optimization of the histology slide production process satisfying the ground growth for digital staining as well as pronounced contrast for OCM imaging, successful imaging of various mouse tissues was achieved. Comparative analyses with conventional staining-based bright field images were executed to evaluate the proposed methodology's efficacy. Moreover, the study investigates the generalization of digital staining color appearance to ensure consistent histopathology, considering tissue-specific and thickness-dependent variations.


Introduction
The observation of thin tissues stained in histopathology through bright-field microscopy (BFM) is a routine as well as an indispensable procedure.The advent of slide scanners capable of automatically imaging massive slides and artificial intelligence (AI) technologies that can automatically detect disease areas in optical images is transforming the field of histopathology from analog to digital [1].Recently, the integration of staining-free optical imaging and AI has accelerated digital histopathology by simplifying the specimen preparation process and improving the efficiency of histopathological diagnosis.Staining-free optical imaging offers the unique advantage of bypassing the cumbersome staining process during specimen preparation and distinguishing the structure of tissue slides based solely on optical contrast.Previous reports indicate that microscope images using auto-fluorescence and phase compete favorably with traditional images obtained from stained slides using BFM [2,3].Furthermore, the implementation of digital staining through AI allows for the digital reproduction of various dyes commonly used in routine practice, not only for representative the hematoxylin and eosin (H&E) staining but also for special stains such as Masson's trichrome or Jones silver stain [4][5][6][7].Digital staining provides additional benefits, allowing correction for variables that may vary with user expertise, changes in the thickness of well-prepared sections, and dye concentrations, ensuring consistent histopathological information.Therefore, the process of digital staining images obtained through staining-free optical imaging is anticipated to become a crucial technology in future histopathology.
Optical coherence microscopy (OCM) is emerging as another promising tool in histopathology, enabling real-time, non-contact imaging of tissue morphology through scattering contrast without staining or labeling [8,9].OCM, utilizing near-infrared (NIR) light wavelengths, allows deep tissue imaging, providing a distinct advantage over conventional optical microscopes commonly used in histopathology [10][11][12].Optical coherence tomography (OCT), the predecessor to OCM, widely used in fields like ophthalmology as 'optical biopsy' has already demonstrated significant potential for application in histopathology [13][14][15].However, its utilization in histopathology remains relatively limited, especially in studies incorporating digital staining, which are still in the early stages.This is primarily due to the difficulty in implementing pairs of OCM images, typically used in supervised learning for digital staining, along with images of stained tissues.In the process of creating histopathological slides, it is typical to cut paraffin-embedded tissues into 3-5 µm thick sections using a microtome, followed by staining.However, OCM imaging of these thin sections on slide glass faces challenges in obtaining clear images for pathological assessment due to weak backscattering signals within the tissue and the pronounced reflection from the glass surface.In contrast, tissue sections with sufficient scattering signals produced by the microtome have thickness limitations, leading to variations in staining concentration.As a result, for the application of the technology combining OCM and digital staining in histopathology, it is essential to consider not only imaging and AI technologies but also a new tissue sectioning protocol.

Sample preparation
Our study conducted optimization experiments targeting thick tissue sections and staining methods specifically tailored for OCM imaging.Various tissues, including the intestine, heart, lung, and skin, were obtained from 6-8 weeks old C57BL/6 mice.The mice were initially anesthetized with 2% isoflurane gas in a chamber and then subjected to perfusion with phosphate-buffered saline (PBS) and 4% paraformaldehyde (PFA).Animal procedures described here were approved by the Ulsan National Institute of Science and Technology (UNISTIACUC-22-54) and carried out in accordance with Institutional Animal Care and Use Committee standards.Figure 1 depicts OCM images and H&E stained BFM images of skin tissue at three different thicknesses: 10, 30, and 50 µm.It is observed that as the tissue thickness increases, the OCM image has good visibility of identify the structure of tissue, but the staining appears significantly darker [16].For more pursuable evaluation, we used root-mean-square contrast (RMS) as a metric to compare the contrast of OCM images.According to visual inspection, there was a tendency for thicker tissues to have higher RMS, and this trend was consistently observed in different types of tissues.Tissues with low scattering coefficients, like the lung, faced challenges in distinguishing structures in 30 µm-thick sections due to light reflection from the slide glass.Consequently, the optimal thickness for consistent OCM imaging across all tissues under the given sectioning conditions was determined to be 50 µm.Tissues are commonly sectioned with a microtome at a thickness of 30 µm or less, but we verified that a thickness of 50 µm is also feasible for all tissues.However, thicker sections showed an increased concentration of staining, as observed in Fig. 1.Furthermore, to assess the stain intensity for each stain component in BFM images, we solved an optimization problem that has the stain color appearance matrix and stain density map as variables based on the sparse non-negative matrix factorization (SNMF) [17].The stain color appearance matrix was assumed to be the same for all thicknesses, and only the stain density changed depending on the thickness.Subsequently, each stain concentration map was averaged and quantified.Analysis revealed relatively stable hematoxylin concentration across different thicknesses, while eosin concentration showed sensitivity.Considering the equilibrium between staining quality and OCM scattering contrast concerning tissue thickness, it was noted that consistent acquisition of paired OCM images and stained images occurred only in thick tissues.

Tissue slide imaging
For the digital histological staining in OCM, various tissues including intestine, heart, lung, and skin were extracted from mice, sectioned to a thickness of 50 µm, and then stained to create histology slides.The prepared H&E tissue slides were sequentially imaged for the acquisition of image pairs from two imaging modalities using the home-built NIR spectral-domain OCM system and the commercial BFM (Olympus BX41).In most digital staining studies, imaging is typically conducted using visible light optical microscopy before staining, followed by the staining process [4][5][6][7].OCM, on the other hand, utilizes the scattering contrast of NIR light, allowing freedom from light absorption caused by hematoxylin and eosin dyes [18].This imparts a unique advantage in minimizing specimen alterations that may occur during the staining process after imaging.
Our OCM system was specifically designed for wide-field imaging in histopathology, allowing for multi-scale analysis of mouse tissue, covering both the micro-scale biological components and the meso-scale geometric anatomy.The OCM system has a broadband light source with 845 nm central wavelength and 60 nm bandwidth.It employs a 2-axis Galvano scanner to produce the raster scan-based 3D image.X-axis scanner captures depth information through the interference of reflected light, forming 2D images.Completing the scanning process along the y-axis then constructs a comprehensive 3D image with dimensions of 0.3 mm x 0.3 mm x 0.2 mm (460 × 460 × 120 pixels).The image acquisition time for a single volumetric dataset was about 24 seconds.For the wide-field OCM image, the specimen underwent imaging through the mosaic technique with motorized stages [19].Following the acquisition of a single OCM volume dataset, the motorized stage moved the sample to the next programmed position, ensuring a 20% overlap with adjacent images to cover the entire tissue on the slide.Through large-scale image processing, we successfully connected these mosaic volumes to create a comprehensive OCM volume.
The en-face OCM images to be used alongside BFM images of H&E slides were extracted through additional processes from the 3D volume.Since the focal plane of OCM and BFM imaging varies, by closely monitoring digitally sectioned en-face OCM images in the depth dimension, we carefully identified OCM images with the highest morphological similarity to their corresponding BFM images, selecting them as matching pairs.In practice, we could find matched OCM image pairs that corresponded to BFM images more frequently from the surface to 20 µm in depth of OCM images.The training image set was acquired using the same magnification objective lens with a consistent focal length in both bright-field microscopy (Olympus UPLXAPO, 20X/0.75NA) and OCM (Thorlabs MY20X-824, 20X/0.45NA).

Digital histological staining with generative model
Despite successive data acquisition, there are inevitable scale mismatches, spatial misalignment, and morphological distortion between BFM and OCM images.To ensure effective model training, it is crucial to prepare a dataset with spatially aligned image pairs from different modalities, preserving detailed structural information.To address global misalignment, a projective transformation was applied to the WSI using the homography matrix calculated by the accelerated-KAZE (AKAZE) algorithm [20] between image pairs.The globally aligned image pairs were then divided into local tiles with a 50% overlap, and local corrections were performed at the tile level using phase cross-correlation to calculate translational offsets [21].To reduce bias caused by an excessive amount of unnecessary background in the dataset, tiles containing over 90% background were removed.Following local correction, empty pixels may appear at the borders of the corrected H&E image.To address this, we trimmed 8 pixels from each side, finalizing the size of each local tile at 1520 × 1520.
The training and validation datasets were created by cropping spatially aligned local tiles into paired small patches of 512 × 512 pixels with a sliding interval of 256 pixels.To enlarge the dataset, we incorporated rotations at four angles (0, 90, 180, and 270 degrees), introduced random Gaussian noise, and adjusted the brightness and contrast for each OCM patch.The augmented dataset was divided into training (80%) and validation (20%) sets.The entire dataset construction process is outlined in Fig. 2. Detailed information of dataset is summarized in Table 1.During training, random flipping and affine transformations were applied to incoming OCM patches to increase the diversity of the training set.However, this online augmentation was not applied during the validation stage to ensure a fair assessment of validation loss.To simulate the pseudo-staining of OCM images to resemble H&E stained samples using a generative approach, we designed model architectures based on GAN, as depicted in Fig. 3.The generator is composed of encoding, residual, and decoding blocks.The encoding blocks extract representative features from the given OCM input, the residual blocks progressively extract more complex features, and the decoding blocks subsequently reconstruct the output to mimic the desired H&E appearance.For the discriminator, we utilized the PatchGAN approach [22], allowing the evaluation of pseudo-staining results with local details compared to the corresponding real H&E stained samples.We acquired pairs of OCM and BFM images of H&E stained tissues through an extensive registration process.However, achieving precise pixel-level alignment posed a challenge, given the utilization of distinct imaging systems and the optical disparities between imaging modalities.To address these subtle misalignments in the dataset, we incorporated a multi-scale loss into the objective function for network training.The generator's loss, described by Eq. ( 1), combines the multi-scale loss, which compares scale-invariant features between the generated output and the target at down-sampled scales [23], and the adversarial loss, utilizing mean-squared error for stable learning [24].
In Eq. ( 1), D and G denote the discriminator and generator, respectively.x, and y are the inputs and targets.In multi-scale terms, i indicates a scale that the length of the image varies 512, 256, 128, and 64 pixels by applying down sampling.F represents the low-pass filter operation by 4 times the Gaussian smoothing.ω is the weight of each scale that is fixed as 25 for four different resolutions.The discriminator loss is divided by 2 for a balanced training process with the generator.
Training begins with the pre-training of the generator over 10 epochs using only the multi-scale loss, followed by adversarial learning.We utilize the Adam optimizer with an initial learning rate of 2.0 × 10 −4 , decreasing linearly after 100 epochs.The batch size is set at 10.The best model, determined by validation loss, is chosen from the networks saved at each epoch.Tissue-specific training is conducted, and the trained networks are employed to translate OCM into H&E color images.

Results
Figure 4 illustrates the digital staining outcomes of OCM applied to mouse lung, heart, intestine, and skin slides.Each tissue underwent sequential processes of sectioning and staining with hematoxylin and eosin dye, followed by imaging with OCM and BFM.The digitally stained images clearly delineated morphological features in each tissue such as alveoli, myocardium, and villi, effectively.This method exhibited a strong correlation with the ground truth images obtained through H&E staining.
For a more quantitative assessment, we utilized the Structural Similarity Index (SSIM), widely employed in the evaluation of image transformation.SSIM measures perceived changes not only in structural information but also in contrast and luminance variations, allowing for a numerical representation of subtle differences between tissues and thicknesses under the same staining [12].The similarity between OCM images before and after digital staining and H&E images was assessed using SSIM values.In particular, various comparisons were conducted by altering color indices in the H&E images to observe the impact of color similarity on the overall similarity assessment.
This study explored structural similarity by comparing OCM images in grayscale before digital staining with H&E stained images after removing color information.L*a*b* color transformation was used to isolate brightness information from H&E stained images.In the L*a*b* color space, L* represents brightness, while a* and b* convey information about distinguishable colors.Projecting H&E stained images into L*a*b* and removing color components resulted in deriving only brightness information (L*).Quantification of SSIM was performed between brightness-only H&E stained images and OCM images.Subsequently, SSIM was computed between digitally stained and H&E stained images.Additionally, the influence of color information on SSIM calculations was investigated by assessing changes in SSIM between digitally stained and H&E stained images in RGB space, considering the presence or absence of structure-preserving color normalization [17].
The SSIM scores, reflecting various conditions across four types of tissues, are presented in Table 2.The similarity between H&E images and OCM images was notably higher when   digitally stained, and this trend was consistently observed across all tissue types.Hematoxylin is commonly used for nuclear staining, while Eosin aids in visualizing the cytoplasmic spaces between cells.Although OCM with scattering contrast allows for non-destructive observation of fine cellular structures like nuclei, it has limitations in distinctly separating cell nuclei, as seen in H&E staining.Consequently, the SSIM between OCM and H&E stained images shows relatively lower scores.In contrast, digitally stained images generated by trained algorithms have been experimentally validated to offer structurally comparable information closer to the ground truth.
It is also confirmed that the structure-preserving color normalization enhanced the SSIM score indicating morphological details that are represented by stained colors could be reconstructed in digitally stained images from OCM images.Figure 5(a) shows the representative comparison between digitally stained OCM image and chemically H&E stained BFM image from skin tissue.The color components corresponding to hematoxylin and eosin dyes were extracted utilizing SNMF and plotted in normalized RGB space.It reveals that two prominent clusters are formed for corresponding stain labels for both digitally and chemically stained images.In addition, we can observe that the stain color representations of digitally stained image are highly correlated to the color representations of chemically stained BFM image.Finally, using this method, we successfully generated images stained only with hematoxylin and eosin.The technique was applied consistently to both digitally stained OCM images and H&E stained BFM images.Figure 5(b) demonstrates the diversity of images when adjusting the RGB distribution of digitally separated hematoxylin and eosin.This technique is efficient in controlling the color intensity in thickly sectioned tissue stains and can be directly implemented in digitally stained OCM images.Additionally, the method of aligning staining intensity with the desired color distribution has broad applications, enabling consistent adjustment of color distribution influenced by factors like staining techniques, tissue preparation, fixation, staining reagent quality, user skill, and experience.

Conclusion
In this research, we introduced a method for imaging histopathological slides utilizing OCM, followed by digital staining.Through the customization of tissue sectioning and staining procedures, we successfully obtained pairs of OCM images and digitally stained slides from the same specimens.The feasibility of our approach was successfully validated by applying a digital staining algorithm to various mouse tissues such as skin, lung, heart, and intestine.Additionally, we presented a digital stain separation and normalization method using H&E stain color vectors to achieve consistent staining.As a result, quantitative analysis revealed that OCM images digitally stained exhibit greater similarity to H&E BFM images compared to their pre-stained counterparts.This approach offers advantages in terms of reducing tissue staining time and decreasing operator dependence during the histochemical staining process.The most unique feature of our research lies in proposing a method to extend the utilization of OCM, initially used in bulk tissue, to conventional pathology slides.Digitally stained OCM images have the potential to find applications in diverse fields involving histopathology.They are particularly well-suited for situations requiring prompt tissue feedback, such as the detection of tumor margins in surgical settings [25].Considering the potential of OCM to provide volumetric data, there is a prospect for expanding conventional 2D histopathological information into 3D in the future [10].Moreover, it is anticipated that the integration of OCM and digital staining technologies could progress towards in vivo histology in the future through continued research.Disclosures.The authors declare no conflicts of interest.

Fig. 1 .
Fig. 1.Quality assessment of OCM and BFM images corresponding to the H&E stained skin tissue thickness.The contrast matrix is calculated based on RMS and is placed on the top-left in the OCM image.The average stain density value is on each bar in the chart painted with the extracted stain color vectors.H; Hematoxylin, E; Eosin

Fig. 3 .
Fig. 3.The architectures of the generator and discriminator.The generator converts a given OCM patch, with dimensions of 512 × 512 pixels, into a pseudo H&E patch resembling the ground truth.In the presence of pairs containing OCM and either pseudo or real H&E patches, the discriminator generates a prediction map indicating the likelihood of each local area being realistic.UpConv represents the transposed convolution block.C, K, and S in the convolution block represent the number of channels, kernel size, and strides, respectively.

Fig. 4 .
Fig. 4. Visualization of the digital stained results for the large size inputs with the original OCM inputs and the ground truths of the mouse tissues; heart, intestine, lung, and skin.White and black scale bars represent 500 µm and 100 µm, respectively.A large part of each wide-field OCM image inside the H&E-colored box was patched into be fed to the network for digital staining.These patches were digitally stained and then rearranged to the original position.DS; digitally stained result, GT; ground-truth.

Fig. 5 .
Fig. 5. Comparison of color distribution in digitally stained result and H&E stained BFM image of skin tissue.(a) Applying the sparse non-negative matrix factorization, hematoxylin and eosin color components were extracted from the digitally stained results and H&E stained images.(b) Adjusting the stain color distribution in digitally stained images for varying color expression in the target position.DS; digitally stained result, GT; ground-truth, H; hematoxylin, E; eosin.