Colorimetrical uncertainty of a hyperspectral imaging microscopy system for assessing whole-slide imaging devices.

A whole-slide imaging (WSI) device is a color medical imaging system whose application in digital pathology is to digitalize stained tissue samples into electronic images for pathologists to diagnose without using a conventional light microscope. Testing the color performance of a WSI device usually implies a color target with known truth that is compared with the device output to estimate color differences. Using stained tissue samples as color targets is challenging because the cellular features cannot be measured with ordinary spectroradiometers unless a hyperspectral imaging microscopy system (HIMS) is used. The goal of this study is to determine the colorimetrical uncertainty of such a reference HIMS that is designed to assess the color performance of WSI devices. A set of optical filters are used for that purpose. The color truth, in terms of spectral transmittance in the visible band, of the optical filters is measured by a reference spectroradiometer. The spectral transmittance is combined with a standard illuminant to generate colorimetrical measures using the CIEXYZ and CIELAB formulas. The differences between the reference HIMS and the reference spectroradiometer are evaluated using the CIE 1976 color difference formulas.


Introduction
A whole slide imaging (WSI) system is an automated digital slide creation, viewing, and management system intended as an aid to the pathologist to review and interpret digital images of surgical pathology slides. The system generates digital images that would otherwise be appropriate for manual visualization by conventional light microscopy [1]. As a medical imaging device, the technical characteristics of a WSI system, such as spatial resolution and focusing accuracy, need to be comparable to the conventional light microscopy that has been used by pathologists for decades. Among the essential technical characteristics of WSI systems, color performance is fundamental because histology is based on staining techniques to color and expose invisible cellular structures [2]. In conventional light microscopy, i.e. using an eyepiece, color is transmitted purely in the optical domain whereas for WSI systems digital conversion usually leads to color discrepancies between the original scene and the device output [3,4], making color performance unique for WSI systems. Characterizing the color behavior of a WSI system requires color measurement of the scene, which is challenging because the cellular structures of tissue samples are too fine to be measured by ordinary spectroradiometers. To assess the color performance of WSI devices, a hyperspectral imaging microscopy system (HIMS) developed to measure the color truth of tissue samples at the pixel level was previously presented [5]. However, the measurement accuracy of the HIMS itself was not reported.
In this study, a test method is presented to determine the measurement accuracy and uncertainty of such a HIMS. The results include transmittance measurements of neutral density filters as well as of color transmittance filters that are compared to reference measurements of the same region of interest (ROI) by a spectroradiometer equipped with a fiber probe. The CIELAB color space coordinates and their uncertainties are derived from the transmittance measurements and the CIE 1976 color difference in the CIELAB color space [6] is used as a color performance assessment metric. Figure 1 presents the experimental setup used to estimate the color performance of a HIMS.

Device under test
The hyperspectral imaging microscope system is based on an upright light microscope (AxioPhot 2, Carl Zeiss Microscopy, White Plains, NY, USA) in bright-field illumination mode. The original lamp housing is replaced with a tunable light source (OL490, Gooch and Housego, TX, USA) using a xenon lamp to provide illumination from λ = 380 nm to 780 nm with adjustable bandwidth (BW). In this study, λ is spanned in steps of 10 nm with BW = 10 nm as the 41 spectra shown in Fig. 2. A liquid light guide directs the light to a f = 40 mm collector lens (MCWHL5-C4, Thorlabs, Newton, NJ, USA) that is followed by a condenser (Achromaticaplanatic NA = 0.9, Carl Zeiss Microscopy, White Plains, NY, USA). The sample is set on a motorized stage system (MAC 6000, Ludl Electronic Products Ltd., Hawthorne, NY, USA) and is imaged using a 20x objective (Plan-Apochromat 20x NA = 0.8, Carl Zeiss Microscopy, White Plains, NY, USA). The image of the sample is acquired by a camera (Grasshopper3 9.1 MP Mono USB3 Vision, Point Grey Research Inc., BC, Canada) with a monochrome sensor (ICX814, Sony Electronics, Newton, NJ, USA). The image format used in this study is a 676 × 844 pixels (2.49 × 3.11 mm 2 ) area situated in the center of the sensor. The camera shutter time and light source intensity are optimized to prevent detector saturation while maximizing the detected signal for each measurement wavelength. Both the focusing and the Kohler illumination adjustment are achieved in broadband illumination, i.e. using the total illumination spectrum provided by the xenon light source of the OL490. The tunable light source, motorized stage, and camera are all controlled by programs written in Matlab (Mathworks, MA, USA) running in the Microsoft Windows 7 Professional 64-bit environment.

Reference instrument
A spectroradiometer (PR-730 with fiber probe FP-730, Photo Research, Syracuse, NY, U.S.A.) is used as the reference instrument. The distal end of the fiber probe is positioned in the eyepiece tube to measure the ROI as observed by the hyperspectral imaging microscope system.

Samples
Both standard and non-standard transmittance targets are used. The standard targets include Kodak Wratten (KW) gelatin neutral density (ND) filters with optical density OD = [0.1, 0.2, 0.3, 0.6, 1.0, 2.0] and color gelatin filters #12 (yellow), #25 (red), #32 (magenta), #47 (deep blue), and #58 (green) (Edmunds Optics, Barrington, NJ, USA). One should note that in order to prevent potential interference patterns due to air gap between film and glass [7], the KW filters are held laterally without setting them on a glass slide. To better represent the color gamut of the hematoxylin-and-eosin (H&E) stained tissue samples, we designed a color phantom populated with a choice of Roscolux color filters (Rosco Laboratories Inc., Stamford, CT, USA) as a set of non-standard transmittance targets. 24 holes (diameter 4.76 mm) are punched on a 1 mm-thick supporting cardboard slab using a dot puncher. A thin cardboard slab with 6.35 mm holes is glued on the supporting slab and 23 locations are filled with a filter dot glued on the supporting cardboard. The additional hole is left empty for measuring the 100% transmittance. A thin covering slab with 24, 4.76 mm-punched holes is glued on top (Fig. 3).

Measurements
Measurements of the transmittance are conducted is two distinct steps. First, the sample is measured using the spectroradiometer with a broadband illumination. Secondly, the same sample is measured using the camera with narrow band illumination. In both cases the illumination is provided by the OL490 light engine. The numerical aperture of the spectroradiometer's detector fiber probe averages out over the sample ROI. For comparison purposes, images of the same ROI captured by the camera are spatially averaged as well. Repeated measurements lead to signal intensities from which a mean value and a standard deviation can be computed. The transmittance of the sample is then expressed as where (I S , I SBg ) and (I W , I WBg ) are the intensities of the signal measured with the sample in the light path and with no sample in the light path (T = 100%), respectively. I SBg and I WBg are the background signals obtained setting the intensity of the light engine to zero. They account for both persistent background illumination observed with the light source and camera background noise. For measuring I S and I W , the intensity of the light source is optimized to obtain the higher signal output without detector saturation. Additionally, for measurements with the camera, both the light source intensity value and the camera shutter time are optimized wavelength by wavelength. For that purpose, we implemented an independent step to define a where S(λ) is the relative spectral power of one of the CIE standard illuminant,x(λ),ȳ(λ) and z(λ) are the CIE 1931 color matching functions [8][9][10], and K is the norlizing factor of Y(λ) corresponding to 100 for a perfectly transmitting sample. In this study, the CIE D65 standard illuminant is used. Converting the integral to summations, we have The CIELAB (L * , a * , b * ) values are then computed as and (X n , Y n , Z n ) are the tri-stimulus values for a perfectly transmitting sample. The CIE 1976 color difference in the CIELAB color space can be computed as the Euclidian distance between two color points of coordinates where

Uncertainty propagation
The usefulness of measurements results is to a large extend determined by the quality of the statement of uncertainty that accompany them [11]. Since the HIMS is our reference instrument to determine the color truth of reference tissue samples further used to assess the color performance of WSI scanners, it is important to first estimate the uncertainty on the measurements results provided by the HIMS. Hence, the uncertainty on the transmittance, CIEXYZ coordinates, CIELAB coordinates are estimated using the law of uncertainty propagation which is based on the Taylor expansion of the functional relationship between the output quantities y and the input quantities x about the mean values of x, µ x [11,12]. In matrix form, assuming that C x is the covariance matrix of the input quantities, the covariance matrix of the output quantities C y is C y JC x J t (8) where J is the Jacobian matrix with From Eq. (1) and Eq. (8), C T = J 1 C I J t 1 at each measurement wavelength λ with The measurements of the sample, the 100% transmittance and background are independent. Hence, C I is a diagonal matrix whose elements are the variances on the measured signals, s 2 x i , where s x i is the standard deviation on the measured signals and x i = (I S (λ), I W (λ), I SBg (λ), I WBg (λ)). In developed form, the resulting variance on the transmittance is From Eq. (3), the tri-stimulus equations are linear functions of the transmittance measurements and Eq. (8) is not an approximation in that case, C CIEXYZ = J 2 C T J t 2 with where N is the number of measurement wavelengths. Since the transmittance measurements over the wavelengths are independent, C T is a N × N diagonal matrix with elements [s T(λ 1 ) 2 , . . . , s T(λ N ) 2 ], i.e. the variance of the transmittance measurements. The tri-stimulus values are correlated because the color matching functions overlap, and they depend on the transmittance spectrum [13]. The resulting covariance matrix, C CIEXYZ , is 3 × 3.
From Eq. (5), the non-linearity of the relationship between the CIELAB coordinates and the CIEXYZ coordinates implies that Eq. (8) is an approximation, C CIELAB J 3 C CIEXYZ J t 3 with [14] Where The resulting covariance matrix, C CIELAB , is 3 × 3. From Eq. (7) the normally distributed CIELAB coordinates (L * 1 , a * 1 , b * 1 ) and (L * 2 , a * 2 , b * 2 ) can be used to form a Euclidian distance in the CIELAB space, ∆E * ab , that is not normally distributed. Hence, the statistical distributions of ∆E * ab between both types of measurements are computed by Monte Carlo simulations of the color points positions using the covariance matrices of the CIELAB coordinates, C CIELAB 1 and C CIELAB 2 . The median of the ∆E * ab statistical distribution, med ∆E * ab , is used as metric to estimate the proximity between (L * 1 , a * 1 , b * 1 ) and (L * 2 , a * 2 , b * 2 ). Here, we limit our estimation to the type A uncertainty (uncertainty evaluated by the statistical analysis of series of observations [11]) by considering: i) the propagation of the uncertainty on a set of measured transmittances under the same measurement conditions, i.e. repeatability of the results, and ii) conducting the experiments several times under changed conditions to account of the reproducibility. The estimated variance, s 2 R , on the results of the reproducibility experiments is added to the square of the uncertainty on the repeatability experiments to compute the total type A variances, u 2 A . The expanded uncertainty is U = ku A where k is the coverage factor. We used k = 2 which correspond to a 95%-confidence interval.

Results
To assess the linearity of the transmittance measurements with the camera, we compare the spatial average of the transmittance images, T SA , to the spectroradiometer transmittance measurements, T S , for a set of KW gelatin ND filters with optical density OD = [0.1, 0.2, 0.3, 0.6, 1.0, 2.0]. The uncertainties on T SA and T S are computed using Eq. (9). Ten reproducibility experiments are conducted for OD = 0.3. The estimated variance from the reproducibility experiments, s 2 R , is used for all samples to compute the total type A variances, u 2 A . Figure 4(a) shows that for most OD values, T SA and T S overlap within the error bars over most wavelengths in the spectral range of measurements but that the differences can be significant for λ = 380 nm and 780 nm. At these wavelengths, the values of the color matching functionsx(λ),ȳ(λ) andz(λ) are small enough that the impact of the transmittance values on the end results CIELAB coordinates is limited. Figure 5 illustrates this assumption by presentingx(λ),ȳ(λ),z(λ) and the Relative Cumulative Weight (RCW) of the color matching functions, expressed as¯x (λ i )+ȳ(λ i )+z(λ i ) N i=1x (λ i )+ȳ(λ i )+z(λ i ) . We compute a weighted linear interpolation of T SA versus T S using the uncertainty on T SA as weight parameters and considering T S as the ground-truth of the transmittance of the KW ND filters. For a broad range of wavelengths, λ = 390 nm to 770 nm , there is a linear relationship between T SA and T S .   Table 1 presents the CIELAB coordinates results (L * , a * , b * ) for the set of neutral density filters, their uncertainties and the median value of the statistical distributions of ∆E * ab , med ∆E * ab , obtained for each sample. One should note that despite the proximity of the (L * , a * , b * ) values issued from T SA and T S , there is not always an overlap within the error bars at k = 2. For OD = 2.0, despite the relative proximity of the CIELAB coordinates mean values their large uncertainties explain the larger med ∆E * ab = 3.51 obtained for this sample for which the T SA and T S values are close to zero over the detection wavelengths. Figure 6(a) presents the transmittance data for OD = 0.3 and shows that T SA and T S overlap within the error bars at all wavelengths. For λ = 780 nm the overlap is explained by the large uncertainty on T SA . Figure 6(b) (c) and (d) present the CIELAB coordinates and 95% confidence regions issued from T S and T SA measurements and their uncertainties in the (a * , L * ), (b * , L * ) and (b * , a * ) projection planes, respectively. There is no overlap between the 95% confidence regions apart from in the (b * , a * ) projection plane which points to a systematic error on the color coordinates. However, the median value of ∆E * ab (med ∆E * ab = 0.68) is relatively small and the agreement between the color coordinates issued from T SA and T S is considered reasonable.

OD
Transmittance Since neutral density filters have low chromaticity values, color filters are measured to assess the color performance of the setup. We first measure a set of KW color gelatin filters and then we measure the color phantom. Ten reproducibility experiments are conducted for samples KW #32 and #47. To maximize the resulting uncertainties on the CIELAB coordinates, the estimated variance from the reproducibility experiments, s 2 R , of sample #32 is used for all KW color filters but for sample #47 to compute the total type A variances, u 2 A , on the measured transmittances. Figure 7 shows that for all samples T SA and T S curves overlap within the error bars at all wavelengths. Again, for λ = 780 nm the overlap is explained by the large uncertainty on T SA .
The CIELAB coordinates, their uncertainty and the median value, med ∆E * ab , of the statistical distributions of ∆E * ab obtained for each KW color filters are presented in Table 2. Again, despite the proximity of the (L * , a * , b * ) values issued from T SA and T S , there is not always an overlap within the error bars at k = 2.
As an illustration of the uncertainty analysis for the KW color filters, Fig. 8(a) presents the T S and T SA spectra with their uncertainty for filter #32 (magenta) and shows that the largest discrepancies indeed occur at λ = 380 nm and 780 nm. The 95% confidence regions around the CIELAB coordinates derived from T S and T SA and their uncertainties presented in Fig. 8(b) (c) and (d) (CIELAB projection planes) do not overlap. Again, this points to a systematic error. For the whole set of KW color filter, med ∆E * ab ranges from 0.74 to 2.31 and the agreement between the color coordinates issued from T SA and T S is considered reasonable.
Based on measurements by the reference spectroradiometer, the gamut of the color filters composing the color phantom is represented in the CIELAB space along with their color appearance ( Fig. 9(a)). Figure 9(b) is a boxplot of the corresponding ∆E * ab values computed from T S , T SA and their uncertainties. A boxplot displays the distribution of data using a five-numbers

Filter
Transmittance  summary. The upper limit of the box is the first quartile Q 1 , the bottom is the third quartile Q 3 with the median values Q 2 set between these two limits. The "minimum" and "maximum" values are defined as Q 1 − 1.5 × IQR and Q 3 + 1.5 × IQR, respectively, with the interquartile IQR = Q 1 − Q 3 . They respectively correspond to the minimum and maximum values excluding any outliers which are represented as red crosses in the boxplot. Median value of ∆E * ab are in the 0.5 to 1.0 range while most outliers are smaller than 3 with some exceptions for a small sub-selection of 5 filters (#24, #342, #46, #347, #59). However, these patches have upper whiskers smaller than 3. Again, there is a good agreement between the color results computed from the T S and T SA measurements. The phantom will be used provide traceability of the measurements by the hyperspectral microscope. Figure 10 presents an example of a tissue image (H&E stained, cancer adjacent cervix tissue, CNS801 B10, US Biomax, Inc., Derwood, MD, USA) obtained by the HIMS. The transmittance values are measured from λ = 380 nm to 780 nm at the pixel level and the CIEXYZ coordinates are computed for each pixel. The sRGB coordinates are computed from the CIEXYZ coordinates for each pixel [15].

Conclusions
Using a HIMS is an effective means of measuring the spectral transmittance of biological tissue slides at the pixel level, which is essential for determining the color truth for evaluating the color performance of WSI systems. WSI systems are medical devices. As a reference instrument, the accuracy and uncertainty of a HIMS need to be quantitatively assessed with an objective method. In this study, a bench test method is described and demonstrated to assess a sample HIMS. A color target consisting of 23 miniature optical filters is crafted to be compatible with standard glass slides to test a HIMS, WSI, or microscope device for traceability. The spectral transmittance of each miniature optical filter is measured with a spectroradiometer as the truth. The errors of the HIMS under test are estimated by computing the Euclidian distance in the CIELAB space, i.e., ∆E * ab , with a Monte Carlo simulation. The results show that the median value of the sample HIMS are less than 0.75 ∆E * ab for neutral patches and less than 1.0 ∆E * ab for chromatic ones. These results are adequate for assessing commercial WSI systems, where the nominal errors are usually greater than 20 ∆E * ab . Thus, the color performance testing data of a WSI device measured by the HIMS under test are considered validated for regulatory purposes.
Other than HIMS, the described test method can also be adapted to assess the other spectrumsensing optical instruments such as colorimeters, photometers, and densitometers. Using alternative metrics such as CIEDE2000 to estimate color errors is part of the future work in addition to crafting color targets for the other staining methods.