mHealth spectroscopy of blood hemoglobin with spectral super-resolution: supplementary material

with spectral super-resolution: supplementary material Sang Mok Park,1,† Michelle A. Visbal-Onufrak,1,† Md Munirul Haque,2 Martin C. Were,3,4,5 Violet Naanyu,6 Md Kamrul Hasan,3 and Young L. Kim1,7,8,9,* 1 Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana 47907, USA 2 R. B. Annis School of Engineering, University of Indianapolis, Indianapolis, Indiana 46227, USA 3 Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee 37212, USA 4 Department of Medicine, Vanderbilt University School of Medicine, Nashville, Tennessee 37212, USA 5 Vanderbilt Institute for Global Health, Vanderbilt University Medical Center, Nashville, Tennessee 37212, USA 6 Department of Behavioral Sciences, Moi University School of Medicine, Eldoret, Kenya 7 Regenstrief Center for Healthcare Engineering, Purdue University, West Lafayette, Indiana 47907, USA 8 Purdue University Center for Cancer Research, Purdue University, West Lafayette, Indiana 47907, USA 9 Purdue Quantum Center, Purdue University, West Lafayette, Indiana 47907, USA


Supplementary
. Initial validation of SSR spectroscopy using the image-guided hyperspectral line-scanning system. a&b. SSR-based hyperspectral reconstruction tested using multiple hyperspectral line-scans within the inner eyelid. The transformation matrix from RGB to spectra is constructed, using the RGB image and the hyperspectral line-scan obtained from Line #1 as a training dataset. Then, the RGB data from Line #2 (i.e. validation dataset) is substituted into the transformation matrix to generate the corresponding reconstructed spectrum in Line #2. By moving the system 5 mm to Line #2, the actual measured hyperspectral data are also acquired. c. Comparison between the measured and the SSR-reconstructed spectra in the spectral-spatial images in Line #2. d. Averaged spectra along the y axis for each wavelength from the spectral-spatial images. The reconstructed spectrum from the RGB data is in excellent agreement with the actual measured spectrum. Multiple hyperspectral line-scans within the inner eyelid using the image-guided hyperspectral line-scanning system. A reflectance spectrum is averaged over the vertical line-scanning area of 0.8 mm × 7.2 mm in the eyelid. c. Four reflectance spectra averaged by the vertical line-scanning spaced 5 mm apart result in nearly identical spectral profiles, independent of the exact horizontal location in the inner eyelid. This vertical averaging approach with a relatively large area is critical to ensure the reproducibility as point-of-care diagnostic testing. The coefficients of determination R 2 (a-c) and the root mean square errors (RMSE) (d-f) between the computed and clinical laboratory blood Hgb levels are calculated in different combination cases of training and validation datasets as a function of the number of principal components. The original dataset (n = 153) is randomly split into a training dataset (n = 138) and a validation dataset (n = 15), and this random split process is repeated 15 times, generating 15 different combination cases. We average the R 2 and RMSE values between the computed and clinical laboratory blood Hgb levels for all combination cases. c. In the training datasets, as the number of principal components increases, the R 2 values increase, and the RMSE values decrease, respectively. f. In the validation datasets, the R 2 and RMSE values have the maximal and minimal values for 18 components as an optimal number of principal components. The R 2 and RMSE values computed with 18 components are highly consistent in all of the different combinations of training and validation datasets, supporting the fidelity of the spectroscopic blood Hgb prediction model.

Count
Blood Hgb (g dL -1 )  Figure 11. Receiver operating characteristic (ROC) curves of spectroscopic and SSR blood Hgb measurements for anemia assessments. The blood Hgb thresholds to define anemia recommended by WHO are used as follows: Hgb < 13 g dL -1 for men (age ≥ 15 years) and Hgb < 12 g dL -1 for women (age ≥ 15 years). Both of the spectroscopic and SSR blood Hgb measurements show excellent performance of anemia assessments, although clinical applications of the reported blood Hgb quantification are not limited to noninvasive anemia assessments.  Table 4 for multiple linear regression details).

Image-guided hyperspectral line-scanning system
To acquire high-resolution spectra reflected from the exact portion of the inner eyelid, we developed an image-guided hyperspectral line-scanning system, inspired by an astronomical telescope system ( Figure  2a and Supplementary Figures 1, 3). This custom-built system had two cameras mounted to a dualchannel spectrograph (LISA, Shelyak Instruments) that allowed for simultaneous acquisitions of spectral and image data from the same object. To provide broadband white-light illumination to the inner eyelid, a white-light LED ring illuminator (Neopixel RGBW 24 LED ring, Adafruit Industries) was used to cover the visible wavelength range of 400 -700 nm. This LED ring illuminator was mounted to a telecentric lens (0.5 ×, Edmund Optics) via a 3D printed ring holder to fit the lens circumference. The intensity of the LED ring was controlled with a microcontroller (Arduino UNO). The image-guided hyperspectral line-scanning system had two data acquisition ports: a hyperspectral line-scanning port mounted with a mono CCD camera (PointGrey Grasshopper3 5.0 MP Mono, FLIR) and an image port mounted with a three-color CCD (PointGrey Grasshopper3 5.0 MP Color, FLIR). For hyperspectral line-scanning (length = 7.2 mm and width = 0.8 mm), the telecentric lens collected light reflected from the inner eyelid, which passed through the slit of the spectrograph and disperses with a diffraction grating. The diffraction grating had a high efficiency in the visible wavelength range of 400 -700 nm with a slit width of 23 μm. For image guiding, the light from the inner eyelid was reflected via another mirror toward the imaging port to generate a field of view (14 mm × 12 mm) with a spatial resolution of ~150 µm. The imageguided hyperspectral line-scanning system was rested on a base with two interlocked x-y-z positioning stages that served to move the system to locate the inner eyelid image centered within the rectangular region of interest (ROI) guide. The data acquisition of hyperspectral line-scans and RGB images, LED light control, and ambient room light subtraction were synchronized by a custom-built LabVIEW interface (National Instruments). In particular, measurements of a white reference reflectance standard (SRT-99-050, Labsphere) were acquired to correct for the system response (both illumination and detection). Two sequential data of the inner eyelid (and the white reflectance standard) were acquired by turning on and off the LED illuminator to remove the ambient room light.

Wavelength calibration and spectral resolution determination
As the image-guided hyperspectral line-scanning system was completely custom-built, we calibrated the wavelength of the system and determined the spectral resolution. In particular, it was important for us to configure the entire desired wavelength range and the high spectral resolution, because hyperspectral data generated by the image-guided hyperspectral line-scanning system were used as a priori information to develop the spectral super-resolution (SSR) algorithm. First, as a calibration light source, we used a Xenon spectral calibration lamp (6033 Xenon Lamp, Newport) generating multiple narrow and intense spectral lines from ultraviolet to near-infrared wavelengths. To determine the actual wavelength in each pixel, we configured the relationship between the pixel numbers and the known peak positions of the calibration light source in the wavelength range of 400 -700 nm. This calibration was well described by linear regression (Supplementary Figure 3a): = 0.1381 × + 383.2064, (S1) where is the wavelength (nm), and is the pixel number. Each pixel size corresponded to an interval of 0.14 nm in the wavelength range of 383 -721 nm. We confirmed that the calibrated wavelength information captures the exact peak positions of the Xenon spectral calibration lamp (Supplementary Figure 3b). Second, we determined the spectral resolution using a helium-neon laser at 543.5 nm (CVI Melles Griot). The spectral resolution (Δλ), which is defined by the full width at half maximum (FWHM), was 1 nm (Supplementary Figure 3c). Accordingly, the spectral interval of the SSR algorithm was 1 nm.

Tissue phantom study
To ensure the image-guided hyperspectral line-scanning system for analyzing the spectral characteristics of blood hemoglobin (Hgb), we had a series of initial tests using tissue-mimicking phantoms with blood microvessels. We prepared two artificial microvessels (low-density polyethylene tubings) filled with solutions having different Hgb concentrations, which were made by lyophilized powdered Hgb (hemoglobin A0, Ferrous Stabilized human lyophilized, Sigma-Aldrich). As shown in Supplementary  Figure 5a, the microvessel in the left contained a higher Hgb concentration of 5.0 g dL -1 , whereas the one on the right had a lower Hgb concentration of 3.0 g dL -1 . Supplementary Figure 5b shows the RGB guiding image obtained from the tissue phantom with blood microvessels. Along the direction of hyperspectral line-scanning (length = 7.2 mm) indicated in the guiding image, we acquired spectra reflected from the two microvessels (Supplementary Figure 5c). A higher Hgb optical absorption can be noted in the hyperspectral line-scan image, where the microvessel with the higher Hgb concentration (5.0 g dL -1 outlined in red) had a lower intensity for the wavelengths between 450 and 550 nm than that with the lower Hgb concentration (3.0 g dL -1 outlined in blue). Selecting only the spatial pixels corresponding to each microvessel and averaging the spectral intensity values across the outlined spatial coordinates for each case provided the distinct spectral profiles of each individual microvessel with the difference Hgb concentrations (Supplementary Figure 5d).

Study enrollment and clinical imaging procedure
Our study data included the spectral and RGB data from 153 individuals (90 females and 63 males) who were referred for complete blood count tests at the AMPATH Accredited Clinical Laboratory of MTRH in Eldoret, Kenya. All of the experiments which involved human subjects were approved by Purdue University Institutional Review Board (IRB) and Moi University/MTRH Institutional Research and Ethics Committee (IREC). The informed consent was obtained from all of the participants. When an individual agreed to be in the study after introduced to the study purpose and background, the trained person looked at the patient's inner eyelid to check if there was any eye disease and ensured that the individual was qualified for the study. We collected the following information about the individual: date of birth, sex, race, ethnicity, blood Hgb counts, and any disease status. We took the inner eyelid image from the participant using the image-guided hyperspectral line-scanning system and Samsung Galaxy J3 with the mHematology mobile application. First, the participant was asked to place his/her chin on the chinrest and forehead against the bar. We asked the patient to pull down his/her inner eyelid and remain still while imaging of the inner eyelid. The white LED light was illuminated on the inner eyelid, ensuring minimal light exposure to the eye. When the imaging began, the image-guided hyperspectral linescanning system took one image with the LED ring on, and the other with LED ring off. The participant was asked to stay still until he/she saw a finished sign. The imaging with the line-scanning system took only 3 seconds. In general, the participant was instructed to select any inner eyelid at his or her convenience. On the other hand, to scrutinize any potential discrepancy in the spectroscopic blood Hgb quantification, we collected hyperspectral measurements from both left and right inner eyelids among a subset of 36 individuals who agreed to image both eyelids. Second, we used our own mobile application. The mHematology mobile application took RGB images of the inner eyelid, in which circular guidelines on the smartphone screen helped the study personnel to locate the eyelid in the center of the image. As in the image-guided hyperspectral line-scanning system, we took one image with flashlight on and the other image with flashlight off through the mHematology mobile application. The entire imaging procedures took approximately 5 minutes for each individual participant.

Optimization of principal components in partial least squares regression
The blood Hgb computation model for predicting actual blood Hgb levels was constructed with PLSR. We determined an optimal number of principal components in a similar manner of cross-validations. The original dataset (n = 153) was randomly split into a training dataset (n = 138) and a validation dataset (n = 15). A randomly selected validation dataset was not used for constructing the spectroscopic blood Hgb prediction model. After this random split process was repeated 15 times, we calculated the coefficient of determination R 2 and root mean square errors (RMSE) values between the computed and clinical laboratory blood Hgb levels in different combination cases of training and validation datasets as a function of the number of principal components (Supplementary Figure 6). For each individual training dataset, a greater number of components better captured variations in the variables and lowered a prediction error (Supplementary Figure 6a, d). For each validation dataset, there was an optimal number of principal components with the maximum R 2 and minimum RMSE values (Supplementary Figure 6b, e). In addition, when we averaged the R 2 and RMSE values between the computed and clinical laboratory blood Hgb levels for all of the combination cases, the optimal number of 18 principal components clearly returned the maximum average R 2 and minimum average RMSE values for the validation datasets, respectively (Supplementary Figure 6c, f). Overall, we ruled out possible overfitting of the spectroscopic blood Hgb prediction model, as we evaluated the ability to predict blood Hgb levels from the completely new sets of data after the model was established properly.

Initial validation test of spectral super-resolution
We conducted an initial validation test for the statistical learning method of SSR, using hyperspectral data and guiding RGB images generated by the image-guided hyperspectral line-scanning system. We collected hyperspectral line-scans of two arbitrary lines 5 mm apart in the eyelid of a volunteer (Supplementary Figure 2a, b). Each line-scan was performed by physically moving the image-guided hyperspectral line-scanning system laterally. The hyperspectral dataset from Line #1 contained 31 spectra along the vertical axis that served as a training dataset to obtain a transformation matrix for hyperspectral reconstruction from RGB data. As a validation set, we substituted the RGB data of the guiding image corresponding to Line #2 into the transformation matrix. The computed reconstructed spectra were highly comparable with the ones measured in Line #2 (Supplementary Figure 2c). The averaged original and reconstructed spectra over the vertical axis were in excellent agreement (Supplementary Figure 2d). Indeed, fixed-design linear regression in an overdetermined framework is advantageous for developing a stable and reliable transformation matrix. This result supported the idea that SSR spectroscopy can be realized using a conventional RGB camera, eliminating a need of bulky and expensive dispersion optical components (e.g. spectrometer, spectrograph, mechanical filter wheel, or liquid crystal tunable filter).

White-light LED safety issues
Even though the LED light illumination of the image-guided hyperspectral line-scanning system and the smartphones was not directly aimed at the human subjects' eyeball during imaging, we evaluated the potential concern on the radiation safety of the LED light sources as follows: Under the FDA guideline, our devices did not require an Investigational Device Exemption (IDE) application, mainly because the image-guided hyperspectral line-scanning system and the smartphone were not considered as custom devices. Technically, these systems were the same as taking photographs using a camera under flashlight. The risk associated with imaging of the eyelid was minimal. To determine any hazard from the LED illumination, we used national and international guidelines for developing ophthalmology instruments. Because the imaging systems were not intended to image the retina in the eyes, it should be noted that our comparison was highly conservative. The FDA guidance for retinoscopes suggested comparing the optical radiation emissions associated with the device to those of a predicate or another legally marketed device. For this purpose, we investigated the optical radiation hazards as defined by the International Organization for Standardization (ISO) 15004 for: (i) ultraviolet and infrared radiation, and (ii) visible light and near IR radiation (blue-light weighted radiance and aphakic-weighted radiance) 1 . Because LED used in our imaging study only had the visible light in the spectral range of 400 -700 nm, the case applicable was as outlined in item (ii). ISO 15004-2.2 classified ophthalmic instruments into: Group 1 (no hazard), or Group 2 (some potential hazard or does not meet criteria for Group 1). Importantly, devices not considered standard ophthalmic instruments, such as smartphones, were classified as Group 2. For Group 2 devices, ISO proposed a safety limit of 706 mW cm -2 , which was the limit of spectral irradiance over a retinal area greater than 1.7 mm 2 . A previous study on the safety of using smartphones for retinal imaging reported spectral irradiance values that met this safety limit 2 . Following the guidelines from ISO 15004-2.2 1 , we measured the spectral irradiance of the LED ring and the smartphone LED flashlight using a power meter (Thorlabs PM100 Optical Power Meter and S120C Standard Photodiode Power Sensor). The power meter was placed at the same distance between LED and the inner eyelid required for properly imaging the inner eyelid. As a reference, we also measured the spectral irradiance under ambient room light conditions. Our results showed that the LED illumination was under the safety limit outlined for Group 2 devices (Supplementary Table 1). Even though our devices were not intended for retinal imaging but rather to image the inner eyelid only, the same hazards assessment for retinal imaging showed that our imaging system was comparable to the bright ambient room light, which was far below the safety limit of 706 mW cm -2 .