Assessing spectral imaging of the human finger for detection of arthritis

: Rheumatoid arthritis causes changes in the optical properties of tissues in the joints, which could be detected using spectral imaging. This has the potential for development of low cost, non-contact method for early detection of the disease. In this work, hyperspectral imaging system was used to obtain 24 images of proximal interphalangeal joints of 12 healthy volunteers. A large inter-subject variability was observed, but still an increase in transmittance in the spectral range of 600 nm – 950 nm could be associated to the joint in all images. The results of experiments were compared to detailed simulations of light propagation trough tissue. For the simulations, voxelized 3D models of unaﬀected and inﬂamed human joints with realistic tissue distributions were constructed from an in-vivo MRI scan of a healthy human ﬁnger. The simulated model of healthy ﬁnger successfully reproduced the experimental data, while the aﬀected models indicated that the inﬂammation introduces detectable diﬀerences in the spectral and spatial features. The results were used to guide the design of a dedicated imaging system for detection of rheumatoid arthritis, that will be used in an upcoming clinical study.


Introduction
Rheumatoid arthritis (RA) is an inflammatory polyarthritis that leads to joint destruction and deformation. The prevalence in Europe is between 0.5% and 1% and the incidence rate increases with age, peaking at an age of around 60 years. RA typically affects small and medium sized joints in a symmetric fashion and has a progressive, destructive course [1]. Without timely and sufficient treatment, the disease leads to significant reduction in the patient's quality of life and ability to work. Several studies have shown that a window of opportunity for alteration or even reversal of the disease exists in the first year and especially in the first 3 months [2]. The first few months therefore represent an important therapeutic window. Early detection and classification of the disease has proven difficult using the existing diagnostic methods, such as clinical inspection, blood tests and imaging using X-rays, ultrasound or magnetic resonance (MRI). All of the conventional imaging techniques have some drawbacks, most notably the use of ionizing radiation for X-rays, operator variability in ultrasound, and the high cost of MRI [3].
Several reports have indicated that optical techniques can be useful for imaging of inflamed joints [3]. The optical techniques offer non-invasive and non-contact imaging of small joints with the possibility to utilize the contrast provided by using multiple wavelengths. Images collected at different wavelengths carry useful information about the spectral variation of optical tissue properties like absorption, scattering, and tissue composition [4]. Specifically for an inflamed joint, the features which can significantly alter the spectral information include vessel proliferation (increased hemoglobin concentration), relative hypoxia and quantitative and qualitative change of synovial fluid (edema, increased scattering).
Diffuse optical tomography (DOT) was studied for detection of arthritis [5,6]. DOT is a complex method, where a joint is exposed to monochromatic light at a single position and the transmitted light is collected at multiple positions around the joint. This process is repeated for several illumination positions resulting in absorption and scattering images of the joint. These images can be used to differentiate between inflamed and non-inflamed joints with relatively high specificity and sensitivity [5][6][7]. Photoacoustic imaging (PAI) is an imaging technique combining the benefits of optics and ultrasound. When irradiated with short laser pulses, biological tissue generates broadband ultrasound waves with amplitudes proportional to the local optical absorption coefficient. Studies on inflamed and normal human joints in vivo demonstrate satisfactory sensitivity in assessing the physiological changes in the joints [8,9]. However, DOT and PAI are rather complex techniques requiring reconstruction of images from measured ultrasonic signals, the simultaneous evaluation of multiple joints is difficult, the system must be typically in contact with the joint, and examination times are usually long. A less complex optical technique is transillumination imaging (TI), an optical technique where light transmitted through articular tissues is measured and then used to form images. It was reported that TI can provide information about the inflammation status of finger joints [10,11]. All these techniques commonly utilize monochromatic light only, neglecting the abundance of spectral information.
To increase sensitivity and specificity of the optical techniques, more wavelengths should be utilized, each wavelength contributing specific features. Optical techniques providing both spatial and spectral information are hyperspectral (HSI) [12] and multispectral imaging (MSI) [13,14]. In a pilot study involving 144 joints from 21 rheumatology patients, transmittance images were obtained at five wavelengths (650, 710, 730, 830, and 930 nm) [15]. Accurate detection of inflamed joints was demonstrated, based on Fourier transform analysis of optical transmission images. A peak in intensity at the joint region was observed for non-inflamed joints, while a dip in transverse mean intensity was manifested for clinically inflamed joints. This differences in spatial information resulted in significant differences in Fourier transform amplitudes.
The sensitivity of hyperspectral imaging to the expected changes in arthritic joints has been demonstrated in our previous work [16]. A 3D Monte Carlo simulation was used to obtain hyperspectral reflectance and transmittance images for models of healthy and affected human finger joints. Two transmission spectral windows in the ranges of 800 nm -900 nm and 1050 nm -1100 nm were identified and significant changes in the transmission intensities and spatial distributions were observed for the affected joints. Thus, these results demonstrated that MSI utilizing appropriate spectral bands could be used to detect RA in human joints. However, a simplified model of the human finger approximated with geometric primitives, such as cylinders and ellipsoids, was used. In this model, passage of light trough the joint, unobstructed by the bone was possible. Yet, the curved shape of the joint surfaces in real fingers prevents the direct passage. Moreover, an unrealistic detector collecting all the reflected/transmitted light was simulated. These drawbacks were addressed in [17]. Realistic human joint models and tissue distributions were constructed from an in-vivo MRI scan of a human index finger [18] and used for more detailed simulations of healthy and affected fingers. The realistic finger anatomy involving all finger tissues lead to more reliable simulation results, clarifying if a realistic joint anatomy transmits enough light to enable the detection of RA. Even with the complexity of the geometry and tissue distributions of a real human finger, notable differences between healthy and inflamed joints were observed.
In the present study hyperspectral measurements of proximal interphalangeal joint (PIP) in transmission mode for 12 healthy subjects are presented. The experimental results are compared to simulations, using which the sensitivity of the method to RA is further explored. Specifically, section 2 presents the experiments and simulation procedure. The results are presented and discussed in sections 3 and 4, which are followed by the conclusions in section 5.

Measurements
The experiments were performed with a custom HSI system developed in-house with the specific aim of performing studies of reflectance and transmittance on a wide range of samples. Using push-broom method, the system achieved a spectral and spatial resolution of 0.5 nm and 60 µm, respectively, while imaging an area of sizes up to 14.4 cm x 5.7 cm. For this study, the images were obtained with fourfold binning of the pixels in the spatial, and twofold binning in the spectral dimensions.
The main hyperspectral imager was composed of a spectrograph (ImSpector V10E, Specim, Finland), monochrome camera (Blackfly S, FLIR, Canada) and an objective lens (Xenoplan, 2,8/50-0902, Shneider-Kreuznach, Germany). A plane of glass provided a solid surface on which the imaged object was fixed. The position of the imager was controlled using a 2D computer controlled stage (8MT195, Standa, Lithuania). In transmittance mode the light produced by a tungsten light with gold reflector (64635 HLX, Osram, Germany) was guided below the glass surface, on which the imaged object was positioned, using an optical fiber (9250-HT, Illumination Technologies, FL, USA). Using a convex lens after the fiber output, a highly uniform illumination was achieved over a circular area with a diameter of approximately 2 cm, suitable for imaging a single finger. The system is illustrated in Fig. 1. With the illumination intensity available, the transmittance imaging was performed with an integration time of 300 ms per line, and sensor gain set to 1. The images of PIP joint were obtained for 500 lines, corresponding to approximately 3 cm length along the finger. This resulted in an imaging time of 2.5 min per joint. In order to obtain absolute transmittance, the measurements were normalized with a reference image of the illumination without the sample in the light path. To obtain the reference image within the dynamic range of the camera, a neutral density filter (NE40A, Thorlabs, NJ, USA) was placed before the objective lens, and spectra were corrected for the spectral dependence of the filter transmittance.
Measurements of human finger optical transmission were obtained using the HSI system in the 400 nm -1000 nm spectral range. Images of middle fingers of the left and right hands near the PIP joint were acquired for 12 healthy volunteers (3 female, 9 male, age 25-40). The volunteers were instructed to position their hand palm down on the glass surface and keep it relaxed during the duration of the scan. No external pressure was applied to the imaged fingers. The temperature of the imaged fingers was not explicitly controlled, but the laboratory temperature was kept at 20°C by an air-conditioner and the volunteers rested in the laboratory for approx. 15 minutes prior to the imaging. Thus it is expected that the imaging conditions were equal for all volunteers. Stray light was stopped using strips of soft, black foam, which the volunteers held between their fingers. An informed consent was obtained from all volunteers and the procedure was performed according to the Declaration of Helsinki. The experimental protocol was approved by the Slovenian National Medical Ethics Committee.

Simulation
Numerical simulations were performed using a custom weighted-photon 3D Monte Carlo model [16,17]. Packets of photons were generated as perfectly collimated light source, uniformly distributed over the whole geometry. Optical propagation trough a voxelized sample was simulated, and a transmittance image was recorded with photons exiting the tissue in the forward (relative to the illumination) direction. Reflectance images and absorbance maps could also be obtained. To obtain the spectral data, the simulation was run for different optical properties of tissues in the 400 nm -1100 nm range, in 10 nm steps. At each wavelength, 10 8 photons were tracked until detection or their weight reaching 0. The finger geometries simulated were based on an ultrahigh resolution in-vivo MRI image of a human finger, obtained at 7 T with a dedicated finger radio frequency coil [18]. We performed tissue segmentation on the MRI data, resulting in a geometry with a spatial resolution (voxel size) of 0.2 mm consisting of 11 different tissues: bone, cartilage, synovial fluid (SF), synovial membrane (SM), tendon, artery, vein, subcutis, dermis, epidermis, and nerve. This geometry was used to represent a normal, healthy finger (N model), shown in Fig. 2. Similar to our previous studies [16,17], three different models of an inflamed joint were simulated: an arthritic joint with synovial membrane thickening (M model), an arthritic joint with SF effusion (SF model), and an arthritic joint with both SF effusion and synovial membrane thickening (SFM model). These three models were constructed from the base geometry using simple 2D linear transformations on transverse slices of the geometry. The transformations resulted in the volumes of SF and SM expanded by factors of approximately 7.5 and 3, respectively, in the vicinity of the inflamed joint. Inflammation of only the proximal interphalangeal joint (PIP) was considered. The affected models used in the simulations are shown in Fig. 3. The optical properties (absorption and scattering coefficients, refractive index and anisotropy) of all the tissues were simulated as precisely as possible using data found in the literature. For all of the references used, the reader is referred to our previous work [16]. Additional properties were added for cartilage [19,20], ligament [21] and nerve [4,22] tissues, while the properties of subcutis [23] and bone [24,25] were further refined. The values of optical properties used for simulation of healthy tissues are shown in Fig. 4.
According to the expected changes due to the arthritic disease [16], the following modifications to the optical properties were made for the three affected joint models compared to the healthy model: for the optical properties of synovial fluid, values measured for synovial fluid from  arthritic knee joint [26] were used instead of values for water; oxygenation of some tissues was decreased; water concentration in the cartilage was increased; blood volume fraction (bvf) in synovial membrane was reduced in the cases of M and SFM models. These changes are summarized in Table 1. While the SF and SFM model represent an already advanced stage of disease, with apparent deformation of the joints, the M model includes only the synovial membrane thickening, without overall increase in dimensions of the finger, and the expected changes in optical properties of tissues involved. This model can therefore represent an early stage of disease.

Data analysis
The data analysis was performed using MATLAB (The MathWorks, Natic, MA, USA) software. On the measured transmittance images, the position of the PIP joint was manually defined for each finger imaged. The transmittance spectra of the PIP joint were then sampled over a small surface around this point. The area of the sampling surface was chosen as a square of approximately 7×7 mm 2 , so as to cover the visible area of increased transmittance for the subjects, taking into consideration also the shape of skin texture, and actual volunteer's finger geometry determined by manual inspection. The procedure was the same for all images. Longitudinal transmission profiles along the length of the finger were acquired in the vicinity of PIP joint, over a width of 30 image pixels (approximately 1 mm).
The output of the simulations were the transmittance images obtained for different wavelengths of the incident light. The transmittance images resulting from simulations were smoothed using a Gaussian filter with a width (σ) corresponding to one voxel, in order to reduce the visual impact of sharp edges due to voxelized tissue geometry. For the simulation results, the transmittance values were sampled over different regions along the finger, as shown in Fig. 5. The proximal interphalangeal joint (PIP) region was defined where the thickness of the bone in the simulated geometry was the smallest and the biggest observable differences due to RA were expected. The middle phalanx (MID) region was defined over the area of bone close to the PIP joint, but far enough so that the expected changes were still small. Similar to the PIP region, the distal interphalangeal (DIP) region was defined over the distal interphalangeal joint. The geometry of this joint was unaffected by arthritis in the simulations. The sizes of DIP, MID and PIP regions were 6×6 mm 2 , 6×4 mm 2 and 8×8 mm 2 , respectively. In case of simulations, the longitudinal profiles were acquired over a width of 5 image pixels, which also corresponds to 1 mm.

Measurements
The transmittance image at 800 nm of a whole right middle finger of one of the volunteers is shown in Fig. 6. It can be seen how the transmittance increases towards the end of the finger, as the total tissue thickness, and in turn the optical absorbance, decreases. As expected considering the local tissue distribution and their optical properties, there is also an increase in transmittance around the PIP region. Also of note is the evident visibility of the skin wrinkles on the transmittance image. For this study, left and right middle fingers of 12 healthy volunteers were imaged using HSI in the vicinity of the PIP region. The obtained images at 800 nm are shown in Fig. 7. Large differences between the subjects are featured in the images. But most importantly, a relative increase in transmittance near the PIP joint is clearly present in almost all images. This is harder to observe mostly for subjects 7 and 12. Motion artefacts can be seen in some images (e.g., Subject 2, left joint), but are small enough that the values sampled over larger regions should not be affected. The spatial characteristics of the imaged joints were also analyzed by plotting the longitudinal transmittance profiles (Fig. 8). This are heavily influenced by the skin texture. While most profiles exhibit a peak near the joint center, no further features can consistently be associated with the measured profiles for all the subjects. The joint is harder to identify using only this 1D information, compared to the 2D images. The transmittance spectra in the vicinity of the PIP joint were sampled from the square areas indicated in Fig. 7. In the resulting spectra (Fig. 9), the differences between the subjects become even more apparent. The observed transmittances vary by almost a factor of 10 between the subjects. However, when considering also the shapes of the spectra, additional difference become apparent: the ratios of the amplitudes of two most prominent peaks in transmittance, at approximately 700 nm and 800 nm, vary between the subjects. No correlation with sex or age was apparent with our sample size. Below 600 nm there is practically no signal, due to scattering and absorption in tissue.
In half of the subjects, a significant difference in absolute transmittance was observed also between the left and right finger. In some part, this can be explained by wrinkling of the skin above the joint. Also, it has been observed in previous work that the joint acts as a light guide, so the exact alignment of the joint during the measurement could also be important. Despite the differences in absolute transmittances, the spectral signatures were consistent between the left and right fingers for all subjects.

Simulations
The comparison of all measured transmittance spectra of the PIP joint and the simulated spectra for two different models of bone optical properties are shown in Fig. 10. It has been observed that the optical properties of bone, especially the absorption coefficient, have an important effect on the resulting spectra, and there are significant differences in this data available in the literature. In Fig. 10 simulation results are shown for bone optical absorption data from two references [24,25]. Both models produced spectral signatures, compatible with measurements, with the absolute transmittance values matching better for optical data from [24]. The latter data was used for all subsequent simulations shown. The light sampled in the PIP region must also pass a significant thickness of subcutis and perfused tissue. By considering optical absorption of subcutis ( Fig. 4(a)) and deoxygenated hemoglobin (see vein in Fig. 4(a)), which have peaks in the 700 and 800 nm region, the decreased transmission in that region can be explained.  The transmittance images obtained from the simulations are shown in Fig. 11. The outlines of the finger are obvious while the transmittance is large in parts near the edges of the finger, where total tissue thickness can be very small. Transmittance is clearly reduced for more central parts and varies with finger thickness. The simulated image of a healthy finger (Fig. 11(a)) is similar to the measurements (Fig. 6), the main difference being the fine texture of the skin surface that was not preserved with sufficient detail in the process of segmentation and voxelization of the MRI data. A feature of increased transmittance near the PIP joint is present in the healthy finger image. It is relatively small and harder to observe than in most experimental images. One possible reason for this difference could be inter-subject variability, with the finger used to construct the simulation model being close in anatomy to Subject 12 in Fig. 7. The subject of the MRI scan was unfortunately not available also for HSI imaging to test this. With the minimally affected M model ( Fig. 11(b)) the transmittance near the PIP joint is reduced. This is even more apparent for SF and SFM models (Fig. 11(c) and 11(d)), where the joint swelling increases even further the optical absorption.
is reduced. This is even more apparent for SF and SFM models (Fig. 11(c) and 11(d)), where the joint swelling increases even further the optical absorption. Fig. 11. Simulated 800 nm transmittance images for models of (a) healthy finger, (b) synovial membrane thickening, (c) synovial fluid effusion, and (d) synovial fluid effusion and synovial membrane thickening. Also shown as the black squares are the areas where the PIP transmittance spectra were sampled.
The spatial distribution of transmittance can be examined more closely by looking at longitudinal profiles, which are shown in Fig. 12 for the length of the finger near the PIP joint (center of projection is shown in Fig. 5). The curves are not smooth due to voxelization of the finger surface. For the healthy finger, a local peak in transmittance near the PIP center (Y = 54 mm) is now more apparent than it was in 2D image. The transmittance near the PIP joint is reduced for the three affected joints.  Fig. 13(a) shows the spectral dependence of transmittance in the PIP area for the four simulated models. The spectra feature tree maxima (at 720, 800 and 1090 nm) and two minima (at 750 and 970 nm). Differences between the models can be observed both in the absolute transmittances ( Fig. 13(a)) and spectral signatures (Fig. 13(b)). When comparing the affected models relative to the healthy model, most notable is the relative decrease in transmittance at lower wavelengths. The ratio of transmittances in 1090 nm and 720 nm peaks increases by 22% for M model relative to N model and 47% for SFM model compared to N model. It is of special interest that differences can be observed in the case of M model, where the total thickness of all tissues was the same as in the case of N model and most of the differences can be attributed to the changes in optical properties of the tissues involved in inflammation. For the SF and The spatial distribution of transmittance can be examined more closely by looking at longitudinal profiles, which are shown in Fig. 12 for the length of the finger near the PIP joint (center of projection is shown in Fig. 5). The curves are not smooth due to voxelization of the finger surface. For the healthy finger, a local peak in transmittance near the PIP center (Y = 54 mm) is now more apparent than it was in 2D image. The transmittance near the PIP joint is reduced for the three affected joints.  Figure 13(a) shows the spectral dependence of transmittance in the PIP area for the four simulated models. The spectra feature tree maxima (at 720, 800 and 1090 nm) and two minima (at 750 and 970 nm). Differences between the models can be observed both in the absolute transmittances ( Fig. 13(a)) and spectral signatures ( Fig. 13(b)). When comparing the affected models relative to the healthy model, most notable is the relative decrease in transmittance at lower wavelengths. The ratio of transmittances in 1090 nm and 720 nm peaks increases by 22% for M model relative to N model and 47% for SFM model compared to N model. It is of special interest that differences can be observed in the case of M model, where the total thickness of all tissues was the same as in the case of N model and most of the differences can be attributed to the changes in optical properties of the tissues involved in inflammation. For the SF and SFM models another notable difference is a relative decrease in transmittance at 970 nm minimum.
The transmittance values at wavelengths of the main peaks and most interesting transmittance ratios are summarized in Table 2.  Simulation results indicate, that a more sensitive indicator of the disease could be obtained by comparing the transmittance of the PIP joint to that obtained for the same finger over a region less affected. Figure 14(a) shows the ratio of transmittances sampled in the PIP and MID regions. The same analysis is also shown in Fig. 14(b) for the case when the DIP region (smaller, unaffected joint of the same finger) was used as reference instead of the MID region. The differences in the spectral signatures of this ratios are significant, especially for the PIP/MID ratio around 700 nm, where even SF and SFM models can be clearly distinguished, and around 970 nm, where the differences in the slopes of ratios between the healthy and affected models seem significant.

Discussion
Simulation studies, previously carried out, showed that indicators of RA inflammation are present in the spectral and spatial information available in the hyperspectral images of affected joints. In this work, hyperspectral imaging of PIP joints of 12 healthy volunteers was performed, with specific aims of assessing the inter-subject variabilities and to validate the simulation model.
The absolute transmittances of the PIP joints measured varied by a factor of almost 10 between the volunteers, as can be expected considering the differences in finger thicknesses. In general, the PIP joint could be identified in 2D images of fingers and its location associated with an area of increased transmittance. When the spatial information was reduced to 1D profiles, little consistency was observed between the fingers measured, beyond a peak of transmittance near the joint center, present in most profiles. The texture of the skin over the joint and exact orientation of the finger during imaging introduced additional variabilities, also observed between the left and right fingers of some subjects. The measured PIP transmittance spectra feature two peaks, one around 700 nm and the other at around 800 nm. Their amplitudes are approximately the same, with some inter-subject variability in the ratio, which is most likely due to differences in amount and composition of tissues, surrounding the joint. Inter-subject variability will be further investigated in a planned clinical study.
The simulation results for healthy PIP joint transmission spectra closely matched the experimental results. The optical properties of bone used for simulation proved to have an important effect on the transmission spectra. The three models simulated with the expected changes due to RA produced observable differences in spectral and spatial information. Most notable was the suppression of the spatial peak in transmission near the joint, consistent with the changes in spatial profiles observed in [15]. However, considering the experimentally observed inter-subject variability, a more sensitive indicator might be needed. One such candidate metric, presented here, is the spectral signature of the ratio of transmittances of the affected joint and over an area of finger, less affected by the disease, such as the middle phalanx or an unaffected DIP joint. Another possibility would be to use machine learning to identify features of inflammation from the whole hyperspectral information, which, however, will require a suitable learning dataset.
The spectra analyzed in this work were sampled over an area of the PIP joint, where the majority of the light passing the joint was observed. These averaged spectra unavoidably contain also contributions of tissues surrounding the joint that are not involved in the inflammation. If needed, the optical signal just from tissues affected by RA could be better isolated by other techniques, such as DOT [5][6][7].
Imaging conditions like pressure exerted on a finger, finger temperature or previous physical activities of a patient, can affect the physiological (e.g., blood perfusion) and anatomical parameters (e.g., thickness of soft tissues) of fingers. Therefore it is important that the imaging conditions are kept as constant as possible to provide reliable measurements.
The HSI system will be used in a planned clinical study to obtain more experimental data, also on inflamed joints. This will help explain some of the observed variabilities in the data collected so far and enable the development of methods for sensitive detection of early RA. Another possible way to reduce the inter-subject variability would be to perform the imaging with fingers immersed in refractive index matching liquid. Such approach was not included in this work due to the issues related to the use of liquids in clinical setting, such as sterility, scan time and patient comfort, but could be used to improve the results of this study in the future.

Conclusions
Hyperspectral imaging of proximal interphalangeal finger joint transmittance was performed for 12 healthy subjects. Peaking in transmittance was measured in the vicinity of the joint in the 600 nm -950 nm wavelength range. Large inter-subject variability was observed in the details of the spectral and spatial response. Simulations of optical transport of light in a human finger, with realistic geometry based on an in-vivo MRI scan, produced results compatible with experiments. Simulated models of joint inflammation suggest that sensitive indicators of disease are present in the hyperspectral data. The results of simulations are used to guide the design of a clinical study, planned for the fall of 2019, to obtain a larger dataset including joints diagnosed with joint inflammations.