Exoscope-based videocapillaroscopy system for in vivo skin microcirculation imaging of various body areas.

The capillary system immediately responds to many pathologies and environmental conditions. Accurate monitoring of its functioning often enables early detection of various diseases related to disorders in skin microcirculation. To expand the scope of capillaroscopy application, it is reasonable to visualize and assess blood microcirculation exactly in the areas of inflamed skin. Body vibrations, breathing, non-flat skin surface and other factors hamper the application of conventional capillaroscopes outside the nailfold area. In this paper, we propose an exoscope-based optical system for high-quality non-invasive computational imaging of capillary network in various areas of the body. Accurate image matching and tracking temporal intensity variations allow detecting the presence of blood pulsations, precise mapping of capillaries and photoplethysmogram acquisition. We have demonstrated the efficiency of the proposed approach experimentally by in vivo mapping and analysis of microvessels in wrist, forearm, upper-arm, breast and hip areas. We believe that the developed system will increase the diagnostic value of video capillaroscopy in clinical practice.


Introduction
Diseases of various etiologies may affect cutaneous microcirculation and result in skin inflammations [1][2][3][4]. Capillaries, which are the smallest and most numerous blood vessels in the human body, respond to various pathologies much earlier than arteries and veins, changes in which indicate that the disease is already in development. Accurate monitoring of capillary system functioning often enables in vivo detection of the disease at early stage [5][6][7]. Quantitative parameters of the microvasculature may be the measure of the pathological processes and treatment efficiency.
Main approaches to non-invasive microcirculation imaging are optical microscopy [8][9][10], laser Doppler perfusion imaging [11,12], laser speckle contrast imaging [13,14] and optical coherence tomography [15]. Advantages and drawbacks of these approaches as well as their clinical applications are well analyzed [16][17][18]. In practice, due to body vibrations, breathing, non-flat skin surface and other factors, microcirculation imaging is now implemented mainly in the nailfold area where capillaries are arranged parallel to the skin surface [19][20][21][22]. It enables assessing their density, shape, dimensions, color and other parameters as well as blood counting [23] and tracking individual red blood cells [24]. To expand the scope of the capillaroscopy application, it is necessary to visualize and to quantify the skin microvasculature exactly in the inflammation area, which may be located in arbitrary region of the human body. For this purpose, specialized tools are in need. In [25], it was demonstrated that imaging system based on two photographic objectives allows video capillaroscopy outside the nailfold area but its applications are still limited to the inspection of the most accessible areas: forearm, cheeks and forehead. This system is quite cumbersome and is barely applicable in non-vertical position.
We propose an exoscope-based concept of the optical system for high-quality computational imaging and mapping of capillary network in various areas of the body. The system utilizes thin and rigid lens probe, which may be easily positioned at any angle and distance to the inspected body area (Fig. 1). For vessels visualization, a stack of images collected by the exoscope is processed to track and quantify temporal changes of the light intensity in each pixel. In this paper, we derive main equations for optical system design, describe the developed experimental set-up and image processing pipeline, demonstrate the efficiency of the proposed approach by multiple experiments, discuss its further improvement and potential applications for video capillaroscopy and photopletismography measurements.

Optical design
The exoscope has to provide a magnified digital image of the inspected microvasculature located in the skin sub-surface layer (dermis). The skin layer between exoscope and the vessels includes epidermis and partly dermis [26]. To align the skin surface and to make it flat, we install a glass plate 2.5 mm thick and 15 mm in diameter in front of the exoscope. With respect to the refractive indices of skin layers (1.36 -1.41) and this glass (1.52), the working distance s of the exoscope should be at least 5 mm (in air).
Proposed exoscope concept is shown in Fig. 2(a). The imaging part consists of the objective, relay lenses, eyepiece, coupler and the image sensor (IDS uEye UI-3060CP-M-GL Rev.2, 1/1.2", 1936 × 1216 pixels, ∆ = 5.86 µm pixel size). The objective forms the skin image that is transferred by the relay lenses to front focal plane of the eyepiece. The eyepiece than forms a magnified infinity-located image that is acquired by the CMOS image sensor) with a coupler lens. The illumination part includes green LED light source (central wavelength λ = 520 nm, FWHM = 30 nm, power 5 W) and light guide. To avoid specular back reflection reflectance and enhance the image of sub-epidermal vessels, we have inserted two crossed film polarizers.
Typical diameter of the small continuous capillaries is 7-10 µm [27]. For reliable detection and quantitative characterization of capillaries using the algorithm described below, the image scale has to be at least 1 pixel per capillary diameter. With regard to aberrations, misalignments and other factors, the magnification should be at least 5 pixels per capillary, e.g. each pixel size in object space (at the capillary level) should be about 2 µm. It means that the image of M × N = 1000 × 1000 pixels corresponds to the skin area of x ob × y ob = 2 × 2 mm 2 . In other words, total optical magnification in this case has to be G = y im /y ob = M·∆/y ob = 2.9.
The optical scheme of the exoscope may be represented as three sequential projection components: objective lens OL, set of relay lenses RL and output system OS consisting of the eyepiece and the coupler ( Fig. 2(b)). Relay lenses have magnification G RL = 1, i.e. do not contribute to the total magnification G = G OL G OS of the exoscope. Magnification of the objective lens is G OL = l/p. With respect to lens formula 1/f OL = 1/l -1/p, it may be written as G OL = f OL /(f OL +p). Magnification of output system is defined by the ratio of coupler and eyepiece focal distances: G OS = f C /f E . Therefore, total magnification of the exoscope may be written as Distance p is a few mm longer than the working distance s calculated from the protective glass. Thus, Eq. (1) shows the dependence of the required magnification G and working distance s on the focal distances f OL , f E and f C of the main components. In our system, objective lens is a combination of an achromatic doublet and a plano-concave lens as a protective glass providing f OL = 6.2 mm, eyepiece is two-lens achromat with f E = 12 mm. To achieve the magnification G = 2.9 mentioned above, we have installed the machine vision lens with f c = 75 mm as a coupler. This lens is interchangeable and may be adapted with respect to required G.

Experimental set-up
Appearance of the set-up is shown in Fig. 3(a). Objective, relay lenses, eyepiece and light guide are packed in the metal tube 10 mm in diameter and 150 mm length. For stable three-dimensional positioning of the probe, we installed the exoscope in the articulating arm of a mounted stand. Thus, the set-up provides easy and fast imaging of the skin in almost any area of the body, for example, forearm ( Fig. 3(b)) and back (Fig. 3(c)). Before the experiments, we have estimated the image quality provided by the developed system. To assess the optimal working distance and depth of field of the assembled prototype of exoscope, we have measured full width at half maximum (FWHM) of the line spread function (LSF) using the modified slanted-edge method [28]. For this purpose, we have acquired the images of the chessboard pattern at different working distances and selected several test zones across the whole field of view at each of them (Figs. 4(a), 4(b)). The calculated LSF FWHM differ from zone to zone due to the residual field curvature aberration and the misalignment of the optical elements ( Fig. 4(c)). For the working distances s in the range 3.3-6.3 mm, mean values of LSF FWHM were calculated. From Fig. 4(d), we can see that optimal working distance is s = 5 mm in the air, i.e. 3.3 mm in the medium with n = 1.5 refractive index. The set-up provides ≥150 µm depth of field (measured at level of 5 pixels LSF FWHM) across the whole field of view and at working distances in the range s = 3.3-6.3 mm (2.2-4.2 mm in medium).
The obtained LSF FWHM may be compared to the diffraction limited resolution that we tried to achieve at the optical design stage. With respect to the exoscope entrance pupil diameter D = 4 mm, the LED central wavelength λ = 0.52 µm and the objective lens focal length f OL = 6.2 mm, we may estimate the size of the best-focused spot as the Airy disk diameter multiplied by the magnification of the output system G OS = 6: and is equal to 5.7 µm which is close to the pixel pitch ∆ = 5.86 µm of the camera. The condition δ ≈ ∆ is necessary to provide the required spatial resolution ∆/ G = 2 µm at the capillary level. From Fig. 4(d), we may see that experimentally measured average resolution at the optimal working distance s = 5 mm is about 5 pixels (10 µm at the capillary level) which is close to the diameter of a single capillary. Due to optical aberrations, imperfect assembly and residual misalignment of the set-up, it is worse than the diffraction limited resolution but is enough for reliable detection of capillaries using the proposed image processing pipeline presented below.

Image acquisition and processing
During each experiment we acquired K = 3000 monochrome 12-bit images A k (x,y) of 1000×1000 pixels at 50 fps within 60 seconds. Figure 5 illustrates the main stages of the image processing pipeline. Due to illumination non-uniformity, body vibrations and breathing, the images ( Fig. 5(a)) should be corrected and matched before joint processing. Calibration procedure is based on the acquisition and processing of N = 10 images B n (x,y) (n = 1,2..N) of uniform test-object -a flat white diffusely reflecting plate located at the working distance of the exoscope. We obtain these images at different lateral positions of the test-object and then average them: y). To get the sensor sensitivity map D(x,y) ( Fig. 5(b)), we subtract low-frequency components related to illumination non-uniformity using a Gauss filter with σ = 20: This allows calculating the corrected image stack I k (x,y) (Fig. 5(c)): To align background, we again eliminate low-frequency components by subtracting a smoothed image G from each image I k of the stack and normalizing the intensity (Fig. 5(d)): where G is a low-frequency Gauss filter, σ is a half-size of the averaging neighborhood, C is a contrast coefficient. Adding 127 is necessary for shifting the mean value to the middle of 8-bit intensity range. To ensure pixel-to-pixel matching of all images, we calculate local motion vectors and matched the images J k with respect to their directions and lengths. For this purpose, we apply the global image stabilization based on phase correlation with reference frames and relative shift compensation [29]. Then, local matching is carried out. For this purpose, first image is divided into blocks whose displacements are estimated by the criterion of the minimum root mean square deviation. Interpolation of these displacements in each pixel allows calculating a continuous coordinate translation matrix. Applying this matrix to the original images J k (x,y) provides compensation of local displacements [30]. Figure 5€ shows the first image from a locally matched stack with crosses indicating the positions of each grid knot in the first (black) and the last (white) image of the sequence. After cropping the common area of overlapped images, we obtain the well-matched and intensitycorrected spatiotemporal data cube I k (x,y) (Fig. 5(f)). In this cube, each image pixel x,y contains temporal dependences of the signal I k = I(t k ) reflected from the inspected skin area. The capillaries and veins differ from the vessel-free surrounding by the presence of blood flow changes. Plotting the spectrum of the temporal signal I(t k ) using Fourier transform allows detection of the dominant frequencies. After the intensity deviation values in each image pixel are obtained, we subtract the blood-free background and consider only the pixels with blood flow related to significant intensity oscillations in the range 0.5-20 Hz. From the series of such blood flow images (Fig. 5(h)), we calculate the vessel image containing the intensities of blood volume changes as the ratio of high-and low-frequency spectral components in the Fourier spectrum ( Fig. 5(g)). A calculated local shifts map (Fig. 5(e)) also allows photoplethysmogram calculation (Fig. 5(j)) by subtracting the low-frequency component from the temporal signal in the selected area (Fig. 5(i)). It illustrates blood volume changes in the inspected skin area and is important for pulse oximetry, cardiovascular activity characterization, breathing monitoring and other tasks [31,32].
For demonstration of the proposed concept, we have carried out multiple vascular imaging experiments using the described set-up in Institute of Immunology (Moscow). To confirm its versatility and efficiency, we have tested it in the nailfold and wrist areas available for conventional video capillaroscopy techniques [19][20][21][22] as well as in breast, back and other areas where these techniques are barely applicable. Rigid-probe-based exoscope mounted in the articulating arm provides reliable and high-quality skin image acquisition when patient is in a sitting or lying position. Presented algorithm for processing such images allows accurate compensation of the body motions and breathing, detection of the blood pulsations and vessel mapping. Figure 6 illustrates the examples of the obtained and enhanced images, calculated vessel maps and their overlay in the nailfold (male, 31 years), wrist (female, 21 years), forearm (female, 44 years), upper arm (male, 36 years), hip (female, 24 years) and right breast (female, 48 years) areas of the people without vascular or inflammatory diseases.

Discussion
Continuous observation and analysis of a capillary network is a powerful approach to the analysis of diseases progression [27,33], studying the effectiveness of drugs [34][35][36] and solving other biomedical tasks. Capillaroscopy techniques are especially effective when the inspection is carried out right in the neoplasm area [37,38] as well as when functional tests (cold [36], occlusion [39], etc.) are applied. The described exoscope-based concept allows to access almost any skin area and to visualize functional microvascular network. By calculating proper parameters (field of view, working distance, magnification and resolution) of the optical system, it may be optimized for the inspection of a particular skin area with maximal performance. Further improvement of the set-up may also include a better aberration correction for higher spatial resolution as well as light throughput increase for higher image acquisition frame rates. For this purpose, a more powerful light source, fibers of a larger diameter or installation of LEDs at the probe's distal end are necessary. For hard-to-reach skin areas inspection, the relay lens (Fig. 2) might be substituted with a flexible imaging fiber bundle.
Spatiotemporal processing of the acquired images is not limited to mapping of the microvessels distribution and photoplethysmography described in this paper. Advanced analysis of the obtained data may include blood vessel morphology study, microcirculation assessment and other methods for quantitative characterization of capillary function and structure under various conditions and diseases.

Conclusion
We have presented an exoscope-based concept for visualization and quantification of skin microvessels in various body areas. Accurate image matching and tracking temporal intensity variations in each pixel allow detecting the presence of blood pulsations and precise mapping of capillaries. This computational imaging approach does not require the optical system performing at a diffraction-limited specification and is based on processing a stack of multiple images. Its effectiveness has been demonstrated on a few different areas of the human body. Due to nice collection of optical and image processing features, the proposed approach provides high-quality vessel mapping even in the presence of hair, slight body vibrations and breathing. Described image processing algorithm enables a reliable visualization of the capillary map and studying its local morphological features. Obtained results may be applied for diagnosis and treatment of inflammatory and other diseases associated with disorders of blood microcirculation.