Non-invasive detection of severe neutropenia in chemotherapy patients by optical imaging of nailfold microcirculation

White-blood-cell (WBC) assessment is employed for innumerable clinical procedures as one indicator of immune status. Currently, WBC determinations are obtained by clinical laboratory analysis of whole blood samples. Both the extraction of blood and its analysis limit the accessibility and frequency of the measurement. In this study, we demonstrate the feasibility of a non-invasive device to perform point-of-care WBC analysis without the need for blood draws, focusing on a chemotherapy setting where patients’ neutrophils—the most common type of WBC—become very low. In particular, we built a portable optical prototype, and used it to collect 22 microcirculatory-video datasets from 11 chemotherapy patients. Based on these videos, we identified moving optical absorption gaps in the flow of red cells, using them as proxies to WBC movement through nailfold capillaries. We then showed that counting these gaps allows discriminating cases of severe neutropenia (<500 neutrophils per µL), associated with increased risks of life-threatening infections, from non-neutropenic cases (>1,500 neutrophils per µL). This result suggests that the integration of optical imaging, consumer electronics, and data analysis can make non-invasive screening for severe neutropenia accessible to patients. More generally, this work provides a first step towards a long-term objective of non-invasive WBC counting.


Optical-device components
Our prototype used for video acquisition in ASCT patients consists of the following elements: 1. Imaging objective. Edmund Optics TECHSPEC 5X. Its features are a 5X magnification and a maximum numerical aperture of 0.15, reduced through a 3D-printed iris of 2.5 mm diameter to maximize depth of focus, a working distance of 16.2 mm, and a maximum FOV size of 1.8 x 1.32 mm. Its fixed-tube and total lengths are 50 mm and 93.81 mm, respectively. Its height, azimuth, and focal position are manually adjustable.
2. CMOS camera. Thorlabs DCC3240N. This camera is mounted to the objective and computer-powered through a USB connection. It comprises a global/rolling shutter. Its FOV is 1280 x 1024 pixels, or 1360 x 1088 um at 5X-magnification, corresponding to a pixel size of 1.0625 x 1.0625 um. Its frame rate is 60 FPS in full frame, which ensures sufficient temporal resolution to detect and track events given the 100-1,000-µm/s range of blood flow speeds in nailfold capillaries. 41,42 It can reach 229 FPS for FOVs restricted to 320 x 240 pixels. Its bit depth is 10 bits per pixel monochrome.
3. Illuminators. Two high-power Luxeon LEDs emit deep-blue light at 420 nm to maximize contrast between RBCs-appearing dark in our videos-and optical-absorption gaps. Each LED emits 161 Lumens at 700 mA using an aspheric collection lens with focal length of 20.1 mm, numerical aperture of 0.6, and an adjustable Thorlabs VA100C collimation slit. Rapidprototyped LED holders are cage-mounted with heat sinks at ~70-degree inclination from the detection axis.
4. Power driver. Drives both LED illuminators continuously at constant DC power level. 5. Disposable hand rest. Rigidly mounted 3D-printed platform used to hold the finger in a stable position for at least one-minute imaging. This platform comprises a one-size-fits-all finger well, with sets of additional padding inserts for smaller fingers. Optical-coupling oil (Johnson & Johnson, refractive index of 1.51) remains in the finger well.
6. Laptop and software. A laptop connected to the CMOS camera is used for power and video acquisition. Specifically tailored LabView software is used for video acquisition and storage. The output data collected for every patient consists of a set of uncompressed videos with time stamps that provide information on the exact acquisition times associated with each frame.

Motion compensation of capillary videos
Motion compensation was carried out automatically based on a specifically tailored algorithm, both implementations being done in MATLAB. Based on one raw video and one rectangular ROI (see the "Creation of capillary videos" section in the main Methods), we devised an algorithm to produce a motion-corrected capillary video-restricted to the user-specified ROIusing rigid registration; more specifically, it aligns all video frames with the first one, assuming that camera movements in the ROI are mere X or Y translations, excluding rotations. While frame movements in the raw video can involve long-range deformations, visual inspection of the registration results proved successful when applied to every capillary ROI separately. As a similarity metric and optimization criterion, our algorithm uses mutual information, which is a measure based on information theory that copes with slight contrast changes and guarantees accurate sub-pixel alignment. R1 Prior to this refined registration process, a coarse frame-prealignment step was performed to ensure a suitable initialization, based on cross-correlation analysis of pixel values and spatial gradients.

Event visualization on videos and ST maps
In order to best visualize and highlight the events labeled by the human raters in capillaries, we developed a method to simultaneously visualize the frames of the capillary video of interest and the corresponding ST-intensity profile of the capillary. This method allows events to be visualized both explicitly, i.e., as a moving object through the corresponding video frames, and as a fixed profile in their ST-map representation. In the context of our study, this visualization technique had allowed us to retrospectively analyze the distribution of labeled events in our videos as well as the relevance of majority rater agreement.
The concept of a ST map for capillary-flow visualization was proposed in previous literature. 18,39,R2 It is motivated by the fact that events can be conveniently observed in that representation: events associated with WBCs are expected to appear as thick, high-contrast, sparse, and unidirectional ST trajectories. R3 These properties also relate to the visual criteria that we defined for our raters, e.g., event brightness. ST maps make our rater-marked events appear as well-defined salient trajectories surrounded by a darker background (Fig. 5).
To create our ST maps, we followed an approach that extracts capillary brightness-as averaged over the cross-section-as a function of time and of the cumulative capillary length, based on segmented capillary boundaries. 39 In order to improve the visualization of event trajectories, we normalized the map values by subtracting their local temporal averages as obtained between 50 frames before and after every time point. We performed the initial capillary-segmentation procedure based on an image extracted from the corresponding registered video. Since the capillary profile may be incomplete in a single video frame due to the presence of absorption gaps in the flow, we chose to extract a temporally integrated image, based on an approach akin to that proposed by Allen et al. R4 for nailfold capillaries, where temporally variable features associated with the capillary flow were also enhanced to maximize contrast between the capillary and its surroundings. This approach relates to the concept of motion-contrast enhancement. R3 Specifically, the image that we used for segmentation was obtained through the integration of temporal-frequency components whose periods were empirically chosen to lie in the [0.25,1.5]second interval.
Capillary boundaries were first segmented manually, and then refined with the help of an activecontour technique. R5 The segmentation of both capillary boundaries was finally automatically resampled so as to include 1,000 points each, such that the center of every point pair at the same index of both boundaries lies on the medial axis of the capillary, where the medial axis is the loci of all circles inscribed in the capillary. Finally, based on this segmentation, separations between the arterial limb, venous limb, and loop of the capillary were defined on a case-to-case basis for visualization ( Supplementary Fig. S14).

Determination of the video acquisition time
The video-acquisition time t was set to one minute because it was practically suitable for clinical settings while remaining long enough to allow sampling substantially higher amounts of events N e in baseline cases compared to severe-neutropenia cases. Even under the shot noise that originates from the quantized nature of events, the count distributions associated with both cases are expected to be disjoint within at least one standard deviation of their means N b and N n , respectively, as expected from the calculations detailed below.
To validate this choice, we assumed a worst-case scenario, using a lower-limit case for baseline (C b = 1,500 neutrophils/µL) and a higher-limit case for severe neutropenia (C n = 500 neutrophils/µL), which is most difficult to discriminate since the difference between cell concentrations C from both categories is minimized. Typical values from the literature 27,41 were then assumed for capillary diameter (D = 15 µm) and flow speed (v = 800 µm/s), which allowed estimating expected amounts of events from concentrations. 33,39 Specifically, N e = π·(D/2) 2 ·C·v·t, which yields N b = 4.24 and N n = 12.72. Finally, the shot-noise statistics imply that counts vary around these means with standard deviations N b 1/2 = 2.06 and N n 1/2 = 3.57, respectively. This resulted in count ranges allowing for clear discrimination ( Supplementary Fig.  S15).

Supplementary figures
Fig. S1. Capillary selection from both experts in raw-video pair from Patient 01, region 1. The videos acquired in baseline and severe-neutropenic states are displayed. The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (C) lack of blood flow, and (D) out-of-FOV movements. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers).

Fig. S2. Capillary selection from both experts in raw-video pair from Patient 01, region 2.
The videos acquired in baseline and severe-neutropenic states are displayed. The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (D) out-of-FOV movements, and (E) occlusions. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers). The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (A) poor illumination, (B) lack of focus, (C) lack of blood flow, and (E) occlusions. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers). The videos acquired in baseline and severe-neutropenic states are displayed. The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (C) lack of blood flow, (E) occlusions, and (F) lack of clear morphology. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers). The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (A) poor illumination, (B) lack of focus, (C) lack of blood flow, (D) out-of-FOV movements, and (E) occlusions. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severeneutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers). The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to noncompliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers).

Fig. S7. Capillary selection from both experts in raw-video pair from Patient 05.
The videos acquired in baseline and severe-neutropenic states are displayed. The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (C) lack of blood flow, (D) out-of-FOV movements, and (E) occlusions. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers).

Fig. S8. Capillary selection from both experts in raw-video pair from Patient 06.
The videos acquired in baseline and severe-neutropenic states are displayed. The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (C) lack of blood flow, (D) out-of-FOV movements, and (F) lack of clear morphology. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers).

Fig. S9. Capillary selection from both experts in raw-video pair from Patient 07.
The videos acquired in baseline and severe-neutropenic states are displayed. The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (A) poor illumination, (B) lack of focus, (D) out-of-FOV movements, and (E) occlusions. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers). The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (D) out-of-FOV movements, (E) occlusions, and (F) lack of clear morphology. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers). The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (A) poor illumination, (B) lack of focus, (C) lack of blood flow, and (E) occlusions. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers). The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (D) out-of-FOV movements, and (F) lack of clear morphology. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers), discarding the rest of them (red numbers).
Fig. S13. Capillary selection from both experts in raw-video pair from Patient 11. The videos acquired in baseline and severe-neutropenic states are displayed. The green boxes outline the selected capillary pairs which, according to each of the two experts complied with the quality criteria for both baseline and severe-neutropenia acquisitions. The red regions/capillaries are discarded due to non-compliance with the quality criteria, i.e., due in this instance to (B) lack of focus, (C) lack of blood flow, (D) out-of-FOV movements, and (E) occlusions. The red lines/corners in the baseline videos outline the effective FOV outside which capillaries must be discarded as they disappear during several frames due to camera movements during acquisition. The yellow boxes outline capillaries that were initially selected in baseline but were discarded later due to non-compliance in the severe-neutropenia acquisition (G). Both experts made the selection process independently and, after that, only capillaries where both experts agreed were kept (black numbers).   and severe neutropenia (red dots) resulting from averaging across N = 1, 2, 3, 4, 5 capillaries per patient, when available. The corresponding cross-capillary variability is also shown for each patient (blue and red bars with notch extremes determined as q 2 ± 1.57(q 3q 1 ) / N 1/2 , where the q i are the respective quartiles, and where N is the amount of available paired data points). For every fixed amount N of capillaries per patient to be assessed, results were averaged over 10,000 trials with randomly picked sets of capillaries, so as to handle amounts of cross-patient capillary combinations that are otherwise intractable. (b) Receiver-operating-characteristic curves for classification of baseline vs. severe neutropenia based on integrating N = 1, 2, 3, 4, 5 capillaries per patient. The patient-level distributions of the resulting count values show that their discriminatory power increases with the amount of combined capillaries per patient. Specifically, the minimum area under curve consistently increases with the amount of combined capillaries from 0.68 to 0.84, 0.88, 0.95 and 1.00 when including 1, 2, 3, 4 and 5 capillaries, respectively.

Fig. S17. ST maps of capillaries with high versus low ratio ratios of majority events.
For each event, the first click from each of the three human raters is displayed with a red, blue, and green dot, respectively. Inter-rater agreement was higher in baseline capillaries (top map) compared to the case of severe neutropenia (bottom map), indicating that events in baseline correspond to more consistently observable physical phenomena. Counts of majority-rater events. Differences in capillary counts between baseline (blue dots) and severe neutropenia (red squares) displayed a substantially lower statistical significance in the single-rater case (Wilcoxon signed-rank test, N=53, P=0.04) than when based on majority events (P=5⋅ 10 -9 ). This indicates that events labeled by single raters are more prone to noise and contain less information than those where multiple raters agreed. Capillary pairs are grouped by patient ID.  Counts of majority-rater events. The median numbers of events observed per minute, when averaging all available capillaries per patient, are shown for baseline (blue dots) and severe neutropenia (red dots) for the 22 acquired video sets and 11 patients of our study. The corresponding cross-capillary variability is also shown for each patient (blue and red bars with notch extremes determined as q 2 ± 1.57(q 3 -q 1 ) / N 1/2 , where the q i are the respective quartiles, and where N=11 is the amount of paired data points). For single-rater events, differences in patient counts between baseline and severe neutropenia did not reach statistical significance (Wilcoxon signed-rank test, N=11, P=0.17), with a median difference of only eight counts per minute. For majority events, differences in patient counts did reach statistical significance (Wilcoxon signed-rank test, N=11, P=0.00097), with a median difference of 25 counts per minute.The optimal thresholds to separate baseline from severe neutropenia are shown as dotted black lines. The X-axis is labeled with the patient IDs together with their amount of analyzed capillaries (brackets). The median amount of capillaries used per patient was four. (b) Diameter of that single capillary before (blue) and after (red) chemotherapy, taking the apex (blue and red circles respectively) as the reference point. (c) Histogram of all capillary diameters used in our study before (N=53, in blue) and after (N=53, in red) chemotherapy. For each capillary, the diameter was sampled in 1,000 locations along its length. The median diameter value was then determined for every capillary to generate the histograms before chemotherapy (baseline) and after chemotherapy (severe neutropenia). No statistically-significant variation in capillary diameter was observed between the two time points (Wilcoxon signed-rank test, N=53, P=0.504).

Fig. S22. Median number of majority events after performing a calibration at baseline. (a)
For every patient, the capillary pair with least majority-event counts at baseline was removed. (b) For every patient, only the capillary pair with most majority-event counts at baseline was retained. In both cases, the separation between baseline (blue dots) and severe neutropenia (red dots) increased compared to the non-calibrated results (Fig. 7), and a perfect classification was achieved. The blue and red bars show the variability in the number of majority events for each patient across each individual capillary. The dotted black lines show the thresholds used to separate baseline from severe neutropenia. The X-axis is labeled with the patient IDs together with their number of available capillaries (in brackets).