Rolling shutter speckle plethysmography for quantitative cardiovascular monitoring

We propose a new speckle-based plethysmography technique, termed rolling shutter speckle plethysmography (RSSPG), which can quantitatively measure the velocity and volume fluctuations of blood flow during the cardiac cycle. Our technique is based on the rolling shutter speckle imaging, where the short row-by-row time differences in the rolling shutter image sensors are used to measure the temporal decorrelation behavior of vertically elongated speckles from a single image capture. Temporal analysis of the speckle field provides rich information regarding the dynamics of the scattering media, such as both the dynamic scattering fraction and the speckle decorrelation time. Using a sequence of images, RSSPG can monitor fluctuations in the blood flow dynamics while separating velocity and volume changes in blood vessels and obtaining high-quality plethysmography waveforms compared to regular photoplethysmography. We demonstrate the quantitative RSSPG based on accurate fitting of the speckle dynamics model, as well as the qualitative RSSPG based on simple row-by-row correlation (RIC) calculation for fast and robust analysis. Based on exploratory in vivo experiment, we show that RSSPG can reliably measure pulsatile waveforms and heart rate variations in various conditions, potentially providing physiologically relevant information for cardiovascular monitoring.


Introduction
Plethysmography is a method used to detect volume fluctuations in various body parts, primarily attributable to blood flow, offering valuable insights into detecting cardiovascular pulse waves [1].Photoplethysmography (PPG), initially proposed in the 1930s [2,3], is a light-based, non-invasive plethysmography technique that has been widely used for cardiac monitoring in both clinical and consumer electronic devices.By utilizing the difference in absorbance between blood and neighboring tissue, PPG waveforms are obtained to monitor fluctuations in transmitted or reflected light intensity corresponding to changes in blood volume.PPG has been applied to various medical practices, such as physiological monitoring, vascular assessment, and autonomic function [4].However, despite its simplicity and direct approach, PPG is vulnerable to significant vasoconstriction [5], along with a relatively poor signal-to-noise ratio (SNR), as it relies on vasomotion rather than directly measuring blood flow itself [6,7].
As an alternative light-based plethysmography approach, speckle plethysmography (SPG) has been developed, utilizing the changes in the random speckle pattern generated from living tissue to measure blood-flow dynamics.SPG reports have utilized spatial laser speckle contrast imaging (sLSCI) technique [8], where the speckle patterns are captured with a camera using finite exposure time, and the spatial contrast values of the captured random speckle interference image patterns are used to quantify the relative velocity of motions of moving scatterers in living tissue, e.g., blood cells [9].The blood flow index (BFI), defined as the square of reciprocal of average speckle contrast in the image, is observed in time-lapse at a sufficiently high framerate to observe fluctuations of local blood flow during a cardiac cycle [9].sLSCI offers single-shot flow measurement, which can be easily adapted to sequential measurements for monitoring pulsatile blood flow with relatively simple processing of raw speckle images, compatible with any type of camera [10].Compared to the intensity-based measurement of conventional PPG, SPG can not only maintain robust SNR under substantial vasoconstriction but also demonstrate higher correlation with electrocardiogram and improved precision in heart rate variability [9,11].However, SPG based on sLSCI is still limited in that it lacks the ability to distinguish dynamic scattering in the presence of static scattering, thus making it difficult to measure relative flow rate changes in complex scattering media such as human tissue [12,13].
In the context of speckle-based blood flow imaging, analyzing the dynamic speckles in the temporal domain provides a more precise and robust flow measurement.For example, temporal LSCI can suppress the effect of static scattering [14][15][16], and sLSCI, using multiple exposure times, can discern more complex temporal decorrelation behavior of the mixture of dynamic and static scattering [17,18].Direct temporal measurement of dynamic light scattering allows precise measurement of the decorrelation behavior of the speckle intensities; however, considering the speed of blood flow in living tissue, high-speed cameras exceeding 10000 fps are required to image spatially-varying blood-flow dynamics in vivo [19,20].
Recently, a single-shot temporal speckle measurement technique named Rolling Shutter Speckle Imaging (RSSI) has been developed, which leverages the small row-by-row time differences in the rolling shutter image sensor to capture speckle intensity at different time points in a single image capture [21].RSSI uses an elliptical aperture to elongate the speckle patterns across multiple rows of the image sensor, where the local temporal speckle decorrelation time can be measured from the row-by-row intensity correlation (RIC)s of the single speckle image.RSSI constructs a comprehensive spatiotemporal intensity correlation model to fit the RIC curves and obtain the scattering parameters such as speckle decorrelation time and the fraction of dynamic scattering.Via temporal analysis, RSSI can effectively decouple speckle decorrelation time, which is indicative of the average velocity of the moving scatterers (e.g.blood cells in tissue), and fraction of dynamic scattering, which represents the ratio of static and dynamic scattering in the scattering volume, while other single shot speckle contrast-based methods cannot distinguish between velocity and volume changes in scattering media.Compared to other temporal measurement techniques, RSSI features less data and low-cost hardware including an economical video-rate rolling shutter camera, which coupled with the precise theoretical model, ensures cost-effective, quantitative measurement and long-term monitoring of in vivo scattering dynamics.
In this paper, we propose a temporal speckle plethysmography technique based on RSSI, termed rolling shutter speckle plethysmography (RSSPG), for continuous measurement of quantitative blood flow variations for cardiac monitoring.A sequence of speckle images is used to monitor fluctuations in the blood flow velocity and the fraction of dynamic scattering, providing rich information about blood perfusion during the cardiac cycle.Unlike conventional SPG, RSSPG can obtain an absolute value of speckle decorrelation time τ c , which can further be converted into the blood flow velocity.This remains effective in the presence of static scattering, by leveraging the derived fraction of dynamic scattering, ρ.RSSPG signals measured in a non-contact manner from a fingertip show clear pulsatile waveforms with high SNR and successfully measure variations of the heart rate and the fluctuations in the blood flow rate in various physiological conditions.In addition to the quantitative speckle dynamics measurement from the model fitting, we also demonstrate a fast and motion-robust data processing technique based on decorrelation estimation, where speckle correlations in two sLSCI image rows are used to estimate initial speckle decorrelation behaviors quickly.To the best of our knowledge, our work represents the initial effort to integrate temporal laser speckle contrast analysis into plethysmography.This integration potentially decouples the velocity and volume changes in the blood vessels during the cardiac cycle, enhancing the clinical significance of plethysmographs in estimating other cardiac or vascular measurements.In the following sections, we provide a detailed exposition of our data acquisition and processing methods and analyze the resulting waveforms of our technique.

Theory
Our RSSI system is implemented with a 4-f optical system and an elliptical aperture (Fig. 1(a)).From the elongated speckle image, RSSI quantifies the intensity correlation between two distinct rows in the image, each captured at different times in a rolling shutter sensor.The role of an elliptical aperture is to extend the spatial speckle correlation length to investigate extended decorrelation behaviors across multiple rows of the image.We chose an elliptical aperture to easily calculate the corresponding speckle sizes based on its well-known transfer function; however, any asymmetric apertures can be used to generate elongated speckles for RSSI.Thus, modeling the correlation of different rows in RSSI must take account of both temporal decorrelations and spatial decorrelations of the speckle field in two image rows.Once the speckle image is captured, the row-by-row speckle intensity correlation is measured from the raw speckle images as follows: where ⟨⟩ indicates the ensemble averaging over all pixels in the window.I(i) and I(i + n) represent the pixel intensities at the (i) th and (i + n) th rows in the image, respectively, with a row difference of n.The value n ranges from 0 to the length of the elongated speckles.The mean speckle intensities can be measured by spatially averaging the pixel intensities at a large enough window size (Fig. 1(b)), assuming ergodicity in the local speckle field.From the row intensity correlation in Eq. ( 1), the dynamics of the speckle field can be extracted by modeling the spatiotemporal speckle intensity correlations and fitting RICs to the model.Speckle intensities separated by n rows in RSSI correspond to spatial displacement of ν = pn and temporal displacement of τ = t r n, where p and t r are the pixel size and the row time, respectively.To model the intensity correlation, both spatial field correlation g A (ν) and temporal field correlation g 1 (τ) must be considered [21].With an elliptical aperture in the imaging system, the spatial correlation is written as follows: where somb is a sombrero function defined by the Bessel function of the first kind, NA is the vertical numerical aperture of the elliptical aperture, and λ is the wavelength of the coherent light source.Temporal field correlation, g 1 (τ), is typically modeled with a negative exponential function.According to the Siegert relation, the spatiotemporal intensity correlation function for the elongated speckle field can be written as: where β s is a parameter for source coherence and spatial averaging.Note that Eq. ( 3) is defined for instantaneous measurement without consideration of detector integration time.The general spatiotemporal intensity correlation model with finite exposure time and the static scattering any type of camera [10].Compared to the intensity-based measurement of conventional PPG, SPG can not only maintain robust SNR under substantial vasoconstriction but also demonstrate higher correlation with electrocardiogram and improved precision in heart rate variability [9,11].
However, SPG based on sLSCI is still limited in that it lacks the ability to distinguish dynamic scattering in the presence of static scattering, thus making it difficult to measure relative flow rate changes in complex scattering media such as human tissue [12,13].
In the context of speckle-based blood flow imaging, analyzing the dynamic speckles in the temporal domain provides a more precise and robust flow measurement.For example, temporal LSCI can suppress the effect of static scattering [14][15][16], and sLSCI, using multiple exposure times, can discern more complex temporal decorrelation behavior of the mixture of dynamic and static scattering [17,18].Direct temporal measurement of dynamic light scattering allows precise measurement of the decorrelation behavior of the speckle intensities; however, considering the speed of blood flow in living tissue, high-speed cameras exceeding 10000 fps are required to image spatially-varying blood-flow dynamics in vivo [19,20].
Recently, a single-shot temporal speckle measurement technique named Rolling Shutter Speckle Imaging (RSSI) has been developed, which leverages the small row-by-row time differences in the rolling shutter image sensor to capture speckle intensity at different time points in a single image capture [21].RSSI uses an elliptical aperture to elongate the speckle patterns across multiple rows of the image sensor, where the local temporal speckle decorrelation time can be measured from the row-by-row intensity correlation (RIC)s of the single speckle image.RSSI effect is derived as [22]: ) where T is the exposure time, τ c is the decorrelation time, and ρ is the ratio of dynamic scattering to the total scattering, i.e., ρ = ⟨I d ⟩ ⟨I d ⟩+ ⟨I s ⟩ where ⟨I d ⟩ and ⟨I s ⟩ represents the average intensities of the dynamic and static scattering elements within the speckle field, respectively.The spatiotemporal intensity correlation model in this work is extended from Ref. [21] to fit decorrelation model for all correlation times including the interval n< T t r , and the sombrero function is coupled to the entire field correlation term in Eq. ( 4).(See Ref. [22] for detailed derivation).
By fitting the model to the row-by-row intensity correlation of elongated speckle images, we can obtain τ c , the speckle decorrelation time, representing the speed of the movement within the scattering media (Fig. 1(c)) and ρ, the fraction of dynamic scattering from total scattering.Notably, the model accommodates the presence of static scattering in the equation.This allows the model to measure τ c from the dynamic scattering component only, independent of the amount of static scattering present in the object.By averaging the post-processed τ −1 c values over time, a time series RSSPG signal can be obtained (Fig. 1(d)).The average τ c values over each frame represent the total flow dynamics of ROI at the instance.The time series RSSPG τ −1 c waveform is plotted in Fig. 2(a) with marked peaks and notches.The representative row intensity correlation curves corresponding to the marked RSSPG regions are illustrated in Fig. 2(b).

Setup
Our experimental setup of RSSPG to measure in vivo plethysmography waveforms is illustrated in Fig. 1(a), which is identical to the proposed RSSI system with modifications in magnification and field of view (FOV) [21].The optical components consist of a 35 mm focal length tube lens and a 50 mm focal length objective lens to implement the 4-f system with 3.42 mm x 3.42 mm FOV.
An elliptical aperture designed and 3D-printed with 0.545 mm in vertical radius and 8.18 mm in horizontal radius that correspond to speckle length of 2-by-30 pixels is inserted in the Fourier plane of the imaging system between the two lenses.A volume holographic grating-stabilized laser diode (LD785-SEV300, Thorlabs) illuminates coherent light, operating with a wavelength of 785 and a power density under 200 mW/cm 2 .Raw speckle image sequences are recorded by a monochrome industrial camera (DMK 37AUX178, Imaging Source) equipped with a rolling shutter CMOS sensor that provides a pixel size of 2.4 m × 2.4 m.

Setup
Our experimental setup of RSSPG to measure in vivo plethysmography waveforms is illustrated in Fig. 1(a), which is identical to the proposed RSSI system with modifications in magnification and field of view (FOV) [21].The optical components consist of a 35 mm focal length tube lens and a 50 mm focal length objective lens to implement the 4-f system with 3.42 mm x 3.42 mm FOV.An elliptical aperture designed and 3D-printed with 0.545 mm in vertical radius and 8.18 mm in horizontal radius that correspond to speckle length of 2-by-30 pixels is inserted in the Fourier plane of the imaging system between the two lenses.A volume holographic grating-stabilized laser diode (LD785-SEV300, Thorlabs) illuminates coherent light, operating with a wavelength of 785nm and a power density under 200 mW/cm 2 .Raw speckle image sequences are recorded by a monochrome industrial camera (DMK 37AUX178, Imaging Source) equipped with a rolling shutter CMOS sensor that provides a pixel size of 2.4 µm × 2.4 µm.

Data acquisition
Time-lapse raw speckle images of 1000-by-1000 pixel region of interest (ROI) were captured as illustrated in Fig. 1(b), with 20 µs row time and 20 µs exposure time, and a frame rate of 140 fps.The experiment simultaneously measured pulsatile blood flow waveforms from the RSSPG setup and a commercial PPG sensor (Ubpulse360, LAXTHA).In vivo data was acquired by measuring changes in blood flow waveforms under baseline, post-workout, and cooling conditions.Simultaneous recordings of RSSPG and PPG signals were obtained by attaching the PPG sensor to one index finger while placing the other hand on a table to illuminate the laser precisely under the nail of the opposite index finger.The workout included a 5-minute running exercise, while the cooling condition entailed a 3-minute cold pressor.During the cold pressor challenge, we placed a towel filled with ice around the index finger of the subject.For each condition, the subject relaxed for the following 10 minutes without relocating the measurement site in order to obtain the baseline blood flow waveforms.Also, we conducted a representative experiment to acquire RSSPG, SPG, and PPG waveforms simultaneously.The study was in accordance with an approved protocol by the Yonsei IRB, and participants provided informed consent (7001988-202312-HR-1936-04).

Speckle image processing
The data processing procedures of RSSPG are adapted from RSSI to process a sequence of images captured at 140 fps.Each frame of the captured raw speckle image is processed by a 50-by-50 pixels sliding window and a step size of 10 pixels (Fig. 3(a)).The window size was determined in consideration of the size of the speckles.Each window comprises enough speckles to successfully represent the statistics of the local speckle field via ensemble averaging while also maintaining spatial resolution to distinguish the spatial variations in the blood flow dynamics within the imaging FOV (Fig. 1(c)).The decorrelation measurement is efficiently obtained by a fast Fourier transform (FFT)-based data processing algorithm (Fig. 3(b)).To begin with, the algorithm computes a row correlation of the entire frame by multiplications in the frequency domain.Then, the normalized RICs are obtained by dividing the term with the mean intensity according to Eq. ( 1).The computation of the row intensity correlations by Nvidia RTX A6000 (10752 CUDA cores, 48GB GDDR6) takes 1 ms for each frame.Following the row intensity correlation computation is a curve fitting process to estimate the flow dynamic parameter τ c , the normalization coefficient β s , and the static scattering ratio ρ.We have used the Scipy library in Python to apply the nonlinear least square curve fitting.site in order to obtain the baseline blood flow waveforms.Also, we conducted a representative experiment to acquire RSSPG, SPG, and PPG waveforms simultaneously.The study was in accordance with an approved protocol by the Yonsei IRB, and participants provided informed consent (7001988-202312-HR-1936-04).

Speckle Image Processing
The data processing procedures of RSSPG are adapted from RSSI to process a sequence of

Quantitative RSSPG signal processing
From each speckle image, 96-by-96 pixel row correlation curves are obtained, which can be individually fitted to obtain 2D maps of τ c and ρ (Fig. 3(c)).To extract a quantitative RSSPG signal with high SNR, the time-lapse maps of τ c and ρ must be properly handled.While the curve fitting method efficiently disentangles τ c and ρ, there are additional errors and noises arising from the fitting process, and overall averaging of the parameters may not effectively remove the effect of these errors.In the case of local speckle patterns with purely static scattering, the estimated τ c is prone to a significant error.Similarly, decorrelation curves that deviate largely from the theoretical model must be excluded due to measurement errors caused by motion artifacts.To handle these errors, we locally select the region of interest within the entire imaging FOV with acceptable fitting results by analyzing the ρ map and selecting the regions with intermediate ρ values between 0.1-0.7,which typically rejects about 50% of pixels in the imaging ROI (Fig. 3(c)).We select the same corresponding region from both ρ and τ c map to compute average <τ −1 c > and average <ρ> values in each frame.Finally, the RSSPG signals are low pass filtered at 10 Hz (Butterworth filter) to remove high-frequency temporal fluctuations.

Qualitative RSSPG estimation for fast processing
While the model-based quantitative analysis of the RSSPG signal allows for the extraction of scattering parameters that represent blood flow rates and blood volume fraction fluctuations, the curve fitting and 2D filtering processing require computation, which may hinder the real-time processing of the data at a high frame rate.For fast analysis of RSSPG data, we developed a fast estimation method.This method only utilizes the first two data points of the row correlation curve to indicate temporal decorrelation behavior without fitting the model into the data.In the row intensity correlation equation, g r (0) represents the intensity autocorrelation of the speckles with zero lag, essentially quantifying the speckle contrast of the raw speckle image.g r (1) denotes the intensity correlation between each adjacent row in the raw speckle image (Fig. 3(d)).We note that the difference between g r (0) and g r (1), defined as ∆g r , signifies the initial drop in the intensity correlation due to the temporal decorrelation of the speckle field, encompassing information of fast-decorrelating or highly correlated speckles.By employing the subtraction procedure, we selectively focus on decorrelation-related information, eliminating the influence of intensity variations in the speckle pattern within the raw image.
∆g r can be obtained directly from the raw speckle images by correlating the image with the single-row-shifted version of itself and does not require the entire row intensity correlation or fitting process described in the previous section.Thus, real-time data processing can be easily achieved with minimal computational resources.In addition, ∆g r is significantly more robust to the subject motion than using the full decorrelation curve due to its usage of two-time points of the data within 20µs of duration.We show that the ∆g r RSSPG signal can achieve higher SNR than the RSSPG τ −1 c signals in the presence of subject motion.To understand the physical meaning of ∆g r , we derived the approximated model of ∆g r from Eq. ( 4).Since the exposure time T is set as 20 µs and τ c falls within the range of 400-600 µs, T/τ c can be assumed to be significantly less than 1.The simplification of each g r (0) and g r (1) can be achieved by the Taylor series expansion, thereby setting the trivial second-order terms aside as follows: where x = T τ c .By subtracting gr(1) from gr(0), and neglecting the higher order terms, ∆g r can be estimated as follows: ∆g r = g r (0) − g r (1) ≃ 4β s T 3 ρ τ c (7) where the exposure time T and normalization coefficient β s are constants.As ∆g r is approximately a linear function of ρ/τ c , it effectively represents plethysmography signals -essentially a multiplication of RSSPG τ −1 c and ρ curves.This contributes to an increased SNR in the resulting plethysmogram.However, the efficacy of this fast algorithm in retrieving physiological blood flow information is tempered by the inherent inseparability of the two parameters τ c and ρ in the ∆g r extraction. .SPG was measured with an exposure time of 10 ms, and the signal was processed as described in Ref. [9].While both SPG and RSSPG sequence shows improved contrast in both systolic and diastolic pulses, RSSPG τ −1 c appears to show more steep rises and falls in both peaks.It is interesting to note that the fluctuation of ρ time series also displays the characteristics of the pulsatile waveform (Fig. 4(c)).Unlike the SPG series illustrated in Fig. 4(b), RSSPG can successfully separate τ −1 c and ρ.It is assumed that the expansion of blood vessels during cardiac activity alters the volumetric change of blood, consequently affecting the fraction of dynamic scattering.Additionally, the behavior at the dicrotic notch is a major distinguishing feature between RSSPG and PPG waveforms.Dicrotic notch is a feature of plethysmography waveforms, indicative of the acute drop of blood flow caused by the aortic valve closure [23].Dicrotic notch differentiates between systolic and diastolic flow in a cardiac cycle, thus correlated with various cardiovascular risk factors, and serves as a marker for arterial stiffness and compliance [24,25].To compare the SNR of the RSSPG and PPG waveforms, we measured the detectability of dicrotic notches in each waveform.RSSPG τ −1 c in Fig. 4(a) exhibits a more distinct and clearer dicrotic notch for each cycle.The quantitative analysis in Fig. 4(f) presents the notch ratio of RSSPG τ −1 c , ρ, SPG, and PPG series, defined as the ratio of the distance between the dicrotic notch and the second peak to the AC amplitude [26].The mean notch ratio of the RSSPG τ −1 c and ρ waveform is 39.28 and 29.91 each, surpassing 2.81 of the PPG notch ratio.SPG waveform had a 37.20 mean notch ratio, which is slightly below the RSSPG τ −1 c waveform.Finally, both the acquired parameters τ −1 c and ρ are capable of estimating the heart rate with accuracy similar to that of the SPG and PPG signals (Fig. 4(e)).

RSSPG under cold pressor condition
The cold pressor experiments were conducted to validate that RSSPG signals represent the fluctuation in the blood flow velocity.Figure 5 illustrates the time series of τ −1 c RSSPG and PPG under relaxed and cold pressor conditions.τ −1 c RSSPG, depicted in Fig. 5 (a), shows clear distinctions of average values of τ −1 c under the two conditions, indicating that the average blood flow velocity decreased in the cold pressor tests.This is consistent with the SPG results reported, where the AC amplitude of the SPG signal decreased during the cold pressor test [24].Our result showcases the ability of RSSPG to effectively measure average flow rates within the measurement area under different physiological conditions.In contrast, Fig. 5(b) shows that as PPG measures the blood volume fraction, the contrast of the resulting PPG signal decreases during the cold pressor test.In both the PPG and RSSPG measurements, a reduction in heart rate was observed during the cold pressor test.

RSSPG Δ𝑔 𝑟 Waveform under Exercise Condition
We conducted a comparative analysis of the PPG, RSSPG  −1  and Δ  measurements obtained from motion-contaminated in vivo measurement data.This data set was captured from a participant directly after an intense workout, therefore the heart rate was high and the subject showed heavy breathing during the measurement.The simultaneously acquired PPG and RSSPG series, illustrated in Fig. 6, demonstrate equivalent average heart rates exceeding 120 bpm throughout the observation period.The PPG series (Fig. 6  We conducted a comparative analysis of the PPG, RSSPG τ −1 c and ∆g r measurements obtained from motion-contaminated in vivo measurement data.This data set was captured from a participant directly after an intense workout, therefore the heart rate was high and the subject showed heavy breathing during the measurement.The simultaneously acquired PPG and RSSPG series, illustrated in Fig. 6, demonstrate equivalent average heart rates exceeding 120 bpm throughout the observation period.The PPG series (Fig. 6(a)) shows steady waveforms without motion artifact, potentially due to the signal filtering applied in the commercial PPG sensor.However, it faces challenges in resolving dicrotic notches in fast heart rates, and the primary peaks were clipped due to the limited dynamic range of the PPG intensity measurement, which failed to capture steep changes in the blood volume fractions.The τ −1 c RSSPG series (Fig. 6(b)) reflects subject movements, where both the baseline and the pulse amplitudes to fluctuate over time.With increased fitting error, the motion-contaminated RICs result in a noisier signal, which manifests in the reduction of the notch contrast in RSSPG τ −1 c signals.In contrast, ∆g r RSSPG series (Fig. 6(c)) demonstrates superior signal quality in capturing blood flow fluctuations with motion.The notch contrast of ∆g r exceeds the other two methods, displaying more distinct and well-defined features.Since ∆g r is proportional to ρτ −1 c , the contrast of the pulse signal is bound to be higher than quantitative RSSPG signals.Taking advantage of the first two row-intensity-correlation points makes the ∆g r RSSPG series extraction remain robust, even in the presence of severe movements.Also, the lack of a model fitting process eliminates additional sources of errors and noises.

Discussion and conclusion
In this work, we introduce RSSPG as a new speckle-based plethysmography technique for quantitative analysis of vascular fluctuations caused by the cardiac cycle.RSSPG is based on the recently-developed RSSI technique, where row-by-row time differences in rolling shutter image sensor is used to directly resolve the fast temporal decorrelation within the elongated speckles due to blood flow.RSSI can be modeled with the spatiotemporal intensity correlation model that accounts for more parameters, such as the absolute flow velocity and the fraction of dynamic scattering, allowing for quantitative analysis of scattering dynamics within a single image capture.RSSPG is an extended variation of RSSI to take advantage of its excellent temporal resolution and model complexity for the purpose of quantifying vascular fluctuations in vivo.We validate the fidelity of the RSSPG signal by comparing the waveforms and the estimated heart rates of both PPG and RSSPG signals in various conditions.
The key advantage of rolling shutter speckle analysis is its ability to quantitatively measure temporal dynamics of light scattering within tissue with a simple measurement setup.While traditional PPG predominantly interprets blood volume changes and speckle-contrast based SPG measures combined effects of blood flow and volume fluctuations, RSSPG can separately measure quantitative parameters to represent blood velocity and volume fluctuations in the vascular pulses.The acquisition of RSSPG data under various conditions reveals that the RSSPG technique is able to detect average blood flow rate changes in the same region of interest.Another promising aspect is that RSSPG provides a more accurate and informative representation of hemodynamics than regular PPG, illustrated by more distinct dicrotic notches in the pulsatile waveform of all measured parameters.Since conventional speckle plethysmography is known to show a stronger correlation with blood pressure measurements than PPG [27,28], quantitative RSSPG waveforms can also be used for estimating physiological function with superior performance than PPG [29].The quantitative capabilities of RSSPG can be advantageous for studying the physiological relationship between cardiac activity, peripheral perfusion, and the potential implications of perfusion-related disorders.On the other hand, we also show that RSSPG signal can be used for fast plethysmography processing using ∆g r RSSPG waveforms, which can be simply obtained without complex model fitting.Our interpretation of the ∆g r model shows that both τ c and ρ conjointly affect ∆g r RSSPG, leading to an enhanced SNR and increased robustness of RSSPG to motion errors.Notably, the ∆g r signal only depends on the subtraction of the first two data points in the RIC curve, and the short temporal resolution of a rolling shutter sensor effectively mitigates motion artifacts arising from movements at the order of 10 cm/s.This work is an initial study to propose RSSPG as a new measurement technique for pulsatile blood flow in cardiac cycles.Although promising, our current demonstration is limited in that it does not fully elucidate the physiological implications of RSSPG signals.The fidelity of the τ −1 c and ρ values measured in RSSI has been experimentally validated via phantom experiments with known flow rates and thicknesses of static scattering layers [22], but more detailed studies are required in order to elucidate the physiological scales and meanings of these parameters in peripheral tissue perfusion.For example, elaborate experimental designs may help probe the exact relationship between τ c and the average blood flow velocity within peripheral tissue with complex capillary vessels.These designs can also aid in understanding the physiological explanation of ρ within tissue in terms of volume fraction of blood flow or the scattering properties of moving blood cells.In addition, more intensive clinical studies are required to validate the clinical significance of RSSPG.Current hardware and the measurement method are limited to be applicable in clinical settings and need to be experimentally validated among a wider range of human subjects with various conditions.As we look to the future, our next intentions include developing a compact system for blood flow rate monitoring for long-term measurement, perhaps in the form of a wearable device [9,30] with additional optimization on the speckle sizes for RSSPG measurement and the method of which optimized speckles are generated.The data processing algorithms also need to be optimized for fast and accurate measurement within a compact device.We are also seeking opportunities to establish a correlation between RSSPG waveforms and other cardiovascular measurements, such as blood pressure and electrocardiogram (ECG), to further investigate the importance of RSSPG with the hope of estimating crucial biomarkers of various circulatory diseases from RSSPG signals.

- 1 ]Fig. 1 .
Fig. 1.Overview of the Rolling Shutter Speckle Plethysmography (RSSPG).(a) RSSPG imaging system including an elliptical aperture and a rolling-shutter camera.(b) Each frame in the speckle image sequence captured with the rolling-shutter camera is analyzed for speckle temporal decorrelation via computing the row-by-row intensity correlations within the window of pixels.The sliding window is used to utilize spatial information.(c) Row-by-row intensity correlation (RIC) within a single measurement window shows spatiotemporal decorrelation of the speckle field, which fluctuates over time with the changes in the blood perfusion.(d) The time-series RSSPG  −1  waveform can be obtained by fitting the spatiotemporal intensity correlation model to each row intensity correlation curve in (c).

Fig. 1 .
Fig. 1.Overview of the Rolling Shutter Speckle Plethysmography (RSSPG).(a) RSSPG imaging system including an elliptical aperture and a rolling-shutter camera.(b) Each frame in the speckle image sequence captured with the rolling-shutter camera is analyzed for speckle temporal decorrelation via computing the row-by-row intensity correlations within the window of pixels.The sliding window is used to utilize spatial information.(c) Row-by-row intensity correlation (RIC) within a single measurement window shows spatiotemporal decorrelation of the speckle field, which fluctuates over time with the changes in the blood perfusion.(d) The time-series RSSPG τ −1 c waveform can be obtained by fitting the spatiotemporal intensity correlation model to each row intensity correlation curve in (c).

Fig. 2 .
Fig. 2. RSSPG waveforms and the corresponding RIC curves.(a) RSSPG 1/  series showing pulsatile flow as a result of the cardiac output.(b) RIC curves at different time points in the cardiac cycle show different decorrelation behaviors and the resulting   and  values.
Time-lapse raw speckle images of 1000-by-1000 pixel region of interest (ROI) were captured as illustrated in Fig.1(b), with 20 s row time and 20 s exposure time, and a frame rate of 140 fps.The experiment simultaneously measured pulsatile blood flow waveforms from the RSSPG setup and a commercial PPG sensor (Ubpulse360, LAXTHA).In vivo data was acquired by measuring changes in blood flow waveforms under baseline, post-workout, and cooling conditions.Simultaneous recordings of RSSPG and PPG signals were obtained by attaching the PPG sensor to one index finger while placing the other hand on a table to illuminate the laser precisely under the nail of the opposite index finger.The workout included a 5-minute running exercise, while the cooling condition entailed a 3-minute cold pressor.During the cold pressor challenge, we placed a towel filled with ice around the index finger of the subject.For each condition, the subject relaxed for the following 10 minutes without relocating the measurement

Fig. 2 .
Fig. 2. RSSPG waveforms and the corresponding RIC curves.(a) RSSPG 1/τ c series showing pulsatile flow as a result of the cardiac output.(b) RIC curves at different time points in the cardiac cycle show different decorrelation behaviors and the resulting τ c and ρ values.

Fig. 3 .
Fig. 3. RSSPG signal processing pipeline.(a) The captured raw speckle image sequence.(b) Row intensity correlation curves are calculated at each window at each frame.(c) Processes for the quantitative parameter extraction.With spatiotemporal model curve fitting, 2D maps of   and  are obtained at each frame.Subsequent steps include spatial filtering based on  values to select dynamic regions in the imaging field of view.(d) An alternative process for fast and robust estimation of RSSPG by computing speckle decorrelations within a single row for qualitative analysis.

Fig. 3 .
Fig. 3. RSSPG signal processing pipeline.(a) The captured raw speckle image sequence.(b) Row intensity correlation curves are calculated at each window at each frame.(c) Processes for the quantitative parameter extraction.With spatiotemporal model curve fitting, 2D maps of τ c and ρ are obtained at each frame.Subsequent steps include spatial filtering based on ρ values to select dynamic regions in the imaging field of view.(d) An alternative process for fast and robust estimation of RSSPG by computing speckle decorrelations within a single row for qualitative analysis.

Figure 4
Figure 4 illustrates typical quantitative RSSPG signals from in vivo measurements.RSSPG waveform of τ −1 c shows distinct systolic peaks with high signal contrast (Fig. 4 (a)), as expected.The SPG and PPG waveforms are used as a reference signal to compare the acquired RSSPG waveform τ −1 c (Fig. 4(b), (d)).SPG was measured with an exposure time of 10 ms, and the signal was processed as described in Ref.[9].While both SPG and RSSPG sequence shows improved contrast in both systolic and diastolic pulses, RSSPG τ −1 c appears to show more steep rises and falls in both peaks.It is interesting to note that the fluctuation of ρ time series also displays the characteristics of the pulsatile waveform (Fig.4(c)).Unlike the SPG series illustrated in Fig.4(b), RSSPG can successfully separate τ −1 c and ρ.It is assumed that the expansion of blood vessels during cardiac activity alters the volumetric change of blood, consequently affecting the fraction of dynamic scattering.Additionally, the behavior at the dicrotic notch is a major distinguishing feature between RSSPG and PPG waveforms.Dicrotic notch is a feature of plethysmography waveforms, indicative of the acute drop of blood flow caused by the aortic valve closure[23].Dicrotic notch differentiates between systolic and diastolic flow in a cardiac cycle, thus correlated with various cardiovascular risk factors, and serves as a marker for arterial stiffness and compliance[24,25].To compare the SNR of the RSSPG and PPG waveforms, we measured the detectability of dicrotic notches in each waveform.RSSPG τ −1 c in Fig.4(a) exhibits a more distinct and clearer dicrotic notch for each cycle.The quantitative analysis in Fig.4(f) presents the notch ratio of RSSPG τ −1 c , ρ, SPG, and PPG series, defined as the ratio of the distance between the dicrotic notch and the second peak to the AC amplitude[26].The mean notch ratio of the RSSPG τ −1 c and ρ waveform is 39.28 and 29.91 each, surpassing 2.81 of the PPG notch ratio.SPG waveform had a 37.20 mean notch ratio, which is slightly below the RSSPG τ −1 c waveform.Finally, both the acquired parameters τ −1 c and ρ are capable of estimating the heart rate with accuracy similar to that of the SPG and PPG signals (Fig.4(e)).

PPGFig. 4 .Fig. 4 .
Fig. 4. Comparison between PPG and RSSPG waveforms.(a) RSSPG  −1  series, indicating the blood flow fluctuations.(b) SPG series from the measurement.(c) RSSPG  series, indicating volume fluctuations.Amplitude and notch distance used to calculate notch ratio are also marked.(d) PPG series from the same subject.(e) Heart rate comparison between RSSPG and PPG.(f) Notch ratio measured from RSSPG and PPG waveforms showing that both  −1  and  waveforms can resolve more distinct dicrotic notch than PPG Fig. 4. Comparison between PPG and RSSPG waveforms.(a) RSSPG τ −1 c series, indicating the blood flow fluctuations.(b) SPG series from the measurement.(c) RSSPG ρ series, indicating volume fluctuations.Amplitude and notch distance used to calculate notch ratio are also marked.(d) PPG series from the same subject.(e) Heart rate comparison between RSSPG and PPG.(f) Notch ratio measured from RSSPG and PPG waveforms showing that both τ −1 c and ρ waveforms can resolve more distinct dicrotic notch than PPG

4. 2 .
RSSPG under Cold Pressor Condition The cold pressor experiments were conducted to validate that RSSPG signals represent the fluctuation in the blood flow velocity.Fig. 5. illustrates the time series of  −1  RSSPG and PPG under relaxed and cold pressor conditions. −1  RSSPG, depicted in Fig. 5 (a), shows clear distinctions of average values of  −1  under the two conditions, indicating that the average blood flow velocity decreased in the cold pressor tests.This is consistent with the SPG results reported, where the AC amplitude of the SPG signal decreased during the cold pressor test [24].Our result showcases the ability of RSSPG to effectively measure average flow rates within the measurement area under different physiological conditions.In contrast, Fig. 5(b) shows that as PPG measures the blood volume fraction, the contrast of the resulting PPG signal decreases during the cold pressor test.In both the PPG and RSSPG measurements, a reduction in heart rate was observed during the cold pressor test.

Fig. 5 .
Fig. 5. RSSPG waveforms under different conditions.Cold pressor and relax state blood flow rate comparison between RSSPG  −1  .(a) show changes in the average  −1

Fig. 5 .
Fig. 5. RSSPG waveforms under different conditions.Cold pressor and relax state blood flow rate comparison between RSSPG τ −1 c .(a) show changes in the average τ −1 c and PPG (b) show changes in the AC amplitude.

4. 3 .Fig. 6 .
Fig. 6.PPG and RSSPG waveforms after running exercise for ten minutes, measured with subject motion.(a) PPG from commercial sensor showing increased AC amplitude while the peaks are clipped due to the limited dynamic range of the detector.(b) RSSPG  −1  showing motion-related fluctuations in the  −1  measurement and reduction in the notch contrast.(c) RSSPG Δ  showing improved signal contrast and robustness without any additional filtering.
(a)) shows steady waveforms without motion artifact, potentially due to the signal filtering applied in the commercial PPG sensor.However, it faces challenges in resolving dicrotic notches in fast heart rates, and the primary peaks were clipped due to the limited dynamic range of the PPG intensity measurement, which failed to capture steep changes in the blood volume fractions.The  −1  RSSPG series (Fig. 6(b)) reflects subject movements, where both the baseline and the pulse amplitudes to fluctuate over time.With

Fig. 6 .
Fig. 6.PPG and RSSPG waveforms after running exercise for ten minutes, measured with subject motion.(a) PPG from commercial sensor showing increased AC amplitude while the peaks are clipped due to the limited dynamic range of the detector.(b) RSSPG τ −1 c showing motion-related fluctuations in the τ −1 c measurement and reduction in the notch contrast.(c) RSSPG ∆g r showing improved signal contrast and robustness without any additional filtering.