Accuracy and Repeatability of the Gait Analysis by the WalkinSense System

WalkinSense is a new device designed to monitor walking. The aim of this study was to measure the accuracy and repeatability of the gait analysis performed by the WalkinSense system. Descriptions of values recorded by WalkinSense depicting typical gait in adults are also presented. A bench experiment using the Trublu calibration device was conducted to statically test the WalkinSense. Following this, a dynamic test was carried out overlapping the WalkinSense and the Pedar insoles in 40 healthy participants during walking. Pressure peak, pressure peak time, pressure-time integral, and mean pressure at eight-foot regions were calculated. In the bench experiments, the repeatability (i) among the WalkinSense sensors (within), (ii) between two WalkinSense devices, and (iii) between the WalkinSense and the Trublu devices was excellent. In the dynamic tests, the repeatability of the WalkinSense (i) between stances in the same trial (within-trial) and (ii) between trials was also excellent (ICC > 0.90). When the eight-foot regions were analyzed separately, the within-trial and between-trials repeatability was good-to-excellent in 88% (ICC > 0.80) of the data and fair in 11%. In short, the data suggest that the WalkinSense has good-to-excellent levels of accuracy and repeatability for plantar pressure variables.


Introduction
During human locomotion the foot acts passively, cushioning the impact forces, and actively, transferring the forces generated by the muscles to propel the body [1]. Thus, it is important to know the magnitude and behavior of ground reaction forces to prevent and treat injuries as well as to enhance human performance in sport. Biomechanical gait analysis is commonly performed by force plates that use three-dimensional force transducers to measure the three components (medial-lateral, anterior-posterior, and vertical) of ground reaction forces and torques [1]. For this reason, force plates are of enormous value for assessing human activity, such as walking, running, or jumping. However, this instrument does not directly provide any information about where these forces are being applied along the plantar surface of the foot. For this reason, new plantar pressure systems for quantitative gait force analysis have become increasingly popular.
The first documented described plantar pressure system, which was based on an air-filled chamber, was developed in the 19th century [2]. This method of measurement evolved to deformable materials that provided an ink impression or used optical methods for recording data [1]. At present, a wide range of systems that use electromechanical sensors allow for a more reliable plantar pressure evaluation. Among these, the capacitive, resistive, and piezoresistive sensors are the most frequently used for foot plantar pressure analysis. When compressed, they calculate the variations of the applied load, measuring the proportional change in voltage, conductance, or resistance, respectively [3][4][5]. These sensors can be used as either matrix measurements or discrete measurements [5]. Matrix measurements are made by arrays of sensors in the configuration of pressure plates for barefoot or in-shoe analyses that record the plantar pressure along the entire foot. Discrete measurements are performed using individual sensors (commonly up to eight sensors) placed at specific locations on the plantar foot surface (usually in-shoe). Compared to matrix measurements, discrete measurements are less complex but require an a priori decision about the regions of interest [5].
In-shoe plantar pressure systems (used to perform either discrete or matrix measurements) have been widely used by researchers and clinicians in the fields of clinical rehabilitation [6][7][8], ergonomics [9][10][11], and sport activities [12,13]. Such systems allow monitoring of pressure in the interface between the plantar surface of the foot and the insole of a shoe during either static or dynamic activities, allowing measurements in real conditions without the limits of laboratorial setup [14,15]. The operating principle behind each inshoe system is generally the same: these systems use different sensors/insoles to collect and send pressure values to a hub, usually attached to the lateral malleolus or pelvic girdle, which records data in a memory card or transfers them in real-time to a computer by cable, Bluetooth, or other wireless means.
Before using a new device, validation studies against goldstandard instruments are conducted to determine the accuracy and repeatability of the system [14]. Accuracy is defined as the difference between the value of a known quantity and the value measured by the tested device [9,16]. This device attribute is also referred to in the literature as "validity" [17,18]. In the gait analysis field, calibration benches or systems considered to be gold standards are used to determine how accurate or valid the tested devices are. Another very important device feature is its repeatability, which is defined as the difference between two or more measurements performed by the same instrument under identical testing conditions [16,[19][20][21]. In gait studies of measurement error, many authors have also used the term "reliability" to refer to this feature [18,20,[22][23][24][25][26]. In some studies the terms "repeatability" and "reliability" have been used indiscriminately [20,21,25]. In a repeatability study, variability in measurements made on the same subject can be ascribed only to errors in the measurement process itself. When gait analysis devices are assessed, the repeatability between stance phases (within-trial repeatability), between trials (between-trial repeatability), and between days (between-day repeatability) is commonly analyzed. One of the in-shoe plantar pressure devices most frequently used by clinicians and researchers is the Pedar in-shoe system (Novel GmbH, Munich, Germany). This system has been demonstrated to be accurate [9] and has shown excellent between-trial [15] and between-day [20,26] repeatability. The knowledge of such device attributes (accuracy and repeatability) is of utmost importance before using it in clinical contexts.
WalkinSense (Kinematix SA, formerly Tomorrow Options, Sheffield, UK) is a user-friendly device designed for in-shoe monitoring and long-term storage of plantar pressure and spatial-temporal parameters during locomotion, such as gait speed, distance traveled, and stride length and frequency, without the need for a standardized calibration and the constraints of a laboratorial setup. One preliminary study [27] has already explored the repeatability of the plantar pressures recorded by the WalkinSense. The system was found to be as repeatable as other plantar pressure measurement systems (i.e., F-scan and Pedar). However, the authors assessed only three subjects and no statistical procedure was performed. The authors [27] highlighted the need for further investigations to truly understand how accurate and repeatable the plantar pressures measured by WalkinSense are. Another preliminary study assessed the spatial-temporal parameters of the WalkinSense [28] in a small sample of 15 participants and found good accuracy and repeatability for these parameters.
In short, there is very little information about how accurate the plantar pressures acquired by WalkinSense are, and it is not clear how repeatable the acquisition of these parameters is. This lack of information becomes a barrier for using the device in research and clinical contexts. Thus, the aim of this study was to measure the accuracy and repeatability of the gait analysis performed by the WalkinSense system. Values recorded by WalkinSense measuring typical gait in an adult population are also presented. We hypothesized that the plantar pressure parameters recorded by WalkinSense would be accurate and repeatable.

Participants.
Forty volunteering university students (20 males and 20 females, with mean age of 21.6 ± 3.4, weight 67.2 ± 11.6 kg, and height 170.6 ± 0.9 cm) were recruited. All subjects were healthy and capable of ambulating independently. The exclusion criteria were any pain or difficulty with independent gait, disabilities that could affect natural gait (musculoskeletal, visual, or hearing impairments), and necessity of walking aids. This study was approved by the local ethical committee, and all participants freely gave their written consent to participate after being informed about the study procedures.

Equipment and Data
Collection. The WalkinSense (weight: 68 g, length: 78 mm, width: 48 mm, and thickness: 18 mm) is a CE mark Class I electronic medical device designed to dynamically monitor human lower limb activity (Figure 1(a)). The device contains a micro-electromechanical system (MEMS) triaxial accelerometer and one gyroscope and is connected to a net of eight force-sensing piezoresistors (weight: 5 g, size: 1.8 cm 2 ) used for foot pressure measurements, which can be freely positioned under or over any insole. This device operates at a sampling frequency of 100 Hz in two modes: offline mode, where data are stored to an SD memory card, and real-time mode, where data are communicated to a PC through Bluetooth technology.
Data collection was carried out into two phases: first, a gold-standard bench-testing comparison was conducted with the WalkinSense and the Trublu calibration device; second, a dynamic experimental test compared the WalkinSense with the Pedar during gait.

Bench Experiment (Trublu).
During the bench testing two WalkinSense devices were assessed. Each one of the eight sensors of the WalkinSense nets were randomly positioned under an insole with 2 mm of thickness (provided by the manufacturer) and attached by adhesive Velcro straps. Next, the insoles were positioned into the Trublu, in which ten levels of pressure were sequentially applied during 10 seconds on each one: 0.00, 23.54, 49.03, 73.55, 99.05, 146. 12,199.07, 294.20, 391.29, and 492.29 kPa. Following this, the Walkin-Sense and the Pedar were assessed together (the WalkinSense nets were positioned under the Pedar insoles, Figure 1(b)), only to determine whether the two systems would work well jointly.

Dynamic Experiment.
For the dynamic experiment, the Pedar system (weight: 400 g, length: 150 mm, width: 100 mm, and thickness: 40 mm) was used. The Pedar records in-shoe plantar pressures through 99 capacitive pressure-sensitive sensors with an area of ≅1.5 cm 2 (depending on the size of the insole) and a sampling frequency of 100 Hz.
The dynamic experimental protocol consisted of recording the plantar pressure during gait simultaneously, with the  WalkinSense and the Pedar overlapped. Thus, we warranted that both systems were measuring exactly the same event. The bench tests suggested that there is no interference between the systems and they work well jointly. Before data collection, the Pedar insoles were checked by the Trublu calibration device in order to verify the performance of all sensors. The eight sensors of the WalkinSense were positioned under the Pedar insole ( Figure 2) in correspondence with the mainreference foot areas, as proposed by and adapted from other studies [7,10]. The centroid for positioning each of the WalkinSense sensors on the Pedar insole was manually identified by pressing a stick with the same area of the sensor on the insole corresponding to the selected regions. The activity of the Pedar sensors was controlled on a computer screen, and when only the aimed sensor was active, the region was marked. Following this, the WalkinSense sensors were attached to the insole using adhesive Velcro straps. This procedure was repeated for all pairs of Pedar insoles. The insoles were put into a neutral pair of shoes (ballet sneakers). Then, the participants stood in an upright position and their weight and height were recorded by a force plate (Bertec Corporation, Columbus, OH, USA) and a stadiometer (Seca, Birmingham, United Kingdom). The participants familiarized themselves with the experimental setup by walking freely over a 12meter walkway at a pace of 100 steps per minute marked by electronic metronome software (Metronome Beat, Andy Stone). Following the familiarization, participants performed a variable number of trials, and three valid ones were used for further analysis. In each trial, about 12 steps were recorded and only the central four stance phases (two with each foot) were used in the statistical analysis. This procedure was adopted to avoid the effects of acceleration in the movement.

Data Analysis.
Data from the Pedar were recorded by the Pedar-X software (Novel GmbH, Munich, Germany) and data from the WalkinSense using the WalkinSense software (Kinematix SA, Sheffield, UK). The sensor pressure values from both systems were exported and then analyzed by MATLAB 7.0 software (MathWorks, Massachusetts, USA) through an appropriate program for data processing and variable calculation. Each step of all trials was considered as one sample during the statistical analysis to avoid hiding differences between the instruments and within or between trials that could be caused by averaged values.

Bench Experiment.
At each of the ten load levels applied by the Trublu, we analyzed the 100 central frames (from 1000 recorded frames-10 seconds of data recording) taken from the two WalkinSense systems. The loads applied by the Trublu were also recorded for later use as reference values to calculate the accuracy of the WalkinSense systems.

Dynamic Experiment.
The following anatomical regions were studied: great toe (G Toe ); medial, central, and lateral forefoot (FF Med , FF Ct , and FF Lat , resp.); medial and lateral midfoot (MF Med and MF Lat , resp.); and medial and lateral rearfoot (RF Med and RF Lat , resp.). For each WalkinSense sensor and the respective Pedar sensor, four dependent variables were calculated: peak pressure ( Peak , in kPa), defined as the highest value displayed by the sensor along the stance phase; peak pressure time ( Time , in % of the stance phase), defined as the instant correspondent to the Peak ; mean pressure ( Mean , in kPa), defined as the mean pressure during the stance phase; and the pressure-time integral ( Integral , in kPa s), defined as the integral along the stance phase.
The gait analysis conducted used the mid-gait method, as this method aptly represents a normal walk [29], and three trials were performed in order to provide a consistent mean [30]. Subjects wore standardized shoes and adopted a controlled gait cadence, since footwear and walking speed have been shown to influence plantar pressures during gait [31]. Gender differences were not considered, as a previous article reported no gender influence on Peak and Integral parameters [32].

Statistical Analysis.
Statistical analysis was performed using SPSS statistics v.20 software (IBM SPSS, Chicago, USA) and Statistica v.8 software (Statsoft, Tulsa, USA). We considered the Two-Way Mixed Model (Type: consistency) intraclass correlation coefficient (ICC) ≤0.69 as poor, 0.70-0.79 as fair, 0.80-0.89 as good, and ≥0.90 as excellent [33]. All ICCs were calculated intraexaminers. The 95% confidence intervals (CI 95% ) were calculated with the ICC and the absolute and percentage differences to verify the uncertainty of these differences [34].

Bench Experiment
(1) Repeatability. The within-WalkinSense (sensor versus sensor from the same WalkinSense net) and between-WalkinSense (8 sensors versus 8 sensors from two Walkin-Sense systems) repeatability was verified by the ICC.

Dynamic Experiments
(1) Repeatability. The overall (considering all regions together) and regional within-trial (first right step versus second right step and first left step versus second left step) and between-trial (four steps from the first trial versus second trial versus third trial) repeatability of the WalkinSense was verified by the ICC.
(2) Accuracy. The relation (accuracy) between the Walkin-Sense and the Pedar records was verified by (i) the ICC (overall and regional), (ii) the overall Person correlation coefficient, and (iii) the overall and regional absolute (Pedar

3.1.2.
Accuracy. An excellent ICC and high and statistically significant ( < 0.001) level of correlation between the applied loads (Trublu) and WalkinSense records were observed ( Figure 3). The absolute differences ranged from −7.88 to 12.81 kPa along the different applied loads. The percentage differences were lower than 9% for nine out of the ten loads applied. At the first load applied (25.54 kPa), the WalkinSense showed pressure values 33.48% lower than the bench. The heaviest applied loads (>294 kPa) showed the smallest differences (<2%).

Repeatability.
Excellent overall within-trial and between-trial repeatability was found in the four dependent variables, and all of the ICC CIs 95% were smaller than 0.02 ( Table 1). The regional within-trial and between-trial ICCs for the Peak were excellent in four (RF Lat , RF Med , FF Ct , and G Toe ), good in two (FF Lat and FF Med ), and fair in one (MF Lat ) out of the seven analyzed regions ( Table 2). For the Time , the regional within-trial ICCs were excellent (G Toe ), good (FF Lat ), and poor (MF Lat ) in one region each and fair in four regions; and the between-trial ICCs were good in five regions and excellent (G Toe ) and fair (MF Lat ) in one region each. All within-trial and between-trial ICCs for Integral and Mean were good or excellent (Table 2).
The regional percentage differences between the systems for Peak ranged from 3.6% at the RF Med to 16.4% at the MF Lat . The percentage differences for the Time were similar among the regions, ranging from 2.3% to 4.3%. In the Integral and Mean , the highest differences were observed at the MF Lat (≅30% for both Integral and Mean ) and the lowest differences in the RF Med (7.7% for Integral and 6.3% for Mean ) ( Table 4).
In seven out of the eight regions, lower values were observed in the WalkinSense (negative percentage differences) for the Time , Integral , and Mean variables. Only at the FF Ct did the WalkinSense show higher values than the Pedar. The range of the CIs 95% of the percentage differences for the Peak was ≅3.5%; for the Time , it was ≅1.5%; and for the Integral and Mean , it was ≅4.5% (Table 4).
Twenty-six out of the 28 regional ICCs (Pedar and WalkinSense) were greater than 0.9, and their 95% confidence interval ranged between 0.2 and 0.35. The remaining two   ICCs (out of the 28) were 0.86 for the Peak at the RF Lat and 0.84 for the Time at the FF Ct .

Description of Values.
Both systems showed the highest and lowest Peak , Integral , and Mean values at the FF Ct and MF Med , respectively. Also, the earliest and latest Time occurred at the same regions in both systems (RF Lat and G Toe , resp.) ( Table 5). In

Discussion
The present study aimed to measure the accuracy, repeatability, and description of values of the plantar pressure parameters of the WalkinSense system. For this purpose, two experiments were carried out: a bench experiment, in which the WalkinSense records were compared to a goldstandard bench test (Trublu), and a dynamic experiment, in which the WalkinSense was compared to one of the most commonly used in-shoe plantar pressure systems (Pedar) in gait analyses. While the majority of the measurement error studies only assess the repeatability or accuracy of the devices using static-bench [9,17,35] or dynamic [19][20][21][22][23][24][25][26]36] experiments separately, the present study focused on a wider approach for statically and dynamically assessing the accuracy and repeatability of a device.

Bench Experiment.
Our first aim was to assess repeatability of the eight WalkinSense sensors from a single net as well as the repeatability of a couple of nets. Secondly, we wanted to verify the accuracy of the measurements when compared to a gold-standard device. Results show an excellent repeatability (ICC > 0.999) of the measurements for both the single net and pair of nets, together with an excellent overall repeatability and a high Pearson correlation coefficient between the WalkinSense and Trublu systems.
However, further analysis reported that the absolute and the percentage differences varied with the applied loads. In fact, the highest percentage differences, −33% and −7%, were observed at the lowest pressures, 24 and 49 kPa, applied. Hsiao et al. [9] reported similar results in a previous study where the accuracy of the Pedar and F-scan (Tekscan, South Boston, USA) systems were analyzed by bench tests. They found a low accuracy at the lowest pressures in both systems and a gradual reduction of the percentage difference at the higher loads. The Pedar showed percentage differences between −57.2% and 1.3% when pressures between 12 and 59 kPa were applied; the F-scan reported a similar trend with percentage differences between 19.4% and 27.9% for loads between 5 and 41 kPa [9]. In our study, on the other hand, we observed the lowest percentage differences, equal to 0.4% and 0.2%, at the highest load magnitudes (≅300 and 500 kPa). In the same way, when the Pedar and F-scan systems were loaded with 300 and 500 kPa by Hsiao et al. [9], low differences, equal to 5.2% and 3.6% and 1.2% and −11%, respectively, were observed. As the thickness of the contact surface in which the sensors are placed decreases their sensitivity at low-pressure ranges (from 10 to 80 kPa) [37], the insole in which the WalkinSense sensors were placed, with its 2 mm thickness, may play an important role in explaining these higher percentage differences found at the lowest applied pressures.  [20], where the repeatability of the Pedar system was assessed by the coefficient of variation, different results were obtained among the regions. As in our study, the authors [20] also found the midfoot (which was considered as a unique region) to be one of the least repeatable regions. In another study in which the betweenday repeatability of a pressure plate (EMED) was assessed in ten-foot regions, Gurney et al. [24] reported that the Peak repeatability was poor-to-fair in four regions and good-toexcellent in six, while the Integral repeatability was poor-tofair in three regions and good-to-excellent in the remaining seven [24]. In our study, overall high degrees of agreement (ICCs > 0.95) were found comparing gait parameters of the Walkin-Sense and the Pedar. The overall percentage differences indicate that the WalkinSense showed lower Peak (≅6%), Integral (≅14%), and Mean (≅13%) and the Time slightly later (≅3%) compared to the Pedar. The regional ICCs between the systems were excellent for almost all regions. When considering the regional percentage differences, Integral and Mean reported the highest differences: ≅30% in the MF Lat and ≅24% in the FF Lat . Considering the differences between the systems (kind of sensor, sensor area, and layout), we may consider these differences as acceptable.

Dynamic
In a preliminary study, Healy et al. [27] assessed three subjects' walking using the WalkinSense and the F-scan (Tekscan Inc., Boston, USA) systems in two different days. The WalkinSense showed a similar level of repeatability when compared to other plantar pressure measurement systems. However, the conclusions were mostly based on subjective analyses; the authors did not provide any measure of variance of the values nor did they perform any statistical procedure. Therefore, it is difficult to compare their results [27] with those found in the present study. Nevertheless, our findings seem to be in agreement with those reported by Healy and colleagues in their preliminary study [27].
In short, in the dynamic experiment (gait analysis) from the present study, the overall within-trial and between-trial repeatability were excellent and the regional within-trial and between-trial repeatability were mostly good-to-excellent (only in one region was repeatability fair). The overall accuracy (ICCs: WalkinSense versus Pedar) showed ICCs higher than 0.95 and Pearson coefficient with correlations statistically significant for all variables and for the regional accuracy ICCs higher than 0.84 were observed for all variables.

Description of Values.
All the Peak values found in the present study fell in the reference range previously proposed for healthy subjects [19]. The Integral magnitudes and the sequence of the Time along the regions were similar to those presented by Putti et al. [19].
In the present study, the highest Peak was in the FF Ct (331 kPa), followed by the RF Med (275 kPa) and the RF Lat (257 kPa), while in the aforementioned study [19] the highest Peak occurred in the G Toe (280 kPa), followed by the rearfoot (which was considered as one region, 264 kPa) and forefoot (metatarsal heads I and II, ≅247 kPa). In another study assessing a healthy population [36], the highest Peak was found in the forefoot (metatarsal heads II-361 kPa and III-330 kPa), followed by the G Toe (321 kPa) and the rearfoot (313 kPa), using a pressure plate. The differences among these studies could be attributed to external variables such as the use of different shoes (neutral shoes versus running shoes) or different reference systems (in-shoe pressure system versus pressure plate). In the present study, we opted to use an inshoe pressure system inside of neutral shoes in order to record gait values during a condition as similar as possible to those found in daily life.

Limitations.
This study presents some limitations, such as (i) the standardized position of the WalkinSense sensors for the four pairs of Pedar insoles that did not necessarily correspond to the point of maximal pressure for all subjects; (ii) the differences between the WalkinSense and the Pedar (i.e., layout, sensor area, and kind); and (iii) the description of values provided in this study that can only be considered for the proposed arrangement of the sensors.

Conclusion
The WalkinSense showed good-to-excellent levels of accuracy and repeatability for plantar pressure variables during static-bench and dynamic gait analysis. Four plantar pressure parameters in healthy adults were presented and can be used as standard values while using this device. Further investigation of gait analysis and of the long-term accuracy and repeatability, the between-day and interexaminer repeatability, and the accuracy and repeatability of the spatialtemporal parameters of the WalkinSense are needed.