Biological reliability of a movement analysis assessment using a markerless motion capture system

Introduction Advances in motion capture technology include markerless systems to facilitate valid data collection. Recently, the technological reliability of this technology has been reported for human movement assessments. To further understand sources of potential error, biological reliability must also be determined. The aim of this study was to determine the day-to-day reliability for a three-dimensional markerless motion capture (MMC) system to quantify 4 movement analysis composite scores, and 81 kinematic variables. Methods Twenty-two healthy men (n = 11; X¯±SD; age = 23.0 ± 2.6 years, height = 180.4.8 cm, weight = 80.4 ± 7.3 kg) and women (n = 11; age = 20.8 ± 1.1 years, height = 172.2 ± 7.4 cm, weight = 68.0 ± 7.3 kg) participated in this study. All subjects performed 4 standardized test batteries consisting of 14 different movements on four separate days. A three-dimensional MMC system (DARI Motion, Lenexa, KS) using 8 cameras surrounding the testing area was used to quantify movement characteristics. 1 × 4 RMANOVAs were used to determine significant differences across days for the composite movement analysis scores, and RM-MANOVAs were used to determine test day differences for the kinematic data (p < 0.05). Intraclass correlation coefficients (ICCs) were reported for all variables to determine test reliability. To determine biological variability, mean absolute differences from previously reported technological variability data were subtracted from the total variability data from the present study. Results No differences were observed for any composite score (i.e., athleticism, explosiveness, quality, readiness; or any of the 81 kinematic variables. Furthermore, 84 of 85 measured variables exhibited good to excellent ICCs (0.61–0.99). When compared to previously reported technological variability data, 62.3% of item variability was due to biological variability, with 66 of 85 variables exhibiting biological variability as the primary source of error (i.e., >50% total variability). Discussion Combined, these findings effectively add to the body of literature suggesting sufficient reliability for MMC solutions in capturing kinematic features of human movement.


Introduction
With a substantial growth in the use and access to evolving technologies, sport science practitioners, as well as others working in the human movement and performance landscape now have the means to collect and communicate data in a more efficient manner than before.Within sports organizations, global positioning systems, quantifying athlete external practice and competition workloads, as well as force platforms and velocitybased training devices, allowing for the quantification of further kinetic and kinematic data are currently some of the most used means to quantify information on athletes.These data are often gathered concurrently as part of ongoing practices or training sessions, making for little interference with the athletes' already busy schedules.On the other hand, other means of gathering information such as motion capture screenings have historically presented with greater logistical challenges, requiring more dedicated time from both the athletes and practitioners.Therefore, technologies allowing for more efficient data collection procedures have been explored over recent years.While generally still considered as the gold-standard in the field of biomechanics, once completely laboratory-based, and requiring the often tedious and time-consuming attachment of reflective markers.Human motion-capture systems using markerless motion capture (MMC) solutions have gained increasing amounts of attention, especially in field-based settings such as sports environments, hospitals or physical therapy clinics (1).Further, when using marker-based motion capture technologies, the location of markers, as well as the movement of the skin with respect to the underlying tissues and bony landmarks, known as soft tissue artifact, may present challenges with regards to the acquisition of repeatable and valid movement data (2,3).Likely however, the overarching benefit of MMC technologies lies in the efficient testing procedures, allowing for the capturing of more frequent data points, which may have positive implications for the health and performance of a broad range of populations (1,4,5).For instance, across a rehabilitation spectrum, more frequent data points may allow clinicians to better understand the patients' progress over time, enabling them to make more informed decisions.In line with this, Mauntel et al. (6) explored the reliability of a MMC system, used concurrently with a movement assessment software in scoring the landing error scoring system test (LESS), from which clinically important movement data may be derived.Authors suggested that the markerless system was able to reliably score the LESS test and provide results that were consistently accurate (5,6).
These previously mentioned evolutions in sport science technologies, and the vast growth of data availability to enhance decision making in sport and health make it more important than before to critically evaluate the validity and reliability of the tools that are being used to take measurements (7).Previous groups have proposed three factors that contribute to a good performance test: (i) validity; (ii) reliability; and (iii) sensitivity (8).In line with the previous, reliability refers to the reproducibility of the values of a test (9), and can be influenced by variation in performance by the test subject (biological reliability), variation in the test methods, and variation in measurement of testing equipment (technological reliability) (10).
Still in its infant stages, some have suggested that this increase in test efficiency of MMC systems may come with a sacrifice in data reliability (11,12).Others have suggested sufficient reliability of MMC systems (1,(13)(14)(15)(16)(17).Within a recent SWOT (i.e., Strengths, Weaknesses, Opportunities, Threats) analysis, authors suggested that a strength of low-cost MMC systems is the emerging agreement between them, when compared to marker-based systems (5).For instance, Sandau et al. suggested that a MMC system was able to reliably produce data within the sagittal and frontal plane of motion during a walking task, while data that was produced in the transverse plane showed lesser degrees of agreement, when compared to a traditional marker-based system (13).Similarly, Tanaka et al. proposed that while absolute measures in the hip joint angles obtained using MMC technology differed from marker-based technology, the MMC system presented usefulness in accurately classifying subjects' movement strategies adopted during a functional reach test (18).Further, Cabarkapa et al. highlighted the repeatability of motionhealth screening score, derived from a MMC system, consisting of 8 cameras (16).More specifically, this study documented that algorithm-based motion health scores, which are often used by applied and clinical practitioners, showed moderate to excellent levels of agreement over six testing sessions (16).Hauenstein recently conducted a study, instigating the interrater (i.e., between-system agreement), and intrarater (i.e., agreement between first and second set within each system) agreement (17).Primary findings suggested that moderate [intraclass correlation coefficient (ICC), >0.50] to excellent (ICC, >0.90) reliability was displayed across all 39 kinematic variables analyzed in this study (17).Lastly, Philipp et al. recently emphasized the technological reliability of a MMC system, quantifying a range of elementary movement patterns, from which a vast number of kinematic metrics were derived (1).In this study, two identical MMC systems consisting of 8 total cameras, positioned in close proximity to each other were used to collect kinematic data on 29 different movements, from which 214 different metrics were derived (1).Primary results suggested that 95.7% of all metrics analyzed showed negligible or small between-device effect sizes, with 91.6% of all metrics showing moderate or better agreements, showing further promise with regards to the adoption of MMC systems (1).
While the previous study highlighted the technological reliability of a MMC system, further reliability research is warranted investigating the variation in performance induced by the test subject, often referred to as the biological variability or reliability.Therefore, the aim of the present study was to add further context to the reliability construct of MMC systems, by studying the dayto-day variability in kinematic metrics derived from a battery of different elementary movements, similar to previous research reports (1).Understanding the biological variability in addition to the technological variability will ultimately aid researchers in creating a more complete picture of the overall reliability of MMC technologies in capturing human movement data.Authors believe this to be the first of its kind exploration to understand human variability in motion testing through markerless solutions.Most often, variability is attributed to technology, however, this allows us to further examine the human element.Finally, it is probable that the complete division between technological and human variability may still be unclear, however, our data should help explain some of the tendencies regarding how much variability is possibly attributed to human variance.
Researchers hypothesized that metrics from respective movements would display sufficient amounts of repeatability and low degrees of variability between the four timepoints studied in this investigation.

Experimental design
To determine the test-retest reliability of a MMC system across four visits, spanning over four weeks, a three-dimensional (3-D) video MMC system was used to compare kinematic features as well as movement analysis scores derived from a movement screen consisting of 14 different movement tasks.An experimental within subjects' design was used to compare biomechanical variables and scores over the four assessment timepoints.More specifically, comparisons were made on the same individual across all four time points.

Subjects
Eleven healthy, recreationally active women ( X + SD; age = 20.8 ± 1.1 years., height = 172.2± 7.4 cm, weight = 68.0 ± 7.2 kg) and eleven men (age = 23.0 ± 2.6 years., height = 180.3± 4.8 cm, weight = 80.4 ± 7.3 kg) volunteered to participate in this investigation.All subjects were physically active a minimum of one hour for three days a week for at least the preceding three months.None of the participants reported a history of current or prior neuromuscular diseases or musculoskeletal injuries specific to the ankle, knee, or hip joints.Subjects demonstrated functional range of motion in hip, knee, ankle, and shoulder joints without limiting mechanical motion and performance during a vertical jump and running.This study was approved by the University's institutional review board for human subjects' research.Each subject read and signed an informed consent form and completed a health history questionnaire prior to participating.

Procedures
Procedures for this investigation were adapted from earlier research out of the same laboratory (1,16).Each subject visited the laboratory for four visits.During the first visit, each subject signed an informed consent document, then performed the movement screen.During all four visits, subjects completed a 10min standardized warm-up protocol prior to performing the movement screen.Each subject completed one session per week at the same time of day.The laboratory temperature (24-28°C) and humidity (38%-42%) remained in a consistent range for all test sessions.All subjects performed a total of 14 different movements, from which a total of 81 kinematic variables were extracted.Unilateral squats and forward lunges were performed on each leg respectively.Similarly, vertical jumps were performed bilaterally, as well as on each leg respectively, with jump height calculated as the difference between the estimated center of mass during the highest recorded frame of the jump and standing height.These variables (with the number of variables in parentheses) included range of motion in degrees for both the right and left shoulders (12), hips (20), knees (16), ankles (16), torso rotation, flexion and extension (3), and knee valgus (4).Also distances for lunge stride length (2) and center of mass displacement were measured (8).Additionally, four movement analysis scores were derived, which are calculated based on different kinematic features, using arbitrary calculations.Table 1, which is adapted from earlier research (1) shows a breakdown and explanation of the 19 different movement tasks.Data was quantified using a 3-D MMC system (DARI Motion, Lenexa, KS, USA) composed of eight high-definition cameras (Blackfly/FLIR GigE) recording at 60 fps.The cameras were attached to a metal frame surrounding the testing area, equidistant to each other.Further, the testing area consisted of a contrasting green floor, and no other persons or objects were allowed in the testing field during data collection.The hull technology model records and subtracts the visual signal minus the background, which is used to generate a pixelated person in order to obtain biomechanical parameters of interest.Following manufacturer guidelines, the system was calibrated prior to testing.Specific movement tasks were explained and demonstrated by the principal investigator of the study.Following this demonstration, the member of the research team running the motion capture system provided the subject with the following command: "three, two, one, go".Following the "go" command, the subject completed the movement task which was being recorded by the 3-D MMC system.After the completion of the respective movement task, the command "done" was provided to the subject, to indicate the end of the movement.In line with earlier research, 3-D MMC joint coordinate systems were defined during movement tracking and calculations for all joints of interest follow the methods prescribed by the International Society of Biomechanics (19-21).

Statistical analyses
Descriptive statistics, means and standard deviations (x¯± SD), were calculated for each dependent variable.The individual movement analysis scores (i.e., athleticism, explosiveness, quality and readiness) for each test day were compared with a 1 × 4 repeated measures ANOVA.A one-way repeated measures multivariate analysis of variance (MANOVA), also referred to as a doubly multivariate MANOVA, was used to determine whether there were any statistically significant differences over time (Test 1-4) for multiple dependent variables classified in each of the fourteen distinct categories [i.e., left shoulder, right shoulder, trunk rotation, overhead squat, left unilateral squat, right unilateral squat, left forward lunge, right forward lunge, vertical jump (VJ), drop VJ, static VJ, left unilateral VJ, right unilateral VJ, multi-hop].If the omnibus MANOVA test was significant, a follow-up analysis of variance (ANOVA) with post-hoc Bonferroni adjustments for multiple comparisons would have been carried out, but none were statistically significant.Mauchly's test for sphericity was not significant for any comparison.Two-way mixed intraclass correlation coefficients (ICC; two-way mixed effects model, subjects -random, tests -fixed, absolute agreement) were used to determine day-to-day reliability for each dependent variable (22).Additionally, absolute agreement for each of the repeated measurements for each dependent variable were used to determine the contributions of biological variability to the previously reported technological variability (1).Statistical significance was set a priori to p < 0.05.All statistical analyses were completed with SPSS (Version 29.0.0.0;IBM Corp., Armonk, NY, USA).

Shoulder abduction
Start with arms at your sides with your palms facing forward.With arms straight, raise them out from your sides and over your head (abduct), keeping palms forward throughout the entire movement   for athleticism, explosiveness, quality and readiness movement analysis scores (see Table 3).For the 81 individual dependent variables, ICCs for all but one variable were either good (11 variables) or excellent (69 variables).Only maximum right ankle flexion during landing from a drop jump exhibited a lower ICC (0.49, fair).When the mean absolute differences for all between test comparisons for each dependent variable were compared with previously reported mean absolute differences between two identical motion capture systems [technical variability; (1)], it was possible to determine how much of the total variability reported in the present study was due to biological variability of the subjects being tested (see Table 4).Biological variability ranged from 0.0%-97.9% of the total variability.When all dependent variables are combined, most of the variability is attributed to biological factors (technical variability = 33.1%,biological variability = 66.9%).In general, 37 dependent variables exhibited contributions from biological variability ≥80%, 28 dependent variables with contributions from biological variability ranging from >50%-79%, while only 16 dependent variables exhibited contributions that were primarily due to technical variability (i.e., technical variability > biological variability).

Discussion
The aim of the present study was to investigate the day-to-day reliability in kinematic metrics derived from a battery of different elementary movements.Findings suggested no significant differences with regards to the four composite movement analysis scores, presenting with ICC values ranging from 0.83 to 0.93.While movement analysis scores, based on arbitrary algorithms may be of interest to most practitioners, fellow researchers may be interested in the reliability data of further, more isolated kinematic measures extracted from the different movement tasks.MANOVA results suggested no statistical significance for any of the fourteen distinct categories [i.e., left shoulder, right shoulder, trunk rotation, overhead squat, left unilateral squat, right unilateral squat, left forward lunge, right forward lunge, vertical jump (VJ), drop VJ, static VJ, left unilateral VJ, right unilateral VJ, multi-hop], into which kinematic variables were grouped.80 out of 81 of the kinematic variables of interest demonstrated good to excellent levels of absolute agreement, with 69 out of 81 variables displaying excellent agreement.Out of all metrics of interest, 91.7% displayed ICCs of 0.70 or higher, which has previously been suggested as a benchmark for minimally acceptable ICC values in the field of exercise and sports science (24,25).These findings are encouraging with regard to the biological (i.e., day-to-day) reliability of kinematic measures derived from a MMC system.This is in agreement with a recent study using the same MMC system, highlighting that all kinematic variables from a similar movement screen displayed at least moderate inter-, and intrarater agreement, with most variables presenting with excellent reliability (ICCs, >0.90) (17).Interestingly, in our study, the only variable displaying less than moderate agreement was right ankle flexion derived from the landing phase of a drop jump (ICC = 0.49).This variability between test days was not observed for left ankle flexion in the same task (ICC = 0.84), nor was it observed for either ankle during the downward phase of the vertical jump (ICC = 0.83, 0.85), often referred to as the eccentric phase.While speculative, this finding may be attributed to the more complex nature of the drop jump, especially when testing recreationally trained individuals, who perform plyometric exercises less frequently.Further, comparisons with technological reliability suggest that for ankle flexion during the drop jump landing, most of the variability in the data is attributed to biological variability (94.7%, 89.9%), rather than technological variability (5.3%, 10.2%), which should be considered when interpreting these results.
In line with the previous suggestions, authors want to highlight data presented in Table 3, where contributions to variability were compared between technological variability and biological variability, using data from an earlier study out of our laboratory (1).In the process of establishing reliability, it is important to distinguish between technological variation (i.e., error associated with the technology) and biological variation (i.e., error introduced from human sources) (26).Recent reviews have even called for a greater distinction between these forms of variability in the field of health and sports performance (27,28).Our data suggests that most variation was attributed to error from human sources, with 37 dependent variables exhibiting contributions from biological variability ≥80%, 28 dependent variables with contributions from biological variability ranging from >50%-79%, while only 16 dependent variables exhibited contributions that were primarily due to technical variability (i.e., technical variability > biological variability).It seems difficult to determine why some variables displayed greater biological variability, while other displayed greater technological variability.However, authors hypothesize that the recreationally trained nature of our sample may in part contribute to the fact that most variation was attributed to error from human sources.Therefore, similar methodologies may be applied in other subject groups such as trained athletes.Further, our findings are in agreement with recent literature studying the interrater (technological variability) and intrarater (biological variability) using the same MMC system used in our investigation (17).Hauenstein et al. suggested that in their investigation, generally, intrarater reliability was lower than interrater reliability for the same measurement of consideration, highlighting that this could be partly attributed to the familiarization or warm-up effect between the first and second set, performed during the same visit to the laboratory (17).Our study therefore effectively adds to the body of knowledge by studying intrarater reliability across more test occasions, not taking place within the same day.While our findings effectively contribute to our current understanding of the reliability of MMC solutions in studying human movement, a few limitations of our work, as well as avenues of future research should be acknowledged.For instance, similar to suggestions by Hauenstein et al., the good reliability between days in our study may in part be attributed to the generally healthy nature of our subjects.In many cases, movement screenings such as the ones used in our investigation are implemented in pathological populations such as athletes returning from injury.Such populations may exhibit alternative movement characteristics (e.g., between-limb asymmetries) that were not observed in our subject pool.Therefore, future investigations should aim to study technological and biological reliability of MMC systems, in populations with abnormal movement characteristics (e.g., injured individuals).
In summary, our findings documented good between-day reliability for a number of kinematic variables extracted from a movement screen consisting of 14 different movements.Movement analysis scores based on proprietary algorithms displayed ICCs ranging from 0.83 to 0.93, while out of 81 other joint-, and movement-specific kinematic variables, 80 displayed good to excellent agreement, with only one variable presenting

Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers.Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Shoulder horizontal abductionStart with your arms out in front of you at shoulder height with your palms facing each other.Bring your arms away from each other and behind your body as far as possible, keeping them at shoulder height throughout Shoulder internal/external rotation Start with elbows and shoulders bent at 90 degrees and palms facing down.Rotate arms up and back as far as possible (externally), and then forward and down (internally).Keeping elbows in the same spot during the movement Shoulder flexion/extension Begin with arms by your side.In one fluid motion, bring hands forward and up above the head, then down and back behind the body, and then return to original position.Trunk rotation Start with elbows and shoulders bent at 90 degrees and palms facing down.In one fluid motion, rotate arms, torso, and head, first to the right, then to the left, and then return to starting position Body weight squat Begin with feet shoulder width apart and toes pointing forward.In one fluid motion, squat as low as possible, then return to the starting position Overhead squat Begin with feet shoulder width apart, toes pointing forward and the dowel rod held above the head, with hands positioned wider than shoulders.In one fluid motion, squat as low as possible, and return to the original position Forward lunge right Begin by striding out with right leg getting as far and deep as possible.Then return to the starting position in one fluid motion.During movement keep arms out for balance Forward lunge left Begin by striding out with left leg getting as far and deep as possible.Then return to the starting position in one fluid motion.During movement keep arms out for balance Unilateral squat right Transfer weight to the right leg, lifting the left foot off the ground and behind the body.In one fluid motion, squat as low as possible, keeping the left foot off the ground, and arms out for balance Unilateral squat left Transfer weight to the right leg, lifting the left foot off the ground and behind the body.In one fluid motion, squat as low as possible, keeping the left foot off the ground, and arms out for balance Vertical jump Begin by standing with feet shoulder width apart.Load and jump as high as possible.Do not step into the jump, but you may use an arm swing Static vertical jump Begin by standing with feet shoulder width apart.Lower into a squat position with arms repositioned to a natural jumping stance.Remain in this position for two seconds.On the signal "jump" immediately jump as high as possible from the squat position Unilateral vertical jump right Begin by standing on right leg with left foot off the ground behind the body.Load and jump as high as possible, using an arm swing, and landing on your right foot again Unilateral vertical jump left Begin by standing on left leg with left foot off the ground behind the body.Load and jump as high as possible, using an arm swing, and landing on your left foot again 5 hop right Begin standing on the right leg with left foot off the ground behind the body.Jump on the right leg five times.Jump as high as possible, and as fast as possible, spending as little time on the ground between jumps as possible 5 hop left Begin standing on the left leg with right foot off the ground behind the body.Jump on the left leg five times.Jump as high as possible, and as fast as possible, spending as little time on the ground between jumps as possible Drop vertical jump Begin standing on a 30-centimeter-high box.With either foot, step off the box landing on two feet.Immediately jump for maximal height, spending as little time as possible on the ground.An arm swing may be used

TABLE 2
Summary statistics for intraclass correlation coefficients (ICC).

TABLE 3
Mean (±SD) values for segmental angles and ranges of motion for four different test sessions performed on separate days.

TABLE 3 Continued
No significant differences were observed for any variable as determined from MANOVA analyses (p > .05).

TABLE 4
Mean absolute differences between test sessions for all kinematic variables, and the relative contributions of biological variability and technical variablility.