Validation of an ergonomic assessment method using Kinect data in real workplace conditions

.


Introduction
In ergonomics, the posture and movement of a worker are important information for determining the risk of musculoskeletal injury in the workplace (Vieira and Kumar, 2004).Different methods and tools have been developed to assess exposure to risk factors for work-related musculoskeletal disorders (MSDs).They can be divided into three groups according to the measurement technique.They are the self-report, direct measurement and observational methods (Li and Buckle, 1999;David, 2005).
Self-report methods can take many various forms such as rating scales, questionnaires, checklists or interviews, however, they are not always reliable and could lead to biased interpretation (Burdorf and Laan, 1991;Wiktorin et al., 1993).Direct methods, which is to collect data directly from sensors attached to the worker's body, are difficult to implement in real work situations (Li and Buckle, 1999).Moreover, wearing these devices may cause discomfort and influence the postural behaviour (David, 2005).Observational methods consist of directly observing the worker and the corresponding tasks, such as the popular RULA method (McAtamney and Corlett, 1993).The accuracy and validity of the results obtained by observational methods directly depend on the input information collected (Fagarasanu and Kumar, 2002).The problem is that data collection is generally obtained by subjective observation or simple estimation of projected angles in videos/pictures.This leads to low accuracy and high intra-and inter-observer variability (Burdorf et al., 1992).Nevertheless, such a method is suitable for many work case and remain a practical way to estimate the risk.
Using observational methods, recent works in ergonomics (Vignais et al., 2013;Battini et al., 2014) have demonstrated that real-time ergonomic feedback based on motion capture systems positively influences the motion of workers and decreases hazardous risk score values.However, these methods were based on wearable inertial sensors, making it difficult to be applied in real work conditions.Other motion capture systems, such as the optical or magnetic systems, have similar limitations.They require positioning sensors or markers on the body and calibrating the system and the skeleton, which are not always possible in real work conditions, as sensors can be incompatible with security constraints and can also be perturbed by the electromagnetic environment.
Recent papers evaluated the accuracy of the Kinect skeleton data mostly for very simple motions with the recommended sensor placement (sensor placed in front of the subject) (Clark et al., 2012;Kurillo et al., 2013;Bonnech ere et al., 2014).It has been shown that the error is dependent of the performed postures (Xu and McGorry, 2015) and this error rapidly increases for complex motions with auto-occlusions and when the sensor is not placed in the recommended position (Plantard et al., 2015).Placement in nonrecommended sensor positions with occlusions can induce large error values, which may be a limitation in real work conditions.
Several methods have been proposed to correct badly reconstructed postures provided by the Kinect.Learning statistical dynamic models (based on a database of examples) as a motion prior can deal with highly non-linear human motion and produce higher quality movements from only a few marker-based motion capture data (Chai and Hodgins, 2007).Applying these methods to correct Kinect postures has a major drawback as each body joint position is assumed to be accurately reconstructed whereas Kinect delivers noisy or even incorrect information.To overcome this limitation, recent works have proposed to take the reliability of the Kinect data into account in the correction process.Reliability can then be integrated into a lazy learning framework to reconstruct a more reliable posture (Shum et al., 2013;Zhou et al., 2014).
However, these previous methods have not been adapted to real workplace conditions, with many occlusions and nonrecommended sensor placement.Indeed, when significantly large occlusions occur, the select few reliable information available would not be sufficient to accurately correct the posture, leading to unrealistic results.To overcome this limitation previous works (Plantard et al., 2016) have proposed a new data structure named Filtered Pose Graph to efficiently preselect a relevant subset of postures before correction, ensuring continuity and maximizing reliability even when important occlusions occur.This enhances both computation speed and correction quality.
The aim of this paper is to design a new method to compute RULA scores at 30 Hz based on corrected Kinect skeleton data, and evaluate its relevance in ergonomic analysis.One of the main challenges in manufacturing plants is the occurrence of many occlusions due to cluttered environments and constrained sensor placement.Computing RULA scores based on corrected Kinect skeleton data should enable us to partly tackle this problem.
Consequently, we proposed to compare RULA scores obtained with corrected Kinect data to 1) those based on ground truth data in controlled laboratory conditions, and 2) those estimated by two professional ergonomics experts in real manufacturing plants with professional workers.

The RULA method
In ergonomics, several observational methods are used such as the revised NIOSH Lifting equation (Waters et al., 1993), the Rapid Upper Limb Assessment (RULA) (McAtamney and Corlett, 1993) or the Rapid Entire Body Assessment (REBA) (Hignett and McAtamney, 2000).One of the most popular observational methods is the RULA.The examiner has to rate a static key posture of the worker based on direct observation or a picture.This evaluation is based on an estimation of the main upper body, trunk and neck joint angles.Each joint angle is associated with a joint score according to a predefined range of angles.These joint scores lead to final grand scores and to recommendations.Readers are referred to (McAtamney and Corlett, 1993) for more details about the RULA method.
Although McAtamney and Corlett (1993) claimed the method to be reliable, statistical calculations were not published and this method suffered from the same biases as other observational methods would.Indeed, only a fair inter-rater reliability of the RULA grand score (ICC < 0.5) was found for observers with a common background in ergonomics (Robertson et al., 2009).The observers need to be trained to accurately fill in the RULA assessment grid (Dockrell et al., 2012).
As this approach is based on isolated key postures (usually the worst case postures), it leads to a discretization of the score that may be less sensitive to noise than methods based on continuous scores.Moreover, isolated scores cannot capture temporal information, such as the time spent performing an unhealthy posture.However, RULA, as most observation methods used in the industry focuses primarily on the evaluation of static postures, mainly due to the lack of suitable human performance analysis tools available for dynamic motion (Chaffin, 2005).Indeed, a postural evaluation at every moment of the task would become labour intensive, despite its actual relevance.Thus, designing a method to continuously assess human motion would provide new relevant information to evaluate potential musculoskeletal risks.

Material and methods
This section describes the developed method to compute required information for RULA while using Kinect data represented by a simplified skeleton.It also describes the two experimental protocols designed to validate this method in simulated and real working conditions.On one hand, simulated conditions in a laboratory enabled us to use a reference motion capture system to quantify inaccuracies in estimating joint angles and the corresponding RULA scores.On the other hand, real work conditions in manufacturing plants may be much more complex, including more occlusions and non-recommended sensor placements.In such a cluttered environment, with real production constraints, placing a reference motion capture system is almost impossible.Alternatively, we compared the resulting RULA scores to those provided by two ergonomic experts who were using a traditional observational method to calculate the RULA scores.

Computation of joint angles using the Kinect data
As shown in (Plantard et al., 2015), the data provided by the Kinect are sensitive to the environmental conditions.Occlusions induced by the performed posture, the position of the worker relatively to the camera and the workstation, lead to inaccurate data.To improve the robustness of the data provided by the Kinect in such conditions, we used a correction method further detailed in (Plantard et al., 2016).
To use the RULA method, relevant joint angles have to be computed based on the Kinect data.The correction method provides a skeleton composed 3D joint position only (see Fig. 1).A posture is defined as p ¼ fx j ; y j ; z j g, where N stands for the number of joints in the posture, and x j ; y j ; z j stand for the 3D Cartesian coordinates of the j th joint.According to the estimated joint positions, joint angles should be computed using the ISB recommendation (Wu et al., 2005).However, the Kinect skeleton is not fully compatible with this recommendation as it does not provide all the required anatomical landmarks.We consequently adapted the joint angle definition to take the available Kinect joints (named with letters in Fig. 1a) into account.
The global coordinate (pelvis coordinate) was defined in accordance with the ISB recommendation (Wu and Cavanagh, 1995).Yaxis is along the trunk axes represented by the vector from the hip center (HC in Fig. 1) to the spine (SP in Fig. 1).The X-axis is defined as the normal of the plane formed by the Y-axis, the left (HL in Fig. 1) and the right (HR in Fig. 1) hips.Finally, the Z-axis is computed as the normal of the X-axis and Y-axis.
For the trunk coordinate system, the Y-axis is represented by the vector from the spine (SP in Fig. 1) to the shoulder center joint (SC in Fig. 1).The X-axis is defined as the normal of the plane formed by the Y-axis, the left (SL in Fig. 1) and the right (SR in Fig. 1) shoulders.Finally, the Z-axis is computed as the normal of the X-axis and Yaxis.
For the shoulder coordinate system, the Y-axis is given by the vector from the elbow joint (EL or ER in Fig. 1) to shoulder joint (SL or SR in Fig. 1).The Z-axis is the normal of the plane formed by the Y-axis and the lower arm defined from wrist joint (WL or WR in Fig. 1) to elbow joint (EL or ER in Fig. 1).The X-axis is the normal of the plane formed by the two previous axes.
These three coordinate systems were placed at the hip center (HC), shoulder center (SC) and shoulder joints (SL or SR) respectively, as depicted in Fig. 1b.The joint angles were then computed according to the ISB recommendation, to obtain the flexion, side bend and twist angles of the trunk and the flexion and abduction angles of the shoulder joint.We changed the matrix decomposition sequences of the shoulder joint angle computation from YXY to ZXY, to isolate abduction and to limit gimbal lock problems as suggested in (Senk and Ch eze, 2006).
The Kinect skeleton also does not provide enough points to compute the neck and elbow local coordinate systems, as recommended by the ISB.We alternatively computed the elbow flexion joint angle using the vector convention detailed in (Bonnech ere et al., 2014).The neck flexion and side bend joint angles were computed by the planar projection of the neck vector (SC to H) expressed into the local trunk coordinate system.The planar projection of the shoulder vector (SC to SL of SR) expressed into the local trunk coordinate system, was used to determine if the shoulder was raised.As there is not enough available information to compute some angles, the "wrist", "wrist twist" and "neck twist" RULA scores have been manually set.
Computing a score based on joint angles, RULA method involves applying joint angle thresholds.These thresholds have been accurately defined for some joint axes, but they have not been defined for the others, such as shoulder abduction/adduction (Battini et al., 2014).For this joint axis, the threshold was set to 20 , as suggested in (Aptel et al., 2000).Other considerations, such as the muscle use (static, dynamic …) and force scores, are set manually.

Experimental procedure in laboratory condition
In this section, we present the experimental protocol used to evaluate the relevance of the proposed method in simulated constrained conditions.To this end, we carried out an experimental protocol with 12 male participants (age: 30.1 ± 7.0 years, height: 1.75 ± 0.05 m, mass: 62.2 ± 7.0 kg).They were equipped with 47 reflective markers positioned at standardized anatomical landmarks, as suggested in (Wu et al., 2005) to measure reference postures.The motion of the participants was recorded by both a Microsoft Kinect 2 system and a 15 cameras Vicon optical motion capture system.
One of the main problem when assessing work tasks is the occurrence of occlusions mainly due to the manipulation of objects.To reproduce this situation in laboratory conditions first, the subjects had to perform lowering and lifting motions with a box with two hands placed on either side of the box, as depicted in Fig. 2. The box dimensions were 40 cm height per 30 cm width per 17 cm depth.The lowering motion consisted in carrying the box from the target position to the front of the hips and the lifting motion involved to put it back to the original position.The box (attached to a magnet) had two target placements, in order to generate two different motions.In the first placement named Front, the target was located in front of the subject, at 1.70 m high, 0.35 m left and 0.50 m in front.In the second placement named Side, the target was located on the left of the subject, aligned with the two shoulders at the same height and 0.55 m left, as illustrated in Fig. 2.
The manipulated box was supposed to generate different levels of occlusion depending on the Kinect placement.We tested different scenarios, with and without box, and with various Kinect placement, to analyse the impact of various types of occlusions: -[NB e No Box condition]: the manipulation of the box was simulated by the subject without actually using a box, to avoid occlusions due to this box.In this condition, the subjects simply reached the position where the box should usually be located.
The Kinect was placed in front of the subjects, as recommended by Microsoft.It enabled us to test the robustness of the Kinect sensor under optimal conditions.-[B e Box]: the manipulation is actually performed with the box, leading to occlusions of parts of the body, simulating working conditions.The Kinect was again placed in front of the subject, as recommended by Microsoft.-[B45 e Box and 45 sensor placement]: as in condition B the subject actually manipulated the box but the Kinect was placed 45 to the right in front of the subject, as it could happen in real cluttered environments.In this condition, risks of occlusion were more likely than in all the previous conditions.The subject repeated each motion (lowering and lifting) 5 times, in each condition and box placement (Front-NB, Front-B, Front-B45, Side-NB, Side-B and Side-B45).The recorded motions were then segmented to keep only the lowering and lifting motion.Thus we eliminated all the fames for which the velocity of the wrists was below a threshold, corresponding to standing poses.In some cases, the Kinect tracking signal was randomly lost and the embedded posture estimation confused the body with the box.In this experiment, this corresponded to 14.6% of the motions, which were excluded from the remaining of the process.

Experimental procedure in real work condition
In this section, we present the experimental protocol used to evaluate the relevance of the proposed method within a real workplace.To this end, we carried out an experimental protocol with 7 male professional workers (age: 49.7 ± 3.9 years, height: 1.75 ± 0.09 m, mass: 70.0 ± 4.9 kg), in a car manufacturer factory.Overall, 5 different workstations were assessed and the work task was recorded by a Microsoft Kinect 2 sensor.Fig. 3 shows the different workstations from the Kinect point of view.One can see that the Kinect was placed in sub-optimal conditions induced by the cluttered working environment: on the side of the worker in these cases.When working, the subjects had to manipulate large objects, leading to large occlusions, as illustrated in Fig. 3.
Overall, 5 different workstations were assessed and the work task was recorded at least twice for each worker.Finally, 22 motion capture sessions were performed in an assembly plant of car seats.The workers performed their routine work tasks, without any kind of perturbation: no wearable sensors, no calibration.
The RULA scores computed using the corrected Kinect data were compared to those obtained by two human observers, similarly to previous works (Diego-Mas and Alcaide-Marzal, 2014).Instead of selecting worst-case postures for RULA assessment as usual, the experts performed the RULA assessment with recorded Kinect colour sequences sampled at 0.2 Hz.A total of 300 different images were consequently assessed by the two experts.The experts independently assessed each body part required by the RULA method.The scores provided by the two experts may be slightly different due to inter-examiner variability.In such a case, the score returned by the method was assumed to be correct if it was in-between the results of the two experts.RMSE between the scores delivered by the experts and the method was calculated using the most different expert's score.

Data analysis
In the laboratory experiment, we were able to compare the joint angles computed with corrected Kinect data to those obtained with a reference motion capture system.Due to the specific lowering and lifting motion, we focused our joint angles analysis on the shoulder flexion angles.Kolmogorov-Smirnov test was used to check the normality of the distribution of the error either these analyses.The distributions did not follow a normal law for this experiment.RMSE and Spearman's rho (r) correlation coefficient were computed for each condition.Then, we compared the resulting RULA scores obtained with both Kinect and reference systems thanks to the RMSE and Proportion agreement index (P o ).
For the workplace experiment, we compared the RULA scores computed with Kinect data to those obtained by the experts.P o and the strength of agreement on a sample-to-sample basis as expressed by unweighted Cohen's kappa (k) were computed, as proposed by (Diego-Mas and Alcaide-Marzal, 2014).

Results in laboratory conditions
Table 1 reports the RMSE of the lowering and lifting motion for all the joint angles computed thanks to the Kinect and the Reference data, in all the target placements and conditions.
These results show that the errors of the different computed joint angles remained low, between an average value of 7.7 (with 9 angles among 26 with an error greater than 10 ) for the simplest case (no box, Kinect in front) and 9.2 (with 12 angles among 26 with an error greater than 10 ) for the worst case (Side-B) The largest error values mostly occur for joints with large motions, such as the shoulder or elbow flexion, leading to an acceptable percentage of error compared to the range of motion.
Correlation between the joint angles computed with the Kinect and the reference motion capture system was also investigated.For joints with small variations, signal to noise ratio leads to unusable correlations.Hence, we focused this analysis to the joint that mainly moves during the motion, also associated with one of the highest absolute error value (see Table 1): the shoulder flexion.For these angles, correlation ranged from 0.98 (for Front-NB condition) to 0.68 (for Side-B condition) and was higher than 0.90 for 16 of the 24 studied motions.
Tables 2 and 3 report respectively the RMSE and agreement values of the lowering and lifting motion for the RULA scores in all the target placements and conditions.
The resulting RULA grand scores computed thanks to the joint angles showed strong agreement.Indeed, the RULA grand scores are correctly computed for more than 70% of the conditions.Moreover, for each RULA score, RMSE was lower than 0.68.
In this experiment, the mean RULA Grand scores for lowering task were 2.82 (SD: 0.59, min: 2, max: 7) and 2.85 (SD: 0.53, min: 2, max: 6) for left and right respectively.For lifting task, the scores were 2.66 (SD: 0.57, min: 2, max: 5), Put Right: 2.66 (SD: 0.53, min: 2, max: 6) for left and right respectively.The agreement found for the RULA grand scores, are slightly lower than those found in the laboratory experiment, but they remain higher than 70%.The kappa index showed a strength of agreement from moderate to substantial according to the scale of (Landis and Koch, 1977).

Discussion
In this section, we discuss the main results reported in this paper and expose some limitations and perspectives.

Main contributions
This paper aimed at proposing and testing a method to estimate RULA scores using Kinect skeleton data, enabling us to compute RULA-compliant joint angles and scores depending on the limited Kinect skeleton data.It has been applied to Kinect skeleton data corrected using a recent method (Plantard et al., 2016) in order to limit the impact of occlusions.To evaluate this method, we carried out two experiments.The laboratory condition enabled us to quantify inaccuracies of the method compared to reference motion capture data.The real condition aimed at evaluating the agreement of the estimated RULA scores, compared to two expert's assessments.
In the laboratory condition, one can notice that the weaker correlations were found for the left shoulder during the lowering motion, especially when the box was placed in the opposite direction of the Kinect.For example in Side-B condition, a rho correlation of r ¼ 0.68 (lowering) and r ¼ 0.79 (lifting) were found for the left shoulder, versus r ¼ 0.94 (lowering) and r ¼ 0.90 (lifting) for the right shoulder, as the box is partly occluding the left arm.We also noticed that the lifting motion generally led to more accurate results than lowering motion.For example, in Front-NB condition the RMSE of the lifting motions were 5.9 ± 4.4 and 7.7 ± 3.1 for left and right shoulder respectively, while the lowering motions led to 12.8 ± 4.4 and 10.7 ± 4.2 for left and right shoulder respectively.Further investigation would be needed to explain this small difference.Differences found in the joints angles were less important in the resulting RULA grand score.Indeed, as RULA is based on angular thresholds, it tends to minimize the effect of noise when the angle is far from the thresholds.
These good results in laboratory conditions do not guarantee a good agreement of the estimated scores with ergonomic experts' assessments, especially in real conditions.In these conditions, the aim was to challenge our method in real workplace environments, with many occlusions and non-optimal Kinect placements.The results showed substantial agreement: the method correctly assessed the RULA grand score 73% and 74% for the right and left body part respectively.However, the reference data was provided by experts' evaluations, where posture could be difficult to be correctly assessed with a unique 2D picture.Let us recall that ergonomic experts used to have this limited information to perform their assessment.Further investigation with more objective reference motion capture system would be required to accurately quantify the relevance of the system (Patrizi et al., 2015).As in previous studies (Dockrell et al., 2012), we also noticed inter-experts variability when assessing the same posture.In this work, our method estimated RULA scores within or very close to the range of values returned by the two experts.Involving more experts would lead to slightly increased variability, which would also lead to improved results with our method.Based on these results, we can conclude that the method could assist the ergonomists as another expert that could complement their observations at 30 Hz.

Limitations
To carry out the experiments described in this paper, we made some choices and hypotheses.Firstly, the skeleton delivered by the Kinect did not contain all the required information to accurately compute all the joint angles in accordance with the ISB recommendations.A possible solution could be to develop a more complex model as proposed by (Bonnech ere et al., 2013).However estimating new anatomical landmarks based on available joints should rely on accurately reconstructed joints.Recent papers (Plantard et al., 2015) have shown that reconstructed joints could exhibit large errors in specific postures where auto-occlusion occurs.The results reported in the first experiment are in accordance with these findings: similar errors as previous work when using no box, and larger angular errors when occlusions occur.Because of these potential large errors, it seems difficult to estimate new anatomical landmarks and further research is needed to improve the quality of the skeleton data first.In this paper, the results have been obtained with skeleton data corrected using the method of Plantard et al. (2016), but some errors remain for highly occluded conditions.
The current computation of the RULA score from Kinect joint positions relies on significant amount of manual input.Indeed, Kinect delivered very noisy and unreliable information for the hand joint(s).Hand configuration is a key point in ergonomics, as reported in the RULA assessment scores.As it is not correctly measured by the Kinect most of the time, further research would be necessary to address this particular point.In addition to hands, there are some parameters that cannot be automated to compute a RULA score, such as the Frequency and Force Adjustments.Most of the times, the legs are occluded by the workstation, which makes it impossible to decide if there is a one-or two-legs support.As a consequence, these parameters have been tuned manually in this study, which enabled us to focus on some of the main joint angles required for RULA.Further development would be needed to take the other parameters into account, such as counting the number of times a posture is shown per minute to fill-in frequency adjustment.
Another limitation is linked to the light condition.The real work experimentation was carried out in a plant, thus with indoor lighting conditions.The Kinect is based on infrared (IR) technology that could suffer from direct daylight conditions.Therefore, the accuracy in the outdoor assessment may be lower with the Kinect.
One of the major problems when applying such a method to real work conditions is the occurrence of occlusions.Using a correction method enabled us to partially overcome this limitation, as shown in the laboratory experiment with controlled occlusion situations.It seems to be in accordance with the real work conditions with various types of occlusions.However, a wider set of experimental conditions, with various types and sizes of occlusions would be required to more accurately understand the robustness of this method to occlusions.
In this work, we evaluated the algorithm using a limited set of subjects with limited anthropometric variation, assuming that the Kinect was designed for any type of population.Further experiments would be necessary to confirm that there is no actual effect of anthropometry on the results.It would require to carry out experiments with a wider set of subjects, and also a wider set of experts to better estimate the capacity of the system to deal with inter-expert variability.More specifically, the thresholds used in this experiment could be adjusted according to the expert's own values, for example using a machine learning approach.Being able to adapt the system for an expert would enable to better help the expert and support their decision-making.
Finally, the distribution of the error did not follow a normal law in the laboratory experiment.Further work and analysis would be needed in order to recognize possible reasons.

Conclusion
This paper proposed and evaluated a RULA ergonomic assessment method based on Kinect skeleton in real work conditions.The results showed that in controlled and real workstation environments, the method accurately assessed the RULA score, even in challenging environments with many occlusions.
Despite the reported limitations, the results of the current study are promising for the ergonomic evaluation of workstations.Kinect has already been considered as a promising tool to evaluate ergonomics on-site, but only with simulated postures (Diego-Mas and Alcaide-Marzal, 2014), with very simple and inaccurate posture representation, and without any joint angles computation (Patrizi et al., 2015).This study shows a practical capacity to correctly assist ergonomists in posture evaluation for working tasks, with a cheap and easy-to-use system.Fig. 4 depicts an example of a potential application based on our method, where joint angles and resulting RULA scores are provided to the ergonomists at 30 Hz, compared to traditional methods based on few key frames.It could provide the amount of time spent above a given score, as a piece of additional information given to the ergonomist.Moreover, applying this method to different geographic sites and at different periods for the same company could improve the standardization of the ergonomic evaluation campaign.
To conclude, using such an approach opens the possibility to assess continuously the postural constraint at 30 Hz with this method, but without the limitation of the direct measurement methods (e.g.complexity, calibration, work disruption, sensors attached to the worker's body).Consequently, the system is easy to use and deploy in real work conditions, without disturbing the workers and without specific engineer skills as no calibration is needed.Moreover, the method could assist the ergonomists, and improve the standardization of assessments performed in various geographic sites and periods.While it assists the ergonomist in measuring postures and scores, it also allows them to understand the work of the subject and interpret the results.Finally, one has to notice that correction runs in real time and allows the possibility to implement real-time user feedback, with potential application in training or virtual prototyping, as suggested by (Vignais et al., 2013).

Fig. 1 .
Fig. 1. a) Skeleton model provided by the method (Plantard et al., 2016).(HC) hip center, (SP) spine, (SC) shoulder center, (H) head, (SL) left and (SR) right shoulders, (EL) left and (ER) right elbows, (WL) left and (WR) right wrists, (HL) left and (HR) right hips.b) Body part coordinates (pelvic, trunk and shoulder).X-axis in red pointing forward, Y-axis in green pointing upward and Z-axis in blue pointing to the right.(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2 .
Fig. 2. Illustration of the lowering and lifting motion with a box in Front and Side target placement, from the Kinect point of view.

Fig. 3 .
Fig. 3. Illustration of the 5 workstations assessed, from the Kinect point of view.
Table 1Mean RMSE (þSD) expressed in degrees of the lowering and lifting motion between left and right shoulder flexion computed from Kinect data and those computed from reference data, in all the target placements and conditions.

Fig. 4 .
Fig. 4. Example of ergonomics application based on Kinect data.In the left part, the blue and yellow skeleton depicts the data provided by the Kinect and the 3D character shows the posture after correction of the Kinect data proposed by (Plantard et al., 2016).The right part depicts joint angle values (below) and the resulting RULA grand score (middle and top).(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) Table 4 reports the RMSE, agreement values and strength of agreement (Cohen's kappa) between RULA scores computed using the Kinect data and expert observations in real work conditions.

Table 2
Mean RMSE (þSD)expressed in RULA score of the lowering and lifting motion between RULA scores computed from Kinect data and those computed from reference data, in all the target placements and conditions.

Table 3
P o of the lowering and lifting motion between RULA scores computed from Kinect data and those computed from reference data, in all the target placements and conditions.RMSE expressed in RULA score, P o and Cohen's kappa index, between RULA scores computed using the Kinect data and expert observations in real work conditions.