Reliability and agreement of the IsoKai isokinetic lift test – A test used for admission to the Swedish Armed Forces

This study was performed to evaluate the reliability and agreement of the IsoKai isokinetic lift test as it is currently administered in admission to the Swedish Armed Forces. The study included an intrarater (n = 534) and interrater reliability sample (n = 137), of Swedish male conscripts who performed the test on two test occasions about two hours apart. Two-to-four lifts were performed at each occasion, and the highest mean (IsoKaiMF) and peak force (IsoKaiPF) produced (N) were used for evaluation. All intraclass coefficients showed excellent reliability. The interrater analyses resulted in intraclass coefficients of 0.942 (95% CI; 0.920–0.959) and 0.858 (95% CI; 0.806–0.896) for the IsoKaiMF and IsoKaiPF, respectively, while the corresponding coefficients for the intrarater analyses were 0.935 (95% CI; 0.923–0.946) and 0.865 (95% CI; 0.842–0.886). Agreement, the capability of a test to detect changes, was assessed by the standard error of measurement (SEM/SEM%) and the smallest real difference (SRD/SRD%). These estimate indicated that it is possible to achieve measurements relevant to use in real practice with the IsoKai isokinetic lift test. Bland and Altman analyses revealed no systematic errors in either sample. Based on these findings, the IsoKai isokinetic lift test is suggested to be a highly reliable test for maximal dynamic muscular strength. The test could be of use in selection procedures in order to accurately evaluate maximal dynamic muscular strength, and for evaluating longitudinal changes in strength.


Introduction
Measuring muscular strength is often of interest and importance in many areas such as sports, rehabilitation, military settings and research [1][2][3]. Measures of muscular strength could either be isometric or dynamic [2]. Standard methods of measuring maximal dynamic strength include the isoinertial one-repetition maximum test (1-RM test), using external weight loading, and isokinetic strength testing performed with various commercial devices [2,4]. In an isokinetic test, the movement velocity is held constant while the resistance adapts to the muscle force PLOS  in 2002. Out of 578 eligible for test, 44 were absent due to sick-leave or other assignments, resulting in a sample of 534 conscripts. Data for the interrater reliability part were collected from another sample of 137 male conscripts, randomly selected from 601 eligible conscript at the end of their 10 months military service in 2001. At the time of data collection, the Swedish military system was based on compulsory military service for males. To be included in the study, conscripts had to be healthy and without any pain which could influence the test procedure. The IsoKai lift tests were administered by personnel from the Swedish Defence Recruitment Agency, which were well-educated in test procedures, and all had several years of experience in physical testing. Participants signed a written informed consent after receiving written information. The individual in this manuscript (Fig 1) has given written informed consent, as outlined in PLOS consent form, to publish these case details. The study was approved by the regional ethic committee in Stockholm, Gothenburg and Orebro, Sweden, Dnr 500:16 307/01. Table 1 presents the characteristics of the two study samples.

Data collection and test procedure
In the intrarater part of the study, the same rater administered the test on every participant on two occasions. In the interater part, two raters, independent of each other, administered the test on every participant once only. In both the intra-and interrater part, all participants conducted the two tests occasions on the same day with about two hours between tests. Raters and participants were blinded to the result from the first test when the second test was conducted. The IsoKai lift test procedure. IsoKai is a device used for measuring isokinetic muscular performance during a vertical lifting procedure. The device consists of a frame holding a vertical lifting bar that, via two wires, is connected to a hydraulic system regulating the speed of the lift at 0.30 m/sec. The IsoKai device allows a small horizontal movement (6-7 cm) of the vertical bar during the lift. The muscular force (N) is, for each cm of the lift, registered by a computer connected to a force plate on which the participant was instructed to stand with feet separated by shoulder width. The test was carried out from a start position where the back was straight while being forwardly inclined and knees bent. Thereafter, a two-handed lift of a vertical bar from 30 cm above the force plate was carried out until in upright standing with the bar at shoulder level (Fig 1). The participants were instructed to use maximal effort during the lift. No shoes were allowed during test. Before the test, participants performed a 10 minutes warmup session using a cycle ergometer. Length was registered by an electronic measure instrument connected to the IsoKai device, and weight was registered by the force plate. This was done in order to adapt the IsoKai devise calculations to each participant. The rater then gave oral instructions on how to execute the lift, and, in addition, demonstrated the lift to secure a safe test. One sub-maximal practice test lift was allowed for the participant to get used to the testing procedure. Each test occasion consisted of two to four maximal lifts. For each lift, a graph in which the force was plotted against each cm of the lift was generated (Fig 2).
The IsoKai mean force (IsoKai MF ) and peak force (IsoKai PF ) measures generated at each lift were registered. The IsoKai MF represents the mean of all the registered forces (plots in the force curve) during one lift, while the IsoKai PF is the maximum force produced during the lift (the top of the force curve). Only the highest IsoKai MF and IsoKai PF measures produced during the

Statistical analyses
Descriptive data were presented with means and standard deviations (SD). The level of significance was set to 95%. Statistical analyses were performed using IBM SPSS Statistics version 23 (IBM Corporation, USA). See the S1 Appendix for the ANOVA results, and some example ICC calculations. Reliability. The intraclass correlation coefficient (ICC) was calculated together with its corresponding 95% confidence interval (95% CI) [17][18][19]. The ICC 1,1 equation was used for the intrarater reliability calculations, and the ICC 2,1 equation for the interrater reliability calculations [17][18][19].
The equation variables were estimated using repeated measures analysis of variance (ANOVA). BMS represents the between-people variability of the measurement, WMS the within-people variability, JMS the between-rater variability, EMS the residual mean square (error) variability, k the number of raters, and n the number of participants (S1 Appendix) [17][18][19]. An ICC coefficient of 1.0 indicates perfect reliability of a measurement while zero indicates a totally unreliable measurement [6]. Evaluation of the ICC coefficients was interpreted using the classification suggested by Cicchetti, with reliability coefficients categorised as: 0.75 to 1.0 = excellent, 0.60 to 0.74 = good, 0.40 to 0.59 = fair and less than 0.40 = poor [20].
Agreement. Agreement was assessed using the standard error of measurement (SEM) and the smallest real difference (SRD), along with their 95% CI [21][22][23]. The SRD shows the limit for the smallest difference (N) that indicates a real change for an individual. Further, the SEM% and SRD% were calculated to be able to evaluate the measurement error and individual changes independently from the units of measurement (N) [23]. SEM% and SRD% are recommended to be used for comparison with other studies, especially to account for the fact that different units of measurement are used [23].Formulas used to assess agreement: where � d represent the overall mean difference between two tests.
Grand mean = (the sample mean force of the IsoKai MF or IsoKai PF from test occasion 1 + the sample mean force of the IsoKai MF or IsoKai PF from test occasion 2) / 2.
Bland and Altman methods. Bland and Altman methods were used to assess the dispersion of data, measurement error and possible systematic bias of the test [7,24]. These methods were based on the analysis of differences between measurements from the repeated test occasions. Using Bland and Altman plots, the mean values from two tests were plotted against the difference between the tests for each participant. The Bland and Altman plots visualised the distribution of the individual test differences around the overall mean difference between tests ( � d ) together with the limits of agreement (LOA).
The LOA which represented the precision of the measurement was calculated by the formula: � d � 2 standard deviations ðSDÞ [24]. Further, as an overall mean difference between the tests occasions ( � d ) significantly different from zero indicates a systematic bias in the test, a possible bias was formally assessed by estimating � d together with the 95% CI [18,24,25]. The 95% CI of � d were calculated using the formula; � d � t nÀ 1 , where t n-1 represents the probability point of the t distribution on n-1 degrees of freedom, and SE the standard error of � d [18]. Table 2 presents the sample mean force and the "Grand mean" for the IsoKai MF and IsoKai PF measures in the intra-and interrater samples respectively.

Reliability
All results showed that the IsoKai isokinetic lift test had excellent reliability, with ICC coefficients ranging from 0.858 to 0.942 (Table 3).

Agreement
The agreement estimates based on the ANOVA analyses (SEM, SEM%, SRD and SRD%) indicated a higher degree of agreement for the IsoKai MF than IsoKai PF , and a higher degree of agreement for the interrater sample compared to the intrarater sample (Table 4).

Bland and Altman methods
Bland and Altman results for the intra-and interrater samples are presented in Table 5 and Figs 3 and 4. The overall mean difference between tests ( � d ) in the intrarater sample was significantly different from zero, with-6 N (95% CI; -10 to -3) for the IsoKai MF and-14 N (95% CI; -23 to -4) for the IsoKai PF .

Discussion
This study revealed that the IsoKai isokinetic lift test is a highly reliable test for evaluating maximal dynamic muscular strength related to lifting. To establish if a measurement gives reliable information, assessing both measures of reliability and agreement is recommended [6,7,22]. Our analyses demonstrated that the IsoKai isokinetic lift test had excellent reliability for measuring maximal strength irrespective of whether the tests were performed by the same rater (intrarater reliability) or by two different raters (interrater reliability). Overall, the ICC  [14]. All the lower limits of the ICC confidence intervals exceeded 0.8 which further support the excellent reliability of the IsoKai isokinetic lift test.
There are no general recommendations set for how to evaluate agreement estimates as the evaluation depends on the context in which the test is used [7]. Therefore, the results in the present study could be evaluated in the light of finding potential changes in maximal muscular strength due to training or de-training [26]. The SEM and SEM% represent the error of the measurement. The SEM in the two samples were 29 and 25 N for the IsoKai MF , and 78 and 73 for the IsoKai PF . If taking the SEM of 29 N IsoKai MF in the interrater sample as an example, this corresponds to a measurement error of only 3 kg (29N � 0.102). We believe this supports the IsoKai test to be a rather precise measurement of changes in dynamic muscular strength. The precision of measuring individual changes represented by the SRD and SRD% was less pronounced, with SRDs between 69 to 217 N (Table 4), or 7 to 22 kg. The Bland and Altman analyses showed no major signs of systematic errors in either the intrarater sample or the interrater sample. The plotted differences between two tests were evenly distributed about the overall mean difference ( � d ) in all analyses. In the intrarrater sample a statistically significant difference from zero was found specifically for the mean difference between tests, with values of -6 N and -14 N for both the IsoKai MF and IsoKai PF , respectively (Table 5). This corresponded to about 1% of the "Grand mean" force values; a minor Table 3

Intrarater reliability (n = 534)
Interrater reliability (n = 137) systematic error which could be negligible in this case. Concerning the interrater sample, about the same differences from zero as in the intrarater sample were found for the overall mean difference ( � d ), however, these were not statistically significant.

Comparison with other studies
Isokinetic strength measures have been criticised as they mostly engage only one or two joints and their associated muscle groups which bear little relationship to the multi-joint and multimuscle actions that take place during movements in real practice [4]. As such, the IsoKai lift isokinetic lift test is unique among isokinetic tests as it engages almost all muscle groups and joints in the body in a movement imitating a normal lifting procedure. As mentioned in the introduction, Larsson et al. (2009) assessed the reliability of the IsoKai test as it was performed in the SwAF before 2013 [16]. By using the mean value of two IsoKai MF registrations as a measure of muscle strength capacity in a sample of Swedish conscripts (n = 427), they found an ICC 3.1 of 0.94, a SEM of 30 and a SEM% of 4.3, findings very similar to ours. The IsoKai PF was not evaluated. In addition to this study, we found two reliability evaluations of multi-joint isokinetic tests similar to the IsoKai isokinetic lift test [27,28]. Bridgeman et al. found an isokinetic multi-joint squat devise to have good test-retest reliability, which was evaluated within 3 test sessions over a 3-week period in a sample of 10 strength trained male athletes. The concentric peak force (N) outcome revealed ICCs ranging from 0.87 to 0.98, and coefficients of variation (CV) from 7.6 to 15.4. The CV was calculated by dividing the method error (ME) by the overall mean difference of two tests ( � d ) and then multiply by 100 [23]. Lexell and Downham states that "if the sample size is sufficiently large and the mean difference small, both highly likely conditions, ME and SEM take similar values", indicating that CV and SEM% could be comparable between studies [23]. In 1997, Wilson et al. examined another isokinetic squat device by measuring concentric peak force (N) in 29 athletic male subjects performing two tests with 3 minutes rest in-between, and found an ICC of 0.89 and a CV of 8.7 [27]. The Iso-Kai PF measures in the current study estimated a muscular force comparable with the outcome in the above mentioned squat tests, and resulted in ICCs of the same magnitude (0.858 and 0.865, Table 3). The IsoKai PK SEM% of 5.4 and 6.2 indicate better precision of the IsoKai lift test compared to the CVs from the squat tests evaluations; however, such a comparison should be done with caution since the small sample sizes might have influenced the results.

Methodological considerations
Today many women apply for enrolment in physically demanding occupations such as in the police force and military service. As such, the lack of women in our samples could be regarded as a limitation. However, unpublished data from the SwAF indicates excellent ICC values for women (n = 18) regarding the IsoKai MF (ICC 1.1 and ICC 2.1 of 0.811) and IsoKai PF (ICC 1.1 and ICC 2.1 of 0.767). Given these preliminary results, we find no reason to believe that the IsoKai isokinetic lift test would not be a reliable test among women. The data in the present study might be regarded as old since it dates back to 2001 and 2002. Still, as the IsoKai device used at time of data collection was exactly the same type as used today, we believe that the results are still applicable. There are several suggested ICC equations recommended in the literature depending on the context of the test procedures, but no standardised system on how to choose the most appropriate ICC exists [7,17,18]. We decided to use the ICC 1,1 equation (intrarater reliability) and the ICC 2,1 equation (interrater reliability) in the analyses, as they corresponded well to recommendations when compared to the test proceedings in which the IsoKai lift test was performed, and how it is used in the Swedish Defence Recruitment Agency [17][18][19]29]. A number of arbitrary recommendations exist regarding the interpretation of satisfactory levels of reliability of a test when using the ICC. Resultantly, we chose the recommendations by Ciccetti [20]. Portney and Watkins suggest that an ICC above 0.75 indicate good reliability, while Currier et al suggest coefficients ranging from 0.80 to 0.90 indicate good reliability and above 0.90 indicate high reliability [6,30]. No matter which of these recommendations used, the Iso-Kai isokinetic lift test would be regarded as a test with good-to-high levels of reliability even if the lower limit of the CI is considered, as is recommended [7,29]. The overall lower ICCs and larger agreement measures for the IsoKai PF compared to the IsoKai MF might be explained by the fact that the IsoKai PF is registered momentarily while the IsoKai MF is averaged during the completed lift. Therefore, the measurement error inherent to the device itself, or differences in the momentarily force produced by the participant, might result in a larger WMS and EMS estimates for the IsoKai PF than the IsoKai MF , causing these differences [22]. This might also affect the relationship between the BMS and the WMS/EMS, as reflected in the F statistics of the ANOVA (S1 Appendix). A high F statistics indicates a large discrepancy between the BMS and WMS/EMS, as in the IsoKai MF , which results in high ICC measures, while a lower F statistics, as in the IsoKai PK , results in lower ICC measures [6,22,31]. We believe that our sample sizes are considered a strength as it widely exceeds recommendations for reliability studies. Bonett et al. suggests a sample size of 108 subjects in order to detect an ICC of 0.70 or higher with a power of 0.80 and an alfa level set to 0.05 [32]. With the same power and alfa level, Walter et al. estimated a sample of 117 subjects is required to find an expected reliability of 0.80 [33]. Bonnet et al. further emphasise that it is unnecessary to include more subjects than needed for the desired power, as it increases the costs of research, and do not necessarily improve the results [32]. We fully agree, but as the data in our study already were collected as a part of the SwAF admission procedure, we found no reason not to use the entire sample. Another strength is that our data were collected in a way that Carter et al. call a "partially standardised approach" which has the purpose of describing reliability with the levels of standardisation that could be achieved in real settings [31]. We believe this to further supports the findings of the IsoKai lift test as a reliable measurement when implemented into real practice. Finally, we have fulfilled all the requirements in the GRRAS checklist for reporting of studies of reliability and agreement [7].

Implications of the results
Military personnel are often exposed to high physical demands in their work tasks, especially during military missions [34][35][36]. Common tasks during military service include marching, digging, carrying and lifting, with lifting to be the most frequent work task [11,34,37]. In 2015, Larsson et al. found that the IsoKai isokinetic lift test had excellent content validity with respect to digging, carrying and lifting tasks [14]. Hence, to use it as an admission test in military settings and other physically demanding occupations with similar work tasks would be of great benefit. Further, as we evaluated the IsoKai isokinetic test using a representative population in comparison with the target population, the results support the use of the test in military settings as well as other similar settings e.g. the police force. The IsoKai lift test is fast, easy to assess and relatively safe as the resistance in the isokinetic test adapts to muscle force [2,5]. This makes it an excellent test in selection procedures targeting large groups especially where time constraints exist. The test also has some drawbacks that have to be mentioned. The device is difficult to transport, needs experienced personnel for calibration, and is quite costly. It is important to notice that the reliability estimates are dependent of the variability of the outcome in the population measured, which might limit the external validity [7,22]. The mean age in our samples was 19 years, which could be argued to restrict our findings to a young population. However, even if our samples are homogenous regarding age, the between subject variability (BMS) of the outcome showed large heterogeneity in all our analyses (S1 Appendix). Therefore, our results could probably be valid even for other age categories as it is not the actual age that matters but the variance of the measures when evaluating reliability [6]. The IsoKai isokinetic lift test could also be used to assess clinical important changes in muscular strength in individuals following an intervention, for example in sport or sports medicine. Measures of agreement, such as the SEM and the SRD, are not dependent on the variability between subjects, since only the measurement error is of importance [22]. Therefore, measures of agreement could to a greater extent be used in various populations other than reliability measures.

Recommendations for future research
The constant speed and varying resistant in an isokinetic strength test have advantages, as discussed earlier, but could also be a disadvantage in relation to muscle performance in real practice where speed typically varies and load remains constant. This could challenge the validity of the IsoKai isokinetic lift test in comparison to isoinertial lifting capacity. One possible solution to this could be to investigate the criterion or concurrent validity of the IsoKai lift test in relation to maximal strength measured with an isoinertial lift test, for example a one repetition maximum lift test. Moreover, we believe that examining the reliability of the IsoKai isokinetic lift test in other populations would be beneficial in order to increase the external validity of the test.

Conclusion
The IsoKai isokinetic lift test was found to be a highly reliable measurement of maximal dynamic muscular strength. The test could be used to monitor dynamic muscular strength for the purpose of distinguishing between individuals. As such, the IsoKai isokinetic lift test could be recommended for use as a test for selection based on capacity levels regarding maximal dynamic muscular strength in military settings as well as other physically demanding occupations. In addition, the test is useful for evaluating changes in maximal muscle strength in individuals following interventions.