Longitudinal measurement invariance of the Multiple Sclerosis Walking Scale-12,☆☆

https://doi.org/10.1016/j.jns.2011.03.008Get rights and content

Abstract

Objective

One primary assumption underlying the unambiguous interpretation of change in Multiple Sclerosis Walking Scale-12 (MSWS-12) scores over time is longitudinal measurement invariance (i.e., Is the same construct being measured over time?). Such an assumption was tested in the present study over periods of 6 and 12 months in persons with relapsing–remitting multiple sclerosis (RRMS).

Method

Participants completed a battery of questionnaires that included the MSWS-12 at baseline (n = 269) and 6-months (n = 260) and 12-months (n = 252) follow-up. The data were analyzed using confirmatory factor analysis and a series of nested model comparisons in Mplus 3.0.

Results

The results indicated that the unidimensional measurement model and all of its parameters (e.g., factor loadings and item intercepts) were invariant over periods of 6 and 12 months.

Conclusion

We provide novel evidence that supports the unambiguous interpretation of scores from the MSWS-12 as a measure of change in walking impairment over time in a sample of persons with MS.

Introduction

Walking impairment is a common outcome for monitoring change in the disease status of persons with multiple sclerosis (MS) [1], [2]. This impairment has often been measured in clinical research and practice using the 500 meter walk portion of the Expanded Disability Status Scale (EDSS), timed walk tests (e.g., timed 25-foot walk or 6-minute walk), and quantitative movement analysis (e.g., gait kinematics) [1], [2]. There are fewer self-rated assessments of walking impairment in MS, despite the recognized importance of patient-reported outcomes [3]. The best validated patient-report outcome is the Multiple Sclerosis Walking Scale-12 (MSWS-12) [4], [5], [6], [7]. This scale has excellent face validity and researchers have provided evidence for its structural integrity, construct validity, and responsiveness in persons with MS [4], [5], [6], [7]. MSWS-12 scores have recently been included as a patient-reported outcome for validating the clinical significance of the timed-walk responder analysis in randomized controlled trials of sustained-release oral fampradine [8], [9].

Importantly, the administration of the MSWS-12 and interpretation of its scores for quantifying changes in walking impairment over time rest upon the assumption of longitudinal measurement invariance; this is true of all patient-reported outcomes, including those with obvious and extensive validity. As stated by Horn and McArdle ([10], p. 117), “The general question of invariance of measurement is one of whether or not, under different conditions of observing and studying phenomena, measurements yield measures of the same attributes.” Essentially, when item responses are combined into a composite score, as is done in clinical research and practice with the MSWS-12, the assumption is that all measurement characteristics of the items (e.g., factor loadings, item intercepts, and item residual variances) and scale itself (e.g., dimensionality, factor variances, and factor means) are equivalent over time. The lack of measurement equivalence for item and scale characteristics over time for the MSWS-12 would seemingly result in successively larger differences in composite scores, and would indicate that perceived walking is changing, when in fact no change in the construct itself has occurred; the opposite is possible too whereby a lack of true change in the construct itself is masked by inherent measurement variability over time. Collectively, longitudinal factorial invariance concerns the equivalence of corresponding parameters in a common factor model over time within a population group [11], [12], [13], [14]. This is a prerequisite condition for both the generation of composite scores and interpretation of change in clinical research and practice [11], [12], [13], [14].

There are four primary forms of factorial invariance, namely configural, metric, scalar, and strict factorial invariance [11], [12], [14]. Each type forms a progressive hierarchy whereby increasing levels of cross-time restrictions are placed on parameters in a common-factor model. Configural invariance, for example, requires the same number of common factors with an identical pattern of item or factor loadings per factor over time. For the MSWS-12, a scale with a hypothesized single common factor, this means all items should have significant loadings on a single factor at each measurement occasion. Metric invariance further requires the corresponding factor loadings to be equal in magnitude over time. The next step in the hierarchy involves scalar invariance and assumes equivalent item intercepts over time. The last step involves strict factorial invariance and tests the equivalence of item residual variances, factor variances, and then factor means over time. This last step is critical for defensible comparisons of composite or mean scores and variance estimates over time. Collectively, the presence of longitudinal measurement invariance is a necessary psychometric property for meaningful interpretation of change in composite or mean scores and variances over time on all patient-report outcomes including the MSWS-12 [11], [12].

To that end, the present study directly tested the longitudinal measurement invariance of the MSWS-12 over periods of 6 and 12 months in an ongoing, prospective observational study of persons with relapsing–remitting multiple sclerosis (RRMS). The two different periods reflected the timing of outcome assessments common in longitudinal observational studies and clinical trials of MS making this more applicable for clinical research and practice. We do recognize that this may be a novel methodological approach in neurology, but reviews of this methodology are common [10], [12], [14], even in medical journals [11], making the application of measurement invariance pertinent and applicable for a broad range of audiences, including disability and rehabilitation in MS [15].

Section snippets

Participants

The complete description of the recruitment, inclusion criteria, and baseline characteristics of the sample has been reported elsewhere [16]. The sample was recruited through a research advertisement posted on the National MS Society (NMSS) website and distributed through 12 chapters of the NMSS. Those who were interested in the study contacted the research team by either e-mail or a toll-free telephone call. This was followed-up by a telephone call from the project coordinator who described

Descriptive and distributional statistics

The mean, standard deviation, range, skewness, and kurtosis for MSWS-12 composite and item scores over time are provided in Table 1. The skewness and kurtosis estimates divided by the standard error all exceeded a z-score of 1.96 and indicated that the data were not normally distributed thereby justifying the selection of the MLR estimator for the invariance analyses. The raw data for computing the zero-order correlations among MSWS-12 items are not reported in the paper based on space

Discussion

Validation of scores from a patient-report outcome measure is an ongoing and evolving process [20] and this study extended the validity of scores from the MSWS-12 by examining its longitudinal measurement invariance. The MSWS-12 demonstrated evidence of configural, metric, scalar, and strict longitudinal factorial invariance over both 6-month and 12-month periods of time. Accordingly, this study provides novel evidence for the unambiguous interpretation of change in composite MSWS-12 scores

References (20)

There are more references available in the full text version of this article.

Cited by (19)

  • Walking measures to evaluate assistive technology for foot drop in multiple sclerosis: A systematic review of psychometric properties

    2018, Gait and Posture
    Citation Excerpt :

    Table 2 presents an overview of the results together with the COSMIN rating and the rating of the quality of the results according to the revised Terwee criteria [24–26]. The methodological quality of the studies was rated according to the COSMIN checklist as ‘good’ (n = 3) [18,33,43], as ‘fair’ (n = 3) [15,34,44] and ‘poor’ (n = 12) [17,35,36,38,40,45–51]. The main reasons for a lower score included not reporting the intraclass correlation coefficient (ICC) or weighted Kappa, not describing the ICC model used, small sample size and the lack of an explicit statement that the repeated measurements were independent.

  • Balance, gait, and falls in multiple sclerosis

    2018, Handbook of Clinical Neurology
    Citation Excerpt :

    There are two versions of this scale, with slightly different scoring, but the same questions and scores for both versions can be transformed to a 0–100 scale (Nilsagard et al., 2007). The MSWS-12 scale has shown excellent test–retest reliability (Motl et al., 2011), internal consistency (McGuigan and Hutchinson, 2004), and concurrent validity (Goldman et al., 2008). An electronic version of this scale is available and includes item response theory-based scoring, error checking, and an estimated timed 25-foot walk derived from MSWS-12 responses (Engelhard et al., 2016).

  • Effect of dronabinol on progression in progressive multiple sclerosis (CUPID): A randomised, placebo-controlled trial

    2013, The Lancet Neurology
    Citation Excerpt :

    We computed MSFC component-wise Z scores15 using results from all participants at visit 2 as reference data.14 We calculated MSFC composite scores from the mean of component-wise Z scores and total scores for MSWS-1216 and RMI using an algorithm analogous to that used for the MSIS-29-PHYS for dealing with missing data. We analysed repeated measures data for MSIS-29-PHYS with multilevel models, which included time (visit) and the other prespecified covariates.

View all citing articles on Scopus

Disclosure: The authors report no conflicts of interest.

☆☆

Funding: This research was supported by a grant from the National Multiple Sclerosis Society (RG 3926A2/1).

View full text