Reliability and smallest detectable change of the Danish version of the Pain Self-Efficacy Questionnaire in patients with chronic low back pain

Introduction

Globally, low back pain (LBP) is one of the most common health problems and the leading cause of disability [1], [2]. Chronic LBP (CLBP) is associated with multiple factors such as disability, reduced quality of life, loss of or limitations in work and lack of personal financial prosperity [1], [3]. Psychological factors including self-efficacy also play an essential role in developing disability [1], [3]. Self-efficacy is defined as a personal belief in one’s ability to successfully cope with difficult situations [4]. Low pain self-efficacy has been identified as an independent predictor of poor LBP outcome six months after consulting the general practitioner [5]. The Pain Self-Efficacy Questionnaire (PSEQ) measures patients’ belief in accomplishing a range of activities despite their experience of pain. The questionnaire is commonly used to assess pain self-efficacy in patients with CLBP [6]. A systematic review of outcome measures to assess self-management in clinical trials involving patients with chronic pain indicated that several important aspects of the measurement properties of the PSEQ such as reliability, responsiveness and interpretability needed to be further studied [7]. Moreover, it is essential to examine the psychometric properties of adapted outcome measures in the population and context where they are going to be used. Psychometric properties may thus vary between versions, populations and settings due to the diversity in translation, study sample or cultural factors [8]. The PSEQ has been translated and cross-culturally adapted to a Danish population of patients with fibromyalgia [9], but the psychometric properties have not yet been assessed in Danish patients with CLBP. Therefore, the objective of this study was to evaluate the reliability and smallest detectable change of the PSEQ-DK in a population of Danish patients with CLBP prior to implementing it as an outcome measure in an inpatient rehabilitation context.

Methods

This was a longitudinal reliability study. The study is presented according to the recommendations from STROBE [10]. Patients were recruited between September 2017 and January 2020 from a Danish multidisciplinary rehabilitation facility. Patients were referred to a rheumatologic multidisciplinary rehabilitation programme consisting of one pre-admission day and after two weeks a four-week inpatient programme. The programme was based on the widely accepted multidisciplinary approach [11] and consisted of more than 30 different clinical activities targeting a wide range of factors driving disability. It was delivered by a multidisciplinary team (rheumatologists, physiotherapists, occupational therapists, and nurses educated as coaches and focussing on the psychological aspect of the programme). The programme was partially standardized and partially tailored, and each day the patients attended an 8–10 h programme alternating between (1) group lecture and dialogue, (2) group sessions (supervised and unsupervised), (3) individual counselling, and (4) unsupervised individual exercise. Further details have been published earlier [12]. The eligibility criteria used in this present study were identical to those used in a randomised controlled trial (RCT) conducted at the same multidisciplinary rehabilitation facility [12], [13], [14]. Thus, patients could be included if they were 18 years or older and if they had experienced CLBP for more than 12 months (with or without widespread pain and/or with or without sciatica).

Patients were excluded if they had severe systemic diseases (American Society of Anesthesiologists Physical Status classification system ≥3), active cancer, active psychiatric pathology, were pregnant, a diagnosis of axial spondyloarthritis, spinal fracture within the last three months, severe osteoporosis, inability to speak and understand Danish or had no e-mail address. The inclusion procedures described below were identical to those in the previously mentioned RCT [12], [13], [14]. Patients were contacted and orally informed about the study before written information and an informed consent form were forwarded by e-mail. Patients were included if they returned a signed informed consent form. A secure electronic database was used to e-email questionnaires and store data. Patients were asked to complete the PSEQ-DK at two time-points (Figure 1); T0 (baseline = three weeks prior to the pre-admission day) and T1 (one week after completing T0). If the questionnaires were not completed at T0 or T1 patients were contacted by telephone and encouraged to complete the questionnaire.

Figure 1:

Overview of the two time-points T0 (baseline = three weeks prior to the pre-admission day) and T1 (one week after completing T0).

Data on sex, age, marital status, smoking habits, employment and education status, pain level and level of disability were collected at baseline.

The PSEQ-DK consists of 10 items, each item is scored on a seven-point Likert scale with responses ranging from 0 (not at all confident) to 6 (completely confident) [15]. A total score is calculated by summing the scores of the 10 items yielding a total sum score ranging from 0 to 60; higher scores indicate a stronger belief in self-efficacy [9], [15].

Results

A total of 160 patients with CLBP were assessed for eligibility (74% women; median age 57.5, IQR 45.5–68). The eight patients who declined to participate differed from those who chose to participate with respect to sex (p<0.05). The 12 patients who were lost to follow-up by T1 did not differ from those who completed the study with respect to age and sex (p>0.05). A flow chart illustrating study inclusion is shown in Figure 2.

Figure 2:

Flow chart of patient inclusion.

Baseline patient characteristics concerning reliability are presented in Table 1.

Reliability of the PSEQ-DK was evaluated in 92 patients. The mean follow-up time between T0 and T1 was 9.26 days (95% CI 8.24; 10.29). No statistically significant systematic differences were found between T0 and T1 on the PSEQ-DK with an estimate of −0.85 (95% CI −2.18; 0.49). The PSEQ-DK mean score for T1 was 29.68 (95% CI 27.44; 31.92). Ninety-five percent limits of agreement were estimated to −13.47 to 11.78 and are presented in Figure 3. Standard error of measurements_agreement was 4.57 and the smallest detectable change was 12.67. The weighted kappa was 0.82 (95% Cl 0.75; 0.88). The ICC_agreement for individual total scores was 0.83 (95% Cl 0.75; 0.88).

Figure 3:

Bland–Altman plot illustrating the mean differences between T0 (baseline = three weeks prior to the pre-admission day), T1 (one week after completing T0) of the Danish version of the Pain Self-Efficacy Questionnaire and the limits of agreement.

The solid line below the centre line represents the mean difference with 95% CI (dashed lines); and the solid outlying lines represent the 95% limits of agreement.

There were no floor or ceiling effects present for the PSEQ-DK total scores at T0 or T1. At T1, item 7 (“I can cope with my pain without medication”) was found to have floor effect with 48% reporting 0 (not at all confident).

Discussion

The present study evaluated the reliability and smallest detectable change of the PSEQ-DK in a rehabilitation context. The results showed that the PSEQ-DK is a reliable outcome measure for measuring pain self-efficacy.

Reliability was found to be good exceeding the acceptable threshold with a weighted kappa of at least 0.81 (8) and an ICC_agreement value of at least 0.70 [16]. This is in agreement with a previous study that obtained an ICC_agreement of 0.82 in a population of patients with CLBP [18]. Other studies of the original and translated versions of the questionnaire found similar results using ICC (range 0.80–0.91) [9, 19, 20] and the Pearson correlation (0.73) [15]. However, since these studies have not specified the definition they have used of the ICC and the Pearson correlation not taking potential systematic differences between the two measurement time-points into account, it was not possible to equate the results directly to the ICC_agreement in the present study.

The 95% limits of agreement and standard error of measurements_agreement found in the present study are fairly comparable to those found in a previous study in patients with CLBP [18]. These estimates together with the smallest detectable change provide relevant knowledge concerning the change that may be attributable to measurement error.

The smallest detectable change estimated in the present study expressed, that only a change of more than 12.6 points on the PSEQ-DK can be considered as a “real” within person change for patients with CLBP on the construct pain self-efficacy.

In addition, information of the smallest detectable change and the minimal clinical important change is of importance to accurately interpreting change scores in both research and clinical practice [8], [16].

To allow further comparisons, future studies on the PSEQ-DK in patients with CLBP in a rehabilitation context should estimate and state standard error of measurements and the smallest detectable change.

The PSEQ-DK total scores showed no floor or ceiling effects, which is in line with a previous study [18]. However, a floor effect was found for item 7 (48%) at T0. Although item 7 has been found problematic in previous studies examining the validation of the PSEQ-DK [9], [15], it is retained in the questionnaire due to its clinical utility [21].

Almost 1/3 of the patients who accepted participation were excluded due to mainly unknown reasons. Further, a considerable proportion of patients were lost to follow-up by T1. This could be due to changes in the administrative organisation at the rehabilitation facility during the course of the study. Thus, the large number of patients lost to follow-up is presumably not caused by systematic errors or associated with the construct under study.

The strengths of this study include reaching the desired sample size for the reliability analyses. Moreover, the reliability analysis was conducted in a stable population of patients with CLBP. There is no standard for the optimal interval between questionnaires when examining reliability, but the literature indicates an interval one to two weeks as appropriate [8], [16]. Thus, we assume the present study is not subject to recall bias in the reliability analysis since the mean follow-up time was approximately nine days. Moreover, it is a strength that the PSEQ-DK is tested in an inpatient rehabilitation context before implementing it as an outcome measure in the same context.

It is considered a limitation that it was not possible to investigate responsiveness and minimal clinical important difference of the PSEQ-DK. Originally the present study was designed to assess the reliability and responsiveness of several patient reported outcome measures planned to be used in the aforementioned randomised controlled trial [12], [13], [14]. The Oswestry Disability Index was planned to be the primary patient reported outcome measure in the RCT. For that reason, an LBP specific Global Rating of Change Scale was chosen to assess the degree of improvement experienced by the patients when analysing responsiveness in the present study. When interpreting the responsiveness results of the PSEQ-DK we realized that assuming PSEQ-DK and LBP specific Global Rating of Change Scale to have similar constructs seemed rather problematic. Thus, being aware of this methodological constraint, we decided not to present the results from our responsiveness analysis on PSEQ-DK. Looking back, more attention should have been payed to the design features of the chosen Global Rating of Change Scale before choosing it [22].

The smallest detectable change should be smaller than the minimal clinical important difference for evaluative purposes [16]. Therefore, it would have provided important knowledge to the interpretability and use of the PSEQ-DK if we could have estimated and presented a minimal clinical important difference.

As regards generalisability, this study was performed in a population of patients with CLBP and therefore its results are not necessarily generalisable to other than comparable populations, thus reliability estimates are dependent and potentially influenced by the characteristics of the population under study [8]. Therefore, future studies ought to focus on evaluation of the measurement properties of the PSEQ-DK in various homogeneous pain populations before being applied in clinical practice.

The present study demonstrated that the PSEQ-DK showed good reliability in patients with CLBP in an inpatient rehabilitation context. For rehabilitation professionals the smallest detectable change could be used as a limit value for individual patients, when considering if the patient has experienced a real change. However, the psychometric properties including responsiveness and interpretability of the PSEQ-DK should be further assessed in future studies.