Development and preliminary validation of the Chronic Pain Acceptance Questionnaire for Clinicians

Phase one: Rasch analysis of the Chronic Pain Acceptance Questionnaire for Clinicians

This was a cross sectional study. Participants (n = 162) were recruited from attendees on a workshop led by one of the authors (KEV) (n = 20), and through multimedia advertisements worldwide. Participants were directed to give informed consent, and subsequently complete a questionnaire, online. Participants were asked to give their occupation, work clinical setting (e.g. hospital pain management clinic, private practice), and number of years qualified.

The Chronic Pain Acceptance Questionnaire for Clinicians (CPAQ-C) (see Appendix 1) was developed by the authors by converting the Chronic Pain Acceptance Questionnaire (CPAQ) [11]. It therefore contained 20 items measuring clinicians’ beliefs regarding the importance of levels of acceptance in patients with chronic pain. Clinicians are asked to rate each statement such as, “Patient’s lives can go well, even though they have chronic pain,” on a scale from, “0 – Never true,” to “6 – Always true.” Items 4, 7, 11, 13, 14, 16, 17, 18 and 20 are reverse scored. Total scores can range from 0 to 120 points, with higher scores indicating higher perceived clinician importance regarding acceptance in patients with chronic pain.

Statistical analysis

Descriptive statistics were calculated for demographic data using Stata 13.1 (Statacorp, Texas, USA). Rasch analysis is used extensively to assess the psychometric properties of scales (see Bond and Fox [19] for a comprehensive overview) and has been used previously to interrogate the patient version of the CPAQ [20]. The analysis was conducted using Winsteps software (v3.92.1, www.winsteps.com) and the Andrich Rating Scale model was chosen due to the relatively small sample and the number of response categories.

Previous studies have suggested the two components of pain acceptance, activity engagement and pain willingness, represent independent constructs [6], [20] and a preliminary regression analysis of raw CPAQ-C scores suggested these components shared only 30% of variance. An exploratory analysis was thus conducted to confirm the presence of two CPAQ-C subscales and ensure the suitability of the sample for scale evaluation. A principal components analysis of residuals (PCA) was conducted and the residual correlation matrix visually inspected to identify item clusters indicative of two independent subscales. Suitability of the data for analysis was assessed by comparing how well the scale items targeted the sample. The following components of the two subscales were then considered independently: item targeting and hierarchy, category ordering, dimensionality, item and person fit, internal consistency and differential item functioning.

To assess subscale item targeting, a comparison between the measures of item endorsibility (i.e. how easy an item was to endorse) and person agreeability (i.e. how agreeable the sample was) was made by visually inspecting the person-item distribution map and by comparing the summary statistics.

To assess the functioning of the Likert-type scale categories, the category ordering was assessed. The CPAQ-C provides seven response categories (1–7) and therefore has six step-calibrations; the points at which the likelihood of endorsing one category is equal to that of endorsing the next. The presence of disordered step-calibrations is indicative of under-utilised categories and can influence the function of the scale. If disordering was detected, an exploratory analysis with the Likert-type categories collapsed was conducted to explore whether fewer categories improved the fit of the items.

Unidimensionality was assessed through analysis of item fit statistics and through PCA. Fit statistics are chi-square-based and indicate how well the data conform to the Rasch Model. They are reported as mean squares (in logits) with an expected value of 1 logit. Both item and person fit statistics were analysed to identify poorly functioning items or persons with unexpected responses. The characteristic curves of items with infit (information-weighted) and outfit (outlier-sensitive) fit statistics >1.4 (model underfit) or <0.6 (model overfit) were analysed further in consideration of the other tests conducted [21]. Poor person fit may reflect respondent carelessness and potentially compromise item fit. For the purpose of scale analysis, persons with outfit statistics >2 logits were removed and the data reanalysed.

The PCA residual correlation matrix was inspected visually to identify the presence of further dimensions. Item clusters with substantial positive or negative loadings equivalent to an eigenvalue greater than 2 were reviewed to ascertain whether the assumption of unidimensionality was breached. Local independence was also assessed as part of the PCA. Local independence assumes the responses to one item do not influence responses to other items. Breaches of this assumption can inflate the reliability of scales. Items with residual correlations greater than 0.3 were reviewed to determine whether they duplicated each other and could thus be considered redundant.

Subscale internal consistency was evaluated using the person separation index (PSI); a marker of how well the subscales differentiate persons of differing agreeability. Scales with PSI values above 0.7 and 0.85 suggest the scale is appropriate for group and individual use, respectively [22].

An analysis of Differential Item Functioning (DIF) was conducted to determine whether person attributes additional to pain acceptance bias the responses to CPAQ-C items, contributing to model misfit. DIF was conducted to assess the influence of years of experience and HC-PAIRS score. Groups were dichotomised using a median split and items with statistically significant (P < .01) contrasts >0.5 logits were further explored.

Activity engagement subscale

Analysis of targeting, shown in Figure 2, suggested the sample found the subscale items relatively easy to endorse. The average person agreeability of 1.02 (1.12) logits (range = −1.90 to 5.51 logits) was comparatively higher than the default average item endorsability of 0 (.67) logits (range = −0.86 to 1.64 logits) suggesting more challenging items are needed to effectively measure clinicians. Table 2 displays the items in hierarchical order, with higher thresholds indicating items that are harder to endorse; an item reliability index of 0.98 suggesting the sample was sufficient to confirm the item hierarchy is reproducible. Clinicians found Item 5 (“It’s not necessary for patients to control their pain in order to handle their life well”) the hardest item to endorse and Item 3 (“It’s OK for patients to experience pain”) the easiest. The hierarchical order of the items provided evidence of construct validity with items suggesting patients are able to live fulfilling lives with pain (items 3, 6, 8 , 9) being more readily endorsed than items suggesting pain will not disrupt life (items 1, 2, 12, 15).

Figure 2:

Item-person threshold map for the Activity Engagement Subscale of the CPAQ-C. Person measures represent 162 clinician responses and items represent the average measure threholds of the 11 items. Note, less agreeable persons and easily endorsed items are located to the left side of the logit scale (i. e., <0 logits) and more agreeable persons and harder to endorse items are located to the right of the logit scale (i. e., >0 logits). Overall average item endorsability is set at 0 logits by default.

Visual inspection of the category ordering showed no disordering of thresholds but Category 2 (Seldom true) appeared to be underutilised. Table 2 also summaries the fit statistics for the Activity Engagement subscale items. Items 3, 5 and 10 demonstrated underfit to the model with each demonstrating excessive infit and outfit. Item 6 demonstrates overfit to the model demonstrating excessive infit and outfit. 11 persons (7%) demonstrated excessive outfit. These persons were removed from reanalyses of the scale.

Visual inspection of the PCA correlation matrix suggested the misfitting items 19 (“It’s a relief to realize that patients don’t have to change pain to get on with their life”) and 10 (“Controlling pain is less important than other goals in patients’ lives”) may represent a different sub-dimension than activity engagement and an eigenvalue of 2.0 warranted further consideration [23]. No clear relationship was evident between these items however. Assessment of local dependence demonstrated a relationship (r = 0.3) between Items 1 (“Patients can get on with the business of living no matter what their level of pain is”) and 2 (“Patients’ lives can go well, even though they have chronic pain”).

A PSI of .90 suggested the Activity Engagement subscale is reliable and suitable for individual use [22]. Analysis of item functioning (DIF) showed more experienced clinicians found the misfitting Item 3 (“It’s OK for patients to experience pain”) significantly harder (0.57 logits, p = 0.004) to endorse than less experienced clinicians. Clinicians with higher HC-PAIRS scores found it significantly easier (0.51 logits, p = 0.002) to endorse Item 1 (“Patients can get on with the business of living no matter what their level of pain is”) and significantly harder (0.54 logits, p = 0.007) to endorse Item 3.

Reanalysis of the Activity Engagement subscale was conducted with poorly functioning Items 3, 5, 10 and 19 excluded (Appendix 2). Hierarchical order of the average item measures remained unchanged and item (0.97) and person (0.87) reliability remained high. As expected, the average person agreeability (1.94 (1.93) logits; range = −3.25 to 6.66 logits) remained comparatively higher than the default (0 (.71) logits; range = −0.80 to 1.42 logits) with the hard to endorse Item 5 removed. No items demonstrated excessive misfit or bias.

Pain willingness subscale

Analysis of targeting, shown in Figure 3, suggested the items targeted the sample well with an average person agreeability of 0.18 (0.61) logits (range = −1.47 to 3.49 logits) comparative with the default average item endorsability of 0 (0.92) logits (range = −0.92 to 1.77 logits). Table 3 displays the items in hierarchical order. Clinicians found Item 16 (“Patients will have better control over life if they can control negative thoughts about pain”) the easiest item to endorse and Item 7 (“Patients need to concentrate on getting rid of their pain”) the hardest. The hierarchical order of the items provided evidence of construct validity with items relating to the need to address thoughts and feelings (items 11, 16) more readily endorsed than items suggesting pain control be prioritised (items 4, 13, 14) or pain be the primary focus (items 7, 17, 18).

Figure 3:

Item-person threshold map for the Pain Willingness Subscale of the CPAQ-C. Person measures represent 162 clinician responses and items represent the average measure thresholds of the 9 items. Note, as these items are reverse scored, more agreeable persons and harder to endorse items are located to the left side of the logit scale (i. e., <0 logits) and less agreeable persons and easily endorsed items are located to the right of the logit scale (i. e., >0 logits). Overall average item endorsability is set at 0 logits by default.

Visual inspection of the category ordering showed no disordering of thresholds but Category 4 (Often true) appeared to be underutilised.

The pain willingness subscale items demonstrated good fit to the Rasch model. Item 16 (“Patients will have better control over life if they can control negative thoughts about pain”), the hardest to endorse item, demonstrated slight underfit to the model, in regards to infit, and no items demonstrated overfit. 15 persons (9%) demonstrated excessive outfit. These persons were removed in reanalyses of the scale.

Visual inspection of the PCA correlation matrix suggested Items 16 and 11 (“Patients’ thoughts and feelings about pain must change before they can take important steps in their life”) could together constitute a second dimension and an eigenvalue of 1.8 provided some support for further consideration. Assessment of local dependence demonstrated a relationship (r = 0.3) between the same two items, 11 and 16, suggesting the possible redundancy of one.

A PSI of 0.73 suggested the pain willingness subscale has reliability suited only to group use (Tennant and Conaghan 2007). No significant differences in item functioning (DIF) were observed for either clinician experience or HC-PAIRS score.

To assess the impact of the poorly functioning Item 16, reanalysis of the Pain Willingness subscale was conducted with it excluded (Appendix 3). Hierarchical order of the average item measures remained largely unchanged although Items 17 and 18 switched order. Item reliability (0.99) remained high and person reliability (0.74) remained stable. Average person agreeability (0.53 (0.74) logits; range = −1.57 to 4.01 logits) increased marginally in comparison to the default (0 (0.95) logits; range = −0.87 to 2.22 logits) with the previously easiest item to endorse excluded. No items demonstrated excessive misfit or bias.

Phase two: construct validity and test-retest reliability of the Chronic Pain Acceptance Questionnaire for Clinicians

Following Rasch analysis questions 3, 5, 10, 16 and 19 were removed from the CPAQ-C. Further analysis was undertaken using the subsequent 15-item version of the CPAQ-C (see Appendix 1).

Construct validity: the full cohort (n = 162) participating in Phase One also completed the Healthcare Providers Pain and Impairment Relationship Scale (HC-PAIRS) [24] to assist with validation of the CPAQ-C. The HC-PAIRS is a valid and reliable questionnaire containing 15 items examining the extent to which healthcare practitioners’ beliefs may justify functional limitations in people with chronic low back pain. Clinicians rate their level of agreement with statements such as, “An increase in pain is an indication that a chronic back pain patient should stop what he is doing until the pain decreases,” on a scale from, “1 – Completely disagree,” to, “7 – Completely agree.” Total scores range from 15 to 105 points. Clinicians indicating higher scores appear to believe that the perception of chronic low back pain justifies greater levels of functional impairment and disability.

Test–retest reliability: for 90% power to detect an ICC of 0.6, alpha set at 0.05, requires the sample size n = 20 [25]. A subset of 24 participants also completed the CPAQ-C online a second time approximately one week later to facilitate examination of the questionnaires test re-test reliability.

Statistical analysis

Descriptive statistics were calculated for scores on the CPAQ-C and HC-PAIRS (n = 162) and Pearson’s correlation co-efficient was used to further examine the questionnaire’s validity. Descriptive statistics were calculated for the CPAQ-C scores at two timepoints for the participants (n = 24) included in examining test re-test reliability. Reliability was examined using the intraclass correlation co-efficient (ICC) (2,1) and standard error of measurement. For 90% power to detect an ICC of 0.6, alpha set at 0.05, requires the sample size n = 20 [25]. Analysis was undertaken using Stata 13.1 (Statacorp, Texas, USA).

Phase three: examination of responsiveness of the Chronic Pain Acceptance Questionnaire for Clinicians and associated constructs

Responsiveness of the CPAQ-C, following an intervention likely to influence CPAQ-C scores, was examined as follows. Nineteen clinicians (one General Practitioner, five nurses, five physiotherapists, eight clinical psychologists/psychotherapists) attending a two-day training workshop on Acceptance and Commitment Therapy (ACT), led by one of the authors (RB), were asked to give written consent and complete paper copies of the CPAQ-C before and immediately after the workshop. At six-month follow-up participants were mailed/e-mailed a further copy of the questionnaire. A reminder was sent two-weeks later in cases where participants had not yet responded.

The workshop involved an introduction to the theory and practice of ACT. It covered the philosophical and theoretical background to the model, with an emphasis on the practical application of ACT interventions. Of note was the experiential nature of the learning, with participants invited to engage in a number of exercises related to the model’s aim of increasing psychological flexibility. Learning via ‘doing’ rather than ‘talking about’ ACT is a key feature of much ACT training and it is common for learners to report personal benefit from training, in addition to the acquisition of knowledge and skills related to their professional roles [26].

Responsiveness of the CPAQ-C was also considered by examining whether CPAQ-C scores were influenced by changes in potentially associated constructs relating to clinicians’ beliefs about pain It was hypothesised that whether clinicians adopted a biomedical or a biopsychosocial belief system, and whether they had contemporary background pain neurophysiology knowledge, may influence clinicians’ beliefs regarding the importance of levels of acceptance in patients with chronic pain. Participants were therefore asked to complete two further questionnaires on the aforementioned three occasions to examine whether these beliefs or knowledge were associated with their beliefs regarding the importance of levels of acceptance in patients with chronic pain . The Pain Attitudes and Beliefs Scale [27] is a valid and reliable questionnaire examining biomedical and biopsychosocial beliefs of clinicians. It contains 31 statements about beliefs regarding management approaches to chronic low back pain such as, “Good posture prevents back pain,” or, “The way patients view their pain influences the progress of the symptoms.” Participants rate their level of agreement with each statement from, “1 – totally disagree,” to, “6 – totally agree.” Biomedical beliefs are examined by statements 4, 5, 9, 10, 13, 14, 20, 22–26, 30 and 31 giving a range of 14–84 points. Biopsychosocial beliefs are examined by statements 3, 6, 7, 11, 12, and 27 giving a range of 6–36 points. Higher scores represent greater levels of biomedical or biopsychosocial beliefs.

Participants also completed the Revised Neurophysiology of Pain Questionnaire [28], a valid and reliable questionnaire measuring knowledge of pain neurophysiology. It contains 12 statements such as, “Descending neurons are always inhibitory,” which participants mark as being true, false or undecided. Based upon whether their answers are correct or not participants can score from 0 to 12 points with higher scores reflecting greater pain neurophysiology knowledge.

Statistical analysis

For a prospective sample size calculation an assumption of a normal sample distribution was made. For a one-way repeated-measures ANOVA to compare sample mean CPAQ-C scores at three time points (pre-workshop, immediately post-workshop, 6-months follow-up) with 90% power to detect an effect size of 0.4, alpha set at 0.05 required the sample size n = 15 (Calculated in G*Power 3.1.9.2). However, following data collection, to allow for missing and skewed data the Skillings Mack test was used instead. The Skillings Mack test or one-way repeated measures ANOVA were also used to examine whether there were any significant changes in participants scores for neurophysiology of pain knowledge, biomedical and biopsychosocial beliefs as appropriate.