Psychometric evaluation of the Danish version of a modified Revised American Pain Society Patient Outcome Questionnaire (APS-POQ-R-D) for patients hospitalized with acute abdominal pain

2.1 Design

A descriptive longitudinal study of the psychometric properties of a modified Danish version of the APS-POQ-R was conducted. The study was part of a pre- and postintervention study of patient-controlled oral analgesia for patients with acute abdominal pain in which we investigated patients’ experience of pain management during their hospital stay (questionnaire one) and after discharge (questionnaire two).

2.2 Setting

The study was performed in an ED observation unit and a general surgical ward in a university hospital in Southern Denmark with a background population for primary referral of approximately 430,000 inhabitants. In Denmark, nearly all nontrauma emergency patients are seen before hospitalization by a general practitioner and are admitted as indicated to the ED before being transferred to an ED observation unit or hospital ward.

2.3 Sample

The number of patients to be included in the intervention study was based on a power calculation using the results from a previous study [30] showing the patient satisfaction of pain management to be 40%. An increase in patient satisfaction to 65% in the intervention group was considered clinically relevant. To achieve a significance level of ≤0.05, a power of 80%, and an expected 20% dropout, a total of 70 participants in each group was required.

A total of 234 patients were approached from December 2015 to October 2016 by the nurse researcher or nurses from the project team when available. The project nurses did not perform care for approached patients. Inclusion criteria were patients admitted with acute abdominal pain and admitted to the ED from the primary health care service, discharged from the ED observation unit or the surgical ward, at least 18 years of age, able to speak Danish, and with a hospital stay longer than 8 h. Patients also had to be considered compliant with the study intervention (patient-controlled oral analgesia) and were evaluated by assessment of their cognitive, physical, and psychological abilities. Assessment of the patients was performed in collaboration between the research nurse/project nurse and the nurse in charge of the patients’ care. Exclusion criteria were end-of-life patients; patients with known chronic pancreatitis, cancer, or inflammatory bowel disease; and those staying in the ICU. Patients with formation of a stoma were excluded, as we prioritized their attention toward training in stoma care.

Of the 234 patients, 45 patients declined (19%), six did not fulfill all inclusion criteria (3%), and four were transferred to another unit (2%). The dropout in questionnaire one was 23 patients (13%) and 62 patients (35%) in questionnaire two. In total, 156 patients (67%) from the control and intervention groups were included in the psychometric tests of questionnaire one and 117 patients (50%) from both groups in questionnaire two.

2.4 The Instrument APS-POQ-R

The APS-POQ-R includes 23 items and five subscales of pain severity (pain), perception of care (satisfaction), pain interference with function (activity), pain interference with emotions (emotion), and side effects of treatment (safety), as well as three items on the use of nonpharmacological pain treatment methods [16].

For the pain severity subscale, the patients had to rate their pain on the 11-point Likert Numeric Ranking Scale (NRS) for pain [31] (0=no pain to 10=the worst imaginable pain) for the first two items concerning “least pain” and “worst pain.” The last item is about frequency of severe pain, where 0% is “never in severe pain” and 100% is “always in severe pain.”

On the patient perception of care (satisfaction) subscale, the patients had to rate their answers on an 11-point Likert scale: one item measures the extent of pain relief (0%=no relief to 100%=complete relief), one concerns the usefulness of information about pain treatment options (0=not at all helpful, 10=extremely helpful), one is about patients being allowed to participate in decisions about pain treatment as much as they wanted (0=not at all, 10=very much so), and one is about patients’ satisfaction with the results of the pain treatment (0=extremely dissatisfied, 10=extremely satisfied). Another item concerns information about pain treatment options (yes/no), another concerns the use of nonpharmacological methods to relieve pain (yes/no), and one is about how often a nurse or doctor encouraged the patient to use nonpharmacological methods (never/sometimes/often).

The third to fifth subscales are measured on an 11-point Likert scale: interference with function (activity) (0=does not interfere, 10=completely interferes), affective experiences (emotion; 0=not at all, 10=extremely) and side effects (safety; 0=none to 10=severe).

2.5 Translation procedure

The APS-POQ-R was translated according to international guidelines [32], [33]. First, the APS-POQ-R was translated from English to Danish (APS-POQ-R-D) by three independent nurses experienced in pain management of acute and general surgical patients and by one professional bilingual translator not trained in health care. During translation, inspiration was collected from a Danish translation of the 1995 APS-POQ [34] and a Norwegian [35] and Swedish version [17] of the APS-POQ and APS-POQ-R due to similarities in health care and language within these countries. A synthesis of the translation was made in collaboration between the experienced nurses and the nurse researcher. The translator clarified uncertainties.

Second, the APS-POQ-R-D was retranslated from Danish to American English by a professional American translator who spoke Danish fluently. The retranslated questionnaire was compared with the original American APS-POQ-R by another group of three nurses, the nurse researcher, and a surgeon experienced in pain management of acute and general surgical patients. This group evaluated the face and content validity of the questionnaire to determine how representative the items were in defining the construct of the subscales. Only minor changes were made after the retranslation.

2.6 Modification of the instrument APS-POQ-R-D

The APS-POQ-R-D was modified as part of the intervention study of patient-controlled oral analgesia for patients with acute abdominal pain in which we intended to investigate the patient experience of pain management during the hospital stay (questionnaire one) and after discharge (questionnaire two). For this purpose, we modified the APS-POQ-R-D by creating two slightly different questionnaires by changing the reference period and adding or deleting items.

Questionnaire one (Table 1) concerned the time during hospital stay. The APS-POQ-R uses a 24-h time frame, which we modified to a 4-h time frame for the time of hospital arrival and represented the time patients stayed in the ED. A second time frame was created for the rest of the hospital stay, representing the time patients stayed in the surgical ward or the ED observation unit and the time frame for the study intervention.

The APS-POQ-R was developed for use during the hospital stay, whereas our study also intended to investigate the time period after discharge. Therefore, the two following items were added to the patient perception of care (satisfaction) subscale to capture issues that had been reported as important for patients with acute abdominal pain upon discharge [36]: “Did you receive information about when to seek medical advice after discharge?” and, “If yes, how useful was the information about when to seek medical advice?”

The modification also included a sixth subscale with six items on patient-related barriers, as a qualitative study of patients with acute abdominal pain revealed those as barriers for pain management [1]. We wanted to investigate whether patient involvement through patient-controlled oral analgesia affected these barriers. Five of the six items were from the Danish BQ-ll [21]: “pain medicine can effectively control pain” (fatalism subscale), “pain medicine is very addictive” (tolerance subscale), “if I talk about pain, nurses and doctors will think that I am a complainer” (communication subscale), “it is easier to put up with pain than the side effects of medicine” (side effects and distract MD subscale), and “pain medicine can keep you from knowing what is going on in your body” (monitor subscale). One item was exclusively from the results of the qualitative study of patients with acute abdominal pain [37]: “it is better to remain at rest than to take painkillers.” The items were rated on a 7-point Likert scale (0=do not agree at all, 6=agree very much).

Overall, questionnaire one included 39 items and six subscales. The subscales for pain and perception of care were included in the time frames for hospital arrival (the first 4 h) and the rest of the hospital stay.

Questionnaire two (Table 2) concerned the time spent at home after discharge. It had seven time intervals and maintained a time frame of 24 h, one for each day for a week after discharge. Questionnaire two included 18 items the first 6 days and 25 items on the seventh day.

For each day, four items were added to capture issues reported as important for patients with acute abdominal pain after their discharge from hospital [36]. Two items were added to the patient perception of care (satisfaction) subscale; “Did you receive information about when to seek medical advice after discharge?” and, “If yes, how useful was the information about when to seek medical advice?” Further, two items were added to the interference with functions (activity) subscale: “How much did the pain interfere with social activities?” and, “How much did the pain interfere with leisure activities?” Finally, one item on the use of analgesics was added. On the seventh day, one item on contact with medical service and one on return to work were added.

At all time intervals, three items from the patient perception of care (satisfaction) subscale were deleted because they were irrelevant after discharge: the extent of pain relief, being allowed to participate in decisions about pain treatment as much as desired, and satisfaction with the results of the pain treatment. In addition, one item on nonpharmacological methods was deleted: how often a nurse or doctor encouraged the patient to use nonpharmacological methods. The following items were deleted the first 6 days to reduce the number of items: information about pain treatment options during the hospital stay, information about in what kind of situations to seek medical advice, and use of nonpharmacological interventions.

The modified APS-POQ-R-Ds were pilot tested in November 2015. The questionnaires were distributed to four patients admitted with acute abdominal pain, who completed the questionnaires in the presence of the nurse researcher, who interviewed the patients about their experience of answering the questionnaires. A few adjustments were made, but overall the patients found the questions easy to understand. Subsequently, the questionnaires were pilot tested on 10 patients admitted with acute abdominal pain who were interviewed during their hospital stays and after discharge by phone by the nurse researcher about their experience of the questionnaires, which resulted in minor adjustments.

2.7 Data collection

The patients completed all items in questionnaire one upon discharge (Fig. 1), along with information on demographic data. The patients delivered the questionnaire in sealed envelopes to the staff before discharge. The medical files of the patients were reviewed to obtain information about type of surgery, if provided; discharge diagnosis; length of hospital stay; and any readmissions within 1 month.

Fig. 1:

Place and time for completion of the questionnaires.

The patients completed questionnaire two daily for the previous 24 h for 1 week after discharge (Fig. 1). The questionnaire was handed out to the patients with a stamped return envelope before discharge.

2.8 Statistical analysis

Data from the questionnaires were doubly entered into a database in REDCap (version 7.0.11 – © 2017 Vanderbilt University, TN, USA) along with data for demographics, surgical procedures, length of hospital stay, discharge diagnosis, and readmissions within 30 days.

All data were transferred to STATA (Version 15.0; StataCorp, TX, USA). Data were analyzed for three time frames: arrival, hospital stay, and home. In the present study, “home” was operationalized as the seventh day after discharge, as it included all subscales and items in questionnaire two and tested the psychometric parameters at the longest time point after discharge. Descriptive statistics were generated on demographics, surgical characteristics, and categorical items. Continuous variables were reported as medians and interquartile ranges (IQRs) and categorical variables as count and percentages. Categorical variables were compared by the χ² test or Fisher’s exact test if the number were lower than five. For each subscale and each item of the subscales with continuous values, minimum and maximum scores, means, and standard deviations (SDs) were calculated. The items “pain relief” and “time spent in severe pain” were converted from 0–100 to 0–10 scales during the analysis to match the other items. Analysis of the subscales was performed by summing the data of each item within the subscale, and the number of items within the subscale was used as the denominator.

For the internal consistency reliability of the subscales, we calculated the Cronbach’s alpha (the τ-equivalent reliability variable) for each item and subscale in addition to the different time frames (arrival, hospital stay, and at home). The Cronbach’s test reveals whether the test accurately measures the variable of interest. A value <0.7 was considered acceptable [38].

To assess whether the subscales measured the desired subjects through the items, we examined the construct validity. For this purpose, an exploratory factor analysis was performed to measure how all items in the questionnaire were clustered in factors. The factor analysis was conducted at the three time frames, and only factors with eigenvalues >1 were included, and we did not rotate the factors [39]. We carried out the Bartlett’s test for sphericity and calculated the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) to investigate the validity of the factor analysis.

2.9 Ethical considerations

At discharge, patients were invited to take part in the study at a time convenient to them. They were given time to consider the invitation, and some took advantage of this opportunity. Patients received oral and written information about the aim of the study. They were informed that they could withdraw from the study at any time without consequences for their care and treatment, and that their statements would be anonymized. The patients gave written consent. The Danish Data Protection Agency (ID: 2008-58-0035) and the Regional Scientific Ethical Committees for Southern Denmark (ID: S-20140160) approved the study.

3.1 Sample and dropout characteristics

The demographics and clinical characteristics of the sample and dropouts are shown in Table 3. The median age in the sample was 53 years (IQR: 38–66), 47 years (IQR: 38–66) for the dropouts in questionnaire one (p=0.66) and 45 years (IQR: 34–59) for the dropouts in questionnaire two (p=0.01). The median length of hospital stay was 77 h (IQR: 48–114) in the sample, 94 h (IQR: 48–141) in dropouts in questionnaire one (p=0.58) and 77 (IQR: 48–113) in questionnaire two (p=0.51).

3.2 Descriptive statistics

For the perception of care subscale (satisfaction), information about pain treatment options provided by the hospital was reported by 62% of patients at hospital arrival, by 71% during hospital stay, and by 63% at home. Information about situations in which to contact medical service after discharge was reported by 81% of patients upon discharge and by 76% of patients at home.

Nonpharmacological pain treatment methods were used by 65% of patients during their hospital stays and by 48% of patients at home. During their hospital stays, 75% of patients reported they were never encouraged to use nonpharmacological methods, 20% reported “sometimes”, and 5% of patients reported “often”. Of those using nonpharmacological pain treatment, the following were reported during the hospital stay (questionnaire one) and at home (questionnaire two): deep breathing 28%/14%, walking 20%/20%, distraction 16%/17%, relaxation 15%/17%, heat 9%/11%, music 8%/8%, imagery or visualization 5%/4%, meditation 4%/4%, cold back 3%/2%, prayer 3%/3%, massage 2%/6%, and other methods 1%/4% for pain relief.

The distribution of minimum and maximum scores, means, and SDs of the rest of the items and subscales are shown in Tables 1 and 2. The minimum and maximum scores on the Likert scale were used in 90% of the items from the first five subscales in questionnaire one. For questionnaire two, the maximum scores were used in 42% of the items.

3.3 Reliability and item correlation

The results of correlations and tests for internal consistency reliability showed a Cronbach’s alpha of >0.7 for the pain, satisfaction, activity, and emotion subscales, but the value was 0.69 for the satisfaction subscale at “hospital stay.” For the safety subscale, Cronbach’s alpha was <0.7, but would be 0.73 if the item “itching” were deleted at the time interval “arrival.” For the patient-barrier subscale, Cronbach’s alpha was ≤0.62 for any combination of the items in the subscale (Table 4).

3.4 Construct validity and factor analysis

Table 5 shows the results of the factor analysis (i.e. how high the items loaded on a factor). The bold numbers illustrate the highest loading on a factor for an item. At “arrival” (questionnaire one), two factors had eigenvalues of >1 each explaining approximately 20% of the variation in the scale. All three items from the pain subscale loaded on factor one, and all four items from the satisfaction subscale loaded on factor two, which confirmed the constructs of the subscales. This analysis resulted in an acceptable KMO of 0.631 and a highly significant Bartlett test (<0.001) indicating acceptable validity of the factor model.

For “hospital stay” (questionnaire one), a five-factor solution occurred, the first factor explaining about 19% of the variation while the other factors explained 5–10% each. The items from the pain, activity, emotion, and safety subscales, except for the items “least pain” and “itching,” loaded on factor one. The results suggest one construct for the four subscales concerning pain and pain interference with physical and psychological functions. In addition, two items from the BQ-ll loaded on one factor and could be named “enduring pain.” Three of five items from the satisfaction subscale loaded on factor two, including the item we created for our study: “How useful was information about medical service after discharge?” This indicated a contribution of the item to the construct of the satisfaction subscale. This analysis resulted in an acceptable KMO of 0.668 and a highly significant Bartlett test (<0.001) indicating acceptable validity of the factor model.

For “at home” (questionnaire two), a three-factor solution emerged, the first factor explaining 43% of the variation while the other two explained 6–7% each. All items from the pain, activity, and emotion subscales, including the two items created for our questionnaire concerning pain interference with social and leisure activities, loaded on factor one. This validates the contribution of the two items to the construct of the subscale “activity.” The loading of three subscales in one factor suggests one construct concerning pain and pain interference with physical and psychological functions. Factor three loaded the items from the safety subscale, except for the item “itching.” This analysis resulted in an acceptable KMO of 0.808 and a highly significant Bartlett test (<0.001) indicating acceptable validity of the factor model.

4.1 Strengths and limitations

The strength of this study is its investigation of patient experiences of pain management during hospital admission and after discharge. It demonstrates that a modified APS-POQ-R can be used for longitudinal studies.

The limitations of the study are the low sample size and the significant differences in the sample and dropouts. Fewer patients in the sample underwent surgery and open surgery and might reflect that patients in the dropouts were more affected by the acute situation than were patients in the sample. This might have influenced the distribution of the minimum and maximum score on the Likert scales in the questionnaires. We were not authorized to collect data on the reasons for dropout.

Another limitation of the study is that the results from questionnaire one reflect patient experiences from approximately 18–141 h; consequently, the results might not be comparable with studies with shorter time frames of patient experiences.