Validity and Utility of Four Pain Intensity Measures for Use in International Research

Plain Language Summary

In order to increase our understanding of pain and its treatment, we must be able to measure it. Although a great deal is known about the strengths and weaknesses of the most commonly used measures of pain intensity, most of our understanding comes from research conducted in Western and developed countries. In order to understand which pain intensity measures are most useful among people everywhere, research on pain measures must be conducted in people with pain from non-Western and developing countries. This study found that 0–10 Numerical Rating Scale has the most utility in our sample of Thai individuals with chronic pain. However, when considered in light of the findings from other countries, the results of this study suggest that the FPS-R may have the most utility for use in cross-cultural and international research.

Introduction

Pain relief and treatment for pain-related conditions are arguably the most common reasons people have for seeking health care.^1,2 Effective pain management remains a critical health concern worldwide.³ To be able to evaluate the efficacy of pain treatments in research – including in multisite research conducted in different countries – pain must be assessed using measures with established reliability and validity.

Pain intensity is the pain domain most often assessed in clinical and research settings.^4,5 Knowledge regarding the psychometric properties of different pain intensity measures across different patient populations is therefore critical for advancing the treatment and scientific understanding of pain. The most commonly used measures of pain intensity are the Visual Analogue Scale (VAS), Verbal Rating Scale (VRS), Numerical Rating Scale (NRS), and Faces Pain Scale-Revised (FPS-R).^6–8 Each of these scales has evidence supporting their test-retest reliability, and ratings of these scales tend to correlate strongly with each other, supporting their validity.^9,10

However, each of these scales also has demonstrated strengths and weaknesses in different populations.^6,11 Although the VAS has the potential to be more sensitive to small changes in pain,^4,7,12 this scale has also been found to be more challenging to comprehend and use by elderly individuals.^7,12 The NRS and VRS have been found to be easier to administer than the VAS, and to have lower incorrect response rates.¹³ However, a recent study in a sample of patients with chronic pain from Nepal reported higher incorrect response rates for the NRS than the VRS, VAS, or FPS-R. This finding was attributed to limited education of the study participants, as well as the fact that numbers are rarely used to describe magnitude in Nepalese culture.⁸ Thus, preliminary findings suggest that both demographic and cultural factors appear to impact the validity and utility of pain measures; no single measure has been found to be universally superior across patient populations.

A significant limitation of the research that has been published to date on this topic is that most of it has been conducted in western populations.^6,14–18 Based on the findings from this research, consensus groups have identified the NRS – specifically, the 11-point 0–10 NRS (also referred to as the NRS-11) – as the best measure of pain intensity in most situations and with most populations.⁴ However, a systematic review by Hjermstad and colleagues concluded that the VRS was preferred over the NRS by those with less education and the elderly, while people with more education and younger individuals preferred the NRS.¹⁵ In addition, although studies conducted in China and Nepal have demonstrated good reliability and validity for all four scales among those who are able to use these scales without errors, the most preferred measure of pain intensity in individuals from these countries has been shown to be the FPS-R.^7,8

Thus, in large part because of the lack of research on pain measures in non-Western countries, it is not yet possible to draw strong conclusions regarding which measure or measures should be used when conducting cross-country or cross-cultural research. Additional research from a variety of countries is needed to help us understand which measure(s) are most universally valid and useful, especially when the goal is to compare research findings across different countries. One group of patients that has not yet been studied in this context is patients from Thailand.

Given these considerations, the aims of the current study were to (1) evaluate the validity and utility of four commonly used measures of pain intensity in a sample of patients with chronic pain from Thailand and (2) compare findings in the current sample with those from other countries, to help identify the measure or measures which might be most appropriate for cross-country research. Based on the available evidence, cited above, we hypothesized that the findings would support the validity of all four rating scales as measures of pain intensity among those who are able to use all measures in the study sample. With respect to scale utility, and given previous research findings in other samples of individuals with pain, we hypothesized that older participants and those with less education would evidence higher rates of incorrect responding than younger participants and those with more education in our sample. We also hypothesized greater preferences for the categorical measures (eg, FPS-R and VRS) than the VAS.

Materials and Methods

Subjects and Procedures

From August to November 2018, 360 patients with chronic pain seen at the outpatient pain clinic at the Siriraj hospital in Bangkok, Thailand, were enrolled in this cross-sectional study. Potential participants were notified about the study via a written pamphlet that was posted on the notice board in the pain clinic. In order to participate in the study, the potential participants had to be aged 18 years or older, report having experienced pain in at least one part of their body for more than three months, and be able to understand and speak Thai. Patients with pain due to cancer and those with a history of cognitive impairment, visual impairment, or significant psychiatric illness per chart review were excluded.

Once participants were enrolled, they were administered a questionnaire asking them to provide demographic data (ie, gender, age, marital status, employment status, place of residence [ie, metropolitan or provincial areas] and educational level). The participants were then asked to rate four domains of pain intensity (worst pain, average pain, and least pain in the past week, and current pain) using four different pain intensity rating scales; the VAS, VRS-6, NRS-11, and FPS-R. As noted in the Introduction section, all four of these scales have been shown to be reliable and valid in previous research, and are commonly used in clinical and research settings.^{5,6,10,12,13,15} The participants were provided verbal instructions on how to use each scale, and the instructions were repeated up to three times if requested by the participants. To control for possible order effects, the four scales were administered randomly using a Latin square design, such that each type of scale was presented in each possible order (first, second, third, or fourth) on a separate sheet. Participants were asked to rate their worst pain, average pain, and least pain in the past week, and current pain, using each scale before moving on to the next.

The participants were then asked to select the scale that they would prefer to use when asked to rate their pain intensity in the future. While the participants rated their pain intensity using each scale, the research staff did not provide any specific guidance or corrections, even if the participant was observed to make an error (eg, rating least pain as being more intense than worst pain). However, incorrect responses were later coded into one of seven categories: (1) providing two or more responses on the same rating scale; (2) providing a range of responses rather than a single response; (3) being unable to provide a response, even after three explanations; (4) marking between two descriptive terms on the VRS-6 or between two faces on the FPS-R; (5) rating worst pain as lower in intensity than average pain; (6) rating worst pain as lower in intensity than the least pain; or (7) rating the least pain as greater than average pain. This study was approved by the Siriraj Institutional Review Board, Faculty of medicine Siriraj hospital, Mahidol University, Bangkok, Thailand (SI429/2018). All participants signed informed consent forms before any data were collected. This study was conducted in accordance with the Declaration of Helsinki.

Measures

Visual Analogue Scale (VAS)

The VAS is a continuous scale, usually presented as a 10-centimeter horizontal line, anchored by 2 verbal descriptors reflecting no pain and extreme pain. In this study, the extreme descriptor used was “Worst pain.”^9,19 This endpoint (which was also used as the endpoint for the NRS-11 in this study) has a literal translation of “most” or “worst” or “maximum” pain, and was translated from that used for the pain intensity ratings from the Brief Pain Inventory (ie, “Pain as bad as it could be”). It has also been deemed to be the cultural equivalent to the Brief Pain Inventory endpoint.²⁰ With the VAS, respondents are asked to make a mark on the line that represents their pain intensity. The distance, in centimeters (ie, 0.0 to 10.0) from the “No pain” end represents the respondent’s pain intensity score.

6-Point Verbal Rating Scale (VRS-6)

The 6-level VRS is a widely used measure of pain intensity that has been validated in 15 languages.^21,22 The six English descriptors used to represent each level of pain intensity are “None,” “Very mild,” “Mild,” “Moderate,” “Severe,” and “Very severe.” For the purpose of this study, we translated these descriptors into Thai. We then linked a specific number (ie, 0, 1, 2, 3, 4, and 5) to each descriptor. The number associated with the descriptor chosen by the respondent was their VRS-6 score.

Numerical Rating Scale-11 (NRS-11)

The 0–10 NRS (also referred to as the NRS-11) is one of the most widely used measures of pain intensity in research and clinical practice.²³ With the NRS-11, respondents are asked to choose the single number that best represents their pain intensity, on a 0 (“No Pain”) to 10 (“Worst pain”) scale.

Faces Pain Scale-Revised (FPS-R)

The original Faces Pain Scale (FPS), developed by Bieri and colleagues,²⁴ was developed to assess pain intensity in children, who were thought to have difficulty using the most commonly used pain intensity measures such as the VAS or NRSs. In 2001, the FPS was revised to make it more suitable for use with the metric scoring 0–10 scale (FPS-R), using visual depictions of faces to represent increasing levels of pain intensity along a 6-face continuum. With the FPS-R, respondents are asked to select the face that best represents their pain intensity. Each face is associated with a number (ie, 0, 2, 4, 6, 8 and 10),²⁵ and the respondent’s pain intensity score is the number associated with the face that was chosen.

Statistical Analysis

We first computed descriptive statistics for the demographic variables, pain history variables, and pain ratings (means and standard deviations for continuous variables, and number and percent for categorical variables) to describe the sample. Next, to evaluate the convergent validity of the scales (ie, the association between each scale and a composite score of all measures), we conducted a principal components analysis.^6,8,12,26 We determined the number of factors that emerged from the factor analysis using a scree test, and examined the loadings of the factor(s) that emerged as an indication of scale validity. We then computed the incorrect responses and preference rates for each measure, and evaluated the associations between each of these and age (dichotomized as younger [≤60 years old] versus older [>60 years old]) and education level (dichotomized as lower education [≤12 years of education or junior high school] and higher education [>12 years of education]), using chi-square analyses. Data analyses were performed using PASW Statistics version 16 (SPSS, Inc., Chicago, IL, USA).

Results

Demographic Data and Pain Characteristics

A summary of the demographic information of the sample is presented in Table 1. Sixty-six percent (N = 293) of the sample were female. Mean age was 53 years (range, 36 to 69 years). A plurality of the participants had neuropathic pain (43%, N=156). The second most common pain problem was muscle/tendon pain (27%, N= 92). Nearly half of participants (49%, N=175) had more pain with movement and 62% (N=223) reported that nothing relieved them from pain. The means and standard deviations of the pain rating scores for each domain of pain intensity (worst, least, average and current pain) are presented in Table 2.

Table 1 Descriptive Information About the Study Participants (N = 360)

Table 2 Means and Standard Deviations of the Pain Ratings (N=360)

Scale Inter-Correlations and Scale Validity

Very strong correlations were found among the four scales for each pain intensity domain (r’s = 0.72 to 0.91, see Table 3). The results of the principal components analyses of the four pain scales provided very strong support that they assessed the same underlying domain, given the very high first eigenvalue (greater than 3.00 in every case) and very low second eigenvalue (see Table 4). In addition, all of the scales evidenced a very strong loading (>0.9) on the single component that emerged. The NRS-11 had the highest loading on the components representing worst (0.94), average (0.94), and current pain (0.97). The VAS had the second highest loading.

Table 3 Inter-Scale Correlation Coefficients

Table 4 Component Loadings from the Principal Components Analyses of the Four Rating Scales

Incorrect Responses

The rates of incorrect responding are presented in Table 5 by type of incorrect response. Two categories of incorrect responses (ie, providing a response that was between two response categories and providing a range of responses instead of a single response) were not made by any participant. As can be seen, the VAS had the highest rate of incorrect responding (N=75; 45%). The total number of incorrect responses to the NRS-11, FPS-R and VRS-6 were all less than 40, with the NRS-11 evidencing the lowest rate of incorrect responding (N=25; 15%). The most common error across all of the scales was rating average pain as higher than worst pain (50%), and the second most common error was rating least pain as being higher than average pain (25%).

Table 5 Rates of Incorrect Responding for the Rating Scales

Factors Associated with Incorrect Responding

Older respondents and those with lower education levels had the highest rates of incorrect responding (see Table 6); the effect of age was statistically significant for the VRS-6 (P = 0.044), and the effect of education was statistically significant for the VAS (P = 0.001).

Table 6 Associations Between Age and Educational Group and Incorrect Response Rates

Scale Preferences

Of the 360 participants, 186 (52%) preferred NRS-11 followed by FPS-R (20%), VRS-6 (19%) and VAS (9%), respectively (see Figure 1 and Table 7). The results regarding the associations between age and educational level and scale preference are presented in Table 7. As can be seen, significant effects were observed for education level; although those with higher and lower education both preferred the NRS-11 over the other scales, this preference rate was greater for those with higher education (56%) than lower education (44%). This difference may have been due to the marked difference in preference rates between these groups for the FPS-R (ie, 14% and 30% for those with higher versus lower education, respectively). This difference in pain scale preference as a function of education level was statistically significant (P = 0.003). No statistically significant differences in preference rates were noted in terms of age (P = 0.523).

Table 7 Associations Between Age and Educational Group and Scale Preference

Figure 1 Number of participants who prefer each pain scale (N=360). Abbreviations: FPS-R, Face Pain Scale-Revised; NRS, Numerical Rating Scale; VAS, Visual Analogue Scale; VRS, Verbal Rating Scale.

Discussion

The key findings from this study were that (1) although all four rating scales are valid measures of pain intensity among those individuals who are able to use them, (2) significant differences in incorrect response rates and preferences emerged for the different rating scales, (3) differences in rates of incorrect responding differed as a function of both age and education level, and (4) differences in preference rates differed as a function of education level. These findings have important implications for determining which measure(s) might be most useful when measuring pain intensity both locally (ie, in this case, in Thailand) and internationally.

Scale Validity

The findings provide strong support for the validity of all four rating scales, at least among the individuals who are able to use each measure. This finding is consistent with those from some other studies that have evaluated the validity of these scales in different populations living in different countries (eg, the USA, China, Nepal, Nigeria, Portugal and Spain).^{7,8,14–18,27} Thus, if clinicians and researchers have the ability to ensure that their patients or research study participants are able to use the measure chosen, each one of the measures could potentially provide valid data regarding the magnitude of felt pain. However, not every clinician or investigator has the resources to evaluate a patient’s ability to use pain scales. In such cases, it would make sense to choose the rating scales that tend to have the lowest incorrect response rates and are most preferred by the population being treated or studied.

Incorrect Response Rates

In a sample of individuals with chronic pain from Nepal, the NRS-11 evidenced the highest rate of incorrect responding, followed by the VAS.⁸ A similar finding emerged in studies conducted in China and Canada.^7,28 Although Thailand is a developing non-Western country with relatively low literacy rates, especially among individuals living in rural areas, the findings from the current study showing the largest incorrect response rates for the VAS and least incorrect response rates for the NRS-11 are similar to those from more developed countries.^12,26,28,29 It is possible that the relatively high education level in the current sample, which is higher than that of the Thai population in general, might explain the similarity of findings to those from developed countries.^{12,26,28–30}

Previous research indicates that when significant effects are found, older individuals, individuals with cognitive impairments, and individuals with lower education levels have higher rates of incorrect responding to pain measures.^8,12,29 This finding appears to more consistent for the VAS than for categorical (ie, FPS-R and VRS) and numerical scales.^28,31 These findings were generally replicated in our sample, with older participants and participants with lower education levels evidencing higher rates of incorrect responding to the VAS, although these differences were only statistically significant for education level. Also, in our sample, a significant effect for education level on the incorrect response rates to the VRS-6 emerged. Overall, these findings support the choice of the NRS-11 or the FPS-R for use in Thai samples who are treated or recruited for research from a large metropolitan city (ie, Bangkok).

Scale Preferences

In this study, NRS-11 was the most preferred scale, followed by the FPS-R. Also, only educational level was significantly associated with scale preference. Our finding of a significant association between education level and preference for the NRS-11 (with those having higher education preferring the NRS-11 more than other scales) has also been found by researchers in other (but mostly developed) countries.^{12,26,28,31–34} However, in general, research among individuals from developing countries or those with populations with lower education levels show that these individuals tend to prefer categorical scales such as the FPS-R,^7,8,33–37 even though this scale was originally developed for use in children. One generally consistent finding across all populations is the low preference rate for the VAS.^8,12,29

No significant differences were found in scale preference in relation to age in our sample. Although previous studies found that the older patients (65 years) and the oldest patients age groups (more than 75 years) prefer VRS-6 to all other scales,^12,34,38 older participants in this study preferred NRS-11 over the other scales across age groups. The reason for this discrepancy in findings is not entirely clear. It is possible that it might be related to one or more differences in the samples used in different studies (eg, effects of country of origin, effects of education levels). Additional research using samples of individuals with pain from Thailand and other countries is needed to determine which factors are most consistently associated with scale preferences.

Implications for Between-Country/International Research

As researchers continue to seek to identify the factors that influence pain and its impact, an important and as yet understudied area is the extent to which the findings from one country and culture do (or do not) generalize to other countries and cultures.^39,40 Much of the pain research that has been and is being conducted has come from studies performed in western countries. Determining the extent to which research findings are “universal” and generalize to individuals from other countries and cultures is very important. Findings regarding the psychometric properties of pain measures that replicate across countries support the generalization of these findings across languages and cultures, and therefore provide evidence that can be used for selecting measures when cross-cultural or cross-country comparisons are a research goal. On the other hand, findings that are unique to specific countries might be less useful for selecting measures in cross-cultural research. Critically, in order to evaluate the cross-country relevance of findings, reliable and valid measures of key pain-related constructs that can be used in populations from different countries are needed. Arguably, measures of pain intensity are among the most important of these.

Based on the current findings, and in light of previous research, one preliminary conclusion that might be drawn is that the FPS-R appears to be the most viable choice for use in cross-country research. Like all of the other measures evaluated in this study, the FPS-R has been shown to be valid in every study that has evaluated this measurement property.^6,25,41,42 In addition, the incorrect response rates of the FPS-R are consistently low across all populations in which these rates have been evaluated.^7,37 Critically, this cannot be said of every measure, including the NRS-11,⁸ perhaps, in part, because of the negative impact that pain can have on an individual’s “number sense.”^43–45 Although the FPS-R is not the measure that tends to be preferred over the others,^15,26,29 the lack of preference for the FPS-R has not translated to a lack of validity or a lack of ability to use the measure. Arguably, validity and ability to use a measure without errors are more important than preference when selecting a measure to use in clinical or research settings. Finally, the FPS-R has already been translated into over sixty-nine languages.⁴⁶ Given these considerations, our current recommendation is for researchers is to consider either (1) using the FPS-R as their primary measure of pain intensity if that measure is deemed appropriate for helping them address their study question and hypotheses, or (2) adding the FPS-R as a secondary outcome measure in their research, in order to facilitate a standardization that would allow for between-country and between-study comparisons of study findings.

Limitations

This study has several limitations that should be considered when interpreting the results. First, it should be kept in mind that although the majority of the Thai population has finished only junior high school (12 years of education),³⁰ most of the participants in the current study had completed a bachelor's degree or had an even higher education level. Thus, it is possible that the findings from this study may not generalize to some subgroups of the Thai population, perhaps especially those living in rural areas where education is more limited. Replication of the study findings in samples of individuals who live in rural areas of Thailand would be helpful to understand the generalizability of the study findings. Second, we did not assess cognitive function in the sample, including in the group that was older. It is possible that a subgroup of the study sample may have an undiagnosed cognitive impairment. The role that this factor plays in the study findings could therefore not be determined. Future researchers should include measures of cognitive function, if possible, to determine the extent to which factor might explain the age effects so often observed on incorrect response and preference rates. Third, we did not assess the participants’ previous experience with and knowledge about the rating scales evaluated in this study. Li and colleagues,⁷ for example, found that the reliability of pain rating scales improved with more exposure and practice. It is possible, for example, that one reason the findings from the current study largely replicated those from developed countries is that the NRS-11 is used so frequently in the hospital where the participants in the current study were recruited. Researchers in this area would do well to measure rating scale familiarity as a potential predictor of scale validity and utility.

Summary and Conclusion

The findings from this study support the validity of four common rating scales of pain intensity for use in Thai samples. We found support for the validity of the NRS-11 in individuals with pain from Thailand. Considerations of the current findings, in light of the findings from studies from other countries, suggest the possibility that the FPS-R might be the best measure for use in international studies in order to facilitate comparisons of findings between countries. Additional research is needed to evaluate the generalizability of the current findings with respect to individuals from rural areas of Thailand.