Asking what matters: The relevance and use of patient‐reported outcome measures that were developed without patient involvement

Abstract Background Patient‐reported outcome measures (PROMs) are increasingly used to establish the value of health care. In order to reflect value, PROMs should measure outcomes that matter to patients. However, patients are not always involved in the development of PROMs. This study therefore aimed to investigate whether PROMs, which were developed without patient involvement, are relevant to patients and whether the level of importance allocated towards aspects of these PROMs varies between patient groups. Methods All patients from 20 Dutch hospitals undergoing hip or knee surgery in 2014 were invited to a PROMs survey. Participants were asked to rate the importance of each of the items in the HOOS‐Physical Function Short form or the KOOS‐Physical Function Short form, the EQ‐5D and the NRS pain. Results Most outcomes were considered important. However, 77.7% of hip surgery patients rated being able to run as unimportant. Being able to kneel (32.7%) or squat (39.6%) was not important to a considerable minority of knee surgery patients. Pain, especially during rest, was considered very important by both hip (68.2%) and knee (66.5%) surgery patients. Patients who were older, male, experienced overall bad health and psychological health considered many items from the PROMs less important than other patients. Discussion Patients differ in what they consider important. Health‐care professionals should explore patients’ preferences and discuss which treatment options best fit patients’ preferences. Additionally, if PROMs are used in performance measurement, further research is needed to look at whether and how variation in patient preferences can be taken into account.


| INTRODUCTION
In several countries such as the United States, the United Kingdom, Sweden and the Netherlands, policymakers try to deal with the ever higher spending on health care for mediocre quality of care. They do so by shifting the focus from contracting or paying for health care based on numbers and price towards contracting based on quality. [1][2][3][4][5][6][7][8] The goal is to achieve the highest value, which is the best possible health outcomes per monetary unit spent. 9 Delivering good quality of care and thereby achieving good health is less expensive than having to deal with poor health. 10 Therefore, a value based health-care system is expected to increase the economic sustainability while benefiting patients, health-care purchasers and health-care providers by improving care. 11 A way to establish value is to measure and compare patient outcomes 12 and weigh these against treatment costs. 5 Patient outcomes are increasingly measured using patient-reported outcome measures (PROMs). 3,4,13,14 PROMs use the patient as a source of information on health outcomes such as quality of life. 3,15 Including patient-reported outcomes is important, as some questions about health care can only be answered by patients. 16 Additionally, patients offer a different view on outcomes, 17 and patients are becoming important stakeholders in health care. 18,19 However, if PROMs are to be used to establish the value of health care, there are two aspects which may need further consideration.
Firstly, research suggests that publicizing health-care performance results leads to a focus on low scoring aspects of performance in quality improvement efforts. 20  PROMs, including several PROMs which were used in this study, suggests that patients are not always involved in PROM development. 21 Consequently, health-care providers may have improved on some aspects of care, but at the same time may have neglected to improve on other aspects of care which are important to patients. Patient outcomes are included because delivering high value for patients should be the main goal of health-care delivery. 10,11 Therefore, failing to improve on aspects of care which are important to patients negates any value PROMs may add.
The second aspect of using PROMs that needs further consideration is that, even if PROMs reflect the patients' perspective, they still only reflect the overall patient population. Usually only the aspects which are considered important by most patients are included, which means that more uncommon symptoms or complaints are neglected.
The focus in health care is shifting towards a more person-centred approach, 22 whereby the patients are actively involved in their care and care is individualized by recognizing that a patient is a person with specific needs, preferences and values. 8 This is also relevant for the use of PROMs, as individual patients may differ in the importance they attach to different outcomes. 12,23 For example, an 80-year-old patient living in a nursing home may be less interested in being able to perform physically demanding functions such as running than an active 60-year-old patient. Measuring and interpreting health outcomes as if patients regard the measures as equally important may not give an accurate view of how patients perceive the quality of their care.
However, before any methods that take individual differences into account are included, it is important to establish whether PROMs reflect the issues that are important to most patients. There are several types of preference based measures which may give more insight into these issues. Examples are standard gamble, time-trade-off and rating scales. 24,25 Standard gamble and time-trade-off ask patients to consider what they would be willing to sacrifice to avoid being in a particular health state. 24 However, it is our aim to explore the importance patients allocate towards aspects of existing PROMs which were developed without patient involvement. Therefore, we used importance rating scales. Importance ratings are an easy way to look at whether patients regard any part of a PROM as important 26

| Participants
This study is part of a study carried out by a Dutch health insurers collaboration. 27 All patients in the Netherlands who underwent hip or knee surgery in 2014 were invited by either their health insurer or their hospital, depending on the hospital's choice of data collection (either coordinated via the health insurers or organized by the hospital itself). For 20 hospitals, patients were asked to complete importance rating scales in addition to the standard questionnaire containing patient-reported experience measures (PREMs) and PROMs. These 20 hospitals were selected based on a high and more or less even number of patients undergoing hip and knee surgery. Hospitals who coordinated their own data collection instead of participating in the data collection via the health insurers were excluded. Patients younger than 16 years and patients who had been invited earlier that year to complete a similar questionnaire were excluded.

| Procedure
In the Netherlands, health insurers are legally allowed to contact their clients for participation in research which may help to improve the quality of care. Of course, in doing so they should guarantee the privacy of the patient in all circumstances. An important aspect of the research health insurers did was setting out PREMs and PROMs for certain types of interventions among their clients.
The health insurers sent their clients a letter within 12 months after surgery inviting them to fill in a questionnaire regarding the care they had received. The letter contained a link to the website with the questionnaire and login details. The letter was accompanied by a card which the client could send back if he or she declined to participate.
A reminder was sent a week after the invitation letter. A second reminder accompanied by a paper version of the questionnaire was sent 2 weeks later. Three weeks after the paper questionnaire was sent, a fourth reminder was sent.

| Measures
The questionnaire was among others comprised of basic information, PROMs, rating scales and a question regarding the main reason for surgery. The basic information used for this study concerned age, sex, education level, overall health, overall psychological health and complications.
Rating scales were added to the HOOS-Physical Function Short form (HOOS-PS) 28 or the KOOS-Physical Function Short form (KOOS-PS), 29 the EQ-5D 30 and the NRS pain. 31 Participants were asked for each of the PROM items how important this item is to them.
For example: "How important is being able to descend stairs to you?" Participants could answer on a four-point scale (1 "Absolutely not important"-4 "Of the greatest importance").
The PROMs were developed without patient involvement.
Participants completed the HOOS-PS or the KOOS-PS depending on whether participants were operated on their hip or knee. The KOOS is based on the WOMAC Osteoarthritis Index, a literature review, an expert panel and a pilot study. 32 The HOOS is an adaptation of the KOOS. 33 The HOOS-PS and KOOS-PS were created by shortening the HOOS and the KOOS using Rasch analysis. 28,29 Both questionnaire are validated extensively. [34][35][36][37] The HOOS-PS 28 and the KOOS-PS 29 measure physical functioning level. Participants rated the degree of difficulty that was experienced during the last month and the month before surgery due to the hip or knee problems on a five-point scale ("None"-"Extreme"). The HOOS-PS consists of five items and the KOOS-PS of seven items.
The EQ-5D was developed by the EuroQol Group and is validated in many settings worldwide. 30 The questionnaire measures health status by asking participants to indicate the degree of difficulty participants experienced ("No problems"-"Major problems") over five dimensions for the day they completed the questionnaire and just before surgery.
The NRS pain 31 is a validated numerical rating scale where participants rate their pain intensity from 0 to 10 (0 "No pain"-10 "Worst possible pain"). Participants rated their pain intensity during rest and while using their hip or knee for the month before surgery and the month preceding the day they completed the questionnaire.
Finally, patients were asked a question regarding their main reason for undergoing surgery: "We can imagine that it is difficult to choose one reason. However, we would like to ask you what your main reason for undergoing hip or knee surgery was?" Answer options were as follows: "Mostly to improve function," "Mostly to lessen the pain" and "I cannot choose."

| Statistical analyses
Univariate analyses were performed to describe the participant characteristics and to give insight into how participants answered their PROM questions and rating scales. To investigate whether different patients allocated different levels of importance towards certain items, a series of linear regression analyses were performed. The importance ratings were included as dependent variables in separate regression analyses. The analyses were controlled for medical complications and time between surgery and questionnaire completion, as the time between surgery and questionnaire completion varied greatly. The outcome item corresponding to the importance rating (which was used as dependent variable), age, sex, overall health, overall psychological health, educational attainment (university, higher vocational education, middle vocational education, high school/secondary education, <high school level) and main reason for surgery (pain, functioning, cannot choose) were included as independent variables. For analyses regarding PROM items which were answered by all participants, the variable type of surgery (hip or knee) was added. To take into account the high number of regression analyses, a cut-off point of .01 for the P value was used. Analyses were conducted using spss 22.0. 38

| Response
A total of 3996 patients from 20 hospitals were invited to fill in the questionnaire which included importance ratings. 1108 patients participated by filling in the questionnaire online, while 1811 patients used the paper version. A total of 589 patients did not react, while 488 patients declined to participate. This is a response rate of 73.1%. This response rate is slightly higher than the response rate of the bigger study, which was 72.0%. Data from 40 patients were removed as these respondents indicated that someone else answered the questions.
Data from 103 patients were removed as these patients completed less than five questions. The final number of included patients was 2776 (69.5%) (Figure 1), which is slightly higher than the bigger study (63.1%). Non-respondents differed only in age from respondents (73.

| Sample characteristics
Most participants were female (65.7%) and received secondary education (56.0%) ( Table 1). Their average age was 72.0 (range=28-98; SD=9.1) and a small majority of the participants underwent hip surgery (52.5%). Although 27.0 percent of the participants experienced complications, on average patients improved on all PROMs (Table 1).

| Importance ratings
Although most PROM questions were considered important, some were deemed more or less important than others (

| Factors which influence what patients consider important
Many factors determine the level of importance patients allocate towards the PROM items. For the HOOS-PS items, the level of importance mainly depended upon age, overall health, and to a lesser extend the reason for surgery (Table 3). In contrast, the importance of the KOOS-PS items was related to the reason for surgery, sex, and overall psychological health (Table 4). Importance ratings for the EQ5D were related to age, sex, overall health and overall psychological health (Table 5)  important. Women considered many EQ-5D items (three of five items) and KOOS-PS items (five of seven items) more important than men. Finally, hip surgery patients who were younger rated a reduction in both pain during rest and pain during use as more important (Table 6).

| DISCUSSION
The present study's aim was to give insight into the relevance of a few well-known PROMs from a patient's perspective and the possible differences in preferences between patients. expect from surgery, the question is whether treatment outcomes will match these patients' preferences and whether patients will benefit optimally from surgery.

| Limitations and strengths
The present findings should be regarded with some caution because of study limitations. First, this study used a retrospective post-thenpre design. Measuring the patient's outcomes after surgery may have influenced the accuracy and completeness of patients' recall.
Research, however, indicates that the impact of measuring afterwards instead of before and after is minimal. 44 It may even improve the accuracy as no response shift takes place. 45 Second, all patients who underwent surgery during the year 2014 were invited to participate, while the questionnaires were sent out all at once. This means that the time between surgery and questionnaire completion varies. Although analyses were controlled for the number of days after surgery, results may be influenced by a recall bias. Third, respondents were on average about a year younger than non-respondents. As our results show that older patients consider many aspects of outcomes less important, inclusion of these non-respondents would probably have led to greater variance in the importance ratings. Fourth, importance ratings were used because it is an uncomplicated method for gaining insight into patient preferences regarding aspects of PROMs. 26 However, the disadvantages of using importance ratings are that it is possible for patients to rate every item as important 42 and that the overall results tend to be skewed. 26 As suggested by Sixma et al., 26  An important strength of this study is that, although patients were not actively involved in the study, by adding importance ratings to a PROMs survey, this study was able to give some important insights into the relevance of PROMs from the patients' perspective. Additionally, the study also gave more insight into patients' preferences regarding outcomes of hip and knee surgery. These results have important consequences for the use of PROMs during medical consultations and for measuring the value of health care. The results of this study are therefore a useful contribution towards the discussion regarding patient involvement in research, health care and medical practice.

| Implications
Our results show that not all aspects of the PROMs are considered  patients need to be positioned very complicated. 49 For example, the easiest function is placed at the beginning of the continuum and the most difficult function at the end of the continuum. If a patient is able to use almost all functions, he or she will be placed near the end of the continuum. Weighing results will mean that some functions are taken into account more than others. This would make it impossible to place the patient on the continuum. Further discussion on whether and how importance ratings should be integrated into PROM results is needed.
However, if the construct is indeed seen as a latent dimension, a suggestion is to not weigh the results. Instead, one could adjust the continuum to ensure that items at the end of the spectrum still represent aspects of the construct patients and their physicians feel are relevant and achievable. For example, in the case of many hip surgery patients being able to run is both unimportant and unachievable. For these patients adjusting the continuum would mean that the most demanding outcome becomes a more attainable outcome such as twisting on a loaded leg instead of running. This way patients are more likely to complete the questionnaire 19 and physicians are not only judged on outcomes which may be a better representation of what is relevant to patients, but they are also judged on outcomes which are actually achievable for a specific patient.
Fifth, variation in preferences between patients is also relevant for patient-provider communication. Patients are becoming more actively involved in their treatment, and awareness of patient's values, needs and preferences is important to individualize care 8 as part of a personcentred approach. 22 For the elective surgery procedures which were investigated in this study, there are several suitable treatment options besides surgery. 50 Taking into account patient's preferences may result in patients receiving the most appropriate treatment, which is both better for the patient and for the value of our health care. 51 As this study indicates that patients' preferences do vary, it is important that patients and their health-care providers discuss and take into account patient preferences and the benefits and risks of the treatment options to come to a well-informed decision. 42,52 Coming to a well-informed treatment decision may be especially important for the patients who considered several outcomes of joint replacement less important than other patients (older patients, male patients, patients who experience overall bad health and psychological health and patients who mainly want to decrease their pain level). For these patients, other treatment options may be able to offer outcomes which are of more importance to these patients without having to undergo surgery.

| CONCLUSION
Although many items from the PROMs included in this study were important to patients, not all aspects are equally important. Preferences also appear to differ between patients. Especially older, male patients, patients who experienced overall bad health and patients who experienced bad psychological health considered many aspects of the PROMs less important than other patients. These results have important consequences for the use of PROMs during medical consultations and for measuring the value of health care. The differences in preferences between patient groups indicate that it is important for health-care professionals to explore patients' preferences and discuss which treatment options best fit the patient's preferences.
Furthermore, as PROMs are used to establish the value of health care, the variations in the importance levels may need to be taken into account to ensure that PROMs give an accurate view on how patients perceived the quality of their care. Further research is needed to investigate whether and how variations in importance can be integrated into PROM results.