Evaluating the ability of patient reported outcome measures to represent the functional limitation of people living with myalgic encephalomyelitis/chronic fatigue syndrome

ABSTRACT Aim To identify activities which people with Myalgic Encephalomyelitis (ME)/Chronic Fatigue Syndrome (CFS) report are impacted by their condition, and evaluate the ability of measures of function used by National Health Service (NHS) ME/CFS Services to represent these experiences. Method 122 participants completed ME/CFS Service questionnaires reporting activities that they had reduced or stopped doing, as well as Patient Reported Outcome Measures (PROMs). These data were coded using the International Classification of Functioning, Disability and Health (ICF) using established linking rules. Matrices identified the agreement rate between the outcome measures and the participant-generated list. Activities which could not be coded against the ICF were grouped using content analysis. Results Responses from participants related to codes from nine subsections of the ICF. The PROMs used by the ME/CFS service had agreement rates between 58% and 62.5% with the participant-generated list. The content analysis identified a range of activities that were meaningful to participants that they could no longer do. These included holidays and day trips, accessing the community independently, and sustaining activity into the evening. These were not captured in either the ICF or the service’s outcome measures. Conclusion The list generated by participants referred to a wide range of activities, including some not captured by the ICF. Comparison against the outcome measures suggests that the measures used in many NHS ME/CFS services nationally capture patients’ experiences moderately well. However, there are activities that patients value that are not captured by these measures.


Introduction
Myalgic encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) is a long-term disabling health condition characterised by debilitating fatigue with post-exertional malaise, disturbed and/or unrefreshing sleep, and cognitive difficulties that have lasted longer than three months [1][2][3].These symptoms can fluctuate rapidly and unpredictably in their severity and exacerbations associated with post-exertional malaise can be disproportionate to peoples' activity levels [4].Therefore, people living with ME/CFS often need to carefully balance their activities with rest, or miss out on some activities altogether, in order to cope [5].There is limited literature detailing the spectrum of activities impacted by ME/CFS, but this knowledge would be of value to healthcare professionals and researchers.This knowledge may also be helpful to individuals living with ME/CFS, given that eligibility for financial welfare is typically determined with a focus on peoples' functional abilities [6].Recognising areas that people find difficult can help to document the difficulties the condition causes.
One way to understand and categorise activities impacted by long-term conditions is to use the World Health Organisation's International Classification of Health, Disease and Function (ICF) [7].This framework provides consistent descriptions with an associated coding scheme for bodily structures and their functions, activities and participation, and environmental factors that relate to health and participation.The ICF has a variety of uses, including in health services to identify changes in peoples' function [8].Gladwell et al. [9] describe the benefits of using the ICF for inductive thematic analysis of qualitative data, facilitating cross-matching against Patient Reported Outcome Measures (PROMs) and highlighted some of the hazards of using PROMs which do not match patient experience.The development of ICF linking rules supports this technique of comparing PROMs against the ICF [10,11] and crossmatching the ICF against patient reports.
Prior studies have linked PROMs to pre-existing ICF core sets for other long-term conditions [12,13].A preliminary core set also exists for ME/CFS and captures 17 different aspects of activities and participation, including items that capture forms of functional impairment [14].Some work has also begun to evaluate PROMs used in ME/CFS services within the U.K. National Health Service (NHS).This work has looked to find PROMs that can identify change as a means of evaluating rehabilitation support [15].This includes the Short Form 36 (SF36) health survey [16], which is prevalent in health services working with ME/CFS.This work identified that measures of functional classification in ME/CFS do not provide a great deal of insight into loss of quality of life due to changes in physical function [15].There are still gaps in the knowledge about how peoples' experiences of ME/CFS are captured by outcome measures regularly used in services [17].
This service evaluation had two primary objectives: 1. To identify activities which patients seen at an ME/CFS service have had to limit or been unable to do because of their condition.2. To investigate whether the PROMs for physical function used by ME/CFS Services nationally captured participants' experiences.We will do this by cross matching the PROMs against the participants' experiences using ICF codes.
This provides an opportunity to benchmark the PROMs used by many ME/CFS services nationally to evaluate the extent to which they capture patients' experiences of disability.As a precedent, this benchmarking has been done with other measures of participation used for a wide range of other health conditions [13].PROMs that accurately capture patients' experiences of functional disability will also help to evaluate effective treatment strategies for the condition.Additionally, this knowledge can help services to make informed choices around service provision and treatment efficacy.Because many ME/ CFS services use the same PROMs, the findings of this study have implications for wider clinical assessment across ME/CFS services in the U.K.

Design
This was a mixed-methods service evaluation project that used existing observational, cross-sectional data to meet the research objectives.This method was selected because it made use of existing data collected for clinical assessment and avoided additional participant burden.Completion via postal questionnaire allowed participants with ME/CFS to pace their completion.It also potentially increased the helpfulness of their generated list of activities, as it gave participants time to naturally encounter and note things that they were unable to do.

Ethics approvals
Ethics approval to commence data processing was obtained from the Research & Development team at the relevant NHS Trust as a service evaluation project (ID: 3821) and ratified by Newman University (project number 2016-08-30-1404324/2903).It is important to note that this work was completed in 2016, prior to the introduction of General Data Protection Regulation (GDPR) legislation.This meant that it was possible to use the data collected for the purpose of clinical assessment for evaluation purposes with appropriate research approvals.However, because of the timing of the research, the raw data is no longer available to comply with GDPR legislation.

Participants
Participants were a cohort of patients attending their first appointment at a specialist ME/CFS service in an NHS Trust.This service provides assessment by a specialist clinician to diagnose ME/CFS where patients' experiences met both the Fukuda and the 2007 NICE Guideline criteria [18,19], ensuring that post-exertional malaise was a required element of the diagnostic criteria.The service also offers courses to support patients to cope with the impact of the condition and optimise self-management of their condition.Patients can also access one-to-one support with specialist clinicians including occupational therapists, psychologists and physiotherapists.This made use of existing data collected for clinical purposes to evaluate the measures used in the service.Participants whose data were included in this dataset were determined based on them meeting the inclusion criteria.Data collection began when research approvals had been granted, with participants who met the inclusion criteria providing the data in the form of completed service questionnaires.More details are provided under 'Procedure'.

Inclusion criteria
Adults (over age 18) referred to the ME/CFS Service who were given a diagnosis of ME/CFS after their initial appointment, and who had completed and submitted the questionnaires described below for this assessment appointment.The participants providing this data were attending their first appointment with the service.Anyone who did not have a diagnosis of ME/CFS did not have their data included in this study.

Measures
An initial questionnaire was created by the service to allow patients referred for assessment to provide information required for the assessment in advance.The intention was to reduce the cognitive load of the assessment by allowing patients to write notes about multiple domains.These domains included symptoms, sleep, general health, medication, and impact on function.This questionnaire had 2 items in particular that were used for this study.These were two free-text questions that asked participants what activities they had to reduce because of their ME/CFS or stopped doing entirely due to their ME/CFS.Participants also completed standard outcome measures used by the service, including the following two measures of physical function: . The Work & Social Adjustment Scale (WSA) [20]: a five-item measure covering domains such as domestic activities and interpersonal relationships.It has demonstrated good reliability and validity, with a range of internal consistency statistics above 0.7, and testretest reliability of .73. .The Short Form 36 health survey physical function subscale (SF36) [16]: a 10-item measure scored by Likert scales, that has been found to have good reliability, with a Cronbach's alpha over .85,as well as good content validity [21].The measure focuses on physical function and activities such as lifting, walking a variety of distances, and moving around inside and outside of the home.These measures were selected not only for their common use in the described ME/CFS service, but because many ME/CFS services across the U.K. used the same set of outcome measures as part of a wider study investigating patient cohorts known as the National Outcome Database [22].We also collected demographic data, including participants' age, gender, duration of symptoms, when they were diagnosed with ME/CFS (if diagnosed before their first appointment with the service).

Procedure
Data collection occurred as part of usual care, with the analyses occurring later as part of a service evaluation.Data were collected for the purpose of contributing information to clinical assessment, to personalise care and help individuals determine treatment goals.The data generated by the two qualitative free-text questions (describing which activities had been reduced or stopped) were compiled in a spreadsheet to facilitate crossmatching against the ICF.We coded responses according to the closest match of the activity against the ICF [7] using a set of established linking rules for the ICF [10,23].We did not relate the codes back to each individual as the aim was to compile a list of activities reported by the cohort as a whole.Any activities participants reduced or stopped because of ME/CFS were matched against the codes in the ICF independently by two separate authors (BJ and PG).A small number of discrepancies in coding were resolved by discussion of the alternative codes to reach consensus.Data which could not be coded were analysed using content analysis [24].Here, the data which could not be matched against the ICF were compiled into a list.Then these data were organised into categories of similar issues to summarise and describe what remained.This included an 'Other' category for activities which came up that did not share similarities with other activities that had not been coded against the ICF.Categories in the content analysis were reviewed and were combined if they shared experiences or covered similar activities.Data were compiled and analysed until the research team determined that no new codes had been generated in the participant-generated list for ten consecutive new participants as a stop rule for data saturation.In total, data were used from 122 patients before data saturation [25].
The activities referred to in the PROMs were also matched against ICF codes in the same fashion.To calculate the agreement rate between the PROMs and the reported participant experiences, matrices were created with the complete list of codes to compare each outcome measure against the participant generated list to calculate a percentage match.

Results
Of 122 participants, 101 (82%) were female with an average age of 38.3 years (range 18-73).Participants had lived with the symptoms of ME/CFS for an average of 5.48 years (standard deviation: 6.36).
Participant-generated list of activities 655 separate activities were provided and 577 were attached to 65 separate codes from the ICF that captured these activities.From the data that could be linked to the ICF, the majority related to the 'Activities and Participation' section.Table 1 reports the breakdown of codes by the different subsections of this section.Responses from participants were related to codes from eight of the nine subsections of the 'Activities and Participation' ICF section.The most common code was Managing Diet and Fitness d5701, with 86 occurrences.Of these, none were diet specific and only related to fitness and exercise.Other common codes were Socialising d9205 (85 occurrences) and Doing Housework d640 (72 occurrences).Identifying which subsections had the highest number of unique codes can help understand life areas where participants reported the most widespread impact on their activities.The list of included codes by ICF subsection is available in Table 1.
Codes which were prominent in the participant-generated list (i.e.mentioned by over 10% of participants) but did not feature in the outcome measures include Swimming d4554, Driving human-powered transportation d4750, Managing diet and fitness d5701, and Arts and Culture d9202.

Outcome measures and ICF codes
18 out of 31 codes identified in the Work and Social Adjustment Scale were also participant-generated codes.This is a 58% agreement rate (see Table 2).20 out of 32 codes identified SF-36 Physical Function subscale were also participant-generated codes, with an agreement rate of 62.5% (see Table 3).

Remaining categories
Most activities described by participants could be linked to the ICF, and those that could not be linked were organised into groups of similar meaning or activities via the content analysis.A list of the categories that the codes were organised into, along with the number of times each category was listed by participants is in Table 4.
The analysis of the remaining data identified that participants often felt like they struggled with 'everything', reflecting that the impact of their condition felt allencompassing.
Another category of activities that participants struggled with that was not specifically captured in the ICF were holidays and day trips.These included mention of specific locations such as restaurants or amusement parks for some participants.For others, it was the broader experience of activities that were leisure-based and outside of the home for extended periods of time.
For some, sustaining activity through the day or doing activity late in the day was not possible because of their ME/CFS.The fluctuations of their symptoms meant that they had to plan the timing of their activities carefully, and this was not identified in PROMs assessing their health.

Discussion
The key findings of this study were that ME/CFS has a significant impact on participants' functional abilities across a wide range of activities, and that there were activities that participants struggled with that are not captured by PROMs used in ME/CFS NHS services.
We note that the participant-generated list should not be considered as a comprehensive list for all activities that patients living with ME/CFS struggle to do.For example, participants did not report that they struggle with bending, kneeling, or stooping (captured by item six on the SF36 physical function subscale) and this was not captured in the existing ME/CFS core set (11).However, the scores on this item (average: 2) indicate that this is an activity that participants struggle with.
There were activities that were difficult to capture with the ICF.One example was watching television.We coded this as d110 Watching, but there is no specific ICF code for watching television.Given that this is a very common activity in the U.K., and has been identified as difficult for patients with ME/CFS to maintain [26], having a code to capture this would be beneficial.
Another commonly mentioned activity which could not be linked to an ICF code was sustaining activity into the evening.This was also not captured in the existing ME/CFS core set [14].Given the specific requirement of 'sustainable activity' to determine a person's eligibility for the Personal Independence Payment criteria [1] capturing the difficulties that this can cause is crucial for getting people the support they need.
Coding activities around spending time with children was also difficult.Participants referred to activities such as childcare, however it was often unclear whether activities should be coded as parent-child relationships d7600, assisting others d660, or play d9200.Further research into the impact of ME/CFS on parenting would add clarity on how this is impacted.The outcome measures related to function typically captured specific physical behaviours.Some duplication in the SF-36 subscale may have impacted the scale's agreement match with the generated list of ICF codes.For example.given that three different items in the SF-36 cover walking, this will impact the agreement rate.
Overall, health care professionals working in the relevant services should be mindful of the moderate agreement rate between the activities that are impacted by patients' ME/ CFS and the PROMs they use to capture functional abilities.Assessment questions would benefit from adding items found in the participant generated list but that didn't appear in the measures, such as the impact on sustaining activity through the day.Working alongside patients with ME/CFS will be crucial to identify a balance between comprehensive measures and the potential burden of excessively long measures.As this evaluation identifies the global impact of ME/CFS on 'everything', there is also a case for considering the use of a global measure of impact such as the Bristol Rheumatoid Arthritis Fatigue Scale Numerical Rating Scale (BRAF-NRS V2) which has been derived from patient experience and is internationally validated.A global measure of impact offers an indication of impact with a significantly reduced patient burden for completion [27].
Limitations of this study relate to the sampling method.For example, sampling from an outpatient service will have excluded potential participants who are being impacted by activities they are no longer able to do, because they are not well enough to complete and return the questionnaires, or even access the service in order to have been eligible to complete the questionnaires.Therefore, patients severely affected by ME/CFS may not have the energy resources to participate and are excluded from research as a result.We cannot assume that the participants are reflective of the wider population of patients living with ME/CFS who have not accessed the service for other reasons, for example as a result of inequalities of access as a result of protected characteristics such as learning disabilities, and ethnic background.We also have a limited catchment as a result of being based in England, and the list of impacted activities in different populations around the world could be quite different.
Exploring what the specific obstacles to participation in the activities identified here that people with ME/CFS find difficult to do could be a target for research in the future.The findings of this research could inform the way in which these behaviours, and the obstacles to carrying them out could be targeted in outcome measures in the future.

Conclusion
People living with ME/CFS reported that a wide range of activities were impacted by their condition, and some indicated that the impact on participation was global.Some of these activities are not captured by the ICF.Comparison against the outcome measures FATIGUE: BIOMEDICINE, HEALTH & BEHAVIOR suggests that the measures used in many National Health Service ME/CFS services nationally capture people's experiences moderately well.However, there are activities that people with ME/CFS value that are not captured by these measures.Further work is required, in partnership with people living with ME/CFS to consider adapted measures of function which more comprehensively represent the impact of the condition.Global measures of impact should also be explored as an alternative in situations where reduced participant burden is more important that a comprehensive representation of activity limitation.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Notes on contributors
Dr Bethan Jones is a research fellow at UWE Bristol with an interest in long-term health conditions and self-management.

Dr
Corin Bourne contributed to this work while a Senior Lecturer at Newman University working in the Psychology department.Dr Peter Gladwell is a Clinical Specialist Physiotherapist and currently works at the Pain Service and ME/CFS Service, North Bristol NHS Trust.He is a visiting Research Fellow at UWE, Bristol and does research in Physiotherapy and Rehabilitation Medicine.

Table 1 .
Breakdown of 'activities and participation' subsections.

Table 2 .
Work and social adjustment scale comparison to ICF.

Table 4 .
Categories of activities which could not be coded against the ICF.