A think-aloud study of the feasibility of patients with end-stage organ failure completing the ICECAP-SCM

Background: The ICECAP-Supportive Care Measure (SCM) is a self-complete measure developed to inform economic decision making at the end-of-life. Previous research has demonstrated its feasibility in hospice and nursing home settings. This is the first study of its use with patients on the organ failure trajectory. Aim: To determine the feasibility of using the ICECAP-SCM with patients experiencing end-stage organ failure in a hospital setting. Design: Participants were asked to ‘think aloud’ when completing the ICECAP-SCM, ICECAP-A and EQ-5D-5L measures. The interviews were transcribed verbatim and examined for errors in comprehension, retrieval, judgement, and response by five raters. Qualitative data were collected to explore reasons for errors in completing the measures and participants’ views about the measures. Setting/participants: Sixty patients (with end-stage renal failure n = 18; end-stage heart failure n = 21; end-stage chronic obstructive pulmonary disease n = 21) participated. Senior clinicians applied prognostic criteria to determine eligibility. Results: Participants reported that the measures were acceptable, clear, and easy to complete. Error rates in completing the measures were low (ICECAP-A = 3%,and ICECAP-SCM = 5.7% and EQ-5D-5L = 6.3%). There was some variation in responses between patients with different end-stage conditions, particularly those with symptom fluctuation. Some patients had not considered their end-of-life (i.e. advance care planning) and reported finding questions about this difficult to answer. Conclusion: It is feasible to use the ICECAP-SCM with patients with end-stage organ failure receiving care in hospital settings. This study provides evidence for researchers and policy makers involved in measuring end-of-life care globally. The ICECAP-SCM can be recommended for research with patients in end-stage organ failure to appropriately capture the broader benefits of end-of-life care.


Introduction
An ageing population and the rising cost of end-of-life care interventions and services have increased the scrutiny of researchers and policy makers on decision making processes at the end-of-life. 1,2 Globally, about 40 million people are estimated to need end of life care annually with significant economic costs. 3 In many countries, decision makers use frameworks that employ a range of implicit and explicit criteria to guide resource allocation decisions in health care. 4 In countries with health systems funded by direct taxation, public institutions such as the National Institute of Health and Care Excellence (NICE) in the United Kingdom, Pharmaceutical Benefits Advisory Committee (PBAC) in Australia and the Canadian Agency for Drugs and Technologies in Health (CADTH); are tasked with improving the process of allocating health care resources. 5 In common with a number of other countries, NICE relies in part on the cost-effectiveness of health interventions to guide resource allocation decisions. 5 Cost utility analysis is recommended by NICE to appraise the economic benefits of health interventions 6 by comparing two or more interventions in terms of their costs and outcomes. 7 Outcomes are generally expressed as Quality-Adjusted Life-Years (QALYs), a measure of health outcome where life following a health intervention is adjusted for quality. 7 Generic preference-based outcome measures are used to assess the cost-effectiveness of health interventions, with the EQ-5D-5L most commonly advised for this purpose. 8 The use of QALYs in evaluating end-of-life care and economic decision making at the end-of-life has been criticised on the basis that end-of-life interventions are not exclusively aimed at improving health-related functioning, they also focus on providing care, managing symptoms, and achieving a good death [9][10][11] Researchers have suggested that a broader focus for decision making than one centred on health-related functioning, may provide a more appropriate basis for evaluating end-of-life care. 12 The capability approach normatively distinguishes between functionings and capabilities. Functionings are defined as 'beings' and 'doings' for example being in good health, walking, being well nourished, while capabilities refers to the freedom of an individual to 'do' and 'be' what they have reason to value. 13 The capability approach offers an alternative theoretical framework that considers the extent that a person is able to do and be the things they have reason to value, as a basis for evaluation. 14 The ICECAP-Supportive Care Measure (ICECAP-SCM) was developed as a self-complete economic measure based on the capability approach to evaluate health and wellbeing associated with the opportunity for a good endof-life. 15 The feasibility of using the ICECAP-SCM with patients receiving palliative care in hospice and nursing home settings has been demonstrated. 16,17 However, there remain questions about its feasibility for use with patients on different dying trajectories 18 and in different care settings. This is important because hospice patients may have different experiences and needs to hospital inpatients or those not accessing, or not able to access, specialist palliative or supportive care services.
Whilst a single measure is commonly used for evaluating the benefits of health interventions, the multi-faceted objectives of palliative care have led to a growing recognition that a single outcome might not suffice at this stage of the life-course. 19 Improving health, managing symptoms and achieving a good death take on varying degrees of prominence at various points along the dying trajectory. 20 It is essential that current evaluative methods account for these shifting objectives in economic decision making.
The aim of the study reported here was to investigate the feasibility of using the ICECAP-SCM, alongside more established economic measures focused on health (EQ-5D-5L) and generic capability (ICECAP-A), 21 with patients on an organ failure trajectory receiving treatment and care in a hospital setting. It was designed to address the following research questions: what difficulties did participants face while completing the ICECAP-SCM alongside other economic measures? What are the views of participants about these measures, in relation to their health and wellbeing?

Methods
Think-aloud interviews involving retrospective verbal probing, followed by a semi-structured interview were used to explore patients' perceptions of end-of-life care and the feasibility of using the ICECAP-SCM, EQ-5D-5L and ICECAP-A measures. The think-aloud interview is a form of cognitive interview in which respondents are asked to verbalise their thoughts as they complete each measure. 22 Think-aloud interviews have been effective in gaining deeper insight into decision making and experiences of people at the end-of-life. 23 Responses were probed retrospectively to clarify any difficulties experienced in completing the measures during the think aloud interviews to help ensure the rigour and trustworthiness of the data. The research was granted ethical approval from the North Wales Research Ethics Commission and the Health Research Authority (REC reference: 17/WA/0022).

Sampling and recruitment
There are no specific recommendations for sample size in think-aloud studies and numbers of participants have ranged from as few as 9 24 to as many as 72. 16 The aim in the present study was to recruit a total of 60 patients including at least 20 with each of end-stage heart failure (HF), endstage chronic obstructive pulmonary disease (COPD) and end-stage renal disease (ESRD). It was anticipated that a sample of 60 patients would be sufficient to enable comparability of scores for the measures and achieve data saturation for the analysis of verbal responses. 25,26 People near the end-of-life were defined as those in the last 12 months of life. 27 There are acknowledged difficulties in identifying patients with end-stage organ failure. 28,29 In view of this prognostic criteria were developed in consultation with specialists in the renal, respiratory and HF units to increase the likelihood that patients in the last 12 months of life were recruited (Appendix 2).
Recruitment teams of a consultant and at least one specialist nurse were set up in the HF and respiratory units, led by SM. The recruitment teams were responsible for identifying patients eligible for inclusion based on condition-specific prognostic criteria (Appendix 2). Patients were screened (by the recruitment team) against the prognostic criteria and those deemed eligible for inclusion eligible were then screened (by HN) against the generic eligibility criteria (Appendix 3). Patients meeting both sets of criteria were invited to participate in the research. Patients with ESRD who were included had declined renal replacement therapy and dialysis in favour of palliative care. They were recruited from a specialist renal out-patient palliative clinic by the hospital renal team. Patients with end-stage HF and end-stage COPD were purposively recruited from in-patient wards.
Following initial contact by the consultant in charge of their care, potential participants were provided with an information sheet by HN, containing the details of the study. Whilst there was no formal patient and public involvement in the study, the Patient Advisory and Liaison Service of the hospital provided feedback on the research design and topic guide.

Measures investigated
The ICECAP-SCM is a measure of capability well-being at the end-of-life. 15 It consists of seven attributes: choice (being able to have a say), love (being able to be with people who care about you), freedom from physical suffering, freedom from emotional suffering, dignity (being able to maintain one's dignity), support (being able to have support), and preparation (having the opportunity to make end-of-life decisions). Each attribute is expressed at four levels of capability and their wish for that capability; ranging from no capability to full capability.
Respondents are asked about their wellbeing 'at the moment'. The ICECAP-SCM has been tested for feasibility with patients receiving care in a hospice 16 and as a proxy instrument for people living with advanced dementia in nursing homes 17 and tariffs have been developed following valuation exercises in samples derived from UK 30 and German adult populations. 31 The ICECAP-A is a measure of the generic capability of adults, and asks respondents to rate their quality of life 'at the moment'. 21 It consists of five attributes (Stability, Attachment, Autonomy, Achievement and Enjoyment), expressed as four levels of capability ranging from no capability to full capability. The ICECAP-A has been subject to extensive validity testing with different groups of patients, 16,32,33 including patients with mental health problems. 34 The EQ-5D-5L is a measure of health functioning recommended for use in economic evaluation of health and social care interventions in the UK. 6 It has five attributes (Mobility, Self-Care, Usual activities, Pain/Discomfort, Anxiety/Depression) expressed at five levels of severity ranging from no problems to extreme problems. 8 The EQ-5D-5L has been used to evaluate end-of-life interventions and services. 35,36 It measures health status in terms of health gains or loss and asks respondents to rate their quality of life 'today'.

Data collection
Think-aloud face-to-face interviews were conducted at the hospital site or the patient's home (dependent on participant preference). All interviews were conducted by HN. Prior to each interview, participants were asked to perform a think-aloud exercise to familiarise them with the think-aloud process. Following the warm-up task, involving them speaking about the number of encounters they had with health professionals on an admission to hospital, participants were asked to confirm their age, race, and gender. Participants were then given the measures in a random order and encouraged to verbalise their thoughts ('think aloud') as they made their response to each item. If participants were silent for longer than 10 s, they were prompted to keep thinking out loud. Following completion of all three measures, respondents were asked additional 'probe' questions to explore any difficulties the participants experienced and clarify any unclear responses.
End of life is a sensitive topic, and two participants became upset during the interview. When this occurred, the issue was handled sensitively, and participants were asked whether they would like to stop or continue with the interview. A distress protocol was also developed to ensure such episodes were managed appropriately (see p.12).

Data analysis
All interviews were audio-recorded and transcribed verbatim. Think-aloud responses to the items in each measure and their scores were analysed by five independent raters (H.N., J.C., P.K., A.H. and C.B.) to increase rigour. Each item was rated to identify if any of four types of response problems 37 were evident: comprehension (any misunderstanding of a word or phrase), retrieval (a recall error or miscalculation of the time frame stated in the question), judgement (correctly judging how recalled information can be used to answer the question), and response (providing a valid response to the question answered). Additionally, a fifth category, 'struggle' (not an error) was included as in previous ICECAP 'think aloud' studies; this was used to capture instances when participants arrived at an appropriate response that reflected their discussion but clearly (from their discussion) found it difficult to get to this response. 16,38,39 Examples of errors and 'struggle' can be found in the supplementary file (Appendix 1).
An error/struggle classification was made when three or more raters identified a specific error/struggle type in an item response. In cases where three or more raters identified an issue but disagreed on the category, discussion took place to reach a consensus on the specific error/ struggle type. When two raters identified an error/struggle, this was discussed in the group to reach a consensus. If only one rater identified an error/struggle, it was not classified as an error/struggle. Error/struggle rates were calculated as percentages of the number of items completed to enable comparability as the ICECAP-SCM contains more questions than the other measures.
Transcripts of the think-aloud interviews were read and re-read, and codes were generated inductively through open coding. 40 Data were compared across transcripts and to the properties of emerging categories. Qualitative analytic accounts 41 were written for batches of transcripts (6-10 patients with each condition) to enable a rigorous process for assessing the reasons for errors and participants' views about the measures. Qualitative data were managed in NVivo© 12.1 software by H.N. Each stage of the analytic process was reviewed by C.B, J.C., A.H. and P.K.

Results
One hundred and sixteen patients were identified as eligible for inclusion based on the prognostic criteria. They included 47 patients with ESRD, 41 with end-stage COPD and 28 with end-stage heart failure. Nineteen patients were excluded by the recruitment team for the following reasons: three patients were too ill to be interviewed; one patient was discharged; one died before an interview could be arranged; six had cognitive issues and were unable to understand the information sheet; and eight could not communicate in the English language. Ninety-seven patients were invited to participate in the research and 37 declined. The reasons given for declining participation are presented in Table 1.
Participants were recruited at various stages of their illness trajectory including those who were very near the end-of-life; 23 of the 60 participants died within 8 months following the interview including 19 who died within 4 months of interview. Interviews were conducted between June and November 2017. Patients included in the study were aged between 35 and 95 years. Interview duration ranged from 14 to 49 min. The socio-demographic characteristics of the participants are presented in Table 2.

Completion of the measures
Thirty-four of the 60 participants completed all three measures without errors, although six of these experienced struggle. The lowest error rate (3%) was demonstrated in completion of the ICECAP-A which 52 of the 60 participants completed without error. The error rate for completion of the EQ-5D-5L was the highest at 6.3%, compared with 5.7% for the ICECAP-SCM. The EQ-5D-5L and ICECAP-SCM were each completed by 45 of the 60 participants with no errors.
Participants with ESRD had the highest percentage error rate when completing the ICECAP-SCM (8.7%) and the lowest percentage error rate when completing the EQ-5D-5L (2.2%); while those with end-stage heart failure and end-stage COPD had the highest percentage error rate completing the EQ-5D-5L (9.5% and 6.7%, respectively). Absolute and percentage error rates for the three patient groups are shown in Table 3.
Participants completing the ICECAP-SCM made the most errors when completing the 'preparation' and 'emotional suffering' items, as shown in Table 4. Comprehension and response errors were the most common type. Data extracts are included below to illustrate such errors. Most errors in 'preparation' were comprehension errors, with some participants appearing to conflate preparation with day-to-day decision making rather than end-of-life decision making. Similarly some participants explained how they were able to make independent decisions in general terms, rather than focussing on preparation for the end-of-life. "I make a decision myself. I have to ask somebody if I want to make a decision very rare now. I make my own decisions".
[PT-63R] Some participants made response errors when completing the preparation item. In most cases, the question was left unanswered. When asked why the question was unanswered, it was because they had not considered or made specific preparations.

. I don't know about that one" [PT-33 R]
Although participants made the fewest errors completing the ICECAP-A, most errors were made in response to the question about 'achievement and progress'. Participants questioned the relevance of 'progress' to their wellbeing.
Participants did not make any comprehension errors while completing the EQ-5D-5L. Most errors were response errors and were due to the description of the response options. Some participants felt unable to make a selection from the options available when they felt that the item described two different dimensions. For example, the EQ-5D-5L includes a question about anxiety and depression and if people had experienced one but not the other they were sometimes unable or unwilling to answer.
"I think they needed to be worded differently. . . like with that one there, anxiety and depression I think it should have like, am I slightly anxious, yes but do I feel a bit depressed? I know anxiety can lead to depression but a lot of people suffer with anxiety for years and years but they don't become depressed. Like myself, I've had you know, I've been anxious over time but I'm not depressed"  Many participants felt the severity of their condition fluctuated on a day-to-day basis which made it difficult to quantify their wellbeing. This was common among those with end-stage HF and end-stage COPD who did not feel the options in the EQ-5D-5L (expressed in terms of severity) described their condition. The EQ-5D-5L appeared to be less sensitive to the frequent fluctuations of end-stage organ failure and so the participants did not feel the options they were offered to describe their condition accurately represented their current state.

How did participants feel about completing the ICECAP-SCM?
Although a few participants struggled to complete some questions, most participants felt the ICECAP-SCM was clear and easy to complete.
"they were simple questions; they were straightforward to me" [PT-38K] Indeed, many participants were able to articulate their responses in terms of their capabilities. While answering the questions, participants spoke about their wellbeing in terms of what they were willing and able to do. The think-aloud approach encouraged reflection and participants were often aware of their imminent death and became emotional while answering the question about 'preparation' in the ICECAP-SCM and the life they would soon leave behind. This was particularly common among participants in the late end-of-life phase (died within 4 months of interview) "I understand how these professional footballers feel when their glittering career comes to an end and the crowd are completely gone. I don't think I'm any different to anybody else on that, I think everyone is like that because nobody on this planet likes losing, nobody likes to go earlier than they should do. Responding to the inevitability of death was challenging and painful, particularly when the participant felt that the end stage was reached quickly, leaving little time for them to prepare and plan for their death.

Summary of findings
This research examined the feasibility of using the ICECAP-SCM, alongside the EQ-5D-5L and ICECAP-A with patients on an organ failure illness trajectory experiencing endstage organ failure as they approached the end-of-life. Most participants felt the ICECAP-SCM was clear and easy to complete and it compared favourably with the other measures in terms of error rate, although the ICECAP-A had the lowest error rate. Most errors made while completing the EQ-5D-5L were response errors and were due to difficulties experienced by participants with heart failure and COPD in quantifying their symptoms. Similarly, most errors in the ICECAP-SCM were in the 'emotional suffering' and 'preparation' items and these errors were due to difficulties in discussing advanced care planning and the impact of their illness on their emotional wellbeing. The ICECAP-A did not address sensitive topics and focuses on capability wellbeing rather than functioning hence most participants may have found it easier to complete.
The descriptive levels of the ICECAP-SCM, expressed in terms of frequency, appeared to reflect the wellbeing of participants more accurately, particularly those with endstage HF and end-stage COPD who frequently experienced fluctuating symptoms. The focus of the ICECAP-SCM on capabilities associated with end-of-life decision making appeared to be difficult for some participants who became upset while reflecting on the impact of their condition on their emotional wellbeing and the inevitability of death. A distress protocol considering appropriate support mechanisms 42 should be in place when using the ICECAP-SCM or any research that involves asking sensitive questions to people near the end of life.

Relationship to wider literature
This is the first study to explore the feasibility of using the ICECAP-SCM in patients with end-stage organ failure. A previous study of the ICECAP-SCM in comparison to other measures in a nursing home setting reported a higher error rate for the ICECAP-SCM, 17 while a study conducted in a hospice setting 16 reported a lower error rate. Consistent with the findings reported here, Bailey et al. noted that participants had difficulties discussing advanced care planning, reflected in more errors in responding to the 'preparation' items. 16 Despite the challenges involved in having discussions with people near the end-of-life, 43 they are generally seen as beneficial to patients 44 and can lead to more efficient use of health care resources. 45 The organ failure trajectory, with its inherent rapidly fluctuating health states, makes measurement of health functioning particularly challenging and use of existing measures can lead to inaccurate estimates of the benefits of interventions. 20,46 The descriptive levels of the EQ-5D-5L, expressed in terms of severity, made it difficult for participants to quantify their health state due to fluctuations in their condition, as found in other studies. 16,47 This problem of fluctuating health states is not unique to end-of-life, and is the subject of ongoing research 46 The descriptive levels of the ICECAP-SCM expressed in terms of frequency may be more appropriate in evaluating endof-life care outcomes in those very near the end of life.

Implications for research and policy
Despite the challenges involved in using the measures for patients with end-stage organ failure, all three measures were feasible to use and accepted by patients. The holistic approach to care advocated in national and international end of life care policy 48,49 underscores the need for a range of measures to capture the diverse benefits of interventions across the end-of-life course. The ICECAP-SCM has been translated in five different languages which will facilitate its use in capturing outcomes beyond health functioning in low-and middle-income and non-English speaking countries. Health, generic capability, and capabilities associated with end-of-life decision making take on varying degrees of relative importance as patients move through the end-oflife course. 50 Using all three measures in an integrated framework may provide a more accurate approach for evaluating the impact of palliative care interventions in end-oflife research for patients on the organ failure trajectory.

Strengths and limitations
The study has several strengths and some limitations. Sixty participants with end-stage organ failure participated in the research, making it the largest patient-completed think-aloud study using any of the ICECAP measures to date. The research was successful in recruiting people who were very near the end-of-life; a challenging group to recruit. The accuracy of the application of the prognostic criteria in selecting participants for the study is confirmed by the fact that 23 of 60 participants died within 8 months of being interviewed. Some participants died within a few days of being interviewed, demonstrating that people at different stages of the organ failure trajectory were included. Furthermore, participants were purposively recruited with different conditions causing end-stage organ failure who received different forms of care ensuring the generalisability of findings given contemporary evolution in palliative care delivery models. The study has some limitations. All participants in the research were recruited from one geographical location, in a single site in England and most self-identified as white-British. There is however no evidence of regional and ethnic differences in end of life patients completing the ICECAP measures. Previous studies 16,17 using similar methods recruited from a single site with participants mainly identifying as white British. Hence, future studies using the ICECAP-SCM would benefit from being repeated with different ethnic groups in a range of geographical locations to explore its use across a more diverse population and identify any potentially challenging aspects of measuring quality of life. Given the need for a range of measures to evaluate the benefits of interventions across end-of-life trajectories, further research on how best to combine the measures and the challenges and feasibility of doing so would be beneficial.

Conclusion
The ICECAP-SCM was specifically developed to evaluate health and capability wellbeing at the end-of-life and this study has shown that use of the measure is feasible and acceptable to patients with end-stage organ failure. Used in conjunction with the EQ-5D-5L and ICECAP-A, it provides researchers and policy makers with an approach to evaluate the impact of end-of-life interventions and provides a more appropriate estimation of the benefits of end-of-life care.

Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors declare no conflict of interest with respect to the research, authorship and/or publication of this article. JC led the development of the ICECAP-A and ICECAP-SCM.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by a PhD studentship provided by the College of Medical and Dental Sciences, University of Birmingham. Joanna Coast is supported by the Wellcome Trust [205384/Z/16/Z]. As this research was funded in part by the Wellcome Trust [Grant number 205384/Z/16/Z], for the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.