Serious Illness Survey for Home-Based Programs

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting qualityassurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit www.rand.org/about/principles.

iii Tables   Table 3.

Introduction
Background There has been rapid growth of community-based programs that provide care for seriously ill individuals in their homes. Although measuring, monitoring, and incentivizing high-quality care is critical for this vulnerable population, until now, no standardized measures have been developed for programs that provide care to the seriously ill.
Experts in care of the seriously ill have highlighted the importance of quality measures that assess the person-and family-centeredness of care. Surveys of seriously ill individuals and their family members are an essential means of understanding whether care is person-and family-centered.
RAND Corporation researchers, supported by funding from the Gordon and Betty Moore Foundation and in close collaboration with Dr. Joan Teno of Oregon Health and Science University and Paul Cleary of Yale University, developed and field-tested a survey to assess the care experiences of individuals who receive care from programs that provide serious illness care to patients in their homes. The goal of this work was to develop survey-based measures that assess the quality of care provided by these programs.

How Was the Serious Illness Survey for Home-Based Programs Developed?
The RAND research team developed the Serious Illness Survey for Home-Based Programs using a systematic literature review of tools that measure experiences with serious illness care; qualitative interviews with patients, family caregivers, and health care providers from a diverse set of serious illness programs nationwide; a technical expert panel composed of experts in serious illness care quality; and cognitive interviews with patients and family caregivers to test draft questions and questionnaires.
Wherever possible, we adapted survey items from previously validated survey instruments, such as the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician & Group Survey and the CAHPS® Hospice Survey.
A draft version of the survey was tested between October 2019 and January 2020 among patients of 32 serious illness programs that provide home-based care across the United States, prior to the onset of the coronavirus disease 2019 (COVID-19) pandemic.
Results of this field test were used to refine and finalize two versions of the survey instrument (a full version and an abridged version for those seeking a shorter survey instrument), as well as recommended survey administration procedures. Details about the abridged version of the survey instrument can be found at www.rand.org/Serious-Illness-Survey.

Who Can Use the RAND Serious Illness Survey?
Serious illness programs, payers, and others can use the Serious Illness Survey to measure patient experiences of care in a standardized manner that can inform quality improvement, promote tracking of performance over time, and enable fair comparisons between programs, where appropriate.
The Serious Illness Survey is made available under a Creative Commons license at no cost. 1 Please attribute to the RAND Corporation. To request use of these survey materials for commercial use, please contact permissions@RAND.org.

How Can I Learn More About the RAND Serious Illness Survey?
This document contains recommended guidelines for implementing the survey, including • an overview of the survey instrument • recommended patient eligibility criteria and sampling procedures • recommended survey administration procedures, including recommended modes of survey administration • recommended approaches to scoring and analysis of survey-based quality measures.
A complete set of survey resources, including survey instruments and guidance on administration, sampling, and analysis, is available at www.rand.org/Serious-Illness-Survey. For more information, contact seriousillness@rand.org.

Survey Instrument
The Serious Illness Survey contains 36 items. It is divided into the following sections: Your Care from This Program, Help in the Last 3 Months, Planning for Your Care, Help for Your Family Members or Friends in the Last 3 Months, and About You. The appendix lists all questions in the survey. An abridged version of the survey is available for use if the cost or time of administering the full version would be prohibitive.
To promote comparability of survey responses over time and across users, we strongly recommend the following: • Users should administer the survey without alteration to question or response option wording, or to the order of questions or response options. • Any other questions added to the survey ("supplemental items") should appear directly before or after the About You section.
Limiting the number of supplemental items will help preserve survey response rates.

Patient Eligibility
The Serious Illness Survey was designed for administration to patients who are currently enrolled in home-based serious illness programs. It was not tested among patients who receive care primarily in clinic-based settings or while living in nursing homes or other long-term care facilities. Table 3.1 lists the recommended eligibility requirements for patients receiving this survey.

Sampling
We recommend that all patients be included in the sample frame unless there is evidence that a patient is ineligible according to the eligibility criteria outlined in Chapter 3. If information is missing on any variable that affects patient eligibility, we recommend that the patient be included in the sample frame.
Importantly, we do not recommend removing patients whose functional status or diagnosis makes them unlikely to be able to complete the survey on their own. The cover of the survey instrument provides instructions for proxy respondents, noting that survey questions refer to the patient's care and that proxy respondents should respond regarding the patient's experience rather than their own experience unless a question indicates otherwise. Proxy respondents should be individuals who are knowledgeable about the care that the patient received from the program. Employees of the program should not be proxy respondents or help patients complete the survey. The survey instrument collects information about whether and how a proxy respondent assisted in completing the survey; we recommend that information on proxy assistance be used in case-mix adjustment of quality measure scores (see Chapter 7 for recommended steps for scoring survey responses).
We recommend that the survey be administered to either (1) all patients of a given serious illness program (a census) or (2) a simple random sample of patients from the program.
Should users wish to make comparisons across programs, we recommend that they set a minimum sample size such that at least 100 completed surveys per program will be achieved. This will allow for sufficient precision to distinguish between programs' performances. However, even if this threshold is not met, survey responses are useful for understanding patient experiences within the program and for tracking changes within the program over time.

Survey Administration
The two recommended modes of survey administration for the Serious Illness Survey are mail only and mixed mode (mail with telephone follow-up). In the field test of the survey, the response rate was 30.4 percent in mail-only mode and 42.5 percent in mixed mode. 2 Table 5.1 provides an overview of the recommended protocol for mail-only administration. We recommend that survey administration begin with a prenotification letter. This letter should be personalized with patient and program name, inform the patient that they will soon be receiving a survey in the mail, explain the importance of the survey, and indicate how their feedback will be used. Use of a prenotification letter has been shown to increase overall survey response rates.

Mail-Only Mode
After approximately one week, a mail survey should be sent to the patient, accompanied by a cover letter and a business reply envelope. The cover letter should be personalized, again with patient and program name, and should include elements of informed consent, as well as language indicating that a family member or friend can assist with or complete the survey for the patient if needed. Cases in which a completed survey has been returned or a patient refuses to complete the survey should be considered finalized.
Finally, a second mail survey should be sent after approximately three weeks if the initial survey has not been returned or the case has not been otherwise finalized. This survey also should be accompanied by a cover letter and a business reply envelope.  Table 5.2 provides an overview of the recommended protocol for mixed-mode administration. The mixed-mode procedure consists of a prenotification letter, a mail survey, and follow-up by phone to nonresponders. We recommend beginning with the prenotification letter, followed by a mail survey one week later, and, after three additional weeks, calls to complete the survey by phone if the mail survey is not returned or the case has not been otherwise finalized (e.g., the patient refuses). All cover letters and introduction scripts should be personalized with patient and program names. The survey cover letter should include language indicating that a family member or friend can assist with or complete the survey for the patient if needed.
Telephone scripts should be used when the survey is administered by telephone. The scripts adapt the mail version of the survey for use in telephone interviewing and offer alternative wording to allow questions to be tailored to either a patient or a proxy respondent. In addition, the scripts include interviewer instructions and specifications for which text should be read aloud. Interviewers should be trained to administer the survey exactly as worded and to follow all instructions in the telephone scripts.
The telephone scripts may be programmed for use with computer-assisted telephone interviewing (CATI) software. They also can be administered without CATI software, if needed. In this instance, interviewers should be trained to carefully follow all skip patterns and conditional text. With mixedmode administration, approximately five to seven telephone follow-up attempts should be made over a period of three to four weeks. This allows sufficient opportunity to reach patients without placing an undue burden on them. Calls should be made at different times of the day and on different days of the week to increase the chances of reaching the patient.
As noted previously, we recommend that patients who are known to have died while receiving care from the program be excluded from the sample frame. However, if it is learned during mail or telephone data collection that a patient died between the date the sample is drawn and the date the survey is administered, the case should be marked as ineligible and no further contact should be made. The global measures are • overall rating of the program • willingness to recommend the program. Table 6.1 lists the questions included in each quality measure, noting their question numbers on the full version of the survey. How often people from the program spent enough time with the patient 4 How often people from the program explained things in a way the patient could understand 5 How often people from the program listened carefully to the patient 6 How often the patient felt that people from the program cared about them as a whole person 7 How often the patient felt heard and understood by people from the program Care coordination 8 How often people from the program seemed to know the important information about the patient's medical history 9 Whether someone from the program talked with the patient about the care or treatment they got from other doctors or health care providers 10 Whether someone from the program talked with the patient about all the medicines they are taking 11 Whether someone from the program talked with the patient about how to get help with everyday activities Whether the people from the program involved the patient's family members or friends in discussions about their health care as much as the patient wanted a 21 Whether the patient's family members or friends got as much emotional support as they wanted from the program a Global measures Overall rating of the program 25 Overall rating of program care from 0 to 10 Willingness to recommend the program 26 Whether the patient would recommend the program to their friends and family a Screening question(s) determines whether this evaluative survey question is applicable to the respondent.

Steps for Scoring Survey Responses
Defining the Denominator Many questions on the survey are applicable only to some respondents (e.g., only to those respondents who experienced pain or contacted the program between visits). Some questions have response options that allow the respondent to indicate that the question is not applicable. For example, question 14 on the full survey asks whether the respondent got as much help as they wanted for pain. The tailored nonapplicable response is "I did not want help for my pain." Tailored nonapplicable responses should be set to missing and excluded from the denominator when calculating scores. In this manner, those who indicate that the question is not relevant to them are considered nonrespondents to the question for the purpose of calculating scores. Finally, if a survey respondent does not respond to a question for which they are eligible, a score is not calculated for that respondent for that question. All responses to questions for which a respondent was eligible are used in the score calculation.

Calculating Top-Box Scores
To calculate scores for individual survey questions or measures based on Serious Illness Survey responses (i.e., above and beyond calculating and reporting the frequency distribution of responses), we recommend the use of "top-box" scores to promote interpretability by lay users.
For all questions, the "top-box" score for each response is calculated as a "100" if the most-positive response category(ies) for that question is selected. Otherwise, it is calculated as a "0." Different questions have different response options. Table 7.1 displays the different response scales and how each response is categorized as a "top-box" score.

Adjustment for Case Mix and Mode
If survey data will be used to calculate and compare the performance of different serious illness programs or to compare one program's performance over time, we strongly recommend adjusting scores for case mix (patient characteristics and use of proxy response) and mode of survey administration. Table 7.2 displays the case-mix variables recommended for adjustment with suggested categories to use in adjustment models. All variables are categorical with the exception of response percentile.

Continuous
Primary diagnosis Cancer, Alzheimer's and other dementias, other Proxy respondent Proxy answered, proxy helped in some other way, no proxy Functional status Not able to get out of bed, able to get out of bed but not house, able to leave house Self-rated physical health Excellent, very good, good, fair, poor Self-rated mental health Excellent, very good, good, fair, poor a Response percentile is calculated as the rank-ordered number of days between the first day of survey administration and survey response for respondents in each program and mode, relative to all eligible patients within each program and mode, scaled from 0 to 1.
In addition to adjusting for case-mix variables, we recommend adjusting for mode of survey administration (e.g., mail-only mode versus mixed mode) when different modes of data collection are used.

Calculating Adjusted Composite and Global Measure Scores for Programs
To calculate top-box scores for the measures listed in Table 6.1, 1. Calculate adjusted top-box scores for each survey item in Table 6.1.
a. Fit a linear model for each survey item that includes as covariates i. each case-mix adjustor ii. mode of survey administration iii. fixed effects for entities to be compared (e.g., program fixed effects when comparing programs). b. For each item, use the fitted linear model from (a) to generate adjusted scores for each program (i.e., generate predicted scores assuming that each program has the same case mix). c. For composite measures, average the adjusted top-box scores for the questions that compose the measure.

Appendix. Question Text in the Full Version of the Serious Illness Survey for Home-Based Programs
In Table A.1, we show the questions and response options for the Serious Illness Survey for Home-Based Programs. In the last 3 months, how many times did someone from this program visit you at home? "None," "1 to 2 times," "3 to 4 times," "5 to 6 times," "7 or more times" 2 In the last 3 months, how many times did you talk to someone from this program on the phone or by video for questions or help with your care?
"None," "1 to 2 times," "3 to 4 times," "5 to 6 times," "7 or more times" 3 If you only see or talk to one person from this program, think about that person when the question says people from this program.
In the last 3 months, how often did people from this program spend enough time with you?
"Never," "Sometimes," "Usually," "Always" 4 In the last 3 months, how often did people from this program explain things to you in a way you could understand?
"Never," "Sometimes," "Usually," "Always" 5 In the last 3 months, how often did people from this program listen carefully to you? "Never," "Sometimes," "Usually," "Always" 6 In the last 3 months, how often did you feel that people from this program cared about you as a whole person?
"Never," "Sometimes," "Usually," "Always" 7 In the last 3 months, how often did you feel heard and understood by people from this program? "Never," "Sometimes," "Usually," "Always" 8 In the last 3 months, how often did people from this program seem to know the important information about your medical history? "Never," "Sometimes," "Usually," "Always" 9 In the last 3 months, did someone from this program talk with you about the care or treatment you get from your other doctors or health care providers?
"Yes, definitely," "Yes, somewhat," "No" 10 In the last 3 months, did someone from this program talk with you about all the medicines you are taking?
"Yes, definitely," "Yes, somewhat," "No," "I do not take any medicines" 11 Everyday activities include things like getting ready in the morning, getting meals, or going places in your community.
In the last 3 months, did someone from this program talk with you about how to get help with everyday activities? "Yes, definitely," "Yes, somewhat," "No," "I did not want help from this program with everyday activities"

Question Number Question Text Response Options 12
In the last 3 months, when you contacted this program between visits for questions or help with your care, did you get the help you needed?
"Yes, definitely," "Yes, somewhat," "No," "I did not contact this program between visits for questions or help with my care"

13
In the last 3 months, did you have any pain? "Yes," "No"

14
In the last 3 months, did you get as much help as you wanted for your pain?

15
In the last 3 months, did you have any trouble breathing? "Yes," "No"

16
In the last 3 months, did you get as much help as you wanted for your breathing?
"Yes, definitely," "Yes, somewhat," "No," "I did not want help for my breathing" 17 In the last 3 months, did you have any feelings of anxiety or sadness?

18
In the last 3 months, did you get as much help as you wanted for your feelings of anxiety or sadness?
"Yes, definitely," "Yes, somewhat," "No," "I did not want help for my anxiety or sadness"

19
In the last 3 months, did you have family members or friends involved in your care?
"Yes," "No" 20 In the last 3 months, did the people from the program involve your family members or friends in discussions about your health care as much as you wanted?

21
In the last 3 months, did your family members or friends get as much emotional support as they wanted from this program?
"Yes, definitely," "Yes, somewhat," "No," "My family members or friends did not want emotional support from this program"

22
Did someone from this program ever talk with you about what you should do during a health emergency?
"Yes, definitely," "Yes, somewhat," "No" 23 Did someone from this program ever talk with you about what is important in your life? "Yes, definitely," "Yes, somewhat," "No" 24 Did someone from this program ever talk with you about what your health care options would be if you got sicker? "Yes, definitely," "Yes, somewhat," "No" 25 Using any number from 0 to 10, where 0 is the worst care possible and 10 is the best care possible, what number would you use to rate your care from this program? 0-10 26 Would you recommend this program to your friends and family? "Definitely no," "Probably no," "Probably yes," "Definitely yes"

27
Are you usually able to leave your home or building to go outside, even if you use a cane, walker, or other people for support?

28
Are you usually able to get in and out of bed? "Yes, by myself," "Yes, with help," "No"