Development and Validation of the Emergency Department Geriatric Readmission Assessment at Yale (ED GRAY): Part 1, Fundamental Measurement

Objective: Our primary objective was to develop a reliable, valid, and efficient screening tool that measures recovery disability among geriatric patients for the Department of Emergency Medicine (ED) Geriatric Readmission Assessments (GRAY). Methods: We conducted a retrospective medical chart review and prospective data analysis of geriatric patients admitted to hospital from the emergency department that were discharged, admitted, or died at a single academic urban university-affiliated hospital to identify items for ED GRAY. Rasch analysis was then used to reduce items and construct an interval/ratio scale of physical and cognitive disabilities. Patients consisted of a cohort of consenting, non-critically ill, English-speaking adults older than 65 years and receiving care in the ED to reduce the number of items. Results: Rasch analyses resulted in infit and outfit statistics that eliminated redundant items or items that did not fit a unidimensional disability construct. From the 158 original items, sixteen items comprise the ED GRAY global health questionnaire, representing five sub-constructs: physical disability, cognitive disability, stress, depression, and isolation. All infit and outfit statistics for the global recovery disability score ranging from 1 (least healthy) to 5 (most healthy) were consistent with forming a unidimensional scale. Conclusions: Our study resulted in an objective measurement tool of physical and cognitive disability using Rasch analyses. This screening tool allows healthcare providers the ability to screen older ED patients on a continuum of risk, with high-risk patients being most likely to benefit from in-depth evaluation—e.g., comprehensive geriatric assessment—followed by intervention (when necessary).


Background
Although the immediate concern for all emergency department (ED) patients is the appropriate and timely treatment of acute medical problems, older patients must also be evaluated for functional abilities and cognitive status and other factors that aid or impede recovery [1,2]. This assessment can dictate whether an older adult is admitted as an inpatient, transferred to an alternative care setting, or discharged home [3]. Admitting older adults to the hospital ED, coordinating a successful course of care, and arranging for appropriate discharge placement is complicated and resource intensive [4]. Therefore, ED providers need tools to assist in evaluating a 23.3 percent patient's status for placement that are both accurate and efficient.
Physical [5,6] and cognitive disabilities [7] are certainly part of a person's placement status and have been operationalized and measured in multiple studies through the process of specification and testing of sets of indicators. The accuracy and efficiency of disability measurement in other settings does not ensure their appropriateness for the hospital ED because of differences in patient characteristics, provider characteristics, and administration context, to name a few. Furthermore, the ED often requires quick assessments. Thus, we sought to develop a reliable, valid, and efficient measure of recovery disability among older ED patients.

Importance
There is a tremendous need for ED providers to conduct brief accurate recovery assessments of older patients, foremostly due to the sheer number of older adults who pass through the ED doors each year. In 2011, an estimated 23.3% of US adults aged 65 years and over made one or more ED visits within the past 12 months; and an estimated 7.8 percent made two or more pastyear visits [8]. When applied to the total number of older adults in the US, those figures represent a remarkable number of older ED patients in total. Unfortunately, conditions associated with age such as cognitive dysfunction and frailty are prevalent and often unrecognized [9]. The number of older ED patients presenting with such conditions can be expected to increase as the US population ages [10-14].
These ED visits allow providers to intervene and minimize the length of time in a dependent state due to recovery disability and its sequelae, thereby potentially preventing hospital admissions, ED revisits, and premature deaths. Important benefits have been reported in intervention studies of comprehensive geriatric assessments in the ED setting [2]. In summary, there is growing recognition that improved ED care for older adults will require providers to consider the influence of factors such as functional disability [15,16]. However, ED providers' ability to conduct functional assessments is constrained by time to screen, availability of accurate, efficient toolsinnovations that have eluded the practice of emergency medicine to date.

Goals of This Investigation
To this end, the overarching aim of this two-part study was to develop and validate an ED recovery disability diagnostic tool. The objective of Part 1 was to develop the ED GRAY (Geriatric Readmission Assessment at Yale Department of Emergency Medicine). The primary objective of Part 2 (see companion submission) was to prospectively validate the ED GRAY by demonstrating its ability to predict or prognosis of return ED visits, hospitalization, or death.

Study Design and Setting
This was a mixed methods study involving both retrospective and prospective data from a large, urban teaching hospital. In the retrospective component, we used existing ED patient records and disability assessments to derive the new disability scale comprised of extant items, revised items from previous measures, and new items. Using baseline data from a prospectively assembled cohort, the scale was then tested for measurement properties and revised as necessary.
Yale Department of Emergency Medicine is an urban, tertiary care center that is designated as a Level 1 Trauma Center. The hospital is located in a mid-sized northeastern city of approximately 130,000 residents. The hospital ED is the busiest in the state with over 81,000 adult visits annually. Among the adult ED patients evaluated annually, approximately 54% are female; 28% are Black; 18% are Hispanic; and 40% receive Medicaid.

Selection of Participants
Patients in the retrospective cohort were randomly selected from patients 65 and over admitted to the hospital from the emergency department during the most recent one year period.
Participants in the prospective cohort were selected by sequential sampling from all patients presenting between the hours of 7 am and 11 pm from February 1-March 22, 2012. Trained Research Associates (RAs) approached ED patients and assessed their eligibility using a standardized screening form. Eligible patients were community-dwelling adults aged 65 years and older requiring a disability assessment. Patients excluded were under 65 years old; not having Medicare; presenting from an extended care facility; having 24-hour home care; being too ill to complete an interview; presenting with acute psychosis, suicidal or homicidal ideation; being non-English speaking; or concurrently enrolled in another research study. To enrich the sample, a group of patients with low acuity (ESI level-4 and level-5 patients), altered mental status, and/or dementia was enrolled from April 27-July 18, 2012. Sample size (n=250) was chosen based on the ability to provide adequate power for the assessment of predictive validity (see companion submission).
Written informed consent was obtained from all participants or their legally authorized representative. The Institutional Review Board of the Yale School of Medicine approved the study.

Methods and Measurements
As part of development of the ED GRAY, a thorough literature review was conducted and many potential questions were examined and discussed by an expert multidisciplinary panel from emergency medicine, geriatrics, statistics, psychometrics, sociology, psychology, engineering, informatics, and healthcare coordinator nurses, methods to determine what patient characteristics should be tested. Existing measures of cognitive and physical disabilities were used along with recommended items from the qualitative focus group of interdisciplinary healthcare providers. Next, we conducted a retrospective chart review to select and analyze potential items for the disability measure. Retrospective review was conducted by trained RAs using a standardized electronic data abstraction form. Medical record abstractors' training and assessment followed the recommendations of Reisch, Fosse and Beverly.
[17] Once trained, RAs obtained data using the electronic medical record system with direct entry into the study database. Variables were developed from pilot research, review of the literature, clinical experience, and collaboration of co-investigators with clinical experience in disabilities. Variables were derived from the past medical/family/social history, history of present illness, and disability diagnoses. The remainder of retrospective data was collected through an extract from the medical record system that was then combined using matching visit ID numbers with the abstracted data.
Based on the retrospective Rasch modeling, a revised and expanded set of items likely to predict cognitive and physical disabilities was created and a prospective cohort was enrolled to assess the revised measure. Primary screening, enrollment, and interviews with the participants were conducted by RAs trained by expert researchers and physicians. The RAs conducted daily surveillance of ED patients and ran initial screens for patients requiring a disability assessment. Data were collected using a standardized electronic form on an iPad® using FileMaker® and stored on the secured server. After written consent from the patient and/or guardian, baseline data collection included demographic characteristics, clinical features, and social support information. Variables included functional disabilities, cognitive disabilities, psychological problems, living circumstance and health status. The final baseline interview assessed 44 items and lasted approximately 90-120 minutes.

Outcomes
The primary objective of Part I was to develop a scale where patients were placed on a continuum of global disabilities from none to complete. These were community-dwelling Medicare patients presenting to the hospital ED. The performance of the ED GRAY was assessed based on its construct validity as evidenced by fit statistics derived from Rasch modeling.

Analysis
Rasch analysis was performed on both the retrospective and prospective data. Rasch models-named after the Danish mathematician Georg Rasch [18]-provide a means for approximating fundamental measurement from discrete observations. Briefly, to approximate fundamental measurement it is necessary to have two classes of entities involved: one class is that of items and the other is that of persons responding to items [18]. Rasch models are based on the idea that items focus on only one attribute of persons at a time (i.e., unidimensionality) [18,19]. Moreover, in order to approximate fundamental measurement, equality of intervals is accomplished through log transformations of raw data odds [19].
Rasch analysis was performed with Winsteps® measurement software (Winsteps® Rasch Measurement). Item fit statistics were estimated to determine how well the empirical data met the requirements of the Rasch model [19]. In order to aid with interpretability, Rasch-derived estimates were converted to an integer scale from 1 to 5, with "1" being the most disabled and "5" being the least disabled.

Characteristics of Study Subjects
During the retrospective chart review phase in 2011, 400 patient records were randomly selected from a total of 9,195 patients aged 65 and older and admitted to the hospital from the emergency department. The mean age was 77.9 years with 57.3% female patients and 82.8% White patients (Table 1).
In the prospective cohort (Feb 1-Mar 22, 2012), 896 patients were screened-of which 424 were deemed eligible and 207 consented and enrolled. A further 43 patients were enrolled (Apr 27-Jul 18, 2012) who met all criteria as well as the additional criteria of being either low acuity or presenting with altered mental status or documented dementia. The mean age of the prospective cohort was 77.3 years with 58.4% of participants being female and 74.3% White.

Main Results
Data derived from the retrospective phase were added to data collected from the prospective cohort data. Both retrospective and prospective datasets were analyzed using Rasch modeling techniques. Items were selected as having a good fit based on mean-square residual summary statistics (infit and outfit) greater than  Each of the five sub-constructs was assessed using between three and six questions. All infit and outfit statistics were within range, with the exception of one item used for assessment of cognitive disability (outfit mean-square = 2.35). Rasch person-scores were transformed to an integer scale ranging from 1 (most disabled) to 5 (least disabled). Table 3 contains the distribution of integer values of person scores for the five disability sub-constructs. Subsequent Rasch analysis was performed on the sub-construct scores to create a global disability score (Table 4). All infit and outfit statistics for the global recovery disability score were consistent with forming a unidimensional scale. The algorithms for scoring the 5 sub-contructs and global disability are shown in Table 4.

DISCUSSION
Avoidable ED visits and hospital admissions of older adults are significant healthcare problems and discharged patients with low functional status and no referral return to the ED repeatedly, are admitted at higher rates or die prematurely [21][22][23]. In this study, a measure of disability was developed based on a scoring system derived from older ED patients' selfreported answers to a small set of screening questions (16 items). The resultant, simple output score can be quickly interpreted to initially assess disability in geriatric patients and orient emergency providers to healthcare issues requiring resolution. The ED GRAY tool is time effective to assess disability and the response burden on older patients has dramatically reduced the golden standard of 158 questions to only 16 questions. Screening Tool (TRST), to be used by ED nursing personnel in the triage setting to determine 'at-risk' older people-defined as having cognitive impairment or expressing ≥2 of 5 remaining risk factors (e.g., living alone or no caregiver willing or able to provide assistance). However, the extent to which the unit amount remains constant with these screening tools cannot be assumed, whereas objective measurement of ED GRAY was achieved through Rasch analyses.

LIMITATIONS
It is possible that the retrospective cohort may have evidenced more disability than the prospective cohort given they were admitted, however, it provided a list of variables to be tested in a prospective rasch disability diagnostic screening test. Additionally, ED patients from the prospective phase were dissimilar to their retrospective counterparts in terms of race and insurance status (Table 1). However, it is important to realize that the role of the retrospective phase was simply to provide a first attempt at item development. What matters more is the performance of the ED GRAY with the prospective participants.
The performance of the ED GRAY was assessed based on fit statistics derived from Rasch modeling. All of the infit and outfit statistics were within range, with the exception of one item used for assessment of the cognitive disability subconstruct: "Is the patient verbally impaired?" Unlike traditional instrument development where poor-performing items are routinely thrown out, misfit identified via Rasch modeling invites the researcher to "enter into a dialectical process in which theory informs practice via measurement and practice informs theory via measurement" (p. xxi) [19].
Finally, the ED GRAY is based on self-reported data from ED patients and/or their guardians. One of the oft-cited barriers associated with screening instruments aimed at identifying highrisk populations of older patients is the requirement for patient self-reporting of information (e.g., low response rates, increased time, increased cost, etc) or a proxy who reports for them. Collection of functional-status data is dependent on patient self-report, proxy report, or clinician-administered performance testing. That is, functional-status measures are not easily extractible from administrative data. Given that such measures are known to be independently associated with hospitalization and ED visits, reliance on patient self-report may be unavoidable (at least for the time being) [15].

CONCLUSION
The overarching aim of this two-part study was to develop and validate ED GRAY. The objective of Part 1 was to develop the ED GRAY tool. While this paper does not address the predictive validity of the ED GRAY instrument (see companion submission), this tool has the potential to allow older ED patients to be classified on a continuum of risk, with high-risk patients being most likely to benefit from in-depth evaluation-e.g., comprehensive geriatric assessment-followed by intervention (when necessary).

ETHICAL APPROVAL
All authors hereby declare that all experiments have been examined and approved by the appropriate ethics committee and have therefore been performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki.

6.
McDowell I, Newell C. Physical disability and handicap.

Scoring Algorithms for Disability Measurement Physical Disability
If question 1 is "Only Occasionally" or "Rarely or Never" assign a score of 5. Otherwise add the scores for questions 1a, 1b, 1c, 1d, 1e, and 1f and use scoring system:

Cognitive Impairment
If question 2 is "Only Occasionally" or "Rarely or Never" assign a score of 5. Otherwise add the scores for questions 2a, 2b, and 2c and use scoring system:

Isolation
If question 4 is "Only Occasionally" or "Rarely or Never" assign a score of 5. Otherwise add the scores for questions 4a, 4b, and 4c and use scoring system: