Development of an instrument to assess social functioning in dementia: The Social Functioning in Dementia scale (SF-DEM)

Introduction Social functioning is a core domain in the life of people with dementia, but there is no accepted instrument to measure it. We aimed to develop the Social Functioning in Dementia (SF-DEM) scale and test its psychometric properties for assessing social function in people with dementia. Methods We interviewed people with mild dementia and family caregivers to develop patient and caregiver-rated SF-DEM versions and refined them through interviews with health care professionals. We tested its psychometric properties in 30 dyads of people with dementia and family caregivers. Results Both SF-DEM versions had content validity and demonstrated concurrent validity against a single item rating overall social functioning (patient rated r = 0.42, 95% CI [0.07–0.68]; caregiver rated r = 0.59, 95% CI [0.29–0.78]). All participants found it acceptable. Analyses showed reliability (test–retest, inter-rater, internal consistency) and indications of responsiveness to change. Discussion SF-DEM shows promise as a valid, reliable, acceptable measure of social functioning in dementia.


Introduction
Dementia diagnostic criteria specify impairment, in activities of daily living or social function, must accompany cognitive decline [1,2]. Changes in social function, "how individuals associate and interact, both in society at large and their own personal environment" [3], such as loss of interest in previously valued hobbies or changes within close relationships, are distressing to people with dementia [4,5] and their families [6,7], especially when the person with dementia lacks awareness of social changes [8]. Changes in social behavior occur in the early stages of a number of dementia subtypes [9] including Alzheimer's disease [10] and frontotemporal dementia [11]. These changes may be caused by emotion recognition [12] or theory of mind [13] deficits, or disinhibition [14] related to amygdala and frontal cortex network disruption [15]. Lower premorbid social functioning has been reported to increase dementia risk [16][17][18] and its progression [19]. Social function is therefore central to the diagnosis of dementia and is a core domain when considering etiology and progression and evaluating the effects of interventions in dementia.
Although measures of general function [20] and quality of life [21] include individual questions about social function, there is no validated instrument available to assess social functioning in people with dementia. We therefore aimed to develop a psychometrically sound and acceptable interviewer-administered measure of social functioning in dementia, the Social Functioning in Dementia (SF-DEM) scale, to be completed in a face-to-face interview with the person with dementia (self-report) or their family caregiver (proxy report).

Method
Westminster NRES Committee (15/LO/0105) gave ethical approval. We used gold-standard methodology [22] to develop and test the instrument in an iterative process ( Fig. 1) in three phases: (1) instrument development-generation of domains and candidate questionnaire items through qualitative interviews with people with dementia and their family caregivers and a structured literature review; (2) expert interviews-qualitative interviews with dementia experts about the test structure and content to refine the draft assessment tools and test content validity; (3) psychometric testing-in structured interviews with people with dementia and their caregivers.

Setting
We recruited participants from two community-based memory clinics in London, UK.

Participants
We purposively sampled dyads of people with dementia and their family caregiver for a range of demographic and clinical characteristics to cover varied experiences of social changes in dementia. We stopped interviewing when no new content arose (theoretical saturation).
We included English speakers with dementia of any subtype (diagnosed clinically by consultant psychiatrists, then validated by A.S. against Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V) criteria for major neurocognitive disorder [1]), of mild severity (Mini-Mental State Examination [MMSE] [23] score 20). We excluded those with severe physical or other mental illness limiting their participation in the interviews or those who lacked capacity to give informed consent.
We included current, English-speaking, unpaid, main caregivers, over 18 years old, in contact with a person with dementia at least weekly. We excluded caregivers with severe physical or mental illness which limited their participation in the interviews or without capacity to give informed consent.

Procedure
A health care professional approached potential participants during clinical contact. Interested participants, after being provided with an information sheet, were approached for informed written consent. One researcher (A.S.)

Analysis
We transcribed recordings and checked transcriptions for accuracy. Two researchers (A.S. and D.S.) analyzed all transcripts to identify a conceptual framework of important social functions, guiding the selection of candidate items for SF-DEM.

Setting and participants
We purposively sampled health care professionals from participating memory clinics to achieve a range of characteristics: sex, experience in clinical and research settings, ethnicity, and specialty. We conducted one focus group and two individual interviews at their workplaces.

Procedure
We developed an interview guide which tested content validity by inquiring whether our draft instruments balanced all important facets of social functioning and assessed clarity, acceptability, and ease of use.

Setting and participants
The setting and inclusion and exclusion criteria for participants were the same as for the qualitative interviews. Consenting participants from those interviews took part in psychometric testing, and we recruited additional dyads.

Procedures
A.S. conducted all semistructured interviews with people with dementia and their caregivers, at their homes at baseline; 4 weeks later to assess test-retest reliability; and 6-8 months later to assess responsiveness to change. Another researcher, D.S., completed SF-DEM for 18 dyads based on audio-recorded interviews, to assess interrater reliability between the two researchers.

Measures
Baseline and 4-week interviews for people with dementia and caregivers included the following: Demographic characteristics: age, sex, ethnicity, marital status, education, employment status, and living arrangements (baseline only). SF-DEM instrument, our newly devised 20-item interviewer-administered instrument by self-report and proxy report. The question "How acceptable did you find this questionnaire?" with ratings either: Very acceptable; Acceptable; Unacceptable; Very unacceptable.
At baseline interviews for people with dementia: We chose the social domain question from Health Status Questionnaire (HSQ)-12 [26], a validated interview as there was neither a "gold-standard" test of social function, nor a social function measure for dementia against which to test our instrument's validity.
-"During the past 4 weeks, to what extent has your memory problem interfered with your normal social activities with family, friends, neighbors, or groups?" Rated as: Not at all; Slightly; Moderately; Quite a bit; or Extremely.

At baseline interview for caregivers:
We chose three social questions from a validated measure of quality of life, QOL-AD [24] to test our tools' construct validity.
-"How is [your relative]'s relationship with their family members?" -"How is [your relative]'s relationship with the person closest to them [could be the caregiver]?" -"How is [your relative]'s relationship with their friends?" Each rated as Poor; Fair; Good; or Excellent, and these ratings were scored as 1 to 4, generating a total score out of 12.
At 6-to 8-month follow-up interviews for people with dementia and caregivers: Changes to living arrangements or bereavement of patient or family caregiver SF-DEM instruments Three social questions from QOL-AD. MMSE (people with dementia only)

Sample size calculation
Our a priori sample size calculation assumed nonparametrically distributed data and provided 80% power and a significance level of 5%. We calculated we would require 28 participants to find a correlation of .0.3 between our instrument and the social function domain of HSQ-12, assuming the true correlation to be 0.7. A sample size of 24 participants would detect a correlation of .0.7 between repeated interviews and 14 participants could detect correlation of .0.6 between raters, assuming the true correlation to be 0.9.

Analysis
We compared results from patient-rated and caregiverrated SF-DEM using paired t-tests. We assessed overall internal consistency using Cronbach's a and item-total and item-item reliability using Spearman's rank coefficient as this tested ordinal data. We tested interrater and test-retest  reliability for total SF-DEM scores using the intraclass correlation coefficient (ICC) [27] using the (2,1) model for interrater reliability and (1,1) model [28] for test-retest reliability, and interrater and test-retest agreement for individual questions using Cohen's k; we used quadratic weighting for k as our ordinal rating scale had nonlinear ratings [29]. For validity testing, we used Spearman's rank coefficient to assess correlation of SF-DEM measures and ordinal data from HSQ-12 or QOL-AD questions and ICC for agreement between patient and caregiver. To test responsiveness to change, we used linear regression to assess the association between change in SF-DEM and an ordinal rating by the participant of the overall social change during the preceding year [30]. We used SPSS version 20 for all descriptive and analytical statistics apart from weighted-k, which we calculated using Stata version 12. Table 1 shows the 18 participants' demographic and clinical characteristics. The mean age of the nine participants with dementia was 79 years (standard deviation [SD] 8) and five (56%) were male. The mean age of the nine caregivers was 68 years (SD 11) and two (22%) were male. We identified the following themes from our analysis and devised candidate items for the questionnaire based on the phrases our participants used and structured around the domains of our conceptual framework (for detail, see Appendix A).

Instrument development
Participants told us about activities they had previously enjoyed, but where engagement had declined when participants developed dementia; including engaging socially with family and friends at home or at others' home and speaking to people on the phone, email or social media, and hobbies enjoyed by people with dementia currently, such as going to caf es or social clubs. Caregivers told us about changes in the person with dementia which impeded social function, such as difficulty making or following conversation, especially in larger groups; increased critical comments; irritability; or loss of interest. People with dementia would frequently deny these changes or attribute them to age or physical ill health.

Professional experts
We interviewed three consultant psychiatrists, two psychologists, four nurses, and two support workers in a focus group or individual interviews. They commented on the range of activities covered in our instruments and suggested additional items and changes to the format of the instruments. We changed our instruments based on these interviews (Appendix A).

Additional items
Professionals suggested adding summary questions asking whether the patient's social life was better or worse than 1 year before and whether they wanted to make changes to their social function. These could make the instruments a stimulus for conversation about past changes, and possible future lifestyle changes.

Suggested changes to instrument format
The changes suggested were as follows: A shorter questionnaire to increase acceptability. Interviewer-administered questionnaire. Increase the time period covered from 2 to 4 weeks to permit a greater range of social activities. A prompt card for the four-item response options to aid understanding.
An accompanying manual with detailed information on administering and scoring the instrument.

Content validity
Participants told us the items in our instrument corresponded with the range of social activities and changes in social functioning that they see in people with dementia.

Format of SF-DEM instruments
The SF-DEM used in the testing phase (accessible at www. ucl.ac.uk/psychiatry/SFDEM) consists of 17 core questions scored, from 0 to 3, and three unscored summary questions. There are separate self-and caregiver-report forms using the same items but rephrased in the caregiver version for proxy response. Higher scores represent better social functioning; the maximum possible score is 51. In section 1, 11 questions cover engagement with social contacts and important social activities and, in section 2, 6 questions cover difficulties in social relationships and these are reverse scored. Table 1 summarizes demographic and clinical characteristics of the 30 recruited dyads. People with dementia had a mean age of 80 years (SD 8) and half were women. The caregivers' mean age was 68 years (SD 13) and 80% were women. Participants came from a variety of ethnic backgrounds, educational levels, and previous employment. Half of dyads were spouses. Nearly three-quarters of participants had Alzheimer's disease. Table 2 details the SF-DEM questions and participants' mean and range of scores. People with dementia used a full range of scores in nine questions and caregivers in 14 questions. The rating by people with dementia was higher than caregivers' (mean difference 4.1, 95% CI [1.9-6.2], P ,.0005) because of difference in rating social relationship  difficulties (section 2 mean difference 3.2, 95% CI [1.5-4.8], P , .0005). There was no significant difference between patient and caregiver rating for frequency of social contact and activity.

Acceptability and time burden
We first considered the presence of missing data and floor/ceiling effects for summary scores. There was 100% completion of all items in both forms, and no participants refused or were unable to complete the interview. There was no floor or ceiling effect as no one scored the minimum or maximum score on SF-DEM. All participants rated the instrument as acceptable (62%) or very acceptable (38%). Administration took a mean of 13 minutes (SD 5) for people with dementia and 11 minutes (SD 4) for caregivers.

Internal consistency
Consistency for the patient-rated (a 5 0.62, 95% CI [0.40-0.84]) and caregiver-rated instruments (a 5 0.64, 95% CI [0.44-0.85]) was at an acceptable level for early-phase research [31]. Four items (4,7,8,10) in the patient-rated instrument and two items (2,15) in the caregiver-rated instrument had low item-total reliability ( Table 3). We considered removing them but judged this would reduce the instruments' content validity. Item-item reliability was ,0.5 between most items, although higher in four instances in each instrument (Appendix B). We considered whether any of these items could be eliminated but this would reduce the overall internal consistency. All items in SF-DEM were therefore retained.

Interrater reliability
Interrater correlation between the two researchers was very high for overall scores on the patient-rated SF-DEM (ICC 5 0.99, 95% CI [0.99-1.00]) and the caregiver-rated SF-DEM (ICC 5 0.99, 95% CI [0.99-1.00]) and exceeded suggested acceptability criteria [32]. Interrater agreement for individual items was good for one question and very good for all other questions (Table 3).
There was significant moderate correlation of both SF-DEM measures with the summary SF-DEM question about overall impression of current social functioning (patient rated r 5 0.44, 95% CI [0.07, 0.68], P 5 .01; caregiver rated r 5 0.60, 95% CI [0.29, 0.78], P 5 .001). We assessed whether validity was affected by the patient's level of cognitive impairment, by dividing the cohort according to median MMSE score. Correlation between patient-rated SF-DEM total score and their summary impression of social functioning was very strong in the 18 participants whose MMSE 26 (r 5 0.82, 95% CI [0.17, 0.83], P , .0005) but we found no significant correlation for the 12 participants with MMSE 25 (r 5 20.44, 95% CI [20.79, 0.24], P 5 .15).

Responsiveness
We repeated testing with SF-DEM, on average, 7.2 months (SD 15, range 182-251 days) later with 29 patients and 27 caregivers (Table 4). Patients' mean MMSE score reduced by 0.3 points (SD 2.5, range 26 to 15) and only one participant had experienced a major change in the intervening period; a family member had moved into their home.
We found a range of SF-DEM change. The mean patient-rated SF-DEM score decreased by 1.2 points (SD 3.1, range from 6 to 27), whereas the caregiver-rated SF-DEM increased by 0.1 points on average (SD 3.9, range from 12 to 28). For details of individual SF-DEM domains' change, see Appendix C. We found preliminary evidence for the participants' overall impression of social change during the past year predicting SF-DEM score. Patient-rated SF-DEM score increased by 1.3 points (95% CI [20.3, 2.9], P 5 .10) and caregiver-rated SF-DEM score increased by 1.4 points (95% CI [20.1, 2.9], P 5 .06) for each point on the five-point ordinal scale of social change. Change in MMSE score for the person with dementia was not associated with SF-DEM change; one-point decline on MMSE during the follow-up period was associated with a 0.24-point improvement on the patient-rated scale (P 5 .20) and 0.04 decline on the caregiver-rated scale (P 5 .84).

Discussion
We have developed the SF-DEM, which measures social functioning of people with dementia, based on interview with them and/or their family caregiver. Both self-and caregiver-report instruments are acceptable, internally consistent and have interrater and test-retest reliability and content, concurrent and convergent validity in people with mild dementia.
Total SF-DEM score and overall impression of current "social life" were rated higher by people with dementia than caregivers (63% of patients vs. 43% of caregivers rated social function as good or excellent). This difference was accounted for by items which required judgment and abstract understanding (section 2, cognitive and psychological barriers to social engagement). Lack of insight is common in dementia [33] and increases as the disease develops [34]. People with dementia rate their quality of life higher than observers [35] possibly partly because they underestimate social changes and the cognitive deficits underlying these changes. However, this fits with the "disability paradox" [36] where differences may be a true reflection of different appraisal by the patient and caregiver, rather than a function of error due to cognitive impairment. This underlines the additional value of caregiver rating and need for further psychometric evaluation in more severe dementia.
Testing the SF-DEM's validity is challenging as there is no psychometrically acceptable existing measure of social function with which to compare this new instrument and researchers cannot feasibly directly measure participants' social function (although families can). There was significant correlation between the SF-DEM and a summary item and social function questions from the QOL-AD. We found no correlation for patients when compared to a question from the HSQ-12, a general social functioning measure, possibly because it required social difficulties to be attributed to memory deficits, and therefore needed insight into cognitive and social changes. We found statistically significant moderate correlation for both patients and caregivers at follow-up, reflecting the instruments' construct validity. The moderate correlation between the patient-or caregiver-rated overall score of SF-DEM and their overall impression of the person's "social life" indicates SF-DEM's concurrent validity, and the agreement between patient and caregiver ratings supports the validity of our instrument. We did not find any association in this small sample between change in MMSE score and change in social functioning as measured by SF-DEM, which may reflect that multiple different factors, rather than the progression of cognitive impairment, contribute to change in social functioning in people with mild dementia. This lack of association with increasing cognitive impairment is also seen in health-related quality of life [37], another multidimensional construct. In addition, there was only a decrease of 0.3 in the mean MMSE score in this study so we had very limited power to detect change.

Strengths and limitations
The main strength of this study is our use of mixedmethodology, meeting quality criteria [38] for the development and testing of SF-DEM. We powered the study for baseline validation but not for ability to detect change. Study participants covered a range of inner city and suburban-dwelling older people with dementia who had contact with memory services and included participants from different settings, ethnic, and social backgrounds. Further work is needed to evaluate SF-DEM's psychometric properties in other populations, such as rural areas, and other cultures where there may be different influences on social function.
One limitation of this study is the restriction of the instrument's development to those with mild dementia which limits the current use of the SF-DEM to this group. Further work is needed to test SF-DEM's psychometric performance in those with moderate and severe dementia, to determine if it accurately measures social function later in the disease process, and also in a cognitively healthy older population, to assess whether early social changes are accurately identified. Another limitation is the relatively small size of the test population meaning that evaluation of the effect of sociodemographic and clinical characteristics on performance on the SF-DEM was not possible. The sample size also precluded factor analysis. The use of the SF-DEM in larger populations would enable the investigation of the effects of variables such as subtype of dementia; insight; affected cognitive domains; premorbid personality; comorbid physical and mental illness; and external factors such as level of support, caregiver coping, and home and neighborhood environment and opportunities.
All our interviews were completed by one researcher, increasing the consistency of questioning style. The psychometric properties may be less good when used by a number of interviewers, although the manual aims to standardize use. We tested interrater reliability, but our very high ratings were based on the second researcher listening to the assessments' audio recordings so would have been affected by the first rater's questioning style.

Clinical implications and further research
Our findings provide encouraging evidence that self-and caregiver-report SF-DEM can be used to measure social function in those with mild dementia. Further research is needed to test the instrument in those with moderate and severe dementia, to test its responsiveness to change more fully, and to assess its use in differing settings. Research in an independent multicenter UK-based clinical sample and other English-speaking countries would test the generalizability of our findings and establish normative data. A larger sample would allow factor analysis to further investigate SF-DEM's internal structure and guide possible refinement of the instruments.
Changes in social function in dementia, such as loss of previously valued social activities, and changes to social relationships are distressing for patients and their caregivers but may often not be assessed systematically in clinical settings. Using SF-DEM in clinical settings could facilitate awareness and conversation about distressing social changes, potentially supporting patients and their caregivers to make changes to improve social function. Detection of early dementia markers is a priority area in dementia research, and there is increasing interest in the concept of mild behavioral impairment, in which social changes are viewed as prodromal dementia symptoms [39]. Testing social function using SF-DEM may enable the detection of such changes. Table 4 Responsiveness to change during 6-to 8-month follow-up period: Association of SF-DEM total score and overall impression of change Abbreviations: SF-DEM, Social Functioning in Dementia; SD, standard deviation; SE, standard error.