The development of a community reintegration outcome measure to assess people with stroke living in low socioeconomic areas

Aims: The aim of this study was to develop and psychometrically evaluate an interviewadministered questionnaire to assess community reintegration for black people with stroke in low socioeconomic rural and urban South African communities. Methods: A sequential mixed methods design was used. A qualitative approach using semi-structured interviews with adults with stroke and/or their caregivers was undertaken to conceptualize and generate community reintegration interview items. Thereafter quantitative methods including the Delphi technique, nominal groups, factor analysis and internal consistency statistics were used to test the validity and reliability of the outcome measure. Results: Thirty-two interviews (13 urban, 19 rural) generated 67 items. These were Maleka Douglas Morake1, Stewart Aimee Vivienne2, Hale Leigh Ann3 Affiliations: 1PhD PT, Associate Professor, HOD Physiotherapy Department Sefako Makgatho Health Sciences University, Molotegi Street, Ga-Rankuwa Gauteng; 2PhD PT, Associate Professor, Physiotherapy Department, School of Therapeutic Sciences, Faculty of Health Sciences, University of the Witwatersrand, Johannes burg, Gauteng Province, South Africa; 3PhD PT, Professor and Dean, School of Physiotherapy, University of Otago, Dunedin, New Zealand. Corresponding Author: Stewart Aimee Vivienne PhD PT, Associate Professor, Physiotherapy Department, School of Therapeutic Sciences, Faculty of Health Sciences, University of the Witwatersrand, Johannes burg, Gauteng Province, South Africa; E-mail: aimee.stewart@wits.ac.za Received: 08 December 2016 Accepted: 14 February 2017 Published: 08 March 2017 reduced to 44 categorized under eight domains during three rounds of the Delphi technique and two nominal group meetings. Factor analysis (FA) and internal consistency statistics on the remaining items, resulted in a two part outcome measure, one for a rural and the other an urban community. The internal consistency of both measures was good (alpha coefficient = 0.95). The new outcome measure, the Maleka Stroke Community Reintegration Measure (M-SCRIM) was compared to the Subjective Index of Physical and Social Outcomes to establish construct validity (urban r = 0.88, p<0.001; rural r = 0.95, p < 0.001). Conclusion: The M-SCRIM is a valid, reliable interview-administered measure to assess community reintegration following stroke for people living in low socioeconomic communities in South Africa. With further refinement it could be validated for use in other developing countries with similar cultures.


INTRODUCTION
The prevalence of disabling stroke in South Africa is as high as that in high income countries and is particularly high amongst the black population living in poor socioeconomic areas, where literacy and education levels are low [1,2]. Following a stroke there tends to be a reduction in previous activity leading to reduced participation in usual activities and social roles [3]. In South Africa, there are no published reports of how well stroke survivors reintegrate back into their communities, although based on known issues of stroke rehabilitation in South Africa, it is likely that reintegration is poor [4]. These issues include; the very short length of stay in hospital for patients following onset of stroke (mean of six days) [5,6], the limited number of rehabilitation treatments [4] and early discharge with limited functional independence [7]. The problem is compounded by the poorly developed and inadequate community-based rehabilitation services in South Africa, [5,8,9] leaving patients ill-prepared for discharge and community reintegration. Measuring community reintegration in South Africa may assist with identification and planning for interventions to mitigate these problems, although community reintegration is a complex phenomenon to measure.
Community reintegration is an integral part of the World Health Organization's International Classification of Disablement, (ICF) [10] embracing both environmental and personal factors that cause participation restriction [11]. Community reintegration following stroke is arguably the ultimate aim of rehabilitation, [4] yet participation restriction is the least measured concept [12]. Measuring community reintegration for people with stroke living in low socioeconomic areas is challenging as there are no context specific and appropriate measures available, and there are no clear definitions of community reintegration as the construct is contextual to the environment in which people live [13][14][15].
We identified nine tools which showed potential for measuring community reintegration following stroke and critiqued them for use in a low socio-economic African population (Craig Handicap Assessment and Reporting Technique-CHART [16]; Community Integration Measure CIM [14]; Community Integration Questionnaire CIQ [17]; London Handicap Scale-LHS [18]; Participation Scale-PS [19]; Reintegration to Normal Living Index-RNLI [20]; Stroke Impact Scale-SIS [21]; Stroke-Specific Quality of Life-SS-QOL [22]; Subjective Index of Physical and Social Outcome-SIPSO [23]. To be relevant to our population, the measure needs to take into account contextual and cultural factors, as well as multi-lingualism, illiteracy, and limited education [6]. All identified measures, with the exception of the Participation Scale [19] were developed in high income countries and thus contain items that we considered not to be contextually relevant for those living in low socio-economic areas, [14,[16][17][18][20][21][22][23] with many being too long and complicated for use in populations with low illiteracy levels where questionnaires are better administered via interview [14,[16][17][18][20][21][22][23]. Furthermore, only six scales have been validated for use in a stroke population [16,18,[20][21][22][23] and many scales are also not solely specific to community reintegration [14,[16][17][18][20][21][22]. We could not find any tool appropriate to measure community reintegration following stroke in a black South African community.
This paper describes the development and psychometric testing of an interviewer-administered outcome measure of community reintegration specific for black people with stroke living in low socioeconomic rural and urban areas of South Africa. By doing this we have also tried to illustrate the importance of ensuring that the specific environmental and cultural factors associated with any group of people wherever they live is taken into account when developing scales like these. Using such a scale will help to focus and measure the integration of people following stroke back into their communities appropriately. Indeed, the process we took in developing this tool will be informative to others wanting to develop a tool relevant for low socioeconomic populations in other developing nations. The aim of this study was thus to develop and psychometrically test an interview-administered outcome measure for community reintegration specific to black South Africans with stroke living in low socio-economic rural and urban areas.
The specific objectives were to conceptualize community reintegration from the perspective of individuals who have had a stroke and their caregivers; to establish the face, content and construct validity of the developed outcome measure and test its reliability. In addition, the outcome measure was tested to establish how it fitted into the ICF framework.

MATERIALS AND METHODS
A sequential mixed methods design [24] was used to develop, validate, and test the reliability of the outcome measure. Firstly, qualitative semi-structured interviews were conducted with people with stroke living in both above mentioned communities to generate questionnaire items [25]. A quantitative approach was then used to validate the developed questionnaire and to evaluate its reliability. Brief descriptions of each are given describing the participants, process and analysis of each stage Because of the complexity of each stage of the methods.
This study was conducted in two culturally, economically and geographically distinct provinces of South Africa namely; the Gauteng (urban) and Limpopo (rural) provinces. The Gauteng province is densely populated and multicultural, whereas the Limpopo province is sparsely populated and mono-cultural and is one of the poorest provinces of South Africa.
Ethical clearance and permission was sought and obtained from the Human Research Ethics Committee www.edoriumjournals.com/ej/dr Douglas et al. 13 (HREC) of the University of the Witwatersrand (M070816) prior to commencement of the study as well as from the respective health authorities and facilities in the two provinces. Signed informed consent was sought and obtained from all participants prior to participation in any of the studies.

Steps taken to generate questionnaire items
The starting point for developing this outcome measure was a qualitative study designed to generate appropriate items for the outcome measure. This part of the study has been published in the reputed journal of South Africa [25]. Thirty-two participants with stroke and their caregivers were involved in the in-depth interviews-19 from the rural province and 13 from the urban province. Their demographic characteristics are listed in Table 1. The in-depth interviews concentrated on establishing how the participants felt they had managed to integrate back into their communities after the stroke relative to their community participation prior to the stroke. The themes generated from this study were then used by the research team in a draft outcome measure for further analysis.
Steps taken to ensure validity and reliability of the draft outcome measure Four phases were undertaken: Firstly Delphi and nominal group techniques were used to establish face and content validity and to determine consensus on items to be included in the outcome measure. Secondly, factor analysis (FA) was undertaken to reduce the number of questionnaire items, and internal consistency was established. Thirdly, the draft questionnaire was then compared with the Subjective Index of Physical and Social Outcome (SIPSO) to establish construct validity [23], and evaluated within an ICF framework.
Forty-four questions remained in the draft questionnaire after three rounds of a Delphi technique that included 10 neurological and community based rehabilitation experts. The experts were identified and chosen for their expertise and experience in community and neurological rehabilitation and are listed in Table  2. The questions remaining after three rounds of the Delphi were considered to be appropriate for measuring community reintegration as a construct and had reached 80% consensus in the group. In addition, agreement was reached on the response scale for the questions. The rigor of the questions and scoring system was then ensured by consulting with a health statistician and a psychologist, both specialists in outcome measure development.
Two nominal group techniques with 10 participants with stroke and their caregivers who had participated in the qualitative study were then conducted to comment on whether the items were understandable, the importance of these items to community reintegration, and how contextually and culturally appropriate they were.
The number of items in the questionnaire were then reduced using factor analysis and internal consistency was established with 112 people with stroke living in the urban area and 104 living in the rural area (216 in total), none of whom previously had been interviewed. The demographic characteristics of the participants for this part of the study are listed in Table 3. Participants for this phase were recruited by the on-site facility physiotherapists in both provinces. Participants were included if they had a first ever stroke, were aged above 18 years, and had been living in their community pre and post stroke for more than six months.
Factor analysis was used to examine the factor loadings to provide information on the underlying dimensions of the measure and provide an estimate of construct validity [26]. The items with low factor loading(s), i.e. below 0.3 were removed from the questionnaire. Following the factor analysis, the items in the preliminary outcome measure were regrouped to form new domains and the scoring system finalized.
An internal overall consistency test was performed using Cronbach alpha [27]. In addition item-total/rest correlations were performed to establish which items may be inconsistent with the outcome measure and contribute error to the instrument. Cronbach alpha values approaching 0.90 indicate high homogeneity of items and item-rest correlations should yield correlations between 0.70.and 0.90 when examining internal consistency [28]. Values between 0.70 and 0.60 demonstrate acceptable consistency and any below this level, low internal consistency [28]. Only factors with an overall Cronbach alpha of 0.70 and above were retained and items with low item-rest correlations <0.6 were removed. Individual item scores were summed to calculate a total score. The total score can then be used to represent the degree of reintegration into the community, with higher scores implying more successful integration.
The construct validity was then established by comparing the new outcome measure to the SIPSO [23]. The SIPSO is a 10-item self-completed questionnaire that measures social/community reintegration following stroke [23] and was chosen because it is the only scale developed specifically to measure participation, and was validated in a stroke population which included younger people (age range 18-65 years) [29]. It is quick to administer. Eighty people (40 rural, 40 urban) with stroke participated in this phase of the study and their characteristics are listed in Table 4. Participants' inclusion and recruitment was the same as described in the previous phase of this study, and no one had participated in any of the previous parts of the study described above. Participants were interviewed using both the SIPSO and the newly developed outcome measure. A Pearson moment product correlation (2-tailed) statistic was used www.edoriumjournals.com/ej/dr Douglas et al. 14 to establish the correlation between the two outcome measures.
The developed outcome measure was compared to the domains of the International Classification of Function (ICF) [11]. This was done using the ICF as a framework to ascertain the fit of the developed items in the MSCRIM to the ICF.
The SSPS (version 17) was used to analyze all the above data and the alpha level was set at p<0.05. At the end of the process described above the outcome measure was named the Maleka Stroke Community Reintegration Measure (M-SCRIM-Appendix 1) by consensus of participants in the Delphi study.

RESULTS
In total 328 people with stroke participated in the different phases of the development of the outcome measure.
A total of 67 items was generated from the qualitative interview data, and they were classified into eleven domains. After the Delphi rounds and the nominal group meetings, the outcome measure was reduced to eight domains comprising 46 items. These eight domains were: activities of daily living, home responsibilities, family responsibilities, community responsibilities, travel and transport, religion, education, relationships, medical care, recreation and productivity.
The factor analysis resulted in two questionnaires, one for a rural and one for an urban setting. The two questionnaires ended with six domains each, which were renamed according to the new groupings derived from the factor analysis. In the rural setting questionnaire 12 items were removed by factor analysis leaving 34 items and six items were removed by FA from the urban setting questionnaire, leaving 40 items. The newly grouped items were found to have excellent internal consistency (Cronbach alpha = 0.95).
The comparison of the MSCIM with the ICF is illustrated in Table 5. The comparison shows how the various domains and items on the MSCIM relate to the activities and participation sub-categories of the ICF.

DISCUSSION
Community reintegration is arguably the ultimate aim of rehabilitation but measuring successful fulfilment of this construct can be problematic. Although some tools exist to measure community reintegration following stroke these were deemed by the research team to be contextually and culturally inappropriate for application with black South Africans living with stroke in impoverished areas. Many outcome measures have been criticised as they were developed with little or no consultation with the population in question [30,31]. Consultation is becoming increasingly important especially with the notion of wanting to produce services that are relevant and appropriate to the intended population [32]. The MSCRIM developed in this study included the narratives of people with stroke; it was subjected to analysis by an expert group; it was tested for internal consistency and compared to an existing outcome measure to establish its construct validity.
Development of the desired outcome measure in this study thus began with a qualitative study to collect the narratives of people living with stroke in both rural and urban low socioeconomic areas. This qualitative step proved critical to the face and content validity of the M-SCRIM, as the impact of context became very apparent during the interviews. For example, for many people living in rural areas, challenges to successful  reintegration include; the type of house (a traditional hut or a small house with an outside toilet), untarred roads making access difficult, and living a long way from public transport [25]. For people living in an urban area, the barriers to reintegration included decreased mobility in the community, confined spaces at home and within the community and lack of access to sustainable livelihood. Yet for both groups being able to help others as part of their community responsibilities, and social interactions and relationships were considered extremely important, as was returning to work or school. People highly valued opportunities to return to some form of employment to gain some degree of independent income and a positive livelihood.
Other considerations that had to be taken into account in the development of the M-SCRIM were that many black people from low socioeconomic areas are illiterate and uneducated and that there are 11 official languages spoken in South Africa (and many more 'unofficial' dialects). Furthermore there are many different cultural beliefs and practices held by black South Africans. These differences needed to be acknowledged and accommodated by the outcome measure. These language and cultural variances along with geographical location differences resulted in two measures being developed, one appropriate for those living in a rural setting, and the other appropriate for an urban setting.

Access to a caregiver
All participants had access to caregivers and all caregivers were members of the family, including a child, a wife or a husband. None had medical training.
All participants had access to caregivers and all caregivers were members of the family, including a child, a wife or a husband. None had medical training.  According to Radhakrishna,[30] the development of an outcome measurement tool should be undertaken in consultation, not only with the patient population in question, but also with those who will need to apply it, in this case, rehabilitation professional experts. Three rounds of the Delphi technique with a group of experts validated this outcome measure and reduced it to eight domains and 44 items. The scoring system was also finalized, in this process with a 4 point Likert scale being chosen. This process also identified the name of the outocme measure.
Factor analysis and internal consistency statistics were used to reduce item numbers in the newly develop outcome measure [33]. To ensure a valid process, the sample size for factor analysis studies must be appropriate; 10-20 respondents per item is recommended [26,34]. In this study 216 respondents were included. There are three values to consider when using factor analysis: (1) the Eigen value, which reflects the amount of variance accounted for by each factor, (2) factor loadings, which represent the correlation between each item and each factor and (3) rotations, which are done to identify meaningful factors that include highly correlated items of the factor [35]. Usually a factor loading of 0.30 or less is not meaningful [35]. In the case of the M-SCRIM, 12 items with low factor loadings were removed from the rural  [23,37] a very high correlation was found, indicating that the two measures assess the same construct, namely community reintegration. The SIPSO was considered inappropriate as an outcome measure for the population in question as it was designed for people living in a developed country, it uses a 5-point Likert scale and some of the items would not have translated meaningfully into local languages. For example, item 6, "Since your stroke, how often do you feel bored with your free time". The phrase "feeling bored" is not easily translated into one word, may need to be explained in writing or to participants during the interview.
The WHO ICF (2001) provides a multi-dimensional framework for health and disability suited to the classification and development of new outcome measures/ instruments [11]. Outcomes may be measured at any of the ICF levels/domains/dimensions. Participation restriction happens when an individual is unable to carry out his/her tasks or responsibilities due to a disease or illness. The restriction may not be due to reduced body structure and function only, but may also be due to the environmental and/or personal factors referred to as contextual factors in the ICF. Participation is a relatively recent concept that is not clearly understood or measured as yet [11]. This is one area of the ICF that deserves much more attention as increasingly participation is considered a pivotal outcome of successful rehabilitation [12].
Many studies have used the ICF as the framework for the development of outcome measures [18,38,39,40]. This study also used the ICF as a framework for the development of the outcome measure to assess the level of participation i.e. community reintegration after stroke. Given the fact that the ICF is a widely used tool in rehabilitation, it is noteworthy that the items in the M-SCRIM fit with most of the ICF domains. Thus given these similarities this outcome measure may well be useful to other developing countries with similar cultures but this would need appropriate further research. What would be useful for such countries is the process we undertook to develop the M-SCRIM, which could be easily replicated.
The strength of our study is that it followed a sequential rigorous methodology in the development of the outcome measure with 328 people with stroke participating in various stages of the study.

CONCLUSION
This paper reported the development and psychometric evaluation of a new outcome measure, the M-SCRIM, for community reintegration of black South Africans stroke survivors living in low socioeconomic areas. We took into consideration the views of the population in question by conceptualising what community reintegration meant to them and what they would consider good reintegration to be. We found the M-SCRIM to have excellent content and construct validity, as well as excellent internal consistency. Further testing in similar communities elsewhere in South Africa, and indeed Africa, would enhance the usability of this measure in a wider context. Community reintegration is the overriding aim of rehabilitation but is often poorly carried out and usually not measured as an outcome of rehabilitation. This study has shown the importance of measuring community reintegration in a relevant and appropriate environmental and cultural manner as well as illustrating the methodology that can be used in studies to develop scales like these for other disabling conditions and contexts.