Abstracts

ORIGINAL ARTICLES

Reliability of the instrument for classifying elderly people's capacity for self-care

Maria Helena Morgani de Almeida^I; Aracy Witt de Pinho Spínola^II; Paula Stefanoni Iwamizu^III; Roberta Irie Sumi Okura^III; Lucia Pereira Barroso^IV; Antonio Carlos Pedroso de Lima^IV

^IDepartamento de Fisioterapia, Fonoaudiologia e Terapia Ocupacional. Faculdade de Medicina. Universidade de São Paulo (USP). São Paulo, SP, Brasil

^IIDepartamento de Prática de Saúde Pública. Faculdade de Saúde Pública/USP. São Paulo, SP, Brasil

^IIIPrograma de Pós-Graduação em Estatística. Instituto de Matemática e Estatística (IME)/USP. São Paulo, SP, Brasil

^IVDepartamento de Estatística. IME/USP. São Paulo, SP, Brasil

^{Correspondence} Correspondence: Maria Helena Morgani de Almeida Departamento de Fisioterapia, Fonoaudiologia e Terapia Ocupacional da FMUSP R. Cipotânea, 51 05360-000 São Paulo, SP, Brasil E-mail: hmorgani@usp.br

ABSTRACT

OBJECTIVE: To evaluate the reliability of an instrument for classifying elderly people regarding their capacity for self-care, which was developed to assist occupational therapists in attending elderly people at primary healthcare units.

METHODS: Stability and internal consistency tests were carried out. To validate the instrument, tests were applied to a sample of 30 individuals aged 60 years and over, on two occasions. The statistical analysis was performed after careful grouping of the responses. This led to the formulation of a simplified version of the instrument. The stability of this version was assessed using the kappa coefficient and the internal consistency by Cronbach's alpha coefficient.

RESULTS: The stability ranged from moderate to excellent. The internal consistency was checked only for areas that were shown to be appropriate for using the methodology, based on calculations of Cronbach's alpha: three of the six questions in the "social profile" area and the blocks of basic and instrumental activities of daily living in the "functional capacity" area, which respectively consisted of nine and eight activities.

CONCLUSIONS: Following the stability and internal consistency tests, the instrument made it possible to succinctly and simply classify elderly people with regard to their functional capacity for basic and instrumental activities, and to characterize them regarding other aspects of self-care. The evidence regarding its reliability and validity could be expanded by means of new studies.

Descriptors: Aged. Self Care, classification. Disability Evaluation. Occupational Therapy. Reproducibility of Results.

INTRODUCTION

The relevance of instruments for assessing and classifying elderly people with regard to aspects of their self-care is due to the increasing proportion of elderly people in the general population and their many social and healthcare needs. According to the 2000 census,¹ 1 Instituto Brasileiro de Geografia e Estatística. Brasil em números: 2000. Rio de Janeiro; 2000. v.8. the proportion of individuals aged 60 years and over is approximately 9% of the total population. By 2025, this proportion of the Brazilian population is expected to have grown 15-fold, thus putting Brazil into sixth place in the world in terms of absolute numbers of elderly people.⁶

Aging increases the susceptibility to chronic and incapacitating illnesses. This leads to increased demand for medical-social services and policies and programs for health promotion and prevention of illnesses and incapacities.⁷

For elderly people to be maintained in their environments with functional abilities and autonomy, they need to receive interdisciplinary care at all levels of complexity of the Brazilian national health system (Sistema Único de Saúde – SUS).² 2 Ministério da Previdência e Assistência Social. Secretaria da Assistência Social. Política Nacional do Idoso: perspectiva governamental. Brasília; 1996. One of the ways of assessing functional abilities and autonomy is by means of the "Elderly people Self-care Capacity Classification" (ESCC) instrument, which was drawn up with the aim of contributing towards systematizing the occupational therapeutic care for elderly people, especially at primary healthcare units.

The validity of an instrument relates to the degree to which the results from a measurement correspond to the true state of the phenomena that are being measured. What indicates whether an instrument has greater or lesser validity is the results obtained through strategies that are used to indicate its validity. It is possible to build up evidence regarding the validity of an instrument by using strategies such as content, construct and criterion validation.⁴

Content validity ensures that the items of an instrument adequately cover and represent what is measured. These items may be obtained by applying questionnaires to patients, be suggested by specialists or come from the literature. The appropriateness, clarity and coverage of the items may be aspects of the validity.^4,8,9 The construct validity reflects the capacity of the instrument to measure a set of behavioral patterns that are related to each other and are known to be associated with the phenomenon that is being measured. The criterion validity is the most popular means of determining validity and described an empirical relationship between a measurement and a reliable criterion of some type.⁴

The validity of the ESCC instrument is so far limited to indications from the results obtained through validating its content by means of the Delphi technique. This consisted of a systematic strategy of listening to and analyzing opinions from specialists with a view to ultimately generating a validated product.³ The ESCC instrument was adjusted in three stages by a jury of 15 specialists, with regard to its clarity of expression, pertinence and organization of questions, thus resulting in a group version. The specialists were occupational therapists who had been active in the field of gerontology for two years or more, in the state of São Paulo. In the first stage, the specialists' judgments were obtained by means of opinion scales, comments and suggestions. During this stage, it was sought to consider the contributions of all the specialists to the topic in question, independent of their frequency, since this was an exploratory stage. In the second and third stages of the study, only items that were approved by 70% or more of the specialists were kept in the instrument.¹⁴ The items in the group version had a mean approval rate of 93.3% for content and 86.7% for expression.

Reliability relates to the extent to which repeated measurements of a relatively stable phenomenon are situated close to each other and can be verified by specific tests.^4,9 The aim of the present study was to evaluate the reliability of the ESCC instrument.

METHODS

To evaluate the stability of the instrument, it was applied to a convenience sample composed of 30 elderly people (60 years old and over), on two occasions with an interval of seven to 15 days. To check the internal consistency, only the results from the first application were taken into consideration. The distribution of the participants according to age group and sex is presented in Table 1.

The participants were selected taking into consideration their willingness (expressed in terms of consenting to participate in the study), capacity to comprehend and express ideas and satisfactory performance in the Clock Completion Test (CCT).¹² Traditionally, the CCT is used to investigate constructional apraxia, which is a dysfunction of the parietal lobe that frequently accompanies the initial stage of dementia. Recent studies have affirmed its practicality and sensitivity for detecting dementia in elderly people.^12,13 A score of zero was attributed to normal results and a score of one to abnormalities in the first three quadrants. Abnormal results in the fourth quadrant are suggestive of dementia and received scores of four. According to the test criteria, results from four to seven are indicative of dementia.¹³

There were no dropouts among the participants, although it is common for there to be losses in test-retest procedures. This was probably achieved because the interviewees had expectations that, on completing the second interview, they would receive guidance regarding their day-to-day difficulties.

Because of the high quantity of response combinations present in the instrument, in relation to the number of elderly people participating, it was necessary to make some groupings of the responses in the stability and internal consistency analyses. These groupings were made on the basis of theoretical conceptual criteria and resulted in the simplified version presented in Table 2. The statistical analysis was performed on this version.³ 3 Lima ACP, Barroso LP, Iwamizu PS, Okura RIS. Relatório de análise estatística sobre o projeto "Validação do Instrumento CICAc (Classificação de Idosos quanto à Capacidade para o Autocuidado)". São Paulo: Instituto de Matemática e Estatística da USP; 2003. (RAE - CEA - 03P02).

Concordance analysis was performed for all the fields and respective variables that made up the instrument. For the test and retest responses, methodology based on kappa coefficients was used, with weighting for the ordinals. The degree of concordance was compared with the criterion that established that kappa values of less than 0.4 indicated weak concordance; between 0.4 and 0.8, moderate; and greater than 0.8, excellent.

The internal consistency was analyzed using Cronbach's alpha coefficient. Values greater than or equal to 0.7 were considered indicative of internal consistency.¹⁰ This coefficient is used when the instrument (or part of it) is composed of items laid out on an ordinal scale, each with the same number of points, such that the sum of these items results in a total value that measures a construct of interest. This coefficient cannot be calculated if there are any missing values in one or more of the items that make up the sum of the scale.

The study followed the standards required by the Helsinki Declaration and was approved by the Research Ethics Committee of FSP/USP.

RESULTS

A large proportion of the responses showed moderate to excellent concordance, such that 60% of the measurements were greater than 0.8 and only 7% were less than or equal to 0.4. From the 248 measurements for which kappa coefficients were calculated, it was found that on average the kappa coefficient was high, such that 75% of the values were greater than 0.65 and more than 50% were equal to one, thereby indicating perfect concordance. The lowest value obtained was -0.09. Although other low values were observed, quantitative analysis on the corresponding frequency tables did not show any great discordance between the responses, as seen in Table 3.

Cases in which the responses to the question were of nominal type with a table of frequencies presenting columns or lines equal to zero did not enter the calculation of measurements for the kappa coefficient, since it is inappropriate to use the kappa coefficient in such situations. It was observed that there were 16 questions with a similar pattern over the course of the process, all with high concordance between the test and retest.

Table 4 shows that the mean value of the kappa coefficient was less than 0.7 only for the field of "domestic arrangements and potential support network". 75% of the kappa coefficients calculated for the field of "social profile" presented values greater than 0.8, which suggests excellent stability.

As presented in Table 2, only 145 kappa coefficients (58%) were calculated for the field of "functional capacity". The questions on "types of difficulty" ane "ways of compensating adopted" (questions 17 to 19) presented a mean kappa coefficient of 0.87. For the questions on opinions such as "degree of satisfaction with present relationships" (question 5) and "possibilities of overcoming or not overcoming difficulties in your activities" (question 20), the mean values obtained were close to 0.4.

The elderly people were more consistent in informing which activities they "liked least" (question 16) than which activities "were most important and included the ones they liked most" (question 15). These presented, respectively, mean kappa coefficient values of 0.95 and 0.66.

For question 21 ("ways imagined for overcoming difficulties") and 22.2 ("ways imagined for achieving desired activities"), the mean kappa coefficient values were 0.80 and 0.73, respectively. This indicated greater consistency among the elderly people than what was presented in question 20 ("possibilities of overcoming or not overcoming difficulties in your activities"), for which the kappa value was 0.41.

In the field of "social profile", the scale corresponding to satisfaction with income was maintained, the scale corresponding to income was constructed and the other scales were adjusted so that they would keep the same number of points. In this field, Cronbach's alpha was -0.078 and was calculated for 29 elderly people.

Table 5 presents the results from Cronbach's alpha coefficient for the sections of basic and instrumental activities of daily living that belonged to the field of "functional capacity". It can be seen that 14 of the elderly people were excluded from the calculation for the section of instrumental activities because they presented missing values in one or more items. Thus, only 16 of the elderly people remained for the calculation. It was impossible to calculate the alpha coefficient for the sections of work and leisure activities, since some items that formed part of these were not cited by any of the elderly people.

In accordance with the criteria adopted,¹⁰ the study showed the existence of consistency for the sections of basic and instrumental activities of daily living in the ESCC.

DISCUSSION

In analyzing the stability of the instrument, although some opinion-providing questions were within the defined criteria for moderate stability, the mean values obtained were lower than for questions providing information on the elderly individual's present behavioral patterns. This was understandable, given that their behavioral patterns were material facts, while opinions (albeit predictors of behavior) did not reach the level of material reality.

Nonetheless, this did not make the opinions less important in an instrument aiming to classify elderly people with regard to their self-care capacity. This capacity includes the ability to discern factors that can be controlled and managed in order to regulate functional capability and human development. The questions providing information such as the "elderly person's degree of satisfaction regarding the social support network" and "imagined compensation" relate to this dimension of self-care.

In analyzing the internal consistency, low values were obtained for the field of "social profile". This may indicate that the ways used to measure the variables of "schooling level", "income" and "satisfaction with income" were not doing so homogenously.

With the aim of correcting for some missing values in the field of "functional capacity" and expanding the number of elderly people used in calculating Cronbach's alpha coefficient, elderly people can be asked about their potential capacity to perform instrumental activities of daily living. Although questioning patients about what they are or are not able to do generates hypothetical responses, this type of approach has been used in instruments for assessing instrumental activities of daily living. Although elderly people may be familiar with these activities and are often capable of performing them, they may not do them for other reasons that are unrelated to their health.⁸

With regard to correcting for missing values for leisure and work activities, it was considered that asking about the potential capacity to perform such activities would imply accepting that the aging process would lead to homogenization. This would go against some important theoretical reference points, in which aging is considered to be a process of individualization, characterized by heterogeneity among aging individuals and diversity in their choices of activities.^2,5,11

Concerning the small sample size, it has to be borne in mind that some studies for evaluating the reliability of scales for activities of daily living (with content relating to ESCC) also used small samples. For example, the study to assess the interrater reliability of the fifteenth version of the Barthel index, which was conducted by Shinar et al and cited by McDowell & Newell,⁸ used a sample of 18 patients. The study to assess the test-retest stability of the Health Assessment Questionnaire (HAQ), which was conducted by Fries and cited by McDowell & Newell,⁸ used a sample of 37 elderly people.

In Brazil, the reliability and validity study on the Portuguese version of Smilkstein's Family Apgar⁴ 4 Duarte YAO. Família: rede de suporte ou fator estressor. A ótica de idosos e cuidadores familiares [doctoral thesis]. São Paulo: Escola de Enfermagem da USP; 2001. can be cited. The test-retest procedure was performed using a sample of 27 elderly people (first application) and 23 elderly people (second application).

Despite the limitations of the present study, the simplified version of the instrument that was generated makes it possible to classify elderly people in a succinct and simplified manner regarding their functional capacity for basic and instrumental activities of daily living, and to characterize them regarding other aspects of self-care. Elderly people's functional capacity can be classified both for the sections of basic and instrumental activities together and for each of their activities separately. From the score attributed, the elderly individual can be positioned within a continuum for basic activities that goes from nine points (fails to perform any activity) to 45 points (presents no difficulties in any activity) and from eight to 40 points for instrumental activities. Even though attributing total scores may have uses in certain situations, such as investigating and managing services, elderly individuals' functional conditions for each of their activities has greater importance with regard to guiding clinical treatment.

Within the limits that have been set out, the simplified version of the instrument provides some valid and reliable reference points regarding elderly people's capacity to care for themselves. In conjunction with other instruments, this version supports the planning, development and assessment of care provided by occupational therapists. The simplified version has been shown to be a useful and appropriate tool for these professionals' daily routine in primary healthcare units, since it guides towards a rapid, succinct and simplified classification of elderly people's functional capacity. However, it is considered that new studies could be conducted to broaden the evidence regarding its reliability and validity.

REFERENCES

Received: 1/17/1/2007

Reviewed: 9/17/2007

Approved: 10/15/2007

Article based on the doctorate thesis by MHM Almeida presented to Faculdade de Saúde Pública of Univerisdade de São Paulo, in 2003.

Publication Dates

History