ABSTRACT

With the technological advancement and democratization of electronic devices, computerized cognitive tests have been increasingly used in the clinical context to evaluate cognitive performance in individuals.

Objective:

This study aimed to propose a brief digital cognitive test based on the paradigm of stimulus equivalence and assess its convergent validity by comparing it with traditionally applied tests.

Methods:

The study was carried out with a non-probabilistic sample of 50 older adults selected from a public call through the communication media, health units, and day centers of a city in the countryside of São Paulo. Participants were assessed by the brief digital cognitive test, Mini-Mental State Examination, Brief Cognitive Screening Battery, and the Five Digit Test.

Results:

Participants had a mean age of 71.23 years (standard deviation [SD]: ±9.36) and a mean of 7.15 years of schooling (SD: ±5.34). The mean time to answer the test was 5.33 minutes (SD: ±1.92). There were statistically significant correlations between traditional and digital tests in most domains evaluated. In addition, considering the total score of the digital test, the test could discriminate participants with and without cognitive impairment: area under the ROC curve=0.765; 95%CI 0.630–0.901.

Conclusion:

The brief digital cognitive test, using the stimulus equivalence paradigm, is an easy-to-apply and valid instrument for the investigation of cognitive impairment in older adults.

Keywords:
Aged; Cognition; Neuropsychological Tests; Learning

RESUMO

Com o avanço tecnológico e a democratização de dispositivos eletrônicos, os testes cognitivos computadorizados vêm sendo cada vez mais utilizados no contexto clínico para avaliar o desempenho cognitivo de indivíduos.

Objetivo:

Este estudo teve como objetivo propor um Teste Cognitivo Digital Breve baseado no paradigma de equivalência de estímulos e avaliar sua validade convergente comparando-o com testes aplicados tradicionalmente.

Métodos:

O estudo foi realizado com uma amostra não probabilística de 50 idosos selecionados por chamada pública pelos meios de comunicação, unidades de saúde e centros dia de uma cidade do interior de São Paulo. Os participantes foram avaliados pelo Teste Cognitivo Digital Breve, Miniexame do Estado Mental, Bateria Breve de Rastreio Cognitivo e Teste dos Cinco Dígitos.

Resultados:

Os participantes tinham média de idade de 71,23 anos (desvio padrão — DP: ±9,36) e média de anos de estudo de 7,15 anos (DP: ±5,34). O tempo médio para a realização do teste foi de 5,33 minutos (DP: ±1,92). Houve correlações estatisticamente significativas entre os testes tradicionais e o Teste Digital na maioria dos domínios avaliados. Além disso, considerando o escore total do teste digital, o teste foi capaz de discriminar participantes com e sem comprometimento: área sob a curva ROC=0,765; IC95% 0,630–0,901.

Conclusão:

O teste digital cognitivo breve com o uso do paradigma de equivalência de estímulos é um instrumento de fácil aplicação e válido para a investigação de comprometimento cognitivo em idosos.

Palavras-chave:
Idoso; Cognição; Testes Neuropsicológicos; Aprendizagem

INTRODUCTION

Cognition is the ability to acquire and process knowledge and is commonly divided into domains for assessing and categorizing individual cognitive performance¹1 Harvey PD. Domains of cognition and their assessment. Dialogues Clin Neurosci. 2019;21(3):227-37. https://doi.org/10.31887/DCNS.2019.21.3/pharvey
https://doi.org/10.31887/DCNS.2019.21.3/... . Cognitive performance gains special importance during the aging process, considering the natural and pathological decline that can occur over the years²2 Brehmer Y, Kalpouzos G, Wenger E, Lövdén M. Plasticity of brain and cognition in older adults. Psychol Res. 2014;78(6):790-802. https://doi.org/10.1007/s00426-014-0587-z
https://doi.org/10.1007/s00426-014-0587-... .

Population aging is a worldwide phenomenon characterized by changes in the age pyramid. In view of this demographic scenario, investigating cognitive changes in this population becomes of the utmost importance. Cognitive assessment is traditionally performed with pen and paper; therefore, the available and validated tests are conducted in this way³3 Abd Razak MA, Ahmad NA, Chan YY, Mohamad Kasim N, Yusof M, Abdul Ghani MKA, et al. Validity of screening tools for dementia and mild cognitive impairment among the elderly in primary health care: a systematic review. Public Health. 2019;169:84-92. https://doi.org/10.1016/j.puhe.2019.01.001
https://doi.org/10.1016/j.puhe.2019.01.0... . With the technological advancement, smartphones and tablets, computerized and digital cognitive tests are increasingly used in the clinical context⁴4 Wong A, Fong CH, Mok VCT, Leung KT, Tong RKY. Computerized Cognitive Screen (CoCoSc): a self-administered computerized test for screening for cognitive impairment in community social centers. J Alzheimer’s Dis. 2017;59(4):1299-306. https://doi.org/10.3233/JAD-170196
https://doi.org/10.3233/JAD-170196... –⁶6 Oliveira RS, Trezza BM, Busse AL, Filho WJ. Use of computerized tests to assess the cognitive impact of interventions in the elderly. Dement Neuropsychol. 2014;8(2):107-11. https://doi.org/10.1590/s1980-57642014DN82000004
https://doi.org/10.1590/s1980-57642014DN... . The advantages of using computerized cognitive tests include the possibility of being self-applied with standardization, the chance to assess a greater number of people, and obtain greater data accuracy according to the test⁶6 Oliveira RS, Trezza BM, Busse AL, Filho WJ. Use of computerized tests to assess the cognitive impact of interventions in the elderly. Dement Neuropsychol. 2014;8(2):107-11. https://doi.org/10.1590/s1980-57642014DN82000004
https://doi.org/10.1590/s1980-57642014DN... . On the other hand, the individual subjected to the test would need skills for handling electronic devices⁷7 Jenkins A, Lindsay S, Eslambolchilar P, Thornton IM, Tales A. Administering cognitive tests through touch screen tablet devices: potential issues. J Alzheimer’s Dis. 2016;54(3):1169-82. https://doi.org/10.3233/JAD-160545
https://doi.org/10.3233/JAD-160545... .

In the elaboration of computerized tests, different approaches can be used, such as the adaptation of a traditional test with minimal changes in its structure⁸8 Oliveira RS, Trezza BM, Busse AL, Jacob-Filho W. Learning effect of computerized cognitive tests in older adults. Einstein (Sao Paulo). 2014;12(2):149-53. https://doi.org/10.1590/S1679-45082014AO2954
https://doi.org/10.1590/S1679-45082014AO... ,⁹9 Coelho LF, Rosário MC, Mastrorosa RS, Miranda MC, Bueno OFA. Performance of a Brazilian sample on the computerized Wisconsin Card Sorting Test. Psychol Neurosci. 2012;5(2):147-56. https://doi.org/10.3922/j.psns.2012.2.04
https://doi.org/10.3922/j.psns.2012.2.04... , the development of a test based on an existing one, or from a paradigm other than the existing ones. A paradigm widely used in learning semantic relations with children and adults with intellectual and cognitive difficulties is stimulus equivalence.

When formalizing the stimulus equivalence paradigm, Sidman and Tailby proposed that symbolic and meaning relations can be operationalized from the verification that the stimuli involved in the class have reflexivity, symmetry, and transitivity properties¹⁰10 Sidman M, Tailby W. Conditional discrimination vs. matching to sample: an expansion of the testing paradigm. J Exp Anal Behav. 1982;37(1):5-22. https://doi.org/10.1901/jeab.1982.37-5
https://doi.org/10.1901/jeab.1982.37-5... . Reflexivity is observed when, from training the relation between two stimuli, for example between A and B, the participant is able to respond to identical stimuli without direct training (AàA and BàB). In symmetry, when learning the AB relation, the inverse relation emerges (if AàB, so BàA). Finally, transitivity refers to the emergence of relations between stimuli that were never directly trained (if AàB and AàC, then BàC and CàB)¹¹11 De Rose JC. Classes de estímulos: implicações para uma análise comportamental da cognição. Psicol Teor Pesqui. 1993;9(2):283-303.,¹²12 Sidman M. Equivalence relations and behavior: a research story. Boston: Authors Cooperative; 1994.. One way of forming the equivalence classes is through the matching-to-sample (MTS) procedure, in which the relations between the arbitrary stimuli are taught¹⁰10 Sidman M, Tailby W. Conditional discrimination vs. matching to sample: an expansion of the testing paradigm. J Exp Anal Behav. 1982;37(1):5-22. https://doi.org/10.1901/jeab.1982.37-5
https://doi.org/10.1901/jeab.1982.37-5... .

Various studies that evaluated this paradigm to assist in the learning of individuals with cognitive difficulties developed tasks for the formation of equivalence classes with the aid of electronic devices¹³13 Arntzen E, Steingrimsdottir HS. Electroencephalography (EEG) in the study of equivalence class formation. An explorative study. Front Hum Neurosci. 2017;11:58. https://doi.org/10.3389/fnhum.2017.00058
https://doi.org/10.3389/fnhum.2017.00058... ,¹⁴14 Steingrimsdottir HS, Arntzen E. Performance by older adults on identity and arbitrary matching-to-sample tasks. Psychol Rec. 2014;64(4):827-39. https://doi.org/10.1007/s40732-014-0053-8
https://doi.org/10.1007/s40732-014-0053-... . These studies also pointed out difficulties in forming equivalence classes in aged individuals with cognitive impairment and a possible positive correlation between performance in class formation and performance in cognitive assessment tests¹⁵15 Aggio NM, Teixeira IO, Rose JC. An exploratory study of directly taught and emergent relations in an elderly woman with Alzheimer’s disease. Psychol Rec. 2021;71:493-7. https://doi.org/10.1007/s40732-020-00441-y
https://doi.org/10.1007/s40732-020-00441... ,¹⁶16 Gallagher SM, Keenan M. Stimulus equivalence and the Mini Mental Status Examination in the elderly. Eur J Behav Anal. 2009;10(2):159-65. https://doi.org/10.1080/15021149.2009.11434316
https://doi.org/10.1080/15021149.2009.11... . Thus, this study aims to propose a brief digital cognitive test based on the stimulus equivalence paradigm, comparing it with traditionally applied tests.

METHODS

Participants

The study was conducted with a non-probabilistic sample composed of 50 older adults. The inclusion criterion was age 60 years or older, and the exclusion criteria were the presence of severe visual or auditory deficits that would hinder the understanding of the tests, the presence of severe clinical or psychiatric comorbidity that would prevent conducting the tests, and self-reported lack of skill with electronic devices. These criteria were verified according to the researchers’ evaluation and based on the volunteers’ reports.

Cognitive assessment

Mini-mental state examination

The mini-mental state examination (MMSE) is an instrument widely used to screen for cognitive impairment that assesses cognition in a general manner. Its total score can vary from 0 to 30 points, and the lower the score, the greater the cognitive impairment¹⁷17 Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-98. https://doi.org/10.1016/0022-3956(75)90026-6
https://doi.org/10.1016/0022-3956(75)900... . In our study, the cutoff points proposed by Almeida¹⁸18 Almeida OP. Mini exame dos estado mental e o diagnóstico de demência no Brasil. Arq Neuropsiquiatr. 1998;56(3B):605-12. https://doi.org/10.1590/s0004-282x1998000400014
https://doi.org/10.1590/s0004-282x199800... were used to separate the participants into groups with and without cognitive impairment, being 19/20 for older adults with no schooling and 23/24 for those with previous school history¹⁸18 Almeida OP. Mini exame dos estado mental e o diagnóstico de demência no Brasil. Arq Neuropsiquiatr. 1998;56(3B):605-12. https://doi.org/10.1590/s0004-282x1998000400014
https://doi.org/10.1590/s0004-282x199800... . The MMSE was used as a measure of general cognition.

Brief cognitive screening battery

The brief cognitive screening battery (BCSB) was developed by Nitrini et al.¹⁹19 Nitrini R, Lefèvre BH, Mathias SC, Caramelli P, Carrilho PE, Sauaia N, et al. Neuropsychological tests of simple application for diagnosing dementia. Arq Neuropsiquiatr. 1994;52(4):457-65. https://doi.org/10.1590/s0004-282x1994000400001
https://doi.org/10.1590/s0004-282x199400... with the objective of obtaining an instrument with less influence from schooling. It includes the figure memory test (incidental, immediate, learning, delayed, and recognition), verbal fluency (animals), and clock-drawing test¹⁹19 Nitrini R, Lefèvre BH, Mathias SC, Caramelli P, Carrilho PE, Sauaia N, et al. Neuropsychological tests of simple application for diagnosing dementia. Arq Neuropsiquiatr. 1994;52(4):457-65. https://doi.org/10.1590/s0004-282x1994000400001
https://doi.org/10.1590/s0004-282x199400... . The accuracy to differentiate healthy older adults from those with great neurocognitive disorder was previously tested in the Brazilian population²⁰20 Fichman-Charchat H, Miranda CV, Fernandes CS, Mograbi D, Oliveira RM, Novaes R, et al. Brief Cognitive Screening Battery (BCSB) is a very useful tool for diagnosis of probable mild Alzheimer’s disease in a geriatric clinic. Arq Neuropsiquiatr. 2016;74(2):149-54. https://doi.org/10.1590/0004-282X20150202
https://doi.org/10.1590/0004-282X2015020... . In our study, we used the sum of the correct answers of the list of figures as indicative of immediate memory, the correct answers of the list of figures after verbal fluency and clock-drawing as late memory, the verbal fluency of animals as language, and the clock-drawing as executive functioning.

Five digit test

The Five digit test (FDT) was developed by Sedó²¹21 Sedó M. FDT. Test de los cinco dígitos. Madrid: TEA Ediciones; 2007 and consists of four stages: reading, counting, choice, and alternation. In the reading stage, the numbers must be read as fast as possible. In the counting stage, individuals are instructed to count the asterisks. In choosing, the participants are instructed to count the numbers that appear in the figure and they must inhibit the reading of the numbers. Finally, in the alternation phase, it is necessary for the participant to count how many numbers are in the quadrants, except in the quadrants with darker edges, which are requested to be read. The time of each stage is measured, providing measures of processing speed (reading and counting), selective attention/inhibitory control (choice), and alternating attention/cognitive flexibility (alternation)²²22 Campos MC, Silva ML, Florêncio NC, Paula JJ. Confiabilidade do teste dos cinco dígitos em adultos brasileiros. J Bras Psiquiatr. 2016;65(2):135-9. https://doi.org/10.1590/0047-2085000000114
https://doi.org/10.1590/0047-20850000001... . In our study, we used the sum of reading and counting as a measure of processing speed, the time of the counting stage as a measure of selective attention, and the time of the alternation stage as a measure of alternating attention.

Brief digital cognitive test

The brief digital cognitive test (BDCT) was elaborated by the researchers according to the paradigm of stimulus equivalence, using OpenSesame 3.1 software²³23 Mathôt S, Schreij D, Theeuwes J. OpenSesame: an open-source, graphical experiment builder for the social sciences. Behav Res Methods. 2012;44(2):314-24. https://doi.org/10.3758/s13428-011-0168-7
https://doi.org/10.3758/s13428-011-0168-... . The test consisted of two classes of stimuli with three stimuli in each class. Thus, Class 1 was composed of a house (stimuli A1), a square (B1), and one abstract figure (C1), while Class 2 was composed of a tree (A2), a triangle (B2), and another abstract figure (C2). The test was constructed with seven blocks (Table 1).

The training and test trials were made through simultaneous MTS trials, in which the model and comparison stimuli appeared on the screen at the same time. The screen of the portable computer was divided into quadrants so that in the middle of the two upper quadrants was the model stimulus and, in the lower quadrants, the two comparison models. In all blocks, the comparison stimuli were randomized to appear half of the time in the lower right-hand quadrant and the other half in the lower left-hand quadrant. In addition, the presentation of the relationships in each trial was randomized within each block. The choice of the comparison stimulus was made by touching the computer screen on the quadrant of the stimulus.

In the training blocks, the answers defined as correct produced a green “V” sign on the screen, and the answers defined as incorrect produced a red “X” sign. The duration of the feedback screen was 750 milliseconds. There was no feedback in the test blocks. Between each trial, a white screen appeared for 500 milliseconds.

At the beginning of the procedure, the participant was positioned in front of the screen of a portable computer and was instructed to perform the test according to the guidelines that would appear on this screen. Before each block, instructions were given to the participants, with the same characteristics, explaining the task to be performed and the presence or not of feedback.

The following variables of the brief digital cognitive test were considered in the study:

• Total number of correct answers: composed of the sum of correct answers in the seven blocks;

• Reflexivity: correct answers for the AA block;

• Baseline training: sum of the correct answers for blocks AB and AC;

• Symmetry: sum of the correct answers in the BA and CA relations;

• Baseline test: sum of the correct answers for AB and AC;

• Late baseline and symmetry: correct answers for blocks AB and BA;

• Transitivity: correct answers for blocks BC and CB; and

• Total test time (Figure 1).

Procedures

The participants were initially evaluated using a battery of traditional cognitive tests. Subsequently, the BDCT was applied through a portable computer with a 14-inch touchscreen. The assessment was performed at the participants’ homes or in health or social centers in an appropriate room, according to the participants’ preferences. The complete assessment lasted a mean of 60 minutes and was conducted individually. All volunteers consented to their participation by signing the Free and Informed Consent Form, which was approved by the Research Ethics Committee of the Federal University of São Carlos (CAAE: 12418019.0.0000.5504).

Data analysis

Descriptive analysis was performed to characterize the sociodemographic profile of the groups. The Kolmogorov-Smirnov test was applied to verify data normality. The differences between the groups with and without cognitive impairment were assessed by the Mann-Whitney test. The Spearman’s Correlation test was used to assess the correlation between the variables. The partial correlations were calculated to adjust the correlations by age and years of schooling. The area under the receiver operating characteristic (ROC) curve and the sensitivity and specificity for the best cutoff point of the total score of the digital test were calculated to discriminate between individuals with and without cognitive impairment. The best cutoff point was determined by the highest sum of sensitivity and specificity. The statistical analyses were performed using the Statistical Package for Social Sciences (SPSS), version 23.0. The level of significance considered was p<0.05.

RESULTS

Sociodemographic characteristics

The mean age of the participants was 71.23 years old (SD: ±9.36), with a median of 71 years old, varying from 60 to 100 years. Regarding the schooling level, 60% of the participants had between 0 and 4 years of study, 32% had between 5 and 8 years, and only 4 had more than 8 years. The mean schooling time was 7.15 years (SD: ±5.34), with a median of 4.5 years, varying between 0 and 18 years. Furthermore, 58% (n=29) were female and 60% (n=30) had no partner, belonging to the group of single, widowed, or divorced individuals.

Performance in the brief digital cognitive test

Of the 50 participants, 32 (64%) always clicked on the answer area and 45 (90%) clicked twice or fewer times outside the answer area. The mean of total correct answers in the test was 51.34 (SD: ±10.02), varying between 35 and 68, which corresponds to a mean rate of correct answers of 75.5%. The mean time to answer the test was 5.33 minutes (SD: ±1.92), varying from 2.43 to 9.34 minutes. Three older adults clicked outside the lower quadrants in more than 10% of the trials, so they were excluded from the inferential analyses. Table 2 presents the means and medians for each variable of the BDCT.

Concerning the transitivity block, it was observed that only seven older adults correctly answered 100% of this block and six answered incorrectly in only one trial. Considering that these participants formed equivalence classes, only 27.65% of the sample managed to form such classes.

Relation between age and schooling and variables of the brief digital cognitive test

Age showed a moderate negative correlation with the scores of the baseline training variable (r=-0.517; p<0.001), and a weak negative correlation with the scores of the variables: reflexivity (r=-0.355; p=0.014); symmetry (r=-0.384; p=0.008); transitivity (r=-0.360; p=0.013), and total score (r=-0.482; p=0.001). Conversely, it presented a weak positive correlation with total time (r=0.499; p<0.001).

In relation to years of schooling, there was a moderate positive correlation with the following variables: baseline training (r=0.575; p<0.001), total score (r=0.542; p<0.001), and total time (r=-0.530, p< 0.001). In addition, there was a weak positive correlation with the correct answers to reflexivity (r=0.462; p=0.001); symmetry (r=0.365; p=0.012); late baseline and symmetry (r=0.347; p=0.017), and transitivity (r=0.366; p=0.011).

Correlations between the cognitive domains and variables of the brief digital cognitive test

Table 3 shows Spearman’s correlations between the BDCT and cognitive variables.

Table 4 shows the partial correlations between the BDCT and cognitive variables adjusted by age and schooling years. All correlations between the total score of BDCT and the scores of general cognitions and domains remain statistically significant.

Variables of the brief digital cognitive test divided by groups with and without cognitive impairment

Table 5 presents the differences between the scores of the participants with and without cognitive impairment groups according to the MMSE scores. There was no difference between groups for age (p=0.118).

Considering the total score of the BDCT, the area under the ROC curve (Figure 2) was 0.765; 95% confidence interval (CI) 0.630–0.901, with the best cutoff point to discriminate participants with and without cognitive impairment being 53/54. At this cutoff point, sensitivity was 72%, and specificity was 68.2%.

DISCUSSION

Most of the correlations between the variables of the BDCT and the scores of the traditional cognitive tests were statistically significant. Moreover, there were significant differences between the scores of older adults with and without cognitive impairment.

Regarding the mechanics of the test, it was observed that only five participants clicked more than twice outside the answer area, which can be an indication that the majority understood the test. It is worth noting that the test presented always uses tasks with the same pattern (matching-to-sample), differently from other digital or computerized cognitive tests²⁴24 Ruano L, Sousa A, Severo M, Alves I, Colunas M, Barreto R, et al. Development of a self-administered web-based test for longitudinal cognitive assessment. Sci Rep. 2016;6:19114. https://doi.org/10.1038/srep19114
https://doi.org/10.1038/srep19114... –²⁶26 Memória CM, Yassuda MS, Nakano EY, Forlenza OV. Contributions of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI) for the diagnosis of MCI in Brazil. Int Psychogeriatr. 2014;26(9):1483-91. https://doi.org/10.1017/S1041610214000726
https://doi.org/10.1017/S104161021400072... that use different strategies to test cognition. One possibility that can be tested in the future would be not to allow the transition to the next trial if the participant clicks outside the answer quadrants.

The mean duration of the test was approximately five minutes, which is compatible with the brief and screening tests most used in our country¹⁷17 Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-98. https://doi.org/10.1016/0022-3956(75)90026-6
https://doi.org/10.1016/0022-3956(75)900... ,¹⁶16 Gallagher SM, Keenan M. Stimulus equivalence and the Mini Mental Status Examination in the elderly. Eur J Behav Anal. 2009;10(2):159-65. https://doi.org/10.1080/15021149.2009.11434316
https://doi.org/10.1080/15021149.2009.11... . The MMSE has a similar mean application time, while the Montreal Cognitive Assessment (MoCA) is applied in 5 to 10 minutes¹⁷17 Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-98. https://doi.org/10.1016/0022-3956(75)90026-6
https://doi.org/10.1016/0022-3956(75)900... ,²⁶26 Memória CM, Yassuda MS, Nakano EY, Forlenza OV. Contributions of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI) for the diagnosis of MCI in Brazil. Int Psychogeriatr. 2014;26(9):1483-91. https://doi.org/10.1017/S1041610214000726
https://doi.org/10.1017/S104161021400072... . However, these traditional tests are migrating to versions requiring the payment of a fee for use²⁷27 Molnar FJ, Benjamin S, Hawkins SA, Briscoe M, Ehsan S. One size does not fit all: choosing practical cognitive screening tools for your practice. J Am Geriatr Soc. 2020;68(10):2207-13. https://doi.org/10.1111/jgs.16713
https://doi.org/10.1111/jgs.16713... .

Concerning age and schooling, it was observed that there was a correlation between the BDCT scores and age and schooling, meaning that the higher the age, the lower the scores, and the higher the schooling, the higher the test scores. These findings are similar to those found in most traditional and computerized cognitive tests²⁸28 Darby DG, Pietrzak RH, Fredrickson J, Woodward M, Moore L, Fredrickson A, et al. Intraindividual cognitive decline using a brief computerized cognitive screening test. Alzheimers Dement. 2012;8(2):95-104. https://doi.org/10.1016/j.jalz.2010.12.009
https://doi.org/10.1016/j.jalz.2010.12.0... –³¹31 Bento-Torres NVO, Bento-Torres J, Tomás AM, Costa VO, Corrêa PGR, Costa CNM, et al. Influence of schooling and age on cognitive performance in healthy older adults. Braz J Med Biol Res. 2017;50(4):e5892. https://doi.org/10.1590/1414-431X20165892
https://doi.org/10.1590/1414-431X2016589... . One aspect that must be considered is that age and schooling are also related to difficulties in operating digital devices⁷7 Jenkins A, Lindsay S, Eslambolchilar P, Thornton IM, Tales A. Administering cognitive tests through touch screen tablet devices: potential issues. J Alzheimer’s Dis. 2016;54(3):1169-82. https://doi.org/10.3233/JAD-160545
https://doi.org/10.3233/JAD-160545... .

The study also evaluated the correlations between the test scores on paper and each stage of the BDCT. The ABBA block was the least correlated with the results of the traditional tests, having only correlated with the results of the MMSE and the FDT. This block was added as a trial to assess the memory of the first block, which was not observed in the results, since there was no correlation with the memory scores. Two possibilities must be considered for future adjustments to the test:

• Removing this block; or

• Including an instruction before this block since, in the test presented, the last three blocks have only one instruction, as shown in Table 1.

Another aspect observed was that the clock-drawing test may not be the most adequate to assess executive functioning in the sample of this study since this domain showed no correlation with three measures of the BDCT. Our sample included many participants with low schooling and it is known that the clock-drawing test presents limitations for assessing cognition in these populations³²32 von Gunten A, Ostos-Wiechetek M, Brull J, Vaudaux-Pisquem I, Cattin S, Duc R. Clock-drawing test performance in the normal elderly and its dependence on age and education. Eur Neurol. 2008;60(2):73-8. https://doi.org/10.1159/000131895
https://doi.org/10.1159/000131895... ,³³33 Fabricio AT, Aprahamian I, Yassuda MS. Qualitative analysis of the Clock Drawing Test by educational level and cognitive profile. Arq Neuropsiquiatr. 2014;72(4):289-95. https://doi.org/10.1590/0004-282x20140004
https://doi.org/10.1590/0004-282x2014000... . Another possibility would be that the tasks of forming equivalence classes involve more processes of memory, language, and attention than executive functioning. The moderate correlation found between this test and the reflexivity block must also be emphasized, pointing to a possible need for abstraction capacity to understand the mechanics of the BDCT.

In the correlation table, it was also possible to notice that the correlations between baseline training and traditional tests were greater than those between baseline and traditional tests, which can be referred to two main factors:

• The number of trials in each block — while the baseline tests consisted of eight trials in total, the baseline training consisted of 24 trials; and

• The presence of feedback — this facilitates the understanding of the task and possibly leads to the use of fewer cognitive resources.

However, these possibilities must be evaluated more systematically in the future.

Additionally, the baseline training blocks were the most correlated with the memory domains. This points out that these domains are essential in the formation of equivalence classes since, without baseline learning, the entire process will be compromised.

In relation to the total BDCT score, a moderate correlation was observed with all scores of the traditional tests, especially the correlations found with general cognition and language, except for the clock-drawing. The language test used in this study deals with the categorical verbal fluency of animals; therefore, it is a task that assesses the participant’s categorization ability more than the language itself. It is also known that the tasks of forming equivalence classes are closely related to categorization ability. The results also corroborate previous studies that pointed to the correlation between the general results in the MMSE and the formation of equivalence classes²¹21 Sedó M. FDT. Test de los cinco dígitos. Madrid: TEA Ediciones; 2007.

Although this study did not evaluate the diagnosis of each participant in relation to the diagnosis of mild or greater neurocognitive disorder, individuals were divided into two groups in order to determine the test’s ability to discriminate between subjects with and without cognitive impairment. It can be observed that there were differences between the groups for all stages of the digital test, except for transitivity, indicating that the test could be used in future research studies to evaluate its properties in clinical samples. Likewise, the area under the ROC curve is similar to that found in studies with other computerized cognitive tests validated to separate individuals with and without dementia⁸8 Oliveira RS, Trezza BM, Busse AL, Jacob-Filho W. Learning effect of computerized cognitive tests in older adults. Einstein (Sao Paulo). 2014;12(2):149-53. https://doi.org/10.1590/S1679-45082014AO2954
https://doi.org/10.1590/S1679-45082014AO... ,³⁰30 Rossetti HC, Lacritz LH, Munro Cullum C, Weiner MF. Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sample. Neurology. 2011;77(13):1272-5. https://doi.org/10.1212/WNL.0b013e318230208a
https://doi.org/10.1212/WNL.0b013e318230... ,³²32 von Gunten A, Ostos-Wiechetek M, Brull J, Vaudaux-Pisquem I, Cattin S, Duc R. Clock-drawing test performance in the normal elderly and its dependence on age and education. Eur Neurol. 2008;60(2):73-8. https://doi.org/10.1159/000131895
https://doi.org/10.1159/000131895... . In general, the sensitivity and specificity results were slightly lower when compared to other studies with traditional screening tests, although the values vary significantly across the studies³3 Abd Razak MA, Ahmad NA, Chan YY, Mohamad Kasim N, Yusof M, Abdul Ghani MKA, et al. Validity of screening tools for dementia and mild cognitive impairment among the elderly in primary health care: a systematic review. Public Health. 2019;169:84-92. https://doi.org/10.1016/j.puhe.2019.01.001
https://doi.org/10.1016/j.puhe.2019.01.0... . However, it must be considered that we used a small sample compared to other validation studies.

An important limitation of our study is its non-probabilistic sample; therefore, it is possible that individuals who did not have ease handling electronic devices may not have volunteered for the research. Despite this, individuals with different schooling levels were included. In addition to that, as a criterion of cognitive impairment, we only used the MMSE cutoff points and the clinical diagnosis could be more reliable to separate these groups and determine discriminant validity. Despite this, we must emphasize that the BDCT proposal is easy to apply and can be available on different platforms, considering its extremely simple dynamics.

In conclusion, the BDCT with the use of the paradigm of stimulus equivalence is an easy-to-apply and valid instrument for the investigation of cognitive impairment in older adults. Future studies must be conducted to analyze different arrangements of instruments using this paradigm such as the number of trials per block, test stages, and presence of feedback.

REFERENCES

Publication Dates

History