Digital Cognitive Assessment Tests for Older Adults: Systematic Literature Review

Background The global health pandemic has affected the increasing older adult population, especially those with mental illnesses. It is necessary to prevent cases of cognitive impairment in adults early on, and this requires the support of information and communication technologies for evaluating and training cognitive functions. This can be achieved through computer applications designed for cognitive assessment. Objective In this review, we aimed to assess the state of the art of the current platforms and digital test applications for cognitive evaluation, with a focus on older adults. Methods A systematic literature search was conducted on 3 databases (Web of Science, PubMed, and Scopus) to retrieve recent articles on the applications of digital tests for cognitive assessment and analyze them based on the methodology used. Four research questions were considered. Through the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology, following the application of inclusion and exclusion criteria, a total of 20 articles were finally reviewed. Results Some gaps and trends were identified regarding the types of digital applications and technologies used, the evaluated effects on cognitive domains, and the psychometric parameters and personal characteristics considered for validation. Conclusions Computerized tests (similar to paper-and-pencil tests) and test batteries (on computers, tablets, or web platforms) were the predominant types of assessments. Initial studies with simulators, virtual environments, and daily-life activity games were also conducted. Diverse validation methods and psychometric properties were observed; however, there was a lack of evaluations that involved specific populations with diverse education levels, cultures, and degrees of technology acceptance. In addition, these evaluations should consider emotional and usability aspects.


√
Selection process 8 Specify the methods used to decide whether a study met the inclusion criteria of the review, including how many reviewers screened each record and each report retrieved, whether they worked independently, and if applicable, details of automation tools used in the process.
There were four reviewers: the two authors, and two student assistants.Everyone worked independently, and no automation tools were used in the process.

Data collection process
9 Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process.
There were four reviewers: the two authors, and two student assistants.Everyone worked independently, and no automation tools were used in the process.

√
Data items 10a List and define all outcomes for which data were sought.Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect.
The outcomes for which data were sought are specified in the PRISMA diagram in Figure 3, and their explanation in the "Data Collection" paragraph.

√
10b List and define all other variables for which data were sought (e.g.participant and intervention characteristics, funding sources).Describe any assumptions made about any missing or unclear information.
The other variables for which data were sought are specified in the "Methods" Chapter, and were based on the research questions.

Item # Checklist item
Location where item is reported Study risk of bias assessment 11 Specify the methods used to assess risk of bias in the included studies, including details of the tool(s) used, how many reviewers assessed each study and whether they worked independently, and if applicable, details of automation tools used in the process.
There were four reviewers: the two authors and two student assistants.Everyone worked independently, the risk of bias assessment was based on the experience of each reviewer, and no automation tools were used in the process.

√
Effect measures 12 Specify for each outcome the effect measure(s) (e.g.risk ratio, mean difference) used in the synthesis or presentation of results.
For the synthesis and presentation of the results, several comparative tables were considered that are included in the text of the Review.Even a Table in Multimedia Appendix 1.

√ Synthesis methods
13a Describe the processes used to decide which studies were eligible for each synthesis (e.g.tabulating the study intervention characteristics and comparing against the planned groups for each synthesis (item #5)).
The decision of which studies were eligible for each synthesis is described in the Methods Chapter paragraphs, and is based on the eligibility criteria, and how they satisfied the research questions.

√
13b Describe any methods required to prepare the data for presentation or synthesis, such as handling of missing summary statistics, or data conversions.
For the synthesis and presentation of the data, comparative tables were used, using summary statistics of the studies, and how they satisfied the research questions.

√
13c Describe any methods used to tabulate or visually display results of individual studies and syntheses.
To tabulate or visually display the results of studies and individual syntheses, electronic spreadsheets were used.And several tables are included in the Review.

√
13d Describe any methods used to synthesize results and provide a rationale for the choice(s).If meta-analysis was performed, describe the model(s), method(s) to identify the presence and extent of statistical heterogeneity, and software package(s) used.
To synthesize the results and justify the choice, eligibility and inclusion criteria were used, according to the research questions and the objective of the Review.A meta-analysis was not performed as it was not included as an objective in the Review.

√
13e Describe any methods used to explore possible causes of heterogeneity among study results (e.g.subgroup analysis, meta-regression).
To explore possible causes of heterogeneity between the results of the studies, the eligibility and inclusion criteria were used, according to the research questions, and the objective of the Review.

√
13f Describe any sensitivity analyses conducted to assess robustness of the synthesized results.
To evaluate the robustness of the synthesized results of the studies, sensitivity and specificity analyzes were done, which are explained in Multimedia Appendix 1.

Reporting bias assessment
14 Describe any methods used to assess risk of bias due to missing results in a synthesis (arising from reporting biases).
To assess the risk of bias due to possible missing results, we involved four reviewers: the two authors and two student assistants.Everyone worked methodically and independently, and the risk of bias assessment was based on the experience of each reviewer.

Certainty assessment
15 Describe any methods used to assess certainty (or confidence) in the body of evidence for an outcome.
To evaluate the certainty or confidence in the body of evidence for an outcome, four reviewers participated: the two authors and two student assistants.Everyone worked methodically and independently, and the assessment of certainty was based on the experience of each reviewer .The results of the search and selection process are described in the diagram in Figure 3.

√
16b Cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded.
The studies that were excluded from the selection and why, are shown in the diagram in Figure 3.

√ Study characteristics
17 Cite each included study and present its characteristics.
As explained in the Review, the included studies were 20, and their main characteristics are shown in Tables 1, 5 and 9 of the Review.

√
Risk of bias in studies 18 Present assessments of risk of bias for each included study.
Of the 20 studies included, and their possible risk of bias, their evaluation is explained in the paragraphs of the Results and Discussion Chapters.

Results of individual studies
19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g.confidence/credible interval), ideally using structured tables or plots.
For the results of the studies, a set of statistics and characteristics of all the digital cognitive tests analyzed are presented, using tables structured in the Review.In particular, Tables 5, 7, 8 and 9.

Results of syntheses 20a
For each synthesis, briefly summarise the characteristics and risk of bias among contributing studies.
The characteristics of the contributing studies are summarized in all Tables (1 to 9).

√
20b Present results of all statistical syntheses conducted.If meta-analysis was done, present for each the summary estimate and its precision (e.g.confidence/credible interval) and measures of statistical heterogeneity.If comparing groups, describe the direction of the effect.
Los resultados de las síntesis estadísticas realizadas, se presentan en las Tablas (1 a 9).A meta-analysis was not performed as it was not included as an objective in the Review.

√
20c Present results of all investigations of possible causes of heterogeneity among study results.
The results of all the investigations on possible causes of heterogeneity between the results of the studies can be seen in Tables 1, 4, 5 and 9 of the Review.

√
20d Present results of all sensitivity analyses conducted to assess the robustness of the synthesized results.
To evaluate the robustness of the synthesized results of the studies, sensitivity and specificity analyzes were done, which are explained in Multimedia Appendix 1.

√
Reporting biases 21 Present assessments of risk of bias due to missing results (arising from reporting biases) for each synthesis assessed.
To assess the risk of bias due to possible missing results, we involved four reviewers: the two authors and two student assistants.Everyone worked methodically and independently, and the risk of bias assessment was based on the experience of each reviewer.Assessments of risk of bias due to possible missing results are discussed in the Results and Discussion Chapters.

Certainty of evidence
22 Present assessments of certainty (or confidence) in the body of evidence for each outcome assessed.
To evaluate the certainty or confidence in the body of evidence for an outcome, four reviewers participated: the two authors and two student assistants.Everyone worked methodically and independently, and the assessment of certainty was based on the experience of each reviewer. √ results of the search and selection process, from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram.