Validity Evidence of an Epidemiological Oropharyngeal Dysphagia Screening Questionnaire for Older Adults

OBJECTIVE: This research aimed to identify evidence of validity of a self-reported Oropharyngeal Dysphagia screening questionnaire for older adults based on test content, response processes, internal structure, relations to other variables, and reliability. METHOD: This is a nonrandomized, cross-sectional study employing the concepts and principles of the Standards for Educational and Psychological Testing. Data were collected from 644 community-dwelling older adults (both genders, age: ≥60 years) who agreed to participate in some steps of the validation process. Statistical methods obtained the content representation of the construct, internal structure validation, discriminant and convergent evidence, and reliability, using a 5% significance level. RESULTS: The screening tool was re-specified in nine questions that provided the best fit and robust reliability, with proper discriminant and convergent evidence. CONCLUSIONS: The screening questionnaire presented valid and reliable results to identify oropharyngeal dysphagia symptoms in older adults, highlighting the importance of the validation process based on the standards to construct an epidemiological instrument.


' INTRODUCTION
Oropharyngeal dysphagia (OD) is a swallowing disorder that may indicate disorder in oral, pharyngeal, or oropharyngeal functioning during the swallowing process. It is associated with noticeable symptoms reported by the individual on ingesting food (1), which presents issues surrounding weight loss, malnutrition, dehydration, and episodes of aspiration pneumonias. These issues are of great concern in the healthcare of older adults. OD has an impact on public health as it can increase the morbidity and mortality of older adults (2).
This clinical condition can increase functional disability and frailty, hospital readmissions, mortality, and the institutionalization of older adults. OD has been recognized as a critical geriatric syndrome by two European Societies (2,3).
The prevalence of OD in older people who live in the community varies between 11% and 38% (4)(5)(6)(7)(8). However, the studies that have reported prevalence did not use screening instruments for epidemiological diagnosis with validity evidence for the older adult population (9).
For a screening instrument to become both epidemiologically relevant and scientifically robust, it must be developed and psychometrically tested for its validity. It should measure what it intends to achieve and must be reliable in producing consistent and reproducible data (10).
Hence, this study aimed to identify the evidence of validity in constructing an epidemiological self-reported Oropharyngeal Dysphagia screening questionnaire for older adults.
' MATERIAL AND METHODS This nonrandomized and cross-sectional study evaluated the evidence of validity according to the test content, response processes, internal structure, relations to other variables and reliability (11).
Data were collected from March 2013 to April 2017. The adjusted versions for each stage of this validation process were used. Participants of either gender aged 60-95 years with preserved cognitive functioning were included in the study. The mean age varied depending on when the data was collected.
This tool aimed to be an epidemiological screening instrument for OD in older adults with no specific diseases. This process excluded participants who were previously diagnosed with comprehensive, psychiatric, neurological, and neurodegenerative disorders or head and neck cancer, who were tracheostomized, who had moderate-to-severe hearing loss (with no hearing aids), or who used feeding tubes.
The study participants were recruited by convenience from social centers for older adults, one long-term care institution for older adults, and the waiting room of a Geriatric and Otorhinolaryngology Outpatient in the University Hospital, Natal, Rio Grande do Norte, Brazil.

Evidence of validity based on test content
An expert technical group (ETG) composed of five researchers (who are the authors of this study) created the initial version of the questionnaire through a free association of ideas based on the most common signs and symptoms of swallowing disorders.
The theoretical basis of this study defined OD as a swallowing disorder resulting from a broad spectrum of disturbances in the interaction between the oral and pharyngeal phases. This disorder can lead to complications in pulmonary and nutritional status (12).
The ETG considered the following qualitative aspects: the educational level of the elderly; how long it took for respondents to answer the questions; the presence of ambiguous words, vague terms, or (medical) jargon; and items with two or more questions.
The initial version of the questionnaire consisted of 17 items focusing on sensations and perceptions related to swallowing disorders of the oral and/or pharyngeal phases. This was judged by a multidisciplinary expert committee (EC) who evaluated the adequacy of the questions. The EC consisted of 22 speech pathologists (3 gerontologists, and 19 dysphagia specialists), five geriatricians, two dietitians, one nurse, one otorhinolaryngologist, and one gastroenterologist.
The analysis used the Content Validity Index by Item (CVI-I) and the Content Validity Index (CVI) to verify the level of agreement of the EC evaluations, estimated as X0.80 and X0.90, respectively. The conceptual framework refinement in a single round of consultations was considered (10). The ETG also considered the suggestions of the EC to modify or insert some questions (13,14).

Evidence of validity based on response processes
After the analysis by the EC, a second version with 16 items was applied to 40 older adults from different social strata (Table 1), according to criteria from a previous study (10).
The participants were asked about the clarity of each item, followed by a request to repeat how they had understood it in a loud voice. Indirect strategies were used to observe their response time and non-verbal reactions to identify problematic sentences that had issues surrounding their meaning or interpretation (15).

Evidence Based on Internal Structure
This version of the epidemiological screening questionnaire, with 14 questions, was applied to 211 older adults, of whom 63 (29.9%) were male and 148 (70.1%) were female (mean age=71.2; SD=7.8).
This evidence was obtained by confirmatory factorial analysis by structural equation modeling (CFA-SEM). The grouping of the questions was organized into two factors, based on highlights from the literature about what each item could mean as an outcome of deglutition disorders ( Table 2). This resulted in a hypothetical model (16).
The CFA-SEM was performed using MPlus 8.0 version software, applying the Robust Weighted Least Squares (WLSMV). This performs well with small samples (nX200) and ordinal categorical variables (23).
The quality of each factor of the model was also analyzed to evaluate the possibility to eliminate any of the variables, considering: an average variance extracted (AVE) X0.60, a mean of the standard factorial loads ( € Xl) X0.70, and composite reliability coefficients (CR) X0.91 for a 10-variables-model (or close to 0.90 if the adjusted model becomes less than 10 variables) (26).

Evidence Based on Relations to Other Variables and Reliability
This stage considered convergent and discriminant validity. A well-fitting model with nine items was applied to 393 older people (mean age=70.4; SD=7.9), with 175 (44.5%) males, and 218 (55.5%) females.
When considering convergent validity, the Associated Factors Questionnaire was used with dichotomized responses ("yes 1 or no 0 "). This questionnaire addressed the presence of comorbidities, discomfort symptoms, negative lifestyle, and inferior self-perception of functionality. This was measured by the sum of the total scores of the worst outcomes.
Convergent validity was verified based on the assumed hypothesis that older people with more referred associated factors had worse screening results, as analyzed by the Spearman's coefficient of correlation (r).
Discriminant validity was assessed by the validated and adjusted Rosenberg Self-esteem Scale (RSES) for Brazilian older adults (29). This was used with part of the sample (n=110; mean age=71.8; SD=8.2), including 30 (27.3%) males and 80 (72.7%) females. The objective was to prove that there was a weak or non-correlation through the Spearman's coefficient.
The reliability of the test-retest reproducibility was assessed using part of the sample (n=75; mean age=70.5; SD=7.1), in which 28 (37.3%) were males, and 47 (62.7%) were females. They came back to a second interview within a period of 5 to 14 days (25).
The statistical reliability analysis used the intraclass correlation coefficient (ICC) of the instrument (scored from 0 to 18) (30), the Standard Error of Measurement (SEM) and the Smallest Real Difference (SRD) (31). The internal consistency was tested using Cronbach's alpha X0.70 (25).

Ethics
The present study complies with all recommended ethical guidelines for research involving human subjects and was approved by the Institution's Ethics and Research Committee (approval number: 1,144,297). '

Evidence of validity based on content
The CVI was 0.72, below the considerable minimum value to affirm that the instrument has good conceptual representation, pertinence, clarity, and comprehensiveness of the questions. This allowed the verification of the CVI-I of each question (Table 3) followed by descriptions of the adjustments made.
The ETG eliminated four questions from the first version (2, 10, 12, and 15) for presenting low CVI-I (first column of Table 3), without providing suggestions for modifications that may improve clarity. The EC suggested restructuring other questions for being considered theoretically relevant.
Despite presenting a good CVI-I, the experts eliminated the fourth question (first column of Table 3) due to focusing on chewing. According to the authors' understanding, chewing is a subject that involves other domains beyond screening for OD and is the study of another screening instrument (32).

Evidence of validity based on response processes
Observing the new order of the 16 items (fourth column of Table 3), the ETG deleted two questions. The ninth question was deleted because more than 70% of the older adults misunderstood if the question was asking about poor consumption of liquid or their weak preference to drink water. The 16 th question was deleted for generating confusion with the 15 th question.
The questions from the second version (fourth and fifth column of Table 3) that generated the most doubt or confusion or created delay in responding among all groups were 01, 02, 05, 08, 10, 12 and 13. Due to the considerations and reflections regarding the clarity and construction of the questions, the authors decided to modify and reorder the questions based on a sequence of symptoms to facilitate their interpretation with small adjustments in the syntactic construction (Table 2).

Evidence of validity based on the internal structure
The 14-question-model with two factors had no proper adjustment for factor 1 (AVE=0.45, CR=0.86; € Xlf1=0.58), which reflected small loads of some items, resulting in the elimination of variables x1, x4, x5, x6, and x7.
After the adjustments, a new structural model reached the well-fit model when compared to the first model (Table 4), as shown in Figure 2.
The 9-item version of the instrument showed robust results for each item as observed from test-retest reliability

Evidence Based on Relations to Other Variables
The convergent validity had a positive and moderate correlation (r=0.43; po0.001) but the discriminant evidence did not demonstrate correlation (r=-0.06; p=0.6).

' DISCUSSION
The results presented in this study are the first to investigate the evidence of validity for the OD screening questionnaire. The investigation was based on the recommended guidelines by the standards that suggest robust premises to be followed from a psychometric perspective (11). This self-reported OD screening questionnaire is designed for older adults living in the community with asymptomatic or initial symptoms. The questionnaire was intended to identify the initial diagnosis of this clinical condition, requiring an immediate diagnostic confirmation, leading to a therapeutic decision (33).
Acquiring evidence of validity, based on content, was an essential step towards obtaining accurate results. The process included the quantitative and qualitative evaluation of each question, along with suggestions from the participants for modifications and further inclusion of relevant items to develop an epidemiological screening tool that could be applied in the older adult population.
The results analysis allowed for the adjustment of semantic, syntactic, and contextual aspects of the initial versions of the instrument. This was essential for the clarification and representativeness of the instrument's relevance to screen for OD.
The quantitative and qualitative evaluations from the ETG, the data from the EC, as well as information from the sample in the response processes favored the modification of the questions. The changes resulted from integrating the given suggestions with empirical observations discussed according to the construct definition (13).
Regarding the evidence of validity related to the response processes, not one screening instrument for OD in this population described this step; therefore, comparisons of the results to other studies was not possible (9). Two studies (34,35) demonstrated their screening tool validation. The first (34) describes a preliminary tool that requires a confirmatory factor analysis to finish its construct validation based on its internal structure. The second (35) provided no evidence on the response processes that collected data on the reactions of the respondents when presented with the questions or the internal structure.
The results emphasize the importance of including detailed descriptions of these steps in studies that are aimed at obtaining evidence of validity from diagnostic tools. This should be considered from the external perception of experts (e.g., the EC and ETG) and the internal perception of who is under evaluation (36).
The findings were relevant, highlighting the importance of describing how the screening tool was constructed and adjusted. This favored the perception of continuous movement in the process of obtaining evidence of validity with interpretations that generated valid and reliable arguments (15).
Evidence of validity, related to the internal structure, wellfitted in nine questions, presented satisfactory quality of the adjustments that were based on the recommendations of the absolute fit indexes (w 2 /df; RMSEA) and incremental indexes (CFI; TLI). This was confirmed by robust indexes of internal structure, consistency, and reliability (26,31).
There are self-reported questionnaires in the literature that presented no clear evidence of their validation process for OD screening in older adults. One of these questionnaires is being used as a screening tool. However, the author has described that its use was to document the initial severity of dysphagia and to monitor the response to treatment in people with a wide range of swallowing disorders (21).
Another study (37) considered the population of older adults in the community; however, there were failures in the methodological description within the analysis parameters that were already discussed in the literature to obtain evidence of validity, reliability, and accuracy (11).  Therefore, this OD screening questionnaire may be considered as the first and, for the time being, the only selfreported questionnaire from the epidemiological perspective that can generate information to provide valid results to identify OD in older adult residents of the community who are asymptomatic or presenting initial symptoms.
The tool's reliability measures were consistent. This indicated that there was a small variability in the scores between the subjects, thus, resulting in adequate stability, and nonsignificant measurement error. Hence, there is a good homogeneity of the items concerning the construct (25).
The convergent validity was significant. This confirms what was found in other studies: only a few factors were associated with OD (e.g., depression (5,6,8); stroke; neck, throat and shoulder tension; limited food intake; social withdrawal; perception as inactive; and heart disease with complaints of difficulty in swallowing (4)).
There are no instruments that present valid and reliable results to establish the prevalence of OD in the older adult residents in the community. The results presented in this study reinforce the need to establish these parameters in population studies.
Regarding the discriminant validity, this screening tool does not have an association with the RSES. This highlights that its measure is empirically unique and represents a phenomenon of distinct interest from this other construct.
A limitation of the present study was the exclusion of older people with cognitive impairment, hearing loss, a history of head and neck cancer, and an absence of satisfactory verbal communication. This can compromise the analysis of the dysphagia scenario in the older adult population living in the community. As the population of older adults with dementia is increasing (37), it is necessary to consider caregivers who can contribute to this screening tool and identify OD in those who lack verbal communication or have cognitive decline.
The final version of this epidemiological screening questionnaire was named according to its initials in the Brazilian Portuguese title: ''RaDI'' (Rastreamento de Disfagia Orofaríngea em idosos -Self-reported Oropharyngeal Dysphagia Screening for Older Adults) (Appendix).

' CONCLUSIONS
The 9-question RaDI, developed in Brazilian Portuguese, presents evidence of validity based on the test content, response processes, internal structure, and relations to other variables.
This screening questionnaire is a simple and time-effective instrument created for epidemiological purposes to detect OD symptoms and initiate actions aimed at caring for older adults. The instrument can be used by any health professional working with older adults.

' ACKNOWLEDGMENTS
We thank the participants of social convenience centers who agreed to spend their time voluntarily answering the questionnaire for our data collection. This study was financed in part by the Coordenac¸ão de Aperfeic¸oamento de Pessoal de Nível Superior -Brasil (CAPES) -Finance Code 001.

' AUTHOR CONTRIBUTIONS
All authors affirm that they have contributed to the development of the manuscript, were involved in all stages of the development, and have approved the submitted manuscript. Magalhães Junior HV conceived and designed the study, collected the data, performed the analysis, verified the analytical methods, evaluated the study findings, discussed the results and conclusions and wrote the manuscript. Pernambuco LA contributed to the study conceptualization, formal analysis, and validation of the methodology. Cavalcanti RVA supported the study conceptualization, investigation, and methodology and contributed to the visualization and manuscript review. Lima KC contributed to the study conceptualization, formal analysis, methodology, and project administration. He lead the study validation and supervised and provided support in the writing of the original draft, review, and editing. Ferreira MAF contributed to the study conceptualization, formal analysis, methodology, and project administration. She also lead the study validation and supervised and provided support in the writing of the original draft, review, and editing.