Development and validation of a questionnaire to assess the knowledge of mechanical ventilation in urgent care among students in their last-year medical course in Brazil

OBJECTIVE: To develop and validate a questionnaire to assess the knowledge of mechanical ventilation among final-year medical students in Brazil. METHODS: A cross-sectional study conducted between October 2015 and October 2017 involving 554 medical students was carried out to develop a questionnaire for assessing knowledge on mechanical ventilation. Reproducibility was evaluated with the intraclass correlation coefficient, internal consistency was evaluated with Cronbach’s alpha, and construct validation was evaluated with a tetrachoric exploratory factor analysis. To compare the means of the competences among the same type of assessment tool, the nonparametric Friedman test was used, and the identification of the differences was obtained with Dunn-Bonferroni tests. RESULTS: The final version of the questionnaire contained 19 questions. The instrument presented a clarity index of 8.94±0.83. The value of the intraclass correlation coefficient was 0.929, and Cronbach’s alpha was 0.831. The factor analysis revealed five factors associated with knowledge areas regarding mechanical ventilation. The final score among participants was 24.05%. CONCLUSION: The instrument has a satisfactory clarity index and adequate psychometric properties and can be used to assess the knowledge of mechanical ventilation among final-year medical students in Brazil.


' INTRODUCTION
The increased time patients spend on mechanical ventilation in the emergency department and the inadequate approach to mechanical ventilation increase mortality and length of hospital stay (1). In addition, many patients require prolonged acute mechanical ventilation (496 hours) in the emergency department, and physicians from multiple specialties take care of these patients (2). Many of these physicians feel uncomfortable when manipulating these patients and often transfer this responsibility to other professionals (3).
In Brazil, many newly graduated physicians work in the emergency room with severe mechanical ventilation patients but have little training in this field (4). There is no existing literature that has evaluated their knowledge of mechanical ventilation.
The objective of this study was to develop and validate an instrument to evaluate the knowledge of mechanical ventilation in the emergency room by a final-year medical students in Brazil.
Medical specialists who published books, chapters or articles on mechanical ventilation or who had teaching experience in intensive care, pulmonology or anesthesiology were considered. The second sample included 60 students in their last-year medical course at UNIFESP and 60 intensive care unit (ICU) physicians with specialist degrees (validation cohort group). The third sample involved 554 medical students in their sixth-year undergraduate course (medical students group). All of the questionnaire respondents signed the voluntary informed consent form before participation in the study.
The validation methods followed the Consensus-based Standards for the Selection of Health Status Measurement Instruments (COSMIN) standard checklist (5).
Regarding content validation, the educational objectives were developed through the Delphi technique (6). The following subjects were selected: respiratory physiology, beginning and maintenance of mechanical ventilation, modes and modalities of mechanical ventilation, and complications and monitoring.
To evaluate the clarity and relevance of each item of the instrument, an interval score of 1 to 10 was created for the two components. For clarity, a score between 1 and 4 was considered confusing; a score between 5 and 7 was considered not very clear; and a score between 8 and 10 was considered clear. For relevance, a score between 1 and 4 was considered irrelevant; a score between 5 and 7 was considered not very relevant; and a score between 8 and 10 was considered relevant. The items were considered for the final version only if they achieved a core of 8-10 points.
Below each component, there was a space for suggestions regarding the content and semantics of the proposed items. The indexes of clarity and relevance were obtained through the mathematical averages of the sums of the score given by the professionals. The statements with relatively low clarity and/or relevance (index below 8.0) were replaced or reworded.
The second version of the questionnaire was evaluated for clarity by twenty medical students with a method similar to that used by the health professionals. The items with scores equal to or less than 8.0 were reformulated, and the final version of the questionnaire, which underwent a process of construct validation and reproducibility, was generated. The established scores for the questions were as follows: correct=1; incorrect or do not know=0. The sum of the question scores represented the final score (7).
The participants in the expert group, sixty medical students and sixty physicians board-certified in intensive medicine, answered the final version of the questionnaire. The final scores of the two groups were compared using the Mann-Whitney test.
Reproducibility was evaluated with 60 (expert group) medical students who agreed to answer the questionnaire twice, with an interval of 14 days (test and retest). The statistical analysis was performed using the interclass correlation coefficient (8). The students did not receive specific information on mechanical ventilation during this interval or any information regarding their performance. For the interclass correlation coefficient analysis, the total scores generated by the instrument were used based on a value higher than 0.8.
The third sample (medical students group) comprised students from 10 educational institutions in Brazil: UNIFESP,

Construct Validation
To validate the construct, an exploratory factor analysis was performed based on the tetrachoric correlation matrix to evaluate the dimensionality suggested by the items of the dichotomic questionnaire. The exploratory factor analysis was performed with the main component method and VARIMAX orthogonal rotation (9,10).
The overall and the subdimension internal consistency was analyzed via Cronbach's alpha coefficient (11).
The sum of the scores of the correct answers in the respective dimensions were generated and were rescaled in such a way that they varied from 0 (minimum) to 100 (maximum).
Once mean differences were detected, the differences were identified with Dunn-Bonferroni tests, with the level of global significance maintained.
The categorical items described were as follows: program schedule with or without mechanical ventilation course; hours spent searching for mechanical ventilation information; number of patients on mechanical ventilation assisted per week; level of comfort in the handling of patients on mechanical ventilation; and professional who handles patients on mechanical ventilation in the emergency room.

' RESULTS
The expert group consisted of 18 specialists; after three rounds of item evaluation, the questionnaire reached clarity and relevance indexes ranging from 8-10. The final questionnaire was answered by the validation cohort comprising 60 medical students from UNIFESP and 60 attending physicians. The final scores among the members of the expert group were compared using the Mann-Whitney test, shown in Table 1.
Subsequently, the questionnaire was answered by 554 medical students from eleven medical schools (Table 2). The number of questionnaires administered was 592; 554 were included. Only those with at least 90% completion were considered.
Reproducibility was tested with the interclass correlation coefficient and had a value of 0.929.
Among the respondents, 15.2% obtained the minimum score of ''zero'', and none reached the maximum score of ''twenty''. The loss percentage was 2.2% among all the questionnaires considered. Of the sample, 83% of the students did not have any mechanical ventilation course on their medical school schedule. The level of knowledge based on the percentages of the final scores is presented in Figure 1. The scores obtained by the students, stratified by different factors, are shown in Table 3.
Moderate positive correlations were found between the level of knowledge and the following variables: hours of mechanical ventilation in the undergraduate course (rho=0.552, po0.001) and information on mechanical ventilationfrom other sources (rho=0,506, po0.001). The professionals who initiated mechanical ventilation in the emergency department were physiotherapists according to 63% of the respondents, and 82.5% of the respondents never participated in the care of a patient on mechanical ventilation.
The exploratory factor analysis is shown in Table 3 and contains the following factors: acute respiratory distress syndrome (ARDS) factor, chronic obstructive pulmonary disease (COPD) factor, complications factor, modality factor, and respiratory factor. We observed the existence of five factors that together accounted for 86.5% of the total explained variance among the items; the eigenvalues of which were higher than 1.0. Item 11 was excluded due to low commonality.
The first factor, the ''ARDS factor'', covered five items involving the following areas of knowledge: the concept of plateau pressure, the concept of respiratory system compliance, positive end-expiratory pressure (PEEP), and mechanical ventilation strategies in ARDS patients. The items explained 24.9% of the total variance.
Factor two, the ''COPD factor'', covered four items: the concept of auto-PEEP, auto-PEEP complications, behavior of patient on auto-PEEP in the emergency room, principles of ventilation in COPD patients, and indications for noninvasive ventilation. The third factor, the ''Complications factor'' covered five items: complications of mechanical ventilation, concept of airway resistance, arterial blood gas testing, and auto-PEEP measurements. The fourth factor, the ''modality factor'', covered three items involving concepts of the most commonly used modalities. The last factor, the ''respiratory factor'', involved concepts of respiratory physiology. After obtaining the different knowledge factors in the tetrachoric factor analysis, the means of each factor were compared with the nonparametric Friedman test due to the violation of the assumption of normality. In the distribution of scores, the Figure 1 -Students' knowledge of mechanical ventilation. Level 1: p25% of the score, Level 2: 425% but p50% of the score, Level 3: 450% but p5% of the score, Level 4: 475% to 100% the score.
highest performance was on ''factor 5'' and the lowest was on ''factor 1'' (Table 4). The internal consistency of the instrument was measured by Cronbach's alpha and had a value of 0.831.

' DISCUSSION
Previous studies have developed instruments for assessing the knowledge of mechanical ventilation among emergency and clinical residents and nurses (12)(13)(14)(15). Our study is the first to validate an instrument to assess the knowledge of mechanical ventilation among medical students.
The data on the internal consistency and reproducibility demonstrated the homogeneity and stability of the instrument and the possibility of obtaining similar and accurate results (16).
The tetrachoric exploratory factor analysis revealed five factors, a finding that demonstrates the multidimensional nature of knowledge on mechanical ventilation. The clusters were related to specific aspects, such as mechanical ventilation in ARDS, COPD, complications of mechanical ventilation, physiology, and the beginning and maintenance of mechanical ventilation.
A very low final score average among the students was observed. This level of knowledge was associated with the absence of self-reported mechanical ventilation teaching programs. The instrument showed that the scores increased as the number of hours of mechanical ventilation activities performed by students increased.
The factor related to physiology obtained the best score. However, the factors that were related to specific knowledge of mechanical ventilation obtained very low scores. The poor performance in factor three, which was related to complications in patients on mechanical ventilation, is highlighted.
Our study has some limitations; the sample of students was not probabilistic, although the study involved a large number of students. Moreover, the relations of the scores measured by the instrument and the time spent learning about mechanical ventilation were based on self-reported, nonmeasured information.
In conclusion, the questionnaire called ''Questionnaire on the knowledge of mechanical ventilation in the emergency room by sixth-year medical school students in Brazil'' presented the psychometric properties necessary to serve as an evaluation tool for this population.

' AUTHOR CONTRIBUTIONS
Tallo FS was responsible for the study conception, data curation, formal analysis, funding acquisition and study investigation. Lopes RD was responsible for the study methodology and support, and manuscript writing and review. Abib SCV supervised the study and was responsible for study support and manuscript writing and review. Baitello AL was responsible for the study investigation and support, project administration and support, manuscript drafting and writing.   A. A decrease in consciousness due to hypercapnia contraindicates the use of NIPPV B. The main monitoring parameter should be TV C. Eye irritation and claustrophobia are the most frequent complications D. The strategy of high flow prompts the rapid relief of respiratory distress E. I don't know