Skip to main content
Download PDF

Developing and validation of COVID-19 media literacy scale among students during the COVID-19 pandemic

BMC Psychology volume 11, Article number: 315 (2023) Cite this article

Abstract

Objective

This cross-sectional validation work evaluated the psychometric features of the COVID-19 Media Literacy Scale (C-19MLs) in Students.

Methods

The study was conducted on 530 students from a medical university in Hamadan, Iran, who were recruited through a stratified cluster random sampling process in June-July 2020. Intraclass Correlation Coefficient (ICC) and internal consistency were used to assess the reliability. Moreover, CFA (Confirmatory Factor Analyses) and EFA (Exploratory Factor Analyses) were carried out to examine construction validity. CVR (Content Validity Ratio) and CVI (Content Validity Index) were used to examine the content validity.

Results

According to the factor analysis, it was indicated that the C-19MLs included 21 items measuring five dimensions (constructedness of credible Covid-19 media messages, contractedness of fake media coronavirus messages, fake media coronavirus messages, audience, with three questions in each factor; format, represented lifestyles in fake media coronavirus messages with six questions in each factor) for an explanation of 58.4% of the prevalent variance. The average scores for the CVI and CVR were respectively 0.94 and 0.77. According to confirmatory factor analysis, the studied model had an appropriate fitting to the data; the relative chi-square (x2/df) = 2.706 < 3, RMSEA = 0.093 ≤ 0.1; CFI = 0.893 ≥ 0.9; TLI = 0.874 ≥ 0.9; GFI = 0.816 ≥ 0.9; and SRMR = 0.06 ≤ 0.08. Further analyses represented acceptable findings for internal consistency reliability values with 0.86 of Cronbach’s alpha.

Conclusions

The results proved that the C-19MLs is a reliable and valid tool, and it is suitable and acceptable now and can be utilized in forthcoming investigations. This highlights educators and stakeholders to realize the importance of participating individuals in the new media ecology and new ‘Infomedia’ ecosystems for enabling people in the current digital society.

Peer Review reports

Introduction

The need to acquire media literacy skills

Media literacy is accessing, analyzing, evaluating, and conveying information in several printed and non-printed media [1]. Media literacy is the cognitive process utilized in critical thinking [2].

The rapid development of media technology in our daily lives causes the novel mode of creating and consuming information particularly attractive to adolescents as space and a platform for activities impossible in face-to-face communication [3, 4]. Media literacy skills can cope with their own media activities and expose themselves intentionally to the media [5]. Moreover, examining the results of meta-analysis investigations of the effectiveness of educational media literacy interventions on the avoidance of performances with high risks [6, 7] presents the facts of the need to implement health interventions to promote media literacy In Iran [8].

A focus on media literacy related to coronavirus

Coronavirus (COVID-19) has been hotly discussed worldwide since January 30, 2020, as an international public health concern [9]. During this period, the youth are confined in their homes in particular situations, and many use social media apps for entertainment. These new opportunities facilitate access to a large body of information for learning and social interaction [10] while exposing them to dangerous individuals, including those with physical and mental health disorders. Based on several surveys, compulsive media use can be represented as an environmental risk factor for youth’s health, including having a sedentary lifestyle [11] or lower life satisfaction [12], anxiety, sleep disturbance [13], and stress [14].

Also, concerning all the draconian measures for the prevention of COVID-19, enabling individuals with media literacy and critically thinking about reports of media, truth or not true, and act using all forms of communication related to COVID-19 and have independent individual decision-making without interference from other environmental factors (media contents) to prevent the coronavirus disease [15, 16] is essential [17].

Adopted theoretical framework of C-19MLs

In addition, there is a global need to study individuals’ health-specific media literacy competence [18], especially on specific pandemic issues [19]. On the other hand, one globally critical need, consultation, and recommendation for further action by the World Health Organization (WHO) and ”a key component of the COVID-19 global response” is increasing COVID-19 infodemic management in individuals [20]. To validate a measure of COVID-19 Media Literacy (C-19ML), applying an appropriate conceptual framework to evaluate the effectiveness of media literacy tools and the designed interventions is necessary. In this regard, the Media Literacy Training Center of the American CML (Theory CML Media Lit Kit) conceptual framework was selected [21], which is presented as a promising tool and evaluates the effectiveness of media literacy interventions [22,23,24,25]. Based on this framework, media literacy has five domains: “purpose, constructedness, audience, format, filter, and omit” [21]. Purpose means most media messages are organized to gain profit and power; “Why is this message being sent?“. The meaning of contractedness is that all media messages are constructedness;“ Who created this message?“. Audience means different people experience the same media message differently; “How might different people understand this message differently from me?“. Format and filter mean media messages are constructed using a language with its rules; “What creative techniques are used to attract my attention?“. The meaning of omitting is that media have embedded value and point of view; “What lifestyle, value and point of view are presented in or omitted from this message?”. These dimensions are the five core concepts of the selected framework [21]. Notably, this framework was used in other and near the scope of the present study [22, 24, 25].

Hence, the current study aimed to assess the psychometric features of C-19ML for measuring the youth’s C-19ML.

Existing instruments and gaps

Several general and specific tools measure media literacy [16, 18, 19, 26, 27]. Each of these tools has introduced dimensions for media literacy (ML). These tools suffer from their narrow scopes and lack of tools targeted at measuring ML in specific issues to assess media literacy comprehensively in multidimensional skills. Moreover, the multidimensional specific media literacy to measure media literacy has mainly been neglected. For example, Chen et al. argue that media literacy is both an understanding producer and a consumer of media content [28]. Buckingham et al. emphasized functional and critical literacy, which define functional media literacy as competencies tools for critical thinking about creator media messages and understanding them at the textual level [29].

In contrast, critical media literacy is the ability to analyze and judge media messages and understand them at various contextual levels [28]. So, these notions about media literacy are traditional [30]. Recently, Li X et al. went beyond and developed a study to construct digital skills scales for primary and middle school children, especially in developing countries [4].

On the other hand, to the best of our knowledge, the number of studies proposing a tool to operationalize media literacy related to emerging diseases is limited [31]. Although some studies have been conducted using the researcher-made questionnaire about media literacy in Iran [16, 32], no prior research has psychometrically evaluated the C-19ML skills in individuals.

Also, several organizations such as the American Academy of Pediatrics, Center for Disease Control and Prevention, European Commission, UNESCO, and European Parliament and several media organizations such as Center for Media Literacy (CML), National Association for Media Literacy Education (NAMLE), and Association for Media Literacy (AML) have vigorously discussed the media literacy [20, 22, 23, 25]. Instead, several studies on Western countries’ media literacy or digital literacy scale [4].

Hence, the current work aimed to assess C-19ML’s psychometric features among college students in Iran, a developing country, and fill the empirical literature gap.

Methods

Statement

The Ethics Committee of Hamadan University of Medical Sciences approved this study with all consent processes (No: IR.UMSHA.REC.1399.229). Informed consent was obtained from all students; they were informed about the confidentiality of the information, the project’s purpose, and their voluntary participation in the study. All methods were performed based on relevant guidelines and regulations. The confidentiality of the information of the students was also assured. Informed consent was obtained from all the contributors.

Instrument and item development

In behavioral research, scale development begins with a thorough understanding of the concept to be measured [33]. So, this measurement instrument’s framework was defined in terms of the Media Literacy Training Center of the American CML[21]. Because no scale exists that measures the C-19 media literacy concepts in the student populations, the concept analysis process begins with exploring COVID-19 media literacy through interviews with people familiar with this concept and personal experience [15]. Then proceeds to write the items that were done [33]. The following processes of questionnaire development and validation [34] are offered in Fig. 1.

Fig. 1
figure 1

Processes of questioner development and validation [67]

Full size image

Based on the results of the previous qualitative study, the concept identification and concept analysis process was done concerning the appearance of new media technology along with emerging diseases, critical thinking about new concepts such as creators of media massages, methods of persuading the audience by creators of media messages and presented/ omit new patterns in lifestyle by misinformation and fake news were identified. So, generally, five concepts or factors consisting of (a) purpose, (b) contractedness, (c) audience, (d) format, and (e) filter and omit were identified [15]. Then, the items were completed by creating a matrix that visually displays the content to be addressed by the items and the dimensions of the concept that would be measured. This matrix helped the researcher identify the number and type of items required to measure the concept adequately. However, for the phrasing of items, the researcher turns to qualitative interviews with people familiar with this concept [33]. Results of the previous study showed that dimensions were required to adequately measure these concepts with the increased number of items of format and technique (8 items) and lifestyles represented in fake media coronavirus messages (8 items). Finally, the items of C-19MLs were extracted from 33 items measured on a 5-point Likert-type scale with “1 strongly disagree” and “5 strongly agree”.

Psychometric properties of the scale

Face validity

The face validity of the C-19MLs was evaluated quantitatively and qualitatively.

Qualitative face validity evaluation

The face validity of the C-19MLs was evaluated qualitatively through the invitation of ten students to assess and comment on the items’ difficulty, suitability, vagueness, and relevancy. In this phase, the time needed for the implementation of this scale was adjusted. The scale amendments were made in terms of the student’s remarks.

Quantitative face validity evaluation

C-19ML’s quantitative face validity was evaluated to adopt the item impact method. Hence, ten students were requested to pilot the scale and determine the item’s rank over a Likert-based scale from 1 (non-important) to 5 (completely important). The effect of every item was ranked as the percentage of frequency multiplied by the importance. The frequency represents the number of students reporting a score of 4–5 to the considered item, and the importance was 4 or 5. When the effect score of an item was higher than 1.5, the item was regarded as appropriate and represented on a scale [35,36,37].

Content validity evaluation

The content validity of the C-19MLs was quantitatively and qualitatively evaluated.

Qualitative content validity evaluation

The content validity was calculated by delivering C-19MLs to 10 experts in health education (ten health education Ph.Ds.). They were then asked to present comments and assessments on the item allotments, phrasing, and ranking of the items [38]. The C-19MLs were revised in terms of feedback and comments.

Quantitative content validity evaluation

CVR (Content Validity Ratio) and CVI (Content Validity Index) were calculated quantitatively. CVR assesses the vitality of every item for the Iranian culture with a 3-point ranking scale (vital, valuable but not vital, and not vital) [39]. The CVR for every item was calculated by CVR = (Ne − (𝑁/2)) / (𝑁/2). Where Ne represents the number of panelists revealing “vital” for each definite item and N shows the overall number of panelists. Utilizing the Lawshe table, the CVR value is numerically obtained, in which a value of 0.62 is adopted for 10 panelists [40]. Using the ordinal scale with four possible responses, CVI was determined for simpleness, relevance, and clearness of every item. The answers included a score of 1 for not simple, unrelated, and unclear to 4 for very simple, very related, and very clear. The students’ judgment about the item as related or clear (with a rate of 3 or 4) was divided by the number of content specialists. CVI = 0.79 was suggested in some studies as a satisfactory lower limit [41, 42].

Target population and a sample

Thus, the target population in this work included students attending bachelor’s degree (BS) education to Ph.D. education. However, in the current research, 530 students from a medical university in Hamadan, Iran, were recruited for a stratified cluster random sampling process in June -July 2020 due to the coronavirus crisis in the Iran context.

Noticeably, the target population for conducting an exploratory factor analysis included 330 students from medical universities, who were recruited through a stratified cluster random sampling process, and the target population for performing confirmatory factor analysis included 200 students from a medical university who were recruited through a stratified cluster random sampling process and finally a total of 530 students from a medical university in Hamadan, Iran, were recruited for a stratified cluster random sampling process in June-July 2020.

Process and ethical considerations

The current research received approval (No: 9,904,102,236 and special ID of the Ethics Committee: IR.UMSHA.REC.1399.229) from the official review board and Ethics Panel at Hamadan University of Medical Sciences.

Procedures

Hamadan City is located in the west of Iran. It has one state medical university, Hamadan University of Medical Sciences. In the present study, strata were schools, and clusters were classes. In the first stage, each stratum has probability proportional to its size. So, eight schools were selected. In the second stage, in these eight schools, three classes were selected, and then 530 students were randomly selected with heterogeneous backgrounds and educational states.

Then, after coordinating with administrators of universities and colleges and obtaining their consent, they were referred to the students if they wanted to interview; aware that written consent was taken from them, they were assured that the information was confidential and then proceeded to collect the information. The inclusion criteria were to be a student of Hamadan University of Medical Sciences, interested in participating, and capable of responding and participating in the work and evaluation of Social Networks and the Internet. If any student wasn’t willing to not contribute to the study, he/she was excluded.

Measures

The students were asked to complete the C-19MLs questionnaires with two sections: [1] items representing the demographic data and [2] the C-19MLs (COVID-19 Media Literacy Scale).

The demographic questionnaires involved items of Sex, Age, Level of education, marital status, Major, Living status, Job status, Inspiration to use social media apps, and time spent utilizing the social media apps.

Measurement scales

Overall, 33 items were extracted for the variables mentioned above. The students were requested to evaluate their quantities of C-19MLs from 1 to 5, considering the items (1 for the minimum, 5 for the maximum) on a Likert-type scale. The sample items are “the objective of creating COVID-19 Media Messages to increase health literacy and self-caring in persons ”, “the WHO is among the Constructedness of credible messages about COVID-19”, “unproductive individuals are among the audience of coronavirus fake media messages,,” “Teaching simple preventive instructions and guidelines for public health Such as using frequent hands washed with ordinary soap and water, wearing a mask are utilized for attracting the attention of the audience incredible messages about Covid-19”, “In coronavirus, fake media messages, often represented in effects of drinking alcohol can prevent infection with the coronavirus.”

C-19MLs uses a five-item Likert-kind scale within the range of completely disagree [1] to completely agree [5]. The items and scores range for every subscale included: Purpose with four items, 4–20 scores range, Contractedness with six items and 6–30 scores range, Audience with seven items and 7 to 35 scores range, format with eight items and 8–40 scores range, and Represent Lifestyles with eight items and 8–40 scores range. The higher scores indicated a higher C-19ML.

Statistical analysis and Validity Assessment

The C-19MLs were validated based on content validity, construct validity, as well as face validity. Thus, the structural validity of the scale was examined using Exploratory Factor Analysis (EFA) with Promax rotation. A factor analysis was conducted by choosing a minimum sample size of 5–10 times the amount per item of the popular instrument [43]. Thus, the target population in this work included 330 students from medical universities in Hamadan, Iran, who were recruited through a stratified cluster random sampling process.

The Kaiser-Meyer-Olkin (KMO) and Bartlett’s Test of Sphericity were used to determine the appropriateness of the sample for factor analysis. Eigenvalues above one and factor loadings greater than 0.40 were considered appropriate to verify the possible underlying factors [43,44,45,46]. Furthermore, confirmatory factor analysis was performed with (AMOS Graphics, version 24. Thus, the target population in this work included 200 students from a medical university in Hamadan, Iran, who were recruited through a stratified cluster random sampling process based on some surveys that have recommended that this phase should be achieved on sample sizes between 100 and 200 participants [47]. Several goodness-of-fit indicators, including the chi-square ratio (χ2/df), the goodness of fit index (GFI), the root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), were selected for reporting the analysis outcomes. The following thresholds were considered to verify the model’s goodness of fit: χ2/df < 2.0, CFI, NFI, NNFI, and GFI ≥ 0.90–0.95, SRMR ≤ 0.05–0.08, and RMSEA ≤ 0.05–0.1 [48, 49].

Two types of validity are carried out for construct validity: convergent validity and discriminant validity. Average variance explained (AVE), construct reliability (CR), and maximum shared variance (MSV) are computed for all factors and are presented in Table 4. The AVE for each construct should be greater than 0.50; CR should be more than 0.7; CR is expected to be greater than AVE [50].

Reliability evaluation

Ultimately, construct reliability, test-retest analyses, and internal consistency were utilized to assess C-19MLs’ reliability. For descriptive studies, Cronbach’s alpha coefficient was used to measure internal consistency. The tool’s reliability was measured by examining the internal consistency by Cronbach’s alpha coefficient. The stability was assessed by estimating the Intraclass Correlation Coefficient (ICC).

Alpha values ≥ 0.50 were regarded as satisfactory. The consistency levels were inferred by choosing the following class, in which α of 0.5 or less was considered improper, 0.50–0.60 as poor, 0.60–0.70 as moderate, 0.70–0.80 as good, 0.80–0.9 as very good, and higher than 0.90 was considered as excellent [36]. For ICC, thirty participants were randomly selected for completion of the scale after 2–4 weeks initially. A comparison was made for test-retest scores for every construct with the Pearson correlation test. The ICC values higher than 0.40 were regarded as acceptable. The consistency levels were inferred by choosing the following class: ICCs of 0.4 or lower were taken as poor to fair, 0.41–0.60 as moderate, 0.61–0.80 as acceptable, and higher than 0.80 as excellent [36].

Results

Sample features

In total, 530 students contributed to this work. The mean age of the respondents was 23.4 ± 5.22. Based on the educational status, Among the 530 participants, 369 participants (69.6%) mainly had bachelor’s degrees (BS), 43 participants (8.1%) had master’s degrees (MS), and 118 participants (22.3%) had Ph.D. Also, 352 participants (66.0%) were women, and 178 (83.6%) were single. Around 292 (55.1%) students lived in the dormitory, and 36 (6.8%) lived in student houses. Moreover, 160 (30.2%) of 530 participants with high social media app accessibility were high users (over 10 h), and 207 students (39.1%) used the internet moderately (for 3–6 h). (Table 1).

Table 1 Summary statistics for characteristics of study participants (n = 530)
Full size table

Psychometric properties of the scale

Face validity

The quantitative face validity indicated that the impact score was > 1.5 for the whole item. Regarding the qualitative face validity, participants represented slight alterations in the phrasing of some items for better elucidation.

Content validity

Assessing the tool qualitatively indicated that the whole criteria, including scaling of the questionnaire, grammar, allocating items, and phrasing, were appropriately chosen. The CVR and CVI values of the whole 33 items of the C-19MLs were respectively 0.77 and 0.94.

Construct validity phase

EFA.

Primary EFA results as presented in the following as Bartlett’s and KMO test indicated the appropriateness of the data for factor analysis (χ2 of 3978.533, KMO index of 0.86, df of 528, P < .001), approving the suitability of the factor model. These two tests revealed the appropriateness of the respondents’ data for EFA, which was conducted on the 33 items of the C-19ML scale by the highest likelihood process with Promax rotation. Based on primary exploratory factor loadings of items and the scree scheme (Fig. 2), eight factors were extracted, reporting an eigenvalue of higher than 1, accounting for 59.273% of the variance.

Fig. 2
figure 2

Primary EFA Scree plot of C-19MLs

Full size image

After careful assessment, some items were omitted for the following reasons:

  1. a.

    There was an item with low commonalities (e.g., less than 0.2) that were not highly correlated with one or more factors.

  2. b.

    Item loading on five items did not satisfy the expected threshold.

  3. c.

    There were three - factors with two or one item which, based on conceptual interpretability, failed to contribute meaningfully to any of the other factors or were ineffective to the items conjugated to other factors conceptually. Thus, in this process, it was decided to omit, and the other hand optimizes scale length.

Thus, item 24 had communalities = 0.16 < 0.2; five items [11, 15, 17, 22, and 23] were eliminated from the model because the factor loadings were less than 0.4. Furthermore, three factors [5,6,7,8] were deleted with fewer than three items ( 1, 2, 3, 4, 32, 33).

A final EFA was conducted to ensure that the factor solution does not change after deleting items and testing the factor structure [51]. The base of the result (χ2 of 2218.389, KMO index of 0.85, df of 210, P < .001), five factors were extracted, reporting eigenvalue of higher than one accounting for 58.41% of the variance with 21 items, remained Table 2, (Fig. 3).

Table 2 The final exploratory Results of Covid-19MLs with five factors
Full size table
Fig. 3
figure 3

Final EFA Scree plot of C-19MLs

Full size image

CFA.

The next step deals with confirming and validating the EFA-obtained factor structure utilizing the CFA (confirmatory factor analysis). According to the GOF (goodness-of-fit) indices, the studied model fits the standard accepted database appropriately. Thus, the CFA proves the model’s adequacy and the decent fitting of its structural model for the participants (Fig. 4). Table 3 represents the model fit indices.

Fig. 4
figure 4

CFA of the C-19MLS questionnaire with Five-domain structure (F1: Constructedness of credible Covid-19 media messages, F2: Constructedness of fake media coronavirus Messages, F3: Fake media Coronavirus Messages audience, F4: Format and F5: lifestyles are represented in fake media coronavirus Messages)

Full size image
Table 3 Measurement model-fit index
Full size table

Validity and reliability

Results of the proposed model fulfils the convergent validity.

To check the discriminant validity, the MSV was compared with AVE, and the square root of each dimension’s AVE was compared with the correlations for each pair of dimensions addressed by AVE and MSV (AVE > MSV) as presented in the correlation matrix Table 4, the MSV of all factors was lower than AVE, except factor one, which might be because of the low number of items (3 items of factor 1). So, for assurance about the reliability of the measurement instrument in addition to Cronbach’s alpha (CA), the reliability analysis was carried out in SPSS 24.0, and the results were presented in Table 5. It can be seen that the value of CA is greater than 0.60 for all the constructs. It can be concluded from the first step that the model is fit for carrying out SEM and path analysis as it meets all the validity issues. Also, internal consistency was used to evaluate the reliability. The mean ICC was 0.893 with a 95% confidence interval from 0.831 to 0.941 (F (600, 30) = 9.388, P < .001) (Table 5).

Table 4 Convergent validity and reliability
Full size table
Table 5 Cronbach’s alpha and ICC of the Factors of the C-19MLS
Full size table

However, the square root of each dimension’s AVE was bigger than the correlations for each dimension, indicating that the proposed factor structure possessed discriminate validity [4] (Table 6).

Table 6 Factor correlation and the squared root of AVE (on a diagonal)
Full size table

External validity

The independent sample t-test examined the external validity among different socio-demographic groups (Table 5). Based on the results, students were assured that lifestyles are represented in fake media coronavirus messages, format, constructedness of credible Covid-19 media messages, constructedness of fake media coronavirus messages, and the audience, respectively. Also, marital status and gender were considered for examining differences among dimensions. The result of the independent sample t-test indicated the constructedness of credible Covid-19 media messages dimension of the married student (M = 12.38, SD = 2.13) were higher than those of single students (M = 10.83, SD = 2.48), but it was not significant. The constructedness of a married student’s fake COVID-19 media messages dimension (M = 10.75, SD = 3.20) was higher than those of single students (M = 10.30, SD = 2.08), but it was not significant. The fake media coronavirus messages audience dimension of the married student (M = 10.50, SD = 2.51) were higher than those of single students (M = 9.78, SD = 2.41), but it was not significant. The lifestyles represented in fake media coronavirus messages dimension of the married student (M = 24.50, SD = 3.85) were higher than those of single students (M = 23.52, SD = 4.01), but it was insignificant.

The result of the independent sample t-test indicated the difference between the men and women was not significant, but the constructedness of credible COVID-19 media messages dimension of the female student (M = 11.42, SD = 2.30) was higher than that of male students (M = 10.20, SD = 3.27) but it was not significant. The constructedness of the female student’s fake Covid-19 media messages dimension (M = 10.46, SD = 3.20) was higher than those of single students (M = 10.30, SD = 2.08) but was not significant. The fake media coronavirus messages audience dimension of male students (M = 10.60, SD = 2.88) was higher than that of female students (M = 9.85, SD = 2.36), but it was insignificant. The lifestyles represented in fake media coronavirus messages dimension of male students (M = 24.00, SD = 6.04) were higher than those of female students (M = 23.73, SD = 3.56), but it was insignificant.

Discussion

Summary of findings

The study aimed to assess the validity and reliability of 21-item C-19ML. As indicated by the EFA, the C-19MLs is a 5-domain structure with five factors F1 (Constructedness of credible Covid-19 media messages) F2 (Contractedness of fake media coronavirus messages), F3 (Fake media coronavirus messages Audience), F4 (Format), and F5 (Represented lifestyles in fake media coronavirus Messages) [52].

Constructedness of credible media coronavirus messages dimension means Who create credible Covid-19 media messages?”. Contractedness of fake media coronavirus messages dimension means “Who create credible Covid-19 media messages?”. Fake media coronavirus messages audience dimension means “Who may deal with fake Covid-19 media messages”. Format dimension means “What creative techniques are used to attract my attention?“. meaning of represented lifestyles dimension is media have embedded value and point of view; “What lifestyle, value and point of view are presented in or omit from this message?” (“Supplementary file”).

Validity

The validity is a fundamental feature of questionnaires aimed at detecting the ability of an instrument to measure the object through its design. Constructed validation is vital for determining a questionnaire’s validity, mainly in psychometrical subjects. The ideal process in this regard is factor analysis [53]. The main frameworks of the research questionnaires included 33 items. In this phase, the operation of EFA caused the elimination of 12 items from the primary questionnaire. Ultimately, a form with 21 items was categorized into a 5-domain structure.

Based on the final EFA with Promax rotation findings, extracting the 5-domain structure is possible along with explicit contractedness of credible Covid-19 media messages, contractedness of fake media coronavirus messages, fake media coronavirus messages audience, format, and lifestyles are represented in fake media coronavirus messages. The former research supported the 5-domain structures. Our findings in this part correspond to developing guidance. CML MediaLit Kit™ develops the educational philosophy of empowerment via education through some records and Internet sources articulating the concept, execution, and application of Media Literacy within the US educational system. Elizabeth Thoman (1994), the CML creator, made her fundamental paper “Skills and Strategies for Media Education” with this kit [21, 54, 55].

C-19MLs measure media literacy related to COVID-19 media messages. According to the qualitative analysis of exploring the experience of people’s COVID-19 Media Literacy, the last 21 items recognized that the scale was accomplished by the C-19MLs measurement [56].

Analyzing the KMO index and Bartlett’s Test of Sphericity indicated the adequacy of the sample size and satisfaction of the factor analysis [48, 57, 58]. Rejecting the null hypothesis of data Sphericity and confirming the KMO statistic were obtained in our study. The five factors accepted here could clarify 60.0% of the variance, and the most pronounced variations were associated with the supposed power. Correspondingly, Koc et al. assessed the New Media Literacy Scale (NMLS) and reported that varimax orthogonal rotation presented the ultimate four-factor model, with the remaining 35 items accounting for 55% of the total variance [30]. We made our C-19MLs based on Primack et al.’s scale for the theoretically similar smoking media literacy [22]; based on their theoretical method of smoking media literacy, we believe the same framework might apply to COVID-19. On the other hand, both Bier et al. and Ashley et al. found a positive association between general media literacy and smoking media literacy regarding the same underlying theoretical framework [25, 59].

Content validity is strong, by which scale items were oriented by a precisely designed framework integrating media literacy models with the greatest acceptability. Furthermore, this strength is associated with items in the resultant scale representing the framework’s core concepts. These findings provided evidence that both factors are statistically applicable and equivalent to measuring media literacy (ML). The only difference between factors in C-19MLs and other scales of Media Literacy. In explaining, it can be said that a special issue of the C-19MLs presents an alternative account of the relationship between factors in media literacy and health education and health promotion themes. That is, C-19MLs is a specific measuring scale in media literacy, and until now, there aren’t specific media literacy scales for measuring individuals.

The present study’s finding examined the external validity, in which all dimensions were different among marital status and gender., mean scores of all dimensions in married students were higher than in singles. Chang et al.‘s study noted an increase in parents’ search for medical information and an increase in e-health literacy, which could be consistent with the present study’s findings [60]. Also, previous evidence found gender differences in computer and information literacy [61]. Notably, studies recommend that boys and girls may perform differently on computer and information literacy skills, and these skills require learning in higher competencies, especially in emergencies such as the COVID-19 pandemic [62, 63].

Reliability

The reliability represents the stableness and consistency of an instrument’s constructs, indicating the questionnaire’s measuring accuracy [36]. The last 21-item scale possesses an excellent internal consistency, with a Cronbach’s alpha of 0.86, indicating the acceptable reliability of the suggested questionnaires. Primack et al. performed a study with a Cronbach’s alpha of 0.87; the last scale with 18 items had exceptional internal consistency [22]. Ashrafi-rizi et al. (2014), investigating the Media Literacy Scale, reported satisfactory reliability with a Cronbach’s alpha of 0.89 [64].

Though the ICC level was desired in all domain structures, that means satisfactory quantities of ICC; nevertheless, an excellent internal consistency existed between these domain structures with a Cronbach’s alpha of 0.89. Thus, it is suggested to consider this fact in approaching studies.

The strengths and limitations

Regardless of some strength in our investigation, such as compared with previous tools, the newly developed C-19MLs is a multidimensional specific media literacy scale related to emerging diseases.

Also, this scale focuses on specific media literacy in developing countries among college students. On the other hand, other scales of ML emphasize that media literacy is both an understanding producer and a consumer of media content, or media literacy is generally managing consuming media, which these notions are traditional.

Also, with the appearance of new media technology along with emerging diseases, critical thinking about new dimensions such as creators of media massages, methods of persuading the audience by creators of media massages [65], and presenting/omitting new patterns in lifestyle by misinformation and fake news [66] could be useful; because these typologies of information can play an essential role in teaching proper health information-seeking behavior when facing with the crises which in other scales to be neglected.

Some limitations exist likewise; first, this work was conducted by a student of Hamadan University of Medical Sciences, which suggested future studies could be conducted in other areas or countries to test the external validity.

Second, it’s impractical to ensure the utilization of the conclusions to the populations in numerous geographical areas or another context because this scale measures COVID-19 media literacy in developing countries. So, it could be validated in future studies in developed countries. Thirdly, we deleted the items based on EFA results; therefore, we suggested adding additional items in future studies.

Also, future studies must compare studies about C-19MLs in students after and before COVID-19 and during seasonal influenza [67].

Conclusions

Concisely, C-19MLs is a scale with reliability and validity for evaluating COVID-19 media literacy among students. Validation assessments with various longitudinal designs and populations should be vitally aimed at purifying, adjusting, or confirming the C-19MLs as an additional, complementary media literacy (ML) tool. Moreover, suggestions for future work in defining and assessing the field of health investigation and media literacy. This indicates the significance for educators and stakeholders to realize the vital participating individuals in the new media ecology and new ‘Infomedia’ ecosystems for empowering people, especially in online health information searches in the youth. Also, this scale could be applied for designing interventional strategies, particularly in cyberchondria, “digital syndrome” prevention in societies.

Data Availability

The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

ML:

Media literacy

C-19MLs:

The COVID-19 Media Literacy Scale

ICC:

Intraclass Correlation Coefficient

CFA:

Confirmatory Factor Analyses

EFA:

Exploratory Factor Analyses

CVR:

Content Validity Ratio

CVI:

Content Validity Index

References

  1. Marchand P. Marshall McLuhan: the medium and the messenger: a biography. MIT Press; 1998.

  2. Brown JA. Media literacy perspectives. J Communication. 1998;48(1):44–57.

    Google Scholar 

  3. Lee L, Chen D-T, Li J-Y, Lin T-B. Understanding new media literacy: the development of a measuring instrument. Comput Educ. 2015;85:84–93.

    Google Scholar 

  4. Li X, Hu R. Developing and validating the digital skills scale for school children (DSS-SC). Information, Communication & Society. 2020:1–18.

  5. Lunt P, Livingstone S. Media regulation: Governance and the interests of citizens and consumers. Sage; 2011.

  6. Vahedi Z, Sibalis A, Sutherland JE. Are media literacy interventions effective at changing attitudes and intentions towards risky health behaviors in adolescents? A meta-analytic review. J Adolesc. 2018;67:140–52.

    PubMed  Google Scholar 

  7. Xie X, Gai X, Zhou Y. A meta-analysis of media literacy interventions for deviant behaviors. Comput Educ. 2019;139:146–56.

    Google Scholar 

  8. Barati M, Jormand H, Bashirian S, Doosti-Irani A, Rezapur-Shahkolai F. The role of media on the intention of adolescents smoking: a systematic review and Meta-analysis. J Educ Community Health. 2020;7(4):311–23.

    Google Scholar 

  9. WHO; novel-coronavirus-2019 [Internet]. 2020. Available from: [cited 2020 July 29]. Available from:https://www.who.int/emergencies/diseases/novelcoronavirus-2019.

  10. Savina E, Mills JL, Atwood K, Cha J. Digital media and youth: a primer for school psychologists. Contemp School Psychol. 2017;21(1):80–91.

    Google Scholar 

  11. Peck T, Scharf RJ, Conaway MR, DeBoer MD. Viewing as little as 1 hour of TV daily is associated with higher change in BMI between kindergarten and first grade. Obesity. 2015;23(8):1680–6.

    PubMed  Google Scholar 

  12. Roberts DF, Foehr UG. Trends in media use. The Future of Children. 2008:11–37.

  13. Lemola S, Perkinson-Gloor N, Brand S, Dewald-Kaufmann JF, Grob A. Adolescents’ electronic media use at night, sleep disturbance, and depressive symptoms in the smartphone age. J Youth Adolesc. 2015;44(2):405–18.URL: https://link.springer.com/article/10.1007/s10964-014-0176-x?r=1&l=ri&fst=0#citeas [accessed 2020-09-27].

    PubMed  Google Scholar 

  14. Ciarrochi J, Parker P, Sahdra B, Marshall S, Jackson C, Gloster AT, et al. The development of compulsive internet use and mental health: a four-year study of adolescence. Dev Psychol. 2016;52(2):272. https://psycnet.apa.org/buy/2015-53033-001. accessed 2020-09-27.

    PubMed  Google Scholar 

  15. Jormand H, Bashirian S, Barati M, Khazaei S, Jenabi E, Zareian S. A qualitative study on people’s experiences of Covid-19 media literacy. Media Lit Acad Res. 2021;4(1):38–52.

    Google Scholar 

  16. Solhi M, Jormand H, Gohari M. Application of media literacy education for changing attitudes about self-medication of slimming supplements. Med J Islamic Repub Iran. 2017;31(1):792–7.

    Google Scholar 

  17. Blaylock RL. COVID UPDATE: what is the truth? Surg Neurol Int. 2022;13.

  18. Frisch A-L, Camerini L, Diviani N, Schulz PJ. Defining and measuring health literacy: how can we profit from other literacy domains? Health Promot Int. 2012;27(1):117–26.

    PubMed  Google Scholar 

  19. Sousa-Pinto B, Anto A, Czarlewski W, Anto JM, Fonseca JA, Bousquet J. Assessment of the impact of Media Coverage on COVID-19–Related Google Trends Data: Infodemiology Study. J Med Internet Res. 2020;22(8):e19611.

    PubMed  PubMed Central  Google Scholar 

  20. World Health Organization. Infodemic management: a key component of the COVID-19 global response – parer aux infodémies: un élément essentiel de la riposte mondiale à la COVID-19. Wkly Epidemiol Record = Relevé épidémiologique Hebdomadaire. 2020;95(16):145–8.

    Google Scholar 

  21. Thoman E, Jolls T. Literacy for the 21st century: an overview and orientation guide to media literacy education. Theory CML MedicaLit Kit: Center for Media Literacy;; 2008.

  22. Primack BA, Gold MA, Switzer GE, Hobbs R, Land SR, Fine MJ. Development and validation of a smoking media literacy scale for adolescents. Arch Pediatr Adolesc Med. 2006;160(4):369–74.

    PubMed  PubMed Central  Google Scholar 

  23. Primack BA, Fine D, Yang CK, Wickett D, Zickmund S. Adolescents’ impressions of antismoking media literacy education: qualitative results from a randomized controlled trial. Health Educ Res. 2009;24(4):608–21.

    PubMed  Google Scholar 

  24. Salgado MV, Pérez-Stable EJ, Primack BA, Kaplan CP, Mejia RM, Gregorich SE, et al. Association of media literacy with cigarette smoking among youth in Jujuy, Argentina. Nicotine Tob Res. 2012;14(5):516–21.

    PubMed  Google Scholar 

  25. Bier MC, Schmidt SJ, Shields D, Zwarun L, Sherblom S, Primack B, et al. School-based smoking prevention with media literacy: a pilot study. J Media Lit Educ. 2010;2(3):1.

    Google Scholar 

  26. Ahmad AR, Murad HR. The impact of social media on panic during the COVID-19 pandemic in iraqi Kurdistan: Online Questionnaire Study. J Med Internet Res. 2020;22(5):e19556.

    PubMed  PubMed Central  Google Scholar 

  27. Tangcharoensathien V, Calleja N, Nguyen T, Purnat T, D’Agostino M, Garcia-Saiso S, et al. Framework for managing the COVID-19 Infodemic: methods and results of an online, Crowdsourced WHO Technical Consultation. J Med Internet Res. 2020;22(6):e19659.

    PubMed  PubMed Central  Google Scholar 

  28. Chen D-T, Wu J, Wang Y-M. Unpacking new media literacy. 2011.

  29. Buckingham D. Media education: literacy, learning and contemporary culture. John Wiley & Sons; 2013.

  30. Koc M, Barut E. Development and validation of new media literacy scale (NMLS) for university students. Comput Hum Behav. 2016;63:834–43.

    Google Scholar 

  31. Austin EW, Austin BW, Willoughby JF, Amram O, Domgaard S. How media literacy and science media literacy predicted the adoption of protective behaviors amidst the COVID-19 PANDEMIC. J Health Communication. 2021:1–14.

  32. Ashrafi-rizi H, Khorasgani ZG, Zarmehr F, Kazempour Z. A survey on rate of media literacy among Isfahan University of Medical Sciences’ students using iranian media literacy questionnaire. J Educ Health Promotion. 2014;3.

  33. DiIorio CK. Measurement in health behavior: methods for research and evaluation. John Wiley & Sons; 2006.

  34. Tsang S, Royse CF, Terkawi AS. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth. 2017;11(Suppl 1):80.

    Google Scholar 

  35. Barati M, Allahverdipour H, Hidarnia A, Niknami S, Bashirian S. Belief-based Tobacco Smoking Scale: evaluating the Psychometric Properties of the theory of Planned Behavior’s constructs. Health Promotion Perspectives. 2015;5(1):59–71.

    PubMed  PubMed Central  Google Scholar 

  36. Hajizadeh E, Asghari M. Statistical methods and analyses in health and biosciences: a research methodological approach using SPSS guide. 1st ed]. Tehran: Publications of the Academic Jihad Organization; 2011. pp. 135–9. [in Farsi].2011.

    Google Scholar 

  37. Soleimani MA, Zarabadi-Pour S, Pahlevan Sharif S, Motalebi SA. The relationship between death anxiety and self-esteem in patients with Acute myocardial infarction. J Qazvin Univ Med Sci. 2019;23(1):14–25.

    Google Scholar 

  38. Bernard HR, Bernard HR. Social research methods: qualitative and quantitative approaches. Sage; 2013.

  39. Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health. 1997;20(3):269–74.

    PubMed  Google Scholar 

  40. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563–75.

    Google Scholar 

  41. Polit DF, Beck CT. Nursing research: principles and methods. Lippincott Williams & Wilkins; 2004.

  42. Jay Lynn S, Surya Das L, Hallquist MN, Williams JC. Mindfulness, acceptance, and hypnosis: cognitive and clinical perspectives. Int J Clin Exp Hypn. 2006;54(2):143–66.

    PubMed  Google Scholar 

  43. Munro BH. Statistical methods for health care research. lippincott williams & wilkins; 2005.

  44. Samuels P. Advice on exploratory factor analysis. 2017.

  45. Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assess Res Evaluation. 2005;10(1):7.

    Google Scholar 

  46. Rahn M. Factor analysis: A short introduction, Part 5: Dropping unimportant variables from your analysis. 2014.

  47. Kline R. Methodology in the social sciences principles and practice of structural equation modeling. Volume URL. New York: Guilford Press; 2005. ftp://158.208.129.61/suzuki/PP_SEM_3e.pdf. [accessed 2020-08-10].

    Google Scholar 

  48. Marsh HW, Hau K-T, Wen Z. In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Struct Equ Model. 2004;11(3):320–41.

    Google Scholar 

  49. Lt H, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Modeling: Multidisciplinary J. 1999;6(1):1–55.

    Google Scholar 

  50. Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res. 1981;18(1):39–50.

    Google Scholar 

  51. DiIorio CK. Measurement in health behavior: methods for research and evaluation. Volume 1. John Wiley & Sons; 2006. pp. 84–94. https://www.wiley.com/en-ir/Measurement+in+Health+Behavior:+Methods+for+Research+and+Evaluation-p-9780787970970. [accessed 2020-09-20]. 2URL.

  52. Barati M, Bashirian S, Jormand H, Khazaei S, Jenabi E, Zareian S. The Association between COVID-19 media literacy and the fear of COVID-19 among students during coronavirus pandemic: a cross-sectional study. J Health Lit. 2022;7(2):46–58.

    Google Scholar 

  53. Ferguson E, Cox T. Exploratory factor analysis: a users’ guide. Int J Selection Assess. 1993;1(2):84–94.

    Google Scholar 

  54. Hobbs R, Frost R. Measuring the acquisition of media-literacy skills. Reading Res Q. 2003;38(3):330–55.

    Google Scholar 

  55. Thoman E. Skills and strategies for media education. Educational Leadersh. 1999;56:50–4.

    Google Scholar 

  56. Jormand H, Bashirian S, Barati M, Khazaei S, Jenabi E, Zareian S. A qualitative study on people’s experiences of Covid-19 media literacy. Media Lit Acad Res. 2021;4(2):38–52.

    Google Scholar 

  57. Harrington D. Confirmatory factor analysis. Oxford university press; 2009.

  58. Tavousi M, Haeri-Mehrizi A, Rakhshani F, Rafiefar S, Soleymanian A, Sarbandi F, et al. Development and validation of a short and easy-to-use instrument for measuring health literacy: the health literacy instrument for adults (HELIA). BMC Public Health. 2020;20:1–11.

    Google Scholar 

  59. Ashley S, Maksl A, Craft S. Developing a news media literacy scale. Journalism & Mass Communication Educator. 2013;68(1):7–21.

    Google Scholar 

  60. Chang F-C, Chiu C-H, Chen P-H, Miao N-F, Lee C-M, Chiang J-T, Cyberpsychology, et al. Behav Social Netw. 2015;18(10):618–24.

    Google Scholar 

  61. Punter RA, Meelissen MR, Glas CA. Gender differences in computer and information literacy: an exploration of the performances of girls and boys in ICILS 2013. Eur Educational Res J. 2017;16(6):762–80.

    Google Scholar 

  62. Aesaert K, Van Braak J. Gender and socioeconomic related differences in performance based ICT competences. Comput Educ. 2015;84:8–25.

    Google Scholar 

  63. Gui M, Argentin G. Digital skills of internet natives: different forms of digital literacy in a random sample of northern italian high school students. New Media & Society. 2011;13(6):963–80.

    Google Scholar 

  64. Ashrafi-Rizi H, Khorasgani ZG, Zarmehr F, Kazempour Z. A survey on rate of media literacy among Isfahan University of Medical Sciences’ students using iranian media literacy questionnaire. J Educ Health Promotion. 2014;3:49.

    Google Scholar 

  65. Rasoolipour S, Emamiifar SN. The effect of creative applicable methods on attracting the audience towards urban commercial advertisements from 2000 to 2020. Int J Arts Technol. 2020;12(3):266–81.

    Google Scholar 

  66. Khweiled R, Jazzar M, Eleyan D. Cybercrimes during COVID-19 pandemic. Int J Inform Eng Electron Bus. 2021;13(2).

  67. Jormand H, Bashirian S, Barati M, Babamiri M, Rezapur-Shahkolai F. The development of a measuring instrument for substance abuse media literacy scale in students. Addict Disorders Their Treat. 2021;20(4):609–20.

    Google Scholar 

Download references

Acknowledgements

We would like to thank all students for participating in this research.

Funding

This study was funded by the Hamadan University of Medical Sciences (No, 9904102236), which participated in the financial support of this study. The funder had no role in the study design, data collection, analysis, or interpretation of the data presented in this paper, the writing of the paper, or the decision to submit the paper for publication.

Author information

Authors and Affiliations

  1. Vice-chancellor for research and technology, of Health Education and Promotion, Hamadan University of Medical Sciences, Hamadan, IR, Iran

    Hanieh Jormand

  2. Education and Promotion, Department of Public Health, School of Public Health, Social Determinants of Health Research Center, Hamadan University of Medical Sciences, Hamadan, IR, Iran

    Majid Barati

  3. Department of Public Health, School of Public Health and Social Determinants of Health Research Center, Professor of Health Education and Promotion, Hamadan University of Medical Sciences, Hamadan, IR, Iran

    Saeed Bashirian

  4. Department of Epidemiology, School of Public and Social Determinants of Health Research Center, Hamadan University of Medical Sciences, Hamadan, IR, Iran

    Salman Khazaei

  5. Hamadan University of Medical Sciences, Hamadan, Iran

    Ensiyeh Jenabi

  6. Vice-Chancellor for Research and Technology, Zareian. Sepideh (MSc), Hamadan University of Medical Sciences, Hamadan, Iran

    Sepideh Zareian

Authors
  1. Hanieh Jormand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Majid Barati
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saeed Bashirian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salman Khazaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ensiyeh Jenabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sepideh Zareian
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MB conceived of the study, obtained funding, and supervised data collection;MB & HJ was responsible for data collection and undertook the factor analyses. SZ prepared questioner; SB, EJ & SKH undertook other data analyses; MB & HJ drafted the manuscript; all authors contributed to the manuscript review and approved the final manuscript.

Corresponding author

Correspondence to Hanieh Jormand.

Ethics declarations

Competing interests

The authors declare no competing interests.

Competing of Interest:

The authors declare that they have no competing interests.

Ethical approval and consent to participate

The current research received approval (No: 9904102236 and special ID of the Ethics Committee: IR.UMSHA.REC.1399.229) from the official review board and Ethics Panel at Hamadan University of Medical Sciences. Also, information was provided for the contributors regarding the objectives and processes of the study. Furthermore, they were ensured voluntary participation. The confidentiality of the information of the students was also assured. Informed consent was obtained from all the contributors.

Consent for publication

Not Applicable.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jormand, H., Barati, M., Bashirian, S. et al. Developing and validation of COVID-19 media literacy scale among students during the COVID-19 pandemic. BMC Psychol 11, 315 (2023). https://doi.org/10.1186/s40359-023-01353-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40359-023-01353-6

Keywords

BMC Psychology

ISSN: 2050-7283

Contact us