Development and Validation of Nutrition Literacy Scale for Middle School Students in Chongqing, China: A Cross-Sectional Study


 Background: Nutrition literacy has a positive effect on health and its measurement has no consensus.This study was performed to develop the Chongqing Middle school student Nutrition Literacy Scale(CM-NLS)and measure its reliability and validity.Methods: Three experiments were conducted. Firstly, a theoretical framework and an initial item pool of CM-NLS were established based on the literature review. Secondly, the two-round Delphi method was used to explore the suitable acceptance indicators and items. Thirdly, item evaluation and reduction were performed using the classical test theory. In addition, the items in the final CM-NLS were tested for their validity and reliability amongst 462 middle school students.The construct validity was assessed using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).The internal consistency reliability and split-half reliability were evaluated using Cronbach’s alpha coefficients.Results: The finalised CM-NLS consisting of 52 items that were based on three primary indicators (functional, interactive and critical) and six sub-indicators (obtain, understand, apply, interact, medial literacy and critical skill) was developed and validated. EFA suggested six factors explaining 69.44% of the total variance (Kaiser–Meyer–Olkin test = 0.916, Bartlett’s test χ2 = 5854.037, P < 0.001). CFA showed that the model fit the data adequately, with χ2/df = 1.911, root mean square error of approximation = 0.063, goodness-of-fit index (GFI) = 0.822 and adjusted goodness of fit index (AGFI) = 0.790. The total CM-NLS Cronbach’s alpha values of internal consistency and split-half reliability were 0.849 and 0.521, respectively, with reasonable reliability.Conclusions: CM-NLS is a valid and reliable instrument to measure nutrition literacy amongst middle school students. It provides the scientific basis for the evaluation of the nutrition literacy level of middle school students in Chongqing and the implementation of nutrition education strategies.


Introduction
The prevalence of childhood obesity has been on the rise in recent years 1  According to studies, childhood obesity is closely associated with chronic non-communicable diseases, such as hypertension, diabetes and cardiovascular disease [4][5][6] . The important factors affecting childhood obesity, such as children's dietary patterns (number, regularity and duration of meals), fast-food consumption habits and food choices, are strongly related to nutrition literacy [7][8][9] .
Nutrition literacy is an emerging term de ned as 'the degree to which people have the capacity to obtain, process and understand basic nutrition information' 10 . Children with high levels of nutrition literacy could develop healthy dietary habits and food purchasing behaviour 11 , whereas low nutrition literacy has been shown to be associated with unhealthy diets 8,12,13 .
Although nutrition literacy has emerged as an area of increasing research focus in many countries, only a limited number of tools for assessing children and adolescents are available. In 2017, Asakura et al. developed the Nutrition Knowledge Questionnaire for primary school children consisting of a four-part test based on the understanding of terms, awareness of dietary recommendations, using the information to make dietary choices, awareness of diet-disease associations 14 . However, interactive nutrition literacy (INL) and critical nutrition literacy (CNL) assessment was lacking. The Adolescent Nutrition Literacy Scale developed by Bari contains 29 attitude statements under three sub-dimensions 15 . Food and Nutrition Literacy evaluated the cognitive and skill domain, which was used to measure the validity and reliability of food and nutritional literacy amongst Iranian children 16 . Nutrition literacy is in uenced by culture and society. The tools used to measure nutrition literacy should be culture-speci c but the two existing evaluation instruments are not suitable for Chongqing 17,18 . Different regions in China have different dietary cultures, thus, the national questionnaire could not be fully applied for regional survey 19 . In China, no consensus tools are available to measure nutrition literacy amongst school children.
This study aimed to develop and validate a scale to assess the nutrition literacy of middle school students in China and provide a scienti c basis for the assessment of the nutrition literacy level of these students.

Study Design and Participants
The study consisted of three consecutive phases. In the rst phase, the Chongqing Middle school student Nutrition Literacy Scale (CM-NLS) test pool items were developed. In the second phase, 18 experts working in child nutrition were invited for a two-round e-Delphi to complete the construction of the framework and initial items. In the third phase, 462 participants were tested, the items were revised and the reliability and validity of CM-NLS were tested. Figure 1 shows the whole process of questionnaire design modi cation.

Page 3/13
Phase 1: Literature review and design of initial items Literature review A comprehensive electronic database search was conducted in PubMed, Web of Science and China National Knowledge Infrastructure from inception to January 2020 to retrieve relevant articles published in English and Chinese. 'Nutrition literacy' and 'Nutritional literacy' were used as search keywords to collect the relevant literature on nutrition literacy. All studies retrieved were read in full, focusing on the de nition, eld and dimension of nutrition literacy. In summary, this step was used to guide the conceptual framework construction and initial item generation.

Theoretical Framework
This step aimed to determine a suitable conceptual framework based on the theoretical framework of health literacy 20 , with a comprehensive review of existing nutrient literacy questionnaires and other assessment tools 16, 21-27 . Phase 2: Assessment of content validity by two rounds of expert consultation The purpose of this step was to identify if the questionnaire items generated in the previous step were su ciently acceptable, comprehensive, and relevant to nutrition literacy from the perspective of professionals. A total of 18 experts (mean age = 41.67 years, mean age of nutrition work = 12.28 years) in the eld of children's nutrition education were invited to evaluate the concepts of nutrition literacy by E-mail, all levels of indicators and the items used in the tool. They were asked to comment on the importance, feasibility of each indicator using a 5-Likert scale, comment on the importance and sensitivity of each item using a 10-Likert scale, respectively. We kept the indicator which mean of importance and feasibility was 3.50 and above; the coe cient of variation was 0.30 and below. We kept the items which mean of importance and feasibility was 7.00 and above; the coe cient of variation was 0.30 and below 28,29 . They were also can put forward some suggestions on the deletion and adjustment of items on the open question, which mainly to learn more about the experts' thoughts on the questionnaire.

Phase 3: Pilot Study And Con rmation Of Validation Item Generation
After the two-round Delphi method was conducted, the CM-NLS, which included21 single-choice questions, three multiple-choice questions, one order question and 34 Likert-type questions, was used in 462 middle school students to con rm the validity of the scale.

Sample
The convenience sampling method was used to recruit students from a middle school in Chongqing from June 2020 to July 2020.The research design of this study utilized a uniform resource locator (URL) invitation, with a QR (Quick Response) code embedded, providing access to the questionnaire to participate.The students who were asked to ll out the questionnaire, anonymous, without mentioning compensation. Based on the exclusion criteria, we excluded those who were unwilling to participate and did not ll in the questionnaires completely. Most participants completed this scale for approximately 15 minutes.

Classical Test Theory (CTT)
The techniques of CTT were applied to measure some observable information (scale scores) to obtain insights into variables (nutrition literacy level of middle school students) that could not be directly observed 30 . The following eight criteria were used to determine the questions included in the tool 31 : 1) Frequency analysis: the item response rates ranged from 98.60% to 100%, with no response rate exceeding 80% on either option.
3) Discriminant validity: according to the scores of 27% percentile before and after the total score of the scale 32, 33 , the high and low groups were divided and the items with no difference in scores between the two groups were deleted (P > 0.05). 4) Intra-class correlation coe cient (R): the R between all items under the same sub-indicator and the items with r value low than 0.2 or greater than 0.9 were deleted. 5) Entry-dimension consistency: the R between all items under the same sub-indicator and the items with r value low than 0.2 or greater than 0.9 were deleted. 6) Item-dimension R: the R between one item and one dimension should be greater than the R between the item and other dimensions. 7) Factor analysis: excluding the maximum factor load < 0.4 or each common factor item containing < three items. 8) Cronbach's alpha coe cient: if an item was deleted and a signi cant increase was present in the alpha coe cient, then deletion was considered.
Items with three or more of the above criteria for deletion should be deleted.

Face Validity
The face validity of the questionnaire was veri ed in accordance with the evaluation of middle students and experts. At the end of the questionnaire, an open question was set, mainly by asking the respondents about the questions they encountered when lling out the questionnaire (e.g. is there any di cult or ambiguous question in the questionnaire?) to verify the face validity.

Content Validity
In this study, content validity index (CVI) was used to evaluate the content validity. Eighteen experts were asked to measure the correlation between each item of the scale and the index content of the scale and were required to use 0 (no) and 1 (yes) to evaluate whether the items could properly re ect the nutrition literacy of students. The items with the CVI value of 0.7 or above were kept 34 (Additional le 1: Supplementary Material S1).

Construct Validity
Exploratory factor analysis (EFA) and con rmatory factor analysis (CFA) were used to evaluate the structural validity only on Likert-type items. The sample was randomly divided into two: one for EFA and the other for CFA. Principal component analysis was rstly performed unrotated by using maximum likelihood extraction and (determining the number of factors to retain) the number of factors was determined using the following criteria: the eigenvalue-greater-than-one rule (K1), the percentage explained by each factor, scree plot, Kaiser-Meyer-Olkin (KMO) and Bartlett's test 35 . Then, CFA was used to validate the content and characteristics of the basic constructs, whilst the choice of items was validated using EFA. Evidence of model t was evaluated using indices of absolute t and GFIs. The reasonable threshold levels of these indices for CFA were considered as χ2 test results P > 0.05, χ2/df < 3, root mean square error of approximation < 0.08, GFI > 0.9 and AGFI > 0.9 36 .

Reliability
The reliability of the scale was evaluated by internal consistency (Cronbach's alpha) and split-half reliability was used to represent the reliability of the total scale and each dimension.
Statistical analysis SPSS 22.0 and STATA 16.0 were used for data analysis and AMOS 23.0 was used to conduct the con rmatory factor analysis, which was only used on Likerttype items, whilst STATA's tetrachoric correlation was used for binary items (10 true/false items).  18, 41 42, 43 . However, none of these instruments is suitable for evaluating the nutrition literacy of middle school students in Chongqing with the deference of region and culture.
Conceptual framework development and initial questionnaire item generation The framework was designed based on Nutbeam's hierarchical model of health literacy, with classical functional, interactive and critical levels 20

Phase 2: Assessment Of Expert Consultation
After two rounds of expert consultation, we deleted, added and modi ed the items according to the experts' opinions, a pool of 59 items was formed to measure three domains (functional, interactive and critical) and six components (obtain, understand, apply, interact, medial literacy and critical) of nutrition literacy (Additional le 1: Supplementary Material S1).

Item Analysis Based On Ctt
Item analysis was conducted using CTT and the instrument included 14 items that did not meet the criteria. However, after the theoretical relevancy of nutrition literacy and the results of group discussion were combined, the nal scale included 56 items (three items removed: Q7_1.   The Cronbach's alpha coe cient of the scale was tested for internal consistency. The results showed that the Cronbach's alpha coe cient of the total scale was 0.849, whilst that of the six subscales was 0.648-0.942, indicating that the reliability of internal consistency of the scale was well stable. The total scale split-half reliability coe cient was 0.521, whilst that of the six subscales was 0.509-0.914 (Table 2).

Discussion
This study was the rst to report the validity and reliability of the assessment instrument to comprehensively evaluate the level of nutrition literacy of middle school students in Chongqing. Based on the conceptual models of health literacy illustrated by Nutbeam, nutrition literacy was divided into FNL, INL and CNL 20 . After a two-round Delphi survey, the items of the nutrition literacy scale of middle schoolers in Chongqing were nally determined, including three primary indicators, six sub-indicators and 52 items. The results of the research con rmed the reliability and validity of CM-NLS, including content and structure.
The expert group had a high degree of agreement on the highest score for FNL. Other researchers also believed that functional literacy is the basis of interactive and critical literacy 27 . FNL is important for middle schoolers to acquire nutrition knowledge and develop healthy dietary habits. Besides, middle schoolers should take the initiative to obtain and understand nutrition information, such as understanding the whole process of food from farm to table (production, processing, transportation, purchase and handling). A notable detail that the authors agreed with Thomas et al., who revealed that nutrition literacy should also emphasize the importance of practice in nutritional information 26,46 . However, the practical ability of middle school students in China was insu cient; they were relatively weak in practical skills, such as food purchasing and food cooking 47,48 . Therefore, the 'application skills' mainly emphasized dietary behavior whilst ignoring the importance of students' application of nutrition information. This problem should be given attention in the future. In addition, the INL score was high. According to the results of the pilot study, middle school students mostly obtained information from families and teachers; it was a one-sided communication between teachers and parents to students, leading to the lack of initiative of students to interact with others. Although the CNL score was the lowest amongst the three subscales, the students' media literacy and critical skills are still necessary to understand. In recent years, adolescents have increasing access to nutrition information 49 , which may lead to information misunderstanding and confusion 50 . Wadsworth advocated that media literacy (the ability to critically view and understand information) should be included in diet-related education 51 . In the framework concept, media literacy and critical skills were placed in the section of CNL, which made up for the de ciency that only a few researchers had explored 16 .
After CTT and structural analysis, some items differing from the results of expert consultation and the purpose of the research were deleted or adjusted. For instance, the item 'It is easy to understand the contents of the Dietary Guidelines for Chinese Residents' was deleted. This deletion could imply that the Dietary Guidelines for Chinese Residents was di cult to understand for middle school students with lacking professional guidance to understand 24 . The CFA results showed that except for the P-value, GFI and AGFI were close to 0.9, whilst other data met the requirements. This nding may be in uenced by the sample size; the chi-square value of the model was very sensitive to the sample size and the larger the sample size was, the more likely it was to reach a signi cant value 52,53 . The EFA and CFA results showed a poor correlation between application skills and the other ve dimensions. The application skills should not belong to FNL and be contrary to the judgment of professional knowledge. This nding could be interpreted as three factors at interplay in obtaining the results. Firstly, according to Bloom's taxonomy (revised), the six levels of cognitive learning are remembering, understanding, applying, analyzing, evaluating and creating 54 .
The cognitive development of middle school students may not be advanced enough to 'apply' the skills 55 . Secondly, the application skills may have some regional differences in the real world; the original literature and guidelines focused on adolescents across China but Chongqing is a multi-ethnic area 56 . Finally, the Chinese education system places more emphasis on route learning and silent learning rather than application, leading to the original hypothesis not being veri ed 57 . The Cronbach's alpha value of the screened scale was 0.851; the values of the other items were between 0.648 and 0.942 and the mean value of CVI containing 56 items was 0.91, indicating that CM-NLS was consistent with the theoretical framework and had reasonable reliability and validity.
This study is based on literature research. A total of 18 experts were consulted in strict accordance with the Delphi method. The results combined subjective assessment with objective analysis. Furthermore, CTT and validity tests were used to screen the items and the relationship amongst theory, experience and data in scale design was well-handled 31 . However, the study still has several limitations. Firstly, the sample size was not representative of the entire population. As CM-NLS only included students from one school, it still needs to be applied to a larger population to verify its feasibility and validity. Besides, due to the COVID-19 epidemic situation, the preliminary survey of this study was mainly conducted online, and some information bias may be present in the self-report of students. Furthermore, the design of nutrition literacy assessment instruments is dynamic and needs to be modi ed and improved under the updated guidelines and literature.

Conclusions
Overall, CM-NLS is a valid and reliable instrument to measure nutrition literacy amongst middle school students in Chongqing. It could be used to identify key problems, such as nutrition education intervention, and pertinence measures to education. It could also provide a scienti c basis for the implementation of nutrition education strategies. Considering that nutrition literacy is dynamic, more samples are needed to modify, optimize and apply the scale based on the actual situation in the future. The study was presented to the Ethics Committee of Chongqing Medical University. All participants were informed about the study and provide informed consent before participation in the Delphi expert consultation and pilot study.

Consent for publication
Not applicable.

Authors' agreements
This manuscript has not been published elsewhere. This manuscript has not and will not be submitted for publication elsewhere until a decision is made regarding its acceptability by your journal. If accepted for publication, it will not be published elsewhere. All authors have contributed substantially to the manuscript and approved nal submission. All authors accept full responsibility for all aspects of the work described.

Availability of data and materials
The data that support the ndings of this study are available from the corresponding author upon request.

Funding
This research did not receive any speci c grant from funding agencies in the public, commercial, or not-for-pro t sectors.
Flow chart of the scale development procedure First-order CFA analysis factor loading construct validity study for CM-NLS Second-order CFA analysis factor loading construct validity study for CM-NLS

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.