Development, Reliability, and Validity of the“Knowledge-Attitude-Practice” Questionnaire of Foreigners on Traditional Chinese Medicine Treatment

Objective To develop a “knowledge-attitude-practice” questionnaire as an evaluating tool of foreigners' cognition on TCM treatment, so as to promote the internationalization of TCM. Methods The questionnaire was based on the “knowledge-attitude-practice” model and adjusted by expert consultation using the Delphi method. After conducting a survey among foreigners, Cronbach's α and exploratory factor analysis were used to test the internal consistency reliability and structural validity of the questionnaire, respectively. Results A total of 10 experts participated in two rounds of expert consultation. The recovery rates of two rounds of expert consultation form were 100.0%. The coefficient authority in two rounds of expert consultation was 0.87 and 0.88, respectively. The concentration of expert opinions in the knowledge, attitude, and practice dimensions was 3.80 to 4.70 points, 3.70 to 4.50 points, and 3.60 to 4.40 points, respectively, in the first round and 4.30 to 4.80 points, 4.10 to 4.60 points, and 4.00 to 4.50 points, respectively, in the second round. The coefficient of variation in the knowledge, attitude, and practice dimensions was 0.10–0.32, 0.16–0.29, and 0.19–0.35, respectively, in the first round and 0.09–0.19, 0.15–0.25, and 0.16–0.31, respectively, in the second round. The W value and significance test x2 in the first round were 0.657 and 218.620 while those in the second round were 0.671 and 181.181(P < 0.001). 8 items were deleted and 1 item was added, and other reserved items were modified according to the statistical analysis results of evaluation items and expert suggestions after the first round and there were no changes after the second round. The revised questionnaire includes three dimensions of knowledge, attitude, and practice, with a total of 30 items. After translating the questionnaire into English, it was conducted in 176 foreigners. Cronbach's α coefficient of the total questionnaire, knowledge dimension, attitude dimension, and practice dimension was 0.908, 0.781, 0.823, and 0.918, respectively. Exploratory factor analysis extracted 3 factors with a cumulative contribution of 54.090%. After testing reliability and validity, 1 item was deleted, leaving 29 items. Conclusions After two rounds of expert consultation based on the Delphi method, the results of expert authority, expert coordination, and expert opinions' concentration were promising, and the expert consultation results were reliable. The “knowledge-attitude-practice” questionnaire of foreigners on TCM treatment in English had good reliability and validity and can evaluate foreigners' cognition on TCM treatment.


Introduction
Traditional Chinese medicine (TCM) had a long history in China and was a precious traditional culture of Chinese people. In the middle of the 19 th century, the British East India Company named Chinese medicine "TCM" in order to distinguish from Western medicine. Until 1936, the "regulations on TCM" made by the Kuomintang government officially established the name of "TCM" [1]. In 1996, China introduced the concept of "internationalization of TCM" for the first time [2], which was aimed at making TCM accessible to the world [3]. Up to now, TCM had spread to 183 countries and regions in the world. According to the World Health Organization (WHO), 103 member states had approved the use of acupuncture [1]. As reported by Overview of International Traditional Chinese Medicine Development and Legislation [4], there were more than 1,500 TCM clinics in the Netherlands, more than 3,000 TCM clinics in the UK, more than 3,000 acupuncture clinics in France, more than 3,000 TCM clinics in Canada, and more than 2,500 TCM clinics in Australia.
However, after more than 2 decades of efforts to the internationalization of TCM, the progress appeared to be stagnated [2]. Although the scale of TCM development was growing, the depth of development was not enough. Many people blindly denied the scientific nature of TCM and questioned the standardization of TCM diagnosis and treatment system, which made TCM difficult to be trusted by more people [5]. Although acupuncture and moxibustion were increasingly valued in many countries because they were relatively easy to learn and had been proven efficient in treating painful diseases, Chinese medicine was restricted or banned in some countries because of the difficulty in learning prescriptions as well as the exceeding content of toxicity, pesticide, and heavy metal in some Chinese medicine. As an important part of TCM treatment, massage was in a more marginal state compared with these treatments in many countries [4].
ere were limited studies on the challenges facing the internationalization of TCM [2]. Some studies had pointed out the reasons hindering the further internationalization of TCM, such as the difficulty to understand the true connotation of TCM [3], the huge barriers between countries' trade [5], the lacking of the strict industrial standard [5], the unscientific export structure [5], the limited input in research [3], and the absence of medical insurance [4]. e "knowledge-attitude-practice" (KAP) questionnaire was generally used to understand the knowledge, attitude, and practice of the target population in health care as well as the demand and acceptance degree of relevant content [6]. Up to now, there was no study of foreigners' knowledge, attitude, and practice on TCM treatment. Our research group developed a KAP questionnaire as an evaluating tool of foreigners' cognition on TCM treatment, which can further promote the internationalization of TCM.

Method
is study included three phases. Phase 1 was from October to November 2019. e questionnaire was based on the "knowledge-attitude-practice" model and adjusted by expert consultation using the Delphi method. Phase 2 was conducted in December 2019. e questionnaire was translated into English and further revision to the questionnaire items was made following a pretest in 20 participants. Phase 3 was a formal survey to test the reliability and validity of the questionnaire from February to March 2020 ( Figure 1).

Developing the KAP Questionnaire Items.
Using the "knowledge-attitude-practice" model as the theoretical framework, the initial KAP questionnaire was developed on the base of a large number of domestic and foreign literatures on TCM, focus group discussions, as well as interviews with TCM experts.

Adjusting the KAP Questionnaire Items through Delphi
Method.
rough exchanging information and revising feedback many times, the Delphi method finally turned the expert opinions into the group consensus [7] and was used to develop evaluation index and scale [8]. After developing the initial KAP questionnaire, the Delphi method was adopted to adjust and confirm the questionnaire items.

Expert Consultation
Group. Expert inclusion criteria were as follows: (1) engaged in TCM-related work for more than 5 years with rich theoretical knowledge, diagnosis, and treatment background as well as practical experience; (2)     from the research group (item which met one of these four criteria can be included in the questionnaire) [6]. Based on the inclusion criterion, the items of the initial KAP questionnaire were adjusted according to experts' suggestions. irdly, all adjustments were summarized and illustrated, and then the new expert consultation form including the revised KAP questionnaire was sent to experts. Until the opinions of all experts tend to be the same, the KAP questionnaire was confirmed. In most studies, two rounds of expert consultation were used.

Expert Consultation Form.
e expert consultation form included 4 parts: introduction of the research background, instruction for filling in the form, general information of expert, and the initial KAP questionnaire. e 5-point Likert rating method was used to determine the importance degree of items, which was set as "very Table 3: Initial "knowledge-attitude-practice" questionnaire of foreigners on TCM treatment. Knowledge K1. Traditional Chinese medicine is extensive and profound K2. Traditional Chinese medicine is the Chinese traditional culture K3. Traditional Chinese medicine is effective in treating chronic disease K4. Traditional Chinese medicine is effective in treating acute disease K5. Traditional Chinese medicine is effective in treating severe disease K6. Chinese medicine is one of the treatments of traditional Chinese medicine K7. Acupuncture and moxibustion is one of the treatments of traditional Chinese medicine K8. Cupping is one of the treatments of traditional Chinese medicine K9. Massage is one of the treatments of traditional Chinese medicine Attitude A1. You are willing to treat the disease by drinking Chinese medicines A2. You are willing to treat the disease by receiving acupuncture and moxibustion A3. You are willing to treat the disease by cupping A4. You are willing to treat the disease by doing a massage Note: if you have not received TCM treatment, please go to A9 A5. You are satisfied with the model of diagnosis and treatment of traditional Chinese medicine A6. You are satisfied with the therapeutic effect of Chinese medicines A7. You are satisfied with the therapeutic effect of acupuncture and moxibustion A8. You are satisfied with the therapeutic effect of cupping A9. You are satisfied with the therapeutic effect of massage A10. You trust the therapeutic effect of traditional Chinese medicine A11. Traditional Chinese medicine should be publicized and prompted A12. Evidence-based research in traditional Chinese medicine should be strengthened Practice P1. When you suffer from chronic diseases, you will choose traditional Chinese medicine P2. When you suffer from acute diseases, you will choose traditional Chinese medicine P3. When you suffer from severe diseases, you will choose traditional Chinese medicine P4. When you have headache and vertigo disease, you will choose traditional Chinese medicine P5. When you suffer from chest tightness and shortness of breath, you will choose traditional Chinese medicine P6. When you suffer from insomnia, you will choose traditional Chinese medicine P7. When you suffer from diarrhea and constipation, you will choose traditional Chinese medicine P8. When you suffer from sterility and infertility, you will choose traditional Chinese medicine P9. When you suffer from hands ache or feet ache, you will choose traditional Chinese medicine P10. When you have a bad appetite, you will choose traditional Chinese medicine P11. When you are tired, you will choose traditional Chinese medicine Note: if you are male, please go to P14 P12. When you suffer from menstruation disorder and dysmenorrhea, you will choose traditional Chinese medicine P13. When you suffer from other gynecological diseases, you will choose traditional Chinese medicine P14. You would like to attend the lecture about traditional Chinese medicine P15. You would like to recommend traditional Chinese medicine to others P16. You would like to use health tea or medicinal diet made by Chinese medicines Evidence-Based Complementary and Alternative Medicine important, important, neutral, unimportant, and very unimportant" and was rated as 5.0 points, 4.0 points, 3.0 points, 2.0 points, and 1.0 point, respectively. Expert familiarity degree was set as "very familiar, familiar, neutral, unfamiliar, and very unfamiliar" and was rated as 1.0 point, 0.8 points, 0.5 points, 0.2 points, and 0.0 points, respectively. Expert was supposed to judge items based on the theoretical analysis, practical experience, domestic and foreign references, and subjective intuition, each of which can be divided into "large, medium, and small" according to the influence degree of judgment. e sum of these four parts' scores formed the judgment coefficient (Table 1). And suggestions for the initial KAP questionnaire can be written in the revision section of the form.

Participants and Study Setting.
In the attitude dimension, 4 items (A1, A2, A3, and A4) were not required for foreigners who had not received TCM treatment. In the practice dimension, 2 items (P8 and P9) were not required for male foreigners. ese 6 items were not suitable for result statistics and analysis, leaving 24 items. According to the sample, size should be 5-10 times of the number of the questionnaire items, and 20% of the sample size should be expanded after considering factors such as invalidity [9], and the final calculated sample size was at least e KAP questionnaire was used to conduct a survey of foreigners in Sir Run Run Shaw Hospital by the convenience sampling method. To make it more specified and reliable, we made some inclusion criteria for foreigners enrolled in our study, including comprehension of the content of the questionnaire and voluntary participation in this questionnaire. e exclusion criterion was participants who had serious illnesses, so that unable to complete the survey. All participants provided informed consent. e study was approved by the Ethics Committee of Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University. Some participants filled in paper questionnaires on site, while others filled in electronic questionnaires on mobile phones after scanning the QR code. Before conducting the survey, a trained researcher gave instructions to the participants. In the process of filling in the questionnaire, no other explanation was given. For participants who have chosen the electronic questionnaire, the same instructions were placed on the front page. Paper questionnaires were collected on site and checked by the researcher. If there were any incomplete answers, participants were asked to clarify. Electronic questionnaires were checked by the researcher from backstage and invalid questionnaires would be eliminated.

Grading Method.
A common grading method was used for each dimension in the KAP questionnaire; 3-point Likert rating method was used for knowledge and attitude dimensions, which was set as "agree, not sure, and disagree" and was rated as 3.0 points, 2.0 points, and 1.0 point, respectively, and 4-point Likert method was used for practice dimension, which was set as "often, sometimes, occasionally, and never" and was rated as 4.0 points, 3.0 points, 2.0 points, and 1.0 point, respectively.

Pretest.
In order to check whether the expression of items could be understood, the KAP questionnaire was piloted among 20 participants on site.

Formal Survey.
e survey was carried out from February to March 2020. Altogether, 176 questionnaires were collected anonymously.

Statistical Analysis.
e data were entered into EpiData 3.1 Entry for data documentation. SPSS 23.0 and Excel 2003 were applied to further analyze the data. Data were presented as the mean ± standard deviation or n (%). Cronbach's α coefficient was applied to evaluate the reliability of the KAP questionnaire. Exploratory factor analysis was used to assess construct validity. e Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett test of sphericity were used to assess the suitability of the data for factor analysis. P < 0.05 was considered statistically significant.

Focus Group Discussions.
ere were three rounds of focus group discussions and a total of 8 members of the research group were joined in. All participants felt free to talk openly and gave honest opinions, who not only expressed their own opinions but also responded to other members and questions posed by the leader. Table 2 shows that 4 (50.0%) were females, 4 (50.0%) were 41-50 years old, 6 (75.0%) had a master's degree, and 6 (75.0%) majored in TCM. e initial KAP questionnaire (Table 3) was determined and formed with three dimensions of knowledge, attitude, and practice and a total of 37 items.

Expert Consultation Based on Delphi Method.
In this study, we adopted two rounds of expert consultation, enrolling a total of 10 experts. e result showed that 9 (90.0%) were females, 7 (70.0%) were 31-40 years old, 7 (70.0%) had were associate chief physicians, and 5 (50%) worked for 11 to 20 years. e detailed information is listed in Table 4. e recovery rates of expert consultation form were 100% in two rounds of expert consultation. And the coefficient authority in two rounds of expert consultation was 0.87 and 0.88, respectively. It was generally composed of expert judgment coefficient and expert familiarity degree, ranging from 0 to 1.00. e data were calculated based on the following formula: the coefficient authority of expert � (expert judgment coefficient + expert familiarity degree)/2. To judge the degree of expert opinion coordination, Kendall's W was adopted. e value of Kendall's W was between 0 and 1, and the higher the value means, the opinion of the experts was more consistent and the coordination of the questionnaire was better [10]. In the first round, the W value was 0.657 (significance test X 2 � 218.620,P < 0.001). In the second round, the W value was 0.671 (significance test X 2 � 181.181,P < 0.001). After two rounds of expert consultation, the experts' opinion on each item tends to be consistent. e degree of concentration of expert opinions was analyzed by the importance value and coefficient of variation. e greater the importance assignment, the smaller the coefficient of variation value indicated that the degree of expert opinion concentration was better [10]. e importance value in the knowledge, attitude, and practice dimensions was 3.80 to 4.70 points, 3.70 to 4.50 points, and 3.60 to 4.40 points, respectively, in the first round and 4.30 to 4.80 points, 4.10 to 4.60 points, and 4.00 to 4.50 points, respectively, in the second round. e coefficient of variation in the knowledge, attitude, and practice dimensions was 0.10-0.32, 0.16-0.29, and 0.19-0.35, respectively, in the first round and 0.09-0.19, 0.15-0.25, and 0.16-0.31, respectively, in the second round. And the detailed information is listed in Table 5. Compared to the first round, the importance value of items in the second round was larger while the coefficient of variation value was smaller.
In the first round of expert consultation, the research group deleted 8 items, modified 5 items, and added 1 item based on the inclusion criterion and experts' suggestions. And there were no changes in items after the second round. . When you suffer from chronic diseases, you will choose traditional Chinese medicine P2. When you suffer from insomnia, you will choose traditional Chinese medicine P3. When you suffer from diarrhea and constipation, you will choose traditional Chinese medicine P4. When you suffer from sterility and infertility, you will choose traditional Chinese medicine P5. When you have a bad appetite, you will choose traditional Chinese medicine P6. When you are tired, you will choose traditional Chinese medicine P7. When you suffer from chills and fever, you will choose traditional Chinese medicine Note: if you are male, please go to P10 P8. When you suffer from menstruation disorder and dysmenorrhea, you will choose traditional Chinese medicine P9. When you suffer from other gynecological diseases, you will choose traditional Chinese medicine P10. You would like to attend the lecture about traditional Chinese medicine P11. You would like to recommend traditional Chinese medicine to others P12. You would like to use health tea or medicinal diet made by Chinese medicines 6 Evidence-Based Complementary and Alternative Medicine us, the revised KAP questionnaire of foreigners on TCM treatment was confirmed, which included 30 items.

Phase 2: Translation and Pretest.
e revised KAP questionnaire was translated into English by 3 experts who had not only a background in TCM but also a good level of translating Chinese into English. In order to avoid bias, one of these experts was not part of the research group. ey did discussions many times and generated the unified English version of the KAP questionnaire. en this KAP questionnaire was piloted among 20 foreigners, and there was no adjustment after pretesting. e formal KAP questionnaire ( Table 6) was formed with three dimensions of knowledge, attitude, and practice and a total of 30 items.

General Information of Participants.
After removing 4 invalid questionnaires, there were 176 useful questionnaires, with a response rate of 97.78%. Of the participants, 90 (51.1%) were males, 103 (58.5%) were 20-39 years old, and the other general information is summarized in Table 7.

Reliability Analysis.
Cronbach's α coefficient was 0.908 overall, 0.781 for the knowledge dimension, 0.823 for the attitude dimension, and 0.918 for the practice dimension, indicating good internal consistency reliability.

Validity Analysis.
Construct validity was evaluated using exploratory factor analysis. In the Kaiser-Meyer-Olkin (KMO) test, the r value of 0.872 indicated that the data definitely lend itself to factor analysis. A Bartlett test of sphericity (X 2 � 2234.69, df � 276, P � 0.000) indicated that the analysis model was appropriate. For these reasons, it was acceptable to adopt factor analysis to test the construct reliability of this questionnaire [11,12].
Under the condition of unrestricted extraction factor values, a total of 6 factors with the characteristic value larger than 1 were extracted, and the accumulated contribution rate was 68.189%. e common factors were extracted by the principal component analysis method and rotated by the maximum difference method. Gravel figure (Figure 2) showed that the slope line gradually flattens after the fourth factor, and 3 dimensions were preset on the questionnaire. e cumulative variance contribution rate of these 3 factors was 54.090%. As shown in Table 8, after using exploratory factor analysis, item P11 should belong to the attitude dimension. Considering that item P11 was definitely the item related to behavior, in a further survey, it should be deleted because of the contradiction.

Discussion
e present study had developed a KAP questionnaire of foreigners on TCM treatment in English with good internal consistency reliability and construct validity. It was hoped that the application of this questionnaire would further our understanding of how TCM treatment affected foreigners and facilitated future promotion in the internationalization of TCM. e brevity and understandability of the item likely facilitated the completion of the questionnaire by participants [12]. e number of items in a questionnaire had a significant influence on the quality of responses, and a time limit of 15-30 min was ideal [12]. e formal questionnaire contained a total of 29 items so that participants can be ensured to complete it about 20 minutes. e questionnaire was scientific and authoritative to some extent since it was build based on the Delphi method.
e Delphi method differed from traditional surveys in that panelists receive feedback as to how the panel was responding throughout the study. is encouraged them to consider their own responses alongside those of the entire panel and clarified or amended their responses based on this information [13]. Choosing appropriate experts was the key to the success of the Delphi method [14]. According to relevant studies, the coefficient authority of expert ≥0.80 indicated that experts had a good grasp of the choice of item [15]. As showed in the results, the coefficient authority in Evidence-Based Complementary and Alternative Medicine two rounds of expert consultation was 0.87 and 0.88, respectively, indicating that the invited experts had a high degree of authority in TCM and the consultation results were reliable. us, with the Delphi method and the high quality of experts, the questionnaire was rather scientific and dependable. e reliability of the questionnaire was evaluated as internal consistency reliability. Cronbach's α represented the internal consistency of the questionnaire which showed whether the items in the same group were testing the same concept [16]. Many researchers maintained that Cronbach's α coefficient ≥0.7 indicated good internal consistency [11,12]. In this study, Cronbach's α coefficient of the questionnaire's total score was 0.908. Cronbach's α coefficient was 0.781 for the knowledge dimension, 0.823 for the attitude dimension, and 0.918 for the practice dimension.
e above results showed that the questionnaire displayed good internal consistency, whether considering overall contents or individual dimensions. e overall content of the questionnaire can reflect an accurate portrayal of foreigners' cognition of TCM treatment while the contents of each dimension can accurately reflect the corresponding elements of TCM treatment.
Construct validity was used to evaluate whether or not the questionnaire corresponded to the theoretical design upon which the questionnaire was based. In this study, by analyzing the items within each common factor, we found that items P6, P2, P5, P3, P7, P4, P1, P12, and P10 reflected the practice dimension, items A3, A10, A2, A9, P11, A1, A4, and A11 reflected the attitude dimension, and items K4, K5, K6, K2, K7, K3, and K1 reflected the knowledge dimension. Results of the analysis showed that the structure of this scale was in accordance with the theoretical framework and original design except for item P11. Item P11 reflected the behavior of recommending TCM to others rather than attitude. Considering this contradiction, item P11 was deleted from the questionnaire, leaving 29 items.  Overall, the KAP questionnaire of foreigners on TCM treatment was simple and convenient and had relatively strong functionality and operability. Our study had several limitations. e experts selected in this study were clinical doctors working in hospitals. Although they had a good TCM background and most of them also taught in medical schools, their career distribution was relatively single. e participants selected in this study were only from a Grade A Class ree hospital in Hangzhou, and the representativeness of the sample was limited. e questionnaire should have been administered again in the same group of participants after at least 2 weeks to ensure test-retest reliability. However, due to the mobility of foreigners in hospitals, doing retest was less feasible. e reliability of the questionnaire was evaluated as internal consistency reliability. In future study, multicenter large sample surveys should be conducted in foreign communities, so that the representativeness of the sample can be increased. Meanwhile, the reliability and validity of the questionnaire should be further analyzed by expanding the sample size. After preliminarily applying the questionnaire in foreign communities, we can evaluate foreigners' cognition and application of TCM treatment, explore the relationship between foreigners' knowledge, attitude, and practice of TCM treatment, and find some influencing factors, so that to provide some suggestions for promoting the internationalization of TCM.

Conclusion
e "knowledge-attitude-practice" questionnaire of foreigners on traditional Chinese medicine treatment in English had been built using the Delphi method based on a panel of qualified experts. After two rounds of expert consultation, the results of expert authority, expert coordination, and expert opinions' concentration were promising, indicating the construction of the questionnaire was scientific. And the questionnaire had good reliability and validity, which can be used to evaluate foreigners' knowledge, attitude, and practice on TCM treatment.

Data Availability
e data came from the questionnaire survey results can be available upon request.
Ethical Approval e study was approved by the Medical Ethics Committee of Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University, reference number: scientific research 20200323-37.

Consent
Written consent was obtained from the study participants.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper. Evidence-Based Complementary and Alternative Medicine 9