A systematic review of measures of healthcare workers’ vaccine confidence

ABSTRACT Healthcare workers (HCW) perceptions toward vaccines influence patient and community vaccine decision making. In an era of rising vaccine hesitancy, understanding HCW vaccine confidence is critical. This systematic review aims to review instruments that have been validated to measure HCW vaccine confidence. We conducted a search in five databases in June 2023. Data was descriptively synthesized. Twelve articles describing 10 different tools were included. Most tools included dimensions or items on vaccine knowledge (n = 9), safety (n = 8), vaccine usefulness (n = 8), recommendation behavior (n = 8), and self-vaccination practice (n = 7). All, except one study, were conducted in high-income countries. There was variability in the quality of the validation process. There is limited existing literature on development and validation of tools for HCW vaccine confidence. Based on the tools currently available, the Pro-VC-Be tool is the most well validated. Further research needs to include low- and middle-income contexts.


Introduction
Vaccination plays a pivotal role in public health by reducing morbidity and mortality associated with infectious diseases and their acute and long-term manifestations. 1Between 2010 and 2018, measles vaccination alone has prevented over 20 million deaths. 2 Among the key drivers behind vaccination success are healthcare workers (HCWs), who serve as front-line providers of immunization services. 35][6][7][8] Some evidence suggested that low confidence in vaccines among HCWs has been associated with low uptake rates of the vaccines. 9,10accine hesitancy is a complex phenomenon.It has been defined as a behavior in delaying or refusing a particular vaccine or vaccination generally despite availability, 11 or as a state of indecisiveness regarding vaccination. 12,136][17][18] For example, the coverage of diphtheriatetanus-pertussis (DTPcv1) containing vaccines decreased by 21% between 2021 and 2022, and one of the factors impacting the drop in coverage is vaccine hesitancy. 19 Strategic Advisory Group of Experts on Immunization working group on vaccine hesitancy proposed the 3Cs of vaccine hesitancy as confidence (trust in vaccines and those delivering them), complacency (low risk, not necessary to take preventative measures), and convenience (affordability and accessibility).11 Terminology around vaccine hesitancy and confidence is not consistent throughout the literature, and we refer to vaccine confidence throughout this paper.In an era marked by evolving vaccine production, increased public scrutiny, and rising vaccine hesitancy, it is imperative to gain a nuanced understanding of waning vaccine confidence among HCWs.20 The validation of survey tools or instruments is an essential component in measuring vaccine confidence among HCWs.Validation promotes reliable, accurate, and effective tools for measuring the complex constructs of vaccine behaviors among this important population.21,22 The process often involves rigorous statistical and methodological evaluation, as well as cross-cultural adaptations to ensure the tools' appropriateness in various contexts.21 Tool validation is therefore important to ensure that the data collected reflect the unique characteristics and dynamics of HCW vaccine confidence.
There has been an increase in literature on HCW's vaccine confidence using a variety of qualitative and quantitative methods.Examining the HCW's vaccine confidence is of particular concern due to HCWs higher risk of transmission of illness, 14 how their vaccine confidence influences their likelihood to recommend vaccines, 23 and since HCWs are trusted sources for vaccine uptake among patients. 7,8hese factors make it important to examine HCW vaccine confidence separately from parental or general population vaccine confidence. 23However, to our knowledge, there is currently no published systematic reviews on the validation of tools used to assess vaccine confidence among HCWs.Therefore, this systematic review aims to review the survey tools/instruments that have been developed and validated to measure HCW vaccine confidence.This review will provide valuable insights into the development of standardized and robust assessment measures of vaccine confidence among HCWs, which can inform more targeted interventions and policies aimed at enhancing vaccine confidence among HCWs.

Methods
This is a systematic review.The review is based on current best practices utilizing the Joanna Briggs Institute (JBI) systematic review framework. 24The population, concept, context (PCC) framework was used to guide the development of our research question. 25The population being HCWs; the concept tools to measure vaccine hesitancy or vaccine confidence/ trust, vaccine acceptance; context including a global setting.This framework as well as the literature review culminated in the research questions: What validated tools exist to measure healthcare worker vaccine confidence/hesitancy/acceptance?No review protocol exists, and the systematic review has not been registered.

Search strategy
The PRISMA checklist and flow diagram were used to guide the search and presentation of results. 26The search strategy was developed in Medline (Ovid) in collaboration with librarians at the Karolinska Institutet University Library.Medline, Web of Science, CABI: CAB Abstracts, and Global Health and Sociological Abstracts were searched, and Publicly Available Content database was used as complementary search.The last search was conducted in 2023-06-08.For each search concept Medical Subject Headings (MeSH-terms) and free text terms were identified on Medline.No language restriction was applied, and databases were searched from inception with no date restrictions.The search was then translated, in part using Polyglot Search Translator. 27The strategies were peer reviewed by another librarian prior to execution.Some of the key search terms used were immunization, immunization programs, exp vaccination, exp vaccines, vaccine confidence, vaccine hesitancy, vaccine acceptance, anxiety, awareness, behavior, choice behavior, communication barriers, health knowledge, attitude, and practice, intention, health personnel, benchmarking, health care surveys, quality assurance, health care, survey and questionnaire.The full search strategy is available in Appendix A.
After running the searches in all databases, a de-publication process was done using the method described by Bramer et al. 28 Finally, DOIs were compared to avoid duplicate articles.For the full search strategy, any articles that assessed vaccine confidence among HCW were included, then the research team narrowed to tool development and validation during the screening process.

Eligibility criteria
We included articles that focused on measuring vaccine hesitancy/confidence/acceptance, vaccine behavior, or vaccine attitudes of HCWs.Specifically, we included peer-reviewed articles describing the tool development and validation process.The term HCWs referred to any group working with healthcare. 29The articles were included if the entire target population for the validation process was focused on healthcare workers or healthcare professionals.We included healthcare students in this definition of HCWs.We excluded systematic review or an intervention study.

Selection process
We used Rayyan.aisoftware 30 for the screening process.After the de-duplication, one researcher (EG) screened article titles and abstracts to create a short list.Initial inclusions were articles that measure healthcare worker vaccine confidence/ hesitancy/acceptance that discuss how it was measured or what tools were utilized.
Two researchers (EG and KA) did a blinded title and abstract screening, which included only articles measuring tool validation with the focus of the studies' results and discussions on how vaccine confidence was measured.From there, the articles were re-blinded and then full articles were screened using the eligibility criteria by EG and KA.Any discrepancies were discussed between EG, KA and senior researchers from the team (SHvW and BA) to determine final inclusion.Ten articles were included for the data extraction and quality assessment. 25

Data extraction and quality assessment
The extraction and quality assessment were done, blinded, by EG and KA using an extraction form developed by the team.The form was developed using guidance from JBI Framework. 24he QAVALS (Quality Appraisal Tool Specifically for Validation Studies) measure for instrument validation was utilized due to the focus on validation methods (Appendix B). 22 The tool had yes, no, or other for each of 24 items on study design and validation methods.The tool items assessed articles study designs, selection criteria, testing procedure, statistical analyses, errors, reporting, face and content validity, criterion validity, and construct validity (known groups, convergent, discriminant).Based on the questions provided and the QAVALS item description guide, the researchers assessed each article as including the item (yes), not including the item (no), or as cannot be determined, not applicable, not reported (other).In the end, based on the quality assessment alone, no articles were excluded from this process.Any disparity in the overall assessment was mediated through discussion.

Data synthesis and analysis
The extraction table was modified for clarity for the tabulation of results.Descriptive statistics were used for year of publication, country income level of study location, type of tool validated, healthcare worker type, vaccine studied, sample size, and topics/dimension assessed with the tool.A heat map was generated to examine the quality of the validation processes based on the QAVALS quality assessment tool. 22ased on the ratings given by EG and KA, each article was assigned a value of 1, 0.5, or 0 based on the 24 items in the assessment tool.Additionally, we added columns to say if the study included face and content validity, criterion validity, construct validity (known group, convergent, and discriminant), and reliability.A value of 1, shown in green on the map, was rated as yes, 0 (red) indicates no (item not included) and 0.5 (light green) indicates others.
[41][42][43][44] One study was based on the Vaccine Attitudes Examination (VAX) tool, 40 which has been previously used for patient vaccine hesitancy, and two studies validated variations of the Health Professionals Vaccine Confidence and Behavior (Pro-VC-Be) tool. 33,36The tools examined different dimensions (illustrated in Figure 2) but

United Kingdom
To develop an instrument to measure nurses' knowledge, risk perceptions, and health behaviors towards influenza, influenza vaccine, and vaccine behaviors.
King's Nurses' Influenza Vaccination Questionnaire Nurses 520 Influenza covered many related topics.36,39,42,45 The results from the studies mostly presented valid and reliable tools, but not all of the results focused on the validation process.Among the nine studies that reported Cronbach alpha scores for reliability, they all had a final score of above 0.70 on the different dimensions (Table 2).In four of the articles, they discussed that some items needed to be removed and adapted from the survey to have a strong reliability score. 33,39,41,43 Vaidity coefficients were presented in the four studies that conducted a criterion validity, and all found good criterion validity among finalized version.33,36,39,41 Half the studies' results and discussions primary focus were the results of the questionnaire itself rather than the validation process.34,35,37,38,40,43 Out of these six studies, four found that HCWs knowledge level is linked with their behavior or attitudes about vaccines. 34,35,37,38

Quality of tool validation
The heat map shows that the quality of the validation processes was mixed across the ten studies.Generally, the study design was well described and appropriate for a tool validation study.41,43 The study sample sizes had a median of 359, but a range of 108 to 2748 participants.
There was also variability in the type of validations conducted.Three studies did all three main forms of validation: face validity, criterion validity, and construct validity. 33,36,41mong the 12 studies that examined face validity, they generally were of good quality, and in Kadir et al. they conducted both a content and face validity separately. 42As the red and light green on the heat map demonstrate, criterion and construct validity were either not conducted or of a medium quality.Four studies conducted a criterion validity utilizing vaccination behavior as the comparison, 33,38,39,41 and one used the long-form Pro-VC-Be tool as the criterion for the shortform version. 36None of the other studies gave a rationale for why they did not conduct a criterion validation.][37]40,42,43 The quality assessments for the articles are illustrated with a heat map in Figure 3.

Discussion
We aimed to identify and evaluate the quality of survey tools or instruments that have been validated to measure HCW vaccine confidence, hesitancy, or acceptance.We identified 10 articles that developed and conducted validation on a tool to measure HCW vaccine hesitancy or confidence.
The tools developed covered a range of dimensions and topics relating to HCW vaccine confidence using inputs from experts.However, only the three articles on the Pro-VC-Be tool clear lay out the dimension selection process. 33,36,41articularly, for the short-form Pro-VC-Be in which the authors ran a confirmatory factor analysis on all combination of dimensions to determine the most suitable item for each dimension.9][40] Tomietto, et al. used the existing VAX tool, and thus had predefined dimensions for the validation process. 40Then Slaunwhite developed the items based on the Theory of Planned Behavior framework and a modified Perceived Behavior Control item. 39espite the variation in reporting of the dimension inclusion, all the studies aimed to examine the main determinants of vaccine confidence and behaviors.All the tools, except versions of Pro-VC-Be, had dimension assessing the HCWs knowledge and/or skills around vaccination.Knowledge and awareness around vaccines has been shown to be associated with positive vaccine attitudes and likelihood to vaccinate. 47imension on attitude, behavior, and influence allows the tools to assess the 3Cs of vaccine hesitancy: confidence, complacency, and convenience. 11Many of the tools also looked at attitudes toward vaccines, such as safety, effectiveness, vaccine usefulness, and moral responsibility, which all have been shown to be important factors influence confidence. 48,49 potentially important part of attitude is trust in vaccines or the health system, which five of the studies included as a dimension. 33,36,38,40,41he Pro-VC-Be tools were the most well-validated tool identified, which was validated three times in a long-form, short-form and international adaption successfully tested in seven countries and four languages. 33,36,41The other tools had less rigorous validation, thus it is important to ensure thorough validation processes in survey development studies.In a scoping review of validated tools to measure vaccine hesitancy from 2010 to 2019, the authors found 26 articles that mostly focused on parent perceptions of routine childhood immunizations and none on validating tools among HCWs. 21The studies presented in our systematic review help to fill the gap between the validated tools on HCW vaccine confidence. 21

Literature gaps
Results show while that there have been efforts to develop and validate tools, there were some key gaps in the literature and limitations with the existing studies.This, despite the fact that, HCW was identified as a core hesitant group during the COVID-19 pandemic. 15urthermore, our systematic review highlighted the limited geographical and income-level scope of existing research.Only one study, conducted in Israel, 37 took place outside of Europe and North America.Additionally, the data is restricted to high-income countries.This could be due to disease prioritization in the geographical area.However, this limitation hinders the generalizability of the findings and fails to account for the unique challenges and dynamics of vaccine confidence in low-and middle-income settings.A study protocol looking to validate the 5Cs scale among HCWs in South Africa was found, which would help expand the literature scope. 31Grjesing et al. explain that the validation of a tool in one region does not mean that it is reliable and valid in another time, culture, and context. 50hus, to ensure comprehensive insights into this global issue, tool validation efforts should extend to other geographic regions.
All except one 43 of the articles reviewed on tool development and validation for assessing vaccine hesitancy among HCWs were published in the last decade.This temporal gap suggests that while awareness of the importance of understanding HCW hesitancy toward has gained increasing attention, research on validated tools for HCW vaccine hesitancy is still emerging. 21,51hile the findings from this study revealed that some studies showed good quality and comprehensive validity, others had gaps in the validation processes.Not conducting a criterion validity or having a limited explanation is a prominent challenge encountered during the validation process.The eight studies, that did not conduct a criterion validation, are then most reliant on the content validation and reliability.This means it could be a reliable tool but knowing if the tool is measuring the correct concept is more challenging and the only data presented for validity in these articles is the content or face validations.The lack of criterion validation could be because of lack of existing, widely recognized tools to measure vaccine hesitancy among HCWs, which makes it difficult to assess the criterion validity of new tools. 52,53onsequently, there is a need for the establishment of standard reference and more validation of tools specifically for HCWs contexts. 52,53nother limitation to the articles included is that several of the studies did not fully elaborate on the effects of the validation processes.The titles, aims, and even parts of methods centered on tool development and validation, while the discussions and conclusions in several of these studies have a primary focus on presenting the survey data. 34,35,37,38,40,43hus, it was difficult for the reader to assess what changes should be made to a tool to improve validation or reliability of the tools.This highlights the importance of refocusing research efforts to place a stronger emphasis on how validation informs tool development and application to other studies. 21

Study design limitations
There are a few limitations to our study.First, the initial screening for short list generation was done by one researcher (EG) due to the high volume of searches returned.Second, although we did not have an English language restriction on the search, the diversity of languages meant that for two tools only one research extracted, and quality assessed the data.

Suggestions for further research
5][56] Our study highlights the necessity for further research specifically dedicated to the validation and reliability of tools designed for assessing vaccine hesitancy among HCWs and considering using existing validated tools, such as the Pro-VC-Be. 33,37,41Alternatively, researchers should engage in full validation processes to ensure the reliability and validity of tools designed for specific contexts.If developing a new tool, researchers should consider  46 including dimensions on knowledge, attitudes, trust in system, and vaccination behavior.
Additionally, research efforts should be expanded to encompass low-and middle-income countries is vital to gain a more inclusive understanding of different socio-economic and healthcare contexts that may present unique challenges and require tailored intervention. 57

Conclusion
We conclude that the Pro-VC-Be tool as the most useful for future research and can be used as a standard for criterion validation because the Pro-VC-Be underwent a robust validation process. 34Additionally, our systematic review emphasizes the critical need for more culturally adapted and standardized tools for assessing vaccine hesitancy among HCWs.Addressing discrepancies in the existing research settings can significantly contribute to the understanding of HCWs vaccine hesitancy and inform targeted interventions and policies in a variety of settings.

Appendix B
QAVALs quality assessment tool with researcher ratings for included articles.

Figure 3 .
Figure 3. Heat map of article quality assessment and validation Process.46 participants in the study representative of the sample population from which they were recruited?study clearly describe the outcome measures to be validated?sample size calculation performed to ensure that the study had sufficient power?study describe and justify any attrition that may have occurred?comparisons were performed, were appropriate statistical adjustments used to control for the likelihood of a type 1 error?study identify potential confounding variables and if so, were measures taken to adjust for these confounders?outcomes, did the study report standard deviations or confidence intervals for normally distributed data?If non-normally distributed data, did the study report inter-quartile ranges for the main outcomes?

Table 1 .
Article summary information: title, authors, year of publication, study setting location, study aim, tool assessed, population type, sample size, and type of vaccine.
USAAimed at improving vaccination rates by targeting physicians, and developed measures of factors influence physician immunization practices and examine the reliability and validity of their measures.Canada 3 studies in the dissertation.Focusing on Study 3. Tested the adequacy of the Theory of Planned behavior framework in explaining intention to receive vaccine including a modified Perceived Behavioral Control item set and tested the predicative ability of intention to receive vaccine on actual uptake.France, Belgium, Canada Aim to validate the Professionals Vaccine Confidence and Behaviors instrument.The instruments' objective is to measure various psychosocial factors that may play a role in vaccine confidence and vaccine behavior among different types of HCPs.
2. Web of Science Core Collection.TS/Topic = title, abstract, author keywords and Keywords Plus • NEAR/x = within x words, regardless of order • * = truncation of word for alternate endings Note: the Exact search-function was used for all the searches OR vaccin*) NEAR/2 (accept* OR anxi* OR attitude* OR awareness OR barrier* OR behavio$r* OR belief* OR choice* OR compulsory OR concern* OR conscious* OR controvers* OR critic* OR decision-make* OR decision-making* OR dilemma* OR distrust OR doubt* OR dropout* OR enable* OR exemption* OR fear* OR intent* OR knowledge OR mandatory OR misconception* OR misinformat* OR mistrust* OR objection* OR objector* OR opposition* OR perception* OR reject* OR reluctan* OR rumo$r* OR trust* OR uptake* OR willing* OR unconscious* OR unwilling*)) OR anxi* OR attitude* OR awareness OR barrier* OR behavio$r* OR belief* OR choice* OR compulsory OR concern* OR conscious* OR controvers* OR critic* OR decision-make* OR decision-making* OR dilemma* OR distrust OR doubt* OR dropout* OR enable* OR exemption* OR fear* OR intent* OR knowledge OR mandatory OR misconception* OR misinformat* OR mistrust* OR objection* OR objector* OR opposition* OR perception* OR reject* OR reluctan* OR rumo$r* OR trust* OR uptake* OR willing* OR unconscious* (health workers OR health care workers OR careproviders OR community health workers OR dentists OR dietitians OR home health aides OR midwives OR nurses OR nutritionists OR physicians OR traditional birth attendants OR traditional healers) OR immunisat* OR vaccin*) NEAR/3 (accept* OR anxi* OR attitude* OR awareness OR barrier* OR behavior* OR behaviour* OR belief* OR choice* OR compulsory OR concern* OR conscious* OR controvers* OR critic* OR decision-make* OR decision-making* OR dilemma* OR distrust OR doubt* OR dropout* OR enable* OR exemption* OR fear* OR intent* OR knowledge OR mandatory OR misconception* OR misinformat* OR mistrust* OR objection* OR objector* OR opposition* OR perception* OR reject* OR reluctan* OR rumor* OR rumour* OR trust* OR uptake* OR willing* OR unconscious* OR unwilling*))