Behavior change techniques to promote healthcare professionals ’ eHealth competency: A systematic review of interventions International of Medical Informatics

Introduction: The of eHealth is rapidly – > increasing; however, many professionals have insuffi- cient eHealth competency. Consequently, interventions addressing eHealth competency might be useful in fostering the effective use of eHealth. Objective: Our systematic review aimed to identify and evaluate the behavior change techniques applied in interventions to promote healthcare professionals ’ eHealth competency. Methods: We conducted a systematic literature review following the Joanna Briggs Institute ’ s Manual for Evidence Synthesis. Published quantitative studies were identified through screening PubMed, Embase, and CINAHL. Two reviewers independently performed full-text and quality assessment. Eligible interventions were targeted to any healthcare professional and aimed at promoting eHealth capability or motivation. We synthe- sized the interventions narratively using the Behavior Change Technique Taxonomy v1 and the COM-B model. Results: This review included 32 studies reporting 34 heterogeneous interventions that incorporated 29 different behavior change techniques. The interventions were most likely to improve the capability to use eHealth and less likely to enhance motivation toward using eHealth. The promising techniques to promote both capability and motivation were action planning and participatory approach . Information about colleagues ’ approval , emotional social support , monitoring emotions , restructuring or adding objects to the environment , and credible source are techniques worth further investigation. Conclusions: We found that interventions tended to focus on promoting capability, although motivation would be as crucial for competent eHealth performance. Our findings indicated that empathy, encouragement, and user-centered changes in the work environment could improve eHealth competency as a whole. Evidence-based techniques should be favored in the development of interventions, and further intervention research should focus on nurses and multifaceted competency required for using different eHealth systems and devices.


Introduction
The increasing use of eHealth in the organization, production, and delivery of healthcare is changing the work culture in healthcare organizations [1]. Although healthcare professionals regularly use eHealth in their work, studies show that their eHealth competency is not developed to the optimal level [2][3][4]. eHealth competency consists of four components, which are (a) psychological capability and (b) physical capability to perform professional tasks related to eHealth, and (c) automatic motivation and (d) reflective motivation toward using eHealth [5,6]. In other words, eHealth competency requires adequate eHealth knowledge, skills, and associated social and communication skills to provide high-quality care; and willingness and positive attitudes toward eHealth [6].
Implementing eHealth without simultaneously ensuring a competent workforce may have unfortunate consequences for the functioning of healthcare organizations, and thus patient health. New working methods that lack competency can disrupt workflow efficiency [7][8][9]. The challenges related to eHealth competency, such as inadequate human-technology interaction, have been associated with safety and privacy incidents, for example, with incompletely recorded patient data, diagnostic results assigned to a wrong patient, and medication errors [10][11][12].
Behavioral theory, the COM-B model [5], proposes that in addition to capability (C) and motivation (M), individuals need to have an opportunity (O) to perform a behavior (B). Opportunity refers to the optimal social and physical environment that enables behavior. Hence, based on the theory, it can be assumed that effective interventions implemented in healthcare organizations might be useful to foster healthcare professionals' eHealth competency. Fig. 1 depicts the COM-B model for eHealth performance in the course of an organizational intervention.
Previous systematic reviews of interventions promoting healthcare professionals' eHealth performance have focused only on electronic health records (EHRs) and a specific healthcare setting [13,14] or type of intervention [15,16]. The review by Gagnon et al. [17] focused on interventions promoting eHealth adoption, but more than ten years have passed from its data collection. Additionally, previous reviews [13][14][15][16][17] have not interpreted which practices could be effective in healthcare organizations because the interventions reviewed were complex, involving various interacting components. Given the rapidly increasing use of eHealth and the uncertainty of effective interventions in previous research, there is a need for using a taxonomy to examine the behavior change techniques (BCTs) for healthcare professionals in the context of the digitalization of healthcare.
This systematic review aimed to synthesize and evaluate the latest behavior change interventions to promote healthcare professionals' eHealth competency through the Behavior Change Technique Taxonomy version 1 (BCTTv1) by Michie et al. [18]. The BCCTv1 is a reliable and valid method for synthesizing the content of interventions as it labels and comprehensively describes 93 BCTs potentially applied in interventions [18]. A BCT is an observable, replicable measure, which directly applies to both the target population and behavior [19]. Identification of BCTs in heterogeneous interventions allows analyzing which common BCTs are associated with effective outcomes [18].
Our specific objectives for the review were to identify (a) which BCTs are applied in interventions to promote healthcare professionals' eHealth competency, (b) which components of healthcare professionals' eHealth competency (i.e. psychological capability, physical capability, automatic motivation, or reflective motivation) can be influenced the most by intervention, and (c) which BCTs, if any, are associated with improvement in healthcare professionals' eHealth competency.

Methods
We conducted a systematic review following the Joanna Briggs Institute's (JBI) Manual for Evidence Synthesis in systematic reviews of effectiveness [20], which includes the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement [21]. Our method focused on quantitative studies and allowed to investigate the extent to which eHealth competency can be improved by implementing BCTs.

Eligibility criteria
Appendix A outlines the inclusion and exclusion criteria for the study selection. We defined eligibility criteria according to the PICOT framework [22]: 1) Participants were healthcare professionals licensed to be credential providing healthcare, employed in a healthcare organization; 2) Interventions aimed to promote eHealth competency, 3) Comparators were any control group, including standard practice or no intervention, or prospective or retrospective baseline measures; 4) Outcome was eHealth competency [6], including (a) eHealth knowledge and cognitive skills, (b) physical eHealth skills, including associated social and communication skills; and (c) willingness and attitudes toward eHealth. Additionally, a measurement that could imply eHealth competency, such as output quality or efficiency, was also considered; and 5) Type of studies included all original peer-reviewed studies with experimental and non-experimental designs. Although qualitative studies could have provided in-depth experiences of interventions, we decided to keep our review focused.
We limited the search to papers published between January 2010 and February 2020 due to the rapid pace of change in the field of information and communication technologies (ICT) and to update the evidence from the previous review [17]. Eligible literature required an English abstract and English, Finnish, Danish, or Swedish full text.

Search strategy
We used PubMed, Embase (Ovid), and CINAHL (EBSCO) as the primary information sources. A three-step search strategy was followed [20]. At first, a limited search was performed on PubMed and CINAHL (EBSCO) to identify the index terms and words from the title and abstract. We consulted a research librarian with expertise in healthcare to optimize the search terms and develop database-specific strategies. We The COM-B model for eHealth performance, adapted from Michie et al. [5]; content based on Konttila et al. [6].
decided to use search terms broadly because the terminology for eHealth or interventions is not yet standardized. Secondly, we implemented the final search strategy, presented in Appendix B, by searching for each included information source. We performed the searches in February 2020 in English. Thirdly, the reference lists of the included full-text publications were screened for additional studies.

Study selection
We collated identified publications from the information sources and removed duplications using EndNote X9 [23]. One reviewer (LV) screened titles and abstracts. Two reviewers (LV and AK) then independently screened the full text of the included publications from the first screening phase and reported the reasoning for exclusion. A kappa value of 0.81 in the full-text screening showed an almost perfect level of agreement [24].

Critical appraisal
Two reviewers (LV and AK) independently appraised the quality of the studies using the Quality Assessment Tool for Quantitative Studies, which had previously been validated [25]. A discussion followed to resolve any disagreements in the rating. No authors of studies were contacted for additional data. The average strength of preventing (a) the extent of bias, (b) selection bias, (c) detection and performance bias, (d) confounders, (e) threats to reliability and validity, and (f) attrition bias in the studies was computed.
Regardless of the quality level, we included each appraised study to the review to achieve a comprehensive synthesis of the latest interventions [26]. One reviewer (LV) performed a sensitivity analysis by excluding methodologically weak studies to examine the robustness of the results.

Data extraction
Our data extraction instrument included details about (a) the author (s) and publication year, (b) methodology, (c) setting, (d) participants, (e) intervention (the type of eHealth, theoretical basis, content, the facilitator(s), and duration), (f) comparator, and (g) outcomes (indicators and effects on eHealth competency).

Data synthesis
Before synthesizing data, one reviewer (LV) completed the BCTTv1 training [27] to improve the interpretation of the content of interventions against the standardized BCT definitions. The training was based on a tutorial which has shown to improve coding skills [28]. To code a BCT, the content of the intervention described in the study had to indicate the presence of the BCT either beyond all reasonable doubt or all probability.
The outcomes from each study were categorized based on their correspondence under one of the four components of eHealth competency in the COM-B model ( Fig. 1) [5]. We chose the COM-B model over technology-specific models because it allowed analyzing intervention effects on competency and combining several outcome indicators with flexible components. Outcomes indicating solely cognitive capability, such as knowledge, fell under psychological capability. Since physical capability also requires psychological capability, distinguishing them would not be appropriate. Thus, physical capability was named as physical and psychological capability, and outcomes indicating physical  [21]. skills were categorized under it. Outcomes indicating motivation, such as reactions and self-efficacy, were categorized under automatic motivation or reflective motivation, respectively.
The identified BCTs and effects on each outcome category were tabulated, and their frequencies calculated. Subsequently, the frequencies of improved effects on eHealth competency associated with each identified BCT were measured. We considered a BCT to be worth further investigation when two or more interventions that applied the technique demonstrated positive evidence. If the certainty of positive evidence was at least moderate (see 2.7.), we considered the technique as promising to promote eHealth competency. We synthesized the results narratively, using standardized statements [29] because heterogeneity between studies impeded the pooling of data in a meta-analysis.

Assessing the certainty of evidence
Following the GRADE guidelines [30], the certainty of evidence was evaluated for the intervention effects on eHealth competency, and the effects of the BCTs on eHealth competency. The assessment was based on the strength of prevention of bias in the studies and the accuracy, consistency, directness, detection, and practical benefits of the evidence. Fig. 2 illustrates the flow of the study selection. A total of 7866 potentially relevant studies were identified in the database search. After removal of 3544 duplicates, 4322 titles and abstracts were screened against eligibility criteria of which 4214 were excluded. The remaining 108 articles were retrieved for full-text examination against eligibility criteria. Of the 108 articles, 76 were further excluded, which are listed in Appendix C. The references of the eligible studies yielded no additional articles. The screening resulted in 32 eligible studies.

Study characteristics
The 32 studies reported a total of 34 interventions promoting healthcare professionals' eHealth competency. The study characteristics are described in Appendix D.

BCTs applied in interventions
Of the 93 BCTs [18], 28 were identified from the interventions. On average, each intervention applied six BCTs, ranging from three [59] to 12 [48]. Additionally, one technique not previously classified in the BCTTv1 [18] was identified and named as a participatory approach. The BCTs were primarily considered being present beyond all reasonable doubt. Fig. 3 illustrates the frequencies of the BCTs.
The most commonly included BCTs were behavioral practice and rehearsal, demonstration of behavior, instruction on how to perform behavior, and practical social support.

Summary table
What was already known on the topic?
• Implementing eHealth without ensuring a competent workforce can affect the efficiency of work, quality of care, and patient safety.
• Behavior change interventions might be useful in promoting eHealth capability and motivation.
What this study added to our knowledge?
• This is the first systematic review using taxonomy and theory to examine interventions addressing insufficient eHealth competency in healthcare professionals. • Interventions tend to focus more on improving capability than motivation although both are crucial for competent eHealth performance.
• Empathic support, encouragement, and user-centered changes in the work environment could improve eHealth competency as a whole. Restructuring the physical environment Redesigning the user interface [53] 247 (2) May improve ⨁⨁©© LOW a,b,e Repairing errors in the current user interface [48] Delegating some work tasks for support personnel [36] Information about others' approval Presenting data from other similar sites that demonstrate their eHealth adoption [62] 99 (2) May improve ⨁⨁©© LOW a,b,g Emphasizing that eHealth is widely used elsewhere in the country [51] Adding objects to the environment Purchasing new eHealth equipment [33,48] 338 ( Note. The certainty of evidence is based on the GRADE Working Group [29] definitions: high-certainty: there is confidence that the true effect lies close to that of the estimate of the effect; moderate-certainty: the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different; low-certainty: the true effect may be substantially different from the estimate of the effect; very-low-certainty: the true effect is likely to be substantially different from the estimate of effect. a A narrative synthesis was conducted where estimates are not precise. b The improved effect sizes were judged to be practically beneficial. c Serious concerns of confounders, reliability and validity of the outcome instruments, and attrition bias. d Concerns about selection bias and confounders. e Serious concerns of selection bias and confounders. f Concerns about the extent of bias. g Serious concerns about selection bias, confounders, and attrition bias. h Serious concerns about selection bias, confounders, reliability and validity of the measurement instruments, and attrition bias.

Influence of interventions on the components of eHealth competency
Appendix F presents the intervention effects on the four components of healthcare professionals' eHealth competency by the indicators used in the studies.

BCTs associated with improvement in eHealth competency
The interventions suggested an association between six BCTs identified in two or more interventions and improvement in healthcare professionals' eHealth competency as illustrated in Appendix G. Two BCTs were promising as there was moderate-certainty evidence that action planning [33,39,45,51] and participatory approach [34,39] probably improve eHealth competency. The other four BCTs were worth further investigation as there was low-certainty evidence that emotional social support [37,45,48,50,61], monitoring emotional consequences [36,53,62], restructuring the physical environment [34,36,39,50,51], and information about others' approval [51,62] may improve eHealth competency.

Discussion
This systematic review examined behavior change interventions promoting healthcare professionals' eHealth competency through taxonomy [18] and behavioral theory [5]. The BCTTv1 [18] ratings demonstrated that different techniques were applied to promote eHealth competency in the reviewed 34 interventions. However, the interventions tended to primarily focus on techniques for practicing, instructing, and demonstrating eHealth performance. Interventions were most likely to improve psychological and physical capability and less likely to enhance automatic or reflective motivation toward using eHealth. We identified two promising BCTs: action planning and participatory approach. Additional six BCTs, information about colleagues' approval,emotional social support,monitoring emotions,restructuring or adding objects to the environment, and credible source, were considered worth further investigation.
Intervention functions have not changed substantially over time, as Gagnon et al. [17] review more than a decade ago similarly found that training was most often used to promote-> eHealth adoption. The focus on the type of eHealth seems to have shifted from electronic databases in Gagnon et al. [17] review toward EHRs as in our review. The widespread implementation of EHRs in the 2010s, partly related to regulations and provisions that promoted implementation [63,64], and the challenges in the adoption such as counterintuitive design [65,66] probably explain the recent focus on EHRs. EHRs can be considered as a catalyst for expanded developments in digitalized patient care. Thus, today's interventions should bend to being multifaceted to address the competency required for different eHealth systems and devices, such as CPOE, patient portals, mobile applications, and wearables. Experiences from interventions focusing on EHRs might be useful in other eHealth areas.
Thus far, Presseau et al. [67] study seems to be the only other study that uses the BCTTv1 for techniques targeting healthcare professionals. Although they addressed professionalism in diabetes care, they identified 17 same BCTs as our review. Our collective findings convince that BCTTv1 [18] can be used to synthesize interventions for healthcare professionals. However, both our review and Presseau et al. [67] study identified less than one-third of the BCTs classified in the BCTTv1, which suggests that some techniques might be more relevant for other target groups than healthcare professionals. Without modification, the taxonomy may be too excessive and time-consuming to evaluate interventions for healthcare professionals.
The main result of our review is that an intervention probably improves healthcare professionals' eHealth knowledge and skills but may overlook the efforts to address their negative attitudes toward digitalization. Although the COM-B model [5] postulates that capability may influence motivation, only 66 % of the reviewed interventions that included behavioral practice improved both capability and motivation to use eHealth. These findings suggest that an intervention that successfully improves capability may not have sufficient effect on enhancing motivation, which is equally crucial for competent eHealth performance.
Self-Determination Theory [68] claims that often some degree of motivation explains human behavior. Motivation in the COM-B model refers to the self-determined, intrinsic motivation. A sense of duty to comply with external demands can create extrinsic motivation for behavior change, which may explain why some interventions improved capability but not motivation. However, without any intrinsic motivation, maintaining the change is unlikely [68,69]. An intervention may provide longer-term benefits if it succeeds in inspiring participants to find a passion for professional development in eHealth [69].
The techniques that we discovered as promising to comprehensively improve eHealth competency or worth further investigation emphasize the importance of the social environment, namely the empathy, encouragement, and acceptance of eHealth shown by competent colleagues [37,45,46,48,50,51,61,62]. The essential role of opinion leaders in motivating the adoption of innovations has also been acknowledged elsewhere [70][71][72].
Social support can also be informational, such as providing feedback [73], but the interventions applied surprisingly little BCTs related to feedback. We did not find an association between feedback and improved eHealth competency. However, previous research has indicated that feedback generally promotes small but potentially essential changes in healthcare professionals' behavior [74].
Techniques related to the modifications in the work environment were scarcely represented in the reviewed interventions. It is recognized that workplace interventions, particularly in the field of health promotion [75], tend to "blame the victim," where employees' behavior is seen as an object of change rather than the environment. We showed, however, that the techniques changing the work environment could promote eHealth competency, particularly by repairing or improving the systems [53], adding supportive elements [31,34,39,48,51,55], and enabling healthcare professionals to participate in the redesign process [34,39].
Our findings suggest that training can be inefficient if incompetency is related to an inadequate digital work environment. The usability researcher, Nielsen [76] has explicitly stated that if the users were not able to use the system, the problem would be in an improperly designed system instead of in the users themselves. Nielsen [77] has, thus, stressed that the user interfaces must be both suitable for the user environment and pleasant to use, which better motivates to use the system. In addition to affecting efficiency and psychological wellbeing of healthcare professionals [78][79][80], the usability of the eHealth systems is vital for safe patient care [81]. If the software or device was sufficiently intuitive, and therefore, experienced easy and meaningful to use, fewer interventions would be needed [82].
Despite the prominence of the single techniques identified, in practice, there may be a synergistic influence of the BCTs combined. The intervention facilitator, dose received, and participant characteristics may affect the results and explain some of the observed inconsistency in the effects. We also discovered that the latest interventions targeted physicians twice as often as nurses, which was a similar finding as a decade ago [17]. It is evident that also other professional groups than physicians are using eHealth in their daily work, and thus, need eHealth competency. An increasing trend for a skill-mixed work community in healthcare organizations [83] implies that even more work tasks can be shared between, for example, physicians and nurses. Therefore, interventions should be targeted at all personnel to ensure the sustainability of the health workforce. Interventions should yet address the potential differences in the challenges created by eHealth between professional groups.

Limitations
This review has limitations. The reliability of the identified BCTs is dependent on the reported details of interventions within the studies, which is a common limitation in reviews that code BCTs [84,85]. In our review, a single reviewer extracted the BCTs and effects, which may predispose the results to subjective interpretation. However, comprehensive training for the BCT coding and adherence with the pre-defined methodology mitigated selective reporting. We may have undermined the internal validity and omitted some relevant records with our decision to use only databases with papers from biomedical, nursing, and allied health disciplines and not to include grey literature to ensure the scientific nature of studies [86]. Lastly, the outcomes may be context-dependent, hampering the external validity, as most studies were from North America, focused on EHRs competency, and targeted physicians.

Implications for practice
Although an intervention for busy providers is challenging to implement, staff would be eager to improve eHealth competency, and the more feasible intervention is compact in duration [42,[50][51][52]. In addition to eHealth training, we recommend that healthcare organizations provide empathic and patient support and encouragement for professionals to improve their motivation in digital changes. Recruited eHealth experts or competent colleagues identified from the staff could emphasize the benefits of eHealth and listen and provide support when challenges arise. We recommend that interventions would be based on behavioral theory to facilitate understanding of and responding to the insufficiencies in the components of eHealth competency. Furthermore, potential deficiencies in the user interfaces and equipment should be addressed.
Future development of eHealth technologies should aim at pursuing a user-centered design [87,88], which is based on the wishes and needs of healthcare professionals. eHealth implementation should incorporate effective techniques to ensure from the beginning that healthcare professionals have adequate capability and motivation to use eHealth effectively. The support for eHealth should, however, be ongoing, which requires that support from ICT personnel, managers, and the work community is always present. Overall, the organizational changes due to digitalization require careful planning and commitment of top management to be successful [89].

Implications for research
We recommend further research on interventions promoting both eHealth capability and motivation. As previous intervention studies have mostly focused on physicians, we propose further research on interventions for other professional groups, such as nurses, to improve understanding whether different groups would need particular support for digitalization. The rapid development of eHealth requires research on interventions promoting multifaceted competency to address the competency needed for also other eHealth solutions than EHRs.
A further review could explore qualitative studies on the experienced efficiency of interventions to understand their role in competency development. Forthcoming systematic reviews could investigate whether certain combinations of BCTs, specific intervention characteristics or participant characteristics, including individual learning styles [90] and orientation to technology, are associated with improved eHealth competency.

Conclusion
The present systematic review has identified 29 different BCTs of the recently implemented interventions to address insufficient eHealth competency in healthcare professionals. Our reviewed interventions tended to improve eHealth capability, but they overlooked the importance of motivation which is also crucial for competent eHealth performance and its maintenance. We have indicated that empathy, encouragement, and user-centered changes in the work environment could improve eHealth competency comprehensively. Evidence-based techniques, such as action planning and participatory approach, should be favored in the development of interventions. Additionally, information about colleagues' approval, emotional social support, monitoring emotions, restructuring or adding objects to the environment, and credible source are techniques worth further investigation. Intervention research can be strengthened by focusing on nurses and multifaceted competency required for the effective use of different eHealth solutions.

Funding
This review was supported by the Strategic Research Council at the Academy of Finland under Grant [number 327145]. The funding source had no involvement in the study design, data collection, analysis, interpretation of data, writing or decision whether to submit the article.

Authors' contributions
All authors have made a substantial, direct, intellectual contribution to this systematic literature review. LV designed the review under the supervision of AK, EL, KG, and TH. LV conducted database searches. LV and AK screened the records and appraised the quality. LV was responsible for data extraction, synthesis and interpretation of results, and drafting of the manuscript. AK, EL, KG, and TH critically revised the draft for important intellectual content. All authors approved the final version ->of the manuscript.

Summary table
What was already known on the topic?
• Implementing eHealth without ensuring a competent workforce can affect the efficiency of work, quality of care, and patient safety. • Behavior change interventions might be useful in promoting eHealth capability and motivation.
What this study added to our knowledge?
• This is the first systematic review using taxonomy and theory to examine interventions addressing insufficient eHealth competency in healthcare professionals. • Interventions tend to focus more on improving capability than motivation although both are crucial for competent eHealth performance. • Empathic support, encouragement, and user-centered changes in the work environment could improve eHealth competency as a whole.

Declaration of Competing Interest
The authors report no declarations of interest.

Appendix A
See    (feedback or abilit* or acceptance or adopt* or attitude* or behaviour or behavior or belief or believe* or capabilit* or comfort* or competenc* or confidence or consider* or experienc* or engag* or knowledge or learn* or motivat* or opinion or opportunit* or perception or perceive* or performanc* or satisf* or self-efficacy or teamwork or recall or recogni* or resist* or skill* or uptake or burnout or stress or willingness (case study or mixed method* or mixed-method* or cohort or implementation study or evaluation study or evaluat* or randomis* or randomiz* or randomly or trial or multicenter or multi center or multicentre or multi centre or controlled or control group* or groups or RCT or CCT or ((pretest or pre test) and (posttest or post test)) or quasi experiment* or quasiexperiment* or time series or repeated measure*).mp. or (effectiveness or efficacy or evidence-based  2 MH ("staff development" OR "education, continuing+" OR "education, competency-based" OR "refresher courses" OR teaching OR "practice guidelines" OR motivation OR "motivational interviewing" OR "program development" OR "personnel management") 197,876 3 interven* OR TI (implement* OR improve* OR facilitat* OR encourag* OR "behaviour change" OR "behavior change" OR change* OR changing OR "organizational change*" OR "organisational change*" OR policy OR policies OR practice* OR procedure* OR "professional development" OR program* OR strateg* OR technique* OR tool* OR "meaningful use" OR "quality improvement") (S13 OR S14) 147,045 16 S12 AND S15 59,561 17 MH ("attitude of health personnel" OR "clinical competence" OR "computer literacy" OR "professional knowledge" OR "attitude to computers" OR "job satisfaction" OR "professional competence") 117,478 (continued on next page)

Appendix C
The list of all excluded studies after full-text examination (n = 76), categorized by the reasons for exclusion.   (S17 OR S18) 2,371,820 20 PT ("adaptive clinical trial" OR "Clinical Study" OR "clinical trial" OR "Controlled clinical trial" OR "Guideline" OR "multicenter study" OR "Pragmatic clinical trial" OR "Randomized controlled trial") 170,202 21 MH ("practice guidelines" OR "randomized controlled trials" OR "interrupted time series analysis") 161,964 22 ("case study" OR "mixed method*" OR mixed-method* OR cohort OR "implementation study" OR "evaluation study" OR evaluat* OR randomis* OR randomiz* OR randomly OR trial or multicenter or "multi center" or multicentre or "multi centre" OR controlled OR control group* OR groups OR RCT OR CCT OR ((pretest OR "pre test") AND (posttest OR "post test")) OR "quasi experiment*" OR quasiexperiment* OR "time series" OR "repeated measure*") or TI (effectiveness OR efficacy OR evidence-based) 2,009,667 23 (S20 OR S21 OR S22) 2,063,587 24 S1 AND S16 AND S19 AND S23 Table E1 summarizes the appraised methodological quality of the studies. The general methodological weakness in the studies was an inadequate methodological reporting, for example, lacking details of their study design, target population, blinding process, allocation concealment, and validation of the measurement instruments. Another frequent methodological weakness was a weak control of potential confounders distorting the study effects and the usage of nonprobabilistic sampling methods. Practical reasons made blinding in the studies challenging, although some reported successfully masking the research aim from the participants or blinding the outcome assessors. Studies also suffered from a low statistical power due to relatively small sample recruited or loss to follow-up. Subjective outcome indicators were used in the majority of the studies and were inevitable in evaluating participants' experiences, but simultaneously introduced a risk of flawed responses. Nevertheless, the prevention of extended bias was, on average, moderately strong due to the study designs utilized.

Appendix E
The sensitivity analysis showed that by excluding methodologically weak studies, three identified BCTs would be missed, namely goal setting [33], reviewing current behavior and goal [45], and information about emotional consequences [50]. Moreover, information about others' approval [51,62] would not be a suggested BCT worth further investigation as it appears in only one study of moderate quality [51]. Overall, however, according to our sensitivity analysis, it seems that the effects of the interventions are not substantially sensitive to the study quality.

(19) 1 day to 2 years
Eleven interventions demonstrated that they may improve reflective motivation toward eHealth. Three interventions observed that they may have little or no difference in reflective motivation. Five interventions showed that they may improve slightly reflective motivation with inconsistent effects: two showed that they may improve attitudes toward eHealth, whereas the other two suggested little or no difference in attitudes; one showed that it may improve perceived eHealth-related behavioral control while another suggested little or no difference in control; one indicated that it may improve confidence in using eHealth, but another suggested little or no difference in long-term confidence.
⨁⨁©© LOW a, b,e,f Note. The certainty of evidence is based on the GRADE Working Group [29] definitions: high-certainty: there is confidence that the true effect lies close to that of the estimate of the effect; moderate-certainty: there is moderately confidence in the effect estimate, i.e. the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different; low-certainty: the confidence in the effect estimate is limited, i.e. the true effect may be substantially different from the estimate of the effect; very-low-certainty: there is very little confidence in the effect estimate, i.e. the true effect is likely to be substantially different from the estimate of effect. a A narrative synthesis was conducted where estimates are not precise, which downgraded the certainty by one point. b The improved effect sizes were judged to be practically beneficial, which upgraded the certainty by one point. c Inconsistent findings between improved effects and an observed little or no difference downgraded the certainty by one point. d Evidence was inconclusive and rated as very inconsistent, which downgraded the certainty by two points. e Inconsistent findings with some sub-indicators showing improved effects in one study while the difference was not observed in another study downgraded the quality by one point. f Concerns of selection bias and confounders downgraded the certainty by one point.