Keywords
Multimorbidity, Polypharmacy, Adherence, General Practice, PPI, Collective Intelligence Design, E-learning tool
This article is included in the HRB Primary Care CTNI gateway.
This article is included in the Public and Patient Involvement collection.
Multimorbidity, Polypharmacy, Adherence, General Practice, PPI, Collective Intelligence Design, E-learning tool
Specific changes made between version 1 and version 2:
Following peer review, the authors have updated this manuscript to incorporate reviewer feedback.
Specifically:
See the authors' detailed response to the review by Ngaire Kerse
See the authors' detailed response to the review by Peter Bower
Healthcare utilisation and cost in both primary and secondary care is significantly increased with multimorbidity (i.e., co-occurrence of two or more chronic conditions). Each additional chronic condition leads to near-exponential increases in both service usage and financial costs (McPhail, 2016). This effect on primary care consultations, hospital outpatient visits and admissions and total healthcare costs occurs independently of age, gender and socioeconomic status (Glynn et al., 2011). The cost of living with multimorbidity is also high for the patient. Living with multiple conditions has debilitating physical, psychological, social, and financial consequences and is associated with impaired functional capacity, reduced quality of life and increased rates of psychological distress (Fortin et al., 2004; Ryan et al., 2015). The treatment burden individuals with multimorbidity experience is also significantly high as each condition can require different treatments, lifestyle adjustments, specialist care, and medications.
In order to increase the quality of care being delivered to patients and reduce spiralling healthcare costs, there is a need to support people to self-manage their chronic health conditions (HIQA, 2015). Supporting people to self-manage their health through education and supportive interventions can increase skills and confidence, enhance self-efficacy, and improve day-to-day quality of life. It can also lead to improvements in clinical outcomes, reduce healthcare utilisation, and decrease hospitalisation (HSE, 2015). For the majority of patients, self-management shows significant benefits when there is increased support from healthcare professionals (Grady & Gough, 2014). Self-management support may include such components as case management, frequent follow-ups and patient education (HSE, 2017). Examples of self-management education may include providing information about the patient’s condition(s), advice on exercise and nutrition, and guidance on appropriate use of medications.
Medication adherence can be a particularly challenging aspect of living with multimorbidity due to high treatment burden. The term ‘adherence’, as opposed to ‘compliance’, is critical, as it highlights the active role of the patient in their treatment (Vrijens et al., 2012). If patients do not take medications as prescribed, then they are unlikely to receive the full benefits of treatments that we know work. Poor adherence can also lead to unnecessary suffering and wasted resources (Cutler et al., 2018; van Boven et al., 2014; Walsh et al., 2019); despite this, patients are often reluctant to tell their doctor or nurse that they do not take their medicines as prescribed. If a patient's medication taking behaviour is not understood, therapy may then be needlessly escalated (Brown & Sinsky, 2018). Escalating therapy when non-adherence is not identified can cause potential harm to the patient, create unnecessary work for the practice, and result in increased costs to the patient and healthcare system. However, the reasons behind medication non-adherence are complex, and can go beyond a lack of information, forgetfulness, or even access to medication itself (Kardas et al., 2013). Understanding the complexity of factors associated with non-adherence is therefore key to addressing the issue, and any tool designed to support patient adherence should be designed with the voice of the patient in mind. Thus, informed by the principles of patient and public involvement, the aim of the current project was to create an e-learning resource that would use evidence-based approaches to help healthcare professionals to support long-term medication taking in multimorbidity. As general practitioners deliver continuous care to patients and are responsible for much of their medication prescribing, the resource that was created, aminuteforadherence.ie, was designed to be an interactive and easily accessible resource that could be integrated in to general practice and clinical education programmes as a supportive tool. Such a training resource was lacking as summaries of treatment adherence have generally been more research than clinical management orientated (Sinclair et al., 2016). The term ‘collective intelligence’ (CI) is used to describe intelligence, or knowledge, that emerges as a result of a group of people working together to come up with solutions to a problem. The CI approach carefully delineates content and process roles, assigning to experts the responsibility for contributing ideas, and to the workshop facilitator responsibility for choosing and implementing selected methodologies for generating, clarifying, structuring, interpreting, and amending ideas. Emphasis is given to balancing behavioural and technical demands of group work (Broome & Chen, 1992) while honouring design laws concerning variety, parsimony, and saliency. There must be enough variety in options to cover all needs, but possible paths and solutions must exist in harmony to avoid creating confusion or disrupting problem resolution (Ashby, 1958; Boulding, 1966; Miller, 1956). Using CI can help to support high quality interdisciplinary work as it includes a set of methods and tools and a facilitated thought and action mapping process that helps groups to develop outcomes that integrate contributions from individuals with diverse views, backgrounds, and perspectives (Hogan et al., 2014; Warfield & Cardenas, 1994). CI has been applied in many different situations to accomplish many different goals, including mediating peacebuilding in protracted conflicts (Broome, 2006), developing a national well-being measurement framework (Hogan et al., 2015), mobilising communities across Europe in response to marine sustainability challenges (Domegan et al., 2016), and understanding and overcoming barriers to the design of personalised nutrition products and services for older adults (Hogan et al., 2017).
The current research project brought together general practitioners, practice nurses, pharmacists, medical educators, psychologists, learning technologists, as well as members of the public living with multimorbidity and receiving polypharmacy. One of the primary advantages of using CI was the facilitation of communication between potential e-learning tool users in relation to usage possibilities and the challenges that may arise for different stakeholders.
A total of 16 stakeholders with lived experience of multimorbidity and taking polypharmacy, and/or general practice, nursing, psychology, education, and pharmacy backgrounds participated in the collective intelligence design work. The workshop took place on April 6 2018 in the School of Psychology in NUI Galway. A purposive sampling strategy was utilised to ensure that a diversity of healthcare professionals, researchers and patient groups (e.g., young and older adults) were represented in the sample. Guidance on patient involvement was sought from PPI specialists within NUI Galway, who facilitated recruitment of people living with multimorbidity. Healthcare professionals and researchers were recruited via existing professional networks. A summary of participant backgrounds can be found in Table 1.
Collective intelligence methodology was used for this research as, while it is more time consuming than other qualitative methods in the design phase, it provides a structured and systematic way of solving a complex problem and utilises a carefully selected set of methodologies that can be matched to the phase of group interaction and the requirements of the situation. For this project, the CI methodology that was used included idea-generation and idea categorisation, combined with scenario-based design. The first stage of the CI process involved systematic analysis of barriers to supporting people with multimorbidity, which helped guard against the use of rigid thinking patterns in relation to the e-learning system and the tendency toward solution-first problem solving and design thinking (Rosson & Carroll, 2002). Stakeholders with expertise related to the problem context were contacted in advance of the session by email, with a request to generate a set of barriers in response to the following trigger question: “What barriers do healthcare practitioners face in supporting people with multimorbidity who are taking multiple medications?". By email submission, experts identified 67 barriers. These responses were subsequently reviewed by the workshop facilitation team and categorised using the Paired Comparison Method (Warfield & Cardenas, 1994). The paired comparison method provides a simple way of summarizing a group of individuals opinions, attitudes or beliefs about a topic in a systematic and objective manner. Responses to a question are clustered into categories, based on similarity, so that the group can see an overview of the issues within each category and work out the most important problems to solve. In this case, eight distinct categories were identified, providing a focus for initial discussions at the CI workshop. Each category highlighted distinct issues or barriers to medication adherence: Training and Education; Conflict; Communication; Ownership and Responsibility; Time Pressure; Resources and Support; Patient Behaviour and Abilities; and Perspective. A sampling of categorisation of barriers can be seen in Figure 1 and a complete list of barriers is presented in Tables 2–Table 9.
On the day of the workshop, the room was set up in such a way that the ‘problem field’ (i.e., the statements received and the categories into which they had been organised) were displayed around the room on poster boards. Participants were divided into four groups by the CI facilitators prior to the workshop, to make sure that a mix of stakeholder expertise was represented at each table. This ensured that a variety of perspectives would be attained during each of the exercises that would be given throughout the day. A short presentation providing an overview of the current position on multimorbidity and treatment adherence was delivered to provide context for the activities that would be completed throughout the day. Participants were advised that their feedback, ideas, and suggestions would be used to inform the design of an e-learning tool to support health practitioners caring for this group of patients. All participants gave permission for the facilitator to record the presentations that were going to take place throughout the day and for their written work to be collected after each exercise. The anonymised audio recordings are stored in a password-protected file on the lead researcher’s password-protected computer and the anonymised worksheets are stored in a locked drawer in a locked office in the University. In line with University policy, the data will be stored in this secure location, with access limited to the researchers on the project, for 10 years. Workshop participants then engaged in an analysis of the eight categories of barriers, with a view to generating options for overcoming barriers faced by healthcare practitioners in supporting people with multimorbidity who are prescribed multiple medications. In order to generate options in response to barriers, small working groups of participants (4–6 persons each) engaged in idea generation in response to assigned barrier categories. During this phase of the workshop, the ideawriting technique was used (Wood & Roth, 1990). Each of the small groups was asked to generate options in writing and to use open dialogue to explore the meaning of ideas generated. Five steps were involved in idea writing: (a) a stimulus question was presented to participants; (b) each participant worked alone to silently generate ideas in writing; (c) written sheets of ideas were exchanged among all group members and individuals had the opportunity to add ideas as they read others’ papers; (d) unique ideas were discussed and clarified; and (e) each working group orally reported the ideas generated in a plenary session. This phase of work focused on the generation of options in response to barrier categories and allowed stakeholders to scope out a broad range of options in response to barriers before focusing attention on specific scenarios of usage for the proposed e-learning support tool.
The next phase involved using scenario-based design methods to generate specific user needs (Rosson & Carroll, 2002). In advance of the CI session, the research team and facilitation team worked together to design a set of scenarios that could be used as inspiration for group idea generation during the workshop (Table 10). Following guidelines provided by Rosson & Carroll (2002), design representations captured in the scenarios were concrete, flexible, generative and did not specify fixed solutions. As such, scenarios were designed to elicit constructive cognitive processes and the development of creative and bespoke solutions that are suitable for the specific context of usage in the scenario. In this way, a collaborative analysis and elaboration of needs is encouraged in the exchange between the design team and stakeholders. The scenarios that were used depicted a series of challenges faced by multiple actors (general practitioners, practice nurses, patients, carers, and pharmacists) in order to help workshop participants to orient themselves in each individual ‘actors’ problem space, so that they could imagine such a situation arising and consider what supports would be useful in each of these circumstances. Specifically, the aim was to prompt user needs in relation to: (1) Information and Knowledge; (2) Communication Needs; (3) Decision-Making Support; (4) Behavioural Support Needs; (5) Relational Needs; and (6) Other Needs. Participants were asked to consider the roles of the different actors in each scenario and to generate a list of needs for each actor and the reasons for these needs. These needs were subsequently discussed by sub-groups and all ideas were collated by the workshop facilitation team. The identification of user needs generated through this scenario-based design phase, informed by the earlier CI analysis of barriers, and generation of targeted options, provided a strong basis for further design work for the final resource that was created.
A descriptive and exploratory approach to textual data analysis was employed, with qualitative content analysis informing the adopted analytical strategy (Elo & Kyngäs, 2008; Elo et al., 2014). An inductive approach was taken (Hsieh & Shannon, 2005) and; given that the overall aim of the analysis was to produce an integrated picture of diverse stakeholders` perspectives on barriers, solutions to those barriers, and the user needs, a low level of data transformation was attempted and the main focus was on the analysis of manifest content and a broad surface structure (Bengtsson, 2016; Sandelowski & Barroso, 2003). Computer assisted software was not used for this process. Qualitative analysis for the purposes of identifying categories of barriers was conducted manually by the team. The phenomena description was achieved by following relevant guidelines and the analytical process included stages of preparation, organisation, and report writing (Elo & Kyngäs, 2008). Specifically, after immersing in the data and obtaining the sense of whole, the researchers engaged in the process of open coding and category creation using the Paired Comparison Method (Warfield & Cardenas, 1994). Pairs of barriers were systematically assessed for conceptual similarity in turn, during an exhaustive, immersive process of comparative analysis. These conceptually similar barriers were then grouped under higher order categories. A description resulted from abstraction as well as the generation of categories and subcategories. The reporting of the analytical process has been supported by figures that represent conceptual models and illustrate the results.
As the work described in this paper relates to a PPI expert advisory group, the project team deemed that those involved those involved should not be classified as ‘research participants’, this was more akin to an expert consensus guideline group. Such an approach aligns with the equal partnership spirit of this PPI work. Therefore the usual processes of seeking ethical approval and informed consent that pertain to health research did not apply in this work. This approach is consistent with guidance provided by the National Institute of Health Research in the UK, which classify this kind of work as an ‘involvement activity’ and consistent with consensus statement papers that have been published by the National Research Ethics Service (NRES) and INVOLVE.
In total, 67 barriers to supporting people with multimorbidity who are taking multiple medications were identified across eight barrier categories: Training and Education; Conflict; Communication; Ownership and Responsibility; Time Pressure; Resources and Support; Patient Behaviour and Abilities; and Perspective. During the CI workshop, participants generated 162 options for overcoming barriers across the eight categories (see Tables 2–9). In the final stage of the workshop, the group focused on scenario-based specification of user needs to inform the design of an e-learning tool for health practitioners. A broad range of information and knowledge needs, communication needs, decision-making support needs, behavioural support needs, relational needs, and ‘other’ needs were identified (see Tables 10 and 11 (extended data (Hanlon et al., 2020)). A summary of some key needs is provided below.
The Information and Knowledge Needs category contains the largest set of generated needs and refers to information and knowledge which would be beneficial in supporting practitioners who work with patients with multimorbidity taking multiple medications. It was suggested that the e-learning tool should provide access to information and guidance on how to address issues of adherence and non-adherence, methods that can be used to identify non-adherent patients, information on how to discuss side-effects, and details of available information resources and supports that would serve to support healthcare professionals in working with patients.
The category of Communication Needs highlights the need to provide prompts and guidelines for healthcare professionals in opening and structuring conversations about adherence, including sample questions which may be useful for both the healthcare professional and patient, to help them to communicate effectively.
The Decision-making Support Needs category includes needs relating to adherence assessment tools and methods, including methods for patients to weigh up costs and benefits associated with medications, and to generate a visualisation of this information, in addition to knowledge on how to optimise decision-making in relation to adherence. It was argued that decision-making methods that support the ability to represent side-effects on one hand and health benefits on the other hand would support more balanced reasoning and decision-making on the part of healthcare professionals and patients as they work together.
Within the category of Behavioural Support Needs, participants identified the need to provide a guide on evidence-based psychological or behavioural supports and interventions, and how to implement these, such as: habit-forming supports, targeting beliefs and concerns, goal monitoring, medication reminders, action planning, and social supports which may be used by healthcare professionals to help patients with their adherence. To make such supports more engaging, it was suggested that they be presented in two forms: written information, and animations.
The Relational Needs category addressed needs such as promoting empathy an understanding, providing personal support and advice, and working with patients to explore key aspects of their specific circumstances and lifestyle choices, such as coping with adherence and organising their routine accordingly. For example, in the context of a patient whose living and relational arrangements have changed, understanding, empathy, support, and advice is needed to explore how patients are managing their medication in this new environment, and whether or not their daily medication routines need to be reorganised to work with this new context.
Finally, some participants also specified ‘other’ needs such as wanting more support from GPs so that they could feel less burdened and wanting more time in consultations so that they could better explain conditions and symptoms. For the full list of categorised needs that were identified see Table 11 (extended data (Hanlon et al., 2020)).
In summary, combining CI with SBD approaches, the CI session reported here was conducted to enhance understanding of complex issues with respect to (1) Barriers to supporting people with multimorbidity who are taking multiple medications; (2) Options for overcoming these barriers; and (3) Specific needs of users of an e-learning tool designed to support health practitioners caring for patients taking multiple medications. Following the session, a detailed report of the CI that was gathered from the group was generated. Further prioritisation of user needs and story-board sequencing of key learning goals was subsequently performed by the research team, which resulted in the design of the e-learning tool, which is available at www.aminuteforadherence.ie. A summary of the process can be found in Figure 2.
Utilising CI and SBD methodology, input was gathered from 16 stakeholders (general practitioners, practice nurses, pharmacists, medical educators, psychologists, patient and public involvement support workers as well as members of the public living with multimorbidity and receiving polypharmacy) to gain insights into barriers to supporting people with multimorbidity who are taking multiple medications, options for overcoming these barriers, and user needs relevant to the design an e-learning tool to support health practitioners caring for patients taking multiple medications. An e-learning expert on the research team provided guidance on presenting this content in the format of an e-learning resource and developing suitable assessment methods. In the short e-learning training resource that was created (www.aminuteforadherence.ie), reasons for non-adherence are explored and advice on how to approach the topic with patients is presented. Furthermore, guidance on advising patients on simple strategies that have been shown to be effective for supporting non-adherence are provided. It is intended that the resource will be used by GPs and practice nurses that are working in a general practice setting. The e-learning training resource takes about an hour to fully complete. There are quizzes built in throughout the resource and when GPs and practice nurses complete the course, they are awarded 1 internal continuous professional development point/credit for this hour of activity by the Irish College of General Practitioners (ICGP). A suggested template for an audit that meets the requirements of the Irish Medical Council is also included on the resource website. The resource is currently being used as a teaching/training tool by the ICGP and on relevant undergraduate programmes at the National University of Ireland, Galway.
The barriers identified by the stakeholder group are numerous and diverse, which reflects the complexity of medication adherence for people living with multimorbidity in daily life. This diversity of barriers identified in our study is largely consistent with previous findings. For example, in their systematic review of systematic reviews, Kardas et al. (2013) identified 771 individual factors associated with treatment initiation, implementation, and persistence, reflecting socio-economic, healthcare team- and system-related, condition-related, therapy-related, and patient-related factors. Although many of these factors are relatively fixed, many more are inherently modifiable through effective clinical and behavioural intervention. Evidence suggests that physicians can be trained to better promote treatment adherence in clinical practice (Dragomir et al., 2019; Schneider et al., 2004). The use of healthcare services to support health behaviour change is vital for promoting public health among our aging population. Therefore designing, testing, and implementing rigorous and feasible training programmes aimed at improving behaviour change communication skills among physicians in the context of chronic disease management is critical to ensure effective delivery of care and treatment success.
Using a CI approach that was informed by PPI principles was highly beneficial for the purposes of this work as it ensured that the insights of key stakeholders were equally represented and that the output that was produced was relevant, useful and acceptable so that research waste could be avoided (Brett et al., 2014; Chalmers et al., 2014; Gray-Burrows et al., 2018). Using CI methodology also ensured that the perspectives and input from the patient representatives was considered from the start (their responses to the trigger question were used to inform the design of the exercises for the workshop); that individuals with multimorbidity and taking polypharmacy were involved at all stages of the process (patient representatives took part in the same activities as healthcare professionals and researchers on the day, were also asked to provide feedback on the report that was generated following the session, and were invited to review the resource that was created to make sure that it was relevant) and; that patient representatives were given equal opportunities to express their insights (the working groups on the day all included a person living with multimorbidity and receiving polypharmacy at each table and each member of the group was given opportunities in both written and verbal form to express insights and opinions). Having the perspective of people living with multimorbidity and taking polypharmacy was thus a vital component of this work and their insights on the day of the workshop were crucial for the project. Equally, having healthcare professionals (GPs, a nurse, and a pharmacist) present ensured that the proposed strategies to overcome the identified barriers were considerate of the practical and resource demands faced in clinical settings, resulting in a resource containing strategies that are both acceptable and implementable. Finally, including health researchers with experience of applying behavioural science to complex contemporary health problems like adherence for people with multimorbidity provided important insight. Having the voices of all of these key stakeholders was vital to achieve the intended outcomes of the research project (i.e., a clinically useful tool to support healthcare professionals to promote adherence for people with multimorbidity). The use of CI methodology ensured that the group interacted effectively with each other during the session, saw problems from each other’s perspectives, and considered obstacles from a systems thinking perspective when working together to generate options for addressing adherence and content for the resource. Cumulatively, this resulted in the production of a highly relevant and comprehensive tool to address the complex clinical problem of poor adherence.
The current study must be considered in light of certain limitations. Necessitating the involvement of such a diverse group of stakeholders, though essential to produce the type of data we needed to address the aims of this work, presented certain practical challenges. Reconciling the needs and preferences of healthcare professionals, researchers, and patients in terms of dates, times and settings for the workshop was a challenge and may have excluded certain people for participating. In addition, requiring participants to commit to a full day on-site may have prevented some people from participating (e.g., people with caring responsibilities, people with diverse physical needs, etc.). Providing other means of contributing to the CI process (e.g., allowing interested parties to submit responses to the trigger question without having to attend the workshop) could ameliorate some of the barriers to participation and improve the representativeness of the data obtained.
Limitations notwithstanding, and consistent with other recent applications (Broome, 2006; Domegan et al., 2016; Hogan et al., 2017), the current study highlights the value of using a scenario-based and collective intelligence approach to system design. The online resource generated from this project represents a promising tool with the potential to support primary healthcare providers to promote optimal adherence among their patients. Further research is now required to evaluate the acceptability and effectiveness of the resource as a professional training tool in routine clinical practice. How best to maximise the impact of such a resource in ever demanding professional contexts merits specific consideration. Future work utilising a similar approach to develop a resource targeting people living with multimorbidity in the community may also be warranted. This could augment the value of this resource by helping individual patients to develop medication-taking strategies that suit their unique situations, and by facilitating improved communication between patients and their healthcare providers.
Raw Data are not publicly available as the transcripts cannot be sufficiently de-identified by redaction. Data will be made available by reasonable request to the corresponding author. A request is considered reasonable where the intended use for the data is clearly outlined, and where this intended use does not violate the protection of participants, or present any other valid ethical, privacy, or security concerns.
Open Science Framework: Designing an e-learning tool to support health practitioners caring for patients taking multiple medications. https://doi.org/10.17605/OSF.IO/QGYU2 (Hanlon et al., 2020)
This project contains the following extended data:
COREQ checklist for ‘Designing an e-learning tool to support health practitioners caring for patients taking multiple medications.’ https://doi.org/10.17605/OSF.IO/QGYU2 (Hanlon et al., 2020)
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).
Casey Donaghey, Public Advisory Panel, PPI Ignite @ NUI Galway, NUI Galway. Casey provided insight on barriers to medication adherence in advance of the workshop, contributed to all exercises completed during the workshop, reviewed the workshop report, and participated in a video that was filmed for the e-learning resource.
Denis Mockler, PPI Contributor, PPI Group @ Primary Care Clinical Trials Network Ireland (CTNI), NUI Galway. Denis provided insight on barriers to medication adherence in advance of the workshop, contributed to all exercises completed during the workshop, and reviewed/provided feedback on the workshop report and e-learning resource.
Edel Murphy, PPI Ignite @ NUI Galway Programme Manager, NUI Galway. Edel facilitated recruitment of people with multimorbidity for the workshop and provided input and guidance on PPI throughout the KEDS project.
Fiona Conconnan, Learning Technologist in the Centre for Excellence in Learning and Teaching (CELT), NUI Galway. Fiona provided integral guidance and expertise on designing the e-learning resource and developing suitable methods for educational assessment, and had a vital role in creating the website content.
Martha Killilea, Research Assistant, PPI Ignite @ NUI Galway and HRB Primary Care CTNI, NUI Galway. Martha provided support with PPI during and after the workshop.
Stacey Grealis, EULAR Research Partner, Arthritis Ireland member and IPPOSI Education programme on Health Innovation. Stacey provided insight on barriers to medication adherence in advance of the workshop, contributed to all exercises completed during the workshop, and reviewed/provided feedback on the workshop report and e-learning resource.
Trish O’ Donavan, PPI Contributor, PPI Group @ Primary Care Clinical Trials Network Ireland (CTNI), NUI Galway. Trish provided insight on barriers to medication adherence in advance of the workshop, contributed to all exercises completed during the workshop, and provided feedback on the workshop report.
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health Services Research.
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Primary care, ageing, multimorbidity.
Is the work clearly and accurately presented and does it cite the current literature?
Partly
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
Are all the source data underlying the results available to ensure full reproducibility?
No
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Primary care, ageing, multimorbidity
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Partly
If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health Services Research.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 2 (revision) 06 Apr 21 |
read | read |
Version 1 01 Sep 20 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with HRB Open Research
Already registered? Sign in
Submission to HRB Open Research is open to all HRB grantholders or people working on a HRB-funded/co-funded grant on or since 1 January 2017. Sign up for information about developments, publishing and publications from HRB Open Research.
We'll keep you updated on any major new updates to HRB Open Research
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)