The loss of reason in patient decision aid research: Do checklists damage the quality of informed choice interventions?

https://doi.org/10.1016/j.pec.2010.01.002Get rights and content

Abstract

Objective

To discuss whether using the International Patient Decision Aids Standards (IPDAS) Collaboration checklist as a gold standard to judge interventions’ quality is premature and potentially detrimental to the validity of resources designed to help patients make treatment choices.

Methods

Conceptual review integrating the science behind individuals’ decision making with the demands of designing complex, healthcare interventions.

Results

Patient decision aids are promoted as interventions to help professionals engage in shared and/or patient-centred care. The IPDAS domains were informed by experts’ opinions of best practice. Decision scientists study how individuals make decisions, what biases their choices and how best to support decisions. There is debate from decision scientists about which component parts are the active ingredients that help people make decisions.

Conclusions

Interventions to help patients make choices have different purposes, component parts and outcomes to those facilitating professional–patient communications. The IPDAS checklist will change to respond to new evidence from the decision sciences.

Practice implications

Adhering uncritically to the IPDAS checklist may reduce service variation but is not sufficient to ensure interventions enable good patient decision making. Developers must be encouraged to reason about the IPDAS checklist to identify those component parts that do (not) meet their intervention's purpose.

Introduction

In 1999, two systematic reviews evaluated over 20,000 articles for interventions to support patients’ decision making about treatment or screening choices [1], [2]. Few authors referred to their interventions as decision aids but rather as education resources. Patient decision aids are complex interventions [3] that include several components to help patients make deliberative choices between two or more screening or treatment options; education resources are interventions that provide information to support a service and/or adherence with a recommended option [2], [4]. Today, there are hundreds of resources claiming to be patient decision aids with health service providers, not for profit and commercial organisations investing in a plethora of materials to support patients’ health, illness and treatment choices [5]. However, the utilisation of patient decision aids outside the research context is low [6], [7], the quality and content of interventions claiming to enable patients’ decision making varies enormously [5], [8], [9], [10], and the analysis of leaflets provided by services suggests most developers are unaware of the evidence and guidance on designing interventions to support patients’ choices [11], [12], [13].

Implementing these complex interventions within existing health services requires an active approach, targeting aspects of the systems that organise services, adoption behaviours of health professionals, and expectations of patients [3], [14], [15]. A key implementation strategy of patient decision aid advocates has been to develop, and support the use of, criteria to guide the content, evaluation and implementation of decision aid interventions [2], [4], [16], [17], [18], [19], [20]. The International Patient Decision Aids Standards (IPDAS) Collaboration is an international, cross-disciplinary group who worked together to develop these criteria [19]. A DELPHI technique was used to develop the IPDAS criteria, drawing on the expertise of patient advocates, clinicians, policy makers and researchers working across a range of areas including: patient-centred health policies, communication training for health professionals and service quality, decision sciences applied to health, and patient decision making [5]. The IPDAS collaboration recommend developers consider a range of domains when designing their decision aids, broadly issues around information content and structure (providing accurate information about options, balancing information, presenting probabilities, using plain language, use of different media), and patient assimilation of information (clarifying values, coaching deliberation) [19].

The IPDAS checklist was designed to be used in the same ways as checklists developed in other areas of evidence-based practice [5]. For example CONSORT [21] and PRISMA [22] which are used to guide investigators’ selection of the methods used in their studies, the format for their write-up, and others’ judgement of the rigour of the study. The IPDAS checklist is promoted as a resource to guide those developing patient decision aids on the intervention's content and structure, to provide a process for evaluating the intervention, and a format for disseminating findings [5]. One adaptation of the checklist (CREDIBLE) is used to elicit data from studies evaluating patient decision aid development and effectiveness for synthesis within a systematic review [10], [18]. More recently, IPDASi is being used by others to assess the degree to which developers operationalised and/or adhered to the IPDAS checklist with a view to judging the quality of the intervention and its evaluation [5], [23], [24]. Reviews suggest more patient decision aid developers are using the IPDAS criteria to inform their patient decision aid development and evaluation [10], [23].

Using the IPDAS checklist should mean that interventions designed to help patients make treatment decisions have a comparable level of quality, so reducing variations in health service delivery [20]. However, to use a checklist effectively assumes investigators know what information is pertinent to [22], and have sufficient knowledge to appraise critically [21], the intervention, its evaluation and implementation. There are several reasons why applying the IPDAS checklist critically may be challenging. First, the IPDAS checklist is informed by several theoretical and health service policy areas [19]. As a result there are many criteria to fulfil, and it is implied all need to be met and/or have equal weight in decision facilitation. Second, many of the statements require significant expertise from the decision sciences to operationalise. For example, ‘provide steps to make a decision’, ‘use visual diagrams for present probabilities of outcomes’, present information in a balanced manner enabling people to compare positive and negative features’. Third, the IPDAS checklist was decided upon by expert opinion [15] as much of the evidence-base to underpin each IPDAS domain was weak [19]. Unlike other evidence-based checklists, there is no distillation of evidence to suggest which criteria or technique results in more effective patient decision aids.

It is increasingly common to see the IPDAS checklist referred to as a model or theory [23]. This change in reference from a set of criteria to consider to a checklist to operationalise suggests developers may not have the relevant knowledge to reason critically about its application. Unlike the explanation and elaboration articles supporting CONSORT [21], little has been written to help developers understand the basic science underpinning patient decision aid interventions [19], [25], [26], [27], [28], [29]. The focus of IPDAS dissemination activities has been to increase awareness of patient decision aids’ efficacy to enhance the shared decision making or patient-centred practices of professionals and provide detailed steps on how do it [4], [5], [6], [9], [10], [19], [30], [31], [32], [33], [34], [35]. This article aims to provide a knowledge brokering of concepts [37] to help reduce the knowledge-action gap between health professionals and behavioural scientists, and enable patient decision aid developers to reason critically about their operationalisation of the IPDAS checklist. It discusses concepts from both the social sciences and health services delivery of relevance to patient decision aid interventions and evaluation with references to the evidence and theories that underpin this synthesis.

Section snippets

Translating decision sciences knowledge for patient decision aid applications

Designing patient decision aid interventions is challenging because the expertise required to develop, evaluate and implement them is drawn from several disciplines, for example (a) the basic decision science investigators explaining how individuals make decisions under risk and uncertainty, (b) the decision science investigators that apply the proof of concept studies to health and illness decisions (translational researchers), (c) the biomedical investigators identifying effective treatments,

Discussion

Using the IPDAS checklist to develop and evaluate patient decision aids will reduce variation in the quality of patient information across services. It is also likely that interventions informed by the checklist will be more useful to patients than information provided routinely by services. Developing the IPDAS criteria has been a key activity to bring together experts interested in developing theory-based interventions for use in the applied health setting, assimilating current knowledge and

Conflict of interest

The author knows of no financial or other relationship with people or organizations that have influenced her work inappropriately. For the last 3 years, the author has been employed as an academic at the University of Leeds and received awards and finances congruent with scholarship rather than collusion: fellowships to attend the Dartmouth Summer Institute for Informed Patient Choice; unrestricted grants to support doctoral student and decision aid research; expenses covering the costs of

Acknowledgements

I am grateful to Professor Anne Stiggelbout, University of Leiden, Dr. Dan Matlock, University of Colorado at Denver, and the two referees for providing useful guidance for this very much revised version of the article. I am indebted to those who have debated with me about issues in decision sciences, health psychology, patient–professional practices and research methods for the last 15 years which has informed my understanding of, and research into, this complex area.

References (111)

  • A.M. O’Connor

    Effects of framing and level of probability on patients’ preferences for cancer chemotherapy

    J Clin Epidemiol

    (1989)
  • L. Fraenkel et al.

    What do patients consider when making decision about treatment for hepatitis C?

    Am J Med

    (2005)
  • S. Sher et al.

    Information leakage from logically equivalent frames

    Cognition

    (2006)
  • R.M. Epstein et al.

    Measuring patient-centred communication in patient-physician consultations: theoretical and practical issues

    Soc Sci Med

    (2005)
  • G. Makoul et al.

    An integrative model of shared decision making in medical encounters

    Patent Educ Couns

    (2006)
  • H.L. Bekker et al.

    Is anxiety an appropriate measure of decision aid effectiveness: a systematic review?

    Patent Educ Couns

    (2003)
  • P.F.M. Stalmeier et al.

    Concise evaluation of decision aids

    Patent Educ Couns

    (2009)
  • H. Bekker et al.

    Informed decision making: an annotated bibliography and systematic review

    (1999)
  • A.M. O’Connor et al.

    Decision aids for patients facing health treatment or screening decisions: a systematic review

    Brit Med J

    (1999)
  • Medical Research Council

    Developing and evaluating complex interventions: new guidance

    (2008)
  • A.M. O’Connor et al.

    The ottawa patient decision aids

    Eff Clin Pract

    (1999)
  • G. Elwyn et al.

    Developing a quality criteria framework for patient decision aids: online international Delphi consensus process

    Brit Med J

    (2006)
  • M. Holmes-Rovner et al.

    Are patient decision aids the best way to improve clinical decision making? Report of the IPDAS symposium

    Med Decis Making

    (2007)
  • A.M. O’Connor et al.

    Decisions aids for people facing health treatment or screening decisions (review). The Cochrane Collaboration

    (2009)
  • A. Coulter et al.

    Sharing decisions with patients: is the information good enough?

    Br Med J

    (1999)
  • A.E. Winterbottom et al.

    Evaluating the quality of patient information provided by Renal Units across the UK

    Nephrol Dial Transplant

    (2007)
  • NHS Centre for Reviews and Dissemination
    (1999)
  • NICE

    How to change practice: understand, identify and overcome barriers to change

    (2007)
  • S. O’Donnell et al.

    Understanding and overcoming the barriers of implementing patient decision aids in clinical practice

    J Eval Clin Pract

    (2006)
  • A.M. O’Connor et al.

    Ottawa personal decision guide

    (2006)
  • Ottawa Health Research Institute

    Toolkit resources

    (2009)
  • A.M. O’Connor et al.

    IPDAS collaboration background document

    (2005)
  • O’Connor et al.

    Toward the ‘tipping point’: decision aids and informed patient choice

    Health Aff

    (2007)
  • D.G. Altman et al.

    The revised CONSORT statement for reporting randomised trials: explanation and elaboration

    Ann Intern Med

    (2001)
  • M.A. Durand et al.

    A review of decision support technologies for amniocentesis

    Hum Reprod Update

    (2008)
  • H.A. Llewellyn-Thomas

    Patients’ health-care decision making: a framework for descriptive and experimental investigations

    Med Decis Making

    (1995)
  • H.L. Bekker et al.

    Patient information and decision making process (editorial)

    Br J Renal Med

    (2009)
  • H.L. Bekker

    Using decision making theory to inform clinical practice

  • H.A. Llewellyn-Thomas

    The decision lab

  • C. Charles et al.

    Treatment decision aids: conceptual issues and future directions

    Health Expect

    (2005)
  • L.A. Siminoff et al.

    A communication model of shared decision making: accounting for cancer treatment decisions

    Health Psychol

    (2005)
  • N. Moumjid et al.

    Shared decision making in the medical encounter: are we all talking about the same thing?

    Med Decis Making

    (2007)
  • V.A. Entwistle

    Patient involvement in decision-making: the importance of a broad conceptualisation

  • J. Lomas

    The in-between world of knowledge brokering

    Br Med J

    (2007)
  • B. Fischhoff

    Decision research strategies

    Health Psychol

    (2005)
  • L. Bryant et al.

    The lure of patient choice (comment)

    Br J Gen Pract

    (2007)
  • I. Greener

    Towards a history of choice in UK health policy

    Sociol Health Ill

    (2009)
  • D. Feldman-Stewart et al.

    A systematic review of information in decision aids

    Health Expect

    (2006)
  • R.M. Hogarth

    Judgment and choice

    (1988)
  • H.A. Simon

    Rationality as process and as product of thought

  • Cited by (67)

    • Preparation for Medical Interventions

      2022, Comprehensive Clinical Psychology, Second Edition
    • Communicative aspects of decision aids for localized prostate cancer treatment – A systematic review

      2019, Urologic Oncology: Seminars and Original Investigations
      Citation Excerpt :

      When reviewing the quality of DAs, researchers often make use of a standardized quality checklist developed by the International Patient Decision Aids Standards (IPDAS) Collaboration [6,23]. Nevertheless, even though the IPDAS checklist is seen as the golden standard for developing and evaluating DAs [24], it is also important to consider other aspects of the communication process that are not covered by the IPDAS. Until now, only one systematic review by Adsul et al. has reviewed the quality of DAs for LPC treatment by using additional items related to implementation (e.g., health literacy) [25].

    View all citing articles on Scopus
    View full text