Edinburgh Research Explorer Computerised decision support systems for healthcare professionals

Purpose Computerised decision support systems are designed to support clinicians in making decisions and thereby enhance the quality and safety of care. We aimed to undertake an interpretative review of the empirical evidence on computerised decision support systems, their contexts of use, and summarise evidence on the eﬀectiveness of these tools and insights into how these can be successfully implemented and adopted. Methods We systematically searched the empirical literature to identify systematic literature reviews on computerised decision support applications and their impact on the quality and safety of healthcare delivery over a 13-year period (1997–2010). The databases searched included: MEDLINE, EMBASE, The Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Eﬀects, The Cochrane Central Register of Controlled Trials, The Cochrane Methodology Register, The Health Technology Assessment Database, and The National Health Service (NHS) Economic Evaluation Data-base. To be eligible for inclusion, systematic reviews needed to address computerised decision support systems, and at least one of the following: impact on safety; quality; or organisational, implementation or adoption considerations. Results Our searches yielded 121 systematic reviews relating to eHealth, of which we identiﬁed 41 as investigating computerised decision support systems. These indicated that, whilst there was a lack of investigating potential risks, such tools can result in improvements in practitioner performance in the promotion of preventive care and guideline adherence, particularly if speciﬁc information is available in real time and systems are eﬀectively integrated into clinical workﬂows. However, the evidence regarding impact on patient outcomes was less clear-cut with reviews ﬁnding either no, inconsistent or modest beneﬁts. Conclusions Whilst the potential of clinical decision support systems in improving, in particular, practitioner performance is considerable, such technology may also introduce new risks resulting not only from technical challenges (such as data inac-curacies) but also from disruption of clinical work-ﬂows. Moving forward, there is a need for system development, procurement and implementation to be characterised by a user ‘pull’ and then tailor systems to the needs of users.


Introduction
The volume of evidence that a clinician needs to be aware of and draw on in making day-to-day clinical decisions -whether for the ordering and interpretation of investigations, making a diagnosis, prescribing or prognostication -is now so large that it is very difficult for clinicians to remain up-to-date. There is thus considerable potential for unacceptable variations in standards of care, and also for unsafe care. Computerised decision support systems aim to support clinicians in such tasks and through so doing enhance both the quality and safety of care. They are also increasingly being seen as a means to address some of the inefficiencies of care through rationalising expenditure on diagnostic tests, treatment decisions and specialist referrals. This article aims to provide an interpretative review of the empirical evidence on computerised decision support systems, summarise available evidence on how these tools can be successfully integrated into work processes and provide pointers to major future developments in this rapidly evolving field. 1 What are computerised decision support systems?
The use of information technology (IT) to support everyday tasks has become a key characteristic of life in the 21st century. The increasing use of and reliance on satellite navigation systems by car drivers is an example of a computerised decision support system in everyday use. These computerised decision support tools also have considerable potential for use in clinical care settings. These are in essence software applications that use patient data, a database of clinical knowledge and 'conditional' logic (e.g. 'if-then' and 'do while') to generate patient-specific recommendations related to care ( Figure 1). [1][2][3] Computerised decision support systems have been defined as '... computer programs that are intended to help healthcare workers in making decisions'. 4 Although they can also aid patient self-care, 2 in this article we focus on investigating the role of computerised decision support systems in supporting the management of patient care by healthcare professionals.
Computerised decision support systems can take several forms, depending on: the task they are designed to support; the approach to utilising patient data (which may involve direct input of data by professionals or, now more commonly, involving automated interrogation of existing electronic health records); the type of knowledge base that is drawn upon; the inference mechanism that is employed; the types of outputs that are generated; and the ways in which these are communicated to healthcare providers (Box 1).

Methods
We focused on identifying the high-quality empirical literature on computerised decision support applications and their impact on the quality and safety of healthcare delivery. We searched major medical databases over a 13-year period (1997-2010) to identify systematic reviews focusing on computerised decision support. Figure 1 Key components of computerised decision support systems As eHealth research tends to be poorly indexed within bibliographic databases, we used broad search strategies using free text and allied MeSH headings. These were based on taxonomies developed for related work and scored for relevance by two independent reviewers. A comprehensive outline of our search strategy is reported elsewhere, 5 but it essentially involved combining terms relating to eHealth (including any type of computerised decision support) with quality, safety, organisational and implementation terms. Once relevant reviews on eHealth applications were identified, we searched these for the inclusion of computerised decision support systems.
The databases searched included: MEDLINE, EMBASE, The Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, The Cochrane Central Register of Controlled Trials, The Cochrane Methodology Register, The Health Technology Assessment Database, Google, LILACS, IndMed, PakMediNet and the National Health Service (NHS) Economic Evaluation Database. We also searched databases of research in progress or unpublished work including The National Research Register, ClinicalTrials.gov and Current Controlled Trials. In order to select relevant systematic reviews, we applied methodological filters devised by SIGN. 6 This approach has been validated and has been used in the development of a number of evidence-based guidelines. To be eligible for inclusion, systematic reviews needed to address computerised decision support systems, and at least one of the following: impact on safety; quality; or organisational, implementation or adoption considerations.
The search for systematic reviews was supplemented by identification of key recent papers and insights from our own contributions to the literature on decision support systems. These were identified through complementary non-systematic literature searches, aiming to identify current challenges and potential solutions to these.
Two reviewers independently assessed the potential relevance of articles for inclusion initially based on scrutinising abstracts, followed by retrieval of full-text copies of potentially relevant articles. We used established quality assessment instruments that were adapted for eHealth systematic reviews resolving any disagreements by discussion. 5 We followed four main steps in conducting an interpretive synthesis of our findings: (1) noting the range of functions and uses of existing systems; (2) developing a preliminary synthesis of the findings of included studies; (3) exploring relationships in the findings; and (4) assessing the robustness of the synthesis produced. 7 In evaluating the overall strength of evidence, we considered the World Health Organization's Health Evidence Network system for public health evidence. 8

Results
Our searches yielded 121 systematic reviews relating to eHealth applications. Of these, we found 41 relevant systematic reviews investigating various aspects of the effectiveness and safety of computerised decision support systems ( Figure 2).  Overall, we found wide variations in definitions, system functionality, settings, methodology, measurements and reporting of outcomes across studies (Table 1).
Computerised decision support systems vary in design and function (Box 2), 50 and can be used by any professional in any healthcare setting. The range of application of these systems is large and can potentially involve supporting the full spectrum of both clinical and non-clinical activities (see Box 3 for some examples).

Improving practitioner performance
Six systematic reviews have demonstrated that computerised decision support systems can improve practitioner performance, especially in some domains such as the promotion of preventive care. For example, Garg et al revealed that computerised decision support systems had positive impacts in 16/21 (76%) of the studies investigating the issuing of reminders for cervical cancer screening, mammography and vaccinations. 9 Computerised decision support systems Box 1 Examples of different types of computerised decision support systems . Range of tasks, e.g. diagnostics, preventive care, prognostics. . Approaches to utilising patient data, e.g. direct input by users or automated interrogation of existing electronic health records. . Types of knowledge-base that are drawn upon and the inference mechanism that is employed, e.g. from different commercial vendors. . Types of outputs that are generated, e.g. warnings, advice. . Ways in which these are communicated to healthcare providers, e.g. presentation of popup boxes.
have further been found to improve guideline adherence and specific tasks in chronic disease management across a range of care settings and disease contexts, albeit with more variability than for prevention. 9,11 The research to date has also identified key correlates of which tools may prove particularly effective in improving these practitioner/process end-points. Correlates of success found by Garg et al, 9 were: automatic prompting to use the system and use of 'home-grown' systems; by Kawamoto et al, 22 that automatic provision of advice is most effective when 'pushed' because it can then be incorporated into clinicians routine workflow; and by Damiani et al, 17 automatic provision of clear recommendations.

Improving patient outcomes
Despite these promising practitioner-related results, there are as yet limited data on changes in practitioner performance translating into improvements in patient outcomes, either directly (i.e. outcomes resulting from providing advice to a clinician at a single point in time) or indirectly (i.e. more remote outcomes that relate to the multitude of factors that stand between the advice and care provision further down the line). [9][10][11][12]18,21,25,30,43,49 This largely reflects the prohibitive size and/or duration of studies needed to demonstrate an effect on clinically relevant patient-level outcomes. As a result, patient outcomes tended not to be studied and, when they have been the studies have often been underpowered and shown little benefit. For example, 52/100 trials retrieved by Garg and colleagues reported on patient outcomes and of these only seven (13%) showed positive patient-level outcomes. 9 Two of these were in the context of medicines management interventions. Similar disappointing patient-level outcomes have also been reported by, amongst others, Georgiou et al, 10 Heselmans et al, 11 Jamal et al, 12 Mador and Shaw 13 (16) Decision support systems (CDSS) may lead to improvements in outcomes, but types of systems and methods vary widely across studies.
Chatellier et al 1998 (35) CDSS for anticoagulant management is effective in control of treatment and reduces adverse events.
Damiani et al 2010 (17) Found significant improvements in process of care after implementation of computerised clinical guidelines.
Delpierre et al 2004 (18) Found that use of electronic health record systems (including CDSS functionality) increased clinician and patient satisfaction and found a positive impact on preventative care; but the impact on patient outcomes was inconclusive.  (11) The authors found no evidence of an effect on patient outcomes, but the evidence is more mixed in terms of process of care.
Hider 2002 (41) CDSS can reduce adverse reactions and length of hospital stay.
Jamal et al 2009 (12) Found a positive impact on guideline compliance amongst practitioners but mixed evidence relating to CDSS impact on patient outcomes. Shiffman et al 1999 (28) Computer-based guideline implementation systems can improve guideline adherence and record keeping. . Diagnostics, e.g. proposing a diagnosis of heart disease based on electrocardiogram results in the patient record. . Disease management, e.g. blood pressure monitoring in people with hypertension. . Guideline-based and prescribing-related decisions and monitoring, e.g. systems alerting for contraindications and inappropriate medication doses. . Prognostics, e.g. helping to predict prognoses of malignant tumours, helping to predict risks based on riskprediction algorithms. . Public health surveillance, e.g. public health alerts to promote awareness of infectious diseases or environmental hazards. . Supporting research, e.g. by capturing user responses to alerts, or by identifying eligible patients and supporting data collection according to pre-defined protocols in enrolled patients.

Box 2 Examples of different designs and functions of existing computerised decision support systems
. Level of integration: Systems can be 'stand-alone' or 'integrated' with other clinical information systems, e.g. electronic health records. . Data entry: Patient data can be inputted by manual entry, transferred from other clinical information systems or transmitted from medical devices. . 'Active' and/or 'passive' engagement by users: Whether the information is 'pushed' to the user -often in real-time so as to be able to support care decisions at the point-of-care -or whether professionals need to retrieve or 'pull' the advice from the computerised decision support system.

Cost-effectiveness of computerised decision support systems
Only few controlled trials measuring the cost-effectiveness of computerised decision support systems in ambulatory care have been undertaken, and most systematic reviews are silent about cost-effectiveness considerations. The recently reported review by Main et al, 14 which aimed to compare the effectiveness of a computerised decision support-based approach when compared with a simpler system that supported test ordering (also sometimes known as computerised physician order entry) for test requests only found two studies with economic data: one showed a cost reduction with computerised decision support, whilst the other found an increase in costs for the tests requested. 51,52 However, models suggest there may be substantial cost savings, suggesting more trials are needed. 53

Empirically demonstrated risks of computerised decision support systems
The included studies reviewed have tended not to report adverse outcomes. This may reflect the fact that these interventions are largely safe or, alternatively, it may represent a failure adequately to consider such risks, whether in the primary studies or the secondary reviews.

Discussion
Procurement considerations: homegrown or commercial systems?
Most of the current evidence of benefits associated with decision support systems comes from evaluation of 'home-grown' systems. [54][55][56] These typically undergo long developmental cycles that involve numerous rounds of iteration; their development is therefore both labour-intensive and expensive, but they have the distinct advantage of being extensively customised to meet local needs. However, the vast majority of healthcare organisations use commercial 'off-the-shelf ' systems. There is as yet very limited evidence on these more generic systems that typically lack the potential to be tailored to individual clinical needs, 54,[57][58][59][60] although studies are now underway investigating the effectiveness of commercial systems.

Ensuring safe implementation of computerised decision support
Computerised decision support systems work best when these meet a perceived need (i.e. they are chosen rather than forced on practitioners), are tailored to clinicians' requirements, and seamlessly integrate with existing clinical information systems. 61 Safe implementation and adoption therefore involves an ongoing process of 'working out', which requires engagement with and empowerment of end-users. 62 Although the evidence of actual harm associated with computerised decision support systems is limited, the introduction of these technologies may have a detrimental effect on care processes and even outcomes, 63,64 especially if not implemented well. Risks to safe implementation may be technical such as faulty algorithms resulting in erroneous advice generated by systems and data inaccuracies in the patient electronic health records that computerised decision support systems draw on. 65,66 Such errors can sometimes result in disastrous consequences for patients. 67,68 Berner et al recently explored the effects of incomplete patient data on computerised decision support accuracy and found that missing data elements resulted in inappropriate and unsafe recommendations in almost 77% of patient encounters. 69 Fernando et al, drawing on simulated test cases to investigate prescribing errors, found that all four prescribing support systems tested failed to alert for at least some of the test scenarios developed by experts. 67 To mitigate these issues, organisations and vendors should track the problems that occur, and systematically implement solutions to prevent them.
In addition to these technical threats to optimal functioning, there are also complex sociocultural considerations surrounding the implementation of computerised decision support systems. For example, alerts are frequently overridden or ignored by users. 57,68,70 This may in part be due to the increasing data that can now be generated on individual patients leading to cognitive overload. 71 For example, in a review of clinician responses to drug safety alerts in an ePrescribing system, van der Sijs et al found that, with the exception of serious alerts for overdoses, safety alerts were overridden in 49-96% of cases, 70 although this does not need to be the case if the alerts are carefully selected. 72 The reasons for these include: alert fatigue (caused by too many perceived false-positive alerts); disagreement with the recommendations; poor presentation of alerts (e.g. too long and difficult to interpret); lack of time to pay adequate attention to the message; knowledge gaps (e.g. failing to recognise the importance of the alert); and irrelevant alerts (particularly in systems that do not take into account patient-specific circumstances). 70,73,74 Unintended adverse consequences may also occur due to a disruption of user workflows resulting from poorly integrated computerised decision support tools. This is particularly well illustrated by studies conducted by Han et al and DelBeccaro et al, who showed that use of the same system integrated within workflows in different ways, contributed to increased patient mortality at one site and improved survival rates at another. 63,75 Users may find alerts so disruptive that they switch them off altogether. [76][77][78][79][80][81] In these cases, complex ethical and liability questions emerge, for example in situations where adverse patient-level consequences result from switching off potentially lifesaving alerts. 82 There are, however, also associated legal considerations associated with organisations investing in poor-quality computerised decision support systems. 83 Central to these deliberations is a need to explore user requirements, differences in human and machine reasoning processes and resulting levels of adherence. For example, existing low levels of user adherence to alerts may in fact be due to the significant differences between human reasoning processes and information processing in clinical decision support systems. Some of these differences are outlined in a recent paper by Marewski and Gigerenzer, who explain how clinical decisions are often based on heuristic shortcuts whilst information technologies are commonly based on 'information greedy models of medical decision making' (p. 81). 84

Future directions
Variations in definitions, settings, system functionality and methodology somewhat complicated the comparability of findings. This highlights the need for more rigorous standards regarding methods and reporting of primary research and systematic reviews. 37 The start date of our searches was chosen for pragmatic reasons, as it was considered unlikely that systematic reviews of computerised decision support technologies would still have clinical relevance if they were conducted before this time period. 85 Future research needs to focus on understanding the contexts in which different decision support tools are most likely to prove effective in supporting clinical care. System designs then need to be configured so that they fit in with existing needs and the routines of different users. [86][87][88] This will require user involvement in all stages of design, development and deployment, and help to ensure that new applications are fit-forpurpose, thereby minimising risks to patient safety. 61 A major limitation in existing systems is the reliance on structured and coded 'computer-readable' data, which requires considerable human effort in allocating clinical data to pre-existing categories that can be processed by computers (such as for example allergies to penicillin, which are then used to trigger an alert). Clinicians vary widely in their ability to code structured data for use by electronic health records, leading to wide variations in the accuracy and completeness of clinical coding. 89 Although not formally identified in reviews identified in our work, new developments may in future offer possibilities in relation to understanding the spoken word and free text entries. These may include, for example, Natural Language Processing applications that can translate human language (both speech and written) into coded data and vice versa. This means that future computerised decision support systems may be able to automatically scan existing patient records and trigger alerts or reminders based on these, thereby facilitating the usability of such systems. A recent new development in this respect is IBM's 'Watson', which is being developed in collaboration with the University of Pittsburgh Medical Center. 90 Although still in development, the system aims to be able to interpret natural language, retrieve and interpret relevant medical literature, generate hypotheses, and thereby use existing patient data (e.g. past medical history, laboratory results) to generate recommendations for care (e.g. on diagnoses and treatments). However, even if found to be effective, such new applications will need to be tailored to different care settings, will still require provider oversight, and are likely to introduce new unanticipated risks, which will necessitate careful piloting and formal evaluation.
Given the considerable costs of developing and maintaining computerised decision support systems as well as the significant potential benefits of some functionalities across settings (e.g. drug allergy alerting), we and others have suggested that there is much to be gained from sharing the content of computerised decision support systems both within and between organisations, and also between providers of electronic health record systems. 91,92 This is particularly true of the 'evidence engine' behind such systems -if clinicians are not able to assess the quality of evidence that systems use and be assured that this evidence is updated when needed, they may be less willing to trust their recommendations. This sharing of content would be particularly valuable to developing countries where the costs of such tools are currently prohibitive. These efforts may also involve developing an inventory -that is periodically updated -of decision support tools that are available for use across care settings. 14,93 In view of the potential for the introduction of new risks, the regulation of computerised decision support systems also needs to be reviewed, 83 as systems are currently outside the remit of the Medicines Health Regulation Authority in the United Kingdom and the Food and Drug Administration in the United States of America. A more integrated approach to regulation might help address the issue of 'defensive' practices by software developers (e.g. the factoring into the algorithms of any conceivable possibility), 94 which may through 'over-alerting' inadvertently increase the risk of adverse events. 82,95 The challenge however is to not over-regulate as this then runs the risk of stifling innovation. 83 Many of the clinical decision support approaches implemented to date have been relatively simplistic. Whilst there is certainly value in pursuing the evaluation and development of such systems due to their lower cost, more straightforward implementation and significant potential for developing countries in particular, there is now also a need to focus on more advanced decision support systems as these become available. Such efforts should investigate both costeffectiveness of clinical decision support systems and their effect on patient outcomes.

Conclusions
We anticipate a proliferation of decision support tools to support clinical practice over the next few years. Based on the available evidence, 96-101 carefully developed and appropriately implemented computerised decision support systems can improve performance in domains such as preventive health and prescribing safety, although the evidence about improvements in patient outcomes remains unclear. Decision support tools need to be developed and refined in conjunction with clinicians if these are to have the desired impact on effectiveness; their cost-effectiveness and potential disruption to user workflows should also be considered. Flexibility in incorporating information from diverse sources and adaptability to varied practice settings are likely to be key quality criteria by which computerised decision support systems are judged in the future; 102 this together with the exciting opportunity of being able to draw on and make sense of natural language has the potential to have a positive impact on provider efficiency. Throughout this journey, there is a need for a shift from an industry-based 'push' model, which has to date dominated the implementations of commercial systems in particular, to a user-led 'pull' model, where end-users are involved in deciding which decision support tools are developed and when and how this information is presented to busy clinicians. 61,103