Edinburgh Research Explorer A review of the empirical evidence of the value of structuring and coding of clinical information within electronic health records for direct patient care

Many of the intended beneﬁts of electronic health records (EHRs) rely upon computers automatically processing the information in them. Such beneﬁts may include: practitioner-level outcomes such as more con-sistent and complete clinical documentation; improved patient outcomes and better quality health care derived from evidence-based and guideline-oriented health care; 1,2 more eﬃcient, eﬀective healthcare systems as a result of greater use of computerised decision support (CDS); enhanced workﬂow management; ABSTRACT Background The case has historically been presented that structured and/or coded electronic health records (EHRs) beneﬁt direct patient care, but the evidence base for this is not well docu-mented. Methods We searched for evidence of direct patient care value from the use of structured and/ or coded information within EHRs. We interro-gated nine international databases from 1990 to 2011. Value was deﬁned using the Institute of Medicine’s six areas for improvement for healthcare systems: eﬀectiveness, safety, patient-centredness, timeliness, eﬃciency and equitability. We included studies satisfying the Cochrane Eﬀective Practice and Organisation of Care (EPOC) group criteria. Results Of 5016 potentially eligible papers, 13 studies satisﬁed our criteria: 10 focused on eﬀectiveness, with eight demonstrating potential for improved proxy and actual clinical outcomes if a structured and/or coded EHR was combined with alerting or advisory systems in a focused clinical domain. Three studies demonstrated improvement in safety outcomes. No studies were found reporting value in relation to patient-centredness, timeli-ness, eﬃciency or equitability. Conclusions We conclude that, to date, there has been patchy eﬀort to investigate empirically the value from structuring and coding EHRs for direct patient care. Future investments in structuring and coding of EHRs should be informed by robust evidence as to the clinical scenarios in which patient care beneﬁts may be realised.


ABSTRACT
Background The case has historically been presented that structured and/or coded electronic health records (EHRs) benefit direct patient care, but the evidence base for this is not well documented. Methods We searched for evidence of direct patient care value from the use of structured and/ or coded information within EHRs. We interrogated nine international databases from 1990 to 2011. Value was defined using the Institute of Medicine's six areas for improvement for healthcare systems: effectiveness, safety, patient-centredness, timeliness, efficiency and equitability. We included studies satisfying the Cochrane Effective Practice and Organisation of Care (EPOC) group criteria. Results Of 5016 potentially eligible papers, 13 studies satisfied our criteria: 10 focused on effectiveness, with eight demonstrating potential for improved proxy and actual clinical outcomes if a structured and/or coded EHR was combined with alerting or advisory systems in a focused clinical domain. Three studies demonstrated improvement in safety outcomes. No studies were found reporting value in relation to patient-centredness, timeliness, efficiency or equitability. Conclusions We conclude that, to date, there has been patchy effort to investigate empirically the value from structuring and coding EHRs for direct patient care. Future investments in structuring and coding of EHRs should be informed by robust evidence as to the clinical scenarios in which patient care benefits may be realised.
Keywords: clinical coding, clinical outcomes, electronic health records, health information technology, structured data entry enhanced interoperability between EHR systems; better informed planning of health services; 3,4 and the reuse of data for research. 5 There are currently major limitations in the extent and reliability with which computers can interpret free-text clinical notes, whereas structured forms and the coding of clinical entries permit computer interpretation and automated analysis for purposes such as decision support. 6,7 The case has therefore historically been presented that the use of structure and coding within EHRs benefits direct patient care beyond computerised physician order entry (CPOE). Many national eHealth programmes include a structured EHR underpinned by interoperability, clinical modelling and terminology standards. The recent specification of 'meaningful use' in the USA, 8  However, there are concerns that clinical systems are not yet usable enough for physicians to structure and/or code all aspects of documentation, resulting in most computerised records still being free-text. Challenges in relation to the professional education and behaviour change implications of adopting standardised EHRs are significant. 9 Previous work has found clinician behaviours to have led to significant volumes of coding errors. 10 The process of recording new information does not adequately include learning from existing EHR data, 11 and there are concerns that the move away from narrative recording might depersonalise health care. 12 Given these challenges, investments in promoting structured EHRs should prioritise those aspects of documentation that bring maximum and near-term value. Some benefits of structuring and/or coding that are well-established, and have been prioritised, include secondary uses such as clinical audit, clinical research, epidemiology, public health, health services research and billing.
However, the evidence for direct care benefits from structuring and/or coding EHRs, and potential harms, has hitherto not been well-documented, making it difficult for eHealth programmes to set appropriate priorities for clinical documentation standards. Building on a recent review of the benefits and risks of structuring and/or coding the presenting patient history, 13 we sought to examine the evidence of structuring and/or coding clinical information within EHRs for value to direct patient care. In so doing, we defined value using the six aims of improvement for 21st century healthcare systems in the Institute of Medicine's (IoM's) Crossing the Quality Chasm report, i.e. improvements in safety, effectiveness, patientcenteredness, timeliness, efficiency or equitability of the care delivered to patients. 14

Methods
The search strategy was developed by iteratively scoping the literature, through discussion with experts and from previously published search strategies. Nine databases were systematically searched for published, unpublished and in-progress research: MEDLINE, EMBASE, CINHAL, PsycINFO, IndMED, LILACS, Paklit, NIHR and Google Scholar. Hand searches of specialist journals and the authors' personal libraries were undertaken in parallel, including snowballing and consultation with other specialists.
To maximise the sensitivity of electronic search terms, an extensive set of keywords and subject headings (MeSH) was developed in three key topic areas: . EHRs 7 . tools and techniques for structuring health records (e.g. templates, terminology browsers and pick lists) . structuring or coding. 15 These were all combined using the 'AND' operator.
Keywords relating to the category of evidence 16 or to the IoM's six dimensions of quality 14 were not included within search strategies as it was found during a pilot study that these were not reliably stated in publications, hence the risk of missing important studies. These criteria were applied manually during title and abstract screening. The final search strategy for MEDLINE, as an example, is given in Box 1.
Title and abstract screening was undertaken on the basis of the criteria defined in Box 2.
Full paper screening applied these same criteria. Some publications were excluded if only a conference abstract existed with no follow-up full paper to adequately assess the intervention, evidence or outcome. Data from empirical studies were abstracted into an evidence table; findings were then first descriptively summarised and then thematically synthesised.

Effectiveness
The majority of relevant publications focused on improved clinical effectiveness (normally assessed through improved adherence to a clinical guideline). [17][18][19][20][21][22][23][24][25][26] These are subgrouped below by disease area. Two studies introduced a structured format as the only intervention: almost all made use of structured data to present prompts as the influential intervention, or evaluated the use of an EHR system as a whole. These studies, however, demonstrate improvements in actual or proxy outcomes when a structured EHR was combined with simple alerts. Studies that referred to the use of coded information offered users a non-hierarchical list of terms (e.g. a pick list) to standardise data entry. No studies employed a hierarchical terminology, in which terms of varying granularity were organised to permit aggregated analysis.

Diabetes
Lobach and Hammond demonstrated that a diabetes clinical guideline could be successfully tailored for specific patients, based on prior information in each patient's EHR. 17 The EHR system at Duke Family Medical Centre was modified to generate a patienttailored encounter form for each diabetes review. Thirty clinical staff were randomised to receive either this tailored form or a generic form covering the whole guideline. The study examined 884 diabetes review encounters. Clinician compliance rates were calculated as the number of recommendations followed over the number of recommendations due for each patient during an encounter. Median compliance for the group receiving the tailored recommendations was 32.0 versus 15.6% for the control group (P = 0.01). Despite this improvement, guideline compliance was lower than the authors had expected, which they attributed to the increased workload that would be required for full compliance and to patient-specific factors that had not been taken into account.
O'Connor et al undertook a controlled study investigating the impact of introducing an EHR system on diabetes care by comparing two community clinics of the HealthPartners Medical Group. 18 Both practices had achieved similar standards of care, including HbA1c control, prior to the study. One practice adopted an EHR system that collected structured diabetes reviews and prompted clinicians if: (a) a patient had no HbA1c test within six months, or (b) if the patient's HbA1c levels were 8%. The other practice used existing paper charts. Frequency of HbA1c tests increased at the EHR clinic compared with the non-EHR clinic (P < 0.001). However, HbA1c levels improved in both clinics (P < 0.05) with no significant differences between clinics after two years (P = 0.10) or four years (P = 0.27).
More recently, Cebul et al demonstrated that intermediate outcomes for diabetes are significantly better in practices using an EHR than those using paper records. 19 Their study included 46 primary care prac-Box 1 Example search strategy for MEDLINE and numbers of hits returned 1. Tools and techniques search records returned = 12 558 (tool OR tools OR technique OR techniques OR questionnaire* OR computer based questionnaire* OR patient-completed history questionnaire* OR computer-administered patient interview* OR form OR forms OR structured form* OR problem-specific report form* OR self-report form* OR structured registration form* OR *template* OR template-generated medical documentation OR documentation template* OR documentation system* OR case histor* OR standardised interview question* OR structured data collection OR medical history-taking device* OR history-taking system OR computer history taking system* OR patient-driven health information OR data entry kiosk* OR pro forma* OR proforma* coded chief complaints OR structured encounter forms OR encounter note OR diary OR diaries OR 'automated speech recognition' OR automated health assessment) 2. Structuring and coding records returned = 1 820 787 ( . a specified introduction of a structured and/or coded format for capturing and/or analysing EHR data as the primary intervention of the research . AND deployment in at least one healthcare setting . AND an empirical evaluation reported of additional benefit obtained from the structured and/or coded EHR data meeting the Cochrane's EPOC Group criteria, namely: a randomised controlled trial, controlled clinical trial, controlled before-and-after study or interrupted-time-series (15) . AND the evaluation could be related to one or more of the six IoM dimensions of quality of patient care.
Exclusion criteria: . ublications describing relevant new or in-progress projects for which evaluations had not yet been performed . descriptions (without evaluation) of novel eHealth solutions to deliver EHRs or supporting applications or to automatically encode or analyse EHR data . feasibility or proof of concept studies demonstrating technical success and/or usability and acceptance of an EHR or clinical application . feasibility or quality assessments of the potential for EHR data to support a quality improvement or clinical outcome, which did not demonstrate a concrete outcome . feasibility assessments or actual secondary uses of EHR data for: clinical audit, clinical research, epidemiology, public health, health services research, clinical guideline development, health service evaluations or reimbursements . the use of an electronic documentation or communication system in which the format of the data was not reported or considered material (i.e. the level of structuring and/or coding was not relevant to the intervention, only that the record is electronic or networked or that a tele-communications channel is used) . benefits from patient diaries in which the format of the data was not material (i.e. the act of keeping a diary is the intervention and the electronic tool simply enhances acceptance over paper diaries, and/or its contents are simply read irrespective of the format) . evidence of increase in data quantity and completeness of data collection or data quality without any evidence of a benefit derived from those additional data . complex interventions in which impact from changes to the capture of EHR data was not capable of being isolated from other causes of the impact (e.g. educational programmes and service model changes in parallel to the introduction of a new EHR template or guideline).  Implementation of a commercial CPOE system A significant 20% reduction in hospitalwide mortality tices in Ohio (33 computerised and 13 paper-based) and follow-up of 27 000 patients over a year. Outcomes were defined as composite quality of care standards derived from outcome measures defined by the American Diabetic Association. Achievement of composite standards for process of diabetes care (e.g. measurement of HbA1c) was 35.1% higher at EHR sites than at paper-based sites (P < 0.001), and achievement of composite standards for outcomes (e.g. HbA1c < 8%) was 15.2% higher (P = 0.005). However, the authors pointed out that improvement in quality could not be attributed to the implementation of EHRs alone due to a number of confounding variables, such as differences in patient populations and known variations in quality standards across practices prior to the study.

Venous thromboembolism
Lecumberri et al demonstrated that significant costs in the hospital management of venouse thromboembolism (VTE) could be avoided through alerts to prompt clinicians to use prophylaxis in high-risk patients. 20 They studied over 25 000 patients before and after introducing the alert. At-risk patients were identified computationally, according to American College of Chest Physicians' guidelines, through preexisting routinely collected EHR data from medical orders, daily nursing reports, surgery registries and laboratory results. Use of VTE prophylaxis in at-risk patients increased significantly from 27 to 60% following introduction of the alert, and hospital VTE incidence decreased by 50% (95% confidence interval [CI] 0.29-0.84). Accounting for the costs of the system and the savings from each prevented episode of VTE, the alerts resulted in an average saving of $8.92 (e6.54) per hospitalised patient. At the University of Illinois Hospital, Galanter et al demonstrated that the introduction of a structured risk assessment form for VTE prophylaxis significantly increased the use of prophylaxis from 25.9 to 36.8% (P < 0.001), and reduced the incidence of VTE. 21 A structured (tick box) form, linked to alertgenerating decision support, was presented via the CPOE system prior to ordering for every new inpatient. This approach did not require the preexistence of relevant data items within the hospital's EHR system, but the data items captured via this form were re-used whenever fresh orders were made for each patient.

Asthma
Bell et al demonstrated that a computerised asthma care plan linked to an EHR system improved measurement of peak expiratory flow rate (8-14%, P = 0.003) and increased the use of controlling medication such as inhaled steroids (from 1 to 7%, P = 0.006). 22 This cluster randomised trial involved 12 Philadelphia primary care practices and almost 20 000 patients. Because the control group of practices had access to the same EHR system, clinical guidelines and educational sessions, the intervention effect could be attributed to the CDS alerts. As this study did not change the way in which clinical data were captured, it is an example of the way in which structured EHR information can be exploited by CDS to improve adherence to guidelines and thereby improve proxy clinical outcomes.
Davis et al found that a template to capture routine asthma monitoring in primary care increased the appropriate use of inhaled corticosteroids. 23 Their template included checkboxes for documenting the number of days per week that the patient had symptoms, nights per month with symptoms, FEV1, recent exacerbations and use of rescue 2 agonists. This study involved 180 records of patients attending the Center for Family Medicine, Spartanburg over two 6month periods before and after introduction of the template. Use of inhaled steroids increased significantly from 39.4 to 51.1% (P = 0.017). Unfortunately the study did not track other outcomes such as emergency department attendances and hospital admissions.

Antibiotic prescribing
Linder et al evaluated a 'Smart Form' for managing acute respiratory infections in primary care, in Massachusetts. The Smart Form was integrated within the EHR system, pre-populated relevant fields from a patient's record when opened, supported the clinical encounter with structured fields (tick boxes and dropdown lists), and simplified the generation of prescriptions and letters. 24 The system advised on whether and which antibiotics were indicated (using published guidelines), advised on streptococcal testing, and checked for drug interactions and allergies. The system therefore combined features of a structured EHR (supporting data entry) with decision support (supporting appropriate prescribing). The evaluation, involving 27 primary care clinics randomly assigned to use Smart Forms or the unmodified EHR system for six months, showed marginal, but non-significant increase in appropriate and reduction in inappropriate antibiotic prescribing, particularly for acute respiratory infections. Using the Smart Form antibiotic prescribing for clinically appropriate acute respiratory diagnoses (e.g. sinusitis) occurred in 88% of encounters compared with 59% in the control group (OR = 5.0; 95% CI 2.9-8.6), and for antibiotic-inappropriate diagnoses (e.g. influenza) occurred in 27% compared with 34% in the control group (OR = 0.7; 95% CI 0.5-1.0). The authors proposed that this limited effect was due to the low frequency of use of the form, which had to be deliberately invoked rather than being automatically triggered by a symptom or diagnosis.
A study by Bourgeois et al in Boston illustrated how the adoption of a new tool to support better quality prescribing, in their case an electronic template to advise on antibiotic use in acute respiratory illness in children, can prove challenging and limit the potential benefits to patients. 25

Immunisations
Ledwich et al demonstrated that an alert, triggered by opening a patient's EHR, indicating whether that patient ought to be offered influenza or pneumococcal vaccination, can more than double vaccination rates (19-41%; P < 0.001, for pneumococcal vaccine). 26 The study of over 750 patients was performed over two consecutive years at a hospital and at a community based practice in Danville, Pennsylvania.

Safety
As no studies were found that specifically demonstrated improved safety from the use of structured or coded data within EHRs it may be concluded that neither the act of entering data via a structured proforma or the review of previously entered structured data have been evaluated from a safety perspective. Whilst there is an established body of work considering the use of CDS within CPOE systems, we found only three papers in which pre-existing structured clinical information within an EHR was the focus of investigation. These three studies are summarised below. Each uses structured data within an EHR to enable relevant alerts to be generated, thereby demonstrating improvements in safety or mortality.
In 1998, Bates et al demonstrated at the Brigham and Women's Hospital, Boston, that the integration of allergy and interaction alerts with a medication ordering system, and the co-presentation of salient laboratory values, reduced serious medication errors by 55%. 27 The impact was primarily on dosing errors and known allergy errors, both of which required structured and computable data entry and the retrieval of relevant data within each patient's EHR. The primary success factor was the use of pre-existing structured EHR data (medication lists and allergy lists), combined with a knowledge base (such as drug-drug interactions), to enable the generation of alerts.
Evans et al demonstrated at the LDS Hospital in Salt Lake City that an antibiotic advisory system can use structured EHR data for diagnosis, medication, microbiology and renal function to provide tailored antibiotic recommendations. 28 A before-and-after evaluation revealed that the system significantly reduced the frequency of allergies, the duration of antibiotic courses and length of stay, thereby also reducing costs.
Implementation of a commercial (locally modified) CPOE system in an academic children's hospital was associated with a significant 20% reduction (95% CI 0.8-40%) in hospital-wide mortality. 29 This observational study of 17 432 hospital inpatient episodes, compared with 80 063 historical pre-implementation controls, sought to account for trends or other factors that might have influenced this rate. The authors concluded that the success factors included the standardisation of the information used to create orders, shared EHRs including vital signs and medication records, and the consequent better support of teambased care. The authors did not specify the data items that contributed to this improvement, and it is therefore difficult to infer the value gained specifically from structuring or coding parts of the record.

Patient-centredness, timeliness, efficiency and equitability
No studies were found that investigated these quality dimensions. Two of the studies summarised above 20,28 included an estimate of cost savings from the quality improvement, but no formal economic evaluations of the impact of introducing structured and/or coded EHRs were identified.

Main findings
The majority of studies we identified focused on effectiveness, demonstrating that proxy, and in some cases, actual clinical outcome can be improved if a structured EHR is combined with prompting or advisory systems in targeted areas: the management of a long-term condition, a preventive intervention, or appropriate choice of therapy. Prescribing was the only example found of improvement to safety (as defined by the IoM). 14 No studies were found reporting value for patient-centredness, timeliness, efficiency or equitability.
Our review focused on studies reporting the structuring or coding of EHRs as the primary intervention, although many of the EHR systems included an alert-generating component. Other published reviews have focused on studies primarily implementing CDS systems and their impact on patient outcomes. 6,7 CDS systems inevitably rely upon computable EHR data, which must therefore be structured and/or coded. Our reading of the CPOE/CDS systematic review literature indicates that where carefully designed these can translate into improvements in practitioner performance, but these signals are relatively weak and high up in the causal chain, with limited opportunities to have substantial impact on practitioner and patientlevel outcomes. In addition, many outcomes of interest (e.g. anaphylaxis to a drug) are relatively uncommon such that the sample size/length of follow-up needed to demonstrate an improvement in outcomes is often prohibitive.

Strengths and limitations
This review used components of previously validated search strategies, a well-accepted definition of quality and a standard definition of evidence to investigate a research area that has not hitherto been welldocumented. Despite searching widely, we found only a small number of publications meeting our review criteria. Some possible reasons for this are: (a) it is difficult to locate the relevant studies due to the way literature is indexed; (b) the papers, if any, which describe added value describe the intervention as the introduction or modification of an EHR system (comprising clinical applications, CDS, clinical workflow support as well as an EHR) where the effect of structuring and/or coding the record is not independently assessed.

Implications for policy, practice and research
Our findings suggest that structuring and coding of EHR information will usually form part of a broader reorganisation and systematisation of a particular aspect of clinical practice, for which the intervention is more holistic, and the health record information effect is difficult to isolate. The findings suggest that the use of structured and/or coded information should be isolated for study as part of a deeper understanding of the implications and imperatives of adopting EHRs, and that these studies should be evaluated using criteria directly related to the delivery of direct patient care.

Conclusions
This review indicates that there has been patchy effort to date to empirically evidence the value from structured and/or coded EHRs for direct patient care, as opposed to secondary use benefits or improved healthcare systems. Structuring and/or coding information is expensive and effortful, and a business case for its promotion should not be made without more robust evidence indicating the priority areas in which patient care benefits are most likely to be realised.

FUNDING
This work was funded by the English National Health Service (NHS) Connecting for Health Evaluation Programme (NHS CFHEP 009). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NHS CFHEP or the UK Government Department of Health.