- Split View
-
Views
-
Cite
Cite
Michael I. Harrison, Ross Koppel, Shirly Bar-Lev, Unintended Consequences of Information Technologies in Health Care—An Interactive Sociotechnical Analysis, Journal of the American Medical Informatics Association, Volume 14, Issue 5, September 2007, Pages 542–549, https://doi.org/10.1197/jamia.M2384
- Share Icon Share
Abstract
Many unintended and undesired consequences of Healthcare Information Technologies (HIT) flow from interactions between the HIT and the healthcare organization's sociotechnical system—its workflows, culture, social interactions, and technologies. This paper develops and illustrates a conceptual model of these processes that we call Interactive Sociotechnical Analysis (ISTA). ISTA captures common types of interaction with special emphasis on recursive processes, i.e., feedback loops that alter the newly introduced HIT and promote second-level changes in the social system. ISTA draws on prior studies of unintended consequences, along with research in sociotechnical systems, ergonomics, social informatics, technology-in-practice, and social construction of technology. We present five types of sociotechnical interaction and illustrate each with cases from published research. The ISTA model should further research on emergent and recursive processes in HIT implementation and their unintended consequences. Familiarity with the model can also foster practitioners' awareness of unanticipated consequences that only become evident during HIT implementation.
Introduction
Healthcare information technologies (HIT), such as electronic medical records (EMR), computerized physician order entry (CPOE), and decision support systems (DSS), may enhance the safety, quality, and patient-centeredness of care, while helping to contain costs and increase efficiency.1–5 Unfortunately, there have been disturbingly mixed reports on HIT's implementation and outcomes.4,6–13 A growing body of research and user reports reveals many unanticipated and undesired consequences of implementation.6,11,13–21 Unanticipated and undesirable consequences, which are usually called unintended consequences,22 often undermine patient safety practices and occasionally harm patients.23,24 Unanticipated consequences with desirable results may be thought of as happy surprises, while anticipated undesirable outcomes present opportunities for decisions, clarification of values, and implementation tradeoffs.22
Managers and clinicians (physicians and nurses) in healthcare delivery systems too often blame undesirable consequences and implementation failures on the performance of the newly introduced technology.25,26 Although technical flaws often cause problems, many harmful or otherwise undesirable outcomes of HIT implementation flow from sociotechnical interactions—the interplay between new HIT and the provider organization's existing social and technical systems—including their workflows, culture, social interactions, and technologies.
These sociotechnical interactions have been richly documented in the literature on HIT's unintended consequences. This literature provides several conceptual models (or frameworks) that help guide researchers and practitioners. Chief among these are a model of relations among categories of CPOE consequences,22 a typology of unintended consequences of patient care information systems,11 a typology of unintended consequences of CPOE,14 and an empirical study of the prevalence of these unintended consequences.27 Other studies describe types of errors in CPOE usage16 and provide a taxonomy of errors associated with CPOE.28
Unfortunately, many IT users and even some IT specialists, are unfamiliar with this literature or with its practical implications. When hospital leaders, clinicians, or IT specialists assume that HIT will deliver the results promised by vendors, they may overlook likely interplays between new technologies and existing sociotechnical conditions. Similarly, those who assume that computerization routinely enhances reliability may underestimate the safety contributions of clinical judgment, unmediated (direct) communication, and teamwork.14,29,30
Conceptual models of HIT's unintended consequences contribute to research on such system interactions. They can also help clinicians, healthcare managers, and HIT specialists anticipate some otherwise unnoticed consequences of implementing HIT and thereby turn unintended consequences into anticipated tradeoffs.22 These models can also help applied researchers and practitioners discern genuinely unforeseeable consequences as these emerge during HIT implementation.
To extend recent work on models of unintended consequences our proposed Interactive Sociotechnical Analysis (ISTA) offers a framework and typology specifying important relationships among new HIT, workflows, clinicians, and organizations. This framework emphasizes the recursive and iterative nature of these relationships and their potential for producing unintended consequences.
Research Background
To develop ISTA, we combine elements from five research areas. First, traditional sociotechnical systems (STS) research documents dynamic, mutual influences among the social subsystem (people, tasks, relationships), the technical subsystem (technologies, techniques, task performance methods, work settings), and their social and organizational environments.31,32 Early STS work focused on attaining an optimum balance between the social system—including the needs of the workers and the technologies used in manufacturing and extraction industries. STS analysts articulated a series of design principles.33,34 Later STS research in healthcare showed how sociotechnical forces shape work processes, which impact employee motivation and patient outcomes.35,36
Sociotechnical studies of office automation indicate that early designers of management information systems adopted a “rational/static” approach that focused mainly on enhancing information processing efficiency and managerial control.37,38 This narrow, mechanistic approach overlooks effects of new information systems on employees and their organization and fails to grasp the dynamic and interconnected nature of organizations as systems. In contrast, sociotechnical analysts advocate applying the STS principles of participative design and autonomous work groups to the introduction of new computer systems.39,40
Second, the closely-related field of ergonomics examines effects of work technologies and physical environments on individuals.41,42 Ergonomics points to crucial interactions among work organization, patient, provider, and organizational factors.43 Macroergonomics examines how work organization features, including physical settings, tasks, technology, and organizational arrangements, affect individual stress and performance.41,44 Ergonomic practitioners design physical, cognitive, and organizational work features that fit peoples' needs, abilities, and limitations.
Traditional sociotechnical and ergonomic analyses stress the benefits of appropriate design of technologies, but they tend to treat designed configurations as stable over time and consistent across contexts. Hence traditional STS and ergonomic research examines how technological features affect people and social systems but does not explore the other side of that relationship: how social systems shape technology and its uses.45 In contrast, three other research streams recognize that technology and the social world are intertwined and influence one another.
The third body of research includes social construction of technology studies, along with the broader negotiated-order and interpretive traditions on which social construction draws.46–48 Researchers in these fields show how interactions among technology users—including managers, clinicians, and other healthcare staff—help select, reinterpret, modify, and even create technologies.49–53 From this vantage point, as people adopt and use technologies, they alter them and transform relations among the technologies and their organizational contexts.
Fourth, technology-in-practice, shows that technologies such as EMR and CPOE are not just shaped by practitioners; they also mediate practice.54 Healthcare emerges through collaborative work and tight interconnections among people, tools, machines, documents, and organizational routines.55,56
Social informatics, the fifth research field on which ISTA builds, applies insights from the research just reviewed. Following social construction and technology-in-practice, social informatics explicitly acknowledges the embeddedness of information technologies within organizations and broader social contexts.45 Thus nearly identical information technologies can be applied and used very differently because of the many, complex interactions among people, between people and equipment, and even between sets of equipment.57
ISTA Framework
Overview
From the foregoing literature we draw four key features of ISTA:
• The importance of examining actual uses of HIT (“HIT-in-use”), rather than uses that were planned or envisioned by designers or managers.
• The impact on HIT use of technical and physical settings of work.
• Users' renegotiation and reinterpretation of HIT features.
• Interaction and interdependence among social and technical systems and recursive relations among sociotechnical subcomponents.
The ISTA framework encourages us to stop viewing HIT innovations as things, but instead treat them as elements within unfolding processes of sociotechnical interaction. From the viewpoint of ISTA, the results of HIT innovation can never be fully determined by the technology. Sociotechnical interactions are dynamic, emergent, hard to understand, and often surprising—conditions characterizing complex adaptive systems.58 ISTA thus rejects popular mechanistic assumptions that HIT implementation problems can be solved with more or better HIT and that proper HIT implementation depends primarily on training and technical support. ISTA thus diverges from the prevailing IT engineering approach, which models existing administrative and clinical practices in new HIT solutions, fine tunes them, and freezes these solutions and required user behaviors.51,59
Figure 1 provides a graphic portrayal of the ISTA framework. It shows interactions among subcomponents of the sociotechnical system that act as major sources of unintended consequences, as reported in the literature and observed by us. Other interaction types could be derived logically, but these do not appear to affect unintended consequences as strongly as the ones shown. The five interaction types are:
New HIT changes existing social system.
Technical & physical infrastructures mediate HIT use—Interaction of new HIT with existing technical and physical conditions affects HIT-in-use.
Social system mediates HIT use—Interaction of new HIT with the social system affects HIT-in-use.
HIT-in-use changes social system—Interaction of new HIT with the social system affects HIT-in-use, which then further changes the social system.
HIT-social system interactions engender HIT redesign—Interaction of new HIT with the social system affects HIT-in-use, which then leads to changes in HIT properties.
Arrows in Figure 1 show the impact of one STS subcomponent on another and correspond to the five interaction types. Arrow 1 shows effects of newly implemented HIT on the Social System subcomponent within the adopting organization. Arrows that pass through a subcomponent show sociotechnical interactions. For example, Arrow 2 depicts interaction between New HIT and the existing Technical and Physical Infrastructures. This interaction in turn shapes HIT-in-Use. Arrow 3 shows the effects on HIT-in-Use of interactions between new HIT and the organization's Social System. Arrows 4 and 5 indicate that, once shaped by the interactions shown by Arrow 3, HIT-in-Use can produce further changes in the Social System (Arrow 4) or in New HIT's formal properties (Arrow 5).
ISTA Types
We now review the five types of interaction presented in Figure 1 and in Table 1 (below).
ISTA Type | Unintended Consequences* |
1. New HIT changes social system |
|
Changes in Communication Patterns & Practices | |
Workflow
| |
2. Technical & physical infrastructures mediate HIT use | Paper Persistence14
|
3. Social system mediates HIT use |
|
4. HIT-in-use changes social system | Changes in the Power Structure14
|
| |
| |
5. HIT-social system interactions engender HIT redesign |
|
ISTA Type | Unintended Consequences* |
1. New HIT changes social system |
|
Changes in Communication Patterns & Practices | |
Workflow
| |
2. Technical & physical infrastructures mediate HIT use | Paper Persistence14
|
3. Social system mediates HIT use |
|
4. HIT-in-use changes social system | Changes in the Power Structure14
|
| |
| |
5. HIT-social system interactions engender HIT redesign |
|
The headings for the types of unintended, negative consequences cited by Campbell and colleagues14 are the short forms that appear in the Discussion section of their paper. A subsequent paper27 uses the same headings with minor variations.
Also treated in Ash and colleagues11(Misrepresenting Communication as Information Transfer—loss of communication). The italicized and bold type headings from the paper by Ash and colleagues are abbreviated versions of headings appearing in italics in the body of their paper. Their subtypes appear in Table 1 in italics but without bold and are shown as modifiers to the main headings.
Also treated in Ash and colleagues11 under Misrepresenting communication … catching errors.
ISTA Type | Unintended Consequences* |
1. New HIT changes social system |
|
Changes in Communication Patterns & Practices | |
Workflow
| |
2. Technical & physical infrastructures mediate HIT use | Paper Persistence14
|
3. Social system mediates HIT use |
|
4. HIT-in-use changes social system | Changes in the Power Structure14
|
| |
| |
5. HIT-social system interactions engender HIT redesign |
|
ISTA Type | Unintended Consequences* |
1. New HIT changes social system |
|
Changes in Communication Patterns & Practices | |
Workflow
| |
2. Technical & physical infrastructures mediate HIT use | Paper Persistence14
|
3. Social system mediates HIT use |
|
4. HIT-in-use changes social system | Changes in the Power Structure14
|
| |
| |
5. HIT-social system interactions engender HIT redesign |
|
The headings for the types of unintended, negative consequences cited by Campbell and colleagues14 are the short forms that appear in the Discussion section of their paper. A subsequent paper27 uses the same headings with minor variations.
Also treated in Ash and colleagues11(Misrepresenting Communication as Information Transfer—loss of communication). The italicized and bold type headings from the paper by Ash and colleagues are abbreviated versions of headings appearing in italics in the body of their paper. Their subtypes appear in Table 1 in italics but without bold and are shown as modifiers to the main headings.
Also treated in Ash and colleagues11 under Misrepresenting communication … catching errors.
Type 1: New HIT Changes Existing Social System
This type captures processes through which HIT interventions in healthcare organizations alter prior patterns of work, communication, or relationships among clinicians. HIT designers often seek to alter work practices but are too often surprised by interactions with HIT that engender undesirable changes. A well-known study on CPOE provides two examples of such unanticipated interactions.15
Introduction of CPOE into a tertiary children's hospital reduced bedside nurse-physician interaction about critically ill infants. Nurses had fewer opportunities to provide feedback that sometimes led to beneficial medication changes. CPOE procedures also altered communication between transport teams and the emergency room (ER). Prior to introduction of CPOE, transport teams radioed the ER so ER staff could order medications and complete admission forms before the patient's arrival. When CPOE was launched, transport staff had to provide patient information in the ER, and treatment was delayed until after ER staff entered the data. Resulting delays and reduced clinician interaction may have contributed to higher mortality rates.
Studies of this sort (also14,60) show how HIT implementation can alter or disrupt oral communication among clinicians, even when talk is faster, more clinically accurate, and safer than transmitting information through HIT. The challenge when introducing HIT is to improve problematic and dangerous forms of communication, such as illegible prescriptions, without undermining vital communication flows among clinicians and with patients.
Type 2: Technical & Physical Infrastructures Mediate HIT Use
Poor fit between new HIT and existing technical and physical infrastructures is a common source of unintended consequences. Most noticeable to HIT specialists are problems of interface between new and existing IT. These problems are a frequent source of technical failure for new HIT and can lead to poor decisions, delays, data loss, errors, and unnecessary testing.12,16,59
“Paper persistence” is another common consequence of inadequate integration of new HIT and existing information systems.14,59 Use of paper was widespread in the five institutions studied by Campbell and her colleagues.14 In some cases, staff had to print out CPOE orders and then manually reenter them into the local department's clinical information system.
Poor interfaces between HIT and the physical settings in which it is deployed may also lead to HIT uses and workarounds which harm safety, quality, or efficiency. IT designers lacking awareness of ergonomics may overlook simple, yet fateful environmental features, such as the physical ease with which computers can be accessed or moved (Sites F, personal communication, October 2006). Other features of work layout, along with noise, overcrowding, and distracting illumination, affect work healthcare performance and safety61 and may have unanticipated, negative effects on HIT use. While strategic workstation location can facilitate HIT use,62 inappropriate physical layouts can extend data-entry time, reduce face-to-face communication, and increase distractions.16 For example, as staff walk across a busy floor to terminals, encounters with several patients and staff may lead them to forget or garble information obtained at the bedside.
Type 3: Social System Mediates HIT Use
Reinterpretations and negotiations about new HIT often lead to different uses and practices from those intended by HIT's designers. When nurses under heavy work loads encounter cumbersome software requiring multiple screens for medication administration, the nurses often delay medication charting until the end of their shifts. This unsanctioned practice can generate inaccurate recording of medication times and quantities, inappropriate duplication of prescriptions, less efficient communication between physicians and nurses, and reduced efficacy of software safety checks.16,63
An important reason that HIT-in-use so often diverges from designed HIT is that the original designs fail to reflect ongoing features of the work system and of social relations among practitioners, including collaborative and interactive work patterns.11 Here is a vivid illustration from an observational study of the introduction of system-wide CPOE into an ICU:64
Before introduction of CPOE, nurses were used to alerting physicians about changes in patients' conditions. Physicians often gave oral orders to nurses and only signed orders after the nurses had transcribed and administered them. The CPOE required physicians to initiate orders, pharmacists to check them, and clerks to deliver them to nurses for administration. Imposition of this linear workflow led to delays in delivery of orders, left nurses in doubt whether physicians had initiated orders, and sometimes produced divergent printed orders: the physician's original and the pharmacist's modification. In response, nurses continued to initiate orders by making suggestions to physicians. Nurses also frequently interrupted physicians to ensure that orders had been entered into the CPOE system. Moreover, nurses assumed responsibility for deciding what to do when the CPOE system presented them with conflicting medication orders.
Workarounds like these may help preserve important forms of collaboration that are not supported by an HIT design. But such workarounds may also make care less efficient and compromise safety and quality.
Type 4: HIT-in-Use Changes Social System
This type reflects the recursiveness of HIT implementation. Use of new HIT is altered by the care organization’ social system (as in Type 3), but then the HIT-in-use leads to new and additional changes in the social system. Consider the following example:
New HIT (i.e., CPOE and EMRs) allows Infectious Disease (ID) fellows to access information on patients for whom housestaff wish to prescribe newer, expensive antibiotics. The ID fellows seek to limit use of expensive, broad-spectrum antibiotics. Housestaff, however, are often eager to try new antimicrobials and to avoid the ID approval process. Housestaff developed “ stealth dosing,” waiting until the ID fellows go off-duty at 10 PM to order the restricted antimicrobials.65 The next morning, ID fellows review the previous night's orders and sometimes demand medication changes. But changing antibiotic regimes can be problematic, so ID fellows let many such orders stand even if they are not ideal. Because ID fellows often take no remedial actions and housestaff can game the system, stealth dosing constrains the ID-fellows' authority and weakens the antimicrobial oversight process.
The beginning of this example sounds like the ICU example (Type 3): Housestaff workarounds lead to unanticipated HIT uses (stealth dosing). But the continuation of the stealth dosing example shows that the HIT-in-use (i.e., workarounds) also affects authority relations.
Type 5: HIT-Social System Interactions Engender HIT Redesign
This type involves an even more dramatic process of recursive change than that described in Type 4. Here users' reactions to new HIT and their local adaptations of it diverge dramatically from the original HIT design. As a result managers and IT designers are forced to reconfigure some HIT features. For instance, in an academic hospital, physicians who enjoyed substantial clinical autonomy objected strenuously to new, fixed-choice EMR templates for recording intake diagnoses. Managers were obliged to accept free-text entry.63 Here is another example of this type of recursive interaction:
When ordering medications via the computer, some systems incorporate off-the-shelf DSS. These generate warnings about doses or interactions, which quickly become overwhelming, and are ignored as “alarm fatigue” sets in.11,12 Physician annoyance and ignored warnings lead managers to deactivate the alarms. Subsequently IT designers seek to reintroduce selected warnings that have been approved by in-house clinicians.66 Another managerial response involves “tiered” alarms,66 where warning levels mirror possible harm levels: physicians may ignore low-harm alarms, must acknowledge more serious alarms, and must explain overrides of the most serious alarms.
In this case, HIT innovations trigger responses by individuals (ignoring warnings), by managers (removing, then introducing new warnings), and by technology designers (tiered alarms).
Comparing ISTA to Existing Models
ISTA contains both continuities and departures from previous models of unintended consequences and from other previous studies of HIT outcomes. The first two ISTA types are widely discussed in STS and ergonomics. Type 3 is prominent in the literature on unintended consequences but has received less attention from other researchers in information technology and healthcare. Types 4 and 5 are less fully developed in most discussions of unintended consequences and virtually absent from the broader literature on HIT.
Table 1 shows the relation of the ISTA types to two well-known typologies of unintended consequences.11,14 ISTA's concepts are more abstract than those in these two studies. Hence ISTA requires just five types to represent eight of the nine categories in the CPOE typology.14 Only the “negative emotions” type in that typology is not covered by ISTA, because emotion is at the level of the individual user, rather than the group or organization. Similarly, all thirteen sub-types of unintended consequences for patient care IT11 fit into just three ISTA categories. ISTA focuses more directly on recursive interactions than the typologies of unintended consequences. The limited number of elements and types within ISTA may make it easier for researchers and practitioners to apply.
Discussion
Growing awareness of HIT implementation's mixed results contributes to its shallow market penetration.67,68 While some vendors and lobbyists still proclaim HIT as a panacea for an ailing healthcare system, users already know that HIT produces many unanticipated and undesired consequences. They also know that unintended consequences can sometimes undermine quality and safety and can lead to implementation failure.
This paper developed a model of the interactions that produce unintended consequences and illustrated them from the research literature The ISTA model builds on earlier models and draws from diverse research streams, several of which are not widely cited in the HIT literature. ISTA emphasizes emergent and recursive interactions among HIT and existing social systems, technologies, and physical environments. Further research, including observational studies actual HIT use, should document and analyze such recursive and emergent processes in HIT implementation. Then, standardized measures of these concepts could be developed.
While not absolving new technology developers from their responsibilities to offer responsive, safe, and useful products, ISTA places analysis of the dynamics of healthcare organizing48 at the center of HIT implementation. Many of the most important consequences of sociotechnical interactions emerge as new HIT becomes intertwined with the ongoing sociotechnical system.11 These developments cannot be fully anticipated during HIT design and implementation planning. To spot such unfolding consequences of sociotechnical interactions, managers, designers, clinicians, and researchers need to carefully track HIT-in-use throughout implementation. This kind of tracking requires continuous or repeated evaluations of IT use and consequences, along with frequent feedback to managers, IT designers, and users. Such formative evaluations—as opposed to post-hoc summative evaluations—can help managers and IT designers address emerging consequences before they harm patients, compromise quality, or undermine implementation.
By emphasizing the complexity and emergent nature of sociotechnical interaction, ISTA encourages practitioners and researchers to view system interactions as opportunities for learning and improvement, rather than as annoying barriers or signs of user resistance. Examining HIT-in-use provides opportunities for understanding and responding to user experiences and emerging needs, for example by changing the pace of implementation or reconfiguring IT properties. Those who successfully installed sophisticated HIT products report on this type of learning. Close and long-term work with vendors, consultants, and internal designers supported an iterative refinement process.1,4,69,70 The successful systems customized and reconfigured HIT to address its sociotechnical consequences.69,71 In making such changes, HIT implementers must balance adaptation to local needs and practices against maintenance of standardization and systems interoperability.
In the long run, some of HIT's unintended consequences may play a role in helping HIT designers and healthcare professionals learn to deploy HIT more effectively. Of course, practitioners of HIT and medicine cannot tolerate developments that harm patients; HIT implementation therefore must proceed with caution. Familiarity with the ISTA model can help clinicians, managers, and designers become more aware of critical, unfolding processes in HIT implementation. This awareness will, in turn, help these practitioners realize HIT's potential for improving healthcare safety, quality, and efficiency.
References
The following individuals made helpful comments on drafts of thismanuscript or reported incidents that contributed to our developmentof the manuscript: David Douglas, MD, Portland VA Medical Center(Portland, OR); Joseph Francis, MD, MPH, Department of VeteransAffairs (Washington, DC); Jennifer Lai, BA, BSN, RN, Hospital of theUniversity of Pennsylvania (Philadelphia, PA); Cynthia Palmer, MS,Agency for Healthcare Research and Quality (AHRQ) (Rockville, MD);Nancy Rodenhausen MS, RN, N.Y. Downtown Hospital (NY); ChristineA. Sinsky, MD, Medical Associates, P.C. (Dubuque, Iowa); FrankSites, RN, Penn E-lert eICU, Univ. of Pennsylvania Health System(Philadelphia, PA); Claudia Steiner, MD, AHRQ (Rockville, MD); JoelL. Telles, PhD, Main Line Health (Bryn Mawr, PA); Prof. Michel Vilette,Argo Paris Tech (Paris, France). The following individuals providedinvaluable technical assistance in preparing the manuscript: SandraCummings and Mary Rolston of AHRQ's Public Affairs Division andCaryn McManus of AHRQ's Information Resources Center. ShirlyBar-Lev's research was part of her doctoral dissertation in Sociology atBar Ilan University. Her research was supported by a DissertationFellowship from that institution and supervised by Michael Harrison.Ross Koppel was supported in part by a grant from the Agency forHealthcare Research and Quality (P01 HS11530).This article is written by an employee of the US Government and isin the public domain. This article may be republished and distributedwithout penalty.
The views expressed in this paper do not necessarily represent thoseof any U.S. government agency, but rather reflect the opinions of theauthors.
IRB approval was not requested.