BACKGROUND

The National Academies of Sciences, Engineering and Medicine (NASEM) report, “Improving Diagnosis in Health Care”1 called on health care organizations (HCOs) to create environments that identify, learn, and reduce diagnostic errors in care. However, most HCOs find diagnostic safety harder to address compared to other safety concerns2, 3—diagnosis is a complex and evolving process involving uncertainty.4, 5 Diagnostic errors are harder to identify and analyze.6,7,8 Consequently, guidance is limited on how to approach measurement and improvement of diagnostic error in clinical settings.9

Learning health systems (LHS) approaches can be useful to advance diagnostic safety.2, 10 According to the National Academies, a learning health care system is “one in which science, informatics, incentives, and culture are aligned for continuous improvement, innovation, and equity - with best practices and discovery seamlessly embedded in the delivery process, individuals and families active participants in all elements, and new knowledge generated as an integral by-product of the delivery experience.”11 Quality and safety are inherent in a LHS where measurement to produce actionable data for improvement is foundational. LHS approaches may be useful to consider in addressing diagnostic errors.9

Diagnostic errors pose unique challenges for learning and improvement due to the complexity of the diagnostic process, the human-system interaction inherent to making a diagnosis, and resources necessary to analyze those errors to determine contributing factors.9 Many HCOs do not have adequate infrastructure or processes in place to identify diagnostic errors let alone learn from them.12 Given the limited practical guidance in the literature on how health care organizations can reduce diagnostic errors, we sought to advance understanding of how organizations are currently approaching diagnostic safety. Using concepts from learning health systems, we interviewed health care quality and safety leaders across the USA to identify both challenges and pragmatic strategies for improving diagnostic safety at an organizational level.

METHODS

Study Setting and Population

We recruited 32 health system quality and safety leaders from a diverse set of 30 US health care organizations. The HCOs are primarily academic health systems with larger hospitals in urban areas and associated primary, urgent, and specialty care locations in surrounding and outlying areas. The leaders identified their primary roles as senior level personnel in quality and patient safety, institutional leadership, and/or frontline clinicians (Table 1). We used two methods for recruitment: (1) an invitation sent by Institute for Healthcare Improvement (IHI) to a list of patient safety professionals in the combined IHI and National Patient Safety Foundation database (including Chief Quality Officers, certified professionals in patient safety, American Society for Professionals in Patient Safety) to explore interest. The invitation included a brief recruitment survey to identify individuals who had the experience and authority to discuss barriers as well as opportunities to help clinicians and their organizations address or improve diagnostic safety. Our team then followed up with interested participants to schedule an interview. (2) Due to the low response rate (n=18), the team sent out direct recruitment emails to quality/safety leaders at different HCOs known to be working on diagnostic safety initiatives (i.e., based on knowledge from published literature, blogs, or personal knowledge based on existing networks). A final list of participants represented geographically diverse academic and non-academic settings caring for adults and/or children.

Table 1 Characteristics of Interview Participants
Full size table

Interview Guide Development

We developed a semi-structured interview guide to facilitate data collection. The content of the interview guide was divided into five domain areas generated by literature review and extensive team discussions. The final interview guide included 23 questions (Appendix 1) with probes, as needed, exploring the challenges of addressing diagnostic error and patient safety within learning health care organizations. The 5 domains assessed by the interview guide included (1) clinical staff/leadership (e.g., are there teams/programs to address diagnostic error, if not, what are the barriers; does your board/senior leadership (C-suite) demonstrate culture and commitment towards addressing diagnostic errors?); (2) culture of reporting and learning (e.g., how do you encourage your staff and employees to voice their concerns?); (3) quality and safety activities (e.g., how do you identify cases, how often do you discuss cases of diagnostic error at venues such as Morbidity & Mortality sessions or peer reviews?); (4) training and education (e.g., what are you doing to reinforce clinician training around diagnostic errors including using strategies such as simulation, cognitive training, and better teamwork?); and finally (5) patient engagement (e.g., how do you encourage patient/families to provide feedback or voice concerns?). Lastly, all participants were asked to make recommendations on how to promote correct and timely diagnosis and move towards high reliability in this area.

Data Collection and Analysis

Individual interviews were conducted by telephone between August 2019 and January 2020. One author (US), a qualitative researcher, conducted all the interviews. The average length of interview was 40 minutes (range 35–45 minutes). All interviews were audio-recorded and transcribed. We conducted an inductive qualitative content analysis allowing codes and categories to emerge from the data and new insights to be included as they arise. The interviewer (US) conducted the initial data analysis by reading transcripts repeatedly. Codes were primarily derived by highlighting the exact words from the text that appeared to capture key thoughts or concepts and a codebook was drafted. This process continued for each transcript, until no more new or meaningful concepts emerged.

After reviewing and revising the initial coding scheme with the study team, a second coder (UM) with expertise in quality and health informatics independently coded all interviews to ensure all information is captured. The two reviewers merged data and reviewed all codes to resolve any discrepancies. The team then met to refine the codebook and group codes with similar meanings into higher order categories representing main barriers to diagnostic safety that the HCO faces. The final codebook and categories were shared with the research team for discussion, and all identified risk areas were reviewed with the entire research team to resolve any questions or discrepancies.

RESULTS

We analyzed interviews of 32 participants including who primarily identified themselves as chief quality and patient safety officer/managers (n=14), facility leadership (n=13), and frontline clinicians (n=5) (Table 1). Qualitative analysis revealed four main barriers to diagnostic safety that HCOs face related to (1) Diagnostic safety culture; (2) Infrastructure for measurement, monitoring, and improvement activities; (3) Leadership investment; and (4) Dedicated team structure to work on diagnostic safety. Participant suggestions for improvement are included in each section. In addition, all participants included patient engagement in diagnostic safety activities as an overarching foundation of this work and provided examples of how their organization was trying to meet this goal. Representative quotes for each theme are presented in Table 2.

  1. (1)

    Diagnostic safety culture

Table 2 Participant Quotes by Theme
Full size table

More than half the participants (n=18) mentioned serious concerns about psychological safety associated with diagnostic error. For instance, the belief that diagnosis is linked with a clinician’s identity—perhaps seen as a failure—made open discussion uncomfortable. Additionally, some participants noted the negative influence of work-related stress on thinking and performance related to diagnosis. This included mounting time pressure and escalating cognitive load that affect diagnostic decision-making.

Participants recommended ways to improve diagnostic safety culture, specifically methods to reduce blaming clinicians. Some suggestions included replacing the word ‘error’ with ‘diagnostic learning’ or ‘learning opportunity’; freely sharing and discussing serious safety events among clinicians and more specifically, how it can lead to improvement; promoting a non-judgmental/non-punitive safety culture by coaching clinicians and fixing problems they or others identify, rather than blaming individual clinicians; and lastly, development of clinician safety champions in department/units to spread a peer-to-peer reminder of high-reliability safety practice in everyday operations.

  1. (2)

    Infrastructure for measurement, monitoring, and improvement activities

Many of the participants (n=25) were struggling on how to measure and monitor diagnostic errors. Their challenges included not having a pragmatic and concise definition of diagnostic error; diversity of settings and situations in which diagnostic error can occur; lack of effective tools to report errors; limited approaches to address cognitive biases; and how to measure and benchmark diagnostic error. Some participants indicated the lack of infrastructure for support staff to collect the data necessary to identify diagnostic errors. They were concerned that even after much research on diagnostic error, the definition remains hard to operationalize and translate to clinical care.

Almost all participants had suggestions to move forward with measuring and monitoring diagnostic errors. For instance, developing a more pragmatic and concise definition of diagnostic error, adapting trigger tools that can identify safety events or near-misses, moving towards another electronic health record (EHR) with better data capture, increasing clinician awareness about cognitive vulnerabilities, developing a safety event classification system (that includes diagnostic error), evaluating EHR functionality for monitoring, structured tools, or processes to think through differential diagnosis, designating diagnostic time-outs in certain risky situations, and lastly, investing in advanced techniques such as artificial intelligence (AI) and machine learning.

Additionally, participants were asked to discuss the role of incident reporting for diagnostic error. Some (n=10) expressed dissatisfaction with low rates of reporting indicating that while most diagnostic errors are identified through incident reporting, they only represent a small proportion of actual diagnostic errors. Some suggested (n=5) that clinicians do not think or do not know they have made an error. It becomes the responsibility of a colleague or staff members who observed an event to decide to report. Moreover, organizations that are diligently collecting safety issues may not be specifically collecting data on diagnostic errors.

Eight participants indicated that they were pleased with their current system of diagnostic event reporting. Many of them had developed specific diagnostic error reporting mechanisms built into the existing workflow: smart phone–based incident reporting system, hotline, and text messaging. These ‘dedicated’ reporting systems also prioritized feedback to facilitate action and learning: a direct feedback system with corrective measures, recognition for reporting, system-wide communication and education about safety, and daily huddles.

Another barrier mentioned was the absence of a clearly laid process on how to address cases of diagnostic error. Participants (n=13) expressed their dissatisfaction over the lack of organizational direction or guidance on how to manage cases of diagnostic errors after they occur. In the absence of formal processes or guidelines to deal with cases of error, participants suggested the need for clinician champions to advocate change.

Some noted there is organizational awareness for the need to create formal processes to address diagnostic error. Organizations are lacking a strategic plan to tackle diagnostic error or if such a plan is present, it is occurring at the front lines of care and poorly coordinated across multiple settings within the institution. Participants indicated that current systems tend to be reactive. However, some participants (n=6) were pleased with their follow-up process for cases of error. Some organizations had invested in this infrastructure and had a robust process and dedicated team in place to address errors.

During the interview, participants also described their concerns on limited sharing of findings and feedback after incident reporting within their organization (n=13). Some attributed this to risk management policies prioritizing confidentiality, some held their senior leadership responsible, and some suggested it was system inefficiency. Additionally, confidentiality requirements can impact feedback, making it hard for clinicians to learn from mistakes or identify appropriate improvement measures.

There were few (n=9) who were satisfied and supported their organizational efforts of sharing findings from incident report throughout their organization. These efforts included a direct feedback system (e.g., outlining what corrective measures were taken, as a result of what clinical staff initially reported); emails sent to all providers that identify learning opportunities based on an incident; and encouraging Morbidity and Mortality conference attendees to disperse information back to their departments.

  1. (3)

    Leadership investment

Participants (n=9, 5 in leadership positions) acknowledged that leadership involvement, commitment, and engagement were essential to influence diagnostic safety. They noted challenges in securing resources for diagnostic safety initiatives and personnel. While they expressed an awareness of the value of leadership engagement in achieving diagnostic safety, they also conveyed concerns about next steps. Participants believed that leaders are aware and understand the importance of diagnostic safety but have not prioritized it into actionable items. Further, they indicated that if leadership (i.e., C-suite, Executive Board) attention is focused solely on publicly reported patient safety issues and measures or other issues of interest (e.g., readmissions or opioid crisis), diagnostic safety will lag. Of note, some participants (n=4, 1 in leadership) indicated they do have highly motivated and engaged leadership, with robust improvement capabilities, and safety culture.

  1. (4)

    Diagnostic safety teams that work on diagnostic safety

Lacking a diagnostic error–focused team at the organization level was discussed in 18 interviews. Reasons included cultural barriers, lack of resources and organizational commitment, need for champions/leader, unclear priorities, and general lack of awareness about diagnostic errors at the system level. Additionally, coordination is challenging as providers, department leads, and other safety professionals sometimes function in isolation and when they do begin to engage as a team, new issues emerge such as lack of clarity in roles and responsibility. Few participants with a team stated it was informal.

Participants suggested building institutional capacity, specifically a dedicated team at the institutional level.

Patient Engagement

Engaging patient and their families to voice their concerns about suboptimal care was documented across all respondents. Almost all the participants indicated having a program to engage patient and families for capturing information on adverse events and near misses (e.g., patient complaint process, patient advocates). Many respondents (n=16) mentioned additional engagement activities, such as having a dedicated patient initiative for improving test results follow-up; having patients on key HCO committees and root cause analysis teams; having a Patient and Family Advisory Council (PFAC) system; adopting methods to increase patient access to EHR notes; and train-the-trainer program.

DISCUSSION

Safety leaders across 30 health care organizations in the USA identified several barriers and facilitators to improving diagnostic safety. Four major themes emerged as barriers—lack of a psychologically safe organizational culture for diagnostic safety; underdeveloped infrastructure for measurement, monitoring, and improvement activities; absence of leadership investment in diagnostic safety; and lack of dedicated teams at the institutional level that focus on diagnostic safety. Several corresponding suggestions were provided to overcome these barriers. These findings align with several core elements of the learning health system: culture, science and informatics, incentives, and engaged patients.13 These four elements provide a robust foundation that HCOs can operationalize and integrate in their efforts to develop into organizations that learn and explore diagnostic excellence (recently described as ‘LEDE organizations’).14

Within HCOs, frontline clinicians are in the best position to identify issues they face in making a diagnosis. Several initiatives to encourage safety reporting have been implemented over the last two decades but few have focused on diagnosis.15,16,17,18,19,20 Clinicians still fear punitive action rather than perceiving reporting as opportunities for learning and improvement21 and a lack of psychological safety results in under-reporting. Making a correct and timely diagnosis is considered a core component of patient care and getting it wrong or delayed may make clinicians uncomfortable because it may invite judgements about their competence. This is somewhat different from safety issues where diagnosticians are not the central focus of discussion such as in safety issues related to line infections, falls, pressure injuries, and medications. While blame culture is a common concern in patient safety, there is data to suggest there is particular sensitivity around the diagnostic process.22, 23 Prior work also suggests that clinicians fear judgement about their knowledge base and are uncomfortable in discussing diagnostic errors in both public and private settings, more so when compared with other types of medical errors.23 Because of these clinicians’ concerns, it is not surprising that we found respondents suggested reframing discussions around diagnostic errors as learning opportunities. However, this also brings up a need to balance these discussions with accountability and disclosure to patients and families who have experienced harm. Additional work will be needed to strike the right balance to ease the tension in these requirements and help advance the implementation of diagnostic safety concepts and tools into practice. Our study identifies that additional work to engage clinicians will be needed to promote diagnostic safety culture, one that supports and encourages clinicians to freely report, learn, and make corrective changes. Recently, Agency for Healthcare Research and Quality (AHRQ) in the USA released supplement items to the core medical office safety culture surveys to assess how medical offices support the diagnostic process, accurate diagnoses, and communication around diagnoses.24 Similar surveys could be developed and used for other care settings as an initial step in measuring diagnostic safety culture and improving it.

Several strategies to promote science and informatics components of LHS are ready for implementation starting with measurement for discovery, analysis, and learning. While currently there are no good measures related to diagnostic safety9, 25, 26, HCOs can identify and analyze diagnostic errors and implement actions to reduce harm based on findings. Recent guidance provides pragmatic recommendations for HCOs to start using measurement for learning and improvement of diagnostic safety using data sources currently available to them.27 For example, clinically rich EHR data can be harvested by using electronic algorithms to identify patients with actual or potential diagnostic harm.28,29,30 However, for moving such algorithms into practice, additional research and development will be needed. HCOs will need to be willing to test and implement these algorithms and integrate them into their existing quality and safety infrastructure.31 Because few HCOs currently conduct robust patient safety–related EHR data analytics, novel forms of embedded research partnerships32 will be needed to accelerate measurement.

Our findings suggest most diagnostic safety work is being conducted at a grass roots level and is poorly coordinated within and across institutions. Publicly reported patient safety issues and measures for payment drive leadership interest. Currently, there are no incentives to promote diagnostic safety as compared to certain other quality and safety targets. We found that leadership incentives must be aligned for generating value and transparency related to diagnostic safety. Diagnostic safety could be prioritized by payers not just to prevent harm, but also because diagnostic errors lead to increased costs and utilization. There is growing interest in measuring diagnostic excellence that may incentivize leadership investment. For example, the Leapfrog Group is developing a survey that aims to help hospitals assess and implement certain best practices to achieve diagnostic excellence.33 Additional incentives could be created by accrediting bodies who aim to protect consumer interests by maintaining high safety standards. Several proposed payment innovations can inform policy in this area.34

Patient and family engagement is a key component of diagnostic safety.14 While all sites in our study indicated that there are opportunities for patients to report concerns, it is unclear how that data is integrated into existing HCOs’ data streams to ensure system-wide learning. To our knowledge, there are no standard patient-reported outcome measures to identify diagnostic safety issues making it difficult to know which data is most actionable. Nevertheless, this is an area of emerging interest.35,36,37 Existing reporting methods, such as patient complaints, have shown promise, including for diagnostic safety,38,39,40,41 and additional methods to capture diagnosis related data from patients are being developed.

Our study has several limitations. Although our sample is national, it may not be representative, and all themes may not be generalizable to other health care organizations, especially smaller HCOs. Additionally, generalizability to non-US health care organizations may be limited because of differences in health system structure and incentives. However, several barriers we identified are quite universal (e.g., how to measure and monitor diagnostic errors,9, 42, 43 diagnostic safety teams). We also collected suggestions for improvements without consideration for effectiveness or feasibility, which would need to be assessed and contextualized. Finally, while several emergent factors related to diagnostic safety culture are known salient patient safety issues, diagnostic errors are particularly vulnerable to clinician fear and discomfort due to possible blame and judgement-related issues.22 Additional qualitative and quantitative exploration is warranted.

CONCLUSIONS

We identified specific barriers for diagnostic safety and strategies to advance improvement efforts at an organizational level. Despite modern patient safety systems in place, a focus on diagnostic safety is not common. Diagnostic errors are more difficult to assess and address even in organizations that have established patient safety systems. Most HCOs do not have processes in place to identify and/or analyzing diagnostic errors in clinical practice. Because the diagnostic process is complex and involves clinician’s knowledge and its application, additional considerations are necessary beyond traditional safety (e.g., medication errors, hospital-acquired infections, falls, surgical safety issues). Findings provide several strategies for development of a learning health system for diagnostic safety at an organizational level including culture that engages clinicians in diagnostic safety, science and informatics focused on using measurement for learning and improvement of diagnostic safety, incentives for diagnostic excellence initiatives, and engaged patients through opportunities to report diagnostic concerns and involvement in diagnostic safety. Implementation of these strategies can bolster development of organizations that learn and explore diagnostic excellence.