Patient Safety Narratives as a Key to Promoting Quality Measurement

The Institute of Medicine’s (IOM) report, To Err is Human, has recommended the nationwide patient safety event reporting as a key strategy in reforming a safe healthcare system [1]. The goal of the reporting is to encouraging healthcare organizations to learn from retrospective incidents and to reduce recurring errors by mitigating contributing factors. In this light, the reporting requires front-line clinicians to report events that are considered unsafe conditions, near misses, and patient harms. These data are collected, aggregated, stored, and retrieved through patient safety event reporting systems housed in hospitals. A team of domain experts are engaged in investigating the key factors, e.g., error types and contributing factors, by reviewing the events.


Introduction
The Institute of Medicine's (IOM) report, To Err is Human, has recommended the nationwide patient safety event reporting as a key strategy in reforming a safe healthcare system [1]. The goal of the reporting is to encouraging healthcare organizations to learn from retrospective incidents and to reduce recurring errors by mitigating contributing factors. In this light, the reporting requires front-line clinicians to report events that are considered unsafe conditions, near misses, and patient harms. These data are collected, aggregated, stored, and retrieved through patient safety event reporting systems housed in hospitals. A team of domain experts are engaged in investigating the key factors, e.g., error types and contributing factors, by reviewing the events.
With the passage of time, increasing volume of reports have been generated and collected. Nevertheless, little evidence has linked reporting to improved patient safety, suggesting a number of challenges and possible defects of the existing reporting systems [2]. Many of the identified problems are associated with the limited data processing capacity, i.e., timeliness of analysis [3], data quality [4], and prioritization [5], considering the overloaded reports. There was a consensus that the current development of reporting calls for a shift from quantity to quality measurement.
Drawing on the narrative perspective of the reports, this paper discusses how the critical clinical information is constructed in the narrative portion of the reports. The paper explores further the current state of informatics methods in processing narrative data. In the end, the paper provides discussion on key challenges and incentives in studying patient safety narratives toward advancing quality measurement.

Construction of patient safety knowledge
In seeking to disclose root causes and many valuable aspects of patient safety events, information is reported in a uniformed format through patient safety event reporting systems. The Agency for Healthcare Research and Quality (AHRQ) developed the Common Definitions and Reporting Formats (a.k.a., the Common Formats) to collect data in a structured format [6]. The collected information will be published in the annual issues of the National Health Quality and Disparities Reports. The Common Formats provide unique advantages in defining incidents into uniformed classifications or taxonomies for aggregate analysis. However, a number of limitations are reported in recent studies. Data consistency is concerned as a major problem that undermines the quality of the reports. In Figure 1, I replicated the major findings from Gong, which demonstrates the discrepancy between mislabelled reports and the reports re-evaluated by domain experts [4]. The re-evaluation procedures identified 12% of 'patient fall' events from events that were originally labelled as 'miscellaneous', and in 'patient fall' category, 4% of 'un-witnessed' events that were labelled as 'others'. Other researchers reported poor to moderate inter-rater reliabilities by re-categorizing the events ( Figure 1) according to the harm scores defined in the Common Formats [7,8].
Rather than using a classification system to determine the definitions and taxonomies of events, I consider how safety-related information is constructed in the narrative portion of the reports. Due to the fact that a considerable amount of information may not be fully represented in the structured format, most reporting systems allow reporters to enter a description of the events in free text. These narratives are created by front-line clinicians, reflecting a dynamic mix of detailed clinical procedures, critical information about patients, and notions of responsibility during the course of care. The importance of narrative data is multifold. First, narrative data provide reporters flexibility with submitting valuable information that is not defined in structured data entry. The problem of data consistency may be mitigated if hospitals and Patient Safety Organizations (PSO) seek out more use of narrative data as a supplement of the structured data. Second, narratives help reconstruct a complete portrait of the care, especially in situations of missing and uncertain information. Third, narrative data provide insights into the system factors that cause the error. In a complex system, i.e., healthcare system, system failures are prevented from many relatively independent components. A component may be clinical administration, treatments, uses of medical equipment, etc., while each one may produce unintended weaknesses. A system failure occurs when by chance many components go wrong at the same time. This argument can be understood by the metaphor of 'Swiss cheese' (Figure 2) [9].
(3) Ensuring that events are assigned to proper teams of specialties and intervention actions.
In addition, recent studies have been focusing on the multilabelled classification of the reports, highlighting a unique advantage of using multi-label classification to disclose the system vulnerability of healthcare [12]. Specifically, a patient safety event may contain one or more error-related factors that also reside in some other reports. The co-occurrences and relations of these factors across reports help disclose multiple yet intrinsically related factors that contribute to the errors. Accordingly, understanding the multi-faceted nature of events becomes a key to disclose system vulnerabilities.

From quantity to quality measurement
Patient safety study calls for a shift from quantity to quality measurement in the way of maximizing the value of patient safety event reporting. At the hospital level, the ever-increasing volume of reports, along with the wide variation of data quality, is becoming a challenge for conventional root cause analysis. Consequential problems such as the lack of timely responses, the defected prioritizing mechanism, and little usefulness of received reports hamper the process of creating actionable knowledge to improve safety. At the PSO level, 'learning from lessons', as one specific aim of patient safety event reporting, is well recognized but not yet accomplished. In present, results of the aggregate analysis are mostly from a cross-sectional survey using structured data, which limits the knowledge to error distribution and trends. It is equally important to investigate system vulnerability and causal relations when advanced analytics is available.
in a story-telling fashion. Reports as such are easier to keep a record of clinically valuable information but create notable technical challenges for large-scale data processing, such as automatic classification of reports, knowledge extraction, etc. Additionally, these reports contain intricate domain knowledge and notable linguistic complexity, as they are produced by clinicians with diverse specialties and levels of experience.
Semantics is one linguistic component that makes the processing of patient safety narratives difficult. In the patient safety event reports, semantics concerns the meaning of terms, phrases, and sentences that are specifically determined by the unique context of reported events. As such, it is necessary to develop a domain specific language system, in which semantic classes, specifications, and senses are defined to distinguish real-world words, phrases, and sentences in the narrative data. Inspired from many existing controlled vocabularies of patient safety data, recent studies are centered on developing an ontological knowledge representation of patient safety that underpins aggregate data analysis and natural language processing (NLP) tasks using narrative data [10]. A mature language system can facilitate advanced analytics using the reports. Classification is recognized as the initial step of data analysis [11]. A comprehensive classification of the received reports helps to maximize patient safety outcomes by (1) Understanding a realistic distribution of week healthcare components; (2) Prioritizing safety efforts to frequent and recurring events;  The growing emphasis on quality over quantity measurement presents significant challenges and opportunities for informatics study. One potential line of study is to improve the data quality from the perspective of reporting systems. Data entry plays a key role in quality control because it is the very first module where undressed reports are accepted. The use of human-computer interaction design is demonstrated useful to improve the accuracy of user entered narratives [13]. The other potential line of study is to advance the development of informatics tools for the narratives. Informatics group has made some progress in building supportive language systems for analysing patient safety narratives. In the effort to make use of reports, much of the existing work lies in the automated classification of reports. It is also expected to demonstrate more progress on extracting valuable information from reports, i.e., co-occurrence probability of safetyrelated components and notions of responsibility shared by the reporters. Accordingly, actionable knowledge can be generated from the report analysis, enabling an evidence-based linkage between individual events and corresponding actions.

Conclusion
In the years following the development of patient safety event reporting systems, a shift from quantity to quality measurement of patient safety is proposed as a crucial incentive of study. To facilitate this shift, it is suggested to enhance the narrative portion of the reports through improved reporting systems and advanced informatics tools.