Compliance with self-reporting quality assurance database entry in a tertiary academic anesthesiology practice

Background. Quality assurance (QA) programs are critical to ensuring patient safety in current anesthesiology practices. While challenging to adapt to daily practice, rates of case entry into other Anesthesiology quality assurance databases have ranged from 48% to 85.1%. The goal of this study was to investigate entry rates at our institution’s voluntary QA database and uncover patterns of reporting. Methods. From 2013 to 2017, a total of 61,492 surgeries were analyzed to determine the rates of voluntary reporting of all cases (with or without a reportable incident) in our QA database by comparing the number of total procedures billed in that time to the total number of cases input into the QA system. Results. The overall compliance rate of entry of cases into the QA database during the 5 year period analyzed was 65.51%. Cases in the Main OR had a higher 5 year rate of entry compared to cases in the Same Day Surgery center. Interestingly, during the 5 year period, July has consistently had the highest rate of case entry (72.54%) into the QA Database, perhaps due to the start of the new academic year for residents. More importantly, there was a decreasing trend of entry into the QA database over the 5 year period of study. From an initial rate of 84.37% in 2013, compliance rates have gradually declined to 78.88% in 2014, 67.30% in 2015, 54.18% in 2016, and 45.16% in 2017. Conclusion. The inherent disadvantages of a voluntary reporting system combined with the growing requirement for quality improvement measures in today’s healthcare system reflect the necessity for particular interventions at our institution aimed at increasing QA compliance rates.


Background
Quality assurance (QA) programs are essential to ensure a high standard of anesthetic care centered on patient safety during the peri-operative period [1]. QA programs vary in their general methodology, ranging from voluntary incident reporting to automatic reporting imbedded in electronic health record systems.
Although challenging to adapt for daily practice, quality assurance programs in anesthesiology generally improve patient post-operative outcomes since their implementation by the Joint Commission on Accreditation of Hospitals (JCAH) in 1979, [2]. After implementation of a voluntary electronic anesthesia incident reporting system, Haller et. al reported a compliance rate ranging from 83 to 87% over a 5 year period [3]. Furthermore, Vigoda et al. demonstrated an increase in the compliance rate of their institutional anesthesia QA database from 48-94% after the implementation of numerous interventions, such as education, workflow integration and individual feedback [4]. Our primary objective of interest was to investigate QA database compliance rates at our home institution to uncover patterns of reporting into the database that may be the future target of interventions aimed at increasing overall compliance.

Methods
The Rutgers New Jersey Medical School (NJMS) Anesthesiology Quality Assurance database (present on hospital intranet with Microsoft Excel backend) along with University Hospital's primary electronic health record system (EPIC) were the two sources of data for this study. The Anesthesiology QA Database of NJMS collects user-input, anesthesia-related information on all cases in which our providers are involved. This information includes: patient identifiers, personnel involved, diagnosis, procedure, surgical location, ASA, start/end times, special procedures, as well as adverse quality incidents such as reintubation, dental injury, etc. (20,483). The overall 5 year compliance rate of entry into the QA database was 65.51% with a rate of 65.25% in the Main OR and 66.04% in the Amb Ctr. (Table 1).
Cases in the Main OR had a higher rate of being logged into the QA database from 2013 to 2015 with monthly rates of compliance ranging from 67 to 85% compared to Amb Ctr. entry rates in the same time period (66 to 81%). However in 2016 and 2017, a reversal in this trend was witnessed as more entries were occurring in the DOC/Amb Ctr. (monthly rates varying from 48 to 57%) compared to the Main OR (43 to 52%) (Fig. 1).
Moreover, there is a decreasing trend of monthly entry rates into the QA database 5 over the 5 year period (Fig. 2). From an initial recording rate of 84.37% in 2013, the rates gradually declined to 78.88% in 2014, 67.30% in 2015, 54.18% in 2016, and 45.16% in 2017.
During the 5-year period, July has the highest monthly rate of case entry into the QA Database at 72.54% while December and June have the lowest rates of entry at 60.5% and 62.09% (Fig. 3).

Discussion
Our study aimed to analyze QA database entry rates at an institution that uses a voluntary electronic reporting structure. We found that the average rate of entry into the QA database over a five-year period was 65.51% (Table 1), a compliance level that is comparable to rates reported in the literature [3][4][5][6]. Although the primary ambulatory surgical center (Amb Ctr.) had different QA completion rates, because of the variation, a clear conclusion cannot be drawn upon whether location of anesthetic procedures affects compliance with database entry. The main Operating Room initially had a higher rate of compliance from 2013 to 2105 and later, the same day surgery center at our institution saw higher rates of QA entry after completion of the cases from 2016 to 2017 with no clear explanation.
A seasonal trend was apparent on a month-to-month basis during the 5 year period.
The month of July had the highest rate of entry into the QA database with a 5 year average of 72.54%. Moreover, the greatest level of reporting for any month in the 5 year period was in July of 2014, with a rate of entry of 91%, at the beginning of the study period. This higher rate of reporting in July coincides with the start of the academic calendar for new anesthesiology residents, which may explain the increased rate of compliance. Residents may also be inclined to record cases more 6 frequently in the QA database in July due to increased education on departmental policies regarding the QA database during this time period, especially during new resident orientation. Fear of medicolegal consequences is generally highest at the beginning of the academic year and has also been said to be a factor that influences QA reporting [6,7]. Furthermore, compliance rates were lowest at the end of the year in December (60.5%) and in June (62.09%), reflecting a decreased inclination to report into the QA database near the holiday season and at the end of the academic year. These findings may elucidate a pattern in anesthesiology resident behaviors specific to QA database entry throughout the academic year.
Over the 5-year period analyzed, rates of entry into the QA database steadily declined from an initial rate of 84.37% in 2013 to 45.16% in 2017. We suspect that this finding is due to a lack of formalized resident and attending education about the availability and importance of entering cases into the QA database. Hall et al. found that deficient in-service education is a major factor in the incidence of reporting adverse outcomes amongst healthcare providers [8]. Furthermore, it has been previously demonstrated by Vigoda et al. that increased education in the form of formal meetings regarding QA documentation departmental policy coupled with regular announcements at grand round meetings increased QA compliance rates from 48-55%. Moreover, when these investigators added an individualized feedback intervention to inform providers of their case load and compliance rates for the month, compliance rates into the database increased to 78% [4]. Regular feedback with providers to use the voluntary QA incident form was also shown by Haller et al.
to be a major contributing factor in achieving their 4 year compliance rates of 85.1% [3]. These other studies shed light on possible interventions that may be effective in increasing QA compliance rates at our institution.
A variety of anesthesiology workflow structures exist to capture quality assurance metrics. While some institutions implement a traditional paper-based Quality Assurance form [7], others have adopted electronic forms that are either voluntary [3] or mandatory. In voluntary reporting systems, providers or externalreviewers can choose whether to report data about an adverse event in the operating room. However, the inconsistency in reporting from such a system leads to compliance rates that are low and ultimately results in ineffective quality improvement interventions [9,10]. Given the inherent flaws in voluntary reporting, Peterfreund et al. customized their Anesthesia information management systems (AIMS) workflow to mandate the completion of an electronic QA form before the endtime of an anesthesia case could be recorded [5]. In reviewing 55,382 cases over the course of a year after implementation of a mandatory QA recording system, the authors found a sustained increase in the number of adverse events that were reported.
The third approach to capture quality assurance information is through active surveillance of anesthesia related data through an institution's Anesthesia Information Management System (AIMS). This structure implements the automatic detection of abnormal patient vital signs using pre-determined criteria built into the AIMS. Automatic reporting of incidents within anesthesiology boasts obvious benefits of increased time for providers and an error-free, unbiased identification of adverse events that can be investigated in quality improvement interventions [11,12]. However, factors such as high perceived costs and low perceived need compared to the available monitoring system have limited the wide-spread implementation of automatic capturing systems [1]. 8

Conclusion
Despite the availability of a quality assurance reporting form at our institution that implements a template of pre-defined categories that have been shown to improve compliance rates [11,3], we found a decreasing trend in QA completion rates over a 5-year period, without ongoing education or feedback mechanism to maintain compliance. This report highlights the necessity of particular interventions, whether in the form of greater education or through a shift towards a mandatory QA capturing system to increase our compliance rate, or implementation of an automated entry system via AIMS, finances permitting.

Declarations
Ethics approval and consent to participate: IRB approval provided by Rutgers University prior to data collection. No patient identifiers used in any collection or analysis.

Consent for Publication: Not applicable
Availability of data and material: All data generated or analyzed during this study are included in this published article [and its supplementary information files] Competing interests: GT and NB declare no conflict of interest.

Funding
No outside funding for this project.

Supplementary Files
This is a list of supplementary files associated with the primary manuscript. Click to download.