Evaluating handoffs in the context of a communication framework

doi:10.1016/j.surg.2016.09.003

Surgery

Volume 161, Issue 3, March 2017, Pages 861-868

Presented at the Academic Surgical Congress, Jacksonville, FL, February 2–4, 2016.

https://doi.org/10.1016/j.surg.2016.09.003 Get rights and content

Background

The implementation of mandated restrictions in resident duty hours has led to increased handoffs for patient care and thus more opportunities for errors during transitions of care. Much of the current handoff literature is empiric, with experts recommending the study of handoffs within an established framework.

Methods

A prospective, single-institution study was conducted evaluating the process of handoffs for the care of surgical patients in the context of a published communication framework. Evaluation tools for the source, receiver, and observer were developed to identify factors impacting the handoff process, and inter-rater correlations were assessed. Data analysis was generated with Pearson/Spearman correlations and multivariate linear regressions. Rater consistency was assessed with intraclass correlations.

Results

A total of 126 handoffs were observed. Evaluations were completed by 1 observer (N = 126), 2 observers (N = 23), 2 receivers (N = 39), 1 receiver (N = 82), and 1 source (N = 78). An average (±standard deviation) service handoff included 9.2 (±4.6) patients, lasted 9.1 (±5.4) minutes, and had 4.7 (±3.4) distractions recorded by the observer. The source and receiver(s) recognized distractions in >67% of handoffs, with the most common internal and external distractions being fatigue (60% of handoffs) and extraneous staff entering/exiting the room (31%), respectively. Teams with more patients spent less time per individual patient handoff (r = −0.298; P = .001). Statistically significant intraclass correlations (P ≤ .05) were moderate between observers (r ≥ 0.4) but not receivers (r < 0.4). Intraclass correlation values between different types of raters were inconsistent (P > .05). The quality of the handoff process was affected negatively by presence of active electronic devices (β = −0.565; P = .005), number of teaching discussions (β = −0.417; P = .048), and a sense of hierarchy between source and receiver (β = −0.309; P = .002).

Conclusion

Studying the handoff process within an established framework highlights factors that impair communication. Internal and external distractions are common during handoffs and along with the working relationship between the source and receiver impact the quality of the handoff process. This information allows further study and targeted interventions of the handoff process to improve overall effectiveness and patient safety of the handoff.

Section snippets

Study population and setting

A prospective, single-institution study was conducted to evaluate the process of handoff of surgical patients at a tertiary care teaching hospital. The conceptual framework published previously for handoffs using communication theory was used to develop evaluation tools for the source (resident giving the handoff), receiver (resident receiving the handoff), and observer.⁴

The observers in this study were involved in the development of the evaluation tools, and consensus was achieved through an

Results

During a 6-month period, 126 handoffs were observed by ≥1 trained observer; 23 handoffs included 2 observers. An evaluation form was completed by the source in 78 handoffs and by a receiver in 82 handoffs. Two receivers completed the evaluation form in 39 handoffs (Table I). The majority of handoffs observed in this study were part of the night-float system (92%). Seven percent of handoffs were completed over the phone, with the evaluation forms completed immediately and returned to the authors.

Discussion

Patient handoffs have become an integral part of patient care. The importance of this communication process has become evident in light of the inception of duty-hour restrictions in 2003; in one study, handoffs increased by 40%.² Intuitively, handoffs in patient care present a risk of loss of vital information and, consequently, the potential for adverse patient events. A large body of literature identifies flaws in patient handoffs, with attempts to improve this process,⁶ but the majority of

References (14)

M. Mohorek et al.
Establishing a conceptual framework for handoffs using communication theory
J Surg Educ
(2015)
N.M. Tapia et al.
Assessment and standardization of resident handoff practices: PACT project
J Surg Res
(2013)
J. Abraham et al.
A systematic review of the literature on the evaluation of handoff tools: implications for research and practice
J Am Med Inform Assoc
(2014)
A.R. Vidyarthi et al.
Managing discontinuity in academic medical centers: strategies for a safe and effective resident sign-out
J Hosp Med
(2006)
B.T. Kitch et al.
Handoffs causing patient harm: a survey of medical and surgical house staff
Jt Comm J Qual Patient Saf
(2008)
L.A. Riesenberg
Shift-to-shift handoff research: where do we go from here?
J Grad Med Educ
(2012)
S. Foster et al.
The effects of patient handoff characteristics on subsequent care: a systematic review and areas for future research
Acad Med
(2012)

There are more references available in the full text version of this article.

Cited by (7)

Integrating machine learning predictions for perioperative risk management: Towards an empirical design of a flexible-standardized risk assessment tool
2023, Journal of Biomedical Informatics
Citation Excerpt :
For example, in situations like OR-ICU handoffs, clinicians experience cognitive fatigue and strain from time pressure, competing demands, and incoherent case information [45,46], sometimes relying on heuristic decision-making [47]. Heuristic biases position clinicians poorly to detect risks, anticipate patient needs, plan for contingencies, and intervene quickly [48,49]. An ML-augmented tool could support more accurate decision-making and improve care by automatically presenting patient-specific risk estimates and identifying underlying risk factors.
Surgical patients are complex, vulnerable, and prone to postoperative complications that can potentially be mitigated with quality perioperative risk assessment and management. Several institutions have incorporated machine learning (ML) into their patient care to improve awareness and support clinician decision-making along the perioperative spectrum. Recent research suggests that ML risk prediction can support perioperative patient risk monitoring and management across several situations, including the operating room (OR) to intensive care unit (ICU) handoffs.
Our study objectives were threefold: (1) evaluate whether ML-generated postoperative predictions are concordant with clinician-generated risk rankings for acute kidney injury, delirium, pneumonia, deep vein thrombosis, and pulmonary embolism, and establish their associated risk factors; (2) ascertain clinician end-user suggestions to improve adoption of ML-generated risks and their integration into the perioperative workflow; and (3) develop a user-friendly visualization format for a tool to display ML-generated risks and risk factors to support postoperative care planning, for example, within the context of OR-ICU handoffs.
Graphical user interfaces for postoperative risk prediction models were assessed for end-user usability through cognitive walkthroughs and interviews with anesthesiologists, surgeons, certified registered nurse anesthetists, registered nurses, and critical care physicians. Thematic analysis relying on an explanation design framework was used to identify feedback and suggestions for improvement.
17 clinicians participated in the evaluation. ML estimates of complication risks aligned with clinicians' independent rankings, and related displays were perceived as valuable for decision-making and care planning for postoperative care. During OR-ICU handoffs, the tool could speed up report preparation and remind clinicians to address patient-specific complications, thus providing more tailored care information. Suggestions for improvement centered on electronic tool delivery; methods to build trust in ML models; modifiable risks and risk mitigation strategies; and additional patient information based on individual preferences (e.g., surgical procedure).
ML estimates of postoperative complication risks can provide anticipatory guidance, potentially increasing the efficiency of care planning. We have offered an ML visualization framework for designing future ML-augmented tools and anticipate the development of tools that recommend specific actions to the user based on ML model output.
Statement of Significance.
- •
  Problem: Implementation of machine-learning (ML) output within clinical workflows is not trivial.
- •
  What is already known: Barriers to ML implementation and appropriate use within clinical workflows include clinicians’ limited understanding of ML, wariness of ML output, and lack of ML output actionability.
- •
  What this paper adds: Qualitative end-user evaluation of postoperative risk predictions found high-levels of agreement between rankings of clinicians' manual risk rankings and ML-augmented risk estimates. The evaluation also highlighted design (functional and visual) enhancements for translational implementation of ML models within clinical workflows.
Dissecting communication barriers in healthcare: A path to enhancing communication resiliency, reliability, and patient safety
2021, Journal of Patient Safety
The Art of Effective Handoff Communication Among Medical and Surgery Residents
2021, Journal of Cognitive Engineering and Decision Making
Noise and turn-taking impact postanesthesia care unit handoff efficiency
2020, Journal of Patient Safety and Risk Management
The handover room: a qualitative enquiry into the experience of morning clinical handover for acute medical teams
2019, Internal Medicine Journal
Who’s my doctor? Team-based management
2018, Controversies in Severe Traumatic Brain Injury Management

View all citing articles on Scopus

: The authors report no conflicts of interest or financial disclosures.

View full text

Presented at the Academic Surgical Congress 2016Evaluating handoffs in the context of a communication framework

Background

Methods

Results

Conclusion

Section snippets

Study population and setting

Results

Discussion

J Surg Educ

J Surg Res

A systematic review of the literature on the evaluation of handoff tools: implications for research and practice

J Am Med Inform Assoc

Managing discontinuity in academic medical centers: strategies for a safe and effective resident sign-out

J Hosp Med

Handoffs causing patient harm: a survey of medical and surgical house staff

Jt Comm J Qual Patient Saf

Shift-to-shift handoff research: where do we go from here?

J Grad Med Educ

The effects of patient handoff characteristics on subsequent care: a systematic review and areas for future research

Acad Med

Presented at the Academic Surgical Congress 2016
Evaluating handoffs in the context of a communication framework