Original Article
Moderate efficiency of clinicians' predictions decreased for blurred clinical conditions and benefits from the use of BRASS index. A longitudinal study on geriatric patients' outcomes

https://doi.org/10.1016/j.jclinepi.2015.08.017Get rights and content

Abstract

Objectives

Accurate prognosis is an essential aspect of good clinical practice and efficient health services, particularly for chronic and disabling diseases, as in geriatric populations. This study aims to examine the accuracy of clinical prognostic predictions and to devise prediction models combining clinical variables and clinicians' prognosis for a geriatric patient sample.

Study Design and Setting

In a sample of 329 consecutive older patients admitted to 10 geriatric units, we evaluated the accuracy of clinicians' prognosis regarding three outcomes at discharge: global functioning, length of stay (LoS) in hospital, and destination at discharge (DD). A comprehensive set of sociodemographic, clinical, and treatment-related information were also collected.

Results

Moderate predictive performance was found for all three outcomes: area under receiver operating characteristic curve of 0.79 and 0.78 for functioning and LoS, respectively, and moderate concordance, Cohen's K = 0.45, between predicted and observed DD. Predictive models found the Blaylock Risk Assessment Screening Score together with clinicians' judgment relevant to improve predictions for all outcomes (absolute improvement in adjusted and pseudo-R2 up to 19%).

Conclusion

Although the clinicians' estimates were important factors in predicting global functioning, LoS, and DD, more research is needed regarding both methodological aspects and clinical measurements, to improve prognostic clinical indices.

Introduction

What is new?

Key findings

  1. Clinicians' prognostic accuracy on patient clinical outcomes at discharge decreases when patients' clinical conditions are not clearly defined.

  2. Overall, clinicians' estimates substantially contribute to prediction accuracy. Furthermore, the Blaylock Risk Assessment Screening Score Index also had a moderate predictive performance.

What this adds to what was known?
  1. Although older hospitalized patients are a substantial proportion of the clinical population, few studies have taken into account their clinical predictions, and to our knowledge, there are only few studies that have assessed clinical prediction in such a population for more than one outcome.

What is the implication and what should change now?
  1. Caution should be used in making prognostic predictions for older patients, especially when the patients' clinical picture is uncertain and the illness course is not linear. However, it is precisely under these conditions that clinicians' prognostic skills are essential to guide treatment options.

  2. Careful consideration of specific clinical information available at admission can help to improve the accuracy of clinical outcomes prediction.

  3. The use of multiple statistical indices and predictive models underlines the importance of delivering more good quality prognostic research, to understand which clinical indices are of prognostic value.

In clinical practice, prognostics are commonly overshadowed by diagnostics and therapeutics [1], [2]. Physicians are often poorly prepared and feel uncomfortable in making predictions, especially when managing chronic diseases [3], [4]. Nonetheless, what patients and family members most expect from doctors is often accurate prediction on the course and outcome of the disease. Correct estimation of prognosis is particularly important both for clinical decision making and for an appropriate choice of the best treatment options [5], [6], [7]. Furthermore, prognostic research entails the “big business” of predicting the future in health care [8], taking into account a combination of multiple prognostic factors by making care more effective thanks to more informed clinical decision making [9], [10], [11].

Several studies have focused on the accuracy of physicians' prognostics for specific clinical populations, such as cancer patients [12], [13], [14], [15] and intensive care unit patients [16], [17], revealing a substantial degree of inaccuracy in predicting patients' actual outcomes. Although older patients are currently increasing in all high-income countries and therefore represent a substantial proportion of clinical populations treated in hospital settings, only a few studies have taken into account their clinical predictions [18], [19]. The prompt and accurate prognostication of patients who require long-term care could improve measured efficiency and the effectiveness of hospital care. This in turn could lead to fewer unnecessary hospital stays or inappropriate early discharge as well as speeding transfer to nursing homes or to long-term care facilities [20], [21], [22]. This would improve resource planning and contribute to make patients' expectations more realistic concerning their length of stay (LoS) [23], [24], [25], [26] and destination at discharge (DD) [27], [28], [29].

To investigate prediction accuracy of hospitalization outcomes for older patients, we examined predictions gathered from the Perdove-Anziani study cohort [30]. This longitudinal prospective study comprehensively evaluated older patients admitted to 10 geriatric units in northern Italy. Our first aim was to evaluate how accurately and under what conditions doctors can predict clinical outcomes for older patients. In particular, we focused on the following outcomes: (1) the patients' global functioning level at discharge, (2) overall LoS, and (3) their DD. Second, we devised predictive models for each outcome, combining the clinicians' estimates and other objective factors, such as the facility where patients were hospitalized and clinical assessment instruments administered at admission, to address future predictions.

Section snippets

Participants and procedures

The Perdove-Anziani study evaluated 329 consecutive older patients (age >64 years) using a multidimensional approach and admitted to 10 geriatric units located in four Italian St. John of God Order facilities in Brescia, Turin, Venice, and near Milan. The study was approved by the Ethics Committee of the St. John of God facilities of north Italy. Each patient was asked to sign an informed consent form. (When medical records indicated incapacity to give informed consent, this was requested from

Characteristics of the study sample

There were 329 patients who met the study entry criteria and were assessed at T0. Table 1 shows the sociodemographic characteristics and outcome descriptive statistics of the sample.

Among the six identified categories for admission, the most common was “motor rehabilitation” (N = 110, 33.4%) and “treatment for behavioral disorders” (N = 93, 28.4%). Twenty-six patients were not included in the analysis because they belonged to a smaller and mixed “other” category.

At admission, the BI mean score

Discussion

We evaluated the predictive accuracy of clinical judgments in both the initial descriptive assessment and in a framework of predictive models incorporating the contribution of objective factors, including the type of hospital structure and assessment measurements available at admission.

Conclusion

Our results suggest that caution should be used in making prognostic predictions in elderly patients' facilities. When formulating a prognosis, clinicians should careful consider useful clinical information gathered at admission. This could improve predictive accuracy especially when patients' clinical picture is not very clear, and for this reason, the evolution of their condition over time might not be linear or obvious.

Taken together, these results suggest the importance that clinicians

Acknowledgments

There were no other financial contributions for this project. The authors are grateful to the research participants and clinicians for their valuable contribution in the study. The authors would like to thank Robert Coates, Medical writer and editor (Università Bocconi, Milan), for his linguistic editing.

References (56)

  • P. Glare

    Predicting and communicating prognosis in palliative care. Prognostic tools can help, but should not be applied blindly

    BMJ

    (2011)
  • N.A. Christakis et al.

    Attitude and self-reported practice regarding prognostication in a national sample of internists

    Arch Intern Med

    (1998)
  • H. Holman

    Chronic disease—the need for a new clinical education

    JAMA

    (2004)
  • K.G.M. Moons et al.

    Prognosis and prognostic research: what, why, and how?

    BMJ

    (2009)
  • K.G.M. Moons et al.

    Prognosis and prognostic research: application and impact of prognostic models in clinical practice

    BMJ

    (2009)
  • T.M. Gill

    The central role of prognosis in clinical decision making

    JAMA

    (2012)
  • G. Peat et al.

    Improving the transparency of prognosis research: the role of reporting, data sharing, registration, and protocols

    PLos Med

    (2014)
  • H. Hemingway et al.

    Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes

    BMJ

    (2013)
  • R.D. Riley et al.

    Prognosis Research Strategy (PROGRESS) 2: prognostic factor research

    PLos Med

    (2013)
  • E.W. Steyerberg et al.

    Prognosis Research Strategy (PROGRESS) 3: prognostic model research

    PLos Med

    (2013)
  • N.A. Christakis et al.

    Extent and determinants of error in doctors' prognoses in terminally ill patients: prospective cohort study

    BMJ

    (2000)
  • P. Glare et al.

    Systematic review of physicians' survival predictions in terminally ill cancer patients

    BMJ

    (2003)
  • K.M.Y.B. Leung et al.

    Challenging the 10-year rule: the accuracy of patient life expectancy predictions by physicians in relation to prostate cancer management

    Can Urol Assoc J

    (2012)
  • S. Frick et al.

    Medical futility: predicting outcome of intensive care unit patients by nurses and doctors—a prospective comparative study

    Crit Care Med

    (2003)
  • W. Meadow et al.

    Power and limitations of daily prognostications of death in the medical intensive care unit

    Crit Care Med

    (2011)
  • B.M. Buurman et al.

    Prognostication in acutely admitted older patients by nurses and physicians

    J Gen Intern Med

    (2008)
  • L.C. Yourman et al.

    Prognostic indices for older adults: a systematic review

    JAMA

    (2012)
  • D.H. Hickam et al.

    Clinicians' predictions of nursing home placement for hospitalized patients

    J Am Geriatr Soc

    (1991)
  • Cited by (6)

    • Clinical utility of scores on the Blaylock Risk Assessment Screen (BRASS): An analysis of administrative data

      2018, Applied Nursing Research
      Citation Excerpt :

      The BRASS has been described as easy-to use, requiring about 15 min to complete, with good inter-rater reliability (Blaylock & Carson, 1992; Chaboyer, Kendall, & Foster, 2002; Dal Molin et al., 2014). It has been translated from English into Dutch (Miastiaen, Duijnhouwer, Prins-Hoekstra, Ros, & Blaylock, 1999) and Italian (Dal Molin et al., 2014; Panella, La Porta, Caselli, Marchisio, & Tennant, 2012; Signorini et al., 2016). Scores for the BRASS are computed based on responses to 10-items addressing patients' age, functional status and mobility, cognition and behavior pattern, sensory abilities, use of healthcare services, number of comorbidities and medications, and social situation.

    • Qualitative experiential analysis of the BRASS scale

      2019, Recenti Progressi in Medicina

    Conflict of interest: This article contains original work that has not been previously published and that is not being submitted for publication elsewhere. All authors declare no conflict of interest.

    Funding: This project was entirely funded by free donations made by private individuals to the St. John of God Order.

    Contributors: For the Perdove-Anziani group: M.E. Boero, C. Geroldi, G. M. Giobbio, P. Maggi, A. L. Melegari, G. Sattin, M. Signorini, D. Volpe, O. Zanetti, E. Chitò, L. Iozzino, G. Kuffenschin, G. Lussignoli, A. Rossetti, M. Cavallaro, N. Cosentino, S. Dessı, A. Lamilia, T. Naldi, C. Nodari, A. Mancuso, D. Rigodanza, R. Loiero, C. Bertinetti, R. Romiti, P. Secreto, and S. Zamburlini.

    View full text