Skip to main content
Log in

Prognosis research and risk of bias

Internal and Emergency Medicine Aims and scope Submit manuscript

Abstract

The interest in prognosis research has been steadily growing during the past few decades because of its impact on clinical decision making. However, since the methodology of prognosis research is still incompletely defined, the quality of published prognosis studies is largely unsatisfactory. Seven major domain for risk of bias in prognosis research have been identified, including study participation, attrition, selection of candidate predictors, outcome definition, confounding factors, analysis, and interpretation of results. The methodology for performing prognostic studies is currently aimed at avoiding such potential biases. Amongst methodologic requirements in prognosis research, the following should be considered most relevant: beforehand publication of the study protocol including the full statistical plan; inclusion of patients at a similar point along the course of the disease; rationale and biological plausibility of candidate predictors; complete information; control of overfitting and underfitting; adequate data handling and analysis; publication of the original data. Validation and analysis of the impact that prediction models have on patient management, are key steps for translation of prognosis research into clinical practice. Finally, transparent reporting of prognostic studies is essential for assessing reliability, applicability and generalizability of study results, and recommendations are now available for this aim.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Hayden JA, Cote P, Bombardier C (2006) Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med 144:427–437

    Article  PubMed  Google Scholar 

  2. Kyzas PA, Denaxa-Kyza D, Ioannidis JPA (2007) Quality of reporting of cancer prognostic marker studies: association with reported prognostic effect. J Natl Cancer Inst 99:236–243

    Article  PubMed  Google Scholar 

  3. Hingorani DA, van der Windt DA, Riley RD, Moons KGM, Steyerberg EW, Schroter S et al (2013) Prognosis research strategy (PROGRESS) 4: stratified medicine research. BMJ 346:e5793

    Article  PubMed  PubMed Central  Google Scholar 

  4. Pugh RN, Murray-Lyon IM, Dawson JL, Pietrni MC, Williams R (1973) Transection of the esophagus for bleeding oesophageal varices. Br J Surg 60:646–649

    Article  CAS  PubMed  Google Scholar 

  5. Kamath PS, Wiesner RH, Malinchoc M, Kremers W, Therneau TM, Kosberg CL et al (2001) A model to predict survival in patients with endstage liver disease. Hepatology 33:464–470

    Article  CAS  PubMed  Google Scholar 

  6. D’Amico G, Garcia-Tsao G, Pagliaro L (2006) Natural history and prognostic factors in cirrhosis: a systematic review of 118 studies. J Hepatol 44:217–231

    Article  PubMed  Google Scholar 

  7. Moons KGM, Altman DG, Reitsma JB, Collins GS, Ioannidis PA, Macaskill P et al (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Int Med 162:W1–W73

    Article  PubMed  Google Scholar 

  8. Grimes DA, Schulz KF (2002) Cohort studies: marching towards outcomes. Lancet 359:341–345

    Article  PubMed  Google Scholar 

  9. Di Pietro NA (2010) Methods in epidemiology: observational study designs. Pharmacotherapy 30:973–984

    Article  Google Scholar 

  10. D’Amico G, Pasta L, Morabito A, D’Amico M, Caltagirone M, Malizia G et al (2014) Competing risks and prognostic stages in cirrhosis: a 25-year inception cohort study of 494 patients. Aliment Pharmacol Ther 39:1180–1193

    Article  PubMed  Google Scholar 

  11. D’Amico G, Morabito A, Pagliaro L, Marubini E, The liver study group of “V Cervello Hospital” (1986) Survival and prognostic indicators of compensated and decompensated cirrhosis. Dig Dis Sci 31:468–475

    Article  PubMed  Google Scholar 

  12. Ratib S, Fleming KM, Crooks JC, Aithal G, West J (2014) 1 and 5 year survival estimates for people with cirrhosis of the liver in England, 1998–2009: a large population study. J Hepatol 60:282–289

    Article  PubMed  Google Scholar 

  13. Donders AR, van der Heijden GJ, Stijnen T (2006) Review: a gentle introduction to imputation of missing data. J Clin Epidemiol 59:1087–1091

    Article  PubMed  Google Scholar 

  14. Royston P, Moons KG, Altman DG, Vergouwe Y (2009) Prognosis and prognostic research: developing a prognostic model. BMJ 338:b604

    Article  PubMed  Google Scholar 

  15. Harrell FE Jr (1985) Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep 69:1071–1077

    PubMed  Google Scholar 

  16. Concato J, Feinstein AR, Holford TR (1993) The risk of determining risk with multivariable models. Ann Intern Med 118:201–210

    Article  CAS  PubMed  Google Scholar 

  17. Moons KGM, Kengne AP, Woodward M, Royston P, Vergouwe Y, Altman GD, Grobbee DE (2012) Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart 98:683–690

    Article  PubMed  Google Scholar 

  18. Royston P, Altman DG, Sauerbrei W (2006) Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 25:127–141

    Article  PubMed  Google Scholar 

  19. Jepsen P, Wilstrup H, Andersen PK (2015) The clinical course of cirrhosis. The importance of multistate models and competing risks analysis. Hepatology 62:292–302

    Article  PubMed  Google Scholar 

  20. Pintile M (2006) Competing risks. A practical perspective. Wiley, Chichester

    Book  Google Scholar 

  21. Fine JP, Gray RJ (1999) A proportional hazards model for the subdistribution of competing risk. J Am Stat Ass 94:496–509

    Article  Google Scholar 

  22. Moons KGM, Royston P, Vergouwe Y, Grobbee DE, Altman DG (2009) Prognosis and prognostic research: what, why and how? BMJ 339:b375

    Article  Google Scholar 

  23. Harrel FE, Lee KL, Mark DB (1996) Tutorial in biostatistics. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361–387

    Article  Google Scholar 

  24. Justice AC, Covinsky KE, Berlin JA (1999) Assessing the generalizability of prognostic information. Ann Intern Med 130:515–524

    Article  CAS  PubMed  Google Scholar 

  25. Steyerberg WE, Vickers AJ, Cook N, Gerds T, Gonen M, Obuchowski N et al (2010) Assessing the performance of prediction models. A framework for traditional and novel measures. Epidemiology 21:128–138

    Article  PubMed  PubMed Central  Google Scholar 

  26. Altman DG, Vergouwe Y, Royston P, Moons KG (2009) Prognosis and prognostic research: validating a prognostic model. BMJ 338:b605

    Article  PubMed  Google Scholar 

  27. Moons KGM, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman GD, Woodward M (2012) Risk prediction models: II. Esternal validation, model updating and impact assessment. Heart 98:691–698

    Article  PubMed  Google Scholar 

  28. Toll DB, Janssen KJM, Vergouwe Y, Moons KGM (2008) Validating, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 61:1085–1094

    Article  CAS  PubMed  Google Scholar 

  29. Reilly BM, Evans AT (2006) Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Int Med 144:201–209

    Article  PubMed  Google Scholar 

  30. Moons KGM, Altman DG, Vergouwe Y, Royston P (2009) Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 338:b606

    Article  PubMed  Google Scholar 

  31. McGinnTH Guyatt GH, Wyer PC et al (2000) User’s guide to medical literature: XXII. How to use articles about clinical decision rules. Evidence based medicine working group. JAMA 284:79–84

    Article  Google Scholar 

  32. Campbel MK, Elborne DR, Altman DC (2014) CONSORT statement extension to cluster randomized trials. BMJ 328:702–708

    Article  Google Scholar 

  33. Hall LM, Jung RT, Leese GP (2003) Controlled trial of effect of documented cardiovascular risk scores on prescribing. BMJ 326:251–252

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Toll DB, Janssen KJM, Vergouwe Y, Moons KGM (2008) Validating, Updating and impact of clinical prediction rules: a review. J Clin Epidemiol 61:1085–1094

    Article  CAS  PubMed  Google Scholar 

  35. Collins GS, Reistma JB, Altman DG, Moons KGM (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Int Med 162:55–63

    Article  PubMed  Google Scholar 

  36. Hemingway H, Riley DR, Altman DG (2009) Ten steps towards improving prognosis research. BMJ 339:b4184

    Article  PubMed  Google Scholar 

  37. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A et al (2013) Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ 346:e5595

    Article  PubMed  PubMed Central  Google Scholar 

  38. Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA et al (2013) Prognosis research strategy (PROGRESS) 2: prognostic factor research. PLoS Med 10(2):e1001380

    Article  PubMed  PubMed Central  Google Scholar 

  39. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S et al (2013) Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med 10(2):e1001381

    Article  PubMed  PubMed Central  Google Scholar 

  40. PLOS Medicine Editors (2014) Observational studies: getting clear about transparency. PLoS Med 11(8):e1001711

    Article  Google Scholar 

  41. The Nordic Trial Alliance Working Group 6 on transparency and registration. 2015 http://nta.nordforsk.org/projects/FINALNTAWPG30032015

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gennaro D’Amico.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Statement of human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

None.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

D’Amico, G., Malizia, G. & D’Amico, M. Prognosis research and risk of bias. Intern Emerg Med 11, 251–260 (2016). https://doi.org/10.1007/s11739-016-1404-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11739-016-1404-z

Keywords

Navigation