Skip to main content
SpringerLink
Log in

Redefining the Critical Value of Significance Level (0.005 instead of 0.05): The Bayes Trace

  • METHODOLOGY OF SCIENTIFIC SEARCH
  • Published:
Biology Bulletin Aims and scope Submit manuscript

Abstract

In 2017, a group of the leading mathematical statisticians published a paper-manifesto having an extremely simple sense: the common critical level of p-values should be decreased by an order of magnitude (0.005 instead of 0.05) (Benjamin, et al., 2017). In this review, the arguments of proponents and opponents of this proposal are discussed. Moreover, the problems related to the “reproducibility crisis” of the scientific results are considered. The corresponding argumentation cannot be understood without consideration of the fundamentals of the theory of statistical derivation. In this connection, the precise sense of some concepts, such as p-value, the Bayes factor, and the minimum a posteriori probability of the zero hypothesis are discussed in the review. This is made mainly with the examples related to the comparison of frequencies. It was shown that, when using p-values, particular attention should be paid to the comparison of low frequencies on the highly abundant samples. Some practical recommendations on application of the Bayes analysis are given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.

Similar content being viewed by others

REFERENCES

  1. Benjamin, D.J., Berger, J., Johannesson, M., Nosek, B., Wagenmakers, E., Berk, R., et al., Redefine statistical significance, Nat. Hum. Behav., 2018, no. 2, pp. 6–10.

  2. Ioannidis, J., Why most published research findings are false, PLoS Med., 2005, vol. 2. e124.

    Article  Google Scholar 

  3. Buck, S., Solving reproducibility, Science, 2015, vol. 348, no. 6242, p. 1403.

    Article  CAS  Google Scholar 

  4. Kolmogorov, A.N., Probability theory, in Veroyatnost’ i matematicheskaya statistika. Entsiklopediya (Probability and Mathematical Statistics. Encyclopedia), Prokhorov, Yu.V., Editor-in-Chief, Moscow: Bol’shaya Rossiiskaya Entsiklopediya, 1999; Moscow: Drofa, 2003, pp. 874–875.

  5. Melton, A.W., Editorial, J. Exp. Psychol., 1962, vol. 64, pp. 553–557.

    Article  Google Scholar 

  6. Wasserstein, R.L. and Lazar, N.A., The ASA’s statement on p-values: context, process, and purpose, Am. Statistician, 2016, vol. 70, no. 2, pp. 129–133.

    Article  Google Scholar 

  7. Lakens, D., Adolfi, F.G., Albers, C.J., Anvari, F., Apps, M.A., et al., Justify Your Alpha, 2018. psyarxiv.com/9s3y6.

  8. Open Science Collaboration, Science. 2015, vol. 349, no. 6251, pp. 1–8.

  9. McShane, B.B., Gal, D., Gelman, A., Robert, C., and Tackett, J.L., Abandon statistical significance, 2017. arXiv:1709.07588 [stat.ME].

  10. Trafimow, D., Amrhein, V., Areshenkoff, C.N., et al., Manipulating the alpha level cannot 1 cure significance testing. comments on “Redefine statistical significance,” PeerJ. Preprints, 2017, vol. 5. e3411v1. https://peerj.com/preprints/3411/.

  11. Perezgonzalez, J.D. and Frías-Navarro, M.D., Retract p < 0.005 and propose using JASP, instead, F1000Research, 2017, vol. 6, p. 2122.

    Article  Google Scholar 

  12. Amrhein, V. and Greenland, S., Remove, rather than redefine, statistical significance, Nat. Hum. Behav., 2018, vol. 2, p. 4.

    Article  Google Scholar 

  13. Esarey, J., Replication data for: lowering the threshold of statistical significance to p < 0.005 to encourage enriched theories of politics, Polit. Methodologist, 2017, vol. 24, no. 2, pp. 13–20. https://thepoliticalmethodologist.com/v24-n2-fix/.

    Google Scholar 

  14. Crane, H., Why “redefining statistical significance” will not improve reproducibility and could make the replication crisis worse, 2017. arXiv:1711.07801v1 [stat.AP].

  15. Head, M.L., Holman, L., Lanfear, R., Kahn, A.T., and Jennions, M.D., The extent and consequences of p‑hacking in science, PLoS Biol., 2015, vol. 13, no. 3. e1002106.

    Article  Google Scholar 

  16. Rubanovich, A.V. and Khromov-Borisov, N.N., Genetic risk assessment of the joint effect of several genes: critical appraisal, Russ. J. Genet., 2016, vol. 52, no. 7, pp. 757–769.

    Article  CAS  Google Scholar 

  17. Wray, N.R., Yang, J., Hayes, B.J., et al., Pitfalls of predicting complex traits from SNPs, Nat. Rev. Genet., 2013, vol. 14, no. 7, pp. 507–515.

    Article  CAS  Google Scholar 

  18. Goodman, S., A dirty dozen: twelve p-value misconceptions, Semin. Hematol., 2008, vol. 45, pp. 135–140.

    Article  Google Scholar 

  19. Dienes, Z., How Bayes factors change scientific practice, J. Math. Psychol., 2016, vol. 72, pp. 78–89.

    Article  Google Scholar 

  20. Held, L. and Ott, M., On p-values and Bayes factors, Annu. Rev. Stat. Appl., 2018, vol. 5, pp. 393–419.

    Article  Google Scholar 

  21. Jeffreys, H., An invariant form for the prior probability in estimation “problems,” Proc. R. Soc. London, Ser. A, 1946, vol. 186, no. 1007, pp. 453–461.

    Article  CAS  Google Scholar 

  22. Sellke, T., Bayarri, M.J., and Berger, J.O., Calibration of p values for testing precise null hypotheses, Am. Statist., 2001, vol. 55, pp. 62–71.

    Article  Google Scholar 

Download references

Funding

This work was supported by the Russian Foundation for Basic Research (project no. 16-06-0046517).

Author information

Authors and Affiliations

  1. Vavilov Institute of General Genetics, Russian Academy of Sciences, 119991, Moscow, Russia

    A. V. Rubanovich

Authors
  1. A. V. Rubanovich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. V. Rubanovich.

Ethics declarations

The author declares that he has no conflict of interest. This article does not contain any studies involving animals or human participants performed by the author.

Additional information

Translated by M. Batrukova

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rubanovich, A.V. Redefining the Critical Value of Significance Level (0.005 instead of 0.05): The Bayes Trace. Biol Bull Russ Acad Sci 46, 1449–1457 (2019). https://doi.org/10.1134/S1062359019110086

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1062359019110086

Keywords:

Search

Navigation