Abstract
Empirical research is ripe for a reality check, as elegantly put by the “elephants in the room” (Ohlson, 2022a. Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: Convivium.) referring to practices to disguise false positives with the aid of statistical engineering. However, the diagnosis points to a deeper problem. The dominant empirical paradigm combines extraordinarily vague hypotheses with ridiculously high desired levels of statistical confidence beatable solely with econometric hacks. Instead, I argue that economic magnitudes measure meaningful theoretical constructs and require far less than conventional significance levels for measurements of sufficient importance. Precisely estimating that an effect is close to zero can be more meaningful than a noisy but significant coefficient. I make several actionable proposals: (1) report standard errors rather than conventional statistical significance (stars) or t-stats, (2) discuss target significance levels likely to change priors and could much higher than weak significance for unsettled questions, (3) report precisely estimated zeros and power analyses, and (4) anchor empirical design on formal theory justified with precise references or structural models.
References
Abadie, A. (2020). Statistical nonsignificance in empirical economics. The American Economic Review: Insights, 2(2), 193–208. https://doi.org/10.1257/aeri.20190252 Search in Google Scholar
Aglietta, M., & Rebérioux, A. (2005). Corporate governance adrift: A critique of shareholder value. Edward Elgar Publishing.10.4337/9781845425470Search in Google Scholar
Angrist, J. D., & Pischke, J.-S. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton university press.10.1515/9781400829828Search in Google Scholar
Appelbaum, D., Alexander, K., Vasarhelyi, M., & Yan, Z. (2017). Impact of business analytics and enterprise systems on managerial accounting. International Journal of Accounting Information Systems, 25, 29–44. https://doi.org/10.1016/j.accinf.2017.03.003 Search in Google Scholar
Arcidiacono, P., & Miller, R. A. (2011). Conditional choice probability estimation of dynamic discrete choice models with unobserved heterogeneity. Econometrica, 79(6), 1823–1867.10.3982/ECTA7743Search in Google Scholar
Ball, R. (2013). Accounting informs investors and earnings management is rife: Two questionable beliefs. Accounting Horizons, 27(4), 847–853. https://doi.org/10.2308/acch-10366 Search in Google Scholar
Bertomeu, J., Beyer, A., & Taylor, D. J. (2016). From casual to causal inference in accounting research: The need for theoretical foundations. Foundations and Trends® in Accounting, 10(2–4), 262–313. https://doi.org/10.1561/1400000044.Search in Google Scholar
Bertomeu, J., Cheynel, E., Floyd, E., & Pan, W. (2021b). Using machine learning to detect misstatements. Review of Accounting Studies, 26(2), 468–519. https://doi.org/10.1007/s11142-020-09563-8 Search in Google Scholar
Bertomeu, J., Cheynel, E., Li, E. X., & Liang, Y. (2021a). How pervasive is earnings management? Evidence from a structural model. Management Science, 67(8), 5145–5162. https://doi.org/10.1287/mnsc.2020.3717 Search in Google Scholar
Bertomeu, J., Liang, Y., & Marinovic, I. (2023). A primer on structural estimation in accounting research. Foundations and Trends® in Accounting, 18(1–2), 1–137.10.1561/1400000074Search in Google Scholar
Bertomeu, J., Marinovic, I., Terry, S. J., & Varas, F. (2022). The dynamics of concealment. Journal of Financial Economics, 143(1), 227–246. https://doi.org/10.1016/j.jfineco.2021.05.025 Search in Google Scholar
Biondi, Y. (2019). Ownership (lost) and corporate control: An enterprise entity perspective. Accounting, Economics, and Law: Convivium, 10(3), 20190025. https://doi.org/10.1515/ael-2019-0025 Search in Google Scholar
Biondi, Y. (2023). Limits of empirical studies in social sciences: A constructive critique from accounting, economics and the law. Accounting, Economics, and Law: Convivium. https://doi.org/10.1515/ael-2017-0026 Search in Google Scholar
Cartwright, N. (2016). Where’s the rigor when you need it? Foundations and Trends® in Accounting, 10(2-4), 106–124. https://doi.org/10.1561/1400000045 Search in Google Scholar
Chemla, G., & Hennessy, C. A. (2019). Controls, belief updating, and bias in medical rcts. Journal of Economic Theory, 184, 104929. https://doi.org/10.1016/j.jet.2019.07.016 Search in Google Scholar
Chemla, G., & Hennessy, C. A. (2020). Rational expectations and the paradox of policy-relevant natural experiments. Journal of Monetary Economics, 114, 368–381. https://doi.org/10.1016/j.jmoneco.2019.05.002 Search in Google Scholar
Chen, X., Yang Cho, H., Dou, Y., & Lev, B. (2022). Predicting future earnings changes using machine learning and detailed financial data. Journal of Accounting Research, 60(2), 467–515. https://doi.org/10.1111/1475-679x.12429 Search in Google Scholar
Cheynel, E., & Zhou, F. (2020). Audit firm rotation and misstatements: A dynamic discrete choice approach. Available at SSRN 3284807.Search in Google Scholar
Christenson, C. (1983). The methodology of positive accounting. The Accounting Review, 58, 1–22.Search in Google Scholar
Comte, A., & Martineau, H. (1855). The positive philosophy (Vol. 2). Ams Press.Search in Google Scholar
Frankel, A., & Kasy, M. (2022). Which findings should be published? American Economic Journal: Microeconomics, 14(1), 1–38. https://doi.org/10.1257/mic.20190133 Search in Google Scholar
Frankel, R. M., Kothari, S. P., & Luo, Z. (2019). Economics of accounting earnings. Available at SSRN 3435218.10.2139/ssrn.3435218Search in Google Scholar
Gassen, J. (2014). Causal inference in empirical archival financial accounting research. Accounting, Organizations and Society, 39(7), 535–544. https://doi.org/10.1016/j.aos.2013.10.004 Search in Google Scholar
Gayle, G.-L., Li, C., & Miller, R. A. (2022). Was sarbanes–oxley costly? Evidence from optimal contracting on ceo compensation. Journal of Accounting Research, 60(4), 1189–1234. https://doi.org/10.1111/1475-679x.12436 Search in Google Scholar
Harvey, C. R., Liu, Y., & Zhu, H. (2016). And the cross-section of expected returns. Review of Financial Studies, 29(1), 5–68. https://doi.org/10.1093/rfs/hhv059 Search in Google Scholar
Hou, K., Xue, C., & Zhang, L. (2020). Replicating anomalies. Review of Financial Studies, 33(5), 2019–2133. https://doi.org/10.1093/rfs/hhy131 Search in Google Scholar
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124 Search in Google Scholar
Johnstone, D. (2018). Accounting theory as a bayesian discipline. Foundations and Trends® in Accounting, 13(1–2), 1–266. https://doi.org/10.1561/1400000056.Search in Google Scholar
Johnstone, D. (2023). Comments on ohlsos elephants in the room. Accounting, Economics, and Law: Convivium.Search in Google Scholar
Keane, M. P., Todd, P. E., & Wolpin, K. I. (2011). The structural estimation of behavioral models: Discrete choice dynamic programming methods and applications. In Handbook of labor economics (Vol. 4, pp. 331–461). Elsevier.10.1016/S0169-7218(11)00410-2Search in Google Scholar
Koopmans, T. C. (1947). Measurement without theory. The Review of Economics and Statistics, 29(3), 161–172. https://doi.org/10.2307/1928627 Search in Google Scholar
Leamer, E. E. (1983). Let’s take the con out of econometrics. The American Economic Review, 73(1), 31–43.Search in Google Scholar
Leamer, E. E. (2008). Macroeconomic patterns and stories. Springer Science & Business Media.10.1007/978-3-540-46389-4Search in Google Scholar
Leone, A. (2022). Earnings management? Working paper.Search in Google Scholar
Liang, P. J. (2023). Bookkeeping graphs: Computational theory and applications. Foundations and Trends in Accounting. https://doi.org/10.1561/1400000070 Search in Google Scholar
Li, J., Liang, P. J., & Hwang, H. (2019). Inter-entity bookkeeping networks: Representations and applications. Technical Report, working paper.Search in Google Scholar
McCloskey, D. N. (1985). Why economic historians should stop relying on statistical tests of significance, and lead economists and historians into the promised land. The American Economic Review, 75, 201–205.Search in Google Scholar
McCloskey, D. N. (1998). The rhetoric of economics. Univ of Wisconsin Press.Search in Google Scholar
Ohlson, J. A. (2015). Accounting research and common sense. Abacus, 51(4), 525–535. https://doi.org/10.1111/abac.12059 Search in Google Scholar
Ohlson, J. A. (2022a). Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: Convivium. https://doi.org/10.1515/ael-2021-0067 Search in Google Scholar
Ohlson, J. A. (2022b). Researchers’ data analysis choices: An excess of false positives? Review of Accounting Studies, 27(2), 649–667. https://doi.org/10.1007/s11142-021-09620-w Search in Google Scholar
Perols, J. L., Bowen, R. M., Zimmermann, C., & Samba, B. (2017). Finding needles in a haystack: Using data analytics to improve fraud prediction. The Accounting Review, 92(2), 221–245. https://doi.org/10.2308/accr-51562 Search in Google Scholar
Popper, K. (2005). The logic of scientific discovery. Routledge.10.4324/9780203994627Search in Google Scholar
Rubinstein, A. (2012). Economic fables. Open book publishers.10.11647/OBP.0020Search in Google Scholar
Rust, J. (2016). Mostly useless econometrics? Assessing the causal effect of econometric theory. Foundations and Trends® in Accounting, 10(2–4), 125–203. https://doi.org/10.1561/1400000049.Search in Google Scholar
Strebulaev, I. A., & Whited, T. M. (2012). Dynamic models and structural estimation in corporate finance. Foundations and Trends® in Finance, 6(1–2), 1–163. https://doi.org/10.1561/0500000035.Search in Google Scholar
Terry, S. J., Whited, T. M., & Zakolyukina, A. A. (2022). Information versus investment. Technical Report, National Bureau of Economic Research.10.3386/w29636Search in Google Scholar
Watson, D. (2004). Death sentence: The decay of public language. Queensland Narrating Service.Search in Google Scholar
Watts, R. L., & Zimmerman, J. L. (1990). Positive accounting theory: A ten year perspective. The Accounting Review, 65, 131–156.Search in Google Scholar
Weinstein, O. (2012). Firm, property and governance: From Berle and Means to the agency theory, and beyond. Accounting, Economics, and Law. https://doi.org/10.1515/2152-2820.1061 Search in Google Scholar
Zakolyukina, A. A. (2018). How common are intentional gaap violations? Estimates from a dynamic model. Journal of Accounting Research, 56(1), 5–44. https://doi.org/10.1111/1475-679x.12190 Search in Google Scholar
Zhou, F. S. (2021). Disclosure dynamics and investor learning. Management Science, 67(6), 3429–3446. https://doi.org/10.1287/mnsc.2020.3638 Search in Google Scholar
Ziliak, S., & McCloskey, D. N. (2008). The cult of statistical significance: How the standard error costs us jobs, justice, and lives. University of Michigan Press.10.3998/mpub.186351Search in Google Scholar
© 2023 CONVIVIUM, association loi de 1901
