Skip to main content
SpringerLink
Log in

An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14171))

Abstract

The Rashomon Effect describes the following phenomenon: for a given dataset there may exist many models with equally good performance but with different solution strategies. The Rashomon Effect has implications for Explainable Machine Learning, especially for the comparability of explanations. We provide a unified view on three different comparison scenarios and conduct a quantitative evaluation across different datasets, models, attribution methods, and metrics. We find that hyperparameter-tuning plays a role and that metric selection matters. Our results provide empirical support for previously anecdotal evidence and exhibit challenges for both scientists and practitioners.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Our code is available at github.com/lamarr-xai-group/RashomonEffect.

  2. 2.

    See project page at github.com/pytorch/captum.

References

  1. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)

    Article  Google Scholar 

  2. Alkhatib, A., Boström, H., Vazirgiannis, M.: Explaining predictions by characteristic rules. In: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD) (2022)

    Google Scholar 

  3. Alvarez-Melis, D., Jaakkola, T.S.: On the robustness of interpretability methods. In: Workshop on Human Interpretability in Machine Learning (WHI@ICML) (2018)

    Google Scholar 

  4. Ancona, M., Ceolini, E., Öztireli, C., Gross, M.: Towards better understanding of gradient-based attribution methods for deep neural networks. In: International Conference on Learning Representations, (ICLR) (2018)

    Google Scholar 

  5. Atanasova, P., Simonsen, J.G., Lioma, C., Augenstein, I.: A diagnostic study of explainability techniques for text classification. In: Conference on Empirical Methods in Natural Language Processing (EMNLP) (2020)

    Google Scholar 

  6. Beckh, K., et al.: Harnessing prior knowledge for explainable machine learning: an overview. In: 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 450–463 (2023). https://doi.org/10.1109/SaTML54575.2023.00038

  7. Bogun, A., Kostadinov, D., Borth, D.: Saliency diversified deep ensemble for robustness to adversaries. In: AAAI-22 Workshop on Adversarial Machine Learning and Beyond (2021)

    Google Scholar 

  8. Breiman, L.: Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat. Sci. 16(3), 199–231 (2001)

    Article  MATH  Google Scholar 

  9. Burkart, N., Huber, M.F.: A survey on the explainability of supervised machine learning. J. Artif. Intell. Res. 70, 245–317 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  10. DeYoung, J., et al.: ERASER: a benchmark to evaluate rationalized NLP models. In: Annual Meeting of the Association for Computational Linguistics (ACL) (2020)

    Google Scholar 

  11. ElShawi, R., Sherif, Y., Al-Mallah, M., Sakr, S.: Interpretability in healthcare: a comparative study of local machine learning interpretability techniques. Comput. Intell. 37(4), 1633–1650 (2021)

    Article  MathSciNet  Google Scholar 

  12. Fisher, A., Rudin, C., Dominici, F.: All models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneously. J. Mach. Learn. Res. 20(177), 1–81 (2019)

    MathSciNet  MATH  Google Scholar 

  13. Flora, M., Potvin, C., McGovern, A., Handler, S.: Comparing explanation methods for traditional machine learning models part 1: an overview of current methods and quantifying their disagreement. arXiv preprint arXiv:2211.08943 (2022)

  14. Guidotti, R., Ruggieri, S.: Assessing the stability of interpretable models. arXiv preprint arXiv:1810.09352 (2018)

  15. Han, T., Srinivas, S., Lakkaraju, H.: Which explanation should i choose? A function approximation perspective to characterizing post hoc explanations. In: Advances in Neural Information Processing Systems (NeurIPS) (2022)

    Google Scholar 

  16. Hancox-Li, L.: Robustness in machine learning explanations: does it matter? In: Conference on Fairness, Accountability, and Transparency (FAT*) (2020)

    Google Scholar 

  17. Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A benchmark for interpretability methods in deep neural networks. In: Advances in Neural Information Processing Systems (NeurIPS) (2019)

    Google Scholar 

  18. Koklu, M., Özkan, I.A.: Multiclass classification of dry beans using computer vision and machine learning techniques. Comput. Electron. Agric. 174, 105507 (2020)

    Article  Google Scholar 

  19. Krishna, S., et al.: The disagreement problem in explainable machine learning: a practitioner’s perspective. arXiv preprint arXiv:2202.01602 (2022)

  20. Leventi-Peetz, A.M., Weber, K.: Rashomon effect and consistency in explainable artificial intelligence (XAI). In: Future Technologies Conference (FTC) (2022)

    Google Scholar 

  21. Liu, F., Avci, B.: Incorporating priors with feature attribution on text classification. In: Annual Meeting of the Association for Computational Linguistics (ACL) (2019)

    Google Scholar 

  22. Lundberg, S., Lee, S.I.: A Unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems (NeurIPS) (2017)

    Google Scholar 

  23. Marx, C.T., Calmon, F.P., Ustun, B.: Predictive multiplicity in classification. In: International Conference on Machine Learning (ICML) (2020)

    Google Scholar 

  24. Molnar, C.: Interpretable Machine Learning. 2nd edn. (2022)

    Google Scholar 

  25. Mücke, S., Pfahler, L.: Check Mate: a sanity check for trustworthy AI. In: Lernen. Wissen. Daten. Analysen. (LWDA) (2022)

    Google Scholar 

  26. Neely, M., Schouten, S.F., Bleeker, M.J., Lucic, A.: order in the court: explainable AI methods prone to disagreement. arXiv preprint arXiv:2105.03287 (2021)

  27. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?: explaining the predictions of any classifier. In: International Conference on Knowledge Discovery and Data Mining (KDD) (2016)

    Google Scholar 

  28. Roscher, R., Bohn, B., Duarte, M.F., Garcke, J.: Explainable machine learning for scientific insights and discoveries. IEEE Access 8, 42200–42216 (2020)

    Article  Google Scholar 

  29. Schramowski, P., et al.: Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nat. Mach. Intell. 2(8), 476–486 (2020)

    Article  Google Scholar 

  30. Semenova, L., Rudin, C., Parr, R.: On the existence of simpler machine learning models. In: Conference on Fairness, Accountability, and Transparency (FAccT) (2022)

    Google Scholar 

  31. Sigillito, V.G., Wing, S.P., Hutton, L.V., Baker, K.B.: Classification of radar returns from the ionosphere using neural networks. Johns Hopkins APL Tech. Digest 10(3), 262–266 (1989)

    Google Scholar 

  32. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: International Conference on Learning Representations (ICLR) (2014)

    Google Scholar 

  33. Smilkov, D., Thorat, N., Kim, B., Viégas, F., Wattenberg, M.: Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017)

  34. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International Conference on Machine Learning (ICML) (2017)

    Google Scholar 

  35. Watson, M., Hasan, B.A.S., Al Moubayed, N.: Agree to disagree: when deep learning models with identical architectures produce distinct explanations. In: Winter Conference on Applications of Computer Vision (WACV) (2022)

    Google Scholar 

  36. Wolberg, W., Street, N., Mangasarian, O.: Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository (1995)

    Google Scholar 

  37. Xin, R., Zhong, C., Chen, Z., Takagi, T., Seltzer, M., Rudin, C.: Exploring the whole rashomon set of sparse decision trees. In: Advances in Neural Information Processing Systems (NeurIPS) (2022)

    Google Scholar 

  38. Yeh, C., Hsieh, C., Suggala, A.S., Inouye, D.I., Ravikumar, P.: On the (In)fidelity and sensitivity of explanations. In: Advances in Neural Information Processing Systems (NeurIPS) (2019)

    Google Scholar 

  39. Zednik, C., Boelsen, H.: Scientific exploration and explainable artificial intelligence. Minds Mach. 32(1), 219–239 (2022)

    Article  Google Scholar 

  40. Zhang, X., Zhao, J.J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems (NeurIPS) (2015)

    Google Scholar 

Download references

Acknowledgments

This research has been funded by the Federal Ministry of Education and Research of Germany and the state of North-Rhine Westphalia as part of the Lamarr-Institute for Machine Learning and Artificial Intelligence Lamarr22B. Part of PWs work has been funded by the Vienna Science and Technology Fund (WWTF) project ICT22-059.

Author information

Authors and Affiliations

  1. University of Bonn, Bonn, Germany

    Sebastian Müller, Vanessa Toborek & Christian Bauckhage

  2. TU Dortmund University, Dortmund, Germany

    Matthias Jakobs

  3. Fraunhofer IAIS, Sankt Augustin, Germany

    Katharina Beckh & Christian Bauckhage

  4. Lamarr Institute, Bonn, Germany

    Sebastian Müller, Vanessa Toborek, Katharina Beckh, Matthias Jakobs & Christian Bauckhage

  5. TU Wien, Vienna, Austria

    Pascal Welke

Authors
  1. Sebastian Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vanessa Toborek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Katharina Beckh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthias Jakobs
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christian Bauckhage
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pascal Welke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Müller .

Editor information

Editors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Danai Koutra

  2. University of Vienna, Vienna, Austria

    Claudia Plant

  3. Max Planck Institute for Software Systems, Kaiserslautern, Germany

    Manuel Gomez Rodriguez

  4. Politecnico di Torino, Turin, Italy

    Elena Baralis

  5. CENTAI, Turin, Italy

    Francesco Bonchi

Ethics declarations

Ethical Statement

In critical contexts, where persons are directly or indirectly impacted by a model, and where explanations are used to verify that model behavior is compliant with a given standard, proper use of explanation methods is of utmost importance. Hyperparameter choices have to be validated for each model individually. For model testing and validation procedures to be reliable they have to integrate this knowledge. Our work demonstrated that it is unreasonable to expect an explanation computed for one model, to be valid for another model, however similar their performance otherwise may be. Re-using explanations from one model to give as an explanation of behavior for another model is not possible and has to be avoided in critical scenarios.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Müller, S., Toborek, V., Beckh, K., Jakobs, M., Bauckhage, C., Welke, P. (2023). An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14171. Springer, Cham. https://doi.org/10.1007/978-3-031-43418-1_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43418-1_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43417-4

  • Online ISBN: 978-3-031-43418-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation