Skip to main content
SpringerLink
Log in

Detecting Evasion Attacks in Deployed Tree Ensembles

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14173))

Abstract

Tree ensembles are powerful models that are widely used. However, they are susceptible to evasion attacks where an adversary purposely constructs an adversarial example in order to elicit a misprediction from the model. This can degrade performance and erode a user’s trust in the model. Typically, approaches try to alleviate this problem by verifying how robust a learned ensemble is or robustifying the learning process. We take an alternative approach and attempt to detect adversarial examples in a post-deployment setting. We present a novel method for this task that works by analyzing an unseen example’s output configuration, which is the set of leaves activated by the example in the ensemble’s constituent trees. Our approach works with any additive tree ensemble and does not require training a separate model. We evaluate our approach on three different tree ensemble learners. We empirically show that our method is currently the best adversarial detection method for tree ensembles.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Computed using Veritas [12].

  2. 2.

    We use Veritas’s binary-search approach to find the closest adversarial example because it is an order of magnitude faster than their mixed-integer linear programming solution.

References

  1. Andriushchenko, M., Hein, M.: Provably robust boosted decision stumps and trees against adversarial attacks. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  2. Biggio, B., et al.: Evasion attacks against machine learning at test time. In: Machine Learning and Knowledge Discovery in Databases, pp. 387–402 (2013)

    Google Scholar 

  3. Biggio, B., Roli, F.: Wild patterns: ten years after the rise of adversarial machine learning. Pattern Recogn. 84, 317–331 (2018)

    Article  Google Scholar 

  4. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  5. Breiman, L., Cutler, A.: Random forests manual (2002). https://www.stat.berkeley.edu/~breiman/RandomForests

  6. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)

    Google Scholar 

  7. Calzavara, S., Lucchese, C., Tolomei, G., Abebe, S.A., Orlando, S.: Treant: training evasion-aware decision trees. Data Min. Knowl. Disc. 34(5), 1390–1420 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chen, H., Zhang, H., Boning, D., Hsieh, C.J.: Robust decision trees against adversarial examples. In: Proceedings of the 36th International Conference on Machine Learning, pp. 1122–1131 (2019)

    Google Scholar 

  9. Chen, H., Zhang, H., Si, S., Li, Y., Boning, D., Hsieh, C.J.: Robustness verification of tree-based models. In: Advances in Neural Information Processing Systems, vol. 32, pp. 12317–12328 (2019)

    Google Scholar 

  10. Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 785–794 (2016)

    Google Scholar 

  11. Cortes, C., DeSalvo, G., Mohri, M.: Learning with rejection. In: Proceedings of the 27th International Conference on Algorithmic Learning Theory (ALT 2016) (2016)

    Google Scholar 

  12. Devos, L., Meert, W., Davis, J.: Versatile verification of tree ensembles. In: Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 2654–2664 (2021)

    Google Scholar 

  13. Diochnos, D., Mahloujifar, S., Mahmoody, M.: Adversarial risk and robustness: general definitions and implications for the uniform distribution. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  14. Einziger, G., Goldstein, M., Sa’ar, Y., Segall, I.: Verifying robustness of gradient boosted models. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 2446–2453 (2019)

    Google Scholar 

  15. Fawzi, A., Fawzi, H., Fawzi, O.: Adversarial vulnerability for any classifier. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 1186–1195 (2018)

    Google Scholar 

  16. Feinman, R., Curtin, R.R., Shintre, S., Gardner, A.B.: Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410 (2017)

  17. Gong, Z., Wang, W., Ku, W.S.: Adversarial and clean data are not twins. arXiv preprint arXiv:1704.04960 (2017)

  18. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: Proceedings of the 3rd International Conference on Learning Representations (2015)

    Google Scholar 

  19. Grosse, K., Manoharan, P., Papernot, N., Backes, M., McDaniel, P.: On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280 (2017)

  20. Grosse, K., Pfaff, D., Smith, M.T., Backes, M.: The limitations of model uncertainty in adversarial settings. In: 4th workshop on Bayesian Deep Learning (NeurIPS 2019) (2018)

    Google Scholar 

  21. Guo, J.Q., Teng, M.Z., Gao, W., Zhou, Z.H.: Fast provably robust decision trees and boosting. In: Proceedings of the 39th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 162, pp. 8127–8144 (2022)

    Google Scholar 

  22. Hendrickx, K., Perini, L., der Plas, D.V., Meert, W., Davis, J.: Machine learning with a reject option: a survey. CoRR abs/2107.11277 (2021)

    Google Scholar 

  23. Kantchelian, A., Tygar, J.D., Joseph, A.: Evasion and hardening of tree ensemble classifiers. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 2387–2396. PMLR (2016)

    Google Scholar 

  24. Katzir, Z., Elovici, Y.: Detecting adversarial perturbations through spatial behavior in activation spaces. In: 2019 International Joint Conference on Neural Networks, pp. 1–9 (2019)

    Google Scholar 

  25. Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. In: Advances in Neural Information Processing Systems, vol. 30, pp. 3146–3154 (2017)

    Google Scholar 

  26. Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., Jana, S.: Certified robustness to adversarial examples with differential privacy. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 656–672. IEEE (2019)

    Google Scholar 

  27. Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  28. Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. In: Proceedings of the 8th IEEE International Conference on Data Mining, pp. 413–422. IEEE (2008)

    Google Scholar 

  29. Liu, X., Li, Y., Wu, C., Hsieh, C.J.: Adv-BNN: improved adversarial defense through robust Bayesian neural network. In: Proceedings of the 7th International Conference on Learning Representations (2019). https://openreview.net/forum?id=rk4Qso0cKm

  30. Metzen, J.H., Genewein, T., Fischer, V., Bischoff, B.: On detecting adversarial perturbations. In: Proceedings of 5th International Conference on Learning Representations (2017)

    Google Scholar 

  31. Pace, R.K., Barry, R.: Sparse spatial autoregressions. Stat. Probab. Lett. 33(3), 291–297 (1997)

    Article  MATH  Google Scholar 

  32. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  33. Raghunathan, A., Steinhardt, J., Liang, P.: Certified defenses against adversarial examples. In: Proceedings of the 6th International Conference on Learning Representations (2018)

    Google Scholar 

  34. Ranzato, F., Zanella, M.: Abstract interpretation of decision tree ensemble classifiers. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5478–5486 (2020)

    Google Scholar 

  35. Ranzato, F., Zanella, M.: Genetic adversarial training of decision trees. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 358–367 (2021)

    Google Scholar 

  36. Roth, K., Kilcher, Y., Hofmann, T.: The odds are odd: a statistical test for detecting adversarial examples. In: Proceedings of the 36th International Conference on Machine Learning, pp. 5498–5507. PMLR (2019)

    Google Scholar 

  37. Schulze, J.P., Sperl, P., Böttinger, K.: DA3G: detecting adversarial attacks by analysing gradients. In: European Symposium on Research in Computer Security, pp. 563–583 (2021)

    Google Scholar 

  38. Shi, T., Horvath, S.: Unsupervised learning with random forest predictors. J. Comput. Graph. Stat. 15(1), 118–138 (2006)

    Article  MathSciNet  Google Scholar 

  39. Sperl, P., Kao, C.Y., Chen, P., Lei, X., Böttinger, K.: DLA: dense-layer-analysis for adversarial example detection. In: 2020 IEEE European Symposium on Security and Privacy (EuroS &P), pp. 198–215 (2020)

    Google Scholar 

  40. Szegedy, C., et al.: Intriguing properties of neural networks. In: Proceedings of the 2nd International Conference on Learning Representations (2014)

    Google Scholar 

  41. Tan, S., Soloviev, M., Hooker, G., Wells, M.T.: Tree space prototypes: another look at making tree ensembles interpretable. In: Proceedings of the 2020 ACM-IMS on Foundations of Data Science Conference, pp. 23–34 (2020)

    Google Scholar 

  42. Tian, J., Zhou, J., Li, Y., Duan, J.: Detecting adversarial examples from sensitivity inconsistency of spatial-transform domain. In: Proceedings of the AAAI Conference on Artificial Intelligence (2022)

    Google Scholar 

  43. Vens, C., Costa, F.: Random forest based feature induction. In: Proceedings of 11th IEEE International Conference on Data Mining, pp. 744–753 (2011)

    Google Scholar 

  44. Vos, D., Verwer, S.: Efficient training of robust decision trees against adversarial examples. In: Proceedings of the 38th International Conference on Machine Learning, pp. 10586–10595 (2021)

    Google Scholar 

  45. Vos, D., Verwer, S.: Adversarially robust decision tree relabeling. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases (2022)

    Google Scholar 

  46. Vos, D., Verwer, S.: Robust optimal classification trees against adversarial examples. In: Proceedings of the AAAI Conference on Artificial Intelligence (2022)

    Google Scholar 

  47. Wang, Y., Zhang, H., Chen, H., Boning, D., Hsieh, C.J.: On LP-norm robustness of ensemble decision stumps and trees. In: Proceedings of the 37th International Conference on Machine Learning, pp. 10104–10114 (2020)

    Google Scholar 

  48. Yang, P., Chen, J., Hsieh, C.J., Wang, J.L., Jordan, M.: ML-LOO: detecting adversarial examples with feature attribution. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, pp. 6639–6647 (2020)

    Google Scholar 

  49. Yang, Y.Y., Rashtchian, C., Wang, Y., Chaudhuri, K.: Robustness for non-parametric classification: a generic attack and defense. In: Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 108, pp. 941–951 (2020)

    Google Scholar 

  50. Zhang, C., Zhang, H., Hsieh, C.J.: An efficient adversarial attack for tree ensembles. In: Advances in Neural Information Processing Systems, vol. 33, pp. 16165–16176 (2020)

    Google Scholar 

  51. Zhang, S., et al.: Detecting adversarial samples for deep learning models: a comparative study. IEEE Trans. Netw. Sci. Eng. 9(1), 231–244 (2022)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This research is supported by the Research Foundation – Flanders (LD: 1SB1322N; LP: 1166222N), the Flemish Government under the “Onderzoeksprogramma Artificiele Intelligentie (AI) Vlaanderen” program (JD), the European Union’s Horizon Europe Research and Innovation program under the grant agreement TUPLES No. 101070149 (JD), and KU Leuven Research Fund (JD: iBOF/21/075; WM: IOF).

Author information

Authors and Affiliations

  1. KU Leuven, Leuven, Belgium

    Laurens Devos, Lorenzo Perini, Wannes Meert & Jesse Davis

Authors
  1. Laurens Devos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lorenzo Perini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wannes Meert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jesse Davis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laurens Devos .

Editor information

Editors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Danai Koutra

  2. University of Vienna, Vienna, Austria

    Claudia Plant

  3. Max Planck Institute for Software Systems, Kaiserslautern, Germany

    Manuel Gomez Rodriguez

  4. Politecnico di Torino, Turin, Italy

    Elena Baralis

  5. CENTAI, Turin, Italy

    Francesco Bonchi

Ethics declarations

Ethical Statement

Machine learning is widely used in many different application areas. With the wide adoption, machine learned models, including tree ensembles, increasingly become high-stake targets for attackers who might employ evasion attacks to achieve their goal. This work proposes a defense method against evasion attacks for tree ensembles. Together with other approaches like robust tree ensembles, this work is a step forward in our ability to protect against evasion attacks. This could further improve the trust in machine learning, and further accelerate its adoption, especially in sensitive application areas.

Improved defenses will likely also result in the development of improved counter-attacks. We strongly feel that it is in the interest of the research community that (1) the research community stays on top of these developments so that machine learning libraries can adapt if necessary, and (2) all work done in this area is open-access. For that reason, all resources and codes are publicly available at https://github.com/laudv/ocscore.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Devos, L., Perini, L., Meert, W., Davis, J. (2023). Detecting Evasion Attacks in Deployed Tree Ensembles. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14173. Springer, Cham. https://doi.org/10.1007/978-3-031-43424-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43424-2_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43423-5

  • Online ISBN: 978-3-031-43424-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation