Skip to main content
SpringerLink
Log in

Explaining the Suspicion: Design of an XAI-Based User-Focused Anti-Phishing Measure

  • Conference paper
  • First Online:
Innovation Through Information Systems (WI 2021)

Part of the book series: Lecture Notes in Information Systems and Organisation ((LNISO,volume 47))

Included in the following conference series:

Abstract

Phishing attacks are the primary cause of data and security breaches in businesses, public institutions, and private life. Due to inherent limitations and users’ high susceptibility to increasingly sophisticated phishing attempts, existing anti-phishing measures cannot realize their full potential. Against this background, we utilize methods from the emerging research field of Explainable Artificial Intelligence (XAI) for the design of a user-focused anti-phishing measure. By leveraging the power of state-of-the-art phishing detectors, our approach uncovers the words and phrases in an e-mail most relevant for identifying phishing attempts. We empirically show that our approach reliably extracts segments of text considered relevant for the discrimination between genuine and phishing e-mails. Our work opens up novel prospects for phishing prevention and demonstrates the tremendous potential of XAI methods beyond applications in AI.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. O’Donnell, L.: Coronavirus ‘Financial Relief’ Phishing Attacks Spike, (2020) https://threatpost.com/coronavirus-financial-relief-phishing-spike/154358/. Accessed 28 Aug 2020

  2. Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L.F., Hong, J.: Teaching johnny not to fall for phish. ACM Trans. Internet Technol. 10, 1–31 (2010)

    Google Scholar 

  3. Parsons, K., Butavicius, M., Pattinson, M., McCormac, A., Calic, D., Jerram, C.: Do users focus on the correct cues to differentiate between phishing and genuine emails? In: 26th Australasían Conference on Information Systems, Adelaide, Australia (2016)

    Google Scholar 

  4. Gupta, B.B., Arachchilage, N.A.G., Psannis, K.E.: Defending against phishing attacks: taxonomy of methods, current issues and future directions. Telecommun. Syst. 67(2), 247–267 (2017). https://doi.org/10.1007/s11235-017-0334-z

    Article  Google Scholar 

  5. Pienta, D., Thatcher, J., Johnston, A.: A taxonomy of phishing: attack types spanning economic, temporal, breadth, and target boundaries. In: Proceedings of the 13th Pre-ICIS Workshop on Information Security and Privacy, AIS, San Francisco, CA, USA (2018)

    Google Scholar 

  6. Hong, J.: The state of phishing attacks. Commun. ACM 55, 74–81 (2012)

    Article  Google Scholar 

  7. Khonji, M., Iraqi, Y., Jones, A.: Phishing detection: a literature survey. IEEE Commun. Surv. Tutorials 15, 2091–2121 (2013)

    Article  Google Scholar 

  8. Nguyen, C.: Learning not to take the bait: an examination of training methods and overlerarning on phishing susceptibility. PhD thesis. University of Oklahoma, Norman, OK, USA (2018)

    Google Scholar 

  9. Albakry, S., Vaniea, K.: Automatic phishing detection versus user training, Is there a middle ground using XAI? In: CEUR Workshop Proceedings, vol. 2151 (2018)

    Google Scholar 

  10. Williams, E.J., Hinds, J., Joinson, A.N.: Exploring susceptibility to phishing in the workplace. Int. J. Hum. Comput. Stud. 120, 1–13 (2018)

    Article  Google Scholar 

  11. Harrison, B., Svetieva, E., Vishwanath, A.: Individual processing of phishing emails: how attention and elaboration protect against phishing. Online Inf. Rev. 40, 265–281 (2016)

    Article  Google Scholar 

  12. Dennis, A.R., Minas, R.K.: Security on autopilot: why current security theories hijack our thinking and lead us astray. Database Adv. Inf. Syst. 49, 15–38 (2018)

    Article  Google Scholar 

  13. Parsons, K., McCormac, A., Pattinson, M., Butavicius, M., Jerram, C.: Phishing for the truth: a scenario-based experiment of users’ behavioural response to emails. In: Janczewski, L.J., Wolfe, H.B., Shenoi, S. (eds.) SEC 2013. IAICT, vol. 405, pp. 366–378. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39218-4_27

    Chapter  Google Scholar 

  14. Blythe, M., Petrie, H., Clark, J.A.: F for fake: four studies on how we fall for phish. In: CHI 2011, pp. 3469–3478, ACM, Vancouver, BC, Canada (2011)

    Google Scholar 

  15. Vishwanath, A., Herath, T., Chen, R., Wang, J., Rao, H.R.: Why do people get phished? Testing individual differences in phishing vulnerability within an integrated, information processing model. Decis. Support Syst. 51, 576–586 (2011)

    Article  Google Scholar 

  16. Vishwanath, A., Harrison, B., Ng, Y.J.: Suspicion, cognition, and automaticity model of phishing susceptibility. Communic. Res. 45, 1146–1166 (2018)

    Article  Google Scholar 

  17. Gunning, D.: Explainable Artificial Intelligence (XAI), 2017, https://www.darpa.mil/program/explainable-artificial-intelligence (Accessed 20 Aug 2020)

  18. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51, 1–42 (2019)

    Article  Google Scholar 

  19. Ribeiro, M.T., Singh, S., Guestrin, C.: Anchors: high-precision model-agnostic explanations. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1527–1535, AAAI, New Orleans, LA, USA (2018)

    Google Scholar 

  20. Martens, D., Provost, F.: Explaining data-driven document classifications. MIS Q. 38, 73–99 (2014)

    Article  Google Scholar 

  21. Hevner, A.R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28, 75–105 (2004)

    Article  Google Scholar 

  22. Jakobsson, M.: The human factor in phishing. In: Priv. Secur. Consum. Inf. (2007)

    Google Scholar 

  23. Kim, D., Hyun Kim, J.: Understanding persuasive elements in phishing e-mails. Online Inf. Rev. 37, 835–850 (2013)

    Article  Google Scholar 

  24. Zeng, V., et al.: Diverse datasets and a customizable benchmarking framework for phishing. In: IWSPA ‘20, pp. 35–41, ACM, New Orleans, LA, USA (2020)

    Google Scholar 

  25. Sheng, S., Wardman, B., Warner, G., Cranor, L.F., Hong, J., Zhang, C.: An empirical analysis of phishing blacklists. In: Sixth Conference on Email Anti-Spam, Mountain View, CA, USA (2009)

    Google Scholar 

  26. Verma, R.M., Zeng, V., Faridi, H.: Data quality for security challenges: case studies of phishing, malware and intrusion detection datasets. In: CCS ‘19, pp. 2605–2607, ACM, London, UK (2019)

    Google Scholar 

  27. Karumbaiah, S., Wright, R.T., Durcikova, A., Jensen, M.L.: Phishing training: a preliminary look at the effects of different types of training. In: Proceedings of the 11th Pre-ICIS Workshop on Information Security and Privacy, AIS, Dublin, Ireland (2016)

    Google Scholar 

  28. Sheng, S., et al.: Anti-phishing phil: the design and evaluation of a game that teaches people not to fall for phish. In: SOUPS 2007, pp. 88–99, Pittsburgh, PA, USA (2007)

    Google Scholar 

  29. Canova, G., Volkamer, M., Bergmann, C., Borza, R.: NoPhish: an anti-phishing education app. In: Mauw, S., Jensen, C.D. (eds.) STM 2014. LNCS, vol. 8743, pp. 188–192. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11851-2_14

    Chapter  Google Scholar 

  30. Moody, G.D., Galletta, D.F., Dunn, B.K.: Which phish get caught? An exploratory study of individuals’ susceptibility to phishing. Eur. J. Inf. Syst. 26, 564–584 (2017)

    Article  Google Scholar 

  31. Wang, J., Li, Y., Rao, H.R.: Overconfidence in phishing email detection. J. Assoc. Inf. Syst. 17, 759–783 (2016)

    Google Scholar 

  32. Volkamer, M., Renaud, K., Reinheimer, B.: TORPEDO: tooltip-powered phishing email detection. In: Hoepman, J.-H., Katzenbeisser, S. (eds.) SEC 2016. IAICT, vol. 471, pp. 161–175. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-33630-5_12

    Chapter  Google Scholar 

  33. Wang, D., Yang, Q., Abdul, A., Lim, B.Y.: Designing theory-driven user-centric explainable AI. In: CHI 2019, ACM, Glasgow, UK (2019)

    Google Scholar 

  34. Lipton, Z.C.: The mythos of model interpretability. Queue 16, 1–27 (2018)

    Article  Google Scholar 

  35. Lei, T., Barzilay, R., Jaakkola, T.: Rationalizing neural predictions. In: EMNLP 2016, pp. 107–117, ACL, Stroudsburg, PA, USA (2016)

    Google Scholar 

  36. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why Should I Trust You?”: Explaining the predictions of any classifier. In: KDD 2016, pp. 1135–1144, ACM, San Francisco, CA (2016)

    Google Scholar 

  37. Lundberg, S., Lee, S.-I.: A unified approach to interpreting model predictions. In: NIPS 2017, pp. 4765–4774, Curran Associates, Long Beach, CA, USA (2017)

    Google Scholar 

  38. Weerts, H.J.P., van Ipenburg, W., Pechenizkiy, M.: A human-grounded evaluation of SHAP for alert processing. In: Proceedings of the KDD Workshop on Explainable AI, Anchorage, AK (2019)

    Google Scholar 

  39. Fernandez, C., Provost, F., Han, X.: Counterfactual explanations for data-driven decisions. In: ICIS 2019, AIS, Munich, Germany (2019)

    Google Scholar 

  40. Förster, M., Klier, M., Kluge, K., Sigler, I.: Evaluating explainable artificial intelligence – what users really appreciate. In: ECIS 2020, AIS (2020)

    Google Scholar 

  41. Burdisso, S.G., Errecalde, M., Montes-y-Gómez, M.: t-SS3: a text classifier with dynamic n-grams for early risk detection over text streams. arXiv:1911.06147 (2019)

  42. Gedikli, F., Jannach, D., Ge, M.: How should I explain? A comparison of different explanation types for recommender systems. Int. J. Hum. Comput. Stud. 72, 367–382 (2014)

    Article  Google Scholar 

  43. Ribera, M., Lapedriza, A.: Can we do better explanations? A proposal of user-centered explainable AI. In: Joint Proceedings of the ACM IUI 2019 Workshop, ACM, Los Angeles, CA (2019)

    Google Scholar 

  44. Bhatt, U., et al.: Explainable machine learning in deployment. In: FAT*20, pp. 648–657, ACM, Barcelona, Spain (2020)

    Google Scholar 

  45. Verheij, B., Wiering, M. (eds.): BNAIC 2017. CCIS, vol. 823. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76892-2

    Book  Google Scholar 

  46. Kaufmann, E., Kalyanakrishnan, S.: Information complexity in bandit subset selection. J. Mach. Learn. Res. 30, 228–251 (2013)

    Google Scholar 

  47. Venable, J., Pries-Heje, J., Baskerville, R.: FEDS: a framework for evaluation in design science research. Eur. J. Inf. Syst. 25, 77–89 (2016)

    Article  Google Scholar 

  48. Doshi-Velez, F., Kim, B.: Considerations for evaluation and generalization in interpretable machine learning. In: Escalante, H.J., et al. (eds.) Explainable and Interpretable Models in Computer Vision and Machine Learning. TSSCML, pp. 3–17. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98131-4_1

    Chapter  Google Scholar 

Download references

Acknowledegments

We kindly thank Rakesh M. Verma (University of Houston) for providing us the dataset.

Author information

Authors and Affiliations

  1. Institute of Business Analytics, University of Ulm, Ulm, Germany

    Kilian Kluge & Regina Eckhardt

Authors
  1. Kilian Kluge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Regina Eckhardt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kilian Kluge .

Editor information

Editors and Affiliations

  1. Faculty of Business Administration and Economics, University of Duisburg-Essen, Essen, Germany

    Frederik Ahlemann

  2. Faculty of Business Administration and Economics, University of Duisburg-Essen, Essen, Germany

    Reinhard Schütte

  3. Department of Computer Science and Applied Cognitive Science, University of Duisburg-Essen, Duisburg, Germany

    Stefan Stieglitz

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kluge, K., Eckhardt, R. (2021). Explaining the Suspicion: Design of an XAI-Based User-Focused Anti-Phishing Measure. In: Ahlemann, F., Schütte, R., Stieglitz, S. (eds) Innovation Through Information Systems. WI 2021. Lecture Notes in Information Systems and Organisation, vol 47. Springer, Cham. https://doi.org/10.1007/978-3-030-86797-3_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86797-3_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86796-6

  • Online ISBN: 978-3-030-86797-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation