Skip to main content
SpringerLink
Log in

Auditing and Testing AI – A Holistic Framework

  • Conference paper
  • First Online:
Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Health, Operations Management, and Design (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13320))

Included in the following conference series:

Abstract

This paper describes a framework that can be used to assess and analyze AI systems in terms of risk. The framework addresses the structure and components of AI systems at five layers and allows taking a holistic view of AI systems while focusing on specific aspects, such as discrimination or data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zweig, K., Hauer, M., Raudonat, F.: Anwendungsszenarien: KI-Systeme im Personal- und Talentmanagement, ExamAI – KI Testing & Auditing. Gesellschaft für Informatik, Berlin (2020)

    Google Scholar 

  2. Waltl, B., Vogl, R.: Explainable artificial intelligence – the new frontier in legal informatics. Jusletter IT 4, 1–10 (2018)

    Google Scholar 

  3. Broy, M., Kuhrmann, M.: Einführung in die Softwaretechnik, Springer, Heidelberg https://doi.org/10.1007/978-3-662-50263 (2021)

  4. Why AI is the future of growth, Accenture, 2016. The economic impact of the automation of knowledge work, robots and self-driving vehicles could reach between EUR 6.5 and EUR 12 trillion annually by 2025 (including improved productivity and higher quality of life in ageing populations). Source: Disruptive technologies: Advances that will transform life, business, and the global economy, McKinsey Global Institute (2013)

    Google Scholar 

  5. AI is part of the Commission's strategy to digitise industry (COM(2016) 180 final) and a renewed EU Industrial Policy Strategy (COM(2017) 479 final)

    Google Scholar 

  6. Russel, S., Norvig, P.: Artificial intelligence: a modern approach (2002)

    Google Scholar 

  7. Handelsblatt. Kartellamt rügt Lufthansa: Solche Algorithmen werden ja nicht vom lieben Gott geschrieben. https://www.handelsblatt.com/unternehmen/handel-konsumgueter/kartellamt-ruegt-lufthansa-solche-algorithmen-werden-ja-nicht-vom-lieben-gott-geschrieben/20795072.html. Accessed 28 Dec 2017

  8. Waltl, B., Vogl, R.: Increasing transparency in algorithmic- decision-making with explainable AI. Datenschutz und Datensicherheit - DuD 42(10), 613–617 (2018). https://doi.org/10.1007/s11623-018-1011-4

    Article  Google Scholar 

  9. Nguyen, G., et al.: Machine learning and deep learning frameworks and libraries for large-scale data mining: a survey. Artifi. Intell. Rev. 52(1), 77–124 (2019)

    Article  Google Scholar 

  10. Došilović, F.K., Brčić, M., Hlupić, Nikica.: Explainable artificial intelligence: a survey. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). IEEE (2018)

    Google Scholar 

  11. Molnar, C.: Interpretable machine learning. A Guide for Making Black Box Models Explainable. https://christophm.github.io/interpretable-ml-book/ (2019)

  12. European Commission, Assessment List for Trustworthy Artificial Intelligence (ALTAI) for self-assessment, 2021, c.f. https://digital-strategy.ec.europa.eu/en/library/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment

  13. Schelter, S., et al.: Automatically tracking metadata and provenance of machine learning experiments. Machine Learning Systems Workshop at NIPS (2017)

    Google Scholar 

  14. Comptroller of the Currency Administrator of National Bank Internal and External Audits: Comptrollers Handbook. https://web.archive.org/web/20101107160153/http://www.ffiec.gov/ffiecinfobase/resources/audit/occ-hb-internal_external_audits-intro.pdf (2003)

  15. Jöckel, L., et al.: Towards a Common Testing Terminology for Software Engineering and Artificial Intelligence Experts. arXiv preprint arXiv:2108.13837 (2021)

  16. Bundesministerium für Arbeit und Soziales KI in der Arbeitswelt: Potenziale erkennen, Transparenz schaffen, Zugriff am 13.10.2021. https://www.bmas.de/DE/Europa-und-die-Welt/Europa/MySocialEurope-Deutsche-Ratspraesidentschaft/Meldungen/ki-in-der-arbeitswelt.html

  17. IEEE Standard for Software Reviews and Audits, in IEEE Std 1028–2008, pp.1–53. https://doi.org/10.1109/IEEESTD.2008.4601584. Accessed 15 Aug 2008

Download references

Author information

Authors and Affiliations

  1. Gesellschaft für Informatik e.V. (GI), Bonn, Germany

    Nikolas Becker & Bernhard Waltl

Authors
  1. Nikolas Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernhard Waltl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolas Becker .

Editor information

Editors and Affiliations

  1. School of Industrial Engineering, Purdue University, West Lafayette, IN, USA

    Vincent G. Duffy

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Becker, N., Waltl, B. (2022). Auditing and Testing AI – A Holistic Framework. In: Duffy, V.G. (eds) Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Health, Operations Management, and Design. HCII 2022. Lecture Notes in Computer Science, vol 13320. Springer, Cham. https://doi.org/10.1007/978-3-031-06018-2_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06018-2_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06017-5

  • Online ISBN: 978-3-031-06018-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation