Skip to main content
SpringerLink
Log in

Design Principles for AI-Assisted Attention Aware Systems in Human-in-the-Loop Safety Critical Applications

  • Chapter
  • First Online:
Engineering Artificially Intelligent Systems

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13000))

  • 866 Accesses

Abstract

AI-assisted, attentiona-ware systems support operators in detecting and managing targets present in visual scenes. Such a system typically attempts to automatically identify targets of interest and increase the probability that an operator can detect them by, for example, modifying their visual saliency in the visual scene. Applications of AI-assisted attention awareness include air-traffic control, submarine de-mining and armored vehicle situational awareness. This chapter explains the key human-machine challenges intrinsic in this design problem and distills six design principles based on a functional design of a general AI-assisted attention-aware system for target identification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baddeley, A.D., Weiskrantz, L.E.: Attention: Selection, Awareness, and Control: A Tribute to Donald Broadbent. Clarendon Press/Oxford University Press, New York (1993)

    Google Scholar 

  2. Bailey, B.P., Konstan, J.A.: On the need for attention-aware systems: measuring effects of interruption on task performance, error rate, and affective state. Comput. Hum. Behav. 22(4), 685–708 (2006)

    Article  Google Scholar 

  3. Bartram, L., Ware, C., Calvert, T.: Moticons: detection, distraction and task. Int. J. Hum. Comput. Stud. 58(5), 515–545 (2003)

    Google Scholar 

  4. Chiappe, D.L., Strybel, T.Z., Vu, K.P.L.: Mechanisms for the acquisition of situation awareness in situated agents. Theoret. Iss. Ergon. Sci. 13(6), 625–647 (2012)

    Article  Google Scholar 

  5. Cutrell, E., Czerwinski, M., Horvitz, E.: Notification, disruption, and memory: Effects of messaging interruptions on memory and performance. In: Proceedings of Interact. pp. 263–269 (2001)

    Google Scholar 

  6. Czerwinski, M., Cutrell, E., Horvitz, E.: Instant messaging: effects of relevance and timing. In: People and Computers XIV: Proceedings of HCI 2000, vol. 2, pp. 71–76 (2000)

    Google Scholar 

  7. Desimone, R., Duncan, J.: Neural mechanisms of selective visual attention. Ann. Rev. Neurosci. 18(1), 193–222 (1995)

    Article  Google Scholar 

  8. D’Mello, S., Olney, A., Williams, C., Hays, P.: Gaze tutor: a gaze-reactive intelligent tutoring system. Int. J. Hum-Comput. Stud. 70(5), 377–398 (2012)

    Article  Google Scholar 

  9. Dostal, J., Hinrichs, U., Kristensson, P.O., Quigley, A.: Spidereyes: designing attention-and proximity-aware collaborative interfaces for wall-sized displays. In: Proceedings of the 19th International Conference on Intelligent User Interfaces, pp. 143–152 (2014)

    Google Scholar 

  10. Dostal, J., Kristensson, P.O., Quigley, A.: Subtle gaze-dependent techniques for visualising display changes in multi-display environments. In: Proceedings of the International Conference on Intelligent User Interfaces, pp. 137–148 (2013)

    Google Scholar 

  11. Driver, J.: A selective review of selective attention research from the past century. Br. J. Psychol. 92(1), 53–78 (2001)

    Article  MathSciNet  Google Scholar 

  12. Einstein, G.O., McDaniel, M.A.: Retrieval processes in prospective memory: theoretical approaches and some new empirical findings. In: Brandimonte, M., Einstein, G.O., McDaniel, M.A. (eds.) Prospective Memory: Theory and Applications, pp. 115–141. Lawrence Erlbaum Associates Publishers (1996)

    Google Scholar 

  13. Endsley, M.R.: Measurement of situation awareness in dynamic systems. Hum. Fact. 37(1), 65–84 (1995)

    Article  Google Scholar 

  14. Endsley, M.R.: Situation awareness misconceptions and misunderstandings. J. Cogn. Eng. Decis. Making 9(1), 4–32 (2015)

    Article  Google Scholar 

  15. Fogarty, J., Hudson, S.E., Atkeson, C.G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J.C., Yang, J.: Predicting human interruptibility with sensors. ACM Trans. Comput.-Hum. Inter. 12(1), 119–146 (2005)

    Article  Google Scholar 

  16. Franconeri, S.L., Simons, D.J.: Moving and looming stimuli capture attention. Percep. Psychophys. 65(7), 999–1010 (2003)

    Article  Google Scholar 

  17. Garrido, J.E., Penichet, V.M., Lozano, M.D., Quigley, A., Kristensson, P.O.: Awtoolkit: attention-aware user interface widgets. In: Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, pp. 9–16 (2014)

    Google Scholar 

  18. Gellersen, H.W., Schmidt, A., Beigl, M.: Multi-sensor context-awareness in mobile devices and smart artifacts. Mobile Netw. Appl. 7(5), 341–351 (2002)

    Article  Google Scholar 

  19. Healey, C.G., Booth, K.S., Enns, J.T.: High-speed visual estimation using preattentive processing. ACM Trans. Comput.-Hum. Inter. 3(2), 107–135 (1996)

    Article  Google Scholar 

  20. Imbert, J.P., Hodgetts, H.M., Parise, R., Vachon, F., Dehais, F., Tremblay, S.: Attentional costs and failures in air traffic control notifications. Ergonomics 57(12), 1817–1832 (2014)

    Article  Google Scholar 

  21. Iqbal, S.T., Bailey, B.P.: Effects of intelligent notification management on users and their tasks. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 93–102 (2008)

    Google Scholar 

  22. Lavie, N., Hirst, A., De Fockert, J.W., Viding, E.: Load theory of selective attention and cognitive control. J. Experiment. Psychol. Gen.l 133(3), 339 (2004)

    Article  Google Scholar 

  23. Lavie, N., Tsal, Y.: Perceptual load as a major determinant of the locus of selection in visual attention. Percep. Psychophys. 56(2), 183–197 (1994)

    Article  Google Scholar 

  24. Lopez-Tovar, H., Charalambous, A., Dowell, J.: Managing smartphone interruptions through adaptive modes and modulation of notifications. In: Proceedings of the 20th International Conference on Intelligent User Interfaces, pp. 296–299 (2015)

    Google Scholar 

  25. Marquardt, N., Diaz-Marino, R., Boring, S., Greenberg, S.: The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 315–326 (2011)

    Google Scholar 

  26. Mehrotra, A., Musolesi, M., Hendley, R., Pejovic, V.: Designing content-driven intelligent notification mechanisms for mobile applications. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 813–824 (2015)

    Google Scholar 

  27. Morris, N.M., Rouse, W.B., Frey, P.R.: Adaptive aiding for symbiotic human-computer control: conceptual model and experimental approach. Tech. rep., Search Technology Inc., Norcross (1985)

    Google Scholar 

  28. Nicosia, L., Kristensson, P.: Inattention-management middleware for human-in-the-loop multi-display applications. In: Proceedings of the IEEE Workshop on Human-Centered Computational Sensing (2018)

    Google Scholar 

  29. Nicosia, M., Kristensson, P.O.: A conceptual design of an inattention management middleware with adaptive target saliency. In: 2020 IEEE Aerospace Conference, pp. 1–11 (2020)

    Google Scholar 

  30. Parasuraman, R., Davies, D.R.: Varieties of attention. Academic Pr (1984)

    Google Scholar 

  31. Parasuraman, R., Molloy, R., Singh, I.L.: Performance consequences of automation-induced’complacency’. Int. J. Aviat. Psychol. 3(1), 1–23 (1993)

    Article  Google Scholar 

  32. Parasuraman, R., Riley, V.: Humans and automation: use, misuse, disuse, abuse. Hum. fact. 39(2), 230–253 (1997)

    Article  Google Scholar 

  33. Posner, M.I.: Orienting of attention. Q. J. Experiment. Psychol. 32(1), 3–25 (1980)

    Article  MathSciNet  Google Scholar 

  34. Posner, M.I.: Cumulative development of attentional theory. Am. Psychol. 37(2), 168 (1982)

    Article  Google Scholar 

  35. Roda, C., Nabeth, T.: Supporting attention in learning environments: attention support services, and information management. In: Creating New Learning Experiences on a Global Scale, pp. 277–291 (2007)

    Google Scholar 

  36. Roda, C., Thomas, J.: Attention aware systems: theories, applications, and research agenda. Comput. Hum. Behav. 22(4), 557–587 (2006)

    Article  Google Scholar 

  37. Rouse, W.B.: Adaptive aiding for human/computer control. Hum. Fact. 30(4), 431–443 (1988)

    Article  Google Scholar 

  38. Simons, D.J., Chabris, C.F.: Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception 28(9), 1059–1074 (1999)

    Google Scholar 

  39. Simons, D.J., Rensink, R.A.: Change blindness: past, present, and future. Trends Cogn. Sci. 9(1), 16–20 (2005)

    Article  Google Scholar 

  40. Vallières, B.R., Hodgetts, H.M., Vachon, F., Tremblay, S.: Supporting dynamic change detection: using the right tool for the task. Cogn. Res. Princ. Implicat. 1(1), 1–20 (2016). https://doi.org/10.1186/s41235-016-0033-4

    Article  Google Scholar 

  41. Wilson, M.D., Farrell, S., Visser, T.A., Loft, S.: Remembering to execute deferred tasks in simulated air traffic control: the impact of interruptions. J. Exp. Psychol. Appl. 24(3), 360 (2018)

    Article  Google Scholar 

  42. Yantis, S.: Goal-directed and stimulus-driven determinants of attentional control. Attent. Perform. 18, 73–103 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, UK

    Max Nicosia & Per Ola Kristensson

Authors
  1. Max Nicosia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Per Ola Kristensson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Max Nicosia .

Editor information

Editors and Affiliations

  1. Paine College, Augusta, GA, USA

    William F. Lawless

  2. University at Buffalo, Director Emeritus, Center for Multi-source Information Fusion, Buffalo, NY, USA

    James Llinas

  3. U.S. Naval Research Laboratory, Washington, DC, USA

    Donald A. Sofge

  4. U.S. Naval Research Laboratory, Washington, DC, USA

    Ranjeev Mittu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Nicosia, M., Kristensson, P.O. (2021). Design Principles for AI-Assisted Attention Aware Systems in Human-in-the-Loop Safety Critical Applications. In: Lawless, W.F., Llinas, J., Sofge, D.A., Mittu, R. (eds) Engineering Artificially Intelligent Systems. Lecture Notes in Computer Science(), vol 13000. Springer, Cham. https://doi.org/10.1007/978-3-030-89385-9_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89385-9_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89384-2

  • Online ISBN: 978-3-030-89385-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation