Skip to main content
SpringerLink
Log in

UAV Operators Workload Assessment by Optical Brain Imaging Technology (fNIR)

  • Reference work entry
  • First Online:
Handbook of Unmanned Aerial Vehicles

Abstract

The use of unmanned aerial vehicles (UAVs) is expected to increase exponentially over the next few years. UAV ground operators are required to acquire skills quickly and completely, with a level of expertise that builds the operator’s confidence in his/her ability to control the UAV under adverse conditions. As UAVs are held to increasingly higher standards of efficiency and safety, operators are routinely required to perform more informationally dense and cognitively demanding tasks, resulting in increased cognitive workloads during operation. Functional brain monitoring offers the potential to help UAV operators meet these challenges. Recent research has demonstrated the utility of near- infrared-based functional brain imaging systems (fNIRs) for the purpose of monitoring frontal cortical areas that support executive functions (attention, working memory, response monitoring). This technology provides portable, safe, affordable, noninvasive, and minimally intrusive monitoring systems with rapid application times for continuous measures of cortical activity. fNIR technology allows continuous monitoring of operators during training as they develop expertise, as well as the capacity to monitor their cognitive workload under operational conditions while controlling UAVs in critical missions. This chapter discusses the utilization of fNIR in the monitoring of a cognitive workload during UAV operation, and as an objective measure of expertise development, that is, the transition from novice to expert during operator training.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 1,399.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 1,999.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • A. Ames III, CNS energy metabolism as related to function. Brain Res. Rev. 34(1–2), 42–68 (2000). doi: 10.1016/s0165-0173(00)00038-2

    Google Scholar 

  • H. Ayaz, M. Izzetoglu, S.M. Platek, S. Bunce, K. Izzetoglu, K. Pourrezaei, B. Onaral, Registering fNIR data to brain surface image using MRI templates. Conference Proceedings - IEEE Engineering in Medicine and Biology Society 2671–2674 (2006). doi: 10.1109/IEMBS.2006.260835

    Google Scholar 

  • H. Ayaz, P.A. Shewokis, A. Curtin, M. Izzetoglu, K. Izzetoglu, B. Onaral, Using MazeSuite and functional near infrared spectroscopy to study learning in spatial navigation. J. Vis. Exp. (56), e3443 (2011). doi:10.3791/3443

    Google Scholar 

  • H. Ayaz, P.A. Shewokis, S. Bunce, K. Izzetoglu, B. Willems, B. Onaral, Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59(1), 36–47 (2012). doi:10.1016/j. neuroimage.2011.06.023

    Article  Google Scholar 

  • M. Beauchamp, A. Dagher, J. Aston, J. Doyon, Dynamic functional changes associated with cognitive skill learning of an adapted version of the tower of London task. Neuroimage 20(3), 1649–1660 (2003)

    Article  Google Scholar 

  • Y. Benjamini, Y. Hochberg, Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B (Methodological) 57(1), 289–300 (1995)

    MATH  MathSciNet  Google Scholar 

  • D. Boas, J. Culver, J. Stott, A. Dunn, Three dimensional Monte Carlo code for photon migration through complex heterogeneous media including the adult human head. Opt. Exp. 10(3), 159–170 (2002)

    Article  Google Scholar 

  • S.C. Bunce, M. Izzetoglu, K. Izzetoglu, B. Onaral, K. Pourrezaei, Functional near-infrared spectroscopy: an emerging neuroimaging modality. IEEE Eng. Med. Biol. Mag. 25(4), 54–62 (2006)

    Article  Google Scholar 

  • R.B. Buxton, K. Uludag, D.J. Dubowitz, T.T. Liu, Modeling the hemodynamic response to brain activation. Neuroimage 23, S220–S233 (2004)

    Article  Google Scholar 

  • B. Cain, A review of the mental workload literature (vol. RTO-TR-HFM-121-Part-II): DTIC document (2007)

    Google Scholar 

  • B. Chance, Z. Zhuang, C. UnAh, C. Alter, L. Lipton, Cognition-activated low-frequency modulation of light absorption in human brain. Proc. Natl. Acad. Sci. USA 90(8), 3770–3774 (1993)

    Article  Google Scholar 

  • B. Chance, E. Anday, S. Nioka, S. Zhou, L. Hong, K. Worden, R. Thomas, A novel method for fast imaging of brain function, non-invasively, with light. Opt. Exp. 2(10), 411–423 (1998)

    Article  Google Scholar 

  • J. Cooper, M.A. Goodrich, Towards combining uav and sensor operator roles in uav-enabled visual search, in Proceedings of ACM/IEEE International Conference on Human-Robot Interaction, Amsterdam, The Netherlands, 2008

    Google Scholar 

  • M. Cope, The development of a near infrared spectroscopy system and its application for non invasive monitoring of cerebral blood and tissue oxygenation in the newborn infant. Ph.D. thesis, University College London, London, 1991

    Google Scholar 

  • M. Cope, D. Delpy, System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination. Med. Biol. Eng. Comput. 26(3), 289–294 (1988)

    Article  Google Scholar 

  • M. Cope, D.T. Delpy, E.O.R. Reynolds, S. Wray, J. Wyatt, P. Van der Zee, Methods of quantitating cerebral near infrared spectroscopy data. Adv. Exp. Med. Biol. 222, 183–189 (1988)

    Article  Google Scholar 

  • J.L. Drury, S.D. Scott, Awareness in unmanned aerial vehicle operations. Int. C2 J. 1, 1–10 (2008)

    Google Scholar 

  • J.L. Drury, J. Richer, N. Rackliffe, M.A. Goodrich, Comparing situation awareness for two unmanned aerial vehicle human interface approaches, in Proceedings of the Conference on Safety, Security, and Rescue Robotics (SSRR), Gaithersburg, MD, 2006

    Google Scholar 

  • J.B. Fishkin, E. Gratton, Propagation of photon-density waves in strongly scattering media containing an absorbing semi-infinite plane bounded by a straight edge. JOSA A 10(1), 127–140 (1993)

    Article  Google Scholar 

  • H. Garavan, D. Kelley, A. Rosen, S.M. Rao, E.A. Stein, Practice-related functional activation changes in a working memory task. Microsc. Res. Tech. 51(1), 54–63 (2000)

    Article  Google Scholar 

  • A. Gevins, M. Smith, Neurophysiological measures of cognitive workload during human-computer interaction. Theor. Issue Ergon. Sci. 4(1), 113–131 (2003)

    Article  Google Scholar 

  • A. Gevins, M. Smith, L. McEvoy, D. Yu, High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. Cereb. Cortex 7(4), 374 (1997)

    Article  Google Scholar 

  • E. Gratton, V. Toronov, U. Wolf, M. Wolf, A. Webb, Measurement of brain activity by near-infrared light. J. Biomed. Opt. 10(1), 11008 (2005)

    Article  Google Scholar 

  • P. Hancock, M.H. Chignell, Toward a theory of mental work load: stress and adaptability in human machine systems, in Proceedings of the International IEEE Conference on Systems, Man and Cybernetics, Atlanta, GA, 1986

    Google Scholar 

  • PA. Hancock, P.A. Desmond, Stress, Workload, and Fatigue (Lawrence Erlbaum, Mahwah, 2001)

    Google Scholar 

  • A.J. Hansen, M. Nedergaard, Brain ion homeostasis in cerebral ischemia. Mol. Chem. Neuropathol. 9(1), 195–209 (1988)

    Google Scholar 

  • S. Hart, L. Staveland, Development of NASA-TLX (task load index): results of empirical and theoretical research. Hum. Ment. Workload 1, 139–183 (1988)

    Article  Google Scholar 

  • J.T. Hing, P.Y. Oh, Development of an unmanned aerial vehicle piloting system with integrated motion cueing for training and pilot evaluation. J. Intell. Robot. Syst. 54(1), 3–19 (2009)

    Article  Google Scholar 

  • J.T. Hing, J. Menda, K. Izzetoglu, P.Y. Oh, An indoor study to evaluate a mixed-reality interface for unmanned aerial vehicle operations in near earth environments. Int. J. Intell. Control Syst. 16(2), 132–141 (2011)

    Google Scholar 

  • Y. Hoshi, M. Tamura, Near-infrared optical detection of sequential brain activation in the prefrontal cortex during mental tasks. Neuroimage 5(4 Pt 1), 292–297 (1997)

    Article  Google Scholar 

  • K. Izzetoglu, S. Bunce, B. Onaral, K. Pourrezaei, B. Chance, Functional optical brain imaging using near-infrared during cognitive tasks. Int. J. Hum. Comput. Interact. 17(2), 211–227 (2004)

    Article  Google Scholar 

  • M. Izzetoglu, K. Izzetoglu, S. Bunce, H. Ayaz, A. Devaraj, B. Onaral, K. Pourrezaei, Functional near-infrared neuroimaging. IEEE Trans. Neural. Syst. Rehabil. Eng. 13(2), 153–159 (2005)

    Article  Google Scholar 

  • K. Izzetoglu, H. Ayaz, A.C. Merzagora, M. Izzetoglu, P.A. Shewokis, S. Bunce, B. Onaral, The evolution of field deployable fNIR spectroscopy from bench to clinical settings. J. Innov. Opt. Health Sci. 4(3), 239–250 (2011)

    Article  Google Scholar 

  • F.F. Jobsis, Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science 198(4323), 1264–1267 (1977)

    Article  Google Scholar 

  • D.B. Kaber, E. Onal, M.R. Endsley, Design of automation for telerobots and the effect on performance, operator situation awareness, and subjective workload. Hum. Factor Ergon. Manuf. 10(4), 409–430 (2000)

    Article  Google Scholar 

  • A. Kelly, H. Garavan, Human functional neuroimaging of brain changes associated with practice. Cereb. Cortex 15(8), 1089 (2005)

    Article  Google Scholar 

  • K.K. Kwong, J.W. Belliveau, D.A. Chesler, I.E. Goldberg, R.M. Weisskoff, B.P. Poncelet, R. Turner, Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. Proc. Natl. Acad. Sci. 89(12), 5675 (1992)

    Article  Google Scholar 

  • J. Menda, J.T. Hing, H. Ayaz, P.A. Shewokis, K. Izzetoglu, B. Onaral, P. Oh, Optical brain imaging to enhance UAV operator training, evaluation, and interface development. J. Intell. Robot. Syst. 61(1–4), 423–443 (2010). doi:10.1007/s10846-010-9507-7

    Google Scholar 

  • N. Meshkati, P.A. Hancock, M. Rahimi, S.M. Dawes, Techniques of mental workload assessment, in Evaluation of Human Work: A Practical Ergonomics Methodology, 2nd edn., ed. by J. Wilson, E.N. Corlett (Taylor and Francis, London, 1995)

    Google Scholar 

  • J.G. Milton, S.S. Small, A. Solodkin, On the road to automatic: dynamic aspects in the development of expertise. J. Clin. Neurophys. 21(3), 134 (2004)

    Article  Google Scholar 

  • S. Ogawa, T. Lee, A. Kay, D. Tank, Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. 87(24), 9868 (1990)

    Article  Google Scholar 

  • R.C. Oldfield, The assessment and analysis of handedness: the Edinburgh inventory. Neuropsy-chologia 9(1), 97–113 (1971)

    Article  Google Scholar 

  • R. Parasuraman, M. Rizzo, Neuroergonomics: The Brain at Work (Oxford University Press, Oxford, 2007)

    Google Scholar 

  • R. Parasuraman, G. Wilson, Putting the brain to work: neuroergonomics past, present, and future. Hum. Factor 50(3), 468 (2008)

    Article  Google Scholar 

  • M.S. Patterson, B. Chance, B.C. Wilson, Time resolved reflectance and transmittance for the noninvasive measurement of tissue optical properties. Appl. Opt. 28(12), 2331–2336 (1989)

    Article  Google Scholar 

  • R.A. Poldrack, A.D. Wagner, M.W. Prull, J.E. Desmond, G.H. Glover, J.D.E. Gabrieli, Functional specialization for semantic and phonological processing in the left inferior Prefrontal Cortex* 1. Neuroimage 10(1), 15–35 (1999)

    Article  Google Scholar 

  • C. E. Rash, P.A. LeDuc, S.D. Manning, Human factors in U.S. military unmanned aerial vehicle accidents, in Human Factors of Remotely Operated Vehicles, vol. 7, ed. by N.J. Cooke, H.L. Pringle, H.K. Pedersen, O. Connor. Advances in Human Performance and Cognitive Engineering Research (Emerald Group Publishing Limited, Boston, 2006), pp. 117–131

    Chapter  Google Scholar 

  • M.W. Scerbo, F.G. Freeman, P.J. Mikulka, A brain-based system for adaptive automation. Theor. Issue Ergon. Sci. 4, 1(2), 200–219 (2003)

    Article  Google Scholar 

  • A.K. Singh, I. Dan, Exploring the false discovery rate in multichannel NIRS. Neuroimage 33(2), 542–549 (2006)

    Article  Google Scholar 

  • G. Strangman, D.A. Boas, J.P. Sutton, Non-invasive neuroimaging using near-infrared light. Biol. Psychiatry 52(7), 679–693 (2002)

    Article  Google Scholar 

  • A. Villringer, B. Chance, Non-invasive optical spectroscopy and imaging of human brain function. Trends Neurosci. 20(10), 435–442 (1997)

    Article  Google Scholar 

  • G.F. Wilson, C.A. Russell, Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum. Factor 45(4), 635 (2003)

    Article  Google Scholar 

Download references

Acknowledgments

The U.S. Army Medical Research Acquisition Activity, 820 Chandler Street, Fort Detrick, MD 21702-5014 is the awarding and administering acquisition office. This investigation was funded under a U.S. Army Medical Research Acquisition Activity, Cooperative Agreement W81XWH-08-2-0573 and in part by W81XWH-09-2-0104. The content of the information herein does not necessarily reflect the position or the policy of the U.S. Government or the U.S. Army, and no official endorsement should be inferred.

The authors would also like to thank Justin Menda and Adrian Curtin for conducting experiments and part of data analyses.

Author information

Authors and Affiliations

  1. School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, USA

    Kurtulus Izzetoglu, Hasan Ayaz & Banu Onaral

  2. Department of Mechanical Engineering and Mechanics, Drexel Autonomous Systems Lab, Drexel University, Philadelphia, PA, USA

    James T. Hing

  3. College of Nursing and Health Professions and Schoolof Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, USA

    Patricia A. Shewokis

  4. Department of Psychiatry, M.S. Hershey Medical Center, Penn State University College of Medicine, Hershey, PA, USA

    Scott C. Bunce

  5. Formerly with Drexel University, Currently: University of Nevada, Las Vegas, NV, USA

    Paul Oh

Authors
  1. Kurtulus Izzetoglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hasan Ayaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James T. Hing
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patricia A. Shewokis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Scott C. Bunce
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Paul Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Banu Onaral
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Kurtulus Izzetoglu , Hasan Ayaz , James T. Hing , Patricia A. Shewokis , Scott C. Bunce , Paul Oh or Banu Onaral .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Denver, Denver, Colorado, USA

    Kimon P. Valavanis

  2. School of Electrical and Computer Engineering, The Georgia Institute of Technology, Atlanta, Georgia, USA

    George J. Vachtsevanos

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media Dordrecht

About this entry

Cite this entry

Izzetoglu, K. et al. (2015). UAV Operators Workload Assessment by Optical Brain Imaging Technology (fNIR). In: Valavanis, K., Vachtsevanos, G. (eds) Handbook of Unmanned Aerial Vehicles. Springer, Dordrecht. https://doi.org/10.1007/978-90-481-9707-1_22

Download citation

Publish with us

Policies and ethics

Search

Navigation