Brian P. Bailey
Skip Abstract Section
Abstract
Notifications can have reduced interruption cost if delivered at moments of lower mental workload during task execution. Cognitive theorists have speculated that these moments occur at subtask boundaries. In this article, we empirically test this speculation by examining how workload changes during execution of goal-directed tasks, focusing on regions between adjacent chunks within the tasks, that is, the subtask boundaries. In a controlled experiment, users performed several interactive tasks while their pupil dilation, a reliable measure of workload, was continuously measured using an eye tracking system. The workload data was extracted from the pupil data, precisely aligned to the corresponding task models, and analyzed. Our principal findings include (i) workload changes throughout the execution of goal-directed tasks; (ii) workload exhibits transient decreases at subtask boundaries relative to the preceding subtasks; (iii) the amount of decrease tends to be greater at boundaries corresponding to the completion of larger chunks of the task; and (iv) different types of subtasks induce different amounts of workload. We situate these findings within resource theories of attention and discuss important implications for interruption management systems.
- Adamczyk, P. D. and Bailey, B. P. 2004. If not now when? The effects of interruptions at different moments within task execution. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 271--278. Google Scholar
Digital Library
- Anttonen, J. and Surakka, V. 2005. Emotions and heart rate while sitting on a chair. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 491--499. Google ScholarDigital Library
- Bailey, B. P., Adamczyk, P. D., Chang, T. Y., and Chilson, N. A. 2006. A framework for specifying and monitoring user tasks. J. Comput. Human Behav. 22, 4, 685--708.Google ScholarCross Ref
- Bailey, B. P. and Konstan, J. A. 2006. On the need for attention aware systems: Measuring effects of interruption on task performance, error rate, and affective state. J. Comput. Human Behav. 22, 4, 709--732.Google ScholarCross Ref
- Beatty, J. 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psych. Bull. 91, 2, 276--292.Google ScholarCross Ref
- Bradshaw, J. L. 1967. Pupil size as a measure of arousal during information processing. Nature 216, 515--516.Google ScholarCross Ref
- Card, S., Moran, T., and Newell, A. 1983. The psychology of human-computer interaction. Lawrence Erlbaum Associates, Hillsdale. Google ScholarDigital Library
- Chen, D. and Vertegaal, R. 2004. Using mental load for managing interruptions in a physiologically attentive user interface. In Proceedings of ACM Conference on Human Factors in Computing Systems. ACM, New York, 1513--1516. Google ScholarDigital Library
- Cutrell, E., Czerwinski, M., and Horvitz, E. 2001. Notification, disruption and memory: Effects of messaging interruptions on memory and performance. In Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction (Tokyo, Japan). 263--269.Google Scholar
- Czerwinski, M., Cutrell, E., and Horvitz, E. 2000a. Instant messaging and interruption: Influence of task type on performance. In Proceedings of the Annual Conference of the Human Factors and Ergonomics Society of Australia (OZCHI) (Sydney, Australia). C. Paris, N. Ozkan, S. Howard and S. Lu, Eds, 356--361.Google Scholar
- Czerwinski, M., Cutrell, E., and Horvitz, E. 2000b. Instant messaging: Effects of relevance and timing. In People and Computers XIV: Proceedings of HCI, S. Turner and P. Turner, Eds, 71--76.Google Scholar
- Dabbish, L. and Kraut, R. E. 2004. Controlling interruptions: Awareness displays and social motivation for coordination. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. ACM, New York, 182--191. Google ScholarDigital Library
- Degani, A. and Wiener, E. 1993. Cockpit checklists: Concepts, design, and use. Human Factors 35, 2, 345--359.Google ScholarCross Ref
- Dey, A. K. and Abowd, G. D. 2000. CybreMinder: A context-aware system for supporting reminders. In Proceedings of 2nd International Symposium on Handheld and Ubiquitous Computing, 172--186. Google ScholarDigital Library
- Dismukes, K., Young, G., and Sumwalt, R. 1998. Cockpit interruptions and distractions. ASRS Direct. 10.Google Scholar
- Donchin, E., Kramer, A. F., and Wickens, C. D. 1986. Applications of brain event related potentials to problems in engineering psychology. Guildford, New York.Google Scholar
- Fogarty, J., Ko, A. J., Aung, H. H., Golden, E., Tang, K. P., and Hudson, S. E. 2005. Examining task engagement in sensor-based statistical models of human interruptibility. In Proceeding of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 331--340. Google ScholarDigital Library
- Gale, A. and Edwards, J. 1983. The EEG and human behavior. Academic Press, New York.Google Scholar
- Gevins, A. and Schaffer, R. 1980. A critical review of electroencephalographic (EEG) correlates of higher cortical functions. CRT Crit. Rev. Bioeng. 4, 113--164.Google Scholar
- Guerlain, S. and Willis, R. 2001. The tactical tomahawk weapons control system: Operator interface design project. In Human Systems Integration Symposium.Google Scholar
- Hart, S. G. and Staveland, L. E. 1988. Development of a multi-dimensional workload rating scale: Results of empirical and theoretical research. In Human Mental Workload P. A. Hancock and N. Meshkati (Eds.), Elsevier, Amsterdam, The Netherlands, pp. 138--183.Google Scholar
- Hess, E. H. 1972. Pupillometrics: A method of studying mental, emotional and sensory processes. In Handbook of Psychophysiology, N. S. Greenfield and R. A. Sternbach, (Eds.) Holt, Rinehart & Winston, New York, pp. 491--531.Google Scholar
- Hess, E. H. and Polt, J. M. 1964. Pupil size in relation to mental activity during simple problem solving. Science 132, 1190--1192.Google ScholarCross Ref
- Hoecks, B. and Levelt, W. 1993. Pupillary dilation as a measure of attention: A quantitative system analysis. Behav. Res. Meth. Instrum. Comput. 25, 16--26.Google ScholarCross Ref
- Horvitz, E. and Apacible, J. 2003. Learning and reasoning about interruption. In Proceedings of the 5th ACM International Conference on Multimodal Interfaces. ACM, New York, 20--27. Google ScholarDigital Library
- Horvitz, E., Jacobs, A., and Hovel, D. 1999. Attention-sensitive alerting. In Conference Proceedings on Uncertainty in Artificial Intelligence, 305-313.Google Scholar
- Horvitz, E., Koch, P., and Apacible, J. 2004. BusyBody: Creating and fielding personalized models of the cost of interruption. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (Chicago, IL). ACM, New York, 507--510. Google ScholarDigital Library
- Hudson, S. E., Fogarty, J., Atkeson, C. G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J., and Yang, J. 2003. Predicting human interruptibility with sensors: A wizard of Oz feasibility study. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 257--264. Google ScholarDigital Library
- Hytintk, J., Tommola, J., and Alaja, A. 1995. Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. Quart. J. Exper. Psych. 48A, 3, 598--612.Google Scholar
- Iqbal, S. T., Adamczyk, P. D., Zheng, S., and Bailey, B. P. 2005. Towards an index of opportunity: Understanding changes in mental workload during task execution. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 311--320. Google ScholarDigital Library
- Iqbal, S. T. and Bailey, B. P. 2005. Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 1489--1492. Google ScholarDigital Library
- Iqbal, S. T. and Bailey, B. P. 2006. Leveraging characteristics of task structure to predict costs of interruption. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 741-750. Google ScholarDigital Library
- Iqbal, S. T. and Bailey, B. P. 2007. Understanding and developing models for detecting and differentiating breakpoints during interactive tasks. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 697--706. Google ScholarDigital Library
- Iqbal, S. T., Zheng, X. S., and Bailey, B. P. 2004. Task evoked pupillary response to mental workload in human-computer interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 1477--1480. Google ScholarDigital Library
- Jackson, T. W., Dawson, R. J., and Wilson, D. 2001. The cost of email interruption. J. Syst. Inf. Tech. 5, 1, 81--92.Google ScholarCross Ref
- Juris, M. and Velden, M. 1977. The pupillary response to mental overload. Phys. Psych. 5, 4, 421--424.Google ScholarCross Ref
- Just, M. A., Carpenter, P. A., and Miyake, A. 2003. Neuroindices of cognitive workload: Neuroimaging, pupillometric, and event-related potential studies of brain work. Theoret. Issues Ergonom. 4, 56--88.Google ScholarCross Ref
- Kahneman, D. 1967. Pupillary responses in a pitch-discrimination task. Percept. Psych. 2, 101--105.Google ScholarCross Ref
- Kahneman, D. 1973. Attention and Effort. Prentice-Hall, Englewood Cliffs, N.J.Google Scholar
- Kok, A. 1997. Event-related-potential (ERP) reflections of mental resources: A review and synthesis. Biological Psychology 45, 19--56.Google ScholarCross Ref
- Kramer, A. F. 1991. Physiological metrics of mental workload: A review of recent progress. In Multiple-Task Performance, D. L. Damos, Ed. Taylor and Francis, London, England, pp. 279--328.Google Scholar
- Kramer, A. F., Schneider, W., Fisk, A. D., and Donchin, E. 1986. The effects of practice and task structure on components of event related brain potential. Psychophysiology 23, 33--47.Google ScholarCross Ref
- Kreifeldt, J. G. and McCarthy, M. E. 1981. Interruption as a test of the user-computer interface. In Proceedings of the 17th Annual Conference on Manual Control, Jet Propulsion Laboratory, California Institute of Technology, JPL Publication 81--95, 655--667.Google Scholar
- Latorella, K. A. 1996. Investigating interruptions: An example from the flight deck. In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting. 249--253.Google Scholar
- Latorella, K. A. 1998. Effects of modality on interrupted flight deck performance: Implications for data link. In 42nd Annual Meeting of the Human Factors and Ergonomics Society. 87--91.Google ScholarCross Ref
- Lee, J. D., Hoffman, J. D., and Hayes, E. 2004. Collision warning design to mitigate driver distraction. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 65--72. Google ScholarDigital Library
- Lin, Y., Zhang, W. J., and Watson, L. G. 2003. Using eye movement parameters for evaluating human--machine interface frameworks under normal control operation and fault detection situations. Int. J. Human-Comput. Stud. 59, 6, 837--873. Google ScholarDigital Library
- Maglio, P. and Campbell, C. S. 2003. Attentive agents. Commun. ACM 46, 3, 47--51. Google ScholarDigital Library
- Marshall, S. P. 2002. The index ofs cognitive activity: Measuring cognitive workload. In Proceedings of the 2002 IEEE 7th Conference on Human Factors and Power Plants. IEEE Computer Society Press, Los Alamitos, CA, 7.5--7.9.Google Scholar
- McCrickard, S., Chewar, C. M., Somervell, J. P., and Ndiwalana, A. 2003. A model for notification systems evaluation--assessing user goals for multitasking activity. ACM Trans. Computer-Hum. Interact. 10, 4, 312--338. Google ScholarDigital Library
- McFarlane, D. C. 1999. Coordinating the interruption of people in human-computer interaction. In Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction. 295--303.Google Scholar
- McFarlane, D. C. and Latorella, K. A. 2002. The scope and importance of human interruption in HCI design. Human-Computer Interaction 17, 1, 1--61.Google ScholarDigital Library
- Miyata, Y. and Norman, D. A. 1986. Psychological issues in support of multiple activities. In User Centered System Design: New Perspectives on Human-Computer Interaction. D. A. Norman and S. W. Draper, Eds. Lawrence Erlbaum Associates, Hillsdale, N.J., pp. 265--284.Google Scholar
- Monk, C. A., Boehm-Davis, D. A., and Trafton, J. G. 2002. The attentional costs of interrupting task performance at various stages. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting.Google Scholar
- Nair, R., Voida, S., and Mynatt, E. 2005. Frequency-based detection of task switches. In Proceedings of the 19th British HCI Group Annual Conference (Edinburgh, Scotland). 94--99.Google Scholar
- Nakayama, M. and Takahashi, K. 2002. The act of task difficulty and eye-movement frequency for the oculo-motor indices. In Proceedings of Eye Tracking Research and Applications. 37--42. Google ScholarDigital Library
- Newell, A. and Rosenbloom, P. S. 1981. Mechanisms of skill acquisition and the law of practice. In Cognitive Skills and their Acquisition, J. R. Anderson, Ed. Erlbaum, Hillsdale, NJ, pp. 1--55.Google Scholar
- O'Donnell, R. D. and Eggemeier, F. T. 1986. Workload assessment methodology. In Handbook of Perception and Human Performance. Volume II, Cognitive Processes and Performance, K. R. Boff, L. Kaufman and J. P. Thomas, Eds., Wiley, New York, pp. 42/41--42/49.Google Scholar
- Phelps, M. P. and Mazziotta, J. 1985. Positron emission tomography: Human brain function and biochemistry. Science 228, 799--809.Google ScholarCross Ref
- Pomplun, M. and Sunkara, S. 2003. Pupil dilation as an indicator of cognitive workload in human-computer interaction. In Proceedings of the 10th International Conference on Human-Computer Interaction, V. D. D. Harris, M. Smith and C. Stephanidis, Eds.Google Scholar
- Rowe, D. W., Sibert, J., and Irwin, D. 1998. Heart rate variability: indicator of user state as an aid to human-computer interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 480--487. Google ScholarDigital Library
- Rubinstein, J. S., Meyer, D. E., and Evans, J. E. 2001. Executive control of cognitive processes in task switching. J. Exper. Psych.: Human Percept. Perform. 27, 4, 763--797.Google ScholarCross Ref
- Shneiderman, B. 1997. Designing the user interface. Pearson Addison Wesley, Reading, MA, Third Edition. Google ScholarDigital Library
- Speier, C., Valacich, J. S., and Vessey, I. 1999. The influence of task interruption on individual decision making: An information overload perspective. Decis. Sci. 30, 2, 337--360.Google ScholarCross Ref
- Stanton, N., Ed. 1994. Human Factors in Alarm Design. Taylor and Francis, London, UK. Google ScholarDigital Library
- Takahashi, K., Nakayama, M., and Shimizu, Y. 2000. The response of eye-movement and pupil size to audio instruction while viewing a moving target. In Proceedings of the ACM Conference on Eye Tracking Research and Applications. ACM, New York, 131-138. Google ScholarDigital Library
- Tobii-Systems. http://www.tobii.se/.Google Scholar
- Trafton, J. G., Altmann, E. M., Brock, D. P., and Mintz, F. E. 2003. Preparing to resume an interrupted task: effects of prospective goal encoding and retrospective rehearsal. Int. J. Human-Comput. Stud. 58, 583--603. Google ScholarDigital Library
- Verney, S. P., Granholm, E., and Dionisio, D. P. 2001. Pupillary responses and processing resources on the visual backward masking task. Psychophysiology 38, 1, 76--83.Google ScholarCross Ref
- Verney, S. P., Granolm, E., and Marshall, S. 2004. Pupillary responses during the visual backward masking task predict cognitive ability. Int. J. Psychophys. 52, 23--36.Google ScholarCross Ref
- Wickens, C. D. 1980. The structure of attentional resources. In Attention and Performance VIII, R. Nickerson, Ed. Lawrence Erlbaum, Hillsdale, NJ, pp. 239--257.Google Scholar
- Wickens, C. D. 1991. Processing resources and attention. In Multiple-Task Performance, D. L. Damos, Ed. Taylor & Francis, London, UK, pp. 3--34.Google Scholar
- Wickens, C. D. 2002. Multiple resources and performance prediction. Theoret. Iss. Ergon. Sci. 3, 2, 159--177.Google ScholarCross Ref
- Zacks, J., Tversky, B., and Iyer, G. 2001. Perceiving, remembering, and communicating structure in events. J. Exper. Psych. General 130, 1, 29--58.Google ScholarCross Ref
- Zijlstra, F. R. H., Roe, R. A., Leonora, A. B., and Krediet, I. 1999. Temporal factors in mental work: Effects of interrupted activities. J. Occupat. Org. Psych. 72, 163--185.Google ScholarCross Ref
Index Terms
- Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management
Recommendations
Towards an index of opportunity: understanding changes in mental workload during task execution
CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsTo contribute to systems that reason about human attention, our work empirically demonstrates how a user's mental workload changes during task execution. We conducted a study where users performed interactive, hierarchical tasks while mental workload ...
Multitasking and monotasking: the effects of mental workload on deferred task interruptions
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsRecent research has found that forced interruptions at points of higher mental workload are more disruptive than at points of lower workload. This paper investigates a complementary idea: when users experience deferrable interruptions at points of ...
Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption
CHI EA '05: CHI '05 Extended Abstracts on Human Factors in Computing SystemsThis work investigates the use of workload-aligned task models for predicting opportune moments for interruption. From models for several tasks, we selected boundaries with the lowest (Best) and highest (Worst) mental workload. We compared effects of ...
Comments