Carrie J. Cai
Jaime Teevan
ABSTRACT
Microtasks are small units of work designed to be completed individually, eventually contributing to a larger goal. Although microtasks can be performed in isolation, in practice people often complete a chain of microtasks within a single session. Through a series of crowd-based studies, we look at how various microtasks can be chained together to improve efficiency and minimize mental demand, focusing on the writing domain. We find that participants completed low-complexity microtasks faster when they were preceded by the same type of microtask, whereas they found high-complexity microtasks less mentally demanding when pre-ceded by microtasks on the same content. Furthermore, participants were faster at starting high-complexity microtasks after completing lower-complexity microtasks, but completion time and quality were not affected. These findings provide insight into how microtasks can be ordered to optimize transitions from one microtask to another.
- David S. Ackerman and Barbara L. Gross. 2005. My instructor made me do it: Task characteristics of procrastination. Journal of Marketing Education 27, 1: 5--13.Google Scholar
Cross Ref
- Piotr D. Adamczyk and Brian P. Bailey. 2004. If not now, when?: the effects of interruption at different moments within task execution. Proceedings of the SIGCHI conference on Human factors in Computing Systems, ACM, 271--278. Google ScholarDigital Library
- Paul S. Adler, Barbara Goldoftas, and David I. Levine. 1999. Flexibility versus efficiency? A case study of model changeovers in the Toyota production system. Organization science 10, 1: 43--68. Google ScholarDigital Library
- David Allen. 2015. Getting things done: The art of stress-free productivity. Penguin.Google Scholar
- Brian P. Bailey and Shamsi T. Iqbal. 2008. Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management. ACM Transactions on Computer-Human Interaction (TOCHI) 14, 4: 21. Google ScholarDigital Library
- Albert Bandura. 1977. Self-efficacy: toward a unifying theory of behavioral change. Psychological review 84, 2: 191.Google Scholar
- Michael S. Bernstein, Greg Little, Robert C. Miller, et al. 2010. Soylent: a word processor with a crowd inside. Proceedings of the 23nd annual ACM symposium on User interface software and technology, ACM, 313--322. Google ScholarDigital Library
- Jelmer P. Borst, Niels A. Taatgen, and Hedderik van Rijn. 2015. What Makes Interruptions Disruptive?: A Process-Model Account of the Effects of the Problem State Bottleneck on Task Interruption and Resumption. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, 2971--2980. Google ScholarDigital Library
- Eric Butler, Erik Andersen, Adam M. Smith, Sumit Gulwani, Zoran Popovic, and W. A. Redmond. 2015. Automatic Game Progression Design through Analysis of Solution Features. Proc. of the SIGCHI Conf. on Human Factors in Computing (CHI'2015). Google ScholarDigital Library
- Carrie J. Cai, Philip J. Guo, James R. Glass, and Robert C. Miller. 2015. Wait-Learning: Leveraging wait time for second language education. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, 3701--3710. Google ScholarDigital Library
- Justin Cheng, Jaime Teevan, Shamsi T. Iqbal, and Michael S. Bernstein. 2015. Break it down: A comparison of macro-and microtasks. Proceedings of CHI. Google ScholarDigital Library
- Lydia B. Chilton, John J. Horton, Robert C. Miller, and Shiri Azenkot. 2010. Task search in a human computation market. Proceedings of the ACM SIGKDD workshop on human computation, ACM, 1--9. Google ScholarDigital Library
- Fergus IM Craik and Robert S. Lockhart. 1972. Levels of processing: A framework for memory research. Journal of verbal learning and verbal behavior 11, 6: 671--684.Google ScholarCross Ref
- Peng Dai, Jeffrey M. Rzeszotarski, Praveen Paritosh, and Ed H. Chi. 2015. And Now for Something Completely Different: Improving Crowdsourcing Workflows with Micro-Diversions. Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, ACM, 628--638. Google ScholarDigital Library
- Steven Dow, Anand Kulkarni, Scott Klemmer, and Björn Hartmann. 2012. Shepherding the crowd yields better work. Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, ACM, 1013--1022. Google ScholarDigital Library
- Darren Edge, Elly Searle, Kevin Chiu, Jing Zhao, and James A. Landay. 2011. MicroMandarin: mobile language learning in context. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 3169--3178. Google ScholarDigital Library
- Joseph R. Ferrari, Judith L. Johnson, and William George McCown. 1995. Procrastination and task avoidance: Theory, research, and treatment. Springer Science & Business Media.Google Scholar
- Linda Flower. 1979. Writer-based prose: A cognitive basis for problems in writing. College English: 19--37.Google Scholar
- Tony Gillie and Donald Broadbent. 1989. What makes interruptions disruptive? A study of length, similarity, and complexity. Psychological research 50, 4: 243--250.Google Scholar
- W. Graves, J. Binder, M. Seidenberg, and R. Desai. "Neural correlates of semantic processing in reading aloud." In M. Faust (Ed.), Handbook of the Neuropsychology of Language. Malden, MA: Wiley-Blackwell.Google Scholar
- Nick Greer, Jaime Teevan, and Shamsi T. Iqbal. 2016. An Introduction to Technological Support for Writing. Microsoft Research Tech Report MSR-TR2016-001.Google Scholar
- Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52: 139--183.Google Scholar
- Suzanne Hidi and Pietro Boscolo. 2006. Motivation and writing. Handbook of writing research: 144--157.Google Scholar
- John Joseph Horton and Lydia B. Chilton. 2010. The labor economics of paid crowdsourcing. Proceedings of the 11th ACM conference on Electronic commerce, ACM, 209--218. Google ScholarDigital Library
- Panagiotis G. Ipeirotis. 2010. Analyzing the amazon mechanical turk marketplace. XRDS: Crossroads, The ACM Magazine for Students 17, 2: 16--21. Google ScholarDigital Library
- Shamsi T. Iqbal and Brian P. Bailey. 2008. Effects of intelligent notification management on users and their tasks. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 93--102. Google ScholarDigital Library
- Shamsi T. Iqbal and Eric Horvitz. 2007. Disruption and recovery of computing tasks: field study, analysis, and directions. Proceedings of the SIGCHI conference on Human factors in computing systems, ACM, 677--686. Google ScholarDigital Library
- Walter S. Lasecki, Jeffrey M. Rzeszotarski, Adam Marcus, and Jeffrey P. Bigham. 2015. The Effects of Sequence and Delay on Crowd Work. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, 1375--1378. Google ScholarDigital Library
- Ulf Lundberg, Marianne Granqvist, Tommy Hansson, Marianne Magnusson, and Leif Wallin. 1989. Psychological and physiological stress responses during repetitive work at an assembly line. Work & Stress 3, 2: 143--153.Google ScholarCross Ref
- Gloria Mark, Daniela Gudith, and Ulrich Klocke. 2008. The cost of interrupted work: more speed and stress. Proceedings of the SIGCHI conference on Human Factors in Computing Systems, ACM, 107--110. Google ScholarDigital Library
- Andrew A. Mitchell. 1979. Involvement: a potentially important mediator of consumer behavior. Advances in consumer research 6, 1: 191--196.Google Scholar
- Stephen Monsell. 2003. Task switching. Trends in cognitive sciences 7, 3: 134--140.Google Scholar
- Robert R. Morris, Mira Dontcheva, and Elizabeth M. Gerber. 2012. Priming for better performance in microtask crowdsourcing environments. Internet Computing, IEEE 16, 5: 13--19. Google ScholarDigital Library
- Mohamed Musthag and Deepak Ganesan. 2013. Labor dynamics in a mobile micro-task market. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 641--650. Google ScholarDigital Library
- Michael Nebeling, Alexandra To, Anhong Guo, et al. 2016. WearWrite: Crowd-Assisted Writing from Smartwatches. Proceedings of CHI. Google ScholarDigital Library
- Christine M. Neuwirth, David S. Kaufer, Ravinder Chandhok, and James H. Morris. 2001. Computer support for distributed collaborative writing: A coordination science perspective. Coordination Theory and Collaboration Technology, ed. GM Olson, TW Malone, and JB Smith, Lawrence Erlbaum Associates, NJ.Google Scholar
- Marion C. Panyan and R. Vance Hall. 1978. Effects of Serial versus Concurrent Task Sequencing on Acquisition, Maintenance, and Generalization. Journal of Applied Behavior Analysis 11, 1: 67--74.Google ScholarCross Ref
- Nathalie Pattyn, Xavier Neyt, David Henderickx, and Eric Soetens. 2008. Psychophysiological investigation of vigilance decrement: boredom or cognitive fatigue? Physiology & Behavior 93, 1: 369--378.Google ScholarCross Ref
- Ilona R. Posner and Ronald M. Baecker. 1992. How people write together {groupware}. System Sciences, 1992. Proceedings of the Twenty-Fifth Hawaii International Conference on, IEEE, 127--138.Google Scholar
- Bennett A. Rafoth and Donald L. Rubin. 1984. The impact of content and mechanics on judgments of writing quality. Written Communication 1, 4: 446--458.Google ScholarCross Ref
- Darryl W. Schneider and John R. Anderson. 2010. Asymmetric switch costs as sequential difficulty effects. The Quarterly Journal of Experimental Psychology 63, 10: 1873--1894.Google ScholarCross Ref
- R. J. Senter and E. A. Smith. 1967. Automated readability index. DTIC Document.Google Scholar
- Gordon Taylor and Peggy Nightingale. 1990. Not mechanics but meaning: Error in tertiary students' writing. Higher Education Research and Development 9, 2: 161--176.Google ScholarCross Ref
- Jaime Teevan, Shamsi Iqbal, and Curtis von Veh. 2016. Supporting Collaborative Writing with Microtasks. Proceedings of CHI. Google ScholarDigital Library
- Jaime Teevan, Daniel J. Liebling, and Walter S. Lasecki. 2014. Selfsourcing personal tasks. CHI'14 Extended Abstracts on Human Factors in Computing Systems, ACM, 2527--2532. Google ScholarDigital Library
- Rajan Vaish, Keith Wyngarden, Jingshu Chen, Brandon Cheung, and Michael S. Bernstein. 2014. Twitch crowdsourcing: crowd contributions in short bursts of time. Proceedings of the 32nd annual ACM conference on Human factors in computing systems, ACM, 3645--3654. Google ScholarDigital Library
- Glenn Wylie and Alan Allport. 2000. Task switching and the measurement of "switch costs." Psychological research 63, 3--4: 212--233.Google ScholarCross Ref
- Ming Yin, Yiling Chen, and Yu-An Sun. 2014. Monetary interventions in crowdsourcing task switching. Second AAAI Conference on Human Computation and Crowdsourcing.Google Scholar
- Judith Lynne Zaichkowsky. 1994. The personal involvement inventory: Reduction, revision, and application to advertising. Journal of advertising 23, 4: 59--70.Google ScholarCross Ref
Index Terms
- Chain Reactions: The Impact of Order on Microtask Chains
Recommendations
Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments
The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that ...
Make Hay While the Crowd Shines: Towards Efficient Crowdsourcing on the Web
WWW '15 Companion: Proceedings of the 24th International Conference on World Wide WebWithin the scope of this PhD proposal, we set out to investigate two pivotal aspects that influence the effectiveness of crowdsourcing: (i) microtask design, and (ii) workers behavior. Leveraging the dynamics of tasks that are crowdsourced on the one ...
Crowd Anatomy Beyond the Good and Bad: Behavioral Traces for Crowd Worker Modeling and Pre-selection
AbstractThe suitability of crowdsourcing to solve a variety of problems has been investigated widely. Yet, there is still a lack of understanding about the distinct behavior and performance of workers within microtasks. In this paper, we first introduce a ...
Comments