Nirwan Sharma
Skip Abstract Section
Abstract
A number of initiatives invite members of the public to perform online classification tasks such as identifying objects in images. These tasks are crucial to numerous large-scale Citizen Science projects in different disciplines, with volunteers using their knowledge and online support tools to, for example, identify species of wildlife or classify galaxies by their shapes. However, for complex classification tasks, such as this case study on identifying species of bumblebee, reaching an agreement between volunteers - or even between experts~-~may require consensus-building processes. Collaboration and teamwork approaches to problem solving and decision-making have been widely documented to improve both task performance and user learning in the real world. Most of these processes and projects are mediated online through feedback delivered in an asynchronous manner, and this article thus addresses a central research question: How do participants involved in species identification tasks respond to different forms of feedback provided in online collaboration, designed to support peer-learning and improve task performance? We tested four different approaches to feedback within a collaboration task, where participants reviewed their previously annotated data based on information curated from their peers on a long running online citizen science initiative. The selected interfaces have a strong foundation in social science and psychology literature and can be applied to citizen science practices as well as other online communities. Results showed that while all four approaches increased accuracy, there were differences based on the types of consensus that existed before collaboration. Such differences highlight the usefulness of different forms of feedback during collaboration for increasing data accuracy of identification and furthering users' expertise on identification tasks. We found that anonymised and goal-directed free text comments posted on social learning interfaces were most effective in improving data accuracy as well as creating opportunities for peer-learning, particularly where the species identification task was more difficult. This study has significant implications for extending the practice of citizen science across formal and informal learning environments and reaching out to a variety of users.
- 2021. iNaturalist. https://www.inaturalist.org/Google Scholar
- 2021. iSpot. https://www.ispotnature.org/Google Scholar
- Paul André, Robert E. Kraut, and Aniket Kittur. 2014. Effects of Simultaneous and Sequential Work Structures on Distributed Collaborative Interdependent Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI '14). Association for Computing Machinery, New York, NY, USA, 139--148. https://doi.org/10.1145/2556288.2557158Google ScholarDigital Library
- Koen Arts, René Van der Wal, and William M. Adams. 2015. Digital technology and the conservation of nature. Ambio 44, S4 (27 11 2015), 661--673. https://doi.org/10.1007/s13280-015-0705--1Google Scholar
- Tom August, Martin Harvey, Paula Lightfoot, David Kilbey, Timos Papadopoulos, and Paul Jepson. 2015. Emerging technologies for biological recording. Biological Journal of the Linnean Society 115, 3 (7 2015), 731--749. https://doi.org/10.1111/bij.12534Google ScholarCross Ref
- Mikhail Bakhtin. 1984. Problems of Dostoevsky's poetics (C. Emerson, Trans. C. Emerson Ed.). University of Minnesota Press, Minneapolis.Google Scholar
- Gregory Bateson. 1979. Mind and nature : a necessary unity. Hampton Press, London.Google Scholar
- Melanie R Beck, Claudia Scarlata, Lucy F Fortson, Chris J Lintott, B D Simmons, Melanie A Galloway, Kyle W Willett, Hugh Dickinson, Karen L Masters, Philip J Marshall, and Darryl Wright. 2018. Integrating human and machine intelligence in galaxy morphology classification tasks. Monthly Notices of the Royal Astronomical Society 476, 4 (1 6 2018), 5516--5534. https://doi.org/10.1093/MNRAS/STY503Google Scholar
- Gerard Beenen, Kimberly Ling, Xiaoqing Wang, Klarissa Chang, Dan Frankowski, Paul Resnick, and Robert E Kraut. 2004. Using Social Psychology to Motivate Contributions to Online Communities General Terms. Proceedings of the 2004 ACM conference on Computer supported cooperative work - CSCW '04, 212--221. https://doi.org/10.1145/1031607Google ScholarDigital Library
- Györgyi Bela, Taru Peltola, Juliette C. Young, Bálint Balázs, Isabelle Arpin, György Pataki, Jennifer Hauck, Eszter Kelemen, Leena Kopperoinen, Ann Van Herzele, Hans Keune, Susanne Hecker, Monika Su?kevi's, Helen E. Roy, Pekka Itkonen, Mart Külvik, Miklós László, Corina Basnou, Joan Pino, and Aletta Bonn. 2016. Learning and the transformative potential of citizen science. Conservation Biology 30, 5 (2016), 990--999. https://doi.org/10.1111/cobi.12762Google ScholarCross Ref
- Tomas J. Bird, Amanda E. Bates, Jonathan S. Lefcheck, Nicole A. Hill, Russell J. Thomson, Graham J. Edgar, Rick D. Stuart-Smith, Simon Wotherspoon, Martin Krkosek, Jemina F. Stuart-Smith, Gretta T. Pecl, Neville Barrett, and Stewart Frusher. 2014. Statistical solutions for error and bias in global citizen science datasets. Biological Conservation 173 (2014), 144--154. https://doi.org/10.1016/j.biocon.2013.07.037Google ScholarCross Ref
- S. Blake, A. Siddharthan, H. Nguyen, N. Sharma, A.-M. Robinson, E. O'mahony, B. Darvill, C. Mellish, and R. Van Der Wal. 2012. Natural language generation for nature conservation: Automating feedback to help volunteers identify bumblebee species. In 24th International Conference on Computational Linguistics - Proceedings of COLING 2012: Technical Papers.Google Scholar
- Rick Bonney, Caren B. Cooper, Janis Dickinson, Steve Kelling, Tina Phillips, V. Kenneth Rosenberg, and Jennifer Shirk. 2009. Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience 59, 11 (12 2009), 977--984. https://doi.org/10.1525/bio.2009.59.11.9Google Scholar
- Eleanor D. Brown and Byron K. Williams. 2019. The potential for citizen science to produce reliable and useful information in ecology. Conservation Biology 33, 3 (1 6 2019), 561--569. https://doi.org/10.1111/COBI.13223Google Scholar
- Matthias Budde, Andrea Schankin, Julien Hoffmann, Marcel Danz, Till Riedel, and Michael Beigl. 2017. Participatory Sensing or Participatory Nonsense? Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (11 9 2017), 1--23. https://doi.org/10.1145/3131900Google ScholarDigital Library
- Luigi Ceccaroni, James Bibby, Erin Roger, Paul Flemons, Katina Michael, Laura Fagan, and Jessica L. Oliver. 2019. Opportunities and Risks for Citizen Science in the Age of Artificial Intelligence. Citizen Science: Theory and Practice 4, 1 (28 11 2019). https://doi.org/10.5334/cstp.241Google Scholar
- Andrea Chester and Gillian Gwynne. 1998. Online Teaching: Encouraging Collaboration through Anonymity. Journal of Computer-Mediated Communication 4, 2 (1 12 1998). https://doi.org/10.1111/J.1083--6101.1998.TB00096.XGoogle Scholar
- Michelene T. H. Chi, Paul J. Feltovich, and Robert Glaser. 1981. Categorization and Representation of Physics Problems by Experts and Novices*. Cognitive Science 5, 2 (4 1981), 121--152. https://doi.org/10.1207/s15516709cog0502_2Google Scholar
- Jeffrey P. JP Cohn. 2008. Citizen Science: Can Volunteers Do Real Research? BioScience 58, 3 (2008), 192. https://doi.org/10.1641/B580303Google ScholarCross Ref
- HM Collins and R Evans. 2002. The third wave of science studies: Studies of expertise and experience. Social studies of science (2002).Google ScholarCross Ref
- Laura Colucci-Gray, Pamela Burnard, Donald Gray, and Carolyn Cooke. 2019. A Critical Review of STEAM (Science, Technology, Engineering, Arts, and Mathematics). Oxford Research Encyclopedia of Education (26 3 2019). https://doi.org/10.1093/ACREFORE/9780190264093.013.398Google Scholar
- Yin Cui, Yang Song, Chen Sun, Andrew Howard, and Serge Belongie. 2018. Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (14 12 2018), 4109--4118. https://doi.org/10.1109/CVPR.2018.00432Google ScholarCross Ref
- Vickie Curtis. 2015. Motivation to Participate in an Online Citizen Science Game: A Study of Foldit. Science Communication 37, 6 (16 10 2015), 723--746. https://doi.org/10.1177/1075547015609322Google ScholarCross Ref
- Marc Esteve Del Valle, Anatoliy Gruzd, Priya Kumar, and Sarah Gilbert. 2020. Learning in the Wild: Understanding Networked Ties in Reddit. Springer International Publishing, Cham, 51--68. https://doi.org/10.1007/978--3-030--36911--8_4Google Scholar
- Janis L. Dickinson, Benjamin Zuckerberg, and David N. Bonter. 2010. Citizen Science as an Ecological Research Tool: Challenges and Benefits. Annual Review of Ecology, Evolution, and Systematics 41, 1 (12 2010), 149--172. https://doi.org/10.1146/annurev-ecolsys-102209--144636Google ScholarCross Ref
- John R.G. Dyer, Christos C. Ioannou, Lesley J. Morrell, Darren P. Croft, Iain D. Couzin, Dean A. Waters, and Jens Krause. 2008. Consensus decision making in human crowds. Animal Behaviour 75, 2 (2 2008), 461--470. https://doi.org/10.1016/j.anbehav.2007.05.010Google Scholar
- Rebecca Ellis. 2011. Jizz and the joy of pattern recognition: Virtuosity, discipline and the agency of insight in UK naturalists' arts of seeing. Social Studies of Science 41, 6 (12 2011), 769--790. https://doi.org/10.1177/0306312711423432Google Scholar
- Elizabeth R. Ellwood, Betty A. Dunckel, Paul Flemons, Robert Guralnick, Gil Nelson, Greg Newman, Sarah Newman, Deborah Paul, Greg Riccardi, Nelson Rios, Katja C. Seltmann, and Austin R. Mast. 2015. Accelerating the Digitization of Biodiversity Research Specimens through Online Public Participation. BioScience 65, 4 (1 4 2015), 383--396. https://doi.org/10.1093/biosci/biv005Google Scholar
- Yrjö Engeström et al . 1999. Activity theory and individual and social transformation. Perspectives on activity theory 19, 38 (1999), 19--30.Google Scholar
- Andrea Forte, Nazanin Andalibi, and Rachel Greenstadt. 2017. Privacy, anonymity, and perceived risk in open collaboration: A study of tor users and wikipedians. Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW (25 2 2017), 1800--1811. https://doi.org/10.1145/2998181.2998273Google ScholarDigital Library
- Steffen Fritz, Linda See, Tyler Carlson, Mordechai (Muki) Haklay, Jessie L Oliver, Dilek Fraisl, Rosy Mondardini, Martin Brocklehurst, Lea A Shanley, Sven Schade, Uta Wehn, Tommaso Abrate, Janet Anstee, Stephan Arnold, Matthew Billot, Jillian Campbell, Jessica Espey, Margaret Gold, Gerid Hager, Shan He, Libby Hepburn, Angel Hsu, Deborah Long, Joan Masó, Ian McCallum, Maina Muniafu, Inian Moorthy, Michael Obersteiner, Alison J Parker, Maike Weisspflug, and Sarah West. 2019. Citizen science and the United Nations Sustainable Development Goals. Nature Sustainability 2, 10 (2019), 922--930. https://doi.org/10.1038/s41893-019-0390--3Google ScholarCross Ref
- Stuart Iain Gray, Judy Robertson, Andrew Manches, and Gnanathusharan Rajendran. 2019. BrainQuest: The use of motivational design theories to create a cognitive training game supporting hot executive function. International Journal of Human-Computer Studies 127 (1 7 2019), 124--149. https://doi.org/10.1016/J.IJHCS.2018.08.004Google ScholarDigital Library
- Oskar L.P. Hansen, Jens Christian Svenning, Kent Olsen, Steen Dupont, Beulah H. Garner, Alexandros Iosifidis, Benjamin W. Price, and Toke T. Høye. 2020. Species-level image classification with convolutional neural network enables insect identification from habitus images. Ecology and Evolution 10, 2 (jan 2020), 737--747. https://doi.org/10.1002/ECE3.5921Google ScholarCross Ref
- Barbara Heinisch. 2020. Knowledge translation and its interrelation with usability and accessibility. Biocultural diversity translated by means of technology and language-the case of citizen science contributing to the sustainable development goals. Sustainability 13, 1 (2020), 54.Google ScholarCross Ref
- Gayle W. Hill. 1982. Group versus individual performance: Are N+1 heads better than one? Psychological Bulletin 91 (1982), 517--539. https://doi.org/10.1037/0033--2909.91.3.517Google ScholarCross Ref
- Toke T. Høye, Johanna -rje, Kim Bjerge, Oskar L. P. Hansen, Alexandros Iosifidis, Florian Leese, Hjalte M. R. Mann, Kristian Meissner, Claus Melvad, and Jenni Raitoharju. 2021. Deep learning and computer vision will transform entomology. Proceedings of the National Academy of Sciences 118, 2 (2021), e2002545117. https://doi.org/10.1073/pnas.2002545117Google ScholarCross Ref
- Alan Irwin. 1995. Citizen science: a study of people, expertise, and sustainable development. Routledge.Google Scholar
- Nick J. B. Isaac, Arco J. Strien, Tom A. August, Marnix P. Zeeuw, and David B. Roy. 2014. Statistics for citizen science: extracting signals of change from noisy ecological data. Methods in Ecology and Evolution 5, 10 (1 10 2014), 1052--1060. https://doi.org/10.1111/2041--210X.12254Google Scholar
- Corey Brian Jackson, Carsten Østerlund, Kevin Crowston, Mahboobeh Harandi, and Laura Trouille. 2020. Shifting forms of Engagement: Volunteer Learning in Online Citizen Science. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020). https://doi.org/10.1145/3392841Google ScholarDigital Library
- Heisawn Jeong and Cindy E. Hmelo-Silver. 2016. Seven Affordances of Computer-Supported Collaborative Learning: How to Support Collaborative Learning? How Can Technologies Help? Educational Psychologist 51, 2 (2 4 2016), 247--265. https://doi.org/10.1080/00461520.2016.1158654Google Scholar
- Aditya Johri. 2011. The socio-materiality of learning practices and implications for the field of learning technology. Research in Learning Technology 19, 3 (2011), 207--217.Google ScholarCross Ref
- Benjamin L. Jones, Richard K.F. Unsworth, Len J. McKenzie, Rudi L. Yoshida, and Leanne C. Cullen-Unsworth. 2018. Crowdsourcing conservation: The role of citizen science in securing a future for seagrass. Marine Pollution Bulletin 134 (1 9 2018), 210--215. https://doi.org/10.1016/J.MARPOLBUL.2017.11.005Google Scholar
- Dick Kasperowski and Niclas Hagen. 2022. Making particularity travel: Trust and citizen science data in Swedish environmental governance. Social studies of science 52, 3 (apr 2022), 447--462. https://doi.org/10.1177/03063127221085241Google Scholar
- Steve Kelling, Daniel Fink, Frank A. La Sorte, Alison Johnston, Nicholas E. Bruns, and Wesley M. Hochachka. 2015. Taking a ?Big Data' approach to data quality in a citizen science project. Ambio 44 (1 11 2015), 601--611. https://doi.org/10.1007/S13280-015-0710--4Google Scholar
- Norbert L Kerr and R Scott Tindale. 2004. Group performance and decision making. Annual review of psychology 55 (2004), 623--655. https://doi.org/10.1146/annurev.psych.55.090902.142009Google Scholar
- Vasily Klucharev, Ale Smidts, and Guillén Fernández. 2008. Brain mechanisms of persuasion: how "expert power' modulates memory and attitudes. Social Cognitive and Affective Neuroscience 3, 4 (12 2008), 353--366. https://doi.org/10.1093/scan/nsn022Google Scholar
- Margaret Kosmala, Andrea Wiggins, Alexandra Swanson, and Brooke Simmons. 2016. Assessing data quality in citizen science. Frontiers in Ecology and the Environment 14, 10 (1 12 2016), 551--560. https://doi.org/10.1002/FEE.1436Google Scholar
- Christopher Kullenberg and Dick Kasperowski. 2016. What Is Citizen Science? -- A Scientometric Meta-Analysis. PLOS ONE 11, 1 (14 1 2016), e0147152. https://doi.org/10.1371/journal.pone.0147152Google Scholar
- Joseph Lampel and Ajay Bhalla. 2007. The Role of Status Seeking in Online Communities: Giving the Gift of Experience. Journal of Computer-Mediated Communication 12, 2 (1 2007), 434--455. https://doi.org/10.1111/j.1083--6101.2007.00332.xGoogle ScholarCross Ref
- J Larkin, J McDermott, DP Simon, and HA Simon. 1980. Expert and novice performance in solving physics problems. Science (1980).Google Scholar
- Patrick R. Laughlin, Bryan L. Bonner, and Andrew G. Miner. 2002. Groups perform better than the best individuals on Letters-to-Numbers problems. Organizational Behavior and Human Decision Processes 88, 2 (7 2002), 605--620. https://doi.org/10.1016/S0749--5978(02)00003--1Google Scholar
- Jean Lave and Etienne Wenger. 1991. Situated learning: Legitimate peripheral participation. Cambridge university press.Google Scholar
- Hjalti Leifsson and Jóhann Örn Bjarkason. 2015. Project Discovery Advancing scientic research by implementing citizen science in EVE Online Supervisor. Ph. D. Dissertation. Reykjavik University.Google Scholar
- Eva Lewandowski and Hannah Specht. 2015. Influence of volunteer and project characteristics on data quality of biological surveys. Conservation Biology 29, 3 (1 6 2015), 713--723. https://doi.org/10.1111/COBI.12481Google Scholar
- D. W. Liang, R. Moreland, and L. Argote. 1995. Group Versus Individual Training and Group Performance: The Mediating Role of Transactive Memory. Personality and Social Psychology Bulletin 21, 4 (1 4 1995), 384--393. https://doi.org/10.1177/0146167295214009Google Scholar
- Yu Pin Lin, Dongpo Deng, Wei Chih Lin, Rob Lemmens, Neville D. Crossman, Klaus Henle, and Dirk S. Schmeller. 2015. Uncertainty analysis of crowd-sourced and professionally collected field data used in species distribution models of Taiwanese moths. Biological Conservation 181 (1 1 2015), 102--110. https://doi.org/10.1016/J.BIOCON.2014.11.012Google Scholar
- Marcelo T. Lopes, Lucas L. Gioppo, Thiago T. Higushi, Celso A.A. Kaestner, Carlos N. Silla, and Alessandro L. Koerich. 2011. Automatic bird species identification for large number of species. Proceedings - 2011 IEEE International Symposium on Multimedia, ISM 2011 (2011), 117--122. https://doi.org/10.1109/ISM.2011.27Google ScholarDigital Library
- Y. Lou, P. C. Abrami, J. C. Spence, C. Poulsen, B. Chambers, and S. d'Apollonia. 1996. Within-Class Grouping: A Meta-Analysis. Review of Educational Research 66, 4 (1 1 1996), 423--458. https://doi.org/10.3102/00346543066004423Google ScholarCross Ref
- M Luczak-Roesch, R Tinati, E Simperl, and Van M Kleek. 2014. Why Won't Aliens Talk to Us? Content and Community Dynamics in Online Citizen Science. ICWSM (2014).Google Scholar
- Robin Martin, Antonis Gardikiotis, and Miles Hewstone. 2002. Levels of consensus and majority and minority influence. European Journal of Social Psychology 32, 5 (9 2002), 645--665. https://doi.org/10.1002/ejsp.113Google ScholarCross Ref
- Serge Moscovici and Willem Doise. 1994. Conflict and consensus: A general theory of collective decisions. Sage.Google Scholar
- Gabriel Mugar, Carsten Østerlund, Katie Devries Hassman, Kevin Crowston, and Corey Brian Jackson. 2014. Planet hunters and seafloor explorers: Legitimate peripheral participation through practice proxies in online citizen science. Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW (2014), 109--119. https://doi.org/10.1145/2531602.2531721Google ScholarDigital Library
- CJ Nemeth and J Wachtler. 1983. Creative problem solving as a result of majority vs minority influence. (1983).Google Scholar
- Jill Nugent. 2018. INaturalist. Science Scope 41, 7 (2018), 12--13.Google ScholarCross Ref
- Takeshi Okada and Herbert A. Simon. 1997. Collaborative Discovery in a Scientific Domain. Cognitive Science 21, 2 (11 4 1997), 109--146. https://doi.org/10.1207/s15516709cog2102_1Google Scholar
- Victoria Palacin, Maria Angela Ferrario, Gary Hsieh, Antti Knutas, Annika Wolff, and Jari Porras. 2021. Human values and digital citizen science interactions. International Journal of Human-Computer Studies 149 (may 2021), 102605. https://doi.org/10.1016/J.IJHCS.2021.102605Google Scholar
- Lorenzo Palamenghi, Serena Barello, Stefania Boccia, and Guendalina Graffigna. 2020. Mistrust in biomedical research and vaccine hesitancy: the forefront challenge in the battle against COVID-19 in Italy. European journal of epidemiology 35, 8 (2020), 785--788.Google Scholar
- Parva Panahi, Parviz Birjandi, and Behrooz Azabdaftari. 2013. Toward a sociocultural approach to feedback provision in L2 writing classrooms: the alignment of dynamic assessment and teacher error feedback. Language Testing in Asia 3, 1 (1 12 2013). https://doi.org/10.1186/2229-0443--3--13Google Scholar
- Roy D Pea. 1993. Learning scientific concepts through material and social activities: Conversational analysis meets conceptual change. Educational psychologist 28, 3 (1993), 265--277.Google Scholar
- Roy D Pea. 1993. Practices of distributed intelligence and designs for education. Distributed cognitions: Psychological and educational considerations 11 (1993), 47--87.Google Scholar
- Jennifer Preece. 2016. Citizen Science: New Research Challenges for Human--Computer Interaction. International Journal of Human-Computer Interaction 32, 8 (2016), 585--612. https://doi.org/10.1080/10447318.2016.1194153Google ScholarCross Ref
- R Core Team. 2021. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/Google Scholar
- Mark S. Reed, Anna C. Evely, Georgina Cundill, Ioan Fazey, Jayne Glass, Adele Laing, Jens Newig, Brad Parrish, Christina Prell, Chris Raymond, and Lindsay C. Stringer. 2010. What is Social Learning? Ecology and Society 15, 4 (2010). http://www.jstor.org/stable/26268235Google Scholar
- Diana Rose, Danielle Efraim, Marie-Claude Gervais, Helene Joffe, Sandra Jovchelovitch, and Nicola Morant. 1995. Questioning consensus in social representation theory. Papers on social representations 4 (1995), 150--176.Google Scholar
- Arthur D. Santana. 2014. Virtuous or Vitriolic. Journalism Practice 8, 1 (2 1 2014), 18--33. https://doi.org/10.1080/17512786.2013.813194Google ScholarCross Ref
- Alan H. Schoenfeld and Douglas J. Herrmann. 1982. Problem perception and knowledge structure in expert and novice mathematical problem solvers. Journal of Experimental Psychology: Learning, Memory, and Cognition 8, 5 (1982), 484--494. https://doi.org/10.1037/0278--7393.8.5.484Google ScholarCross Ref
- Nirwan Sharma, Laura Colucci-Gray, Advaith Siddharthan, Richard Comont, and René Van Der Wal. 2019. Designing online species identification tools for biological recording: The impact on data quality and citizen science learning. PeerJ 2019, 1 (28 1 2019), e5965. https://doi.org/10.7717/peerj.5965Google Scholar
- Nirwan Sharma, Sam Greaves, Advaith Siddharthan, Helen B. Anderson, Annie Robinson, Laura Colucci-Gray, Agung Toto Wibowo, Helen Bostock, Andrew Salisbury, Stuart Roberts, David Slawson, and René van der Wal. 2019. From citizen science to citizen action: Analysing the potential for a digital platform to cultivate attachments to nature. Journal of Science Communication 18, 1 (2019). https://doi.org/10.22323/2.18010207Google ScholarCross Ref
- Advaith Siddharthan, Christopher Lambin, Anne-Marie Robinson, Nirwan Sharma, Richard Comont, Elaine O'mahony, Chris Mellish, and Van Der René Wal. 2016. Crowdsourcing Without a Crowd. ACM Transactions on Intelligent Systems and Technology 7, 4 (5 5 2016), 1--20. https://doi.org/10.1145/2776896Google ScholarDigital Library
- Jonathan Silvertown, Martin Harvey, Richard Greenwood, Mike Dodd, Jon Rosewell, Tony Rebelo, Janice Ansine, and Kevin McConway. 2015. Crowdsourcing the identification of organisms: A case-study of iSpot. ZooKeys 480, 480 (2 1 2015), 125--46. https://doi.org/10.3897/zookeys.480.8803Google Scholar
- Gerry Stahl, Ulrike Cress, Sten Ludvigsen, Nancy Law, G Stahl, U Cress, S Ludvigsen, and N Law. 2014. Dialogic foundations of CSCL. International Journal of Computer-Supported Collaborative Learning 2014 9:2 9, 2 (may 2014), 117--125. https://doi.org/10.1007/S11412-014--9194--7Google Scholar
- Jonathan Steinke, van Jacob Etten, and Pablo Mejía Zelan. 2017. The accuracy of farmer-generated data in an agricultural citizen science methodology. Agronomy for Sustainable Development 2017 37:4 37, 4 (24 7 2017), 1--12. https://doi.org/10.1007/S13593-017-0441-YGoogle Scholar
- Alexandra Swanson, Margaret Kosmala, Chris Lintott, and Craig Packer. 2016. A generalized approach for producing, quantifying, and validating citizen science data from wildlife images. Conservation Biology 30, 3 (6 2016), 520--531. https://doi.org/10.1111/cobi.12695Google Scholar
- E.J. Theobald, A.K. Ettinger, H.K. Burgess, L.B. DeBey, N.R. Schmidt, H.E. Froehlich, C. Wagner, J. HilleRisLambers, J. Tewksbury, M.A. Harsch, and J.K. Parrish. 2015. Global change and local solutions: Tapping the unrealized potential of citizen science for biodiversity research. Biological Conservation 181 (2015), 236--244. https://doi.org/10.1016/j.biocon.2014.10.021Google ScholarCross Ref
- Ramine Tinati, Elena Simperl, and Markus Luczak-Roesch. 2017. To help or hinder: Real-time chat in citizen science. Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017, 270--279.Google ScholarCross Ref
- Stefan Trausan-Matu, Rupert Wegerif, and Louis Major. 2021. Dialogism. (2021), 219--239. https://doi.org/10.1007/978--3-030--65291--3_12Google Scholar
- René Van der Wal, Nirwan Sharma, Chris Mellish, Annie Robinson, and Advaith Siddharthan. 2016. The role of automated feedback in training and retaining biological recorders for citizen science. Conservation Biology 30, 3 (1 6 2016), 550--561. https://doi.org/10.1111/COBI.12705Google Scholar
- Ivar E. Vermeulen and Daphne Seegers. 2009. Tried and tested: The impact of online hotel reviews on consumer consideration. Tourism Management 30, 1 (2009), 123--127. https://doi.org/10.1016/j.tourman.2008.04.008Google ScholarCross Ref
- David Evans Walter and Shaun Winterton. 2007. Keys and the crisis in taxonomy: extinction or reinvention? Annual review of entomology 52 (12 1 2007), 193--208. https://doi.org/10.1146/annurev.ento.51.110104.151054Google Scholar
- Guiming Wang. 2019. Machine learning for inferring animal behavior from location and movement data. Ecological Informatics 49 (1 1 2019), 69--76. https://doi.org/10.1016/J.ECOINF.2018.12.002Google Scholar
- Etienne Wenger. 1999. Communities of practice: Learning, meaning, and identity. Cambridge university press.Google Scholar
- Sarah West and Rachel Pateman. 2016. Recruiting and Retaining Participants in Citizen Science: What Can Be Learned from the Volunteering Literature? Citizen Science: Theory and Practice 1, 2 (31 12 2016), 15. https://doi.org/10.5334/cstp.8Google Scholar
- Senuri Wijenayake, Niels van Berkel, Vassilis Kostakos, and Jorge Goncalves. 2020. Impact of contextual and personal determinants on online social conformity. Computers in Human Behavior 108 (1 7 2020), 106302. https://doi.org/10.1016/J.CHB.2020.106302Google Scholar
- J C Woodcock, A Greenhill, K Holmes, G Graham, J Cox, E Y Oh, and K Masters. 2017. Crowdsourcing citizen science: exploring the tensions between paid professionals and users. Journal of Peer Production 10 (2017).Google Scholar
- Gamze Yilmaz and Reef Youngreen. 2016. The Application of Minority Influence Theory in Computer-Mediated Communication Groups. Small Group Research 47, 6 (12 2016), 692--719. https://doi.org/10.1177/1046496416661033Google ScholarCross Ref
- Lixiu Yu, Paul André, Aniket Kittur, and Robert Kraut. 2014. A comparison of social, learning, and financial strategies on crowd engagement and output quality. Proceedings of the 17th ACM conference on Computer supported cooperative work and social computing - CSCW'14 (2014), 967--978. https://doi.org/10.1145/2531602.2531729Google ScholarDigital Library
- Lixiu Yu and V. Jeffrey Nickerson. 2011. Cooks or cobblers? Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11, 1393. https://doi.org/10.1145/1978942.1979147Google ScholarDigital Library
- Dana L Zeidler. 1997. The central role of fallacious thinking in science education. Science Education 81, 4 (1997), 483--496.Google ScholarCross Ref
- Haiyi Zhu, Robert Kraut, and Aniket Kittur. 2012. Organizing without Formal Organization: Group Identification, Goal Setting and Social Modeling in Directing Online Production. CSCW '12 Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, 935--944. https://doi.org/10.1145/2145204.2145344Google ScholarDigital Library
- B J Zimmerman. 2001. Social Learning, Cognition, and Personality Development. In International Encyclopedia of the Social Behavioral Sciences, Neil J Smelser and Paul B Baltes (Eds.). Pergamon, Oxford, 14341--14345. https://doi.org/10.1016/B0-08-043076--7/01765--4Google Scholar
Index Terms
- Consensus Building in On-Line Citizen Science
Recommendations
Species Identification in Citizen Science: Effects of Interface Design and Image Difficulty on User Performance and Workload
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsSpecies identification is an important scientific activity for ecological monitoring and digital technology has enabled citizen science projects to recruit volunteers for collecting and processing species information. Experts can easily identify species ...
Technology and work practices in citizen science
ASIS&T '10: Proceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem - Volume 47Citizen science is a form of research collaboration that actively involves the public in scientific research to address real-world problems. Research designed specifically for public participation is a form of information work for which the design of ...
The Science of Citizen Science: Theories, Methodologies and Platforms
CSCW '17 Companion: Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social ComputingCitizen science is a form of collaboration that engages non-professionals as contributors to scientific research, typically through the processes of gathering, transforming or analyzing data. To date, research has documented examples of hugely ...
Comments