Skip to main content
SpringerLink
Log in

Crowdsourcing a Text Corpus is not a Game

  • Conference paper
  • First Online:
Digital Libraries: Providing Quality Information (ICADL 2015)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9469))

Included in the following conference series:

Abstract

Building language corpora for low resource languages such as South Africa’s isiXhosa is challenging because of limited digitized texts. Language corpora are needed for building information retrieval services such as search and translation and to support further online content creation. A novel solution was proposed to source original and relevant multilingual content by crowdsourcing translations via an online competitive game where participants would be paid for their contributions. Four experiments were conducted and the results support the idea that gamification by itself does not yield the widely expected benefits of increased motivation and engagement. We found that people do not volunteer without financial incentives, the form of payment does not matter, they would not continue contributing if the money is taken away and people preferred direct incentives and the possibility of incentives was not as strong a motivator.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Statistics South Africa: Census 2011 Census in Brief. Statistics South Africa, Pretoria (2012)

    Google Scholar 

  2. Eiselen, E., Puttkammer, M.: Developing text resources for ten South African languages. In: Proceedings of the LREC (2014)

    Google Scholar 

  3. Webb, V.N.: African Voices: An Introduction to the Languages and Linguistics of Africa. Oxford University Press (2000)

    Google Scholar 

  4. Johnson, K.K.: Xhosa-English Machine Translation: Working with a Low-Resource Language (2011)

    Google Scholar 

  5. Drummer, A.: Phrase-Based Machine Translation of Under-Resourced Languages (2013)

    Google Scholar 

  6. Jackson, C.B., Osterlund, C., Mugar, G., Hassman, K.D., Crowston, K.: Motivations for sustained participation in crowdsourcing: case studies of citizen science on the role of talk. In: 2015 48th Hawaii International Conference on System Sciences (HICSS), pp. 1624–1634 (2015)

    Google Scholar 

  7. Howe, J. Crowdsourcing: How the Power of the Crowd is Driving the Future of Business. Random House (2008)

    Google Scholar 

  8. Ross, J., Irani, L., Silberman, M., Zaldivar, A., Tomlinson, B.: Who are the crowdworkers?: shifting demographics in mechanical turk. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems, pp. 2863–2872 (2010)

    Google Scholar 

  9. Zaidan, O.F., Callison-Burch, C.: Crowdsourcing translation: professional quality from non-professionals. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 1220–1229 (2011)

    Google Scholar 

  10. Munro, R.: Crowdsourced translation for emergency response in Haiti: the global collaboration of local knowledge. In: AMTA Workshop on Collaborative Crowdsourcing for Translation (2010)

    Google Scholar 

  11. Geiger, D., Seedorf, S., Schulze, T., Nickerson, R.C., Schader, M.: Managing the crowd: towards a taxonomy of crowdsourcing processes (2011)

    Google Scholar 

  12. Negri, M., Mehdad, Y.: Creating a bi-lingual entailment corpus through translations with mechanical turk: $100 for a 10-day rush. In: Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pp. 212–216 (2010)

    Google Scholar 

  13. Callison-Burch, C.: Fast, cheap, and creative: evaluating translation quality using amazon’s mechanical turk. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 286–295 (2009)

    Google Scholar 

  14. Ambati, V., Vogel, S.: Can crowds build parallel corpora for machine translation systems? In: Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pp. 62–65 (2010)

    Google Scholar 

  15. Deterding, S., Sicart, M., Nacke, L., O’Hara, K., Dixon, D.: Gamification. Using game-design elements in non-gaming contexts. In: CHI 2011 Extended Abstracts on Human Factors in Computing Systems, pp. 2425–2428 (2011)

    Google Scholar 

  16. Eickhoff, C., Harris, C.G., de Vries, A.P., Srinivasan, P.: Quality through flow and immersion: gamifying crowdsourced relevance assessments. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 871–880 (2012)

    Google Scholar 

  17. Farzan, R., DiMicco, J.M., Millen, D.R., Brownholtz, B., Geyer, W., Dugan, C.: When the experiment is over: deploying an incentive system to all the users. In: Proceedings of the Symposium on Persuasive Technology, in conjunction with the AISB (2008)

    Google Scholar 

  18. Farzan, R., DiMicco, J.M., Millen, D.R., Dugan, C., Geyer, W., Brownholtz, E.A.: Results from deploying a participation incentive mechanism within the enterprise. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 563–572 (2008)

    Google Scholar 

  19. Montola, M., Nummenmaa, T., Lucero, A., Boberg, M., Korhonen, H.: Applying game achievement systems to enhance user experience in a photo sharing service. In: Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era, pp. 94–97 (2009)

    Google Scholar 

  20. Anderson, A., Huttenlocher, D., Kleinberg, J., Leskovec, J.: Steering user behavior with badges. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 95–106 (2013)

    Google Scholar 

  21. Denny, P.: The effect of virtual achievements on student engagement. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 763–772 (2013)

    Google Scholar 

  22. Dominguez, A., Saenz-de-Navarrete, J., De-Marcos, L., Fernández-Sanz, L., Pagés, C., Martínez-Herráiz, J.-J.: Gamifying Learning Experiences: Practical Implications and Outcomes. Comput. Educ. 63, 380–392 (2013)

    Article  Google Scholar 

  23. Fitz-Walter, Z., Tjondronegoro, D., Wyeth, P.: Orientation passport: using gamification to engage university students. In: Proceedings of the 23rd Australian Computer-Human Interaction Conference, pp. 122–125 (2011)

    Google Scholar 

  24. Halan, S., Rossen, B., Cendan, J., Lok, B.: High score! - motivation strategies for user participation in virtual human development. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 482–488. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for ICT4D, Department of Computer Science, University of Cape Town, Cape Town, South Africa

    Sean Packham & Hussein Suleman

Authors
  1. Sean Packham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hussein Suleman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sean Packham .

Editor information

Editors and Affiliations

  1. Yonsei University, Seoul, Korea (Republic of)

    Robert B. Allen

  2. School of ITEE, University of Queensland, St. Lucia, Queensland, Australia

    Jane Hunter

  3. School of Library & Info Sci, Kent State University, KENT, Ohio, USA

    Marcia L. Zeng

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Packham, S., Suleman, H. (2015). Crowdsourcing a Text Corpus is not a Game. In: Allen, R., Hunter, J., Zeng, M. (eds) Digital Libraries: Providing Quality Information. ICADL 2015. Lecture Notes in Computer Science(), vol 9469. Springer, Cham. https://doi.org/10.1007/978-3-319-27974-9_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27974-9_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27973-2

  • Online ISBN: 978-3-319-27974-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation