Abstract
Group behavior experiments require potentially large numbers of participants to interact in real time with perfect information about one another. In this paper, we address the methodological challenge of developing and conducting such experiments on the web, thereby broadening access to online labor markets as well as allowing for participation through mobile devices. In particular, we combine a set of recent web development technologies, including Node.js with the Socket.io module, HTML5 canvas, and jQuery, to provide a secure platform for pedagogical demonstrations and scalable, unsupervised experiment administration. Template code is provided for an example real-time behavioral game theory experiment which automatically pairs participants into dyads and places them into a virtual world. In total, this treatment is intended to allow those with a background in non-web-based programming to modify the template, which handles the technical server–client networking details, for their own experiments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bishop, M. (2002). Computer security: Art and science. Addison-Wesley Professional.
Boer, K., Kaymak, U., & Spiering, J. (2007). From discrete-time models to continuous-time, asynchronous modeling of financial markets. Computational Intelligence, 23(2), 142–161.
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5.
Cao, M., Morse, A. S., & Anderson, B. D. (2008). Agreeing asynchronously. IEEE Transactions on Automatic Control, 53(8), 1826–1838.
Charness, G., Oprea, R., & Friedman, D. (2012, April). Continuous time and communication in a public-goods experiment (University of California at Santa Barbara, Economics Working Paper Series No. qt5404914p). Department of Economics, UC Santa Barbara. Retrieved from http://ideas.repec.org/p/cdl/ucsbec/qt5404914p.html
Cialdini, R. B., & Goldstein, N. J. (2004). Social influence: Compliance and conformity. Annual Review of Psychology, 55, 591–621.
Dale, R., Fusaroli, R., Duran, N., & Richardson, D. C. (2014). The self-organization of human interaction. Psychology of Learning and Motivation, 59, 43–96.
Deck, C., & Nikiforakis, N. (2012). Perfect and imperfect real-time monitoring in a minimum-effort game. Experimental Economics, 15(1), 71–88.
Friedman, D., & Oprea, R. (2012). A continuous dilemma. The American Economic Review, 102(1), 337–363.
Goldin, G., & Darlow, A. (2013). TurkGate (version 0.4.0) [Computer software manual]. Providence, RI.
Goldstone, R. L., Ashpole, B. C., & Roberts, M. E. (2005). Knowledge of resources and competitors in human foraging. Psychonomic Bulletin & Review, 12(1), 81–87.
Helbing, D., Schönhof, M., Stark, H.-U., & Hołyst, J. A. (2005). How individuals learn to take turns: Emergence of alternating cooperation in a congestion game and the prisoner’s dilemma. Advances in Complex Systems, 8(1), 87–116.
Hughes-Croucher, T., & Wilson, M. (2012). Node: Up and running: Scalable server-side code with JavaScript. O’Reilly Media, Incorporated.
Lau, S.-H. P., & Mui, V.-L. (2008). Using turn taking to mitigate coordination and conflict problems in the repeated battle of the sexes game. Theory and Decision, 65(2), 153–183.
Majumder, S. R., Diermeier, D., Rietz, T. A., & Amaral, L. A. N. (2009). Price dynamics in political prediction markets. Proceedings of the National Academy of Sciences, 106(3), 679–684.
Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44(1), 1–23.
Mason, W., & Watts, D. J. (2010). Financial incentives and the performance of crowds. ACM SigKDD Explorations Newsletter, 11(2), 100–108.
McDonnell, J., Martin, J., Markant, D., Coenen, A., Rich, A., & Gureckis, T. (2012). psiturk (version 1.02) [Computer software manual]. New York, NY. Retrieved from https://github.com/NYUCCL/psiTurk
Michinov, N., & Primois, C. (2005). Improving productivity and creativity in online groups through social comparison process: New evidence for asynchronous electronic brainstorming. Computers in Human Behavior, 21(1), 11–28.
Miller, N., Garnier, S., Hartnett, A. T., & Couzin, I. D. (2013). Both information and social cohesion determine collective decisions in animal groups. Proceedings of the National Academy of Sciences, 110(13), 5263–5268.
Moussaïd, M., Helbing, D., Garnier, S., Johansson, A., Combe, M., & Theraulaz, G. (2009). Experimental study of the behavioural mechanisms underlying self-organization in human crowds. Proceedings of the Royal Society B: Biological Sciences, 276(1668), 2755–2762.
Olfati-Saber, R., Fax, J. A., & Murray, R. M. (2007). Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE, 95(1), 215–233.
Paolacci, G., Chandler, J., & Ipeirotis, P. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), 411–419.
Pettit, J., Friedman, D., Kephart, C., & Oprea, R. (2014). Software for continuous game experiments. Experimental Economics. doi:10.1007/s10683-013-9387-3
Reips, U.-D. (2002). Standards for internet-based experimenting. Experimental Psychology, 49(4), 243–256.
Spivey, M., & Dale, R. (2006). Continuous dynamics in real-time cognition. Current Directions in Psychological Science, 15(5), 207–211.
Suri, S., & Watts, D. J. (2011). Cooperation and contagion in web-based, networked public goods experiments. PLoS One, 6(3), e16836.
Takada, M. (2012). Mixu’s Node book: A book about using Node.js. Available at mixu.net.
Tilkov, S., & Vinoski, S. (2010). Node. js: Using JavaScript to build high-performance network programs. IEEE Internet Computing, 14(6), 80–83.
Author Note
This research was in part supported by National Science Foundation REESE grant 0910218 to the Percepts and Concepts Laboratory. We thank Shoshana Berleant and the members of the Santa Fe Institute mailing list for participating in usability tests of the framework and also Rob Goldstone and Johan Bollen for comments on early drafts.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hawkins, R.X.D. Conducting real-time multiplayer experiments on the web. Behav Res 47, 966–976 (2015). https://doi.org/10.3758/s13428-014-0515-6
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13428-014-0515-6