New! Apply now for postdoctoral research position in Online Experimental Social Science at MSR-NYC
Web platforms such as Amazon’s Mechanical Turk are revolutionizing our ability to conduct human behavioral experiments of the kind historically performed in physical labs. In recent years, researchers have replicated dozens of classical psychology and behavioral economics experiments in “virtual labs,” with remarkably consistent results, and have also begun to study networked experiments, in which many subjects participate simultaneously. Experiments of this kind can shed new light on the role of network structure, and even dynamics, to longstanding social scientific problems such as cooperation, learning, and collective problem solving. Beyond research on networks, virtual labs allow for individual-level psychology and economics experiments to be carried out with unprecedented scale and speed.
Virtual labs have a number of important advantages over their physical counterparts. In particular, virtual lab experiments are much cheaper and faster to conduct, allowing for larger experiments, many more experimental conditions, and many more replications per treatment. In this way, smaller effects will be detectable and wider ranges of parameters can be included in an experiment’s design. In addition, the speed with which new experiments can be designed and launched will lead to a much shorter hypothesis-testing loop than is possible in physical labs—potentially, hypotheses can now be modified and new experiments designed and conducted in a matter of hours rather than months or years. Finally, as the size and tenure of subject pools increase, it will become possible to sample subjects based not only on their demographic attributes, but also on their previous behavior, either in the same game or in different games.
Online experimental social science therefore requires a diverse mix of skills, including a deep understanding of the theoretical and experimental social science literature, experimental design, statistical modeling and analysis, significant programming expertise, and even UI design. As no one individual possesses all these skills in equal depth, we work in a collaborative and interdisciplinary manner.
- W. Mason and D.J. Watts, Collaborative learning in networks, in Proceedings of the National Academy of Sciences, vol. 109, no. 3, pp. 764–769, National Acad Sciences, 2012
- W. Mason and D.J. Watts, Financial incentives and the performance of crowds, in ACM SIGKDD Human Computation, 2009