For best experience please turn on javascript and use a modern browser!
Our Research

Experiments

Laboratory experiments can show causal relations while controlling much better for disturbing factors than is possible in field studies. Those who are skeptical about external validity of experimental findings might be surprised to learn how accurately they predict the same effect in the field, and how much better they can be replicated than field studies can:

  • Herbst, Daniel and Mas, Alexandre (2015) Peer effects on worker output in the laboratory generalize to the field. Science 350: 545-549.
  • Camerer, Colin et al. (2016) Evaluating replicability of laboratory experiments in economics. Science 351: 1433-1436. 

Of well-conducted experiments, the question is not if the results are externally valid, which they oftentimes are at the scale at which the results are obtained, but if the causal mechanism found at that scale, say a small group, can be generalized to different scales, e.g. national states.

Introduction

  • Jackson, Michelle and Cox. D.R. (2013) The principles of experimental design and their application in sociology.  Annual Review of Sociology 39: 27-49. 
  • Al-Ubaydli, Oma rand List, John A. (2015) External validity of field experiments. American Economic Review 105: 462-466.
  • Susan D. Hyde (2015) Experiments in international relations: Lab, survey, and field. Annual Review of Political Science 18: 403-424.
  • Baldassarri, Delia and Abascal, Maria (2017) Field experiments across the social sciences. Annual Review of Sociology 43: 41-73.
  • Ernst Fehr and Herbert Gintis (2007) Human motivation and social cooperation.  Annual Review of Sociology 33: 43-64.
  • Uskali Mäki (2005) Models are experiments, experiments are models. Journal of Economic Methodology 12: 303-315.

Textbooks

  • Colin F. Camerer (2003) Behavioral game theory.
  • Morton, Rebecca, and Kenneth Williams.  (2010).  Experimental Political Science and the Study of Causality. From Nature to the Lab.  Cambridge: Cambridge University Press.
  • Druckman, James, Donald Green, J. Kuklinski, and A. Lupia, eds.  2011.  Cambridge Handbook of Experimental Political Science.  Cambridge: Cambridge University Press. 

Practicalities: (1) recruiting subjects

Universities with a lab usually have a website to recruit students. Another option is to recruit them online through  Mechanical Turk, although its reliability is disputed:

  • Bohannon, Joseph (2016) Mechanical Turk upends social sciences. Science 352: 1263. 

Most experiments in Europe and the VS are done on WEIRDos: they are from Western, educated, industrialized, rich, and democratic cultures, according to

  • Joseph Henrich, Steven J. Heine, and Ara Norenzayan (2010), Most people are not WEIRD.  Nature 466: 29.

The implication is that experiments should be replicated across the planet: See Joseph Henrich's work, or a laid back introduction by:

  • Jean Ensminger (2002) From nomads in Kenya to Small-Town America: experimental economics in the bush.  Engineering and Science 2: 7-16.

Notice that experiments conducted in the field are not the same as field experiments, because for the latter there is no random assignment of subjects to treatments. For all experiments, researchers must obey strict protocols on how to treat subjects, for ethical reasons and to obtain valid results.

Fortunately, within Western societies, people who are recruited for experiments (usually students) do not differ from the remainder population, but Western people are not representative for the rest of the world: 

  • Filippos Exadaktylos, Antonio M. Espin and Pablo Brañas-Garza (2013) Experimental subjects are not different. Scientific Reports 3: 1213.
  • Daniel Hruschka e.a. (2018) Pressing questions in the study of psychological and behavioral diversity. Special issue of PNAS 115: 11366 ff.

(2) computer interface

Subjects often get to see the instructions for the experiment, while researchers collect their data, through a computer interface. Currently, the widely used z-tree program, which is a monster, is giving way to more user friendly software, among others the open source program oTree.

(3) data and analysis

A file of experimental data is not any different from a survey study, i.e. each subject has one row; columns are treatments, choices, and payoffs. The statistical analysis of experimental data is most of the times much simpler than of survey data, and usually consists of comparing groups with different treatments pair-wise; see this  overview of options. 

Examples

An early one:

  • Alex Bavelas (1950) Communication patterns in task-oriented groups.  Journal of the Accoustical Society of America 22: 725–730.

An anthropological experiment in the field:

  • Coren Apicella e.a. (2012) Social networks and cooperation in hunter gatherers.  Nature 481: 497-501.

A Web experiment:

  • Matthew Salganik, Peter Sheridan Dodds, and Duncan Watts (2006) Experimental study of inequality and unpredictability in an artificial cultural market.  Science 311: 854-856.

Experts at the AISSR are Theresa Kuhn, Bertjan Doosje, and Abbey Steele for field experiments.