| | |
Eytan Bakshy, Dean Eckles, Michael Bernstein
| |
WWW: International World Wide Web Conference, 2014
|
|
Online experiments are widely used to compare specific design alternatives, but they can also be used to produce generalizable knowledge and inform strategic decision making. Doing so often requires sophisticated experimental designs, iterative refinement, and careful logging and analysis. Few tools exist that support these needs. We thus introduce a language for online field experiments called PlanOut. PlanOut separates experimental de- sign from application code, allowing the experimenter to concisely describe common designs, such as "A/B tests" and factorial designs, as well as more complex designs involving conditional logic or multiple experimental units. It also allows experimenters to implement designs involving multiple randomization units, enabling them to identify causal mechanisms that underlie complex user dynamics. We describe two field experiments conducted on Facebook using PlanOut, and demonstrate how PlanOut can implement experiments from the literature. For common scenarios in which experiments are run iteratively and in parallel, we introduce a namespaced management system that encourages sound experimental practice.
|
|
|
This project is known for:
online experiments , data science