Title: Optimal Experimental Design in Online Experiments
Speaker: Chris Riedl
Abstract: We illustrate and extend a dynamic method to design and execute Bayesian optimal experiments. The method extends seminal work in statistics and computer science on Bayesian optimal design that have not taken hold due to computational limitations. Our procedure takes as primitives a class of parametric models of strategic behavior, a class of experimental designs, and priors on the behavioral parameters. The method then selects the experimental design that maximizes the information from the experiment and thus increases statistical power. We propose and evaluate two computational improvements to make the application of our approach more tractable. First, we develop a probabilistic bootstrapping procedure to reduce the number of hypothetical experimental outcomes that need to be considered. Second, we apply modern Artificial Intelligence methods to learn a multi-dimensional function to efficiently optimize the space of possible experiments. Together, these two improvements reduce computational complexity by several orders of magnitude over the naive approach while simultaneously increasing precision and reducing regret. This allows for a completely automated approach to designing experiments using only the space of possible experiments as input, which is critically important for the large scale experimentation performed by internet companies that run thousands of experiments daily.
Bio: Christoph Riedl is currently the Joseph G. Riesman assistant professor for Information Systems at the D’Amore-McKim School of Business at Northeastern University. He hold a joint appointment with the College of Computer & Information Science and is a core faculty at the Network Science Institute. He is a fellow at the Institute for Quantitative Social Science (IQSS) at Harvard University. He is recipient of a Young Investigator Award from Army Research Office for his work on social networks in collaborative decision-making. Before joining Northeastern University he was a post-doctoral fellow at Harvard Business School and IQSS. He received a PhD in Information Systems from Technische Universität München (TUM), Germany in 2011, a MSc in Information Systems in 2007, and a BSc in Computer Science in 2006. His work has been funded by NSF and published in leading journals including Management Science, Information Systems Research, and Academy of Management Discoveries.
Tuesday, November 7, 2017 at 11:00am to 12:00pm
Northeastern University, 511 Lake Hall