Crowd-sourcing is increasingly being used for providing answers to online polls and surveys. However, existing systems, while taking care of the mechanics of attracting crowd workers, poll building, and payment, provide little that would help the survey-maker or pollster to obtain statistically significant results devoid of even the obvious selection biases.
InterPoll is a platform for programming of crowd-sourced polls. Polls are expressed as embedded LINQ queries, whose results are provided to the developer. InterPoll supports reasoning about uncertainty, enabling t-tests, etc. on random variables obtained from the crowd. InterPoll performs query optimization, as well as bias correction and power analysis, among other features. Making InterPoll queries part of the surrounding program allows for optimizations that take advantage of the surrounding code context. The goal of InterPoll is to provide a system that can be reliably used for research into marketing, social and political science questions.
Our ultimate goal is to make self-service crowd-sourced opinion polling available and affordable. To do this well, we are exploring ideas that come from programming languages, crowd-sourcing, databases, statistics, as well as security and privacy.
Hypothesis testing with power analysis
- Benjamin Livshits and Todd Mytkowicz, Saving Money While Polling with InterPoll using Power Analysis, AAAI - Association for the Advancement of Artificial Intelligence, 2 November 2014
- James Bornholt, Todd Mytkowicz, and Kathryn S. McKinley, Uncertain<T>: A First-Order Type for Uncertain Data, Architectural Support for Programming Languages and Operating Systems (ASPLOS), March 2014
- Benjamin Livshits and Todd Mytkowicz, InterPoll: Crowd-Sourced Internet Polls (Done Right), no. MSR-TR-2014-3, 7 January 2014
- Programming for Social Scientists (long presentation, about 70 slides)