Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Belief Propagation Algorithms for Crowdsourcing

Speaker  Qiang Liu

Affiliation  UC Irvine

Host  Dengyong Zhou

Duration  00:53:24

Date recorded  29 January 2013

Crowdsourcing on platforms like Amazon's Mechanical Turk have become a popular paradigm for labeling large datasets. However, it has given rise to the computational task of properly aggregating the crowdsourced labels provided by a collection of unreliable and diverse annotators. On the other side, graphical models are powerful tools for reasoning on systems with complicated dependency structures. In this talk, we approach the crowdsourcing problem by transforming it into a standard inference problem in graphical models, and apply powerful inference algorithms such as belief propagation (BP). We show both the naive majority voting and a recent algorithm by Karger, Oh, and Shah are special cases of our BP algorithm under particular modeling choices. With more careful choices, we show that our simple BP performs surprisingly well on both simulated and real-world datasets, competitive with state-of-the-art algorithms based on more complicated modeling assumptions. Our work sheds light on the important tradeoff between better modeling choices and better inference algorithms.

©2013 Microsoft Corporation. All rights reserved.
> Belief Propagation Algorithms for Crowdsourcing