Speaker Dengyong Zhou
Host Ofer Dekel
Date recorded 26 October 2012
In many real-world machine learning applications, we have training examples labeled by a crowd of workers with varied expertise rather than by an expert. Collecting labels from a crowd is usually cheap, but the label quality is low. For solving this problem, we propose a minimax entropy principle to simultaneously estimate worker expertise, task ambiguities, and true labels with a game theoretic interpretation. Moreover, we suggest an objectivity requirement for reasonably measuring worker expertise and task ambiguities, and show that the proposed method is unique in meeting the objectivity requirement. Experiments on real crowdsourcing data show that our minimax entropy approach dramatically outperforms states of the arts. It is a joint work with Sumit Basu, Yi Mao and John Platt.
©2012 Microsoft Corporation. All rights reserved.