Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Crowdsourcing Gold-HIT Creation at Scale: Challenges and Adaptive Exploration Approaches (Position Paper)

ittai abraham, Omar Alonso, Vasilis Kandylas, Rajesh Patel, Steven Shelford, Aleksandrs Slivkins, and Hai Wu

Abstract

Gold HITs — Human Intelligence Tasks with known answers — are commonly used to measure worker performance and data quality in industrial applications of crowdsourcing. We suggest adaptive exploration as a promising approach for automated, scalable Gold HIT creation. We substantiate this with initial experiments in a stylized model.

Details

Publication typeTechReport
NumberMSR-TR-2013-106
PublisherMicrosoft Research
> Publications > Crowdsourcing Gold-HIT Creation at Scale: Challenges and Adaptive Exploration Approaches (Position Paper)