Crowdsourcing Gold-HIT Creation at Scale: Challenges and Adaptive Exploration Approaches (Position Paper)

Gold HITs --- Human Intelligence Tasks with known answers --- are commonly used to measure worker performance and data quality in industrial applications of crowdsourcing. We suggest adaptive exploration as a promising approach for automated, scalable Gold HIT creation. We substantiate this with initial experiments in a stylized model.

TR.pdf
PDF file

Publisher  Microsoft Research

Details

TypeTechReport
NumberMSR-TR-2013-106
> Publications > Crowdsourcing Gold-HIT Creation at Scale: Challenges and Adaptive Exploration Approaches (Position Paper)