Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Decision-Theoretic Crowdsourcing

Speaker  Mausam Mausam

Affiliation  University of Washington

Host  A Kumaran

Duration  01:12:34

Date recorded  10 September 2012

Crowdsourcing continues to rise in popularity today and is considered as the future of labor by many. Presently, crowdsourcing marketplaces (e.g., Amazon Mechanical Turk) have enabled the construction of scalable applications for tasks ranging from product categorization and photo tagging to audio transcription and language translation. These vertical applications are typically realized with complex, self-managing workflows that guarantee quality results. But constructing and controlling such workflows is challenging, with a huge number of alternative decisions for the designer to consider. We argue the thesis that artificial intelligence methods can greatly simplify the process of creating and managing complex crowdsourced workflows. We present the design of CLOWDER, which uses decision-theoretic techniques to dynamically optimize the workflows. Preliminary evaluations suggest that these optimized workflows are significantly more economical and return a much higher quality output than those generated by human designers.

©2012 Microsoft Corporation. All rights reserved.
> Decision-Theoretic Crowdsourcing