Human Computation

Tasks like image recognition are trivial for humans, but continue to challenge even the most sophisticated computer programs. This talk introduces a paradigm for utilizing human processing power to solve problems that computers cannot yet solve. Traditional approaches to solving such problems focus on improving software. I advocate a novel approach: constructively channel human brainpower using computer games. For example, the ESP Game, described in this talk, is an enjoyable online game – many people play over 40 hours a week – and when people play, they help label images on the Web with descriptive keywords. These keywords can be used to significantly improve the accuracy of image search. People play the game not because they want to help, but because they enjoy it. The ESP Game has been licensed by a major Internet company and will soon become the basis of their image search engine.

I describe other examples of “games with a purpose”: Peekaboom, which helps determine the location of objects in images, and Verbosity, which collects common-sense knowledge. I also explain a general approach for constructing games with a purpose.

In addition, I describe my work on CAPTCHAs, automated tests that humans can pass but computer programs cannot. CAPTCHAs take advantage of human processing power in order to differentiate humans from computers, an ability that has important applications in practice.

The results of this work are currently in use by hundreds of Web sites and companies around the world, and over 100,000 people have played some of the games presented here. Practical applications include improvements in areas such as: computer vision, image search, adult-content filtering, spam prevention, common-sense reasoning, accessibility, and security in general.