Dan Weld · University of Washington
Human Intelligence Needs Artificial Intelligence

You need Flash player 8+ and JavaScript enabled to view this video.

Crowdsourcing platforms, such as Amazon Mechanical Turk, have enabled the construction of scalable applications for tasks ranging from product categorization and photo tagging to audio transcription and translation. These vertical applications are typically realized with complex, self-managing workflows that guarantee quality results. But constructing such workflows is challenging, with a huge number of alternative decisions for the designer to consider.

We argue the thesis that "Artificial intelligence methods can greatly simplify the process of creating and managing complex crowdsourced workflows." We present the design of CLOWDER, which uses machine learning to continually re-fine models of worker performance and task difficulty. Using these models, CLOWDER uses decision-theoretic optimization to 1) choose between alternative workflows, 2) optimize parameters for a workflow, 3) create personalized interfaces for individual workers, and 4) dynamically control the workflow. Preliminary experience suggests that these optimized workflows are significantly more economical (and return higher quality output) than those generated by humans.

Daniel S. Weld is Thomas J. Cable / WRF Professor of Computer Science and Engineering at the University of Washington. After formative education at Phillips Academy, he received bachelor's degrees in both Computer Science and Biochemistry at Yale University in 1982. He landed a Ph.D. from the MIT Artificial Intelligence Lab in 1988, received a Presidential Young Investigator's award in 1989, an Office of Naval Research Young Investigator's award in 1990, was named AAAI Fellow in 1999 and deemed ACM Fellow in 2005. Dan is an area editor for the Journal of the ACM, was a founding editor for the Journal of AI Research, was guest editor for Computational Intelligence and Artificial Intelligence, and was Program Chair for AAAI-96. Dan is an active entrepreneur with several patents and technology licenses. In May 1996, he co-founded Netbot Incorporated, creator of Jango Shopping Search and later acquired by Excite. In October 1998, Dan co-founded AdRelevance, a revolutionary monitoring service for internet advertising which was acquired by Media Metrix. In June 1999, Dan co-founded data integration company Nimble Technology which was acquired by the Actuate Corporation. In January 2001, Dan joined the Madrona Venture Group as a Venture Partner and member of the Technical Advisory Board.