CS547 Human-Computer Interaction Seminar (Seminar on People, Computers, and Design)
Fridays 11:30am-12:30pm PT · Gates B3 · Open to the public|
Dan Weld University of Washington High-Quality Crowdsourcing
May 19, 2017
Requestors often complain about the low-quality of crowd work, but whose fault is this? We argue that techniques like majority vote and expectation maximization (EM) miss the point and donât solve the true, underlying problems: confusing task instructions and poor worker training. Instead we advocate three new methods: 1) gated instruction, 2) adaptive testing, 3) micro-argumentation, and 4) self-improving workflows. These methods fuse ideas from HCI with AI methods such as partially-observable Markov decision problems & reinforcement learning. As a bonus, weâll present recent work on active learning, where the crowd does more than just label examples. |
|
