PCD 12/8/06 Lau&Cypher

We have developed a system called Koala that enables users to record their actions in a web browser, play them back to automate those actions, and publish them on a wiki to share with a community. Within a corporation or other community, Koala acts as a "wikipedia of how-to knowledge". For instance, when one person figures out how to fill in all of the corporate purchasing codes and customer numbers to purchase a flat panel display, that person's colleagues can play back the recorded actions to purchase a monitor themselves.

Koala records actions as commands in plain English, using the textual labels that appear on a web page. When it interprets written commands in order to perform them, it loosely matches the words in the command with the words on the web page. As a result, Koala's language is largely understandable by both humans and Koala: People can freely write and edit commands without worrying about a rigid syntax, and instructions written for a person are often executable by Koala.

We will demonstrate the program, show how it combines programming by demonstration with social networks, describe various domains where we hope to apply Koala, and discuss our plans for development and research.

Tessa Lau is a Research Staff Member at IBM's Almaden Research Center. She completed her Ph.D. in computer science at the University of Washington in 2001. Her research goal is to give people tools to improve their productivity, enhance their creativity, and make them more effective. She is interested in information management, particularly personal information, and how people interact with and customize their working environment. She has done significant work in the area of programming by demonstration, giving end users the ability to automate repetitive tasks simply by showing the system how to perform the task a few times. More generally, she is interested in finding patterns in human behavior and human-centric information and building tools that exploit these patterns to enable people to do more with less work.

Allen Cypher is a Research Staff Member at IBM's Almaden Research Center. His main research interests are programming by demonstration and end-user programming -- giving all computer users capabilities that have traditionally belonged to programmers. Allen is a co-inventor of Stagecast Creator -- a program that enables children to create their own games and simulations and publish them on the Web. He is the editor of Watch What I Do: Programming by Demonstration. He created the Eager program, which was one of the first intelligent agents. Eager constantly monitored a user's actions on the computer, and when it detected a repetitive activity, it would write a program to perform that activity automatically. He received a B.A. in mathematics from Princeton University in 1975, a Ph.D. in computer science from Yale University in 1980, and spent several years as a post-doc in cognitive science at the University of California, San Diego.

Titles and abstracts for previous years are available by year and by speaker.

Koala: End user programming on the web