High Performance Pen Interfaces

   Ken Hinckley, Microsoft Research
    kenhmicrosoft.com

Seminar on People, Computers, and Design
Stanford University October 22, 2004

Much of the user experience with current pen-operated mobile devices is clumsily adapted from the point-and-click interfaces of desktop graphical interfaces. I will discuss some efforts at Microsoft Research to advance the state-of-the-art in pen interfaces, including new capabilities made possible by using the pen for collaborative interactions, as well as my take on the classic problems of gesture-based interaction. The talk will focus on two projects, Stitching and Scriboli.

Stitching is a new interaction technique that allows users to forge wireless network connections between pen-operated mobile devices, by using pen gestures that span multiple displays. To stitch, a user starts moving the pen on one screen, crosses over the bezel, and finishes the stroke on the screen of a nearby device. This can be used as the basis for a variety of collaborative tasks, such as copying images from one tablet to another that is nearby, expanding an image across multiple screens, or establishing a persistent shared workspace. I will also discuss design issues that arise from proxemics, that is, the sociological implications of users collaborating in close quarters.

Scriboli is a pen interaction testbed. Our fundamental goal in Scriboli is to design a grammar for pen interfaces, including nouns (scopes), verbs (commands), and delimiters that help the system determine the structure of pen gesture input phrases. I will discuss our initial progress towards this goal and demonstrate how Scriboli can support integrated scope selection, command activation, and direct manipulation all in a single fluid pen gesture. Hence the work offers a nice extension of Stanford alumni Francois Guimbretiere's work with the FlowMenu (Francois is also a collaborator on Scriboli). The name "Scriboli" is a play on the Tivoli electronic whiteboard (Xerox PARC) that evokes the fast and informal "scribbling" nature of the interactions we seek to design.

Ken Hinckley is a research scientist at Microsoft Research. His interests span many areas of human-computer interaction including input devices, interaction techniques, sensors and sensing systems, mobile devices, and ubiquitous computing. For recent papers, see his publications.

The projects I will discuss are collaborative efforts. Collaborators on Stitching are Gonzalo Ramos, Francois Guimbretiere, Patrick Baudisch, and Marc Smith; for Scriboli, they are Patrick Baudisch, Gonzalo Ramos, and Francois Guimbretiere.

Yang Li (first author), Zhiwei Guan, and James Landay of the University of Washington conducted an ink/gesture mode switching study on which I was a collaborator, and will discuss in the context of Scriboli.

.

View this talk on line at CS547 on Stanford OnLine

Titles and abstracts for previous years are available by year and by speaker.