Ken Hinckley
Microsoft Research
High Performance Pen Interfaces
October 22, 2004
Much of the user experience with current pen-operated mobile devices is clumsily
adapted from the point-and-click interfaces of desktop graphical interfaces.
I will discuss some efforts at Microsoft Research to advance the state-of-the-art
in pen interfaces, including new capabilities made possible by using the pen
for collaborative interactions, as well as my take on the classic problems
of gesture-based interaction. The talk will focus on two projects, Stitching
and Scriboli.
Stitching is a new interaction technique that allows users to forge wireless
network connections between pen-operated mobile devices, by using pen gestures
that span multiple displays. To stitch, a user starts moving the pen on one
screen, crosses over the bezel, and finishes the stroke on the screen of a
nearby device. This can be used as the basis for a variety of collaborative
tasks, such as copying images from one tablet to another that is nearby, expanding
an image across multiple screens, or establishing a persistent shared workspace.
I will also discuss design issues that arise from proxemics, that is, the
sociological implications of users collaborating in close quarters.
Scriboli is a pen interaction testbed. Our fundamental goal in Scriboli is
to design a grammar for pen interfaces, including nouns (scopes), verbs (commands),
and delimiters that help the system determine the structure of pen gesture
input phrases. I will discuss our initial progress towards this goal and demonstrate
how Scriboli can support integrated scope selection, command activation, and
direct manipulation all in a single fluid pen gesture. Hence the work offers
a nice extension of Stanford alumni Francois Guimbretiere's work with the
FlowMenu (Francois is also a collaborator on Scriboli). The name "Scriboli"
is a play on the Tivoli electronic whiteboard (Xerox PARC) that evokes the
fast and informal "scribbling" nature of the interactions we seek
to design.
|
Ken Hinckley is a research scientist at Microsoft Research. His interests
span many areas of human-computer interaction including input devices, interaction
techniques, sensors and sensing systems, mobile devices, and ubiquitous computing.
For recent papers, see his
publications.
The projects I will discuss are collaborative efforts. Collaborators on Stitching
are Gonzalo Ramos, Francois Guimbretiere, Patrick Baudisch, and Marc Smith;
for Scriboli, they are Patrick Baudisch, Gonzalo Ramos, and Francois Guimbretiere.
Yang Li (first author), Zhiwei Guan, and James Landay of the University of
Washington conducted an ink/gesture mode switching study on which I was a
collaborator, and will discuss in the context of Scriboli.
|
|