CS547 Human-Computer Interaction Seminar   (Seminar on People, Computers, and Design)

Fridays 11:30am-12:30pm PT · Gates B3 · Open to the public
Previous | Next
Philip R. Cohen
Computer Dialogue Laboratory, Artificial Intelligence Center, SRI International
The Role of Natural Language in a Multimodal Interface
January 13, 1993

Although graphics and direct manipulation are effective interface
technologies for some classes of problems, they are limited in
many ways. In particular, they provide little support for identifying
objects not on the screen, for specifying temporal relations,
for identifying and operating on large sets and subsets of entities,
and for using the context of interaction. On the other hand,
these are precisely strengths of natural language. This talk
discusses how to build interfaces that blend natural language
processing and direct manipulation technologies, using the characteristic
advantages of each to overcome weaknesses of the other. Specifically,
I will show how to use natural language to describe objects and
temporal relations, and how to use direct manipulation for overcoming
hard natural language problems involving the establishment and
use of context and pronominal reference. This work has been implemented
in SRI's Shoptalk system, a prototype information and decision-support
system for manufacturing.


Dr. Cohen is Senior Computer Scientist in the Natural Language
Group of the Artificial Intelligence Center at SRI, is a Principal
Researcher with Stanford's Center for the Study of Language and
Information, and a Consulting Associate Professor with the Symbolic
Systems Program at Stanford. His current research interests include
multimodal interface design and its application to mobile computing
and information management; theoretical, computational, and empirical
models of dialogue; theories of intention, speech acts, and collaboration;
and collaborative simulation-based training.