CS547 Human-Computer Interaction Seminar   (Seminar on People, Computers, and Design)

Fridays 11:30am-12:30pm PT · Gates B3 · Open to the public
Previous | Next
Ted Selker
IBM, Thomas J. Watson Research Center, Yorktown Heights, NY
Graphics as Visual Language
April 24, 1991

Graphics as Visual Language Visual language is the systematic
use of visual presentation (graphic) techniques to convey meaning.
Current computer user interfaces make extensive use of graphic
techniques in many areas. This talk defines structural elements
of visual language as a framework for describing computer graphics,
and describes a program called VIsual Elements Workspace (VIEW);
A Hypertext and Construction Tool for Learning about Visual Language.
Graphics presentations can be described as having the fol- lowing
visual elements: graphical alphabet the set of visual primitives
used in a visual language graphical syntax the composition of
primitives which form visual statements interaction language
the set of user to system primitives structure rules that combine
sub-languages to form a language The classification of the visual
elements can be viewed as a linguistic description of visual
language. VIEW uses a formally defined Elements of visual language
to help a user build a working visual language interface. A user
drives through VIEW using a mouse and keyboard defining a visual
language. The system started as and includes hypertext for describing
the elements of visual language as a visual linguistics. The
user scenario for defining a language starts by defining the
set of visual primitives or visual alphabet by selecting or creating
images, icons or symbols or gestures which will be later put
together into visual utterances. For alpha- betic items which
will be placed or moved in a visual lan- guage an input syntax
is selected and attached which uses pointing gesture or keyboard
actions. To define what an ut- terance in the language would
be the user associates a pars- ing syntax; using position size
time or other spatial temporal parsing definition associated
with the language. Finally, semantics should be associated with
the language to make it do something. The definition of semantics
is beyond the scope of the visual language definition, We allow
a user to associate one of our predefined semantics with the
lan- guage or write a script of their own.


Ted Selker is currently a Research Scientist at the IBM T.J.
Watson Research Center. His work focuses on developing new paradigms
for using computers: The COACH research system demonstrates the
use of adaptive user models in help/tutoring environments. The
VREP project is developing a "linguistics" for the
use of graphic techniques in computer interfaces. The J-key project
has developed an analog input device with many advantages over
a mouse. Previously, Mr. Selker has conducted research at Xerox
PARC, Atari, and the Robotics Laboratory at Stanford University.