Graphics as Visual Language

Ted Selker, IBM, Thomas J. Watson Research Center, Yorktown Heights, NY

Seminar on People, Computers, and Design
Stanford University April 24, 1991

 

Graphics as Visual Language Visual language is the systematic use of visual presentation (graphic) techniques to convey meaning. Current computer user interfaces make extensive use of graphic techniques in many areas. This talk defines structural elements of visual language as a framework for describing computer graphics, and describes a program called VIsual Elements Workspace (VIEW); A Hypertext and Construction Tool for Learning about Visual Language. Graphics presentations can be described as having the fol- lowing visual elements: graphical alphabet the set of visual primitives used in a visual language graphical syntax the composition of primitives which form visual statements interaction language the set of user to system primitives structure rules that combine sub-languages to form a language The classification of the visual elements can be viewed as a linguistic description of visual language. VIEW uses a formally defined Elements of visual language to help a user build a working visual language interface. A user drives through VIEW using a mouse and keyboard defining a visual language. The system started as and includes hypertext for describing the elements of visual language as a visual linguistics. The user scenario for defining a language starts by defining the set of visual primitives or visual alphabet by selecting or creating images, icons or symbols or gestures which will be later put together into visual utterances. For alpha- betic items which will be placed or moved in a visual lan- guage an input syntax is selected and attached which uses pointing gesture or keyboard actions. To define what an ut- terance in the language would be the user associates a pars- ing syntax; using position size time or other spatial temporal parsing definition associated with the language. Finally, semantics should be associated with the language to make it do something. The definition of semantics is beyond the scope of the visual language definition, We allow a user to associate one of our predefined semantics with the lan- guage or write a script of their own.

 

Ted Selker is currently a Research Scientist at the IBM T.J. Watson Research Center. His work focuses on developing new paradigms for using computers: The COACH research system demonstrates the use of adaptive user models in help/tutoring environments. The VREP project is developing a "linguistics" for the use of graphic techniques in computer interfaces. The J-key project has developed an analog input device with many advantages over a mouse. Previously, Mr. Selker has conducted research at Xerox PARC, Atari, and the Robotics Laboratory at Stanford University.

 

Titles and abstracts for all years are available by year and by speaker.

For more information about HCI at Stanford see