Transforming Graphical User Interfaces into Auditory User Interfaces
May 26, 1995
Computers in non-desktop environments have become ubiquitous
over the past five years. But now we are faced with the chore
of learning three applications just to read electronic mail at
our desk, on the phone, and with a pager. One vision for the
future is that a set of applications would follow the user from
the desk, to the car, to the warehouse, and to the board room.
The application interfaces would be transformed to suit each
new environment while remaining familiar to the user. One piece
of this vision is transforming graphical interfaces into auditory
interfaces.
In this talk, I will describe a methodology for automatically
transforming graphical interfaces into auditory interfaces. This
methodology addresses analyzing and modeling the existing graphical
interface, as well as designing auditory interfaces to represent
the interface model. In this transformation, the salient characteristics
of the graphical interface are conveyed using combinations of
speech and nonspeech auditory output. The underlying model of
the graphical interface, an annotated tree structure, forms the
basis for navigating the auditory interface. I have investigated
these ideas by designing an interface for blind users accessing
graphical applications. With my system, Mercator, blind users
can work with auditory versions of the graphical applications
used by their sighted colleagues.
Elizabeth Mynatt is a Research Scientist in the Graphics,
Visualization and Usability Center at Georgia Tech. She directs
the Multimedia Computing Group which focuses on research in auditory
and collaborative interfaces. Elizabeth is the designer of the
Mercator interface which transforms graphical interfaces into
auditory interfaces that are accessible for blind computer users.
Her main interest is in integrating computers into the physical
environment. To that end, she will be continuing to investigate
transforming user interfaces to meet user and environmental constraints,
as well as designing peripheral interfaces for collaborative
systems.