Robin Jeffries||Jim Miller · Human Computer Interaction Department HP Laboratories||Human Computer Interaction Department HP Laboratories
User Interface Evaluation in the Real World: A Comparison of Four Techniques

A good user interface is the optimal combination of a few good ideas, a thousand tradeoffs, and a million details. How do we find out which parts we didn't get quite right in the first few design iterations? Usability testing is the time-honored technique for discovering the places where the application doesn't fit the users. However, everyone complains that usability testing is too expensive, can't be done until it's too late to make changes, and requires people with expertise that is not readily available. Several other evaluation techniques have been proposed to overcome some or all of these limitations, but the relative utilities of these techniques are largely unknown. Our study applied four evaluation techniques -- usability testing, heuristic evaluation, guidelines, and cognitive walkthroughs -- to the same user interface, under as realistic conditions as we could manage. We will report on how the various techniques uncovered different kinds of problems with this interface, and will discuss the relative advantages of the four techniques. If time permits, we will discuss the differences in the ways evaluators and developers think about usability problems, and ways to improve communication between them.

Robin Jeffries is currently a project manager in the Human-Computer Interaction Department at HP Labs. She has a Ph.D. in Cognitive Psychology from the University of Colorado and was at Carnegie-Mellon University before coming to HP. Her research has been on various aspects of human problem solving and complex cognitive skill acquisition, ranging from software design to information access. Jim Miller is also a project manager in the Human-Computer Interaction Department at HP Labs. He has a Ph.D. in Cognitive Psychology from UCLA, and came to HP Labs from the MCC Human Interface Laboratory. His current research is focused on user interface design tools, especially those that enable non-programmers to develop task-specific interfaces and applications.