Creation-oriented software applications such as photo editors and word processors are often difficult to test with traditional laboratory usability testing methods. A diversity of creation goals and strategies results in a diversity of usability problems encountered by users. This diversity in problems translates into the need for a large pool of participants in order to identify a high percentage of the problems. However, recruiting a large pool of participants can be prohibitively expensive, due to the high costs of traditional, expert-moderated thinkaloud usability testing. To address this problem, this dissertation contributes a new usability evaluation method called backtracking analysis, designed to automate the process of detecting and characterizing usability problems in creation-oriented applications. The key insight is that interaction breakdowns in creation-oriented applications often manifest themselves in simple backtracking operations that can be automatically logged (e.g., undo operations, erase operations, and abort operations). Backtracking analysis synchronizes these events to contextual data such as screen capture video, helping the evaluator to characterize specific usability problems.
|
