autumn 2007
Assignment 8: User Testing — Grade Value: 12.5%
Due by 11am on Thursday, November 29 (Submitted Online)
brief
Your goal is to test your functional prototype with at least three different people, generate user feedback and insights, and use this to inform revisions to your prototype your final project.
assignment
Develop a user test plan. Your should first decide what type of study to run: an experiment or a user study. Base this decision on what you would like to learn. You should then prepare an outline or agenda of how the user study will be run, written introductions that you will read to participants, and any other materials that will be used during the study (e.g. post-test questionnaires and interview questions). This will be similar to what you did for the paper prototype, but with greater depth.
Your high-level goal is to find problems with your interface. Look for breakdowns and pain points in your interface (you might want to look at Nielson's 10 Usability Heuristics), and try to understand what the problems are and how you might fix them. If possible, try modifying or updating your prototype before continuing to the next user.
Consider the different types of user tests that one can run, and choose the type of user test that you believe makes the most sense for your project. Make sure you can justify this decision! A great way to do this is to begin by writing down exactly what you hope to learn from the study and then use this list to inform your study planning. Different styles of studies were discussed in the "evaluation" lecture, and are detailed in the Sharp, Rogers, and Preece reading accompanying that lecture.
As mentioned in the previous assignment, the more realistic your prototype is, the more likely you are to uncover unanticipated design errors as the application can be used more naturally.
You will use the results of this assignment to inform your final iterations and updates for your group project. For more information on the scope of the final project, see the final project page.
design goals
- Develop an evaluation plan
- Get feedback from a variety of users
- Plan further iteration on your final project based on user feedback
submit online
- A description of your evaluation plan (200 words max), including justification for the plan you chose based on what you hope to learn
- Any questionnaires or surveys you wrote (either as text or a scanned image)
- A writeup of what revisions and changes you made to your prototype and why, based on what you learned from your study (200 words max)
- A paragraph on design reflections on this user test process, potentially in contrast with paper prototyping (100 words max)
- Photographs of your user test in action
evaluation criteria
- Experiment Design — how well-planned was the user test?
- Execution — how well did was it executed?
- Revision Rationalization — what did you decide to change and why?
in studio
- Be prepared to present findings from your evaluation (about 5 minutes)
- Bring your prototype for further heuristic evaluation from the rest of your studio members
grading rubric
The assignment was graded out of 125 points. The points were broken down as follows:
- 15 points for presentation
- 25 points for writeup and pictures
- 5 points - description of their evaluation plan
- 5 points - any questionnaires or surveys you wrote (either as text or a scanned image)
- 5 points - writeup of what revisions and changes you made to your prototype and why, based on what you learned from your study
- 5 points - paragraph on design reflections on this user test process, potentially in contrast with paper prototyping
- 5 points - photographs of your user test in action
- 30 points for experiment design
- how well-planned was the user test? did you justify your approach to the user study, what components they choose or choose not to do? is the test unbiased and designed to draw out interesting information?
- 20 points for execution
- how well was it executed? did your experiment succeed? did you find substantial issues with their design? if it didn't, did you make changes to their design to avoid further problems?
- 30 points for revision rationalization
- did they come up with multiple changes to make to their project based on user feedback? is it clear what that user feedback is? do you have justified reasons for making or not making particular changes?
- 5 points for interesting insights in your reflections
- did you find anything interesting while reflecting on the process?
student examples
