spring 2011

CS376: Research Topics in Human-Computer Interaction

Groups · Ideas · Project Abstract · Progress Meeting · Presentation · Paper

Project Overview

In this course, you will complete a quarter-long research project. This project will be completed in groups of two.

At a high level, successful projects will raise an important research question, and plan and execute a methodology for answering that question. Often, this methodology will include building and evaluating a prototype system, but hacking is not strictly necessary. All projects require a study — obviously a much more thorough study will be expected of projects that do not involve system building. The goal of the project abstract draft (described below) is to help you scope your work appropriately.

To get a sense of what a good scope for a project is, here are some examples of final papers from the last year:

2010 final papers

And a couple from the previous years:

For information on how the project will be evaluated, see the grading page.

Forming Groups

This project will be completed in groups of two (email cs376@cs if you'd like a larger group). Project groups will be self-paired.

At the end of class Monday April 6th, use the online submission system to submit the name(s) of who you will be working with. (All group members should complete a submission.)

To assist in finding a group, a message board will be available on the course submission site to post your ideas and communicate with others. Also, take a look at the opportunities for collaboration with individuals outside of the class.

Project Ideas

This year, there are two research themes: diverse form factors, and online collaboration. All of the research projects must in some way explore, touch on, or illustrate the increasing diversity of computing devices and/or modes of online collaboration. Here is a list of collaboration opportunities; we encourage you to take advantage of them as a way of participating in a long-term research endeavor. It will take some thinking to find a project where you can ask and answer a research question inside a 10-week quarter. You'll likely iterate on your idea several times -- take care not to fixate on one idea early on.

Project Abstract (Draft and Final Versions)

A draft of your project abstract is due at 7am on Friday, April 8th. Course staff will provide feedback on the draft to assist in the preparation of a final version, due at 7am on Friday, April 22. Both are submitted online.

The project abstract should cover the following topics:

Research Question: what are you trying to answer? State this as clearly as possible in one sentence.
Hypothesis: what do you think the answer to your question is, and why? State your hypothesis in terms that you will actually be able to deliver on within the space of a quarter. (For example, having a technology increase someone's income might be your ultimate goal, but you may not be able to measure a change in income in 10 weeks. In this case, increasing income qould be part of your motivation, but not your hypothesis. Your hypothesis needs a more proximal measure.)
Method: how will you explore your hypothesis, and why is that the right approach? (This should include the design of your study.) Grounding this in methodologies that other researchers have used (e.g. by drawing from the class readings) is a good idea.
- Study design: What are you going to do? Be very detailed and precise.
- Evaluation: How will you know you succeeded? What will you measure? How will you measure it?
- Ecological Validity: Why does your study answer your research question? Why does your evaluation address your hypothesis? Make sure your study, and the variables you're measuring, properly address the question you are asking.
Study Recruitment Plan: how will you get participants for your study? For pilot studies, we suggest you recruit from within the class -- "trading" participation with other groups is a great way to learn about what others are doing. For larger studies (e.g. for those not building a system), you need a clear recruitment plan.
Related Work: Please describe several pieces of published research, how they inform your project, and where your work trancends this knowledge. (You need not have related work in the draft version.)
Biggest Risk: what's the riskiest component of your project? (may not be able to get the hardware you need, robustly implementing the ___ algorithm may take too long, the difference between conditions may not be measurable, ...)
Motivation: (final only) What motivates you to explore this issue? What leads you to believe this is a problem/opportunity?
Design Space: (final only) What is the structure of the space of possibilities that your work explores. what are the major decisions from a design perspective and what are their relative merits. If it's helpful, you can use the design space in the Juxtapose paper as a guide.

For the draft, we expect you to cover all topics in ~3 paragraphs--be concise but concrete in your descriptions. For the final version, you'll want to go into greater depth (approximately 2 paragraphs for each issue, with the exception of the research question, which should still be be one precise sentence).

We encourage you to iterate multiple times on this abstract. While there is only one formally defined point for receiving feedback from course staff, you should seek out more informal feedback as you work on this. E-mail us at any point if you'd like us to take a look at your current submission, or come to office hours if you'd like to discuss in person. You are free to change directions after submitting your draft, but the sooner you nail down a direction, the better your project is likely to be.

Progress Meeting

Course staff will meet individually with each project group to provide feedback on your progress. These meetings will happen in the last 30 minutes of our scheduled class time. Please use this google doc to sign up for a time slot. You should be prepared to present your working system, discuss your study plan, and have pilot results. Use the online submission system to submit any materials you'd like to discuss (e.g., prototypes, data, draft writing.) Come to the meeting prepared to show and tell with preliminary results and how you plan to course-correct based on your early experimentation. How will you revise your system/design/experiment/framing so that your project really pops. What will the title of your final paper be? In other words, how will you summarize your research contribution in just a few words? This exercise will helps you focus and sharpen your efforts on what will best address your research question. This focus will be especially important as time gets tight: some things will matter more than others.

Final Presentation

At the end of the quarter, you will present your research results to the class and outside guests. We have invited a couple HCI luminaries. Feel free to invite interested friends and colleagues!

Each group has will have 5 minutes to present, plus 2 minutes for questions. This time limit will be strictly enforced – groups should set up during the question session of the group before them. To enable this, unplug the video cable from your laptop before answering questions.
Test (and debug) your laptop video projection before presentations begin. Time spent fiddling with display settings will count against your presentation time.
Structure your presentation like a pyramid — begin with a one-sentence statement of your research result. This will get everyone on the same page. Then, offer a short (e.g., 1 slide, 4 sentences) description of what you did and why you did it. Then, explain things in detail.
This presentation is short enough that you can write out everything you want to say long-hand. Do this! This will allow you to convey information efficiently and effectively. Read through it enough times so that you have it basically memorized, but not so memorized that you get flustered if you skip a word or someone asks a question.
Know your audience! You can expect that everyone in the class knows everything you learned in class. So, you don't need to re-introduce the whole field of HCI. A sentence or two to situate your work in the field is good, but spend the rest of the time telling us what you did.
When presenting, stand near your slides. And look at the audience.

Final Paper

In addition to the presentation, you will present your findings in a final paper. Final papers should be 3-5 pages long in the UIST format. While this may sound short, it is much harder to write an effective, complete short paper than it is to ramble.A good approach to writing a great short paper is to write a long one first, and then trim it down to the most vital parts. Much of the advice from above for preparing your presentation applies to the paper as well. Here are a few more suggestions for preparing your paper:

Find a paper that you particularly like because of how it's written, and use it as a template. This paper needn't be on the same topic, but a close mapping in terms of type of contribution (e.g. a tool paper vs. a theory paper) will give you more guidance as to how to structure your paper.
The title and abstract are the most important parts of a paper, and should clearly convey what you did. Motivate your specific problem (not the field as a whole), and focus on what you did. After reading the abstract, the reader should know what your contribution is – don't speak in generalities. For example, instead of saying "We analyze different methods for preparing cookies with interesting ingredients by running a user study.", say "We present three new recipes for chocolate chip cookies each employing a unique ingredient: jellybeans, tofu, and corn nibblets. Cookies were compared using a blind, within-subjects taste test with 30 individuals. The cookie with tofu was found to have superior mouth feel when compared with the other two, but subjects preferred the taste of the corn cookie by a 2:1 margin."
Review the Project Abstract assignment. Make sure you clearly address each of the important bullets from the abstract in your final paper.
Please use the APA heading structure to describe your study & results. Clearly tie your analysis to your hypotheses.
Use pictures to show your interface and graphs to present your data. Graphs should generally aggregate across participants, and show standard error bars. (Only show individual data points if the reader learns something more by doing so.) Figures should be captioned with what you believe the reader should infer from the figure.

Groups who do excellent projects will be encouraged to submit their research to UIST 2011's poster session. These submissions are due July 1st.

Tips on Running a User Study

If you have instructions, present them to participants in written form. You'll have a lot on your mind. Likely too much to remember to say everything you want. Written instructions also help insure that everything is consistent across participants.
Remember to instrument your software so that it logs all relevant user actions. How often did people ___?
Whatever you try first is almost guaranteed to have bugs so iteratively prototype your experiments just like you iteratively prototype interfaces. Test early and often and leave time for iteration in your schedule.