spring 2013

CS376: Research Topics in Human-Computer Interaction

Groups · Ideas · Project Abstract · Progress Meeting · Presentation · Paper

Project Overview

In this course, you will complete a quarter-long research project. This project will be completed in groups of two.

At a high level, successful projects will raise an important research question, and plan and execute a methodology for answering that question. Often, this methodology will include building and evaluating a prototype system, but hacking is not strictly necessary. All projects require a study — obviously a much more thorough study will be expected of projects that do not involve system building. The goal of the project abstract draft (described below) is to help you scope your work appropriately.

To get a sense of what a good scope for a project is, here are some examples of final papers. Because several of these projects went on to become a work-in-progress publications, their page length may differ from your final deliverable:

For information on how the project will be evaluated, see the grading page.

Forming Groups

This project will be completed in groups of two. E-mail cs376@cs.stanford.edu if you'd like a larger group. You will be subject to lengthy and brutal questioning. No teams with a single member. Project groups will be self-paired. Use Piazza to help find team members. When discussing a potential partnership with someone, you should discuss your background (e.g., programming proficiency or other skills you bring), availability (e.g., do you plan to primarily work evenings or mornings? weekdays or weekends?), and motivation level (ambition for a turing award? Or to just barely graduate?). It's important to be honest with your partner up front, and follow through on commitments you make.

At the end of class Monday April 8th, use the online submission system to submit the name(s) of who you will be working with. (All group members should complete a submission.)

For assistance in finding a group, go to the Piazza forum to post your ideas and communicate with others. Also, take a look at the opportunities for collaboration with individuals outside of the class.

Project Ideas

You are encouraged to generate your own project idea. There are two important keystones to making this work: 1) read a lot of papers in an area of interest, and 2) generate a lot of ideas. Here is a list of collaboration opportunities; we encourage you to take advantage of them as a way of participating in a long-term research endeavor. It will take some thinking to find a project where you can ask and answer a research question inside a 10-week quarter. You'll likely iterate on your idea several times -- take care not to fixate on one idea early on.

Project Abstract (Draft and Final Versions)

You must submit a draft of your project abstract. Course staff will provide feedback on the draft to assist in the preparation of a final version (see homepage for deadlines). Everything is submitted online.

The project abstract should cover the following topics:

Research Question: what are you trying to answer? State this as clearly as possible in one sentence.
Hypothesis: what do you think the answer to your question is, and why? State your hypothesis in terms that you will actually be able to deliver on within the space of a quarter. (For example, having a technology increase someone's income might be your ultimate goal, but you may not be able to measure a change in income in 10 weeks. In this case, increasing income could be part of your motivation, but not your hypothesis. Your hypothesis needs a more proximal measure.)
Related Work: Please describe several pieces of published research, how they inform your project, and where your work trancends this knowledge. (You need not have related work in the draft version.)
Theoretical contribution: What motivates you to explore this issue? What leads you to believe this is a problem/opportunity? And what are the theoretical contributions of your work to existing research in your area? What is the structure of the space of possibilities that your work explores? What are the major decisions from a design perspective and what are their relative merits? If it's helpful, you can use the design space in the Juxtapose paper (Table 1) as a guide.
Method: how will you explore your hypothesis, and why is that the right approach? (This should include the design of your study.) Grounding this in methodologies that other researchers have used (e.g. by drawing from the class readings) is a good idea.
- Study design: What are you going to do? Be very detailed and precise.
- Evaluation: How will you know you succeeded? What will you measure? How will you measure it?
- Ecological Validity: Why does your study answer your research question? Why does your evaluation address your hypothesis? Make sure your study, and the variables you're measuring, properly address the question you are asking.
Study Recruitment Plan: (final only) how will you get participants for your study? For pilot studies, we suggest you recruit from within the class -- "trading" participation with other groups is a great way to learn about what others are doing. For larger studies (e.g. for those not building a system), you need a clear recruitment plan.
Biggest Risk: what's the riskiest component of your project? (may not be able to get the hardware you need, robustly implementing the ___ algorithm may take too long, the difference between conditions may not be measurable, ...)

For a guide to the APA format, go to APA Style. Note that the information on the site is possibly too detailed for the abstract. If you want a good example of the detail expected for the final paper, look at Dynamic Speedometer: Dashboard Redesign to Discourage Drivers from Speeding, Manu Kumar and Taemie Kim.

For estimating number of users, see tutorial slides, or the handout from PSYCH 252 (posted here in a hurry---I haven't asked Prof Thomas for permission. Please don't redistribute.)

For the draft, we expect you to cover all topics in ~3 paragraphs--be concise but concrete in your descriptions. For the final version, you'll want to go into greater depth (approximately 2 paragraphs for each issue, with the exception of the research question, which should still be be one precise sentence).

We encourage you to iterate multiple times on this abstract. While there is only one formally defined point for receiving feedback from course staff, you should seek out more informal feedback as you work on this. E-mail us at any point if you'd like us to take a look at your current submission, or come to office hours if you'd like to discuss in person. You are free to change directions after submitting your draft, but the sooner you nail down a direction, the better your project is likely to be.

Progress Meeting

Course staff will meet individually with each project group to provide feedback on your progress. These meetings will happen in the last 30 minutes of our scheduled class time. Please use this google doc to sign up for a time slot. You should be prepared to present your working system, discuss your study plan, and have pilot results. Use the online submission system to submit any materials you'd like to discuss (e.g., prototypes, data, draft writing.) Come to the meeting prepared to show and tell with preliminary results and how you plan to course-correct based on your early experimentation. How will you revise your system/design/experiment/framing so that your project really pops? What will the title of your final paper be? In other words, how will you summarize your research contribution in just a few words? This exercise will helps you focus and sharpen your efforts on what will best address your research question. This focus will be especially important as time gets tight: some things will matter more than others.

Final Presentation

At the end of the quarter, you will present your research results to the class and outside guests. We have invited a couple HCI luminaries. Feel free to invite interested friends and colleagues!

Each group will have 4 minutes to present, plus 2 minutes for questions. This time limit will be strictly enforced – groups should set up during the question session of the group before them. To enable this, unplug the video cable from your laptop before answering questions.
Test (and debug) your laptop video projection before presentations begin. Time spent fiddling with display settings will count against your presentation time.
Structure your presentation like a pyramid — begin with a one-sentence statement of your research result. This will get everyone on the same page. Then, offer a short (e.g., 1 slide, 4 sentences) description of what you did and why you did it. Then, explain things in detail.
This presentation is short enough that you can write out everything you want to say long-hand. Do this! This will allow you to convey information efficiently and effectively. Read through it enough times so that you have it basically memorized, but not so memorized that you get flustered if you skip a word or someone asks a question.
Know your audience! You can expect that everyone in the class knows everything you learned in class. So, you don't need to re-introduce the whole field of HCI. A sentence or two to situate your work in the field is good, but spend the rest of the time telling us what you did.
When presenting, stand near your slides. And look at the audience.

Final Paper

In addition to the presentation, you will present your findings in a final paper.

Page limit: Final papers should be 3-5 pages long in the UIST format. While this may sound short, it is much harder to write an effective, complete short paper than it is to ramble. A good approach to writing a great short paper is to write a long one first, and then trim it down to the most vital parts. Appendices are acceptable and optional (they don't count towards the page limit), but won't be graded. Add one for materials you want an interested reader to see (for example, when we post your project on the website for next year), but don't need to be graded. Page limit includes references.

Much of the advice from above for preparing your presentation applies to the paper as well. Here are a few more suggestions for preparing your paper:

Find a paper that you particularly like because of how it's written, and use it as a template. This paper needn't be on the same topic, but a close mapping in terms of type of contribution (e.g. a tool paper vs. a theory paper) will give you more guidance as to how to structure your paper.
The title and abstract are the most important parts of a paper, and should clearly convey what you did. Motivate your specific problem (not the field as a whole), and focus on what you did. After reading the abstract, the reader should know what your contribution is – don't speak in generalities. For example, instead of saying "We analyze different methods for preparing cookies with interesting ingredients by running a user study.", say "We present three new recipes for chocolate chip cookies each employing a unique ingredient: jellybeans, tofu, and corn nibblets. Cookies were compared using a blind, within-subjects taste test with 30 individuals. The cookie with tofu was found to have superior mouth feel when compared with the other two, but subjects preferred the taste of the corn cookie by a 2:1 margin."
Review the Project Abstract assignment. Make sure you clearly address each of the important bullets from the abstract in your final paper.
Please use the APA heading structure to describe your study & results. Clearly tie your analysis to your hypotheses.
Use pictures to show your interface and graphs to present your data. Graphs should generally aggregate across participants, and show standard error bars. (Only show individual data points if the reader learns something more by doing so.) Figures should be captioned with what you believe the reader should infer from the figure (e.g. Participants rated tofu cookies to have 25% better mouth feel. Differences between jellybeans and corn nibblets were non-significant).

Groups who do excellent projects will be encouraged to submit their research to the UIST poster track. These submissions are due in early summer.

Tips on Running Effective Studies with People

If you have instructions, present them to participants in written form. You'll have a lot on your mind. Likely too much to remember to say everything you want. Written instructions also help insure that everything is consistent across participants.
Remember to instrument your software so that it logs all relevant user actions. How often did people ___?
Whatever you try first is almost guaranteed to have bugs so iteratively prototype your experiments just like you iteratively prototype interfaces. Test early and often and leave time for iteration in your schedule.
If you're asking someone to do a task, it helps to provide a good motivating scenario. Appendix A of this paper provides a pretty good example of a scenario for a design tool.