Assignment 3: Project Introduction

Due: Wednesday, October 16 at 4:00pm. Submit on Canvas.

Summary

Now that you understand the literature and the bit flip that you're making, it's time to write up a project proposal in the form of an Introduction section for a paper. As researchers, we often write up a paper introduction like this as we embark on the project as a check that we can explain a clear vision of the project. If we can't explain the project clearly in writing, then we don't understand the project and its contribution well enough to move forward!

The result of this assignment will be an Introduction section that you can eventually adapt for your final paper. Think of this Introduction as also serving the role of a project plan for the quarter.

Outline your introduction

While there are many approaches to write an introduction, in Computer Science most of these approaches need to explain two things: what is the problem, and how are you solving it?

In my papers, generally I follow the following paragraph-by-paragraph outline for an Introduction:

  • Problem: what's the problem you're solving? Cite other work that helps motivate the problem. Make the reader feel the urgency of the problem, so that by the end of the paragraph they care a lot that someone finds a solution.
    • Example topic sentence: We all want to build IoT applications that are responsive to our behaviors — for example, coffee machines that make coffee for us after we wake up, or phones that know not to ring while we're in meetings — but we lack the data to endow our algorithms with this information about our patterns and behaviors.
  • Set up the bit: This paragraph usually answers the question, "Why isn't this problem solved yet?" by setting up the bit that you're going to flip. The topic sentence typically summarizes existing approaches to solving the problem by articulating the bit that they share, and points out why those approaches and the bit more broadly hasn't been able to solve the problem. So, it's a brief summary of the most important related work — but it's a summary, not a survey, meaning that its goal is to set up the bit, not to cover everything.
    • Example topic sentence: Existing solutions to this problem require manually authored rules (e.g., coffee is a morning drink, we drink morning drinks after we wake up), but the required manual effort results in incomplete data about human situations and behaviors.
  • Flip the bit: Here you introduce your insight by flipping the bit. The topic sentence of this paragraph is the thesis statement for the entire paper. It should lay out the big idea that you'll be pursuing. Spend this paragraph explaining your insight clearly, why it flips the bit that you set up, and what the implications of flipping that bit will be for the problem at hand. For the purposes of this assignment, it's fine to assume that your bit flip will work out — we all know that research is iterative and projects will evolve.
    • Example topic sentence: In this paper, we suggest that data mining a large dataset of modern fiction offers a promising alternative to manual annotation.
  • Instantiate that bit flip in a solution: At this point, the reader understands the idea that you're proposing, but it's still very high level. In this paragraph, map that idea onto a concrete instantiation. This is where you introduce the system you've built, the model you're creating, the study design, or whatever is the main artifact of your research, and how it instantiates that insight.
    • Example topic sentence: We introduce Augur, a knowledge base that relates human activities and the objects around them by mining 1.8 billion words of modern fiction from the online writing community Wattpad into a vector space model for predicting human activities.
  • Evaluate the solution: We haven't discussed evaluation yet in the class, but a good introduction summarizes the evaluation that you performed to demonstrate that flipping the bit has the effects that you suspect. For the purposes of this assignment, ask yourself: how would you prove to a critical reader that flipping the bit has the effect you promised in solving the problem? We expect you to articulate what you will be manipulating and measuring, but we don't require any actual results right now and we won't hold you to this evaluation strategy — especially since sometimes your evaluation strategy evolves as your project evolves.
    • Example topic sentence: To evaluate Augur's ability to predict human behaviors, we first compared Augur's predictions to human estimates of behaviors given the same inputs, and second performed a field deployment to test Augur's precision and recall in a realistic application.
  • Implications: If you're right about this bit flip being the right way for the field to go, what implications will there be for the field? Don't overplay your hand here by stating that all of computing will shift, but do be clear and forceful in pushing the case as hard as you feel that the case warrants.
    • Example topic sentence: This work demonstrates that fiction can provide deep insight into the minutiae of human behavior, and that modern NLP algorithms can structure those minutiae into useful hooks for IoT applications.

This outline will depend a bit on the kind of paper you're writing. For example, start by answering: are you an old problem / new solution paper, or a new problem / old solution paper? Old problem / new solution papers already have a warrant for the problem, so that part of the Introduction is easy, but they need to argue for the solution very carefully since there's no warrant there. Similarly, new problem / old solution papers need to be extra careful to argue the problem persuasively, but they can draw on existing warrants for the solution and only argue for how they're adapting it to the problem.

Start by creating the outline for your Introduction, using only the topic sentences that you will eventually expand into full paragraphs. This is harder than it sounds! It means that you only have those six sentences to tell the entire story of your paper. So, you can't add any unnecessary detail or extra concepts, or those six sentences won't be possible for a reader to understand. Just like a pro swimmer makes no unnecessary motions as they focus all energy on executing a stroke, and just like a pro chess player makes no unnecessary moves as they focus their actions on enacting their strategy, your topic-sentence outline should introduce no unnecessary concepts or ideas as it focuses on explaining the project.

You may want to get feedback from your TA on your outline before moving forward. We recommend office hours. If you find that you need to stray from the outline above for your project, definitely reach out to the TA first with your proposed deviation.

From outline to Introduction

Now, to write the Introduction section itself! Each topic sentence becomes the anchor of a paragraph in your Introduction, and the rest of each paragraph should set out to prove the point made by that topic sentence. If you need to split a paragraph into multiple paragraphs to be clearer, do so — but keep the rough proportions of those original six paragraphs. (In other words, don't spend three paragraphs on the problem, then rush through everything else.)

Don't forget to cite any work you're referencing that motivates your problem, sets up the bit, or helps explain why the flip is a good idea.

Here are example introduction sections for each section, selected by that section's TA. Bear in mind that these papers will stray from the outline format above—which is fine, experienced writers play with the format—but we still feel that they represent strong examples of Introduction sections.

Your Introduction should be 700-900 words. Include a References section with citations at the bottom; references are not included in the word count.

Timeline

Executing this project will be your team's focus for the rest of the course. Here is a rough sense of the timeline from here on out:

  • Week 4: Introduction due; start executing the project
  • Weeks 5-8: work on the project, with weekly check-in assignments
  • Week 8: Evaluation plan due in addition to the weekly check-in assignment
  • Week 9: Draft paper due
  • Week 10: Final presentation in class
  • Finals: Final paper due

AI Section: Specific Tools

After this assignment, you will start working on your projects. We strongly recommend you start learning the tools necessary for your projects. These are optional, but highly recommended.

  • For all groups: Go through the wandb pytorch tutorial: https://colab.research.google.com/drive/1zkoPdBZWUMsTpvA35ShVNAP0QcRsPUjf
  • Image generation: Reproduce the tasks from the generation paper and run inception scores
  • Training DNNs for high performance inference: Understand PyTorch-Lightening
  • Stochasticity: Attack a model with the 10 different attacks from the advex_uar codebase
  • Verification: Reproduce the results of 1-3 verification papers

Submission

Submit a PDF with (1) your outline, and (2) your Introduction on Canvas. This is a group assignment; create a group for your team, and one member should submit on behalf of the group.

Grading

Your introduction will be graded on the following rubric:

  • Outline: does your outline read as a clear and complete explanation of the project? (5pt)
  • Problem and bit: does the set up of the project make sense and prepare the reader for the bit that will later be flipped? (5pt)
  • Bit flip and description: does the bit flip follow through on the bit that was set up, and does the system description make clear how that bit flip gets instantiated in a system? (5pt)
  • Clarity: is the writing overall clear and easy to follow for a technical expert in the field? (5pt)