Ultimate statistics
The big picture: To find the most efficient way to input the most relevant event data (statistics) of an ultimate game real-time at the side-line.
In this project we test the initial assumptions/designs for pen/voice input and gather some material for the next prototype rounds (field-view preference, subjective accuracy in positioning the throwers, subjective accuracy while multi-tasking, response/reaction times from event to drawing/speaking, and initial vocabulary for speech input)
General comment: ultimate is probably more suitable for this kind of data gathering than for example basketball, football, soccer, and ice hockey. This is because in ultimate players are not allowed to run with the disc. This allows the stat keeper to just point separate locations of the screen (thrower locations) and he does not need to draw dribble/run patterns of the person with the ball. Also the voice input is easier to synchronize with these discrete locations on the field.
There are different uses for the statistics

    * To help coach on field to make game-time decisions (change of strategy, hot players, players needing rest)
    * To help coach to prepare for the next opponent (scouting)
    * To analyze the game to find out why some teams/players are good
    * To provide background for calculating ratings for players (although usually the ultimate culture dislikes the idea of pointing out people from a team) 


Hypothesis
People prefer bird-eye field of he field to perspective view.
People can position players on the field pretty accurately and the accuracy does not degrade much even if they are speaking out loud throw-by-throw commentary (the locations are compared to the positions from the other test subject, because the field setting did not allow to position the game tracking camera high enough to allow accurate positioning later from the video stream.
People can position the thrower and draw the marker location for every throw, even while speaking.
The time between drawing the thrower location and saying out loud the catcher name in pretty constant, so synchronizing the drawing and the speech is straightforward (also this would allow to distinguish the high stall-count situations)

Evaluation plan
A test with two test subjects was carried out last weekend during a scrimmage tournament at Stanford. The setup was: two video cameras, a tripod, two binders with various field templates, two pens, and two test subjects having knowledge of the game and also knowing the names of most of the players in one team currently playing.
One video camera standing on a tripod was placed at the one end of the playing field allowing to capture the events on the field without operator.
The other video camera was also running all the time during the test but operated manually by the tester to capture the actions of the test subjects and also capturing the audio (part of the test required the test subject to speak out loud the events on the field while drawing the trowing locations).
The both videos have been transferred to a PC, they will be later synchronized to allow to study the response times of the test subjects.
A number of different tests were carried out; the field templates required for each of the sub tests were divided from others using interleaves. One template for one possession (to reduce the possible clutter on the paper).
Tests:

    * Pesrpective field vs. Bird-eye field (continue the rest of the tests with Bird-eye field) marking the throwers as crosses
    * Base material for positioning accuracy, positioning the throwers only (crosses) - both
    * Position the throwers (crosses) and markers (lines from shoulder to shoulder) - both
    * Position the throwers and markers and speak out throw-by-throw commentary (player, throw style, etc) -first the other and then the other 


Current prototype
The prototype is paper prototype. Field templates are printed on paper, a stack of these templates are put into a binder (with hard covers, allowing the test subjects to draw on the papers even without a table).
No feedback (Wizard of Oz -like feedback) is provided to the users.

Further development
Already the short interview and test subjects' comments changed some of the assumptions and required to re-evaluate some input ideas.
To do:

    * Analyze test data
    * Refine hypothesis, draw conclusions for further (after this quarter) development (target is TabletPC prototype during 1H/2005)
    * Write the final report 


Related research
There are three areas of related research

    * multimodal (pen and voice) interaction
    * data gathering methods for team sport statistics
    * data gathering methods for ultimate specifically 

Multimodal interaction
Oviatt's paper (discussed during this course) provides a lot of background material for this.
QuickSet is one of the relevant projects, as here voice is used to add properties to the locations pointed by the user.
To my knowledge there has not been studies where pen/voice interfaces have been used to record real time events.

Team sports in general
Many vendors for statistics software, each of them tailored for a specific sport.
Only one application for basketball uses voice input.
A company, which provides professional statistical services for 26 different sports (STATS Inc.) uses real time statistician only for baseball, all the other sports are analyzed from a videotape and using keyboard input.
Video analysis of player movements is possible, but it requires the video camera to be high above the ground (not always possible) and the raw data requires manual smoothing (hypothesis). The same need for smoothing is true for using location sensors for each of the players, but this might be impossible also because the players might not accept wearing a device (however small it would be). Videoanalysis and sensor usage could be excellent for physiological analysis of the game (to measure the distance run, combine that with heart rate, etc.).

Ultimate statistics
RUFUS was a paper and pen based statistics system developed in late 80s. It required one person to speak out loud the events on the field while the other person was writing down the events using a shorthand writing. The data was later entered to a computer, and some Perl scripts were available for creating summaries.
UltiStats is a Palm OS application for ultimate statistics gathering. The event data is inputted to the application using various forms (on-screen buttons and like) using the stylus. As the screen is small and pointing the buttons requires accuracy (one has to remember the real-time requirements) it helps having another person speaking out loud the events on the field. The system also provides some summary screens of the data. the application is Open Source and available at sourceforge.net.
There are also a couple of electronic score keeping systems for ultimate. One uses mobile phones and touch tone input of scores through a central server (http://www.scoreomatic.com) and a J2ME

Hartti