Marc Davis · MIT Media Lab and Interval Research
Media Streams: Representing Video for Retrieval and Repurposing

What do you do when you have a thousand video clips? What tools do we need to build to support the reuse of large archives of video and audio information? In order to answer these questions, we need to design representations of video content. Although some aspects of video can be automatically parsed, a detailed representation requires that video be annotated. We will discuss the design issues in building representations of video content with special attention to the challenges of creating archives of repurposable video. Our prototype system, Media Streams, enables users to create multi-layered, iconic annotations of streams of video and audio data. Media Streams takes a departure from traditional 'clip-based' representations of video content inherited from single-use applications and utilizes a 'stream-based' representation in order to support the repurposing of annotated temporal media. In addressing the challenges of representing video information, we combine automatic and human-assisted annotation of media content, use multiple views of video data at different spatial and temporal granularities, have built a mixed representation similar to semantic and episodic memory structures, and have developed a visual language of cascading, compound, and animated icons for describing the complex structure of video and audio. We will also focus on the particular problems of creating a representation of action for video, as well as describing transitions in video.

Marc Davis is completing his Ph.D. from the MIT Media Laboratory while interning at Interval Research Corporation in Palo Alto. His thesis prototype, Media Streams, is a system for annotating, browsing, retrieving, and repurposing digital video. After receiving a B.A. in the College of Letters (an interdisciplinary program in history, literature, philosophy and language) from Wesleyan University in Connecticut, Marc Davis was awarded a two year research fellowship from the German Academic Exchange Service. In 1987, he received an M.A. in literary theory and philosophy from the University of Konstanz in Germany. In 1990, Marc Davis began his doctoral studies at the MIT Media Laboratory and co-founded (with Mike Travers) the Narrative Intelligence Reading Group.