Intelligent Analysis/Synthesis of Facial Imagery

Trevor Darrell (Interval Research ) and Chris Bregler (Interval Research and UC Berkeley)
trevor@interval.com

Seminar on People, Computers, and Design
Stanford University October 31, 1997

 

The intelligent processing of images of faces is a key resource in building interactive systems with natural and intuitive interfaces. In this presentation we describe two different systems for the analysis/synthesis of facial imagery, carried out by ourselves and colleagues at Interval Research.

The first system, the Magic Morphin Mirror, is an example of real-time, on-line vision processing to detect faces and drive an interactive graphics display. We use multi-modal integration of stereo, color, and pattern cues to robustly segment people and track their faces in crowded, open-background environments. Our first application was an interactive virtual mirror, demonstrated at SIGGRAPH '97, where a user's facial expression was distorted into a surreal effect in real-time.

The second system, Video Rewrite, is an off-line, photo-realistic facial animation system that uses existing footage to create automatically new video of a person mouthing words that she did not speak in the original footage. The system reorders the mouth images in the training footage to match the phoneme sequence of the new audio track. A new automatic morphing technique is used to combine these mouth gestures into the final video sequence. The resulting video combines the dynamics of the original actor's articulations with the mannerisms and setting dictated by the background footage.

These systems and their future extensions hold out the promise of creating fully synthetic characters from video examples, and animating them using real-time, untethered human performance.

This talk describes joint work with Malcolm Slaney, Michele Covell, Gaile Gordon, John Woodfill, Mike Harville and others at Interval Research.

 

Chris Bregler is a Ph.D student in Computer Science at UC Berkeley and a consultant at Interval Research Corp. His research centers around analysis and synthesis of human motion. Trevor Darrell recieved his Ph.D from MIT in 1996, and is presently a Member of Research Staff at Interval Research Corp. His research interests include computer vision, machine learning, and computer graphics.

 

Titles and abstracts for all years are available by year and by speaker.

For more information about HCI at Stanford see