Hrvoje Benko · Microsoft Research
Beyond Flat Displays: Interactivity on Any Surface

Much of human-computer interaction today is confined to flat, rectangular displays. This has never been more apparent than with the mainstream adoption of touchscreens, where the same flat surface facilitates both the output and the input capabilities of the device. Recent advances in sensing and display technologies enable new modes of interaction that break the traditional boundaries of the screen, potentially enveloping the user's physical environment with interactive content. In this talk, I argue that high-fidelity interactivity, through touch and gestures, can be facilitated on every surface and between surfaces. To illustrate my claims I present a series of research projects which demonstrate how depth-sensing cameras can be used to create augmented reality experiences far richer than previously imagined. I discuss how such experiences can be authored and created, and how to achieve high-fidelity interactions with virtual content without requiring the user to wear any additional gear. I draw from experiences in designing several highly publicized projects such as LightSpace, OmniTouch, MirageTable, and IllumiRoom to illustrate the concepts.

Hrvoje Benko is a researcher at Microsoft Research. He explores novel interactive computing technologies and their impact on human-computer interaction. In particular, his research interests include augmented reality, computational illumination, surface computing, new input form factors and devices, as well as touch and freehand gestural input. He is fascinated by the intersection point where the digital technology world meets the curved, physical, 3D space we live in. He is the author of more than 30 scientific papers and journal articles. For his publications he received the best paper awards at both ACM UIST and ACM SIGCHI. His work has been featured in the mainstream media and on popular technology blogs. In 2010, he worked with Microsoft Hardware to turn one of his research projects into a product called Microsoft Touch Mouse. He received his Ph.D. in Computer Science from Columbia University in 2007 with prof. Steven Feiner. More detail can be found on his website: http://research.microsoft.com/~benko/.