 |
I'm a post doc researcher in the Sensors and Devices Group, part of the Computer Mediated Living Group at Microsoft Research in Cambridge, UK.
Email: otmarh [at] microsoft.com |
Research Interests
My research focuses on the intersection of input sensing technologies, display technologies, computer graphics, and human computer interaction. My main objective is to push the boundaries of what humans can do with computers and how they interact with them. Touch enabled phones and tablets have created novel user experiences and have now entered the mainstream market. Motion sensing cameras such as Kinect promise to have a similarly profound effect on how we use computers. In my research I aim to combine emerging input and output technologies in interesting and novel ways to create and study the next generation of non-traditional user interface technologies.
In particular I’m interested in multi-touch, physical and tactile input, 3D and augmented reality (AR) technologies and applications. I have worked on novel, direct-touch volumetric displays, room-based sensing for user interaction from multiple depth-sensing cameras, dense 3D surface reconstruction from a single moving depth camera for AR applications, and situated AR setups allowing for free-form dexterous interactions with spatially registered graphics. My approach covers low-level input sensing technologies, algorithms to process and interpret this data, through representations of this sensor data within virtual worlds, to displaying graphics, and finally understanding the implications for user interaction.
Latest Projects
| HoloDesk: Direct 3D Interactions with a Situated See-Through Display: | |
|
O. Hilliges, D. Kim, S. Izadi, M. Weiss To appear in Proceedings of Human Factors in Computing Systems (ACM CHI) Austin, Tx, USA, Apr 2012 |
|
| KinectFusion: | |
|
Shahram Izadi, Richard Newcombe, David Kim, Otmar Hilliges, David Molyneaux, Steve Hodges, Pushmeet Kohli, Jamie Shotton, Andrew Davison, Andrew Fitzgibbon “KinectFusion: Real-Time Dynamic 3D Surface Reconstruction and Interaction” ACM SIGGRAPH 2011 Talk “KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera” In Proceedings ACM User Interface and Software Technologies (ACM UIST) Santa Barbara, CA, USA, Oct 2011
|
 |
| Vermeer: | |
|
A. Butler, O. Hilliges, S. Izadi, D. Molyneaux, S. Hodges “Vermeer: Direct Interaction with a 360° Viewable 3D Display” In Proceedings ACM User Interface and Software Technologies (ACM UIST) Santa Barbara, CA, USA, Oct 2011 |
|
Selected Publications
| Full-length conference papers | ||||
| O. Hilliges, D. Kim, S. Izadi, M. Weiss
"HoloDesk: Direct 3D Interactions with a Situated See-Through Display" To appear in Proceedings of Human Factors in Computing Systems (ACM CHI) Austin, Tx, USA, Apr 2012
| ||||
|
S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, A. Fitzgibbon “KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera” To appear in Proceedings ACM User Interface and Software Technologies (ACM UIST) Santa Barbara, CA, USA, Oct 2011 | ||||
|
A. Butler, O. Hilliges, S. Izadi, D. Molyneaux, S. Hodges “Vermeer: Direct Interaction with a 360° Viewable 3D Display” To appear in Proceedings ACM User Interface and Software Technologies (ACM UIST) Santa Barbara, CA, USA, Oct 2011 | ||||
|
R. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. Davison, P. Kohli, J. Shotton, S. Hodges, A. Fitzgibbon “Real-Time Dense Surface Mapping and Tracking with Kinect” To appear in Proceedings IEEE Augmented and Mixed Reality (IEEE ISMAR) Basel, Switzerland, Oct 2011 | ||||
|
D. Kirk, S. Izadi, A. Sellen, S. Taylor, R. Banks, O. Hilliges “Opening up the Family Archive” In Proceedings ACM Computer Supported Collaborative Work (ACM CSCW). Savannah, Georgia, USA, February 2010 (Best Paper Award) | ||||
|
O. Hilliges, S. Izadi, A. D. Wilson, S. Hodges, A. Garcia-Mendoza, A. Butz “Interactions in the Air: Adding Further Depth to Interactive Tabletops” In Proceedings of ACM User Interface and Software Technologies (ACM UIST) Victoria, Canada, Oct 2009 | ||||
|
A. D. Wilson, S. Izadi, O. Hilliges, A. Garcia-Mendoza, D. Kirk Monterey, CA, USA, Oct 2008 (Best Paper Award) | ||||
|
M. Hancock, O. Hilliges, C. Collins, D. Baur, S. Carpendale “Exploring Tangible and Direct Touch Interfaces for Manipulating 2D and 3D Information on a Digital Table” In Proceedings of Interactive Surfaces and Tabletops (ACM ITS) Banff, Alberta, Canada, Oct 2009 | ||||
|
O. Hilliges, D. Baur, A. Butz “Photohelix: Browsing, Sorting and Sharing Digital Photo Collections” In Proceedings of the IEEE Tabletop and Interactive Surfaces Newport, RI, USA, Oct 2007 | ||||
|
O. Hilliges, L. Terrenghi, S. Boring, D. Kim, H. Richter, A. Butz “Designing for Collaborative Creative Problem Solving” In Proceedings of ACM Creativity & Cognition Washington D.C., USA, June 2007 | ||||
|
Short conference papers | ||||
| ||||
|
Journal articles | ||||
|
L. Terrenghi, O. Hilliges, A. Butz: “Kitchen Stories: Sharing Recipes with the Living Cookbook”
| ||||
|
Book chapters | ||||
|
O.Hilliges, A.Butz, S.Izadi, A. Wilson “Interaction on the Tabletop: Bringing the Physical to the Digital” In Tabletops - Horizontal Interactive Displays, Springer 2010, C. Müller-Tomfelde (Ed.)
|
Professional Activities
|
Demonstrations Co-Chair, ACM Symposium on User Interface Software and Technology (UIST '10 & '11) |
| Program Committee Member, ACM Conference on Interactive Tabletops and Surfaces (ACM ITS '10) |
|
Student Volunteer Chair, ACM Conference on Interactive Tabletops and Surfaces (ACM ITS '08 & '09) |
|
Program Committee Member, ACM Conference on Tangible and Embedded Interaction (ACM TEI '09) |
Previous Work
Before I joined Microsoft Research Cambridge I was a member of the Media Informatics Group, University of Munich, Germany. I received my PhD in computer science in summer 2009. In my dissertation, “Bringing the Physical to the Digital – A New Model for Tabletop Interaction” I explored several interaction techniques that allow for richer human computer interaction more akin to the flexible and continuous ways in which we manipulate objects in the real world. In the course of this work I developed a model for more natural interactions with 3D data on interactive surfaces. The model enables various ways to map rich input from vision-based interactive surfaces within virtual worlds where object behaviour is controlled by a sophisticated physics simulation. The model enables a variety of open-ended, non-scripted interactions allowing users to leverage their fine-grained manipulation skills in ways similar to the real world.
However, because the interaction – the sensed input and the displayed output – is still bound to the surface, there is a fundamental limitation in manipulating objects using the third dimension. To address this issue, I proposed a novel approach to creating a continuous interaction space – allowing for rich, natural interaction on the surface and above the surface – conceptually allowing users to pick objects off the surface and manipulate their position in 3D as if it were a real object. We presented several ways to enable “in-the-air” interactions that allow users to interact with 3D data (rendered on a 2D display) in a as-direct-as possible fashion without the need for user instrumentation. This approach utilizes several interesting hardware and software components such as a switchable diffuser, infrared illumination, IR-based depth estimation, holographic projection screens and a depth-sensing camera.
Additonal Publications
| Full-length conference papers |
|
L. Terrenghi, D. Kirk, H. Richter, S. Krämer, O. Hilliges, A. Butz “Physical Handles at the Interactive Surface: Exploring Tangibility and its Benefits.” Conference on Advanced Visual Interfaces, (AVI). Napoli, Italy, May 2008
|
|
O. Hilliges, M. Wagner, L. Terrenghi, A. Butz “The Living-Room: Browsing, Organizing and Presenting Digital Image Collections in Interactive Environments.” Conference on Intelligent Environments. Ulm, Germany, Sept 2007
|
|
S. Boring, M. Altendorfer, G. Broll, O. Hilliges, A. Butz “Shoot & Copy: Phonecam-Based Information Transfer from Public Displays onto Mobile Phones.” Conference on Mobile Technology, Applications and Systems (Mobility). Singapore, Sept 2007
|
|
O. Hilliges, P. Kunath, A. Pryakhin, A. Butz, H.P. Kriegel “Browsing and Sorting Digital Pictures Using Automatic Image Classification and Quality Analysis.” Conference on Human-Computer Interaction (HCI International). Beijing, P.R. China, July 2007
|
|
S. Boring, O. Hilliges, A. Butz “A Wall-Sized Focus Plus Context Display.” IEEE Conference on Pervasive Computing and Communications (PerCom). New York, NY, USA, Mar 2007
|
|
O. Hilliges, P.Holzer, R.Klüber, A. Butz “Audio Radar: A Metaphorical Visualization for the Navigation of Large Music Collections.” International Symposium on Smart Graphics. Vancouver, Canada, July 2006
|
|
Doctoral Symposium |
|
O. Hilliges “Informed Browsing: Scaling Up Co-Experienced Access to Digital Media.” Doctoral Symposium of ACM Symposium on User Interface Software and Technology (ACM UIST). Newport, RI, USA, Oct 2007
|
|
Short conference papers |
|
D. Baur, O. Hilliges, A. Butz “Flux: Enhancing Photo Organization through Interaction and Automation.” International Symposium on Smart Graphics. Rennes, France, Aug 2008
|
|
O. Hilliges, C. Sandor, G. Klinker: “Interaction Management for Ubiquitous Augmented Reality User Interfaces.” International Symposium on Intelligent User Interfaces (IUI). Sydney, Australia, Jan 2006 |
|
Workshop papers |
|
O. Hilliges “Finding the Unknown – Serendipitous Discovery in Co-Located Consumption of Digital Photo Collections.” Collocated Social Practices Surrounding Photos in conjunction with the ACM CHI. Florence, Italy, Apr 2008 |
|
O. Hilliges, L. Terrenghi “Overcoming Mode-Changes on Multi-User Large Displays with Bi-Manual Interaction.” International Workshop on Multi-user and Ubiquitous User Interfaces (IUI). Sydney, Australia, Jan 2006
|
|
L. Terrenghi, O. Hilliges “Design for Collaboration on a Tabletop Display in the Home.” International Workshop on Multi-user and Ubiquitous User Interfaces (IUI). Sydney, Australia, Jan 2006
|
|
O. Hilliges, C. Sandor, G. Klinker “A Lightweight Approach for Experimenting with Tangible Interaction Metaphors.” International Workshop on Multi-user and Ubiquitous User Interfaces (IUI). Funchal, Madeira, Jan 2004
|
|
M. Bauer, O. Hilliges, A. MacWilliams, C. Sandor, M. Wagner, G. Klinker, J. Newman, G. Reitmayr, T. Pintaric, T. Fahmy, D. Schmalstieg “Integrating Studierstube and DWARF.” International Workshop on Software Technology for Augmented Reality Systems (ISMAR). Tokyo, Japan, Oct 2003
|
