I am a Post Doc Researcher in the Interactive 3D Technologies (I3D) group at Microsoft Research. My current research focuses on novel input and output technologies for "Freeform 3D Interactions in Everyday Environments".
Natural User Interfaces (NUI) have received a great deal of attention in recent years. Much of this attention is around the Kinect and gaming in the living room. My research looks at ways of taking NUI out of the living room, into our everyday lives. Through the design, prototyping and evaluation of novel NUI systems, I am exploring how NUI can become truly ubiquitous, allowing such interactions to be incorporated into new user experiences and scenarios, in particular in rethinking traditional mobile and desktop interactions.
A secondary theme within my research is exploring the convergence of NUI with Augmented Reality (AR). Over the years we’ve seen AR technologies such as glasses, tracking and mapping, and mobile computing maturing. A less researched topic is how to make such AR technologies interactive. I’m interested in fusing together ideas from NUI with AR. These areas can be seen as very complimentary; NUI is focused on sensing new forms of input, and AR traditionally focuses on new forms of output. Bringing these two fields together can lead to new user experiences that radically change the way we interact and even think about computing systems.
Selected Research Projects
- T. Sharp, C. Keskin, D. Robertson, J. Taylor, J. Shotton, D. Kim, C. Rhemann, I. Leichter, A. Vinnikov, Y. Wei, D. Freedman, P. Kohli, E. Krupka, A. Fitzgibbon, S. Izadi. Accurate, Robust, and Flexible Real-time Hand Tracking. ACM CHI, Seoul, South Korea, April 2015 (to appear).
- C. Rendl, D. Kim, S. Fanello, P. Parzer, C. Rhemann, J. Taylor, M. Zirkl, G. Scheipl, M. Haller, S. Izadi. FlexSense: A Transparent Self-Sensing Deformable Surface. ACM UIST, Honolulu, HI, USA, Oct 2014
- S. Fanello, C. Keskin, S. Izadi, P. Kohli, D. Kim, D. Sweeney, A. Criminisi, J. Shotton, S.B. Kang, T. Paek. Learning to be a depth camera for close-range human capture and interaction. ACM SIGGRAPH and Transation On Graphics, Vancouver, BC, Canada, Aug 2014
- P. Dietz, G. Reyes, D. Kim. The PumpSpark Fountain Development Kit. ACM DIS, Vancouver, BC, Canada, June 2014
- D. Kim, S. Izadi, J. Dostal, C. Rhemann, C. Keskin, C. Zach, J. Shotton, T. Large, S. Bathiche, M. Nießner, A. Butler, S. Fanello, V. Pradeep. RetroDepth: 3D Silhouette Sensing for High-Precision Input On and Above Physical Surfaces. ACM CHI, Toronto, ON, Canada, April 2014
(Best Paper Award)
- C. Weichel, M. Lau, D. Kim, N. Villar, H. Gellersen. MixFab: A Mixed-Reality Environment for Personal Fabrication. ACM CHI, Toronto, ON, Canada, April 2014
(Best Paper Award)
- D. Kim, O. Hilliges, S. Izadi, A. Butler, J. Chen, I. Oikonomidis, P. Olivier. Digits: Freehand 3D Interactions Anywhere Using a Wrist-Worn Gloveless Sensor. ACM UIST, Cambridge, MA, USA, Oct 2012
(Best Demo Award 2nd place)
- D. Molyneaux, S. Izadi, D. Kim, O. Hilliges, S. Hodges, X. Cao, A. Butler, and H. Gellersen. Interactive Environment-Aware Handheld Projectors for Pervasive Computing Spaces. PERVASIVE, Springer, Newcastle upon Tyne, UK, Jun 2012
(Best Paper Award)
- O. Hilliges, D. Kim, S. Izadi, M. Weiss, A. Wilson. HoloDesk: Direct 3D Interactions with a Situated See-Through Display. ACM CHI, Austin, TX, USA, May 2012
- S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, and A. Fitzgibbon. KinectFusion: Real-Time 3D Reconstruction and Interaction using a Moving Depth Camera. ACM UIST, Santa Barbara, CA, USA, Oct 2011
- R. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. Davison, P. Kohli, J. Shotton, S. Hodges, A. Fitzgibbon. Real-Time Dense Surface Mapping and Tracking with Kinect. IEEE ISMAR. Basel, Switzerland, Oct 2011
(Best Paper Award)
- A. Butler, O. Hilliges, S. Izadi, S. Hodges, D. Molyneaux, D. Kim, D. Kong. Vermeer: Direct Interaction with a 360° Viewable 3D Display. ACM UIST, Santa Barbara, CA, USA, Oct 2011
- A. Butler, S. Izadi, O. Hilliges, D. Molyneaux, S. Hodges, D. Kim. Shake'n'Sense: Reducing Interference for Overlapping Structured Light Depth Cameras. ACM CHI, Austin, TX, USA, May 2012
- D. Kim, P. Dunphy, P. Briggs, J. Hook, J. Nicholson, J. Nicholson, P. Olivier. Multi-Touch Authentication on Tabletops. ACM CHI, Atlanta, Georgia, USA, Apr 2010
- O. Hilliges, D. Kim, S. Izadi. Creating Malleable Interactive Surfaces using Liquid Displacement Sensing. IEEE Tabletop and Interactive Surfaces, Amsterdam, Netherlands, Oct 2008
- O. Hilliges, L. Terrenghi, S. Boring, D. Kim, H. Richter, A. Butz. Designing for Collaborative Creative Problem Solving. ACM Creativity & Cognition, Washington D.C., USA, Jun 2007
- Moving object segmentation using depth images (US 8401225 B2). Richard Newcombe, Shahram Izadi, Otmar Hilliges, David Kim, David Molyneaux, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler.
- Real-time camera tracking using depth maps (US8401242 B2). Richard Newcombe, Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler.
- Authentication by multi-level pressure exertion on multi-touch tabletop interfaces. Paul Dunphy, David Kim, James Nicholson, John Nicholson, Jonathan Hook Pamela Briggs, Patrick Olivier
- Gesture Recognition Techniques (US 20120306734 A1). David Kim, Otmar D Hilliges, Shahram Izadi, Patrick L Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G Molyneaux, Stephen E Hodges, Andrew W Fitzgibbon.
- Mobile Camera Localization Using Depth Maps (US 20120194644 A1). Richard Newcombe, Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler.
- Using a three-dimensional environment model in gameplay (US 8570320 B2). Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler.
- Three-dimensional environment reconstruction (US 8587583 B2). Richard Newcombe, Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Stephen Edward Hodges, David Alexander Butler, Andrew Fitzgibbon, Pushmeet Kohli.
- Human Body Pose Estimation (US 20110210915 A1). Jamie Daniel Joseph Shotton, Shahram Izadi, Otmar Hilliges, David Kim, David Geoffrey Molyneaux, Matthew Darius Cook, Pushmeet Kohli, Antonio Criminisi, Ross Brook Girshick, Andrew William Fitzgibbon.
Other Co-authored Patent Applications
- User interaction in augmented reality. 04 Nov 2010
- Augmented reality with direct user interaction. 04 Nov 2010
- Reducing interference between multiple infra-red depth cameras. 31 Jan 2011
- Using a three-dimensional environment model in gameplay. 31 Jan 2011
- Generating computer models of 3d objects. 6 Jun 2011.
- Wearable sensor for tracking articulated body-parts. 04 Oct 2012
- Grasping virtual objects in augmented reality. 17 Oct 2012
- Learning Image Processing Tasks from Scene Reconstructions. 15 Dec 2011
- Distributed asynchronous localization and mapping for augmented reality. 2 Jun 2011
Reviewer: CHI 2011, CHI 2012, CHI 2013, CHI 2014, CHI 2015, UIST 2011, UIST 2012, UIST 2013, UIST 2014, ITS 2012, ITS 2013, UbiComp 2012, DIS 2012, TEI 2013, IUI 2013, IEEE Pervasive Computing (2014), IEEE Computer Graphics and Applications (2013), International Journal of Human-Computer Studies (2014), The Visual Computer (2014)
Chair, First Workshop on Kinect in Pervasive Computing at PERVASIVE 2012
- Program Committee Member, ACM CHI 2014
- Program Committee Member, ACM CHI 2015
- Program Committee Member, ACM ITS 2015
- Student Innovation Contest Co-chair at UIST 2013
- Student Innovation Contest Co-chair at UIST 2014
- Poster Co-chair ACM ITS 2015