Depth-Sensing Video Cameras for 3D Tangible Tabletop Interaction

Andrew D. Wilson
Microsoft Research

Recently developed depth-sensing video camera technologies provide precise per-pixel range data in addition to color video. Such cameras will find application in robotics and vision-based human computer interaction scenarios such as games and gesture input systems. We present an interactive tabletop system which uses a depth-sensing camera to build a height map of the objects on the table surface. This height map is used in a driving simulation game
that allows players to drive a virtual car over real objects placed on the table. Players can use folded bits of paper, for example, to lay out a course of ramps and other obstacles. A projector displays the position of the car on the surface, such that when the car is driven over a ramp, for example, it jumps appropriately. A second display shows a synthetic graphical view of the entire surface, or a traditional arcade view from behind the car. Micromotorcross is a fun initial investigation into the applicability of depth-sensing cameras to tabletop interfaces. We present details on
its implementation, and speculate on how this technology will enable new tabletop interactions.

Wilson, A.D. Depth sensing video cameras for 3D tangible tabletop interaction. In Proceedings of Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems, 2007. TABLETOP 2007. (10-12 Oct, 2007). 201-204.