DepthTouch: Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface

DepthTouch is an exploratory interactive surface system that combines the benefits of multi-touch-sensitive surface with the ability to interact in the hover space in front of the surface. DepthTouch uses a depth-sensing camera, which reports a range value per pixel in addition to color, to track the 3D position of the user’s head and hand through a transparent vertical display screen (DNP HoloScreen). The camera location behind the screen minimizes situations in which one hand occludes the other and allows for tracking of the user’s interactions on and off the surface by segmentation of range data. We outline our system hardware, the tracking implementation, and 3D interactions enabled. We also discuss feedback from hundreds of preliminary users and implications for future research.