Andy Wilson

Your Phone or Mine?: Fusing Body, Touch and Device Sensing for Multi-User Device-Display Interaction

Mahsan Rofouei1,2, Andrew D. Wilson1, A.J. Brush1
1Microsoft Research, 2UCLA

Determining who is interacting with a multi-user interactive touch display is challenging. We describe a technique for associating multi-touch interactions to individual users and their accelerometer-equipped mobile devices. Real-time device accelerometer data and depth camera-based body tracking are compared to associate each phone with a particular user, while body tracking and touch contacts positions are compared to associate a touch contact with a specific user. It is then possible to associate touch contacts with devices, allowing for more seamless device-display multi-user interactions. We detail the technique and present a user study to validate and demonstrate a content exchange application using this approach.

Mahsan Rofouei, Andrew Wilson, A.J. Brush, and Stewart Tansley. 2012. Your phone or mine?: fusing body, touch and device sensing for multi-user device-display interaction. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 1915-1918.