Looking At You: Fused Gyro and Face Tracking for Viewing Large Imagery on Mobile Devices

To appear at ACM SIGCHI 2012

Neel Joshi1                   Abhishek Kar1,2                   Michael Cohen1

1Microsoft Research       2Indian Institute of Technology, Kanpur


We present a touch-free interface for viewing large imagery on mobile devices. In particular, we focus on viewing paradigms for 360 degree panoramas, parallax image sequences, and long multi-perspective panoramas. We describe a sensor fusion methodology that combines face tracking using a frontfacing camera with gyroscope data to produce a robust signal that de?nes the viewer's 3D position relative to the display. The gyroscopic data provides both low-latency feedback and allows extrapolation of the face position beyond the the ?eldof-view of the front-facing camera. We also demonstrate a hybrid position and rate control that uses the viewer's 3D position to drive exploration of very large image spaces. We report on the ef?cacy of the hybrid control vs. position only control through a user study.

[Silverlight viewer | MP4]

Back to projects page