User-Defined Gestures for Surface Computing

Jacob O. Wobbrock The Information School DUB Group University of Washington

Merdith Ringel Morris, Andrew D. Wilson Microsoft Research Redmond, WA

Many surface computing prototypes have employed gestures created by system designers. Although such gestures are appropriate for early investigations, they are not necessarily reflective of user behavior. We present an approach to designing tabletop gestures that relies on eliciting gestures from non-technical users by first portraying the effect of a gesture, and then asking users to perform its cause. In all, 1080 gestures from 20 participants were logged, analyzed, and paired with think-aloud data for 27 commands performed with 1 and 2 hands. Our findings
indicate that users rarely care about the number of fingers they employ, that one hand is preferred to two, that desktop idioms strongly influence users’ mental models, and that some commands elicit little gestural agreement, suggesting the need for on-screen widgets. We also present a complete user-defined gesture set, quantitative agreement scores, implications for surface technology, and a taxonomy of surface gestures. Our results will help designers create better gesture sets informed by user behavior.

Wobbrock, J. O., Morris, M. R., and Wilson, A. D. 2009. User-defined gestures for surface computing. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 1083-1092.