The Power of Freehand Interaction

This session of the 2005 Microsoft Research Faculty Summit describes two iCampus research projects that focus on using freehand input on Tablet PCs. We want people to be able to sketch, gesture, and talk about their ideas with a computer in the way that they do when interacting with each other.

• The first project gives people a kind of “magic paper” that understands the messy freehand sketches, casual gestures, and fragmentary utterances that are part and parcel of such interaction. Magic Paper will let computer tools capture and understand the kinds of ideas that are today captured in pencil on scraps of paper, introducing design capture into the earliest stages of the design process and radically shortening the design cycle.
• The Classroom Learning Partner (CLP) project is focused on improving student experience and learning in large classes. CLP uses as its framework Classroom Presenter, a Tablet PC-based presentation system that supports student wireless submission of digital ink answers to in-class exercises. CLP will allow this system to work in large classes by using AI techniques to interpret and aggregate student ink answers into a small number of equivalence classes, presenting summary information to the instructor and students.