Microsoft Research

Interactive Machine Learning

A large body of human-computer interaction research has focused on developing metaphors and tools that allow users to effectively issue commands and directly manipulate informational objects. However, with the advancement of computational techniques such as machine learning, we now have the unprecedented ability to embed 'smarts' that allow machines to assist users in completing their tasks. We believe that trying to fully automate tasks is extremely difficult and even undesirable, but instead there exists a computational design methodology which allows us to gracefully combine automated services with direct user manipulation.

 

 

Project Team

CueTIP: Mixed-Initiative Handwriting Recognition

Handwritten input is inherently ambiguous, and recognition systems will always make errors. Unfortunately, work on error recovery mechanisms has mainly focused on interface innovations that allow users to manually transform the erroneous recognition result into the intended one. In our work, we propose a mixed-initiative approach to error correction. CueTIP is a novel correction interface that takes advantage of the recognizer to continually evolve its results using the additional information from user corrections. This significantly reduces the number of actions required to reach the intended result.

CueFlik: Interactive Concept Learning in Image Search

Popular image search engines have begun to provide tags based on simple characteristics of images (such as tags for black and white images or images that contain a face), but such approaches are limited by the fact that it is unclear what tags end users want to be able to use in examining image search results. CueFlik is an image search application that allows end users to quickly create (and reuse) their own rules for re-ranking images based on their visual characteristics.

 

Publications

CueFlik: Interactive Concept Learning in Image Search

James Fogarty, Desney S Tan, Ashish Kapoor, Simon Winder

CHI 2008 Conference on Human Factors in Computing Systems

CueTIP: A mixed-Initiative Interface for Correcting Handwriting Errors

Michael Shilman, Desney S Tan, Patrice Simard

19th ACM Symposium on User Interface Software and Technology, pp. 323-332, 2006