Microsoft Research
Computational User Experiences

Interactive Machine Learning

A large body of human-computer interaction research has focused on developing metaphors and tools that allow users to effectively issue commands and directly manipulate informational objects. However, with the advancement of computational techniques such as machine learning, we now have the unprecedented ability to embed 'smarts' that allow machines to assist users in completing their tasks. We believe that trying to fully automate tasks is extremely difficult and even undesirable, but instead there exists a computational design methodology which allows us to gracefully combine automated services with direct user manipulation.

 

Project Team

CueT

CueT     

Network alarm triage refers to grouping and prioritizing a stream of low-level device health information to help operators find and fix problems. Today, this process tends to be largely manual because existing tools cannot easily evolve with the network.

CueT is a system that uses interactive machine learning to learn from the triaging decisions of operators. It then uses that learning in novel visualizations to help them quickly and accurately triage alarms. Unlike prior interactive machine learning systems, CueT handles a highly dynamic environment where the groups of interest are not known a-priori and evolve constantly.

ManiMatrix

ManiMatrix     

ManiMatrix is a system that provides controls and visualizations that enable system builders to refine the behavior of classification systems in an intuitive manner. With ManiMatrix, users directly refine parameters of a confusion matrix via an interactive cycle of reclassification and visualization.

EnsembleMatrix

EnsembleMatrix     

Machine learning is an increasingly used computational tool within human-computer interaction research. While most researchers currently utilize an iterative approach to refining classifier models and performance, we propose that ensemble classification techniques may be a viable and even preferable alternative.

 

In ensemble learning, algorithms combine multiple classifiers to build one that is superior to its components. We designed and developed a new interactive visualization system that presents a graphical view of confusion matrices to help users under-stand relative merits of various classifiers.

CueFlik: Interactive Concept Learning in Image Search

Popular image search engines have begun to provide tags based on simple characteristics of images (such as tags for black and white images or images that contain a face), but such approaches are limited by the fact that it is unclear what tags end users want to be able to use in examining image search results. CueFlik is an image search application that allows end users to quickly create (and reuse) their own rules for re-ranking images based on their visual characteristics.

 

CueTIP: Mixed-Initiative Handwriting Recognition

Handwritten input is inherently ambiguous, and recognition systems will always make errors. Unfortunately, work on error recovery mechanisms has mainly focused on interface innovations that allow users to manually transform the erroneous recognition result into the intended one. In our work, we propose a mixed-initiative approach to error correction. CueTIP is a novel correction interface that takes advantage of the recognizer to continually evolve its results using the additional information from user corrections. This significantly reduces the number of actions required to reach the intended result.

Publications

Learning to Learn: Algorithmic Inspirations from Human Problem Solving

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

To appear in the Proceedings of AAAI 2012, July 2012

Performance and Preferences: Interactive Refinement of Machine Learning Procedures

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

To appear in the Proceedings of AAAI 2012, July 2012

Human-Guided Machine Learning for Fast and Accurate Network Alarm Triage

Saleema Amershi, Bongshin Lee, Ashish Kapoor, Ratul Mahajan, Blaine Christian

Proceedings of IJCAI 2011, July 2011

CueT: Human-Guided Fast and Accurate Network Alarm Triage

Saleema Amershi, Bongshin Lee, Ashish Kapoor, Ratul Mahajan, Blaine Christian

Proceedings of ACM CHI 2011, May 2011 (Best paper nomination)

Interactive Optimization for Steering Machine Classification

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

Proceedings of ACM CHI 2010, pp. 1343-1352.

EnsembleMatrix: Interactive Visualization to Support Machine Learning with Multiple Classifiers

Justin Talbot, Bongshin Lee, Ashish Kapoor, Desney S Tan

Proceedings of ACM CHI 2009, pp. 1283-1292.

CueFlik: Interactive Concept Learning in Image Search

James Fogarty, Desney S Tan, Ashish Kapoor, Simon Winder

Proceedings of ACM CHI 2008

CueTIP: A mixed-Initiative Interface for Correcting Handwriting Errors

Michael Shilman, Desney S Tan, Patrice Simard

19th ACM Symposium on User Interface Software and Technology, pp. 323-332, 2006

Contact Us Terms of Use Trademarks Privacy Statement ©2010 Microsoft Corporation. All rights reserved.Microsoft