Microsoft Research: Computational User Experiences

A.J.
Brush

Siddharth
Khullar

Bongshin
Lee

Dan
Morris

T. Scott
Saponas

Greg
Smith

Desney
Tan

Shwetak
Patel

Logan
Kendall

Nicki
Dell

See our alumni page for previous CUE members.

Phoneprioception:
Enabling Mobile Phones to Infer Where They Are Kept

We collected data from 693 participants to understand where people keep their phone in different contexts and why. Using this data, we identified three placement personas: Single Place Pat, Consistent Casey, and All-over Alex. We also built prototypes employing capacitive, multispectral, and accelerometer sensing to infer phone placements automatically.

Always-Available Mobile Interfaces

This work seeks to make interaction more ubiquitous by appropriating both the human body and the surrounding environment as interaction canvases. We leverage sensors used in medical contexts, and we apply signal processing and machine learning techniques that extract data about gesture and behavior.

Home Automation

We are developing systems to simplify technology management and application deployment in the home. We focus in particular on the challenge of reducing home energy use: our PreHeat system uses occupancy prediction to control home heating.

SketchInsight

We develop a radically new way for people to create information visualizations and interact with their data through simple interactive sketches. We aim to make manipulating and analyzing data as easy as creating napkin sketches, and much more powerful.

Information Visualization

We are designing new visualization and interaction techniques that take advantage of pattern recognition, color and motion perception, spatial memory, and reasoning or sensemaking skills of users.

Patient-Friendly Medical Information Displays

Electronic medical records increasingly allow patients to access our medical information, but rarely help patients understand that information. Consequently, patients are still frequently under-informed about our own hospital courses. In this work, we address design and technology questions around patient-friendly medical information displays.

Interactive Machine Learning

We are exploring the dynamic interaction of human users with machine learning systems. Examples include CueTIP, a mixed-initiative interface for correcting handwriting recognition errors, and CueFlik, a system that interactively reranks image search results based on visual properties of examples provided.

Computational Tools for Music

We use signal processing and machine learning tools, combined with insights from traditional music creation, to give novices better access to musical creativity. Our group’s work in this area led to the development of Songsmith.

See our archived projects page for information about older CUE projects.

2013

Nudging People Away From Privacy-Invasive Mobile Apps Through Visual Framing

Eun Kyoung Choe, Jaeyeon Jung, Bongshin Lee, Kristie Fisher

Proceedings of Interact 2013, To appear

Phoneprioception: Enabling Mobile Phones to Infer Where They Are Kept

Jason Wiese, T. Scott Saponas, A.J. Bernheim Brush

Proceedings of ACM CHI 2013, May 2013

Benevolent Deception in Human Computer Interaction

Eytan Adar, Desney S Tan, Jaime Teevan

Proceedings of ACM CHI 2013, May 2013 (Best paper nomination)

Data Visualization on Interactive Surfaces: A Research Agenda from the DEXIS Workshop

Petra Isenberg, Tobias Isenberg, Tobias Hesselmann, Bongshin Lee, Ulrich von Zadow, Anthony Tang

IEEE CG&A, Vol. 33, No. 2, pp. 16-24, 2013 (Visualization Viewpoints)

2012

Foveated 3D Graphics

Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, John Snyder

Transactions on Graphics, 31(6), article 164

TouchWave: Kinetic Multi-touch Manipulation for Hierarchical Stacked Graphs

Dominikus Baur, Bongshin Lee, Sheelagh Carpendale

Proceedings of ITS 2012, November 2012

FaceReview: Supporting Interactive Exploration of Linked Heterogeneous Datasets for Unilateral Cleft Lip and Palate

Sungwon Lee, Boeun Kim, Bongshin Lee, Bohyoung Kim, Bohyung Han, Nina K. Anderson, Richard A. Bruun, Stephen Shusterman, Jinwook Seo

Proceedings of AMIA 2012, November 2012

Beyond Mouse and Keyboard: Expanding Design Considerations for Information Visualization Interactions

Bongshin Lee, Petra Isenberg, Nathalie Henry Riche, Sheelagh Carpendale

IEEE TVCG (InfoVis 2012), Vol. 18, No. 12, pp. 2689-2698

Understanding Pen and Touch Interaction for Data Exploration on Interactive Whiteboards

Jagoda Walny, Bongshin Lee, Paul Johns, Nathalie Henry Riche, Sheelagh Carpendale

IEEE TVCG (InfoVis 2012), Vol. 18, No. 12, pp. 2779-2788

An Ultra-Low-Power Human Body Motion Sensor Using Static Electric Field Sensing

Gabe Cohn, Sidhant Gupta, Tien-Jui Lee, Dan Morris, Joshua Smith, Matthew Reynolds, Desney Tan, Shwetak Patel

Proceedings of ACM Ubicomp 2012, September 2012 (Best paper award)

Learning to Learn: Algorithmic Inspirations from Human Problem Solving

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

Proceedings of AAAI 2012, July 2012

Performance and Preferences: Interactive Refinement of Machine Learning Procedures

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

Proceedings of AAAI 2012, July 2012

DiffMatrix: Matrix-based Interactive Visualization for Comparing Temporal Trends

Hyunjoo Song, Bongshin Lee, Bohyoung Kim, Jinwook Seo

Proceedings of EuroVis 2012, June 2012

Exploring the Design Space of Interactive Link Curvature in Network Diagrams

Nathalie Henry Riche, Tim Dwyer, Bongshin Lee, Sheelagh Carpendale

Proceedings of AVI 2012, May 2012

SpeechToast: Augmenting Notifications with Speech Input Focus

A.J. Bernheim Brush, Paul Johns

Proceedings of ACM AVI 2012, May 2012

Using Machine Learning to Support Pedagogy in the Arts

Dan Morris, Rebecca Fiebrink

Personal and Ubiquitous Computing, April 2012

Trajectory-Aware Mobile Search

Shahriyar Amini, A.J. Bernheim Brush, John Krumm, Jaime Teevan, and Amy Karlson