Microsoft Research
Computational User Experiences

Overview

The Computational User Experiences (CUE) group applies expertise in machine learning, visualization, mobile computing, sensors, and devices to advance the state of the art in human-computer interaction, health care, home technologies, and computer-assisted creativity.

 

People

Current Research Areas

Automatic Exercise Recognition

Workout: Automatic Exercise Analysis

Although numerous devices exist to track and share exercise routines based on running and walking, these devices offer limited functionality for strength-training exercises. We introduce a system for automatically tracking repetitive exercises – such as weight training and calisthenics – via an arm-worn inertial sensor, with no user-specific training and no intervention during a workout.

 
Always-Available Mobile Interfaces

Always-Available Mobile Interfaces

This work seeks to make interaction more ubiquitous by appropriating both the human body and the surrounding environment as interaction canvases. We leverage sensors used in medical contexts, and we apply signal processing and machine learning techniques that extract data about gesture and behavior.

 
Home Automation

Home Automation

We are developing systems to simplify technology management and application deployment in the home. We focus in particular on the challenge of reducing home energy use: our PreHeat system uses occupancy prediction to control home heating.

 
SketchInsight

SketchInsight

We develop a radically new way for people to create information visualizations and interact with their data through simple interactive sketches. We aim to make manipulating and analyzing data as easy as creating napkin sketches, and much more powerful.

 
Information Visualization

Information Visualization

We are designing new visualization and interaction techniques that take advantage of pattern recognition, color and motion perception, spatial memory, and reasoning or sensemaking skills of users.

 
Patient-Friendly Medical Information Displays

Patient-Friendly Medical Information Displays

Electronic medical records increasingly allow patients to access our medical information, but rarely help patients understand that information. Consequently, patients are still frequently under-informed about our own hospital courses. In this work, we address design and technology questions around patient-friendly medical information displays.

 
Interactive Machine Learning

Interactive Machine Learning

We are exploring the dynamic interaction of human users with machine learning systems. Examples include CueTIP, a mixed-initiative interface for correcting handwriting recognition errors, and CueFlik, a system that interactively reranks image search results based on visual properties of examples provided.

 
Computational Tools for Music

Computational Tools for Music

We use signal processing and machine learning tools, combined with insights from traditional music creation, to give novices better access to musical creativity. Our group’s work in this area led to the development of Songsmith.

 

Past Projects and Research Areas

See our archived projects page for information about older CUE projects.

Publications

2014

A combined segmenting and non-segmenting approach to signal quality estimation for ambulatory photoplethysmography

Jeremiah Wander, Dan Morris

To appear in the Journal of Physiological Measurement

Interactively Stylizing Camera Motion

Neel Joshi, Dan Morris, Michael Cohen

Proceedings of ACM CHI 2014, April 2014

Using a Wearable Sensor to Find, Recognize, and Count Repetitive Exercises

Dan Morris, T. Scott Saponas, Andrew Guillory, Ilya Kelner

Proceedings of ACM CHI 2014, April 2014 (Best talk award)

Understanding Quantified-Selfers’ Practices in Collecting and Exploring Personal Data

Eun Kyoung Choe, Nicole B. Lee, Bongshin Lee, Wanda Pratt, Julie A. Kientz

Proceedings of ACM CHI 2014, April 2014 (Best paper nomination)

DemoWiz: Re-Performing Software Demonstrations for a Live Presentation

Pei-Yu (Peggy) Chi, Bongshin Lee, Steven M. Drucker

Proceedings of ACM CHI 2014, April 2014

2013

Persuasive Performance Feedback: The Effect of Framing on Self-Efficacy

Eun Kyoung Choe, Bongshin Lee, Sean Munson, Wanda Pratt, Julie A. Kientz

Proceedings of the AMIA Annual Symposium, November 2013

SketchStory: Telling More Engaging Stories with Data through Freeform Sketching

Bongshin Lee, Rubaiat Habib Kazi, Greg Smith

IEEE Transactions on Visualization and Computer Graphics, 19(12): 2416-25, December 2013

A Deeper Understanding of Sequence in Narrative Visualization

Jessica Hullman, Steven Drucker, Nathalie Henry Riche, Bongshin Lee, Danyel Fisher, Eytan Adar

IEEE Transactions on Visualization and Computer Graphics, 19(12):2406-15, December 2013

The Sound of Touch: On-body Touch and Gesture Sensing Based on Transdermal Ultrasound

Adiyan Mujibiya, Xiang Cao, Desney S. Tan, Dan Morris, Shwetak N. Patel, Jun Rekimoto

Proceedings of ACM Interactive Tabletops and Surfaces (ITS), October 2013

AirWave: Non-Contact Haptic Feedback Using Air Vortex Rings

Sidhant Gupta, Dan Morris, Shwetak Patel, Desney Tan

Proceedings of Ubicomp 2013, September 2013

There's No Such Thing as Gaining a Pound: Reconsidering the Bathroom Scale User Interface.

Matt Kay, Dan Morris, mc scraefel, Julie Kientz

Proceedings of Ubicomp 2013, September 2013 (Best paper award)

Nudging People Away From Privacy-Invasive Mobile Apps Through Visual Framing

Eun Kyoung Choe, Jaeyeon Jung, Bongshin Lee, Kristie Fisher

Proceedings of Interact 2013, September 2013

Phoneprioception: Enabling Mobile Phones to Infer Where They Are Kept

Jason Wiese, T. Scott Saponas, A.J. Bernheim Brush

Proceedings of ACM CHI 2013, May 2013

Benevolent Deception in Human Computer Interaction

Eytan Adar, Desney S Tan, Jaime Teevan

Proceedings of ACM CHI 2013, May 2013 (Best paper nomination)

Data Visualization on Interactive Surfaces: A Research Agenda from the DEXIS Workshop

Petra Isenberg, Tobias Isenberg, Tobias Hesselmann, Bongshin Lee, Ulrich von Zadow, Anthony Tang

IEEE CG&A, Vol. 33, No. 2, pp. 16-24, 2013 (Visualization Viewpoints)

2012

Foveated 3D Graphics

Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, John Snyder

Transactions on Graphics, 31(6), article 164

TouchWave: Kinetic Multi-touch Manipulation for Hierarchical Stacked Graphs

Dominikus Baur, Bongshin Lee, Sheelagh Carpendale

Proceedings of ITS 2012, November 2012

FaceReview: Supporting Interactive Exploration of Linked Heterogeneous Datasets for Unilateral Cleft Lip and Palate

Sungwon Lee, Boeun Kim, Bongshin Lee, Bohyoung Kim, Bohyung Han, Nina K. Anderson, Richard A. Bruun, Stephen Shusterman, Jinwook Seo

Proceedings of AMIA 2012, November 2012

Beyond Mouse and Keyboard: Expanding Design Considerations for Information Visualization Interactions

Bongshin Lee, Petra Isenberg, Nathalie Henry Riche, Sheelagh Carpendale

IEEE TVCG (InfoVis 2012), Vol. 18, No. 12, pp. 2689-2698

Understanding Pen and Touch Interaction for Data Exploration on Interactive Whiteboards

Jagoda Walny, Bongshin Lee, Paul Johns, Nathalie Henry Riche, Sheelagh Carpendale

IEEE TVCG (InfoVis 2012), Vol. 18, No. 12, pp. 2779-2788

An Ultra-Low-Power Human Body Motion Sensor Using Static Electric Field Sensing

Gabe Cohn, Sidhant Gupta, Tien-Jui Lee, Dan Morris, Joshua Smith, Matthew Reynolds, Desney Tan, Shwetak Patel

Proceedings of ACM Ubicomp 2012, September 2012 (Best paper award)

Learning to Learn: Algorithmic Inspirations from Human Problem Solving

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

Proceedings of AAAI 2012, July 2012

Performance and Preferences: Interactive Refinement of Machine Learning Procedures

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

Proceedings of AAAI 2012, July 2012

DiffMatrix: Matrix-based Interactive Visualization for Comparing Temporal Trends

Hyunjoo Song, Bongshin Lee, Bohyoung Kim, Jinwook Seo

Proceedings of EuroVis 2012, June 2012

Exploring the Design Space of Interactive Link Curvature in Network Diagrams

Nathalie Henry Riche, Tim Dwyer, Bongshin Lee, Sheelagh Carpendale

Proceedings of AVI 2012, May 2012

SpeechToast: Augmenting Notifications with Speech Input Focus

A.J. Bernheim Brush, Paul Johns

Proceedings of ACM AVI 2012, May 2012

Using Machine Learning to Support Pedagogy in the Arts

Dan Morris, Rebecca Fiebrink

Personal and Ubiquitous Computing, April 2012

Trajectory-Aware Mobile Search

Shahriyar Amini, A.J. Bernheim Brush, John Krumm, Jaime Teevan, and Amy Karlson

Proceedings of ACM CHI 2012, May 2012

"You're Capped!" Understanding the Effects of Bandwidth Caps on Broadband Use in the Home

Marshini Chetty, Richard Banks, A.J. Bernheim Brush, Jonathan Donner, and Rebecca Grinter

Proceedings of ACM CHI 2012, May 2012

Your Phone or Mine? Fusing Body, Touch and Device Sensing for Multi-User Device-Display Interaction

Mahsan Rofouei, Andrew D. Wilson, A.J. Bernheim Brush, and Stewart Tansley

Proceedings of ACM CHI 2012, May 2012

GraphTrail: Analyzing Large Multivariate, Heterogeneous Networks while Supporting Exploration History

Cody Dunne, Nathalie Henry Riche, Bongshin Lee, Ronald Metoyer, George Robertson

Proceedings of ACM CHI 2012, May 2012

Understanding the Verbal Language and Structure of End-User Descriptions of Data Visualizations

Ronald Metoyer, Bongshin Lee, Nathalie Henry Riche, Mary Czerwinski

Proceedings of ACM CHI 2012, May 2012

Voice Typing: A New Speech Interaction Model for Dictation on Touchscreen Devices

Anuj Kumar, Tim Paek, Bongshin Lee

Proceedings of ACM CHI 2012, May 2012

GyroTab: A Handheld Device that Provides Reactive Torque Feedback

Akash Badshah, Sidhant Gupta, Dan Morris, Shwetak Patel, Desney Tan

Proceedings of ACM CHI 2012, May 2012

Humantenna: Using the Body as an Antenna for Real-Time Whole-Body Interaction

Gabe Cohn, Dan Morris, Shwetak Patel, Desney Tan

Proceedings of ACM CHI 2012, May 2012 (Best paper nomination)

SoundWave: Using the Doppler Effect to Sense Gestures

Sidhant Gupta, Dan Morris, Shwetak Patel, Desney Tan

Proceedings of ACM CHI 2012, May 2012

Making Gestural Input from Arm-Worn Inertial Sensors More Practical

Louis Kratz, T. Scott Saponas, Dan Morris

Proceedings of ACM CHI 2012, May 2012

Using Mobile Phones to Present Medical Information to Hospital Patients

Laura Pfeifer Vardoulakis, Amy Karlson, Dan Morris, Greg Smith, Justin Gatewood, Desney Tan

Proceedings of ACM CHI 2012, May 2012

An Operating System for the Home

Colin Dixon, Ratul Mahajan, Sharad Agarwal, A.J. Bernheim Brush, Bongshin Lee, Stefan Saroiu, Paramvir Bahl

Proceedings of NSDI 2012, April 2012

2011

Emerging Input Technologies for Always-Available Mobile Interaction

Dan Morris, Scott Saponas, Desney Tan

Foundations and Trends in Human-Computer Interaction, 4(4), pp. 245-316, November 2011

Data Analysis on Interactive Whiteboards through Sketch-based Interaction

Jeffrey Browne, Bongshin Lee, Sheelagh Carpendale, Nathalie Riche, Timothy Sherwood

Proceedings of ITS 2011, November 2011

PocketTouch: Through-Fabric Capacitive Touch Input

T. Scott Saponas, Chris Harrison, Hrvoje Benko

Proceedings of UIST 2011, October 2011

Research Directions for Data Wrangling: Visualizations and Transformations for Usable and Credible Data

Sean Kandel, Jeffrey Heer, Catherine Plaisant, Jessie Kennedy, Frank van Ham, Nathalie Henry Riche, Chris Weaver, Bongshin Lee, Dominique Brodbeck, Paolo Buono

Information Visualization, Vol. 10, pp. 271-288, October 2011

Motion and Attention in a Kinetic Videoconferencing Proxy

David Sirkin, Gina Venolia, John Tang, George Robertson, Taemie Kim, Kori Inkpen, Mara Sedlins, Bongshin Lee, Mike Sinclair

Proceedings of Interact 2011, September 2011

PreHeat: Controlling Home Heating Using Occupancy Prediction

James Scott, A.J. Bernheim Brush, John Krumm, Brian Meyers, Mike Hazas, Steve Hodges and Nicolas Villar

Proceedings of ACM Ubicomp 2011, September 2011

Skinput: Appropriating the Skin as an Interactive Canvas

Chris Harrison, Desney Tan, Dan Morris

Communications of the ACM 54.8, August 2011

Human-Guided Machine Learning for Fast and Accurate Network Alarm Triage

Saleema Amershi, Bongshin Lee, Ashish Kapoor, Ratul Mahajan, Blaine Christian

Proceedings of IJCAI 2011, July 2011

Honest Signals in Video Conferencing

Byungki Byun, Anurag Awasthi, Philip A. Chou, Ashish Kapoor, Bongshin Lee, Mary Czerwinski

Proceedings of ICME 2011, July 2011

SpeakerSense: Energy Efficient Unobtrusive Speaker Identification on Mobile Phones

Hong Lu, A.J. Bernheim Brush, Bodhi Priyantha, Amy Karlson, Jie Liu

Proceedings of Pervasive 2011, June 2011 (Best paper nomination)

Learning Time-Based Presence Probabilities

John Krumm and A.J. Bernheim Brush

Proceedings of Pervasive 2011, June 2011 (Best paper award)

CueT: Human-Guided Fast and Accurate Network Alarm Triage

Saleema Amershi, Bongshin Lee, Ashish Kapoor, Ratul Mahajan, Blaine Christian

Proceedings of ACM CHI 2011, May 2011 (Best paper nomination)

Which Version is This?: Improving the Desktop Experience within a Copy-Aware Computing Ecosystem

Amy K. Karlson, Greg Smith, Bongshin Lee

Proceedings of ACM CHI 2011, May 2011 (Best paper nomination)

Home Automation in the Wild: Challenges and Opportunities

A.J. Bernheim Brush, Bongshin Lee, Ratul Mahajan, Sharad Agarwal, Stefan Saroiu, Colin Dixon

Proceedings of ACM CHI 2011, May 2011

Your Noise is My Command: Sensing Gestures Using the Body as an Antenna

Gabe Cohn, Dan Morris, Shwetak Patel, Desney S Tan

Proceedings of ACM CHI 2011, May 2011 (Best paper award)

Characterizing Patient-Friendly “Micro-Explanations” of Medical Events

Lauren Wilcox, Dan Morris, Desney Tan, Justin Gatewood, Eric Horvitz

Proceedings of ACM CHI 2011, May 2011

ClassSearch: Facilitating the Development of Web Search Skills through Social Learning

Neema Moraveji, Meredith Ringel Morris, Dan Morris, Mary Czerwinski, Nathalie Riche

Proceedings of ACM CHI 2011, May 2011 (Best paper nomination)

2010

Identifying Opportunities for Inpatient-Centric Technology

Meredith Skeels, Desney Tan

Proceedings of ACM IHI 2010, November 2010

Physician Attitudes About Patient-Facing Information Displays at an Urban Emergency Department

Lauren Wilcox, Justin Gatewood, Dan Morris, Desney Tan, Eric Horvitz, Steven Feiner

Proceedings of AMIA 2010, November 2010

SparkClouds: Visualizing Trends in Tag Clouds

Bongshin Lee, Nathalie Henry Riche, Amy K. Karlson, Sheelagh Carpendale

IEEE TVCG (InfoVis 2010), Vol. 16, No. 6, pp. 1182-1189

ManiWordle: Providing Flexible Control over Wordle

Kyle Koh, Bongshin Lee, Bohyoung Kim, Jinwook Seo

IEEE TVCG (InfoVis 2010), Vol. 16, No. 6, pp. 1190-1197

NetClinic: Interactive Visualization to Enhance Automated Fault Diagnosis in Enterprise Networks

Zhicheng Liu, Bongshin Lee, Srikanth Kandula, Ratul Mahajan

Proceedings of VAST 2010, pp. 131-138

AllocRay: Memory Allocation Visualization for Unmanaged Languages

George Robertson, Trishul Chilimbi, Bongshin Lee

Proceedings of SoftVis 2010, October 2010

The Home Needs an Operating System (and an App Store)

Colin Dixon, Ratul Mahajan, Sharad Agarwal, A.J. Brush, Bongshin Lee, Stefan Saroiu, and Victor Bahl

HotNets, October 2010

Brain-Computer Interaction: Applying our Minds to Human-Computer Interaction

Desney S Tan, Anton Nijholt (eds)

Springer-Verlag: London. ISBN 987-1-84996-271-1, e-ISBN 978-1-84996-272-8, 2010

Exploring End User Preferences for Location Obfuscation, Location-Based Services, and the Value of Location

A.J. Bernheim Brush, John Krumm, James Scott

ACM UbiComp 2010, September 2010

User Experiences with Activity-Based Navigation on Mobile Devices

A.J. Bernheim Brush, Amy Karlson, James Scott and Menlo Team

ACM Mobile HCI 2010, September 2010

User-Specific Learning for Recognizing a Singer’s Intended Pitch

Andrew Guillory, Sumit Basu, Dan Morris

Proceedings of AAAI 2010, July 2010

Understanding Interactive Legends: a Comparative Evaluation with Standard Widgets

Nathalie Henry Riche, Bongshin Lee, Catherine Plaisant

Proceedings of IEEE EuroVis 2010, June 2010

Interfaces on the Go: Enabling Mobile Micro-Interactions with Physiological Computing

Desney Tan, Dan Morris, and Scott Saponas

ACM Crossroads Magazine, June 2010

Multi-Modal Selection During Text Entry on Mobile Devices

David Dearman, Amy Karlson, Brian Meyers and Benjamin Bederson

Proceedings of ACM Graphics Interface 2010, May 2010

iChase: Supporting Exploration and Awareness of Editing Activities on Wikipedia

Nathalie Henry Riche, Bongshin Lee, Fanny Chevalier

Proceedings of AVI 2010, pp. 59-66

Mobile Task Flow in Context: A Screen Shot Study of Smartphone Usage

Amy K. Karlson, Shamsi T. Iqbal, Brian Meyers, Gonzalo Ramos, Kathy Lee and John C. Tang

Proceedings of ACM CHI 2010, April 2010

Interactive Optimization for Steering Machine Classification

Ashish Kapoor, Bongshin Lee, Desney Tan, Eric Horvitz

Proceedings of ACM CHI 2010, pp. 1343-1352

A Comparative Evaluation on Tree Visualization Methods for Hierarchical Structures with Large Fan-outs

Hyunjoo Song, Bohyoung Kim, Bongshin Lee, Jinwook Seo

Proceedings of ACM CHI 2010, pp. 223-232

Newport: Enabling Sharing During Mobile Calls

Junius A. Gunaratne and A.J. Bernheim Brush

Proceedings of ACM CHI 2010, April 2010

Video Playdate: Toward Free Play Across Distance

Svetlana Yarosh, Kori, M. Inkpen, A.J. Bernheim Brush

Proceedings of ACM CHI 2010, April 2010

Embodied Social Proxy: Mediating Interpersonal Connection in Hub-and-Satellite Teams

Gina Venolia, John Tang, Ruy Cervantes, Sara Bly, George Robertson, Bongshin Lee, Kori Inkpen

Proceedings of ACM CHI 2010, April 2010

Designing Patient-Centric Information Displays for Hospitals

Lauren Wilcox, Dan Morris, Desney Tan, Justin Gatewood

Proceedings of ACM CHI 2010, April 2010 (Best paper nomination)

Making Muscle-Computer Interfaces More Practical

Scott Saponas, Desney Tan, Dan Morris, Jim Turner, and James Landay

Proceedings of ACM CHI 2010, April 2010

Skinput: Appropriating the Body as an Input Surface

Chris Harrison, Desney Tan, Dan Morris

Proceedings of ACM CHI 2010, April 2010 (Best paper award)

Automatic Classification of Daily Fluid Intake

Jonathan Lester, Desney Tan, Shwetak Patel, A.J. Bernheim Brush

Proceedings of Pervasive Health 2010, March 2010 (Best paper award)

Understanding Family Communication across Time Zones

Xiang Cao, Abigail Sellen, A.J. Bernheim Brush, David Kirk, Darren Edge, Xianghua Ding

Proceedings of CSCW 2010, Feb. 2010, pp. 155-158

Revealing Uncertainty for Information Visualization

Meredith Skeels, Bongshin Lee, Greg Smith, George Robertson

Information Visualization, Vol. 9, No. 1, 70-81

2009

Selected Human Factors Issues in Information Visualization

George Robertson, Mary Czerwinski, Danyel Fisher, Bongshin Lee

Reviews of Human Factors and Ergonomics, Volume 5, pp. 41-81

iSee: Interactive Scenario Explorer for Online Tournament Games

Greg Smith, Desney S Tan, Bongshin Lee

Proceedings of ACE 2009, pp. 190-197

Enhancing Input On and Above the Interactive Surface with Muscle Sensing

Hrvoje Benko, Scott Saponas, Dan Morris, Desney Tan

Proceedings of ACM Tabletop 2009, November 2009

Enabling Always-Available Input with Muscle-Computer Interfaces

Scott Saponas, Desney Tan, Dan Morris, Ravin Balakrishnan, Jim Tuner, James Landay

Proceedings of ACM UIST 2009, October 2009

Optically Sensing Tongue Gestures for Computer Input

Scott Saponas, Daniel Kelly, Desney Tan, Babak Parviz

Proceedings of ACM UIST 2009, October 2009

Collabio: A Game for Annotating People within Social Networks

Michael Bernstein, Desney Tan, Greg Smith, Mary Czerwinski, Eric Horvitz

Proceedings of ACM UIST 2009, October 2009

Overview-Based Example Selection in Mixed-Initiative Interactive Concept Learning

Saleema Amershi, James Fogarty, Ashish Kapoor, Desney Tan

Proceedings of ACM UIST 2009, October 2009

Relationships Between Lyrics and Melody in Popular Music

Eric Nichols, Dan Morris, Sumit Basu, Chris Raphael

Proceedings of ISMIR 2009, October 2009

GeneShelf: A Web-based Visual Interface for Large Gene Expression Time-Series Data Repositories

Bohyoung Kim, Bongshin Lee, Susan Knoblach, Eric Hoffman, Jinwook Seo

IEEE TVCG (InfoVis 2009), Vol. 15, No. 6, pp. 905-912

A Comparison of User-Generated and Automatic Graph Layouts

Tim Dwyer, Bongshin Lee, Danyel Fisher, Kori Inkpen Quinn, Petra Isenberg, George Robertson, Chris North

IEEE TVCG (InfoVis 2009), Vol. 15, No. 6, pp. 961-968

PhyloDet: A Scalable Visualization Tool for Mapping Multiple Traits to Large Evolutionary Trees

Bongshin Lee, Lev Nachmanson, George Robertson, Jonathan M. Carlson, David Heckerman

Bioinformatics, Vol. 25, No. 19, pp. 2611-2612

Designing Phrase Builder: A Mobile Real-Time Query Expansion Interface

Tim Paek, Bongshin Lee, Bo Thiesson

Proceedings of MobileHCI 2009

Investigating the Use of Voice and Ink for Mobile Micronote Capture

Adrienne Andrew, Amy K. Karlson, and A.J. Bernheim Brush

Proceedings of Interact 2009

Understanding Multi-touch Manipulation for Surface Computing

Chris North, Tim Dwyer, Bongshin Lee, Danyel Fisher, Petra Isenberg, George Robertson, Kori Inkpen

Proceedings of Interact 2009

Designing Novel Image Search Interfaces by Understanding Unique Characteristics and Usage

Paul Andre, Edward Cutrell, Desney Tan, Greg Smith

Proceedings of Interact 2009

It’s no secret: Measuring the security and reliability of authentication via ‘secret’ questions

Stuart Schechter, A. J. Bernheim Brush, and Serge Egelman

2009 IEEE Symposium on Security and Privacy

Toward Technologies that Support Family Reflections on Health

Andrea Grimes, Desney Tan, Dan Morris

Proceedings of ACM Group 2009

It’s Not Easy Being Green: Understanding Home Computer Power Management

Marshini Chetty, A.J. Bernheim Brush, Brian R. Meyers, Paul Johns

Proceedings of ACM CHI 2009, pp. 1033-1042

Can I Borrow Your Phone? Understanding Concerns When Sharing Mobile Phones

Amy Karlson, A.J. Bernheim Brush, Stuart Schechter

Proceedings of ACM CHI 2009, pp. 1647-1650

Exploring Awareness Needs and Information Display Preferences Between Coworkers

A.J. Bernheim Brush, Brian R. Meyers, James Scott, Gina Venolia

Proceedings of ACM CHI 2009, pp. 2091-2094

FacetLens: Exposing Trends and Relationships to Support Sensemaking within Faceted Datasets

Bongshin Lee, Greg Smith, George G Robertson, Mary Czerwinski, Desney S Tan

Proceedings of ACM CHI 2009, pp. 1293-1302

EnsembleMatrix: Interactive Visualization to Support Machine Learning with Multiple Classifiers

Justin Talbot, Bongshin Lee, Ashish Kapoor, Desney S Tan

Proceedings of ACM CHI 2009, pp. 1283-1292

Dynamic Mapping of Physical Controls for Tabletop Groupware

Rebecca Fiebrink, Dan Morris, Meredith Morris

Proceedings of ACM CHI 2009, pp. 471-480

Investigating Web Search Strategies and Forum Use to Support Diet and Weight Loss

mc shraefel, Ryen White, Paul Andre, Desney S Tan

Proceedings of ACM CHI 2009, pp. 3829-3834

“The Calendar is Crucial”: Coordination and Awareness through the Family Calendar

Carman Neustaedter, A.J. Bernheim Brush, and Saul Greenberg

ACM Transactions on Computer-Human Interaction, Vol 16, Issue 1, April 2009, Article 6

Data-Driven Exploration of Musical Chord Sequences

Eric Nichols, Dan Morris, Sumit Basu

Proceedings of Intelligent User Interfaces (IUI) 2009, pp. 227-236.

Exploring communication and sharing between extended families

Kimberly Tee, A.J. Brush, Kori Inkpen

International Journal of Human-Computer Studies, Vol. 67, No. 2, pp. 128-138, Elsevier

2008

SPARCS: Exploring Sharing Suggestions to Enhance Family Connectedness

A.J. Bernheim Brush, Kori M. Inkpen, Kimberly Tee

Proceedings of ACM CSCW 2008, November 2008

Family Accounts: A new paradigm for user accounts within the home environment

Serge Egelman, A.J. Brush, Kori Inkpen

Proceedings of ACM CSCW 2008, November 2008

Search Vox: Leveraging Multimodal Refinement and Partial Knowledge for Mobile Voice Search

Tim Paek, Bo Thiesson, Y.C. Ju, Bongshin Lee

ACM UIST 2008, pp.141-150

Effectiveness of Animation in Trend Visualization

George Robertson, Roland Fernandez, Danyel Fisher, Bongshin Lee, John Stasko

IEEE TVCG (InfoVis 2008), Vol. 14, No. 6, pp.1325-1332

Complementary Computing for Visual Tasks: Meshing Computer Vision with Human Visual Processing

Ashish Kapoor, Desney S Tan, Pradeep Shenoy, Eric Horvitz

IEEE Conference on Automatic Face and Gesture Recognition 2008

Combining Brain Computer Interfaces with Vision for Object Categorization

Ashish Kapoor, Pradeep Shenoy, Desney S Tan

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2008

Exposing Parameters of a Trained Dynamic Model for Interactive Music Creation

Dan Morris, Ian Simon, Sumit Basu

Proceedings of AAAI 2008

GOTreePlus: An Interactive Gene Ontology Browser

Bongshin Lee, Kristy Brown, Yetrib Hathout, Jinwook Seo

Bioinformatics, Vol. 24, No. 7, 2008, pp.1026-1028

Revealing Uncertainty for Information Visualization

Meredith Skeels, Bongshin Lee, Greg Smith, George Robertson

Advanced Visual Interfaces 2008, pp.376-379

TapGlance: Designing a Unified Smartphone Interface

Daniel C. Robbins, Bongshin Lee, Roland Fernandez

Designing Interactive Systems 2008, pp.386-394

Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces

T Scott Saponas, Desney S Tan, Dan Morris, Ravin Balakrishnan

Proceedings of ACM CHI 2008

Feasibility and Pragmatics of Classifying Working Memory Load with an Electroencephalograph

David Grimes, Desney S Tan, Scott Hudson, Pradeep Shenoy, Rajesh Rao

Proceedings of ACM CHI 2008

Human-Aided Computing: Utilizing Implicit Human Processing to Classify Images

Pradeep Shenoy, Desney S Tan

Proceedings of ACM CHI 2008

CueFlik: Interactive Concept Learning in Image Search

James Fogarty, Desney S Tan, Ashish Kapoor, Simon Winder

Proceedings of ACM CHI 2008

IMPROMPTU: A New Interaction Framework for Supporting Collaboration in Multiple Display Environments and its Field Evaluation for CO-located Software Development

Jacob Biehl, William Baker, Brain Bailey, Desney S Tan, Kori Inkpen, Mary Czerwinski

Proceedings of ACM CHI 2008

SuperBreak: Using Interactivity to Enhance Ergonomic Typing Breaks

Dan Morris, A.J. Brush, Brian Meyers

Proceedings of ACM CHI 2008

SearchBar: A Search-Centric Web History for Task Resumption and Information Re-finding

Dan Morris, Meredith Ringel Morris, Gina Venolia

Proceedings of ACM CHI 208

MySong: Automatic Accompaniment Generation for Vocal Melodies

Ian Simon, Dan Morris, Sumit Basu

Proceedings of ACM CHI 2008

Predictability and Accuracy in Adaptive User Interfaces

Krzysztof Gajos, Katherine Everitt, Desney S Tan, Mary Czerwinski, Daniel Weld

Extended Abstracts of ACM CHI 2008

Brain-Computer Interfaces for Human-Computer Interaction and Games

Anton Nijholt, Desney S Tan, Brendan Allison, Jose del R Millan, Bernhard Graimann, Melody Moore Jackson

Workshop at ACM CHI 2008

2007

Yours, Mine and Ours? Sharing and Use of Technology in Domestic Environments

A.J. Brush, Kori Inkpen

Proceedings of Ubicomp 2007, September 2007

CandidTree: Visualizing Structural Uncertainty in Similar Hierarchies

Bongshin Lee, George G. Robertson, Mary Czerwinski, Cynthia Sims Parr

Information Visualization, Vol. 6, 2007, pp.233-246

AdaptiviTree: Adaptive Tree Visualization for Tournament-Style Brackets

Desney S Tan, Greg Smith, Bongshin Lee, George G. Robertson

Proceedings of IEEE InfoVis 2007, pp.1113-1120

Visualizing Set Concordance with Permutation Matrices and Fan Diagrams

Bohyoung Kim, Bongshin Lee, Jinwook Seo

Interacting with Computers, Vol. 19, Issues 5-6, 2007, pp.630-643

Drawing Graphs with GLEE

Lev Nachmanson, George Robertson, Bongshin Lee

Graph Drawing 2007, pp.389-394

EcoLens: Integration and interactive visualization of ecological datasets

Cynthia Sims Parr, Bongshin Lee, Benjamin B. Bederson

Ecological Informatics, Vol. 2, Issue 1, 2007, pp.61-69

NetLens: Iterative Exploration of Content-Actor Network Data

Hyunmo Kang, Catherine Plaisant, Bongshin Lee, Benjamin B. Bederson

Information Visualization, Vol. 6, 2007, pp.18-31

Supporting Human Memory with a Personal Digital Lifetime Store

Desney S Tan, Mary Czerwinski, Gordon Bell, Emma Berry, Jim Gemmell, Steve Hodges, Narinder Kapur, Brian Meyers, Nuria Oliver, George Robertson, Ken Wood

Personal Information Management: Challenges and Opportunities (eds. William Jones and Jaime Teevan), University of Washington Press, 2007

Investigating the Querying and Browsing Behavior of Advanced Search Engine Users

Ryen White, Dan Morris

Proceedings of ACM SIGIR 2007

Instrumenting the Dynamic Web

Andy Edmonds, Ryen White, Dan Morris, Steven Drucker

Journal of Web Engineering (JWE), Vol. 6, No. 3 (2007), 243-260.

New Frontiers in Desktop Task Management

George Robertson, Greg Smith, Patrick Baudisch, Mary Czerwinski, Eric Horvitz, Brian Meyers, Daniel Robbins, Desney S Tan

Beyond the Desktop Metaphor: Designing Integrated Digital Work Environments (eds. Victor Kaptelinn and Mary Czerwinski), MIT Press, 2007

FASTDash: A visual dashboard for fostering awareness in software teams

Jacob Biehl, Mary Czerwinski, Greg Smith, George Robertson, Brian Bailey

Proceedings of CHI 2007

Using Job-Shop Scheduling Tasks for Evaluating Collocated Communication

Desney S Tan, Darren Gergle, Regan Mandryk, Kori Inkpen, Melanie Kellar, Kirstie Hawkey, Mary Czerwinski

Journal of Personal and Ubiquitous Computing (Special Issue on User-Centered Design and Evaluation of Services for Human-Human Communication and Collaboration), 2007

InkSeine: In Situ Search for Active Note Taking

Ken Hinckley, Shen Zhao, Raman Sarin, Patrick Baudisch, Ed Cutrell, Michael Shilman, Desney S Tan

CHI 2007 Conference on Human Factors in Computing Systems, pp. 251-260

An Evaluation of Extended Validation and Picture-in-Picture Phishing Attacks

Collin Jackson, Dan Simon, Desney S Tan, Adam Barth

Usable Security 2007

Understanding Memory Triggers for Task Tracking

AJ Brush, Brian Meyers, Desney S Tan, Mary Czerwinski

Extended Abstracts of ACM CHI 2007, pp. 947-950.

Health-OS: A Position Paper

Darko Kirovski, Nuria Oliver, Mike Sinclair, Desney S Tan

Poster at the 1st International Workshop on Systems and Networking Support for Healthcare and Assisted Living Environments (HealthNet07)

Playing with your Brain: Brain-Computer Interfaces and Games

Anton Nijholt, Desney S Tan

Organizing paper for Brainplay Workshop at Advances in Computer Entertainment 2007

2006

FacetMap: A Scalable Search and Browse Visualization

Greg Smith, Mary Czerwinski, Brian Meyers, Daniel Robbins, George Robertson, Desney S Tan

IEEE Transactions on Visualization and Computer Graphics, 12(5), pp. 797-804, 2006

An Exploration of User Interface Designs for Real-time Panoramic Photography

Patrick Baudisch, Desney S Tan, Drew Steedly, Eric Rudolph, Matt Uyttendaele, Chris Pal, Richard Szeliski

Australian Journal of Information Systems, 13(2), pp. 151-166, 2006

Large Displays Enhance Optical Flow Cues and Narrow the Gender Gap in 3D Virtual Navigation

Desney S Tan, Mary Czerwinski, George Robertson

Human Factors: The Journal of the Human Factors and Ergonomics Society, 48(2), pp.318-333, 2006

Physically Large Displays Improve Performance on Spatial Tasks

Desney S Tan, Darren Gergle, Peter Scupelli, Randy Pausch

ACM Transactions on Computer-Human Interaction, 13(1), pp. 71-99, 2006

Using a Low-Cost Electroencephalograph for Task Classification in HCI Research

Johnny C Lee, Desney S Tan

19th ACM Symposium on User Interface Software and Technology, pp. 81-90, 2006

CueTIP: A mixed-Initiative Interface for Correcting Handwriting Errors

Michael Shilman, Desney S Tan, Patrice Simard

19th ACM Symposium on User Interface Software and Technology, pp. 323-332, 2006

FaThumb: A facet-based interface for mobile search

Amy Karlson, George Robertson, Daniel Robbins, Mary Czerwinski, Greg Smith

CHI 2006 Conference on Human Factors in Computing Systems

SWISH:Semantic Analysis of Window Titles and Switching History

Nuria Oliver, Greg Smith, Chintan Thakkar, Arun Surendran

International Conference on Intelligent User Interfaces (IUI 2006)

Phosphor: Explaining Transitions in the User Interface using Afterglow Effects

Patrick Baudisch, Desney S Tan, Maxime Collomb, Dan Robbins, Ken Hinckley, Maneesh Agrawala, Shengdong Zhao, Gonzalo Ramos

19th ACM Symposium on User Interface Software and Technology, pp. 169-178, 2006

Tumble! Splat! Helping Users Access and Manipulate Occluded Content in 2D Drawings

Gonzalo Ramos, George Robertson, Mary Czerwinski, Desney S Tan, Patrick Baudisch, Ken Hinckley, Maneesh Agrawala, Daniel Robbins

Advanced Visual Interfaces 2006, pp. 428-435

Exploring the Design Space for Adaptive Graphical User Interfaces

Krzysztof Gajos, Mary Czerwinski, Desney S Tan, Daniel Weld

Advanced Visual Interfaces 2006, pp. 201-208

Clipping Lists and Change Borders: Improving Multitasking Efficiency with Peripheral Information Design

Tara Matthews, Mary Czerwinski, George Robertson, Desney S Tan

CHI 2006 Conference on Human Factors in Computing Systems, pp. 989-998

Large Display Research Overview

Mary Czerwinski, George Robertson, Brian Meyers, Greg Smith, Daniel Robbins, Desney S Tan

Extended Abstracts at CHI 2006 Conference on Human Factors in Computing Systems, pp. 69-74, 2006

Before 2006

Large Display User Experience

George Robertson, Mary Czerwinski, Patrick Baudisch, Brian Meyers, Dan Robbins, Greg Smith, Desney S Tan

IEEE Computer Graphics and Applications, 25(4), pp. 44-51, 2005

Panoramic Viewfinder: Providing a Real-time Preview to Help Users Avoid Flaws in Panoramic Pictures

Patrick Baudisch, Desney S Tan, Drew Steedly, Eric Rudolph, Matt Uyttendaele, Chris Pal, Richard Szeliski

OZCHI 2005 Conference for the Computer-Human Interaction Special Interest Group of the Human Factors Society of Australia

Understanding Research Trends in Conferences using PaperLens

Bongshin Lee, Mary Czerwinski, George Robertson, Benjamin B Bederson

Extended Abstracts of CHI, pp. 1969-1972, 2005

Understanding Eight Years of InfoVis Conferences using PaperLens

Bongshin Lee, Mary Czerwinski, George Robertson, Benjamin B Bederson

Posters Compendium of InfoVis, pp. 53-54, 2004

Fishnet, a Fisheye Web Browser with Search Term Popouts: a Comparative Evaluation with Overview and Linear View

Patrick Baudisch, Bongshin Lee, Libby Hanna

Advanced Visual Interfaces 2004, pp. 133-140

Display space usage and window management operation comparisons between single monitor and multiple monitor users

Dugald Hutchings, Greg Smith, Brian Meyers, Mary Czerwinski, George Robertson

Advanced Visual Interfaces 2004, pp. 32-39

Scalable Fabric: Flexible task management

George Robertson, Eric Horvitz, Mary Czerwinski, Dugald Hutchings, Patrick Baudisch, Daniel Robbins, Greg Smith

Advanced Visual Interfaces 2004, pp. 85-89

GroupBar: The TaskBar Evolved

Greg Smith, Patrick Baudisch, George Robertson, Mary Czerwinski, Brian Meyers, Daniel Robbins, Donna Andrews

(2003) OZCHI 2003 Conference for the Computer-Human Interaction Special Interest Group of the Human Factors Society of Australia

Toward characterizing the productivity benefits of very large displays

Mary Czerwinski, Greg Smith, Tim Regan, Brian Meyers, George Robertson, Gary Starkweather

(2003) Interact 2003

Press

2013

New tech allows computers to sense gestures without camera

KOMO News, 3 Jan 2013

2012

Control your laptop with a wave of your hand

CNN Money, 7 August 2012

Next Up in Kinect-Style Motion Sensing: Ultrasound?

Popular Mechanics, 25 May 2012

Microsoft Turns Jazz Hands Into Gesture Commands Using Sound Waves

IT World, 10 May 2012

Microsoft Research Projects Offer New Takes on Gesture Sensing

IDG News Service, 9 May 2012

Beyond Kinect: Gestural Computer Spells Keyboard Death

New Scientist, 9 May 2012

Gesture Sensing Alternatives Use Radio Interference, Doppler Effect

PC World, 9 May 2012

Laptop Uses Sound for Gesture Control

Discovery News, 9 May 2012

Cool Microsoft Research Takes Kinect To Another Level

PC Magazine, 7 May 2012

Gesture Control System Uses Sound Alone

Technology Review, 7 May 2012

Here’s Looking at You (but I’m Still Texting)

New York Times, 11 Feb 2012

Stealth Texting

Technology Review, 1 Jan 2012

10 Tech Research Projects to Watch (featuring PocketTouch)

PC World, 3 Jan 2012

CES 2012: Printing dress, virtual dress-up and Art of Touch at Microsoft

Seattle Times, 12 Jan 2012

2011

The Knee Bone's Connected To The Smartphone

Fast Company, 17 May 2011

A Virtual View Beneath the Skin

Technology Review, 17 May 2011

How to Make a Human Antenna (at ABC News) (at MSNBC)

Discovery News, 12 May 2011

Turn your entire home into a game controller

New Scientist, 10 May 2011

Talking to the Wall

Technology Review, 3 May 2011

TechFest: A thermostat that knows when you’re home

Seattle Times, 9 March 2011

2010

Finding Our Way with Digital Bread Crumbs

Technology Review, 18 August 2010

Kids experiment with ‘video playdates’

CNN Tech, 11 June 2010

Microsoft’s Skinput turns hands, arms into buttons

CNN, 19 April 2010

Skinput Makes the Entire Body a Touch Interface

PC World, 13 April 2010

Sensors turn skin into gadget control pad

BBC News, 26 March 2010

Body acoustics can turn your arm into a touchscreen

New Scientist, 1 March 2010

‘Skinput’ Turns Your Body Into Touchscreen Interface

TechNews, 3 March 2010

Skinput Turns Any Bodily Surface Into a Touch Interface

Popular Science (popsci.com), 3 March 2010

Skinput Turns Your Arm into a Touch-Screen

Wired, 3 March 2010

The Intersection of Work & Life

850 Business Magazine, 7 January 2010

2009

Muscle-Based PC Interface Lets You Literally Point and Click, No Mouse Required

Popular Science (popsci.com), 29 October 2009

Muscle-Bound Computer Interface

MIT Technology Review, 28 October 2009

The Revolutionaries: Radical Thinkers and their World-Changing Ideas - Desney Tan

Forbes, 22 September 2009

Technology Shows What’s on a Customer’s Mind

Financial Times, 16 September 2009

The Quest for a Better Keyboard

Forbes Magazine, September 2009

Top Ten Amazing Research Projects (featuring Microsoft Research Songsmith)

PC Authority, June 2009

Password reminders: hard to remember, but easy to hack

Ars Technica, 19 May 2009

Study: Secret Questions Don’t Safeguard Passwords

PC World Business Center, 19 May 2009

Are Your “Secret Questions” Too Easily Answered?

Technology Review, 18 May 2009

The Songsmith Flap

EQ Magazine, April 2009

A Review of Microsoft Research Songsmith

Sound on Sound, April 2009

Inside These Lenses, a Digital Dimension

The New York Times, 25 April 2009

Microsoft tests custom hardware controls on Surface-style device

Techflash, April 2009

Mixing Real and Virtual Controls

Technology Review, April 2009

A Look at Microsoft Songsmith

Ars Technica, January 2009

Microsoft Songsmith Backs Up Your Vocals

PC Magazine, January 2009

Microsoft’s Songsmith Gives Singers an Algorithmic Band

TechFlash, January 2009

Microsoft’s Songsmith Makes Music Using PC

Seattle Post-Intelligencer, January 2009

Microsoft Releases Songsmith: Karaoke in Reverse

CNET, January 2009

2008

Make Beautiful Music With MySong

Computer Power User, September 2008

The Interface: Gestures will Force the Mouse into Retirement

Financial Times, 16 September 2008

Microsoft Research: How We Watch the Computer and How it Watches Us

Channel 10: HealthBlog, 18 August 2008

Collabio Game Explores Social-Network Data Mining... And Social Psychology

Microsoft Research News & Highlights, August 2008

Microsoft Research: How we watch the computer, how it watches us

Bill Crounse’s HealthBlog, August 2008

My Awesome IT Job: Researcher and Manager, Microsoft

ZDNet, 25 July 2008

An Algorithm to Turn Melodies Into Songs

IEEE Spectrum, July 2008

Upgrading Humans

Discovery Channel, 16 June 2008

High-tech Armband Puts your Fingers in Control

The New Scientist, 24 April 2008

Microsoft creates ‘instant backing band’ for singers

The New Scientist, April 2008

Giving Back to the Machines

digit, 22 January 2007

Microsoft’s Investigation into the Subconscious

MarketWatch from DowJones, 18 January 2008

2007

The Year in Hardware

MIT Technology Review, 26 December 2007

I Spy with my AI, Something Beginning with T(hought)!

Information Technology Magazine, 1 October 2007

Work on Paralysis Shows Just How Hard it is to Read Minds

Wall Street Journal, 26 September 2007

Teaching Computers to Read Minds

MIT Technology Review, 15 August 2007

Human-Aided Computing: Microsoft Researchers are Trying to Harness Untapped Brain Power

MIT Technology Review, 22 June 2007

2006

Computer Will Try to Adjust to Your Moods: This is your Brain Plugged In

Puget Sound Business Journal, 19 May 2006

Software Notebook: Helping Users Peer into the Minds of Users

Seattle Post-Intelligencer, 13 March 2006

Contact Us Terms of Use Trademarks Privacy Statement ©2010 Microsoft Corporation. All rights reserved.Microsoft