Fun physical interfaces for everyday tasks  

Enhanced window management capabilities in the Windows taskbar

Logging and visualization of user activity

A PA Music and Physiology-based Personal Trainer on your Smartphone

FacetMap

FacetMap is an interface for exploring large, metadata-rich data stores that moves beyond traditional text keyword search and list-based results. It combines information retrieval techniques with information visualization techniques to create a highly graphical tool capable of searching, browsing, and sense-making across a wide range of SQL-based data stores. FacetMap's recursive, space-filling display algorithm adapts gracefully to many different screen sizes, and it uses direct manipulation of graphical objects throughout the interface to allow the user to build complex queries with simple interactions. We're running user studies to explore the efficacy (and the limitations) of this more graphical approach to searching and browsing. A video describing the FacetMap project is available (here). More info...

FaThumb

FaThumb is a novel interface for exploring large datasets from a mobile phone. By exploiting the spatial mapping between the numeric keypad and onscreen graphical objects, FaThumb deemphasizes tedious keyword entry in favor of iterative data filtering on hierarchical metadata. Like FacetMap, users can build complex queries with simple manipulations to rapidly browse and search large datasets. Keyword text search is also integrated into the interface, but our lab studies have demonstrated that the "facet navigation" is preferred and more effective when the name of the search target is not known precisely in advance. More info...

Brain Computer Interfaces

Recent advances in cognitive neuroscience and brain imaging technologies provide us with the unprecedented ability to interface directly with activity in the brain. Beyond traditional neuroscience work, researchers have begun to explore brain imaging as a novel input mechanism. This work has largely been targeted at allowing users to bypass the need for motor movement and to directly control computers by explicitly manipulating their brain signals. We take a radically different approach. In our work, we use brain imaging to passively sense and model the user’s cognitive state as they perform their tasks. We will use brain imaging to explore human cognition in the real world, to evaluate interface design, and to build interfaces that adapt based on cognitive state. More info...

SWISH

SWISH (Semantic analysis of Window titles and Switching History) is an application of machine learning techniques to the analysis of an individual's computer usage in order to facilitate a better user experience on the desktop. We use a Probabilistic Latent Semantic Indexing (PLSI) analysis of window titles to generate possible window groupings based on keyword clusters. We combine this with a graph-based model of temporal window-switching behavior to create an aggregated real-time probability distribution of which windows in the system the user might consider to be part of the same "task", and therefore might want to treat as a single multi-window unit. We are currently working on refining the learning techniques and integrating the system into a larger infrastructure for longer term task tracking and multi-tasking support. More info...

HealthGear

HealthGear is a wearable, smart health monitoring system. It consists of a number of physiological sensors that wirelessly transmit their sensed data to a Smartphone via Bluetooth. HealthGear allows for any number of physiological signals to be constantly monitored in a non-intrusive manner. We are developing both hardware and software components to collect, store, analyze and visualize the users’ physiological data. In particular, we are interested in finding patterns of behavior in the physiology of the users when correlated with other external variables, such as workload, stress levels, personal interactions, traffic conditions, exercise, diet, sleep, etc. We believe that daily monitoring of physiological signals will be one of the enabling factors to a better understanding of our bodies and how they relate to the environment. Finally, non-intrusive, at-home, cheap health monitoring may be the key to the future of a sustainable health system. More info...

TeamTracks

We have developed prototype tools that can help developers come up to speed with unfamiliar code. The tools use new visualizations, with code instrumentation and model checking behind the scenes. This project is in collaboration with the Human Interactions in Programming group (HIP). More info...

VibeLog

VibeLog is a logging tool that allows us to research the ways that work practice might change as users move in between various sized displays throughout their work day. Once we understand work practice changes and issues from marrying our logging tool with ethnographic research data, we should have a good understanding of what parts of the designs of Windows and Office do not scale well across different display sizes. This fresh understanding, based on large amounts of log data, will justify where we should expend our research efforts in novel visualization and interaction, with an eye toward designing more elegant UIs.

Memex

The Memex project is dedicated to UI design, visualization, and user studies for the exploration of digital memories. Our early prototypes show advanced clustering and timeline visualizations, organization and annotation tools, search, and browsing. We show how detailed logging information can be used to filter and recommend information. The focus of the research is to develop automated tools for collecting new memories and activities, as well as useful ways to use, explore, and share the information. The store encompasses a wide range of datatypes, including documents, photos, email, sensor data, GPS tracks, and calendar events). Done in conjunction with the MyLifeBits project.

ZoneZoom

Traditional pan and zoom techniques pose a challenge when navigating large information spaces on smartphones and other small-form-factor mobile devices. Our most recent project is ZoneZoom.

• Divide view into nine segments
• Map each segment to one of the number keys on a cell phone
• Press number key: animate view to show selected segment
• Quickly release key (tapping): view stays on selected segment
• Releases key after holding down for more than a moment: view returns to the original view ("glance" at different areas)
• Redivide current by pressing on the pound # key

More info...
 

ClippingLists and ChangeBorders

Information workers often have to balance many tasks and interruptions. In this work, we explore peripheral display techniques that improve multitasking efficiency by helping users maintain task flow, know when to resume tasks, and more easily reacquire tasks. Specifically, we have compared two types of abstraction that provide different task information: semantic content extraction, which displays only the most relevant content in a window, and change detection, which signals when a change has occurred in a window. Results from our user study suggest that semantic content extraction improves multitasking performance more so than either change detection or our base case of scaling. Results also show that semantic content extraction provides significant benefits to task flow, resumption timing, and reacquisition.  More info...

Tumbler and Splatter

Accessing and manipulating occluded content in 2D drawings can be difficult. In this work, we characterize the design space of solutions that facilitate access to occluded content in 2D drawings. We have designed two new tools, Tumbler and Splatter, which represent unexplored areas of the proposed design space. By design Tumbler and Splatter can co-exist and complement each other, as well as more traditional methods such as layer palettes. We have conducted two usability studies that showed how Splatter outperforms existing solutions, such as Palette, both in selection and layering tasks. Participants from our studies commented almost unanimously that they could see themselves using different tools for different scenarios. More info...

SchemaMapper

A sophisticated visualization for mapping schemas from source to destination. Current approaches to visually defining such a mapping fail when the schemas or maps become large. The new approach uses various information visualization techniques to simplify the view, making it possible for users to effectively deal with much larger schemas and maps. A user study verifies that the new approach is useful, usable, and effective. The primary contribution is a demonstration of novel ways to effectively present highly complex information.

Snap-and-go

Snapping is a widely used technique that helps users position graphical objects precisely, e.g., to align them with a grid or other graphical objects. Unfortunately, whenever users want to position a dragged object close to such an aligned location, they first need to deactivate snapping. We propose snap-and-go, a snapping technique that overcomes this limitation. By merely stopping dragged objects at aligned positions, rather than “warping” them there, snap-and-go helps users align objects, yet still allows placing dragged objects anywhere else. While this approach of inserting additional motor space renders snap-and-go slightly slower than traditional snapping, snap-and-go simplifies the user interface by eliminating the need for a deactivation option and thereby allows introducing snapping to application scenarios where traditional snapping is inapplicable. In our user studies, participants were able to align objects up to 138% (1D) and 231% (2D) faster with snap-and-go than without and snap-and-go proved robust against the presence of distracting snap targets.

Summary Thumbnails

In order to display web pages originally designed for desktop-sized monitors, some small-screen web browsers provide single-column or thumbnail views. Both have limitations. Single-column views affect page layouts and require users to scroll significantly more. Thumbnail views tend to reduce contained text beyond readability, so differentiating similar looking areas requires users to zoom. Summary Thumbnails are thumbnail views enhanced with readable text fragments. Summary Thumbnails help users identify viewed material and distinguish between similar looking areas. In our user study, participants located content in web pages 41% faster and at a 71% lower error rate when using the Summary Thumbnail interface than when using the Single-Column interface, and zoomed 59% less than when using the Thumbnail interface. Nine of the eleven participants preferred Summary Thumbnails over both the Thumbnail and Single-Column interfaces.

TimeQuilt

For the large percentage of users who don't want to invest the effort of manually organizing their photo collections, only (1) the photo creation date and (2) the visual contents of the photo itself are available to support retrieval. Linear timeline layouts maximize use of the former; space-filling photo layouts (e.g., PhotoMesa) maximize the use of the latter. We propose time quilt, a layout designed to combine the benefits of both approaches. A time quilt layout is created by packing a timeline layout into a rectangular screen space using a "line break" algorithm. While still conveying temporal order, the time quilt layout makes reasonable use of screen space. In an experimental comparison of space-filling, timeline, and time quilt layouts, participants carried out the task of finding photos in their personal photo collections averaging 4,000 items. They performed 45% faster on time quilt. In addition, we have used the time quilt photo browser to investigate what we call "semantic zooming based on representative photos". When zoomed out, our browser collapses each cluster of thumbnails into a single large thumbnail. This allows users to get an overview of collections that are so large that normal thumbnails would become unreadably small when zoomed out.

Curvature Dial

This is a technique designed to extend gesture-based interactions like FlowMenus with eyes-free parameter entry. Flow-Menus, let users enter numerical parameters with “dialing” strokes surrounding the center of a radial menu. This centering requires users to keep their eyes on the Menu in order to align the pen with its center before initiating a gesture. Curvature dial instead tracks the curvature of the path created by the pen: since curvature is location-independent, curvature dialing does not require users to keep track of the menu center and is therefore eyes-free. We have used curvature dial to implement an example application that allows users to scroll through a document eyes-free.

Collapse-to-zoom

Overview visualizations for small-screen web browsers were designed to provide users with visual context and to allow them to rapidly zoom in on tiles of relevant content. Given that content in the overview is reduced, however, users are often unable to tell which tiles hold the relevant material, which can force them to adopt a time-consuming hunt-and-peck strategy. Collapse-to-zoom addresses this issue by offering an alternative exploration strategy. In addition to allowing users to zoom into relevant areas, collapse-to-zoom allows users to collapse areas deemed irrelevant, such as columns containing menus, archive material, or advertising. Collapsing content causes all remaining content to expand in size causing it to reveal more detail, which increases the user’s chance of identifying relevant content. Collapse-to-zoom navigation is based on a hybrid between a marquee selection tool and a marking menu, called marquee menu. It offers four commands for collapsing content areas at different granularities and to switch to a full-size reading view of what is left of the page.

PaperLens

PaperLens is a novel visualization that reveals trends, connections, and activity throughout a conference community.  It tightly couples views across papers, authors, and references.  PaperLens was developed to visualize 8 years (1995-2002) of InfoVis conference proceedings and was then extended to visualize 23 years (1982-2004) of the ACM SIGCHI conference proceedings. PaperLens has won first prize at the InfoVis 2004 Information Visualization contest.  The visualization contributes to the field by allowing users to discover research trends, patterns and relationships not possible with existing tools.  A video of the visualization tool for the Info Vis proceedings can be viewed here.

WinCuts

Each window on our computer desktop provides a view into some information. Although users can currently manipulate multiple windows, we assert that being able to spatially arrange smaller regions of these windows could help users perform certain tasks more efficiently. In this paper, we describe a novel interaction technique that allows users to replicate arbitrary regions of existing windows into independent windows called WinCuts.

Multiblending

Alpha blending allows the simultaneous display of overlapping windows—such as palette windows in visual workspaces. Although alpha blending has been used in some applications, such as games, it has not been widely adopted. One reason for the limited acceptance is that in many scenarios, alpha blending compromises the readability of content. We introduce a new blending mechanism called multiblending that uses a vector of blending weights, one for each class of features, rather than a single transparency value. Multiblending can in most cases be automatically optimized to preserve the most relevant features of both the palette and the background window. We present the results of a user study in which multiblended palettes provided higher recognizability of both the background and the palette than the best participating version of alpha blending.

Flat Volume Control

The hardware-inspired volume user interface model that is in use across all of today’s operating systems is the source of several usability issues. One of them is that restoring the volume of a muted application can require an inappropriately long troubleshooting process: in addition to manipulating the application’s volume and mute controls, users may also have to visit the system’s volume control panel to find and adjust additional controls there. The “flat” volume control model eliminates this and other problems by hiding the hardware-oriented volume model from the user. Using the flat model, users use one slider per application to indicate how loud they want the respective applications to play; the slider then adjusts all hardware volume variables necessary to obtain the requested output. This simplifies controlling the volume of—and unmuting—any application, as there now is a single point of control for each application, rather than an entire hierarchy of such points. In our studies, participants completed all four volume control and mixing tasks faster and with less error when using the flat model than when using the existing hardware-oriented volume control model. Participants also indicated a subjective preference for the flat model over the existing model. Demo

Fishnet

Fishnet is a web browser that always displays web pages in their entirety, independent of their size. Fishnet accomplishes this by using a fisheye view, i.e. by showing a focus region at readable scale while spatially compressing page content above and below that region. Fishnet offers search term highlighting, and assures that those terms are readable by using “popouts”. This allows users to visually scan search results within the entire page without scrolling. In our user study, fishnet outperformed a regular web browser that offered the same search term highlighting but no fisheye effect. Download (Microsoft internal only)

Mouse ether

When acquiring a target located on a different screen, multi-monitor users face a challenge: differences in resolution and vertical and horizontal offsets between screens cause the mouse pointer to get warped, making the attempt to acquire the target difficult. Mouse ether eliminates warping effects by applying appropriate transformations to all mouse move events. In our user study, mouse ether improved participants’ performance on a target acquisition task across two screens running at different resolutions by up to 28%. 7 of the 8 participants also strongly preferred using mouse ether to the control. Download (Microsoft internal only)

New User Interaction Models and Corresponding Visualizations 

Larger display surfaces expose the limitations of some of the most fundamental Windows interaction primitives, especially minimize/ maximize and move/resize. It is possible we can create new interaction mechanisms that extrapolate better to displays of arbitrary sizes and configurations, by giving more consideration to the mapping between a user's attention space and the desktop layout space. We are exploring more sophisticated mechanisms for window movement, window placement, window grouping, and task switching. We are also running studies with external users to refine and optimize our designs in this area.   Download (Microsoft internal)   Download (External)

High-Density Cursor 

As screen sizes increase, e.g. as multiple monitor configurations become more popular, users use higher mouse cursor speeds as well as stronger mouse acceleration in order to traverse the screen from side to side reasonably fast. The faster the mouse cursor moves, however, the more likely users are to lose track of it. One key reason is that the cursor is rendered only once per frame, which makes it visually jump from one rendering position to the next, with the distance increasing with the cursor's speed. high-density cursor addresses this issue by using a specific type of motion blur. By filling the space between the current cursor position and the previous one with additional fill-in cursor images, high-density cursor bridges the gaps between cursor positions, resulting in an effect similar to increasing the display frame rate. Since all cursor images exist only for a single frame, the proposed technique does not introduce any lag, which makes it different from similar-looking techniques, such as the MS Windows mouse trail. Demo, 

Drag-and-pop

Drag-and-pop is an interaction technique designed to accelerate drag-and-drop on large screens. By animating potential targets and bringing them to the dragged object, drag-and-pop reduces the user effort required for dragging an object across the screen to a desired target. To preserve users' spatial memory, targets are not moved away from their original location, but are instead stretched using a rubber band-like visualization. Demo