Adaptive Systems and Interaction

The Adaptive Systems and Interaction group pursues advances in principles of intelligence and interaction and applications of these advances to enhance computational systems and interfaces. Our team includes groups exploring foundations of sensing, learning, and decision making, search & retrieval, and human-computer interaction.


The Adaptive Systems & Interaction group (ASI) pursues research on automated reasoning, adaptation, and human-computer interaction. Interests of the group include principles and applications of decision making and learning, computation in the face of complexity, techniques for information management and search, and the development and evaluation of innovative designs for visualization and interaction.

Research goals include both the pursuit of basic science and the development of computing and communications applications that demonstrate new functionalities and flexibility. ASI is at the center of user modeling at Microsoft Research, focused on inferring the goals and needs of users from multiple sources of information about activity and interests.

The group is also home to research on information retrieval and management, including work in automated text classification and clustering. The ASI team continually attempts to identify new means for enriching the user experience through advances in automated reasoning and user interface design.

Areas of Focus

User Modeling and Intelligent User Interfaces. We pursue methods for enhancing human-computer interaction via architectures that mesh models of a users' context-sensitive interests, needs, and goals with expressive event systems that sense activity and related information. A focus in this arena is the development of Bayesian and decision-theoretic user models. Representative projects include Lumiere, the technology behind the user-requested services provided by the Office Assistant in MS Office, Lookout, an exploration of mixed-initiative interaction, and efforts within the Attentional User Interface (AUI) project, centering on principles and architectures for learning and reasoning about the attention- and context-sensitive costs and benefits of information and services. AUI projects include models, theory, and systems explored in Priorities and Notification Platform work, and a series of psychological studies of disruption.

Information Access, Filtering, and Management. We are pursuing principles and applications of technologies that allow information retrieval, filtering, and management. In this realm, we have refined collaborative filtering(CF) algorithms and methods for recommending content or services to a user based on the analysis of the behavior of a large number of users. Check the top of your page while browsing through the MS Research site to see an example of a CF page recommender we put together for this site. Other projects incolude enhanced search, text classification and clustering, and background query services providing related information by watching what a user is reviewing or writing. Other work has explored the use of classification technologies to filter information and to organize and present the results of searches, the use of Bayesian models infer the goals and actions of a user during search, and the development of systems for answering queries. Such work includes the Bayesian text analysis methods first shipped as Answer Wizard in Office '95, and the later integrated with the Office Assistant, appearing in Office '97 and Office 2000.

HCI Design and Visualization. Work on the ASI team includes efforts to develop new approaches and design metaphors for human-computer interaction, including 3D user interfaces and advances in the display of complex data and information. We are looking at new techniques for navigating and making use of large information spaces. We are also investigating opportunities and challenges with replacing the current 2D desktop with a 3D environment. Other work pursues strategies for enhancing memory and minimizing the costs of disruptions. Research efforts include the Task Gallery, a 3D desktop metaphor informed by intuitions about the way our own minds work to identify and track objects in physical space, and the Data Mountain, an alternative to IE Favorites or the Explorer. Recent work includes glanceable visualizations of multiple notifications, otherwise known as The Scope. One of our current projects is ZoneZoom, a technique for rapidly navigating large information spaces on Smartphones. Stuff I've Seen (SIS) is a prototype tool that makes it easy for you to find information you've seen before, whether it came as email, attachments, files, web pages, appointments, tablet journal entries, etc.

Psychological Studies. ASI is the center of user studies at Microsoft Research. The team hosts a state-of-the-art psychological studies laboratory. Topics addressed with empirical investigation of human subjects include perception, attention, search, and personality. Beyond the investigation of principles, user studies probe attributes of HCI designs for legacy and novel interfaces. A central area of interest focuses on the value of alternative designs for information visualization and navigation.

Conversational Systems. We are pursuing the long-term dream of fluid conversation between people and computers. The Conversational Architectures and the Persona projects center on exploration of principles, architectures, and prototypes for supporting such conversational dialog.

Novel Input Devices and Sensors.We are pursuing the development of new input devices and sensors that promise to provide users with new abilities to directly manipulate objects and visualizations, and to support our user modeling work with additional evidence about context. Ongoing work in this area includes extending mouse and keyboard with touch sensors, enabling systems to sense contact from the user's hands.

Computation, Systems, and Networks. We are exploring the use of flexible and proactive computational methods and decision theory to identify bottlenecks, to harness available resources effectively, and to optimize the functionality of operating systems and applications. Sample work in this arena includes efforts to prefetch and presend information based on probabilistic user models and decision-theoretic analysis of the value of alternative actions. Other areas of interest include the relationships between computational complexity and perception, exploring principles and methods for exploiting perception in computational approximations.

Diagnostics, Troubleshooting, and Sensor Fusion.We have developed and applied diagnostic reasoning methods to a range of problems, extending from software debugging to troubleshooting software and hardware systems. In our collaboration with Microsoft Technical Support, we have developed decision-theoretic troubleshooters that are available via the worldwide web. Visit Microsoft Technical Support Troubleshooters to access several decision-theoretic troubleshooters that have been deployed in an operational setting. Another diagnostics project, named Aladdin, has explored the application of decision-theoretic case-based reasoning to troubleshooting and customer support, the result of a collaboration between DTAS and Microsoft Technical Support. Our Windows-based application for Bayesian belief network construction and inference tool called Microsoft Belief Networks (MSBN), is available free for non-commercial purposes.

Selected Publications

Reasoning, diagnosis, information retrieval

HCI Design and Studies

Input Devices

Tools Available

The Adaptive Systems & Interaction group at Microsoft Research has created a Windows application for Bayesian belief network construction and inference tool called Microsoft Belief Networks, or MSBN. This software is free for non-commercial purposes.

Bayesian Network Interchange Format

Bayesian Network Interchange FormatA standardization effort to define an interchangable text file format for belief networks and influence diagrams.

ASI Research Communities

ASI researchers are active in several research communities including Uncertainty and Artificial Intelligence (UAI), CHI, User Modeling, FOCS, STOC, Institute for Operations Research and the Management Sciences (INFORMS), American Association for Artificial Intelligence (AAAI), International Society for Bayesian Analysis (ISBA), InfoVis, Knowledge Discovery and Datamining (KDD), and American Statistics Association (ASA). Here are some relevant links to these communities:

ASI in the Press