The Adaptive Systems and Interaction group pursues advances in principles of intelligence and interaction and applications of these advances to enhance computational systems and interfaces. Our team includes groups exploring foundations of sensing, learning, and decision making, search & retrieval, and human-computer interaction.
The Adaptive Systems & Interaction group (ASI) pursues research on automated reasoning, adaptation, and human-computer interaction. Interests of the group include principles and applications of decision making and learning, computation in the face of complexity, techniques for information management and search, and the development and evaluation of innovative designs for visualization and interaction.
Research goals include both the pursuit of basic science and the development of computing and communications applications that demonstrate new functionalities and flexibility. ASI is at the center of user modeling at Microsoft Research, focused on inferring the goals and needs of users from multiple sources of information about activity and interests.
The group is also home to research on information retrieval and management, including work in automated text classification and clustering. The ASI team continually attempts to identify new means for enriching the user experience through advances in automated reasoning and user interface design.
Areas of Focus
User Modeling and Intelligent User Interfaces. We pursue methods for enhancing human-computer interaction via architectures that mesh models of a users' context-sensitive interests, needs, and goals with expressive event systems that sense activity and related information. A focus in this arena is the development of Bayesian and decision-theoretic user models. Representative projects include Lumiere, the technology behind the user-requested services provided by the Office Assistant in MS Office, Lookout, an exploration of mixed-initiative interaction, and efforts within the Attentional User Interface (AUI) project, centering on principles and architectures for learning and reasoning about the attention- and context-sensitive costs and benefits of information and services. AUI projects include models, theory, and systems explored in Priorities and Notification Platform work, and a series of psychological studies of disruption.
Information Access, Filtering, and Management. We are pursuing principles and applications of technologies that allow information retrieval, filtering, and management. In this realm, we have refined collaborative filtering(CF) algorithms and methods for recommending content or services to a user based on the analysis of the behavior of a large number of users. Check the top of your page while browsing through the MS Research site to see an example of a CF page recommender we put together for this site. Other projects incolude enhanced search, text classification and clustering, and background query services providing related information by watching what a user is reviewing or writing. Other work has explored the use of classification technologies to filter information and to organize and present the results of searches, the use of Bayesian models infer the goals and actions of a user during search, and the development of systems for answering queries. Such work includes the Bayesian text analysis methods first shipped as Answer Wizard in Office '95, and the later integrated with the Office Assistant, appearing in Office '97 and Office 2000.
HCI Design and Visualization. Work on the ASI team includes efforts to develop new approaches and design metaphors for human-computer interaction, including 3D user interfaces and advances in the display of complex data and information. We are looking at new techniques for navigating and making use of large information spaces. We are also investigating opportunities and challenges with replacing the current 2D desktop with a 3D environment. Other work pursues strategies for enhancing memory and minimizing the costs of disruptions. Research efforts include the Task Gallery, a 3D desktop metaphor informed by intuitions about the way our own minds work to identify and track objects in physical space, and the Data Mountain, an alternative to IE Favorites or the Explorer. Recent work includes glanceable visualizations of multiple notifications, otherwise known as The Scope. One of our current projects is ZoneZoom, a technique for rapidly navigating large information spaces on Smartphones. Stuff I've Seen (SIS) is a prototype tool that makes it easy for you to find information you've seen before, whether it came as email, attachments, files, web pages, appointments, tablet journal entries, etc.
Psychological Studies. ASI is the center of user studies at Microsoft Research. The team hosts a state-of-the-art psychological studies laboratory. Topics addressed with empirical investigation of human subjects include perception, attention, search, and personality. Beyond the investigation of principles, user studies probe attributes of HCI designs for legacy and novel interfaces. A central area of interest focuses on the value of alternative designs for information visualization and navigation.
Conversational Systems. We are pursuing the long-term dream of fluid conversation between people and computers. The Conversational Architectures and the Persona projects center on exploration of principles, architectures, and prototypes for supporting such conversational dialog.
Novel Input Devices and Sensors.We are pursuing the development of new input devices and sensors that promise to provide users with new abilities to directly manipulate objects and visualizations, and to support our user modeling work with additional evidence about context. Ongoing work in this area includes extending mouse and keyboard with touch sensors, enabling systems to sense contact from the user's hands.
Computation, Systems, and Networks. We are exploring the use of flexible and proactive computational methods and decision theory to identify bottlenecks, to harness available resources effectively, and to optimize the functionality of operating systems and applications. Sample work in this arena includes efforts to prefetch and presend information based on probabilistic user models and decision-theoretic analysis of the value of alternative actions. Other areas of interest include the relationships between computational complexity and perception, exploring principles and methods for exploiting perception in computational approximations.
Diagnostics, Troubleshooting, and Sensor Fusion.We have developed and applied diagnostic reasoning methods to a range of problems, extending from software debugging to troubleshooting software and hardware systems. In our collaboration with Microsoft Technical Support, we have developed decision-theoretic troubleshooters that are available via the worldwide web. Visit Microsoft Technical Support Troubleshooters to access several decision-theoretic troubleshooters that have been deployed in an operational setting. Another diagnostics project, named Aladdin, has explored the application of decision-theoretic case-based reasoning to troubleshooting and customer support, the result of a collaboration between DTAS and Microsoft Technical Support. Our Windows-based application for Bayesian belief network construction and inference tool called Microsoft Belief Networks (MSBN), is available free for non-commercial purposes.
Reasoning, diagnosis, information retrieval
- Special Issue on Bayesian Networks: Communications of the ACM, March, 1995, vol 38, no. 3.
- J. Breese, D. Heckerman., C. Kadie, Empirical Analysis of Predictive Algorithms for Collaborative Filtering Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, Madison, WI, July, 1998.
- J. Breese, D. Heckerman. Topics in Decision-Theoretic Troubleshooting: Repair and Experiment Technical Report MSR-TR-96-06, Microsoft Research, March, 1996.
- S. T. Dumais, J. Platt, D. Heckerman and M. Sahami (1998). Inductive learning algorithms and representations for text categorization. (Word file) In Proceedings of ACM-CIKM98, Nov. 1998.
- E. Horvitz, A. Jacobs, D. Hovel. Attention-Sensitive Alerting, Proceedings of UAI '99, Conference on Uncertainty and Artificial Intelligence, July 1999, pp. 305-313.
- E. Horvitz, J. Breese, D. Heckerman, D. Hovel, and K. Rommelse. The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, July 1998.
- E. Horvitz. Continual Computation Policies for Utility-Directed Prefetching. Proceedings of the Seventh ACM Conference on Information and Knowledge Management, November 1998.
- T. Lau and E. Horvitz, Patterns of Search: Analyzing and Modeling Web Query Refinement. Proceedings of the Seventh International Conference on User Modeling, Banff, Canada, June 1999. New York: Springer Wien, 119-128.
HCI Design and Studies
- Czerwinski, M., Dumais, S.T., Robertson, G., Dziadosz, S., Tiernan, S. & van Dantzich, Visualizing Implicit Queries for Information Management and Retrieval, , M. ACM CHI'99 Conference on Human Factors in Computing Systems.
- E. Horvitz. Principles of Mixed-Initiative User Interfaces. Proceedings of CHI '99, ACM SIGCHI Conference on Human Factors in Computing Systems, Pittsburgh, PA, May 1999.
- Larson, K. & Czerwinski, M.P. Web Page Design: Implications of Memory, Structure and Scent for Information Retrieval. In Proceedings of CHI 98, Human Factors in Computing Systems (LA, April 21-23), ACM press, 25-32.
- Risden, K., Hoffman, H., Czerwinski, M.P., et al., Interactive advertising: Patterns of use and effectiveness. In Proceedings of CHI 98, Human Factors in Computing Systems (LA, April 21-23, 1998), ACM press, 219-224.
- Robertson, G. , Czerwinski, M., Larson, K., Robbins, D., Thiel, D. & van Dantzich, M., Data Mountain: Using Spatial Memory for Document Management, In Proceedings of UIST '98, 11th Annual Symposium on User Interface Software and Technology, pp. 153-162.
- Robertson, G., van Dantzich, M., Robbins, D., Czerwinski, M., Hinckley, K., Risden, K., Gorokhovsky, V., Thiel, D., The Task Gallery: A 3D Window Manager, to appear in CHI'2000.
- Balakrishnan, R., Hinckley, K., Symmetric Bimanual Interaction, to appear in CHI'2000.
- Hinckley, K., Sinclair, M., Touch-Sensing Input Devices, ACM CHI'99 Conference on Human Factors in Computing Systems, pp. 223-230.
- Hinckley, K., Sinclair, M., Hanson, E., Szeliski, R., Conway, M., The VideoMouse: A Camera-Based Multi-Degree-of-Freedom Input Device, ACM UIST'99 Symposium on User Interface Software & Technology, pp. 103-112.
The Adaptive Systems & Interaction group at Microsoft Research has created a Windows application for Bayesian belief network construction and inference tool called Microsoft Belief Networks, or MSBN. This software is free for non-commercial purposes.
Bayesian Network Interchange Format
Bayesian Network Interchange FormatA standardization effort to define an interchangable text file format for belief networks and influence diagrams.
ASI Research Communities
ASI researchers are active in several research communities including Uncertainty and Artificial Intelligence (UAI), CHI, User Modeling, FOCS, STOC, Institute for Operations Research and the Management Sciences (INFORMS), American Association for Artificial Intelligence (AAAI), International Society for Bayesian Analysis (ISBA), InfoVis, Knowledge Discovery and Datamining (KDD), and American Statistics Association (ASA). Here are some relevant links to these communities:
- Uncertainty and Artificial Intelligence (UAI)
- American Association for Artificial Intelligence (AAAI)
- Decision Analysis Society of INFORMS
- ACM SIGIR: Special Interest Group on Information Retrieval
- ACM SIGCHI: Special Interest Group on Computer-Human Interaction
- IEEE InfoVis
- User Modeling
- Institute for Operations Research and the Management Sciences (INFORMS)
- International Society for Bayesian Analysis (ISBA)
- Data Mining and Knowledge Discovery in Databases (KDD)
- HFES: Human Factors and Ergonomics Society
- ACM SIGKDD: Special Interest Group on Knowledge Discovery and Data Mining
- American Statistics Association (ASA)
- ASA Section on Bayesian Statistical Sciences
- Society for Medical Decision Making (SMDM)
ASI in the Press
- Intelligent agent technology staging a comeback (CNET News, October 1999).
- Designer envisions Windows in 3-D (Houston Chronicle, November 1999)
- "Artificial Intelligence Gets Real" (ZDnet, August 1997)
- "Is AI Going Mainstream at Last: A Look Inside Microsoft Research" (IEEE Intelligent Systems, March-April 1998, .pdf file).
- "Microsoft Sees Software Agent as Way to Avoid Distractions" (New York Times, July 2000)
- "From the Ether" (InfoWorld, September 2000)
- "Digital valets might happen in 2001" (USA Today, Jan 2001)
- "Microsoft is Calling You" (Red Herring, June 2001)
- "The Next Computer Interface" (MIT Technology Review, December 2001)
- "Just Beyond Our Windows" (LA Times, January 2002)
- "Continual Computation" (ComputerWorld, January 2002)