Imagining What Comes Next

Microsoft Researchers are innovators whose jobs are never finished: to continue to advance the state of the art in computer science. With the freedom to pursue the subjects they are passionate about, they look for gaps in current technologies and dedicate themselves to filling them. They envision what comes next, without worrying about product-delivery deadlines. When their ideas lead to real-world applications, the researchers work with a team of technology-transfer agents or directly with product teams to weave them into Microsoft software and services. Occasionally, researchers involved with a project transfer to a product-development group where their initial ideas will take shape. Nearly every Microsoft product in the marketplace today has been influenced by this approach, including the Windows Server® System, SQL Server®, the Xbox® gaming system and Kinect game controller, MSN®, Bing™, and Windows Media®. The projects described in this section represent just a fraction of the many ways that Microsoft researchers are advancing the state of the art in computing.

Making Interactions with Computers Easier and More Natural

The Sensors and Devices Group is exploring how new forms of hardware can be combined with innovative software to enrich the user experience when interacting with computer systems of all types. The group’s projects include SenseCam, a badge-like wearable device that automatically captures images and sensor data as people move through their day, creating a “visual blog” that can then be played back at high speed to share particular experiences with friends or as a way to reflect upon a day’s activities.

As the devices that surround us become more powerful and connected — from mobile phones to in-car computers — there is greater demand for user interfaces that transcend the keyboard and the mouse. Microsoft Research has made significant advances in speech and handwriting recognition that help us interact with today’s PCs and mobile devices as we interact with each other. But the work doesn’t end there.

“Solving today’s toughest problems takes more than a good idea. It also requires sharing ideas and thinking about solutions by interacting with people at all levels to understand the problems they have and how we can solve those problems with technology.”

— Susan Dumais, Principal Researcher, Adaptive Systems and Interaction Group

The Commute UX project aims to make interacting with in-car computers easier by moving beyond today’s complex menu structures and limited voice commands to adopting a “say anything, anytime” approach to handling user input. The result is a system that is safer and more responsive in situations when drivers’ hands and eyes should be focused on the road.

The Virtual Receptionist is a “situated conversational agent” that can act as a front-desk receptionist, with a lifelike avatar interacting with visitors, giving directions, or welcoming and registering guests. This project incorporates key advances that can be applied to many other areas of computing, such as the ability to notice, recognize, and interact with multiple individuals, to understand the user’s intentions and offer helpful information, or to sense when users are frustrated or have been waiting too long.

To make teleconferencing a more realistic, comfortable experience, Microsoft researchers are developing technologies that can identify different speakers and “place” voices to make it appear they are in different parts of a room. By exploiting humans’ ability to hear voices as positioned in space, the technology makes remote communication feel natural — a significant improvement to technology with the potential to enhance collaboration and reduce the need for business travel.

Since its 2010 debut, the Kinect for Xbox 360 has been wildly successful. Advances in skeletal tracking, facial-recognition and audio technology have come together to create the fastest selling consumer electronic device to date according to Guinness World Records. The software development kit (SDK) released has offered new opportunities across widely different industries, from health care and education, and to potentially new manufacturing techniques using the advanced modeling features.

Improving Software Quality and Reliability

As computing makes its way into more aspects of our lives — and spreads across more devices and the Internet cloud — maintaining security, performance, quality, manageability, and reliability becomes an increasingly important requirement. Microsoft Research is inventing new programming tools, methodologies, and techniques to help developers build software that better meets these needs.

To help administrators quickly identify and resolve the myriad problems that can occur in today’s data centers — which can include tens of thousands of individual servers that could fail for any number of reasons — Microsoft researchers are using advanced visualization and machine-learning techniques to predict and anticipate problems well in advance. Additionally, the Microsoft Research Extreme Computing Group (XCG) is looking to reinvent hardware and software technologies for the future of Cloud Computing. This work will greatly improve energy efficiencies, quality control processes, plus datacenter startup and operating costs.

The Rigorous Software Engineering team is working to improve the quality of software tools by combining traditional program analysis with statistical methods. The approach is able to identify security vulnerabilities, generate specifications for APIs, point to potential root causes of bugs, and find relevant information from bug repositories and version control during debugging. In initial testing, one tool built using the approach has demonstrated an ability to identify more potentialvulnerabilities than existing tools and processes.

Other Microsoft Research projects advance design, development, debugging, develop formal techniques and models to better understand programs, programming abstractions, and languages; and improve the methods, notation, and tool support for high-level system design and analysis. A number of Microsoft researchers are conducting security-related work that includes researching new encoding methods and applications to enhance privacy and security, collaborating with standards bodies to develop security protocols, and providing internal security consulting on Microsoft products.

Improving How Systems Store, Retrieve, and Present Information

Our lives are more data-driven than ever, yet computing hasn’t achieved its full potential to help us manage, protect, visualize, and understand that data. For businesses, it is important to identify quickly the information that matters and make the right decisions in real time. Individuals want to manage their photos, contacts, and communications easily and intuitively. Microsoft researchers are exploring ways to design systems, architectures, and components that can help computers manage and present data in more useful ways, while also preparing us for an era when everything will be digital.

“Unlike other industrial research labs, Microsoft Research conducts a lot of external research programs. I wished to keep publishing papers, going to conferences to meet with people, and visiting universities. Microsoft Research Asia allows all those things, and that provides me with an excellent research environment.”

— Yasuyuki Matsushita, Lead Researcher, Microsoft Research Asia

The Web Search & Data Mining Group is dedicated to improving how people locate, analyze, organize, retrieve, and visualize information. Although most Web-search methods simply rank and retrieve data from documents, this group uses methods that delve much deeper, searching at the object level to transform that data into knowledge and useful insights.

The Socio-Digital Systems Group combines psychology, sociology, computer science, and hardware engineering to design technology capable of supporting people throughout their lives — such as by building digital “timelines” to capture memories and experiences, extending the reach of today’s social networks to people who can’t or don’t use a PC and helping people create rich digitalmedia collections.

OCRLess is technology that improves the reach and quality of multilingual search by converting user queries into images and then matching them against scanned documents. By eliminating the scanning and recognition of source documents, this technology increases the volume of searchable information in many different languages, including Arabic, Chinese, Hebrew, and even hieroglyphics.

Exploring Tomorrow’s Problems

Thirty years ago — when mainframe computers cost millions of dollars and filled entire rooms — only a few dreamers imagined that computers several orders of magnitude more powerful would fit into our pockets. As technology continues to evolve — including the advent of multicore and many-core processors, and continued progress toward quantum computing — the computers we’ll use decades from now will be just as different. Microsoft researchers are engaged in some of the tough theoretical problems that need to be solved before we can take advantage of tomorrow’s technology and make it truly useful.

The Theory group at Microsoft Research is engaged in fundamental problems in mathematics and theoretical computer science, including probability theory, combinatorics, statistical physics, metric geometry, fractals, algorithms, and optimization — all of which have the potential to help computers work faster, solve problems, and make decisions.

The Algorithms Research Group is working in advanced areas such as streaming algorithms, directed data sampling, machine learning, approximation algorithms, and integration of numerical and combinatorial methods. Its aim is to be at the forefront of conceptual breakthroughs in the way computers can work with massive amounts of data.

Microsoft Research XCG's research spans several critical areas in computing research including cloud computing hardware and software, topological quantum computing, operating system and application software focused on heterogeneous many-core computers and all aspects of computer security from the mathematics of cryptographic primitives and protocols through system design supporting practical privacy, integrity and confidentiality in heterogeneous trust environments. A hallmark of the group is its ability to employ broad investigative techniques from the development of fundamental new theory through development of practical prototype systems that concretely demonstrate potential technology impact in Microsoft products.

Breakthroughs, Large and Small

Microsoft Research takes a long-term view on innovation. No matter how far-reaching their projects might be, Microsoft researchers consistently strive for results that will solve real-world technology challenges.