Rick Rashid, Microsoft Chief Research Officer, presents his vision for the future of computing research and the resources Microsoft Research can bring to bear on the evolution of computing.

This project explores ways for people to experience search that are complementary to fast, relevant search in response to queries. In particular, these concepts focus on new ways to spend time, rather than save time on the Web.

High-Fidelity Facial-Animation Capturing presents a new approach for acquiring high-fidelity, 3-D facial performances with realistic dynamic wrinkles and finely scaled facial details. This approach leverages state-of-the-art motion-capture technology and advanced 3-D scanning technology for facial-performance acquisition.

Beamatron is a new, augmented-reality concept that combines a projector and a Kinect camera on a pan-tilt moving head. The moving head is used to place the projected image almost anywhere in a room. Meanwhile, the depth camera enables the correct warping of the displayed image for the shape of the projection surface and for the projected graphics to react in physically appropriate ways. For example, a projected virtual car can be driven on the floor of the room but will bump into obstacles or run over ramps. As another application, we consider the ability to bring notifications and other graphics to the attention of the user by automatically placing the graphics within the user’s view.

A huge amount of climate data is available, covering the whole of the Earth surface. But even the experts find it ludicrously difficult to get the climate information they need: locate data sets, negotiate permissions, download huge files, make sense of file formats, get to grips with yet another library, filter, interpolate, regrid, etc! Enter FetchClimate, a fast, intelligent climate-data-retrieval service that operates over Windows Azure. FetchClimate can be used through a Silverlight web interface or from inside any .NET program. FetchClimate works at any grid resolution from global to a few kilometers, in a range of years from 1900 to 2010, on days within a year, and for hours within a day. When multiple data sources could answer your query, FetchClimate automatically selects the most appropriate, returning the requested values along with the level of uncertainty and the origin of the data. The entire query can be shared as a single URL, enabling others to retrieve the identical information.

This project presents next-generation webcam hardware and software prototypes. The new prototype webcam has an extremely wider view angle than traditional webcams and can capture stereo movie and high-accuracy depth images simultaneously. Users can chat with stereoscopic video. Accurate depth-image processing can support not only all Kinect scenarios on a PC, but also a gesture-control user interface without a touch screen. Besides computer vision, the webcam includes a hardware accelerator and a new image-sensor design. The cost of the design is similar to that of current webcams, and the webcam potentially could be miniaturized as a mobile camera. The project showcases new user scenarios in playing games with this webcam.

Presented at Microsoft TechFest 2012
This project presents a glassless 3-D display with a correct camera pose and view pose for a live view-dependent 3-D Window Telepresence experience.

A freely available, open source Windows application that lets almost anyone use their computer to do stuff automatically across all their devices. Just select an event (e.g. your favorite stock hit a trigger value, a change in the weather, say something to your Kinect, etc.) and then select a reaction (e.g. advance a PowerPoint slide, turn on a lamp, start playing a movie, etc.), and within seconds, you have a connection running.

Wearable Multitouch Interaction is a depth-sensing and projection system that enables interactive multitouch applications on everyday surfaces. Beyond a shoulder-worn system, there is no instrumentation of the user or the environment. Foremost, on such surfaces—without calibration—Wearable Multitouch Interaction provides capabilities similar to those of a mouse or a touchscreen: X and Y locations in 2-D interfaces and whether fingers are “clicked” or hovering, enabling a wide variety of interactions. Reliable operation on the hands, for example, requires buttons to be 2.3 centimeters in diameter. Thus, it is now conceivable that anything one can do on today’s mobile devices can be done in the palm of a hand.

Modern touch devices allow one to interact with virtual objects. However, there is a substantial delay between when a finger moves and the display responds. Microsoft researchers, Albert Ng and Paul Dietz, have built a laboratory test system that allows us to experience the impact of different latencies on the user experience. The results help us to understand how far we still have to go in improving touch performance.

A still photograph is a limited format for capturing moments that span an interval of time. Video is the traditional method for recording durations of time, but the subjective “moment” that one desires to capture is often lost in the chaos of shaky camerawork, irrelevant background clutter, and noise that dominates most casually recorded video clips. This work provides a creative lens used to focus on important aspects of a moment by performing spatiotemporal compositing and editing on video-clip input. This is an interactive app that uses semi-automated methods to give users the power to create “cliplets”—a type of imagery that sits between stills and video from handheld videos.

Presented at TechFest 2012, Microsoft Translator Hub implements a self-service model for building a highly customized automatic translation service between any two languages. Microsoft Translator Hub empowers language communities, service providers and corporations to create automatic translation systems, allowing speakers of one language to share and access knowledge with speakers of any other language. By enabling translation to languages that aren’t supported by today’s mainstream translation engines, this also keeps less widely spoken languages vibrant and in use for future generations. This Azure based service allows users to upload language data for custom training, and then build and deploy custom translation models. These machine translation services are accessible using the Microsoft Translator APIs or a Webpage widget.

Presented at TechFest 2012
Seeing Displays: Uses flat lenses (wedge) to see through a semi-transparent OLED for novel above screen gesture and scanning interactions.

Presented at TechFest 2012
Behind the Screen Overlay Interactions: Behind-the-screen interaction with a transparent OLED with view-dependent, depth-corrected gaze.

Holoflector is a unique, interactive augmented-reality mirror. Graphics are superimposed correctly on your own reflection to enable a novel augmented-reality experience. Presented at Microsoft Research’s TechFest 2012, Holoflector leverages the combined abilities of Kinect and Windows Phone to infer the position of your phone and render graphics that seem to hover above it.

Seen at TechFest 2012, IllumiShare enables remote people to share any physical or digital object on any surface. It is a low-cost, peripheral device that looks like a desk lamp, and just like a lamp lights up a surface at which it is pointed, IllumiShare shares a surface. To do this, IllumiShare uses a camera-projector pair where the camera captures video of the local workspace and sends it to the remote space and the projector projects video of the remote workspace onto the local space. With IllumiShare, people can sketch together using real ink and paper, remote meeting attendees can interact with conference room whiteboards, and children can have remote play dates in which they play with real toys.

Steven Bathiche, Director, Microsoft Applied Sciences, shares his team's latest work on the next generation of Smart Interactive Displays.