By Rob Knies
May 6, 2010 1:00 PM PT
One of the hallmarks of Microsoft Research is that it often gets to contribute the results of its projects directly to the company’s business units.
Over the years, just about every Microsoft product has benefited from technology transferred from Microsoft Research. It’s an advantage that has proved invaluable.
And for a model of how such efforts can pay handsome dividends, you need look no further than the relationship between Microsoft Research and Bing Maps, some of which were featured May 6 during the Silicon Valley TechFair.
The latter has become one of the most dynamic Microsoft products, regularly delivering new features and services that are defining the cutting edge of mapping technology, such as geotagged panoramas, astronomical imagery, and bird’s-eye-image stitching.
This, says Heather Warncke, is no accident.
“Our relationship with Bing Maps has grown as the service has grown,” says Warncke, who serves as principal research program manager for Microsoft Research. “They’ve always been a great partner for Research, because they immediately reached out to us, from the beginning. We’ve been a contributor to them all along.
“Over the last year, you’ve seen much more of an emphasis as we see mapping as a core competitive area. You’ve seen much more investment on the business side, and you’ve seen a corresponding investment on this side and a lot of good collaboration. When you can start to point to things—and we’re there now—that Microsoft Research has contributed to directly, that’s an important milestone.”
Warncke is uniquely situated to provide that perspective. One of her roles is to work with researchers to learn about their projects as they evolve and to connect them with her contacts on the product side—in this case, Microsoft’s Online Services Division—once the technologies are mature.
“The sweet spot for transfers,” she says, “is when researchers are working on things that also happen to be really high-value to the business units. Because Bing Maps is trying to do things that are very edgy, that push the envelope, there’s a natural relationship with the kinds of things people are doing in research. They’re hungry for exciting, new, flashy stuff that no one else is doing, and that’s stuff that frequently we can provide as a research organization, particularly when you have two groups that have worked together really closely like this.”
She’ll get no argument from Chris Pendleton, Bing Maps technical evangelist. His team has been on the receiving end of a number of Microsoft Research technology transfers recently, and he’s well aware of the value they convey.
“Leveraging the brain trust from Microsoft Research has allowed Bing Maps to extend into areas we’ve never thought possible,” Pendleton says. “We’re advancing mapping and data visualization to bring more scientific and technical features to our product, yet retaining the usability of the different applications.”
“The collaboration,” he says, “between our Interactive Visual Media Group and the Bing Maps team has been incredible.”
One of those collaborative efforts involved Destination Maps, a project that enables users to provide simplified maps to help friends and colleagues find a specific location within a region, for example, to attend a party or a business meeting. Johannes Kopf and Michael Cohen of IVM worked on that project.
“Bing Maps provides a unique opportunity for researchers to make real many ideas related to geographically located images and for enhancing maps for a variety of applications,” Cohen says. “The interactions have been collaborative and supportive, with developers and researchers working closely in sync.”
Having contributed a tech preview of Destination Maps to the Map Apps section of Bing Maps, Kopf and Cohen are now exploring new ways to enhance the Bing Maps experience. Other techniques already have been implemented:
Photosynth has received lots of media attention in recent years for its ability to mash together collections of photos, of spaces ranging from the Taj Mahal to your living room, into a 3-D scene that can be explored naturally and intuitively.
It’s an entrancing, exhilarating project, and, now, 3-D experiences enabled by Photosynth are available from Bing Maps, where users will find exploration environments from Siberia to South America, from Austria to Australia. You could—be careful, you might!—spend hours roaming the globe, or simply your own hometown.
IVM’s Image Composite Editor (ICE)—work from Matt Uyttendaele, principal research software-development lead for IVM, and his colleagues—enables the stitching together of overlapping images into high-resolution panoramas. Those, too, can now appear on Bing Maps.
“The latest release of ICE takes advantage of multiple cores to stitch faster than ever,” explains Eric Stollnitz, principal research software-development engineer. “It has a ‘structured panorama’ feature to handle hundreds of images taken by robotic panorama-capture devices. And, most important, it allows users to upload their stitched panoramas to the Photosynth Web site.
“Now users have a place to store, view, and share their panoramas. And panoramas, like synths, can be geotagged so they’ll show up on Bing Maps.”
As Szeliski notes, the addition of panoramic support to Bing Maps is the latest step in expanding the reach of the Photosynth experience.
“Photosynth had its origins in research done in our group, in collaboration with the University of Washington,” he says, “and we continue to deliver innovations such as better 3-D modeling techniques into this product.
“Getting our panoramic-stitching algorithms integrated into Photosynth is a great milestone, as it brings together the two best ways to experience and navigate immersive environments under one umbrella.”
Another amazing effort from Microsoft Research is the WorldWide Telescope, which gathers imagery from the world’s best ground- and space-based telescopes to enable your computer to function as a virtual telescope. Users can swoop through the sky, examine exotic galaxies, and experience narrated guided tours from astronomers and educators.
Now, users also can use WorldWide Telescope within the Bing Maps interface. Via integration with the Bing Maps Street View, the WorldWide Telescope Map App enables the viewing of the sky above a street view.
“In Bing Maps, you can spot your viewing location from the map view,” says Jonathan Fay, principal software architect for Microsoft Research’s External Research group and one of the driving forces behind the WorldWide Telescope. “You can fly in to the street side, look up, and see the night sky, blended with the buildings and landmarks around you. You can move the time control to see what the sky looked like at any time, to answer questions like ‘When will the moon come over the Space Needle?’ or ‘What was that I saw in the sky last night around 8:30?”
Fay particularly appreciates his collaboration with Mark Dawson and Gonzalo Ramos of the Bing Maps team.
“It was great Mark and Gonzalo were able to take our Web control and, with a minimum of change, integrate the code into their Bing Maps client as a Map App,” Fay says. “They then created a customized UI experience to blend the features of WorldWide Telescope into the Bing Maps street view.
“It was nice seeing what started as experimental code move into production. It was also great seeing how they respected the expertise I brought to the table while taking the experience in a new, fresh direction.”
Then there’s the Enhanced Bird’s Eye mode in Bing Maps, which uses techniques that enable seamless stitching of bird’s-eye-view images, work that developed out of a 2005-’06 Microsoft Research project by Drew Steedly.
“Bird’s-eye images are images that are captured at an oblique angle,” Steedly explains. “This means they are not captured looking straight down, but are tilted roughly at 45 degrees. They give you a very nice sense of what the city looks like, because you can see the sides of buildings instead of just the rooftops. Bird’s eye images have been some of the most popular things that users look at on our maps.”
The fact that the images are stitched together creates an immersive, lifelike experience.
“These start out as just a bunch of individual bird’s-eye images,” he continues. “What I worked on was stitching these together into a mosaic that covered the entire city. This lets you seamlessly pan around the city without having to jump from image to image. It also lets you zoom out to see the area covered by several images at once.”
The bird’s-eye work represents more than just technology transfer from Microsoft Research to Bing Maps. The researcher himself has been transferred. After working on this project with Microsoft Research and then moving to Live Labs to develop the tool that creates synths, Steedly joined the Bing Maps team in 2008, giving him a unique perspective on the nature of tech transfers.
“By being on both sides of the fence, both in Microsoft Research and in the product group, I’ve gotten a much better sense of what it takes to be successful at transferring technology,” he says. “It is important for both sides to really understand and communicate the constraints of the problem. From a product-group perspective, you have to be very crisp about what your problem actually is, what the inputs are, and what the desired output is. From a Microsoft Research perspective, you need to be equally crisp in describing your proposed solution—the assumptions you are making, the robustness of your algorithm, what its possible failure modes are, the computational cost and memory usage.
“Also, it is very important to try to come up with the simplest solution possible. There is an enormous, often underestimated, cost to complex solutions. Kitchen-sink-type solutions that rely on every possible input often end up with many parameters that have to be tuned. Tuning parameters incurs substantial test and development costs. And complex solutions have many more failure points and tend to be much more brittle than simpler ones.”
That’s not all. Microsoft Research has made other significant contributions to Bing Maps. Simon Winder, an IVM senior research software-development engineer, has provided feature detectors and descriptors that outperform existing state-of-the-art technology and are used broadly across Bing Maps, including Photosynth, aerial stitching, and street-side pose estimation.
And such collaborative accomplishments, it seems, will extend far into the future.
“Over the next year,” Warncke says, “it will be really interesting to see what kind of traffic we’re driving to these exciting new features. We are shipping things that are novel compared to our competitors. That’s very exciting!”
From the Bing Maps perspective, Pendleton can only agree.
“Because of the innovation released to the Web by Bing Maps and Microsoft Research,” he says, “we’re now viewed by many as one of the fastest advancing and most innovative groups at Microsoft.”