High-quality Image-based Interactive Exploration of Real-World Environments

Interactive scene walk-throughs have long been an important computer graphics application area. Starting with Fred Brooks pioneering work, efficient rendering algorithms have emerged for visualizing large architectural databases. More recently, researchers have developed techniques for constructing photorealistic 3D architectural models from real-world images.Real-world tours based on panoramic images also exist, as we describe in the "Panoramic Imaging" sidebar. These systems all aim to create a real sense of being there, a sense of virtual presence that lets users experience a space or environment in an exploratory, interactive manner. This article presents an image-based rendering system that brings us a step closer to a compelling sense of being there. Whereas many previous systems have used still photography and 3D scene modeling, we avoid explicit 3D reconstruction because it tends to be brittle. Instead, we film a tour of an environment that we wish to explore and then use image-based rendering techniques to replay the tour interactively. We call such experiences interactive visual tours. In designing this system, our goal was to let users move freely along a set of predefined tracks, choose between different directions of motion at decision points, and look in any direction. In addition, the displayed views should have a high resolution and dynamic range. The system easily incorporates multimedia objects such as navigation maps, video textures, audio enhancements, and pop-up stills. Our system is not the .rst to propose interactive videobased tours (see the "Panoramic Imaging" sidebar). We believe, however, that our system is the first to deliver fully interactive, photorealistic image-based tours on a personal computer at or above broadcast video resolutions and frame rates. Moreover, to our knowledge, no other tour provides the same rich set of interactions or visually complex environments.

PDF file

In  IEEE Computer Graphics & Applications (CG&A)

Publisher  IEEE Computer Society
Copyright © 2007 IEEE. Reprinted from IEEE Computer Society. This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it.


InstitutionMicrosoft Research

Previous Versions

Matthew Uyttendaele, Antonio Criminisi, Sing Bing Kang, Simon Winder, Richard Hartley, and Richard Szeliski. High-quality Image-based Interactive Exploration of Real-World Environments, October 2003.

> Publications > High-quality Image-based Interactive Exploration of Real-World Environments