High-Quality Video View Interpolation Using a Layered Representation

  • Larry Zitnick ,
  • Sing Bing Kang ,
  • Matt Uyttendaele ,
  • Simon Winder ,

ACM SIGGRAPH |

Published by Association for Computing Machinery, Inc.

The ability to interactively control viewpointwhilewatching a video is an exciting application of image-based rendering. The goal of our work is to render dynamic scenes with interactive viewpoint control using a relatively small number of video cameras. In this paper, we showhowhigh-quality video-based rendering of dynamic scenes can be accomplished using multiple synchronized video streams combined with novel image-based modeling and rendering algorithms. Once these video streams have been processed, we can synthesize any intermediate view between cameras at any time, with the potential for space-time manipulation.

In our approach,we first use a novel color segmentation-based stereo algorithm to generate high-quality photoconsistent correspondences across all camera views. Mattes for areas near depth discontinuities are then automatically extracted to reduce artifacts during view synthesis. Finally, a novel temporal two-layer compressed representation that handles matting is developed for rendering at interactive rates.

Publication Downloads

MSR 3D Video Dataset

March 11, 2014

This data includes a sequence of 100 images captured from 8 cameras showing the breakdancing and ballet scenes from the paper “High-quality video view interpolation using a layered representation”, Zitnick et al., SIGGRAPH 2004. Depth maps, computed from stereo, are also included for each camera along with the calibration parameters.