Back to group page


Sudipta N. Sinha     Johannes Kopf     Michael Goesele     Daniel Scharstein     Richard Szeliski
Microsoft Research     Microsoft Research     TU Darmstadt     Middlebury College     Microsoft Research

Two-layer decomposition for the Guitarist image sequence. From left to right: sample input image, its decomposition into the front and rear layers, detail from an interpolated view using the one-layer model, and using a two-layer model. Notice how the one-layer rendering shows significant ghosting (doubled edges).


We present a system for image-based modeling and rendering of real-world scenes containing reflective and glossy surfaces. Previous approaches to image-based rendering assume that the scene can be approximated by 3D proxies that enable view interpolation using traditional back-to-front or z-buffer compositing. In this work, we show how these can be generalized to multiple layers that are combined in an additive fashion to model the reflection and transmission of light that occurs at specular surfaces such as glass and glossy materials. To simplify the analysis and rendering stages, we model the world using piecewise-planar layers combined using both additive and opaque mixing of light. We also introduce novel techniques for estimating multiple depths in the scene and separating the reflection and transmission components into different layers. We then use our system to model and render a variety of real-world scenes with reflections.
    author  = {Sudipta Sinha and Johannes Kopf and Michael Goesele
               and Daniel Scharstein and Richard Szeliski},
    title   = {Image-Based Rendering for Scenes with Reflections},
    journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2012)},
    year    = {2012},
    volume  = {31},
    number  = {4},
    pages   = {to appear}
Click here
Supplementary Video
Click here
Supplementary Material
Click here
Download Data
Click here

Back to projects page