Image-space bidirectional scene reprojection
ACM Trans. Graphics (SIGGRAPH Asia), 30(6), 2011.
Real-time temporal upsampling through image-based reprojection of adjacent frames.
We introduce a method for increasing the framerate of real-time rendering applications. Whereas many
existing temporal upsampling strategies only reuse information from previous frames, our bidirectional
technique reconstructs intermediate frames from a pair of consecutive rendered frames. This significantly
improves the accuracy and efficiency of data reuse since very few pixels are simultaneously occluded in
both frames. We present two versions of this basic algorithm. The first is appropriate for fill-bound
scenes as it limits the number of expensive shading calculations, but involves rasterization of scene
geometry at each intermediate frame. The second version, our more significant contribution, reduces both
shading and geometry computations by performing reprojection using only image-based buffers. It warps and
combines the adjacent rendered frames using an efficient iterative search on their stored scene depth and
flow. Bidirectional reprojection introduces a small amount of lag. We perform a user study to investigate
this lag, and find that its effect is minor. We demonstrate substantial performance improvements (3-4X)
for a variety of applications, including vertex-bound and fill-bound scenes, multi-pass effects, and motion
One nice contribution is the idea of using an iterative implicit solver to locate the source pixel
given a velocity field.
In the more recent project
Automating image morphing
using structural similarity on a halfway domain
we adapt this idea to evaluate an image morph directly in a pixel shader,
without requiring any geometric tessellation, i.e. using only “gather” operations.
ACM Copyright Notice
Copyright by the Association for Computing Machinery, Inc. Permission to make digital or
hard copies of part or all of this work for personal or classroom use is granted without fee provided that
copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the
full citation on the first page. Copyrights for components of this work owned by others than ACM must be
honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to
redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications
Dept, ACM Inc., fax +1 (212) 869-0481, or firstname.lastname@example.org. The definitive version of this paper can be
found at ACM's Digital Library http://www.acm.org/dl/.