Rate-distortion optimized 3D reconstruction from noise-corrupted multiview depth videos

Wenxiu Sun, Gene Cheung, Philip A. Chou, Dinei Florencio, Cha Zhang, and Oscar C. Au

Abstract

Transmitting compactly represented geometry of a dynamic

scene from a sender can enable a multitude of 3D imaging

functionalities at a receiver, such as synthesis of virtual images

from freely chosen viewpoints via depth-image-based

rendering (DIBR).While depth maps can now be readily captured

using inexpensive depth sensors, they are often corrupted

by non-negligible acquisition noise. In this paper, we

derive 3D surfaces of a dynamic scene from noise-corrupted

depth maps in a rate-distortion (RD) optimal manner. Specifically,

unlike previous work that finds the most likely (e.g.,

maximum likelihood) 3D surface from noisy observations

regardless of representation size, we judiciously search for

the best fitting (i.e., minimum distortion) 3D surface subject

to a bitrate constraint. Our RD-optimal solution reduces

to the maximum likelihood solution as the rate constraint is

loosened. Using the MVC codec for compression of multiview

depth video and MPEG free viewpoint test sequences

as input, experimental results show that RD-optimized 3D reconstructions

computed by our algorithm outperform unprocessed

depth maps by up to 2.42dB in PSNR of synthesized

virtual views at the decoder for the same bitrate.

Details

Publication typeInproceedings
PublisherIEEE Internation Conference on Multimedia & Expo (ICME)
> Publications > Rate-distortion optimized 3D reconstruction from noise-corrupted multiview depth videos