This paper presents a method for joint stereo matching

and object segmentation. In our approach a 3D scene is

represented as a collection of visually distinct and spatially

coherent objects. Each object is characterized by three different

aspects: a color model, a 3D plane that approximates

the object’s disparity distribution, and a novel 3D connectivity

property. Inspired by Markov Random Field models

of image segmentation, we employ object-level color models

as a soft constraint, which can aid depth estimation in

powerful ways. In particular, our method is able to recover

the depth of regions that are fully occluded in one input

view, which to our knowledge is new for stereo matching.

Our model is formulated as an energy function that is optimized

via fusion moves. We show high-quality disparity

and object segmentation results on challenging image pairs

as well as standard benchmarks. We believe our work not

only demonstrates a novel synergy between the areas of image

segmentation and stereo matching, but may also inspire

new work in the domain of automatic and interactive objectlevel

scene manipulation.

PDF file

In  CVPR

Details