Simultaneous Matting and Compositing

Recent work in matting, hole filling, and compositing allows image elements to be mixed in a new composite image. Previous algorithms for matting foreground elements have assumed that the new background for compositing is unknown. We show that, if the new background is known, the matting algorithm has more freedom to create a successful matte by simultaneously optimizing the matting and compositing operations. This observation was motivated by the problem of recomposing the elements within a single image or within a set of related images. For example, many cameras are by default set for wide angle shots to successfully capture landscapes. Foreground characters often appear quite small relative to the image frame. This issue is exacerbated when one wants to display the image on a small mobile devices. We propose a new algorithm, that integrates matting and compositing into a single optimization process. The system is able to compose foreground elements onto a new background more efficiently and with less artifacts compared with previous approaches. In our examples, we show how one can enlarge the foreground while maintaining the wide angle view of the background. We also demonstrate composing a foreground element on top of similar backgrounds to help remove unwanted portions of the background or to rescale or rearrange the composition. We compare and contrast our method with Bayesian Matting, Iterative Matting, Photomontage, and the Image Retargetting systems.