Matting
This page describes the work about matting. In this work we
are also interested with the photometry of the image, not only with
the geometry and so we examine the pixel values to recover
measurements that are smaller than one pixel. In the first work, Alpha Matting model is used. This means that
the measurement in each image pixel is due to a combination of
two layers. Here we show how to estimate these layers from
multiple images, and how to compensate degenerate cases that arise
from daily images with priors. We then extend this to Environment Matting in which the foreground
layer has photometrically active elements that can bend the
light ray (e.g. a lens) and so each pixel measurement is a combination
of arbitrary parts of the background.
Alpha Matting
Experiment 1: Fence Sequence
The table below shows four out of seven input images alongwith the
computed clean plate:
 |
 |
 |
 |
Four out of seven input images taken with
hand-held camera.
Inter-layer motion is due to parallax.
Here are results of embedding the fence in a new environment:
| Composite on checkerboard |
Composite on real image |
|---|
|
|
Experiment 2: Monkey Sequence
 |
 |
 |
 |
Four out of twenty input images taken with
static camera.
Background layer is moved explicitly.
We used the two alpha-mattes to place the monkey between the building
and the fence:
 |
Combined image. We used images with different
lighting to emphasize the composition process. Note that the alpha
mattes are sub-pixel accurate
Environment Matting
When the scene contains optically active elements
alpha-matting framework is not applicable as the foreground object may
collect light from any location in the background. This work
shows how this can be recovered from real images.
Experiment 1: Magnifying Glass
 |
 |
 |
 |
Four out of 42 input images taken with
static camera.
Result
After learning the distortion, we can embed the magnifying glass in a
new image. Click the image for a video.
 |
 |
 |
Embedding the lens in a new image. The area
below the line shows the alpha-matting and so does not reproduce the
distortion of the lens
Experiment 2: Distorted Window
In this experiment we model the imperfections of an old window. Two
images were taken: one through the window and a clear one. The
distortions are estimated under the assumption that they are smooth
and small.
 |
 |
The two input images
Result
 |
 |
 |
Distortion is applied to a static baloon
image. Click on the images to see the whole video.
Experiment 3: Complex glass vase
Results
The above model was composed on the following two movies:
 |
 |
 |
 |
 |
 |
Input. Click here for the full giraffe movie
(2.1Mb) and here for the gnu movie (1.6Mb)
