Resynthesizing Facial Animation through 3D Model-Based Tracking

Frédéric Pighin, David H. Salesin, and Richard Szeliski

Abstract

Given video footage of a person's face, we present new techniques to automatically recover the face position and the facial expression from each frame in the video sequence. A 3D face model is fitted to each frame using a continuous optimization technique. Our model is based on a set of 3D face models that are linearly combined using 3D morphing. Our method has the advantages over previous techniques of fitting directly a realistic 3-dimensional face model and of recovering parameters that can be used directly in an animation system. We also explore many applications, including performance-driven animation (applying the recovered position and expression of the face to a synthetic character to produce an animation that mimics the input video), relighting the face, varying the camera position, and adding facial ornaments such as tattoos and scars.

Details

Publication typeInproceedings
Published inSeventh International Conference on Computer Vision (ICCV'99)
Pages143-150
AddressKerkyra, Greece
PublisherIEEE Computer Society
> Publications > Resynthesizing Facial Animation through 3D Model-Based Tracking