Individual 3D Face Synthesis Based on Orthogonal Photos and Speech-Driven Facial Animation

  • Shigong Shan ,
  • Wen Gao ,
  • Jie Yan ,
  • Hongming Zhang ,
  • Xilin Chen

Published by Institute of Electrical and Electronics Engineers, Inc.

Publication

In the paper, a methodology for individual face synthesis using given orthogonal photos is proposed. And an integrated speech-driven facial animation system is presented. Firstly, in order to capture given subject’s personal facial configuation, a novel coarse-to-fine strategy based on facial texture and deformable template is proposed to localize some facial feature points in the image frontal view. And the corresponding feature points in the profile are extracted by using polygonal approximation. Secondly, all these feature points are aligned to fit the generic 3D face model to a specialized one to reflect the given person’s facial configuration. Then a multi-direction texture-mapping technique is presented to synthesize a lifelike personal face. Finally, muscle-based expression an lip-motion models are built up. All above technolgies are integrated into a speech-driven face animation system. We are aiming at a MPEG-4 compatible video-driven face animation system.