Microsoft Research
current research
visiting msr
university relations



Tiling Layered Depth Images - click for more information.

Face Modeling

3D modeling. Generating realistic 3D human face models and facial animations has been a persistent challenge in computer vision and graphics. We have developed a system that constructs textured 3D face models from videos with minimal user interaction. Our system takes a video sequence of a face with an ordinary video camera. After five manual clicks on two images to tell the system where the eye corners, nose tip and mouth corners are, the system automatically generates a realistic looking 3D human head model and the constructed model can immediately be animated (different poses, facial expressions and talking). Optionally, in order to achieve higher accuracy, a user can click an additional set of three points to indicate the lower part of the face. A user, with a PC and an ordinary camera, can use our system to generate his/her face model in a few minutes.

Facial expression. We are also interested in facial expression mapping. Facial expressions exhibit not only facial feature motions, but also subtle changes in illumination and appearance (e.g., facial creases and wrinkles). These details are important visual cues, but they are difficult to synthesize. Traditional expression mapping techniques consider feature motions while the details in illumination changes are ignored. We propose to capture the illumination change of one person's expression in what we call an expression ratio image (ERI). Together with geometric warping, we map an ERI to any other person's face image to generate more expressive facial expressions.

There is an internal website for the Face Modeling project.

People involved are: Zhengyou Zhang, Zicheng Liu, Dennis Adler, Michael Cohen, Erik Hanson, Chuck Jacobs, Ying Shan, Andreas Soupliotis, and Matt Uyttendaele.


Model-Based Bundle Adjustment with Application to Face Modeling. Ying Shan, Zicheng Liu, Zhengyou Zhang. In Proceedings of the 8th International Conference on Computer Vision (ICCV01), Vol. II, pages 644-651, July 20001, Vancouver, Canada.

Rapid Modeling of Animated Faces From Video. Zicheng Liu, Zhengyou Zhang, Chuck Jacobs, Michael Cohen. In Proceedings of The Third International Conference on Visual Computing (Visual 2000), pages 58-67, September 2000, Mexico City. Also available as Technical Report MSR-TR-99-21.

Robust Head Motion Computation by Taking Advantage of Physical Properties. Zicheng Liu, Zhengyou Zhang. IEEE Workshop on Human Motion (HUMO 2000),  Austin, USA, December 2000.

Expressive Expression Mapping with Ratio Images. Zicheng Liu, Ying Shan, Zhengyou Zhang. in SIGGRAPH 2001, pages 271-276, Los Angeles, August 12-17.  The color plate has a few cool examples.

Media coverages:

Red Herring Magazine by Niall McKay (4/5/2001): Let's play with Microsoft

CBS Saturday Early Show with Renay San Miguel (4/21/2001) : MPEG file (77 MB).

MicroNews: 3-D Talking Heads on the Horizon (1/12/2001)


<% Call GetFooter() %>