[   Kentaro Toyama's Home Page   |   Projects   |   Vision Technology   |   MSR   ]

Projects: Robust Face Pose Tracking

Robust Face Pose Tracking

Here are two movies of my work with robust face tracking:

In these MPEG files, you will see live video of me at my workstation with what appear to be moving faint colored scribbles all over my face. Those scribbles are what happens to overlaid polygons, rectangles, and octagons after they are put through a scan converter to Beta video tapes, and then from Beta video tapes through an MPEG encoder board into greatly reduced-resolution MPEG format. If you can make sense of them, this is what they mean:

• The color indicates the system's self-assessment of its degree of tracking success. Red means that it is in search mode and cannot say where the face is. Yellow means it is very likely to be the target face. Green indicates a lock on the correct target.
• Different shapes indicate what type of processing is taking place. The squiggly polygons show the perimeter of a star-shaped polygon fit to the part of the image underneath it that corresponds to skin color. The rectangle indicates the result of linearized sum-of-squared-differences (SSD) tracking (a la Greg Hager), as it tracks 2D position and orientation. The octagon (with projected normal) shows full 3D, 6-DOF tracking. The octagon is also accompanied by feature trackers on the eyes, nostrils, and lips, but those graphics are hardest to see.

Relevant Publications

K. Toyama, "Prolegomena for Robust Face Tracking", MSR Technical Report, MSR-TR-98-65, November 1998 (presented at the Post-ECCV Workshop on Advances in Facial Image Analysis and Recognition Technology, Freiburg, May 1998).

K. Toyama, "'Look, Ma -- No Hands!' Hands-Free Cursor Control with Real-Time 3D Face Tracking", In Proc. Workshop on Perceptual User Interfaces (PUI'98), San Francisco, November 1998. 

K. Toyama, G. Hager Incremental Focus of Attention for Robust Vision-Based Tracking, Int'l J. Computer Vision 35(1):45-63, Nov, 1999.

[   Kentaro Toyama's Home Page   |   Projects   |   Vision Technology   |   MSR   ]