CVPR 2001 Short Course
Computer Vision and the Art of Special
Effects
Instructors:
Steve Sullivan (Industrial Light and Magic), Luc Robert
(REALVIZ), Irfan
Essa (Georgia Institute of Technology), Steve Seitz (University of
Washington), and Eugene Vendrovsky (Rhythm & Hues)
DESCRIPTION
Over
the past 10 years, computer graphics has come to dominate special effects
production. Despite the sophisticated tools available, however, artists still
rely on essentially manual processes to complete many of their shots. As manual
approaches prove too expensive, time-consuming, or inexact to meet directors'
escalating demands, studios are reaching deeper into current technical research
to achieve their artistic goals.
In
this course, we discuss how computer vision techniques are impacting feature
film production. After an overview of the production process, representatives
from ILM and Rhythm and Hues will describe their use of vision algorithms on
recent shows such as "Star Wars - The Phantom Menace",
"Babe", "The Mummy", "Nutty Professor",
"Pearl Harbor", "A.I.", and "Harry Potter". Luc Robert of REALVIZ will then discuss the
challenges of commercializing vision algorithms for both large and small-scale
production, and Irfan Essa
will describe how special effects projects are used to motivate and supplement
a computer vision course at Georgia Tech. We'll close the session with
discussion of problems with current algorithms as well as production issues
ripe for computer vision solutions.
COURSE
OUTLINE
Intro:
computer vision and film production
Steve
Sullivan, ILM
Description
of how an effects studio works, and how computer vision algorithms make their
way into artists' hands.
- How are studios and shows organized?
- How is a shot put together?
- What is the production pipeline?
- R&D in a production environment
- How do algorithms make their way to the
screen?
- How does vision fit in?
- When do we buy, and when do we roll our own?
Computer
vision at ILM
Steve
Sullivan, ILM
An
overview of past and current computer vision techniques in use at Industrial
Light + Magic. Topics will be discussed by deconstructing shots from the films
"Star Wars - Episode I", "Mighty Joe Young", "The
Mummy", "Magnolia", "Pearl Harbor", and "A.I. -
Artificial Intelligence".
Topics
covered:
- Camera calibration
- Matchmoving
- Image-based modeling
- Image-based rendering
- Motion capture
- On-set visualization
- Rotoscoping
- Object removal
- The gap between theory and practice
Commercializing
vision algorithms for production
Luc
Robert, REALVIZ
Luc
will speak about his experiences founding REALVIZ to commercialize computer
vision algorithms for a production environment. The presentation will be
illustrated with a number of production examples and technical demonstrations.
Topics include:
- Vision algorithms and film production
tools: how big is the gap?
- Camera calibration, Matchmoving,
Image-based Modeling, Retiming, Image Stitching
Break
Special
effects and computer vision education
Steve Seitz,
Irfan Essa, Georgia Tech.
- Steve will give an overview of "Vision
for Graphics", a course he co-taught with
- Irfan will discuss
his course at Georgia Tech which combines the technical aspects of computer vision
and computer graphics, especially applied to digital video special effects
production. He'll briefly describe the
topic covered, concepts discussed and show some of his
students' completed projects. A similar course is also now offered at MIT. Further
information and video projects are available from http://www.cc.gatech.edu/dvfx/.
Computer
vision at Rhythm and Hues
Eugene
Vendrovsky
Eugene
will present an overview of computer vision techniques in use at Rhythm and
Hues Studios. He'll discuss the use of proprietary and commercial vision
algorithms at R&H, as well as present the perspective of a matchmove artist using these tools. Shot breakdowns will
include material from the films "Sum of All Fears", "Harry
Potter", and "Dr. Doolittle 2".
Topics
covered:
- 2D tracking
- Camera matchmove
- Object pose estimation and matchmove
- Geometry estimation
- Interactive tools for performing the above
Current
and future problems
Steve
Sullivan, ILM
A
look at how current algorithms can fall short under real production conditions,
as well as a discussion of the algorithms we'd like to see in the future.
Topics
include:
- Image-based modeling
- Lighting recovery
- Material/reflectance estimation
- Recovery of arbitrary deforming geometry
- "Daylight" motion capture
- Automatic shape tracking/matting
- Object removal/plate restoration
BRIEF
BIOGRAPHIES:
STEVE
SULLIVAN:
Steve
is a Principal Engineer in ILM's Research and
Development group, specializing in computer vision applications for film and
video production. He received a PhD in Electrical Engineering in 1996 from the
University of Illinois at Urbana-Champaign, with an emphasis on automatic
object modeling, recognition, and surface representations. After graduation, he
joined Rhythm and Hues Studios in
LUC
ROBERT:
Luc
is Chief Technology Officer at REALVIZ (www.realviz.com),
and was instrumental in the founding of the company in 1998. From an initial
team of 6, REALVIZ has now grown to a total of 95 staff members, with
headquarters in Sophia Antipolis,
Luc
received a PhD in Computer Science from the Ecole Polytechnique (
STEVE
SEITZ:
Steve
is an assistant professor in the Department of Computer Science and Engineering
at the
IRFAN
ESSA:
Irfan is an
Assistant Professor at the
EUGENE
VENDROVSKY:
He
received a PhD in Computer Graphics in 1986 from the
CONTACT
INFORMATION:
Steve
Sullivan
Industrial
Light + Magic
Tel:
415.448.3707
Back to
Main