CVPR 2001 Short Course

Image-based Lighting

Instructor: Paul Debevec (USC Institute for Creative Technologies)

Duration: 3.5 hours

ABSTRACT:

Using images of real light to illuminate computer-generated scenes provides new levels of realism and new avenues for creativity. This course teaches everything from the theory behind the techniques to the nuts and bolts of using image-based lighting in actual production with commercially available software. Topics include high-dynamic range imagery (HDRI), lighting acquisition, illuminating real and synthetic objects with measurements of real light, global illumination, real-time techniques, and photoreal compositing.

EXPANDED COURSE STATEMENT:

This course will present recently developed techniques for realistically integrating computer-generated imagery with live-action photography that use measurements of real-world lighting to illuminate CG objects. The course will present background material in high dynanmic range photography, global illumination, and light reflection, and will describe both the theory and practice of acquiring measurements of light in the real world and injecting this light into computer animations. The techniques will be illustrated with detailed breakdowns of the animations "Rendering with Natural Light" and "Fiat Lux" and shots from recent feature films. Examples of applying the techniques using commercial packages such as LightWave 3D will also be presented. The course will conclude with an overview of techniques to extend image-based lighting to people and objects in the real world.

PREREQUISITES:

Participants should have basic knowledge of or experience with modeling and rendering in a traditional modeling/rendering package, for example 3D Studio Max, LightWave 3D, Maya, or SoftImage, and be familiar with basic traditional CG lighting and compositing. No advanced ray tracing or global illumination knowledge is assumed.

TOPICS:

This course will cover high-dynamic range photography, lighting acquisition, image-based lighting and compositing, real-time techniques, and ongoing research in illuminating real people and objects with sampled light. Use of the techniques in actual productions and with particular software packages will be explained. Additionally, the course will cover the basics of light reflection and global illumination necessary for understanding the techniques presented.

OUTLINE:

0:00-0:45

1) Introduction

What is image-based lighting?

How does it differ from traditional techniques?

2) An Overview of How Light Works

What light is and how it is generated

How light reflects off of (and scatters through) surfaces

Specular, diffuse, retro-reflection, arbitrary BRDFs

Measuring BRDFs

How everything reflects everything else

Effects and examples of real-world light interreflection

How environmental illumination determines the appearance of an object

3) Global Illumination Overview

Basics of radiosity techniques

Basics of Monte-Carlo global illumination techniques

Introduction to the RADIANCE global illumination package

4) How Cameras measure light

Lenses, Shutters, Apertures, CCD arrays, and ADCs

Luminance response

Color response

Imperfections

Flare, chromatic abberration, vignetting

How to compensate for these imperfections

0:45-1:30

5) Recovering High Dynamic Images (HDRI) from Photographs

Taking a series of photographs

Algorithm for deriving the response curve

Combining the photographs into a radiance map

Using the "mkhdr" program to recover high dynamic range radiance maps from photographs

Viewing and Editing HDR imagery

Related paper:

"Recovering High Dynamic Range Radiance Maps from Photographs", SIGGRAPH 97

6) Recording Real-World Illumination

What to look for in a camera

Types of mirrored balls

Acquiring a light probe image using a mirrored ball

Alternate light probe techniques

Panoramic camera techniques

Low-dynamic range techniques

Real-time light probes

Rendering synthetic probe images

7) Illuminating Synthetic Objects with Real Light

Mapping a light probe image into a CG environment

Simulating light from the environment on the object

Converting light probe images to light source constellations

8) Making "Rendering with Natural Light" (SIGGRAPH 98 Electronic Theater)

Modeling the scene

Acquiring the light

Rendering the scene

Communicating the sense of brightness

Vignetting

Defocus and effects thereof

Motion Blur

Glare

Using the custom "pflare" RADIANCE program to produce these effects

Re-rendering with Natural Light - creating the DVD edition

Improved sampling of curved surfaces

1:30-1:45 Break

1:45-2:10

9) Rendering Synthetic Objects into Real Scenes

Computing shadows, reflections, caustics, and interreflections

Compositing into background plates

Differential Rendering

Rendering into image-based environments

Related paper:

"Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography", SIGGRAPH 98

10)Making "Fiat Lux" (SIGGRAPH 99 Electronic Theater)

Combining Image-Based Lighting with Image-Based Modeling and Rendering

Creating "illum" light sources for direct lighting

Integrating animated objects

Avoiding interframe flickering

Reflectance property tricks to avoid sparkling

Inverse global illumination for the floor of St. Peter's

Distributed rendering

2:10-2:35

11)Image-Based Lighting with Commercial Renderers and in Commercial Production

Image-Based Lighting in Lightwave 3D 6.0

Image-Based Lighting in Standard Ray-Tracers (LightWave 5.5, BMRT)

Image-Based Lighting with Scanline Rendering

2:35-2:50

12)Overview of real-time image-based lighting techniques

Environment Mapping (including history thereof)

Prefiltering techniques

What accuracy is lost?

Related papers:

Ned Greene, "Environment Mapping and Other Applications of World Projections"

Gibson and Murta, "Interactive Rendering with Real-World Illumination", EGRW 2000

2:50-3:30

13)Image-Based Lighting Real Objects and Actors

Acquiring Reflectance Fields with a Light Stage

Illuminating reflectance fields

Real-time image-based lighting of human faces

Acquiring reflectance fields of objects

Combining with Environment Matting techniques

Reflectometry from reflectance fields and surface geometry

Related papers:

Debevec et al, "Acquiring the Reflectance Field of the Human Face", SIGGRAPH 2000

Zongker et al, "Environment Matting", SIGGRAPH 99

CONTACT INFORMATION

Paul Debevec

Executive Producer, Graphics Research

USC Institute for Creative Technologies

13274 Fiji Way, 5th Floor

Marina del Rey, CA 90292

Email : paul@debevec.org

Back to Main