Ph.D. Thesis of Tsinghua University
[ Thesis 50MB ]

Yue Dong, Xin Tong, Fabio Pellacini, Baining Guo

      We present AppGen, an interactive system for modeling materials from a single image. Given a texture image of a nearly planar surface lit with directional lighting, our system models the detailed spatially-varying reflectance properties (diffuse, specular and roughness) and surface normal variations with minimal user interaction. We ask users to indicate global shading and reflectance information by roughly marking the image with a few user strokes, while our system assigns reflectance properties and normals to each pixel. We first interactively decompose the input image into the product of a diffuse albedo map and a shading map. A two-scale normal reconstruction algorithm is then introduced to recover the normal variations from the shading map and preserve the geometric features at different scales. We finally assign the specular parameters to each pixel guided by user strokes and the diffuse albedo. Our system generates convincing results within minutes of interaction and works well for a variety of material types that exhibit different reflectance and normal variations, including natural surfaces and man-made ones.

ACM SIGGRAPH Asia 2011
[ Project Page ] [ Paper ] [ Video ]

Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Yanxiang Lan, Moshe Ben-Ezra, Baining Guo

      Manifold bootstrapping is a new method for data-driven modeling of real-world, spatially-varying reflectance, based on the idea that reflectance over a given material sample forms a low-dimensional manifold. It provides a high-resolution result in both the spatial and angular domains by decomposing reflectance measurement into two lower-dimensional phases. The first acquires representatives of high angular dimension but sampled sparsely over the surface, while the second acquires keys of low angular dimension but sampled densely over the surface.
We develop a hand-held, high-speed BRDF capturing device for phase one measurements. A condenser-based optical setup collects a dense hemisphere of rays emanating from a single point on the target sample as it is manually scanned over it, yielding 10 BRDF point measurements per second. Lighting directions from 6 LEDs are applied at each measurement; these are amplified to a full 4D BRDF using the general (NDF-tabulated) microfacet model. The second phase captures N=20-200 images of the entire sample from a fixed view and lit by a varying area source. We show that the resulting N-dimensional keys capture much of the distance information in the original BRDF space, so that they effectively discriminate among representatives, though they lack sufficient angular detail to reconstruct the SVBRDF by themselves. At each surface position, a local linear combination of a small number of neighboring representatives is computed to match each key, yielding a highresolution SVBRDF. A quick capture session (10-20 minutes) on simple devices yields results showing sharp and anisotropic specularity and rich spatial detail.

ACM SIGGRAPH 2010
[ Project Page ] [ Paper 3MB ] [ Video 50MB ]

Yue Dong, Jiaping Wang, Fabio Pellacini, Xin Tong, Baining Guo

      Many real world surfaces exhibit translucent appearance due to subsurface scattering. Although various methods exists to measure, edit and render subsurface scattering effects, no solution exists for manufacturing physical objects with desired translucent appearance. In this paper, we present a complete solution for fabricating a material volume with a desired surface BSSRDF. We stack layers from a fixed set of manufacturing materials whose thickness is varied spatially to reproduce the heterogeneity of the input BSSRDF. Given an input BSSRDF and the optical properties of the manufacturing materials, our system efficiently determines the optimal order and thickness of the layers. We demonstrate our approach by printing a variety of homogenous and heterogenous BSSRDFs using two hardware setups: a milling machine and a 3D printer.

ACM SIGGRAPH 2010
[ Project Page ] [ Paper 5MB ] [ Video 36MB ]

Jiaping Wang, Yue Dong, Xin Tong, Zhouchen Lin, Baining Guo

      We propose a kernel Nystrom method for reconstructing the light transport matrix from a relatively small number of acquired images. Our work is based on the generalized Nystrom method for low rank matrices. We introduce the light transport kernel and incorporate it into the Nystrom method to exploit the nonlinear coherence of the light transport matrix. We also develop an adaptive scheme for efficiently capturing the sparsely sampled images from the scene. Our experiments indicate that the kernel Nystrom method can achieve good reconstruction of the light transport matrix with a few hundred images and produce high quality relighting results. The kernel Nystrom method is effective for modeling scenes with complex lighting effects and occlusions which have been challenging for existing techniques.

ACM SIGGRAPH 2009
[ Project Page ] [ Paper 2MB ] [ Video 40MB ]

Yue Dong, Sylvain Lefebvre, Xin Tong, George Drettakis

      Existing solid texture synthesis algorithms generate a full volume of color content from a set of 2D example images. We introduce a new algorithm with the unique ability to restrict synthesis to a subset of the voxels, while enforcing spatial determinism. This is especially useful when texturing objects, since only a thick layer around the surface needs to be synthesized. A major difficulty lies in reducing the dependency chain of neighborhood matching, so that each voxel only depends on a small number of other voxels. Our key idea is to synthesize a volume from a set of pre-computed 3D candidates, each being a triple of interleaved 2D neighborhoods. We present an efficient algorithm to carefully select in a pre-process only those candidates forming consistent triples. This significantly reduces the search space during subsequent synthesis. The result is a new parallel, spatially deterministic solid texture synthesis algorithm which runs efficiently on the GPU. Our approach generates high resolution solid textures on surfaces within seconds. Memory usage and synthesis time only depend on the output textured surface area. The GPU implementation of our method rapidly synthesizes new textures for the surfaces appearing when interactively breaking or cutting objects.

Eurographics Symposium on Rendering 2008
[ Project Page ] [ Paper 12MB ] [ Video 46MB ] [ YouTube ]

Jiaping Wang, Shuang Zhao, Xin Tong, Stephen Lin, Zhouchen Lin, Yue Dong, Baining Guo, Heung-Yeung Shum

      In this paper, we propose techniques for modeling and rendering of heterogeneous translucent materials that enable acquisition from measured samples, interactive editing of material attributes, and real-time rendering. The materials are assumed to be optically dense such that multiple scattering can be approximated by a diffusion process described by the diffusion equation. For modeling heterogeneous materials, we present an algorithm for acquiring material properties from appearance measurements by solving an inverse diffusion problem. Our modeling algorithm incorporates a regularizer to handle the ill-conditioned inverse problem, an adjoint method to dramatically reduce the computational cost, and a hierarchical GPU implementation for further speedup. To display an object with known material properties, we present an algorithm that performs rendering by solving the diffusion equation with the boundary condition defined by the given illumination environment. This algorithm is centered around object representation by a polygrid, a grid with regular connectivity and an irregular shape, which facilitates the solution of the diffusion equation in arbitrary volumes. Because of the regular connectivity, our rendering algorithm can be implemented on the GPU for real-time performance. We demonstrate our techniques by capturing materials from physical samples and performing real-time rendering and editing with these materials.

ACM Transactions on Graphics, Volume 27, Issue 1
[ Project Page ] [ Paper 20MB] [ Video 46MB]