eHeritage Projects

Displaying Unique Historical Treasures

eHeritageEach nation's heritage represents its past glory, its present identity, and its future inspiration. The eHeritage projects preserve cultural heritage through the application of advanced computing technologies.

Semantic Annotation Services For 3-D Models of Cultural Heritage Artifacts

Jane Hunter, The University of Queensland, Australia

Craig Koomeeta, Wuungkam [Barramundi] Craig Koomeeta, Wuungkam [Barramundi] Cultural heritage institutions are increasingly using three-dimensional (3-D) digital preservation techniques to preserve fragile museum artifacts and to develop virtual online collections. Such 3-D representations improve access to physical collections and provide precise digital surrogates for scholarly analysis. As 3-D data acquisition tools become more affordable and available, the amount of 3-D data requiring storage, indexing, and search services will grow dramatically. Because high quality metadata is prohibitively expensive, museums are eager to explore how they might exploit community tagging and annotation services. Such services enable users to easily attach information, comments, and keywords to 3-D objects and their sub-parts and hence, add significant contextual value to the collection, enhance discovery, and enable much richer online exhibitions. Annotation services also facilitate collaboration and knowledge exchange between geographically dispersed communities of curators, scholars, and indigenous owners. Moreover, if the annotations are drawn from pre-defined ontologies, they provide machine-understandable semantics that enable further reasoning about the object and the derivation of new knowledge. The aim of this project is to develop simple, semantic annotation services for 3-D digital objects (and their sub-parts) that will facilitate the discovery, capture, inferencing, and exchange of valuable cultural heritage knowledge.

A Low-cost Image-based Rendering System for eHeritage

Bingfeng Zhou, Institute of Computer Science and Technology, Peking University, P. R. China

High-quality rendering of the visual aspect of the culture heritage content is an important task in eHeritage systems. In this project, we mainly focus on the application of image-based rendering techniques in this domain.

Rendering objects in different environment mapping with sampled light resourcesRendering objects in different environment mapping with sampled light resources

A low-cost system that can efficiently reconstruct 3-D geometry of objects from photos, and render them in virtual environment with high photo-reality, will be proposed. Related with this system, researches will focus on the following topics: image-based geometry reconstructing and processing, GPU-accelerated rendering, surface appearance modeling, and optionally image-based dynamic object rendering and modeling.

Style-Consistency Chinese Calligraphic Character Synthesis

Yueting Zhuang, Jiangqin Wu, Kai Yu, College of Computer Science and Technology, Zhejiang University, P. R. China

Chinese calligraphy is one of the most important parts of Chinese culture heritage. In order to make those precious arts easily accessible and sharable to people worldwide over the Internet, many pieces of Chinese calligraphy works have already been digitized for preservation and exhibition. At Zhejiang University, much research has been conducted on digitized Chinese calligraphy, including retrieval, recognition, tablet(牌匾) design, new style generation, and historical Chinese calligraphy verification. In this proposed project, we explore the ways of style-consistency Chinese calligraphic character synthesis by computer, consequently generate Chinese calligraphic character in the style of some famous calligraphists.

 Calligraphic Character Synthesis SystemFramework of the Chinese Calligraphic Character Synthesis System

We will introduce style features into the synthesis and propose an approach of style-consistency Chinese calligraphic character synthesis. Firstly, the layout of the character to be generated, including the structure and the relative positions of strokes and radicals extracted, is calculated and then adjusted by the style features. Secondly, the radicals and strokes are chosen and put together to combine the calligraphic character as an optimization problem. Finally, the generated calligraphic character is estimated and adjusted by an evaluation model based on the style features.

China-US Million Book Digital Library Project

The Realtime Simulation of the Ancient Festival and Construction Process

Jizhou Sun, Nobuo AOKI, Subin Xu, Yan Liu, School of Computer Science and Technology (Jizhou SUN, Yan Liu), School of Architecture (Nobuo AOKI, Subin XU), Tianjin University, P. R. China

In this project, Motion Capture, Motion Editing, and Intelligent Virtual Human technology will be used in simulating the festival and complex construction process of ancient people in Confucian Temple, which is an important part of Intangible Cultural Heritage. The basic action will be achieved through Motion Capture technology. Then, motion editing technology will make these actions suitable for the digital actor. Moreover, the Intelligent Virtual Human technology will realize the real-time simulation of the complex festival and construction process. Finally, a demo about the festival—“shidian”—played in Dacheng Dian will be finished.

Some results from our IBMR researchSome results from our IBMR research

Appearance Modeling for Objects with Complex Reflectance Properties

Yue Qi, Beihang University, P. R. China

This project will develop methods and devices for capturing and modeling the complex appearance. An acquiring device will be set up, which can capture high-precision reflectance data effectively. Modeling methods for spatially-varying bidirectional reflectance distributed function (SVBRDF), which is used to represent reflectance properties of the object, will be developed. An intuitive SVBRDF acquiring and modeling system will be released.

Capture and Reassemble Fractured Heritage Objects

Xinguo Liu, School of Computer Science and Technology, Zhejiang University, P. R. China

Damaged heritage artifacts are often found in archaeological activities. It is important, but difficult, to reconstruct the original shapes and textures using the fractured patches for archaeologists. We will build a capture dome for scanning the shape and appearance of heritage artwork. Our dome is equipped with a set of synchronized video cameras and LED lights. We capture the shape of heritage artworks and patches using structured light techniques, and capture the reflectance field under a set of basis light conditions.

Our acquisition setup under constructionOur acquisition setup under construction

With the captured data, we can synthesize realistic images of the heritage objects in novel environments. Second, we develop a hybrid approach to resemble the fractured heritage patches. We use both the local geometry and texture features to evaluate if two patches are neighboring patches of the same object. Local features are not enough when there are tiny or missing patches. We then propose to use global shape features to address this problem. At the conclusion of the project, we will share our results throughout the world via the Internet. 

Image-based Large-scale Monuments Modeling

Long Quan, The Department of Computer Science, The Hong Kong University of Science and Technology, Hong Kong SAR

The Microsoft Research Asia eHeritage initiative intends to preserve the world heritage in digital formats for various modern applications. Image-based approach for eHeritage monument modeling is probably the most appropriate methodology to achieve the large-scale heritage monument modeling. As the pure image-based approach is non-invasive, it can model arbitrarily large-scale objects, and it can capture not only the geometry properties of the heritage objects, but also the photometric properties.

In this project, we focus on the quasi-dense approach that we have been developing for many years. The quasi-dense is unique in that it is not only robust and accurate in Structure from Motion that is the fundamental prerequisite of the whole image-based methodology, but also the semi-density of the reconstructed point clouds allows the effective computation of the underlying object models. In the past, we have already successfully used the quasi-dense approach to model plants and trees, facades and buildings. These are de facto part of the heritage monuments and sites and constitute an initial stage for the full development of large-scale heritage monument modeling. This proposal is built upon these prior works, and one preliminary example of heritage monument modeling is given in the figure below.

The preliminary results on Hennepin Avenue, MinneapolisThe preliminary results on Hennepin Avenue, Minneapolis. Data from Microsoft Virtual Earth

Polarization Stereo for Modeling the Small-Scale Structure of Cultural Assets

Daisuke Miyazaki, Hiroshima City University, Graduate School of Information Sciences, Japan

This project aims to develop a measurement system based on polarization, which can obtain the surface normal and the depth of the scene. The system is consisted of multiple cameras, which a polarization filter or a polarization prism is set in front of each. The polarization stereo is a passive measurement system; thus, it can obtain the small-scale structure of huge cultural buildings or statues since we do not need to illuminate the target objects. Cameras and polarizers are less expensive than the laser range sensor.


Visual and Tactile Rendering of Japanese Traditional Drawings and Costumes

Hiromi T. Tanaka, Shinichi Hirai, and Xin Yin, Ritsumeikan University
Naoki Saiwaki, Nara Women’s University, Japan

Noh is a Japanese historical play lasted for 600 years and Noh costumes are precious cultural heritage. And Ukiyo-e is one famous traditional woodblock type Japanese drawing. They are thin with beautiful appearance and cannot be touched as their value. We call this type cultural heritage as tissue type cultural heritage. There are two subjects of this project. One is developing an efficient image-based method for rendering appearance of tissue type cultural heritage. Another is developing techniques which permit user touch their surface at same time.

To render the appearance, a lot of reflection models were proposed based measured reflection data in last decades. To improve reality of rendering results, it is also need to construct anisotropy reflection model for this tissue type cultural heritage based on measured meso-structure. Two types of data are measured. One is BTF data measured by using the four axes machine, which can put the camera and lightning on arbitrary position of a half-sphere dome. The other is high precision 3-D meso-structure data (resolution is 2.0 μm) measured by using a 3-D digital microscope. Then, the anisotropy reflection model can be construed from this measured meso-structure geometry and its parameters are fit to the measured BTF data. As all the parameters are saved as texture, the rendering can be run in real-time based on GPU. About tactile rendering studies, developing tactile devices were focused on and a little of tactile rendering algorithms were proposed. Using some trigonometric signals such as sine vibration signals to excite nerve in skin, some type tactile can be represented. But the result is far from real one and there is far way need to go. When finger touch and move on the object surface, how the tactile vary according to the surface meso-structure is not studied well. This tactile variation according to time is important to represent real tactile and has been ignored in previous work.

 A VR system for watching tissue type cultural heritageA VR system for watching tissue type cultural heritage

In this project, the finger deformation is simulated based on measured meso-structure. Compare simulate result to the measured surface strain variation when finger move on the surface, how the meso-structure on surface effect the tactile nerve can be understood. Then the tactile variation according to time can be rendering and is shown using vibration type tactile device. To evaluate touch feeling, psychophysics methods, such as the semantic differential (SD), method is used to evaluate whether the tactile rendering result is near to real one or not. After all the goal is achieved, user can watch appearance of tissue type cultural heritage and touching their surface at same time. This is one type of next-generation digital museum.

Learn more about this project

Interactive Global Illumination of Massive eHeritage Data

Sung-Eui Yoon, Korea Advanced Institute of Science and Technology, Korea

Massive cultural heritage data are easily generated due to the advances of geometric and photometric modeling techniques. These data can take multi-GB size and it is likely that the complexity of cultural heritage data will continue to grow. In order to interactively visualize this massive cultural heritage data with a high image quality, we propose to design scalable and interactive physically-based global illumination methods. Particularly, we propose three different research directions to achieve our goal: multi-resolution based global illumination, cache-coherent global illumination, and random-accessible compressed data. We plan to distribute our research results and source codes on the Internet to achieve higher visibility of this research and encourage other researchers to extend our proposed research approaches.

Ray tracing the St. Matthew ModelThis figure shows an image shot of ray tracing the St. Matthew model with seven lights, shadows, and reflections

 Scalable Graphics/Geometric Algorithm Lab, Korea Advanced Institute of Science and Technology

A Low-Cost System for Effective Large-Scale Reconstruction

In So Kweon, School of Electrical Engineering and Computer Science, Korea Advanced Institute of Science and Technology, Korea

The main objective of this research is to develop a low-cost system for capturing three-dimensional (3-D) information and the associated effective 3-D reconstruction methods for indoor/outdoor structures. Recently several heritages such as historical sites have been reconstructed and archived as a digital structure. Usually they are captured by expensive sensors and equipments (for example, 3-D laser scanners) for accurate reconstruction. In this research, we aim at developing a low-cost sensory system, consisting of multiple sensors: a 2-D laser scanner, multiple perspective color cameras, an omni-directional camera, and a GPS, while achieving comparable accuracies as those of commercial 3-D laser scanners. In addition, we develop effective 3-D reconstruction methods to solve the three fundamental issues in the 3-D recovery:

  1. Error accumulation problem
  2. The correspondence problems (laser-to-laser, image-to-image, laser-to-image, image-to-aerial_images)
  3. The occlusion problem (self-occlusion and occlusion by moving objects)

Five laser correspondences between two successive imagesFive laser correspondences between two successive images [Bok2007]

Noise Estimation of Scanned Point Sets

Seungyong Lee, Department of Computer Science and Engineering, Pohang University of Science and Technology (POSTECH), Korea

Three-dimensional (3-D) scanning is an effective approach for reconstructing the shapes of eHeritage objects. Due to hardware and measurement errors, 3-D scanning process inevitably introduces noise to the acquired point set. To obtain accurate shapes of eHeritage objects, this noise should be resolved before or during the process of surface reconstruction.

Noisy point setsNoisy point sets. The second and fourth columns show the estimated noise distributions

In this research, we develop a statistical technique to estimate the noise in a scanned point set. The estimated noise is represented as normal distributions at points and the variances of the distributions determine the amount of the noise. To estimate the noise distribution, we adopt a variational Bayesian method, which is known to provide more robust estimation than point estimates, such as Maximum Likelihood and Maximum a Posteriori. The results of noise estimation can be used to guide subsequent processing, for example, point set denoising and surface reconstruction.

Displacement Mapping-based Real-time eHeritage Visualization

JungHyun Han, Korea University, Korea

From the computer graphics perspective, the research and development efforts for eHeritage focus on modeling and rendering of cultural heritage content. When the geometry data of eHeritage content are highly complex, it is not easy to transmit the content though the limited bandwidth of Internet and to display it at the low-end PCs.

Ancient gilt-bronze incense burners of Baekje Dynasty Ancient gilt-bronze incense burners of Baekje Dynasty

We propose a skeleton-based mesh simplification algorithm that can preserve the geometric details of the original complex mesh. The differences between the original mesh and the simplified one are recorded into a displacement map. The pair of simplified mesh and displacement map is appropriate for both transmission and rendering. At the rendering stage, global illumination is emulated using an ambient occlusion technique based on the displacement map.

Interactive 3D Media Lab, Korea University

Reviving the Mau Gong Ding of National Palace Museum by Integrating Technologies Developed by Microsoft Research Asia and NTU

Yi-Ping Hung, Graduate Institute of Networking and Multimedia, Department of computer Science and Information Engineering, National Taiwan University (NTU), Taiwan

The Microsoft Research Asia eHeritage initiative serves as a platform for worldly heritage digitally displayed that is without spatial and aging constraints. The images of ancient artifact can be taken and displayed virtually in 3-D and re-constructed to present the effect of revived objects to its original mint state using a simulation technology.

In this project, we combine the digital simulation de-weathering processes with human interactive interface using a breathing-based biofeedback for the signaling of the simulation process. A piece of invaluable artifact, the Mau Gong Ding, from National Palace Museum (NPM) has been chosen for this implementation. The Mau Gong Ding is one of the most precious bronze work in Chinese culture, dated back 800 B.C. Since the bronze appearance has been aged with patina, this project will adopt a de-weathering technique developed by Microsoft Research Asia to digitally revive and return the images of this bronze work to the condition that it was originally minted.

The breathing-based interaction with the artifactIllustration of the breathing-based interaction with the artifact

In this project, a 3-D modeling process will be performed first using the techniques developed by NTU. Next, a de-weathering technique will be applied with the assistance from Microsoft Research Asia. Finally, breathing-based biofeedback will be implemented as a human interface to trigger the de-weathering process for the virtual artwork which will be displayed in the Future Museum of NPM at Tao Yuan International Airport of Taiwan.

Contact Us

Microsoft Research Asia
5th Floor, Beijing Sigma Center
No.49 Zhichun Road, Haidian District
Beijing 100190, P. R. China

Send us an email message
Tel: (86-10)5896-8888
Fax: (86-10)8809-7305