Technology Brings Chinese Cultural Heritage to Life

Walk within the painting: Along the River During Qing-Ming Festival.

Publication date: April 27, 2011

Visitors to the Palace Museum in Beijing can step into an ancient city and hear the birds chirping, watch people shop, witness boats being loaded, and gaze upon animals working in the fields. All of this is possible through a state-of-the-art multimedia exhibition of China’s most famous historic panoramic painting, Along the River During Qing-Ming Festival

Part of the urban section of the ancient Chinese painting, "Along the River During the Qing-Ming Festival"Part of the urban section of the ancient Chinese painting, "Along the River During the Qing-Ming Festival" 

 

A Sensory Immersion into Ancient Culture

Imagine stepping into the Hall of Martial Valor at the Palace Museum in Beijing, where you see an ancient painting slowly rolling out on a wide screen. When you put on a headset beside the screen, the sounds of classical music, hawkers shouting, birds chirping, and dogs barking fill your ears. The screen displays a vast field and crisscross canals—the scenery of a vivid early spring day during the Song Dynasty (AD 960–1127). You are totally absorbed into the rural beauty of the painting.

Experiencing the interactive exhibitionExperiencing the interactive exhibition

A boy’s loud voice jars the peaceful setting. In the painting, he is driving a donkey to a bridge. Imagine that you touch the image of the boy and suddenly hear him shout, “Go ahead,” with an accent of Henan Province, the capital of China during the Song Dynasty. You follow him by moving your fingers along the screen. The sounds of running water and people’s voices become louder. The bridge ahead is filled with people. You make a gesture to enlarge the image for a better view of the individual people—you can even see their worried expressions as they maneuver to get out of the way of the donkey.

Amidst the imagery and sound of this bustling setting, you can imagine that you are living in the Song Dynasty now. You can use the interface to move along a street and watch an old woman feed her pigs; squeeze into a crowd and enjoy a fantastic storytelling performance; head for a vendor’s stand to buy a drink—until you suddenly realize that you are not truly in the painting, but taking control of a virtual, interactive, multimedia browser system.

Up Close with Chinese Ancient Paintings

“This painting is not only a miracle in the arts, but also a great reference to the culture of ancient China.”

— Jianhua Lin, vice president of Peking University

 

Along the River During Qing-Ming Festival is a historical, panoramic painting of ink-on-silk by the Chinese artist Zeduan Zhang from the Northern Song Dynasty. The piece is painted on a hand scroll and portrays three scenes of daily life during the Song Dynasty: suburban, wharf, and urban areas. The painting contains more than 600 representations of people, dozens of houses, and a view of the picturesque trees and nature of the setting. The content reveals the lifestyle of all stratums of society as well as the different economic activities in the rural areas and the city.

“This painting is not only a miracle in the arts, but also a great reference to the culture of ancient China,” said Jianhua Lin, the vice president of Peking University. “It is comparable to the west’s Mona Lisa in importance to the Chinese people.”

However, because the ancient painting is often removed from public display and protected because of its artistic and cultural heritage significance, visitors have not been able to appreciate it in up-close detail. That is, until the development of the interactive browser system, which provides a real-time, user-friendly interface and a realistic audio-visual experience.

Digital Techniques Reproduce Ancient Paintings

“The technique can also be applied to other paintings and even vases...It will enable artists and art historians to study works at a level never before accomplished.”

— Yingqing Xu, lead researcher, Microsoft Research Asia, Beijing

 

The interactive browser system was jointly developed by the Palace Museum, Microsoft Research Asia, and Peking University. The exhibition, open to the public for free at the Palace Museum in Beijing since April 2010, shows high-resolution images of the painting, accompanied by hundreds of voice recordings and background sounds. 

The browser system presents a three-dimensional (3-D), detailed, interactive, and multi-layered digital rendition of the painting, with annotated gigapixel images and HD View developed by Microsoft Research by using Microsoft Silverlight as a platform. The collaborators believe this 3-D simulation—which provides different multimedia experiences (depending upon the viewer’s “position” in the painting) without any loss of the original painting’s clarity—is the first-ever such breakthrough in the history of museum exhibitions.

Enhanced objects with embedded voice annotation in the paintingEnhanced objects with embedded voice annotation in the painting

“When a visitor points to any part of the painting on the multi-touch screen, it activates the corresponding story and realistic stereo audio recordings, including voices and sounds of the street,” said Xin Ma, the program leader at Microsoft Research Asia.

Example of audio annotations positioned in the 3-D layout of an imageExample of audio annotations positioned in the 3-D layout of an imageMost objects in the painting are enhanced by auditory and textual annotations. The hidden 3-D layout of all the objects is first inferred from the two-dimensional (2-D) picture. Visitors experience the exhibition through a human-computer interaction (HCI) interface, with the 3-D layout as the core. The system intelligently presents the hybrid-media annotations of the objects and presents them in a way that the viewer can experience. The 3-D layout acts as a mixture modulator. The HCI interface is planned to be realized on the Microsoft Surface platform or Window 7 in future iterations.

“It provides a virtual three-dimensional walkthrough of the painting with sound,” said Yingqing Xu, a lead researcher at Microsoft Research Asia in Beijing, China. “The level of detail for a work of art like this has never been created before. The technique can also be applied to other paintings and even vases and the Terracotta Warriors. It will enable artists and art historians to study works at a level never before accomplished.”

New Era of Cultural Heritage

Before the browser system was produced, museum visitors and researchers could not fully appreciate famous ancient artworks because the easily damaged pieces must be protected from oxidation and light, making them less accessible (or inaccessible) to the public. A Microsoft Research Asia eHeritage project, the interactive browser system helps preserve and make cultural heritage accessible to the public through the interactive exhibition—without endangering the actual ancient artifacts.

Installed in the Hall of Martial Valor of Palace Museum in Beijing, the exhibition attracts visitors daily. Its user-friendly interface enables visitors to experience a period of ancient China in a way that can help them better understand and appreciate Chinese culture and heritage.