Treasured Chinese Painting Comes to Life Through Technology

Published

Explore how a treasured Chinese painting comes to life through technology

As Chinese citizens celebrate Chinese Cultural Heritage Day, an annual event that takes place on the second Saturday in June, thousands will likely take a virtual walk along a river and a stroll through time, as they digitally explore one of their country’s most famous paintings. Courtesy of a collaborative effort among Microsoft Research Asia (opens in new tab), the Beijing Palace Museum (opens in new tab), and Peking University (opens in new tab), visitors to the museum can experience a detailed, interactive digital representation of Along the River during the Qing-Ming Festival, one of China’s most treasured scroll paintings.

The centuries-old, original ink-on-silk painting is only rarely displayed publicly; most of the time it’s safely stored for preservation. A scroll that measures more than 5 meters long and approximately 25 centimeters high when completely unfurled, it depicts a variety of panoramas of daily life in the Northern Song Dynasty (960 to 1127): from farmers in their fields, to boatmen and shopkeepers plying their trades, to government officials collecting taxes. Unlike traditional Western paintings, which have a single focus, Along the River during the Qing-Ming Festival employs the “moving focus” technique of Chinese scroll painting, which presents multiple focal points as the viewer works his or her way down the scroll.

Spotlight: On-demand video

AI Explainer: Foundation models ​and the next era of AI

Explore how the transformer architecture, larger models and more data, and in-context learning have helped advance AI from perception to creation.

The three-dimensional (3-D) digital representation of the painting allows viewers to pan, zoom, and pause as they explore the richly detailed artwork by using a multi-touch screen. As viewers navigate the digital painting, the software uses their actions to calculate their viewpoint. Enhancing the experience, stereophonic audio has been added, with hundreds of voices creating dialogues that track to the actions in the painting. The scripts were created by experts from the Beijing Palace Museum, and the dialogue uses a dialect of Mandarin that reflects the speech of the Northern Song Dynasty. The audio also includes ambient sounds of nature and city life, linked to the corresponding place in the painting.

This tour-de-force of software development and 3-D modeling was the work of Ying-Qing Xu, a lead researcher with Microsoft Research Asia in Beijing, who developed the detailed, interactive, and multi-layered digital rendition of the painting by using annotated gigapixel images and HD View (opens in new tab) on a Microsoft Silverlight (opens in new tab) platform. This work is part of Microsoft Research’s eHeritage project, which facilitates collaborative initiatives between Microsoft Research and academia that use technology to preserve and display the cultural heritage of the Asia-Pacific region.

The digital exhibition is open daily to the public for free, allowing visitors to experience a period of China in a way that can help them better understand ancient Chinese culture and heritage. “This is having a profound effect on museums globally and putting Beijing firmly on the map of cutting-edge development,” said Chui Hu, former director of information center at the Palace Museum in Beijing.

—Xin Ma, University Relations Program Manager, Microsoft Research Connections Asia

Learn More