Two of the five sunflowers have already withered, one is still budding and two are open and vibrant like flames. "Vase of Five Sunflowers" is highly representative of the 11 paintings Vincent van Gogh created of his favorite plant. "Only unclear photos of this masterpiece can be found on the Internet today, though, as the original copy was destroyed during World War II," said Dr. Moshe Ben-Ezra, Lead Researcher of Visual Computing Group at Microsoft Research Asia, with a tone of regret when we discussed the relationship between digital cameras and cultural heritage.
Unlike most of the other researchers at Microsoft Research Asia, Moshe spends many of his office hours in a dark room designed specifically for the extra-large camera he works with. After digital enhancement, pictures generated by the camera allow people to examine the microscopic world. "I hope to be able to pass these historic items on to future generations,” he said, “not only to protect, but also to provide immersive details and a real impact."
Researchers from Internet Graphics Group, Visual Computing Group, Natural Language Computing Group, Speech Group, and Wireless & Networking Group at Microsoft Research Asia share Moshe’s aspiration to "rescue" these treasures, which may soon disappear or be destroyed. They proposed a number of research subjects on the protection and presentation of cultural relics, working with universities in the Asia-Pacific region.
"Our technologies could contribute and be of great assistance to the entire human society, so we chose to do this," said Xin Ma, Manager of the University Relations Group at Microsoft Research Asia. Research projects at Microsoft not only involve the protection and preservation of world heritage sites and cultural relics — including the use of Computer Generated Appearance Modeling techniques, Terra-cotta Warriors Display System and the digital preservation of Angkor Wat, to name a few — but also covers intangible elements from our cultural past. Microsoft Research Asia has chosen to dig into the past for a better future.
"Science and technology are the basis of the protection of cultural heritage," said Jinshi Fan, President of the Dunhuang Academy. Like other research institutions seeking digitalized solutions, Microsoft Research Asia as an enterprise-backed research facility also puts all its effort into making its technologies available for the purposes of cultural heritage protection.
The University Relations Group of Microsoft Research Asia officially launched in 2008 the eHeritage program. Apart from applying cutting-edge computing technologies to the protection of cultural relics, it aims to gain better understanding of their origin, formation and development from a digital perspective.
In addition to using wireless and sensor technology, natural language processing, Internet search, computer vision and graphics and multimedia for cultural heritage protection purposes, researchers at Microsoft Research Asia also hope to bring mature technologies to developing countries at a lower cost to aide in cultural preservation.
"For instance, special devices may be needed to conduct a highly detailed, 360-degree, 3-D sampling on an extremely rare or irregularly structured artifact that allows absolutely no physical contact or change of location,” said Xin Ma. In this situation the first step in the digitalization of cultural relics — namely, data collection — would seem nearly impossible.
Moshe showed me a special camera completed a few months ago exclusively for this task. It is an eye into history that is capable of capturing the minute details of each brush stroke and of every color.
During his job interview with Microsoft Research Asia, Moshe mentioned an idea that he had conceived many years ago to develop a special camera. Moshe had spent two years in Columbia University working on this idea, but it was at Microsoft Research Asia that his dream finally came true.
Moshe and his colleagues built a camera specifically designed for digital art documentation in museums and electronic cultural heritage. It was the first camera to successfully combine very large format lens to work with a digital sensor. This device, originally designed to shoot films, can generate images of over one gigapixels (1,000 megapixels) each, while commercial digital cameras today are only capable of images of less than 160 megapixels. Most importantly, the lens of this camera does not need to move, which makes it ideal for taking pictures of objects with 3-D texture. Exposures under different lighting circumstances are synthesized by computers to obtain 3-D images.
"Image size is not the only criteria by which one can judge a camera. This one allows a contrast ratio greater than 1:3,000,000, 300 times better than the best displays available today. Its resolution is great enough to enable us to see over 30 lines on a needle point,” said Moshe. “This is six times as sharp as normal human vision.
Powerful hardware products realize their full potential on top of an even more powerful software platform at Microsoft. With the right software, you can easily access data that is well beyond the capacity of commercial cameras.
Besides helping to save and study cultural heritage, Microsoft Research Asia hopes to display the mysteries of history in a more accessible and interesting way.
Abiding by the principles of historical objectivity, Microsoft Research Asia works closely with research institutions to apply pioneering computing technologies to the confirmation, protection, preservation, and popularization of heritage and culture. At the same time, Microsoft Research Asia enables the general public to access cultural relics that are packed-away securely in museum vaults, thus creating a new dialogue between olden and modern times.
The audio-video display system based on "Qingming Riverside" (a well-known traditional painting created during the Ming Dynasty) is a joint heritage protection project of Microsoft Research Asia, Peking University and the Forbidden City Museum that disseminates historical knowledge in an entertaining manner. "The Forbidden City is a mysterious place for an average person because it used to be a royal palace with lots of treasure collections. Side by side with the Forbidden City Museum, we are going to figure out better ways to showcase these invaluable collections," Xin Ma said.
Through communication with culture heritage experts as well as analysis of academic literature, Microsoft Research Asia has managed to revive the figures in the painting.
Through multi-touch technology, viewers can freely browse the entire scroll, and immerse themselves in the painting’s scenarios through an interactive background sound system. "With this system, we hope to enable audiences to actually hear peddlers hawking their wares, the chants of boatmen, or the graceful whispers of a well-to-do young lady in the street,” said Xin Ma. “Our 3-D technology is able to precisely project sounds to figures and scenarios where they belong, in an effort to fully reflect the characteristics of the multiple perspectives in the painting. This project is pretty challenging due to the massive data and numerous scenarios contained in the scroll as well as the difficulty in predicting what the audience appreciates.”
"Development of technology must be a few steps ahead of application, but it should also match the needs of users,” Ma added. “We wish that someday in the future, these mysterious yet valuable relics will be made more easily accessible, and that cultural heritage will be digitally presented on your mobile phone.” Ma would like to see more cooperative projects among Microsoft Research Asia, universities and cultural heritage authorities. In her words, what matters about a research project is not its size, but its quality and purpose.