By Rob Knies
October 25, 2010 1:00 PM PT
On one side of the room, researchers explained how to enhance a seminal Chinese painting from almost a millennium past. Nearby, a pair of projects explored ways to make the current fascination with Twitter more useful for users. And then there were university collaborators focused on ways to make humans even more productive and aware of their surroundings in years to come.
“Innovation Day is a technology extravaganza,” Hsiao-Wuen Hon, Microsoft distinguished scientist and managing director of Microsoft Research Asia. “It’s an opportunity for us to demonstrate how we are turning ideas into reality in ways that solve our most challenging problems. For 12 years, we have harnessed the best talent in Asia and worked with our academic partners to push forward the state of the art of computer science.”
The 22 demos on display during Innovation Day were a highlight of three days of collaborative events in which thought leaders from Asia Pacific universities and research institutes gathered in Shanghai to hear and learn of the latest research efforts from Microsoft Research and its partner institutions. Also presented during the Oct. 18-20 whirlwind were the Microsoft Research Asia Faculty Summit 2010 and the 12th annual Computing in the 21st Century Conference.
Representatives from academia and other Microsoft Research facilities also were present to help underscore current efforts to define the future of computing.
Innovation Day demos were subdivided into three themes: Client + Cloud, Natural User Interfaces, and How Computer Technology Contributes to Human Society, all pointing to ways in which groundbreaking technologies will transform the way people work, enjoy entertainment, and advance toward a more advantageous future.
The variety of ingenious technologies and strategies displayed during the day was broad and inspiring, as can be deduced from a few samplings:
Life Along the Bian River During the Pure Brightness Festival is one of the most famous artworks in Chinese history. Painted by Zeduan Zhang and measuring almost 10 inches tall and more than 17 feet wide, it details, via a technique called moving focus, a series of fascinating, dynamic scenes along a canal in Bianjing, the capital of the Northern Song Dynasty a thousand years ago.
You might be forgiven for assuming that such an ancient treasure would hold limited appeal for modern-day observers. But that—thanks to researchers from Microsoft Research Asia, Beijing’s National Palace Museum, and Peking University—couldn’t be further from the truth, as visitors to an installation within Beijing’s Forbidden City can attest.
“We are working together to develop an immersive, interactive, multimedia system to show a very famous painting,” says Ying-Qing Xu, lead researcher in Microsoft Research Asia’s Internet Graphics Group. “It may be the most expensive painting in Chinese history.”
It’s certainly one of the most engaging. The painting consists of three expansive scenes. The rightmost features a tranquil village, with a collection of cottages and footpaths anchored by a stand of willow trees. An arched bridge over the river is the focus of the central scene, with a stream of humanity—some pulling carts, some on horseback, some selling wares—passing overhead while boatmen below struggle to keep their vessel from being swept downstream. At left, a bustling cityscape depicts a broad, intensely detailed swath of urban life while a procession of camels passes through a city gate.
The researchers on the project—which also include Qiong Li, Wei Ma, and Yizhou Wang—have digitized the huge, gigapixel image using the Microsoft Image Composite Editor, an advanced panoramic image stitcher from Microsoft Research Redmond. Many of the specific, Silverlight-enabled interactions within the painting are supplemented with audio dialogues, with which users can interact using a 63-inch multitouch screen in a 5.1 surround-sound environment in a special room within the Forbidden City.
“We provide more than 50 stories and 700 dialogues,” Xu explains, “based on the history research of the Palace Museum researchers.”
This project is inspired by an earlier effort called Animating Gigapixel Images, a collaboration with Michael Cohen, a principal researcher in the Interactive Visual Media Group at Microsoft Research Redmond. That work was demonstrated in Dunhuang, China, in June 2009 during Microsoft Research Asia’s eHeritage Workshop, and it caught the eye of then-director of the Information Department of the National Palace Museum, Chui Hu.
“He saw this demo and thought it was really cool,” Xu recalls. “He wondered if we could do something like this to ancient Chinese painting.”
Traditional Chinese painting differs from that of western painting. Most western paintings adopt a pinhole-perspective model with a fixed viewpoint. Ancient Chinese paintings do not have a fixed focus. Instead, they deploy moving focus, because the painter’s viewpoint varies.
“To do annotation, we must know the object’s depth in the 3-D space,” Xu says. “But computing the object’s depth in moving-focus-based Chinese painting is a challenge.”
Digitization work on the painting began earlier this year. National Palace Museum research provided direction on effective ways to annotate the moving-focus perspective of the image, including the historical context.
The resultant installation has generated plenty of interest from Chinese reporters and television accounts. For Xu and colleagues, the attention has been gratifying.
“It’s a new way for a museum to show this very expensive artwork,” he enthuses. “This is a commitment from Microsoft to work for society and the Chinese culture, the heritage of our country, so I’m very happy.”
A second Innovation Day demo set its sights squarely on the future—with inspiration from Arnold Schwarzenegger.
Aided Eyes: Eye Sensing for Enhancing Everyday Life, a project from the University of Tokyo, features a small eye sensor, implanted in a pair of spectacles, that detects gaze direction and eye movement. It holds promise for assisting the disabled and improving a user’s short-term memory.
“We are creating a new, wearable device that can support human memory,” says Jun Rekimoto, a professor in Interfaculty Initiative in Information Studies at the University of Tokyo, “by using gaze detection and eye-movement technology.”
Aided Eyes measures eye movement via the use of phototransistors, infrared LEDs, and a miniature video camera. Gaze-indexed images are captured for recognizing focused objects, eye contact with other people, and text read by the user.
How did the project come about? Credit Ah-nold.
“I am inspired by science-fiction movies,” Rekimoto smiles, “like Terminator 2 and science-fiction animation.”
The Terminator’s glowing eye-processors provided particular motivation for the Aided Eyes project, which began in mid-2009.
“I want to make the human eye better,” Rekimoto says. “This is still very much at an early stage. We are creating a prototype input device, and we want to combine this with information retrieval and make it a complete system.”
That system, which might take another two or three years to refine, could even help the user to understand the emotional status of the people he or she encounters.
“We want to understand how a human can be enhanced by using technology,” Rekimoto concludes. “I am quite happy in developing new things that can create a new effect on humans.”
Visitors to Innovation Day had their choice of a pair of demos that address one of today’s hottest technologies, Twitter. Upon entering the room in which the event was held, an immediate turn left led the entrant to QuickView: Semantic Search of Tweets, while a sharp right turn would encounter Project Emporia.
At first glance, the projects might seem similar. Both are based on analysis of the voluminous stream of tweets from Twitter’s estimated 145 million users worldwide. But the manner in which the research proceeds varies significantly.
QuickView, a project by Ming Zhou, Xiaohua Liu, and Long Jiang of Microsoft Research Asia, is focused on the search scenario, extracting key information to provide advanced data-mining capabilities and semantic search.
“Our technology,” says Zhou, a senior researcher and research manager of the Natural Language Computing Group, “builds a comprehensive index—what we call semantic indexing—including named entities: people’s names, product names, or opinions toward some topic.
“We can support semantic search. With that, people can search what is happening today—the hot topics. What is a famous person saying? What is the prevailing grassroots opinion about a new product? People can leverage numerous tweets to get fresh information on the topic he or she is interested in.”
With Twitter’s tens of millions of daily tweets, the volume of information the service offers is enormous. That’s the good news. But the flip side is that the never-ending stream of content can include pointless babble, incomprehensible acronyms, spam, and offensive remarks. It’s a challenge to separate the wheat from the chaff.
QuickView is an attempt to help.
“People don’t want to read tweets,” Zhou confirms, “because they’re very noisy and very ungrammatical. Users would like to know the overview about some collection of tweets, and semantic analysis can enable a computer to understand the tweets on some topic and provide overview information, positive or negative, on the key topics.”
Actually, the noise inherent in the Twitter stream is part of what makes the QuickView research compelling. Zhou and his colleagues are interested in the natural-language consequences of extracting useful information from the noisy data. That, in turn, could help bolster the Bing search experience.
“What’s the best way to consume Twitter information?” Zhou asks. “Bing has a social-search team in Silicon Valley, developing Twitter search and Facebook search, and we want to contribute to their product.”
The QuickView project evolved a year ago from earlier work on social-media text mining, and the effort has proved rewarding.
“I’m very happy with the progress,” Zhou says. “It’s a very challenging project, but I am confident that this is the right direction for Microsoft Research.”
Project Emporia, too, attempts to present useful information by digesting the Twitter stream, but uses an alternative approach. Instead of semantic analysis, this effort—a joint endeavor between FUSE Labs U.K. and Microsoft Research Cambridge—represents a personalized news-recommender system based on the power of Twitter.
“On Twitter, people point to URLs, recommend them in their tweets,” says Thore Graepel, senior researcher in the Online Services and Advertising group at Microsoft Research Cambridge. “We pick up those URLs, rank them by popularity, and categorize them, so people can look for the URLs in the category they like.
“Furthermore, if people want to sign into the service, they can provide feedback for the articles or links we present to them, positive or negative, and the system can then learn their individual preferences and, next time, give them even better, more personalized recommendations for articles to read.”
Project Emporia began at the start of 2010 and combines a browser-based interface and services from FUSE Labs with Matchbox, a large-scale recommendation system from Microsoft Research Cambridge. The result is a service in which you can benefit from other users’ feedback, and they can benefit from yours.
“What Twitter can give you is general popularity,” Graepel says, “and what a recommender system can give you is that individual relevance that comes from your rating of the content that is presented to you. It’s that combination that prompted this project.
“As a collaboration, it has been very nice. We take machine-learning technology developed at Microsoft Research and use that in a project in which FUSE Labs brings in the social component. It has been good fun.”