Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Web Search—With a Little Help from Your Friends
By Rob Knies
May 15, 2007 5:00 PM PT

Who knows you better than anybody?

Your friends and family, of course.

Who is best-positioned to understand your wants and needs?

Your closest acquaintances, naturally.

Doesn’t it make sense, therefore, to take the opinions of your dearest companions into account when attempting a Web search?

If you agree, count Dahlia Malkhi, Lidong Zhou, and Coby Fernandess among your friends.

Malkhi and Zhou, from Microsoft Research Silicon Valley, are working on a research project called Nocturnal that aims to use an established online community to provide a mechanism for giving reviews and recommendations from your social circle a higher priority when you search the Web. They received invaluable assistance from former Microsoft Research intern Fernandess, the main system architect and developer on the project.

“Nocturnal,” explains Malkhi, a senior researcher at the Silicon Valley lab, “is a collaborative, personalized Web-search tool built of two essentially independent components. One is a generic infrastructure for sharing information in a very large network, built over the Windows Live™ Messenger network structure. The idea is to propagate information among social contacts using existing instant-messaging infrastructure. You could use this for sharing any type of information: pictures, files, history—whatever.

“The second component is an enhanced Web-search tool. We have our own Internet Explorer® toolbar that you can search the Web through. It retrieves information from Live Search but reorganizes it and sorts it based on information that you’ve shared about recommendations about Web sites with your Messenger contacts.”

In addition, Nocturnal lets users tag pages with keywords. When a user searches the Web, Nocturnal retrieves pages visited by the user’s social community based on a match between the search keyword and tags, complementing the reorganized Live Search results.

This combination, using one of the largest social networks extant, that of Windows Live Messenger, can provide improved, personally relevant Internet search results.

“From a general search engine, you get rankings that are not customized to your needs,” says Zhou, a researcher with Microsoft Research Silicon Valley. “You might look for particular Web pages on particular topics. It’s likely that your friends or your friends’ friends share common interests with you, so the pages they like are likely to be more interesting to you.”

And, as he indicates, Nocturnal enables users to select the size of the social circle with which information and recommendations are shared. It can be limited to one’s set of friends, as determined by the people on your Messenger buddy list. It can be your friends’ friends: everybody on the buddy lists of the people on your buddy lists. It can be extended to your friends’ friends’ friends and so on. The user chooses the number of “hops” to utilize.

The project got started in the summer of 2005. Fernandess was responsible for most of the implementation of the prototype, and his many nightly hours spent debugging the technology led directly to the project’s name. He worked as a project consultant during the 2005-’06 academic year and returned to the lab for a second internship last winter.

Fellow Microsoft Research Silicon Valley researcher Udi Wieder and Fabian Kuhn, a former postdoctoral researcher at the lab, also helped bring the project to life.

“Nocturnal,” Malkhi says, “is a culmination of two visions. One is that there is a shift in the way users use the Internet today. You see much more powerful workstations on people’s desktops. They’re persistent, they’re online all the time, and they’re not only consuming information or using information, they’re also servicing other users and collaborating and exchanging information. They’re putting out stuff, not just consuming stuff.

“Nocturnal is a manifestation of this completely decentralized system that doesn’t employ any centralized servers, doesn’t store anything on a persistent, centralized server. It uses the power of users at the end points to share and collaborate. It is, essentially, a cost-free operation.”

In that regard, the system is similar to peer-to-peer networks, in that the computing work is performed by the PCs at the edge of the network. But then there’s the other vision upon which Nocturnal relies.

“Everybody is talking about the power of social networks, the power of social communities, the power of experts,” Malkhi says. “But there is a real difficulty in bootstrapping these communities or these expert sites. It’s very difficult to elicit information from users about their preferences, to get them to subscribe to the particular service that you want, to initiate such services.”

Unless, of course, you can utilize an already robust, existing service, such as Windows Live Messenger.

“We saw that there is a very large social network out there that’s not being utilized for anything other than instant communication,” she continues. “We put these two very powerful trends together in our own research and figured that this should be able to tremendously enhance your search experience. You’ll be able to filter out all junk, all spam, all intruders, and just use the information that we can pull together in a personalized, completely decentralized manner.”

The result is a personalized search experience, fostered by the social relationships existing within a collection of friends, that provides better results amid an environment of trust and improved privacy.

“The idea,” Zhou says, “is to get this kind of personalization into the search experience. We believe that’s a very powerful paradigm. We believe it’s what the next generation of search engines should be doing. It’s hard to support such functionalities in a traditional and centralized way. But the paradigm fits naturally with a decentralized social network.

“Another important issue is trust and privacy on the Internet. Anything you do ultimately has the issue of whether you trust the source of information and whether you want to share information with strangers. We believe a social network achieves a delicate balance between trust and privacy, because you tend to trust your friends and you are willing or inclined to share information with your friends. In Nocturnal, we also let users control what information they disclose to their friends and how far such information is propagated in the social network.”

Adds Malkhi: “It’s the digital analog of what you do in your daily life. You want to buy a bicycle—how do you know which store to use? You ask your neighbor. Even if they just moved to your street, their recommendation carries a lot of weight, because it comes incentive-free.”

Many people, when they hear about the concept behind Nocturnal, immediately wonder about putting the gathered information on a centralized server.

“Everybody raises the question,” Malkhi stipulates. “Why not do everything centralized? Why not just ship all the data to a central server, and we can do everything better and easier there?

“That’s kind of a barrier. As soon as you centralize things, you globalize them, and you lose the individual point of view. We have the personalized, individualized point of view that you can emulate centrally for each user, but you might as well do it at the end point.

“The other big issue is security. As soon as you centralize any service, you become a honey pot for spammers, and there’s no way to distinguish the good from the bad. With a social network, that’s exactly where you have more control.”

Members of a social network self-select for people they trust. Malevolent forces need not apply, and Nocturnal benefits.

The technology offers other functionality that could prove invaluable. Users can rate Web pages and share the ratings with their friends. The friends then can determine how much to value the ratings they receive.

“Suppose I have a lot of common interests with Dahlia,” Zhou says. “This will be reflected in the high weights we assign to ratings from each other. And when I browse the Web and I like some page, I can rate that page—say, give it five stars or four stars. All those numbers will eventually influence how the search results are ordered and how they are presented to me and my friends.”

Users also can tag information in any way they desire, including by how many hops away it originated. Information and grades can be shared, but the relative weight they receive is completely under the user’s control.

“Everybody,” Malkhi says, “has their own personalized view of that information. Even though we share a lot in common, no two users have exactly the same data in their files. Information that came from three hops away from me may be four hops away from you, because you got it through me and you may place less trust in it, or more trust in it, depending on your preferences.”

Nocturnal is fully operational on Windows® XP and Windows Vista™ and has been installed internally for testing.

“We are looking for someone in the company to seize Nocturnal,” Malkhi says, “and incorporate it as the means for pushing Web-search functionality to the edges of the network.”

The concept behind Nocturnal remains alluring.

“You can,” Zhou states, “find more relevant information than you can find otherwise.”

Malkhi agrees—and cites additional benefits.

“It’s no hassle,” she says. “You don’t need to do anything. You don’t need to subscribe to anything new. It helps protect your privacy, and it’s absolutely effective.”

Trust, discretion, and support—hmm, aren’t those what friends are for?