A large-scale evaluation and analysis of personalized search strategies

[[Note that Zhicheng Dou's email address has been changed to zhichdou at microsoft.com now. The old email douzc@yahoo.com.cn is unavailable. Sorry for this.]]

Although personalized search has been proposed for many years and many personalization strategies have been investigated, it is still unclear whether personalization is consistently effective on different queries for different users, and under different search contexts. In this paper, we study this problem and provide some preliminary conclusions. We present a large-scale evaluation framework for personalized search based on query logs, and then evaluate five personalized search strategies (including two click-based and three profile-based ones) using 12-day MSN query logs. By analyzing the results, we reveal that personalized search has significant improvement over common web search on some queries but it has little effect on other queries (e.g., queries with small click entropy). It even harms search accuracy under some situations. Furthermore, we show that straightforward click-based personalization strategies perform consistently and considerably well, while profile-based ones are unstable in our experiments. We also reveal that both longterm and short-term contexts are very important in improving search performance for profile-based personalized search strategies.

wwwfp495-dou.pdf
PDF file

In  WWW '07: Proceedings of the 16th international conference on World Wide Web

Publisher  ACM Press
Copyright is held by the International World Wide Web Conference Committee (IW3C2). Distribution of these papers is limited to classroom use, and personal use by others.

Details

TypeInproceedings
URLhttp://doi.acm.org/10.1145/1242572.1242651
Pages581–590
ISBN978-1-59593-654-7
AddressNew York, NY, USA
> Publications > A large-scale evaluation and analysis of personalized search strategies