Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Improving Similarity Measures for Short Segments of Text

Wen-tau Yih and Christopher Meek

Abstract

In this paper we improve previous work on measuring the similarity of short segments of text in two ways. First, we introduce a Web-relevance similarity measure and demonstrate its effectiveness. This measure extends the Web-kernel similarity function introduced by Sahami and Heilman (2006) by using relevance weighted inner-product of term occurrences rather than TF×IDF. Second, we show that one can further improve the accuracy of similarity measures by using a machine learning approach. Our methods outperform other state-of-the-art methods in a general query suggestion task for multiple evaluation metrics.

Details

Publication typeInproceedings
Published inProceedings of AAAI 2007
PublisherAmerican Association for Artificial Intelligence
> Publications > Improving Similarity Measures for Short Segments of Text