On the use of Words and N-grams for Chinese Information Retrieval

Jian Zhang, Jian-Yun Nie, Jiangfeng Gao, and Zhou Ming

Abstract

In the processing of Chinese documents and queries in information retrieval (IR), one has to identify the units that are used as indexes. Words and n-grams have been used as indexes in several previous studies, which showed that both kinds of indexes lead to comparable IR performances. In this study, we carry out more experiments on different ways to segment documents and queries, and to combine words with n-grams. Our experiments show that a combination of the longest-matching algorithm with single characters is the best choice.

Details

Publication typeInproceedings
> Publications > On the use of Words and N-grams for Chinese Information Retrieval