A Discriminative Model Based Entity Dictionary Weighting Approach for Spoken Language Understanding

Spoken language understanding (SLU) systems use various features to detect the domain, intent and semantic slots of a query. In addition to n-grams, features generated from entity dictionaries are often used in model training. Clean or properly weighted dictionaries are critical to improve model’s coverage and accuracy for unseen entities during test time. However, clean dictionaries are hard to obtain for some applications since they are automatically generated and can potentially contain millions of entries (e.g. movie names, person names) with significant noise in them. This paper proposes a discriminative model based approach to weight entities in noisy dictionaries using multiple knowledge resources. The model makes use of features extracted from query click logs, knowledge graph and live search results for accurate entity weighting. Experiments for both intent detection and slots tagging tasks in entertainment search covering five domains show significant gains over the baselines.