Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Deep Belief Network based Semantic Taggers for Spoken Language Understanding

Anoop Deoras and Ruhi Sarikaya

Abstract

This paper investigates the use of deep belief networks (DBN) for semantic tagging, a sequence classification task, in spoken language understanding (SLU). We evaluate the performance of the DBN based sequence tagger on the well-studied ATIS task and compare our technique to conditional random fields (CRF), a state-of-the-art classifier for sequence classification. In conjunction with lexical and named entity features, we also use dependency parser based syntactic features and part of speech (POS) tags [1]. Under both noisy conditions (output of automatic speech recognition system) and clean conditions (manual transcriptions), our deep belief network based sequence tagger outperforms the best CRF based system described in [1] by an absolute 2% and 1% F-measure, respectively. Upon carrying out an analysis of cases where CRF and DBN models made different predictions, we observed that when discrete features are projected onto a continuous space during neural network training, the model learns to cluster these features leading to its improved generalization capability, relative to a CRF model, especially in cases where some features are either missing or noisy.

Details

Publication typeInproceedings
Published inISCA Interspeech
PublisherISCA
> Publications > Deep Belief Network based Semantic Taggers for Spoken Language Understanding