Deep Learning for Natural Language Processing and Related Applications (Tutorial at ICASSP)

Deep learning techniques have enjoyed tremendous success in the speech and language processing community in recent years (especially since 2011), establishing new state-of-the-art performance in speech recognition, language modeling, and some natural language processing tasks. The focus of this tutorial is on deep learning approaches to problems in language or text processing, with particular emphasis on important applications including spoken language understanding (SLU), machine translation (MT), and semantic information retrieval (IR) from text.

In this tutorial, we first survey the latest deep learning technology, presenting both theoretical and practical perspectives that are most relevant to our topic. We plan to cover common methods of deep neural networks and more advanced methods of recurrent, recursive, stacking, and convolutional networks. Next, we review general problems and tasks in text/language processing, and underline the distinct properties that differentiate language processing from other tasks such as speech and image object recognition. More importantly, we highlight the general issues of language processing, and elaborate on how new deep learning technologies are proposed and fundamentally address these issues. We then place particular emphasis on three important applications:1) spoken language understanding, 2) machine translation, and 3) semantic information retrieval from text. For each of the three tasks we discuss what particular architectures of deep learning models are suitable given the nature of the task, and how learning can be performed efficiently and effectively using end-to-end optimization strategies.

Beyond providing a systematic tutorial of the general theory, we also present hands-on experience in building state-of-the-art SLU/MT/IR systems. In the tutorial, we will share our practice with concrete examples drawn from our first-hand experience in major research benchmarks and some industrial scale applications which we have been working on extensively in recent years.