NLify: Lightweight Spoken Natural Language Interfaces via Exhaustive Paraphrasing

This paper presents the design and implementation of a programming

system that enables third-party developers to add

spoken natural language (SNL) interfaces to standalone mobile

applications. The central challenge is to create statistical

recognition models that are accurate and resource-efficient

in the face of the variety of natural language, while requiring

little specialized knowledge from developers. We show

that given a few examples from the developer, it is possible

to elicit comprehensive sets of paraphrases of the examples

using internet crowds. The exhaustive nature of these paraphrases

allows us to use relatively simple, automatically derived

statistical models for speech and language understanding

that perform well without per-application tuning. We have

realized our design fully as an extension to the Visual Studio

IDE. Based on a new benchmark dataset with 3500 spoken

instances of 27 commands from 20 subjects and a small developer

study, we establish the promise of our approach and

the impact of various design choices.

PDF file

In  Proceedings of UbiComp 2013

Publisher  ACM
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from


> Publications > NLify: Lightweight Spoken Natural Language Interfaces via Exhaustive Paraphrasing