Implementation of a Multimodal Dialog System Using Extended Markup Languages

In this paper, we describe an implementation of a plan-based multimodal dialog system using the extensible markup language (XML). The dialog manager receives semantic objects representing the user’s utterance at the end of each user’s turn. We define a semantic markup language (SML), based on XML, to describe these semantic objects. Following the principles of XML Schema, we define the schema of SML in another XML called semantic definition language (SDL). In addition to supporting many discourse and dialog features, SDL is also designed to represent the domain knowledge via the application schema and the hierarchy of the semantic objects. We show that, with a thoughtful design in SDL, SML can be expressive enough that the behavior of a dialog planner can be fully specified in the extensible stylesheet language (XSL), a standardized language with a logical programming model that is most popular for implementing intelligent systems.