Where to Go: Interpreting Natural Directions Using Global Inference

Giving and interpreting directions is almost a daily activity, and as such, is a necessary system component of household robots or autonomous wheelchairs. In particular, we would like to be able to parse naturally given human verbal directions, and compute the likely corresponding sequence of physical regions. We cast this as an inference problem and use a modification of Hidden Markov Models to compute the most likely physical paths corresponding to a single set of directions. We evaluated this model on a campus office floor and compared it to human performance. Though the original model works fairly well for the first environment, we have since tested it in much larger office environments. To handle such environments, I will describe two directions of current work into improving autonomous direction inference. The first consists of extending the model to handle a more complicated mapping between verbal directions and physical regions to tackle some of the common challenges found in verbal directions, such as failure to describe some regions along the path. The second effort is on autonomously generating questions to further improve the accuracy of the inferred region sequence. Initial results from both directions make us optimistic about the potential benefit of posing direction understanding as inference. This is joint work with Thomas Kollar, Dimitar Simeonov, Sachi Hemachandra, Yuan Wei, and Nicholas Roy.