Lattice-Based Discriminative Training: Theory and Practice

Lattice-based discriminative training techniques such as MMI and MPE have been increasingly widely used in recent years. I will review these model based discriminative training technique and also the newer feature-based techniques such as fMPE. I will discuss some of the practical issues that are relevant to discriminative training, such as lattice generation, lattice depth and quality, probability scaling, I-smoothing, language models, alignment consistency, and various other issues for feature-based discriminative training, and will discuss more recent improvements such as frame-weighted MPE (MPFE), and give an overview of some recent unrelated work that I have been doing.

Speaker Details

Dan Povey received his Bachelor’s degree in Natural Sciences from Cambridge University, England, and his Master’s and PhD in speech recognition from the Engineering Department of Cambridge University. His PhD topic was “Discriminative Training for Large Vocabulary Speech Recognition” in which various practical improvements necessary for large-scale MMI training were introduced or popularized (e.g. modified smoothing constants for Extended Baum-Welch equations; I-smoothing; probability scaling; unigram language model) and in which a new and generally better objective function called Minimum Phone Error (MPE) was introduced. Since 2003 he has been working at IBM’s T.J. Watson Research Center in Yorktown Heights, NY and has mainly been focusing on acoustic modeling techniques; while at IBM he introduced a feature-based form of MPE (fMPE) which has been widely used in IBM and has since been used in various modified forms at a number of other sites. His interests include world domination and kites.

Date:
Speakers:
Dan Povey
Affiliation:
IBM's T.J. Watson Research Center in Yorktown Heights, NY