Gradient Boosting Learning of Hidden Markov Models

Rusheng Hu, Xiaolong Li, and Yunxin Zhao


In this paper, we present a new training algorithm, gradient boosting learning, for Gaussian mixture density (GMD) based acoustic models. This algorithm is based on a function approximation scheme from the perspective of optimization in function space rather than parameter space, i.e., stage-wise additive expansions of GMDs are used to search for optimal models instead of gradient descent optimization of model parameters. In the proposed approach, GMD starts from a single Gaussian and is built up by sequentially adding new omponents. Each new component is globally selected to produce optimal gain in the objective function. MLE and MMI re unified under the H-criterion, which is optimized by the extended BW (EBW) algorithm. A partial extended EM algorithm is developed for stage-wise optimization of new omponents. Experimental results on WSJ task demonstrate that he new algorithm leads to improved model quality and recognition performance.


Publication typeInproceedings
Published inProceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'06)
AddressToulouse, France
PublisherInstitute of Electrical and Electronics Engineers, Inc.
> Publications > Gradient Boosting Learning of Hidden Markov Models