Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Calibration of confidence measures in speech recognition

Dong Yu, Jinyu Li, and Li Deng


Most of the speech recognition applications in use today rely heavily on confidence measure for making optimal decisions. In this work, we aim to answer the question: what can be done to improve the quality of confidence measure if we have no access to the internals of speech recognition engines? The answer provided in this paper is a post-processing step called confidence calibration, which can be viewed as a special adaptation technique applied to confidence measure. We report three confidence calibration methods that have been developed in this work: the maximum entropy model with distribution constraints, the artificial neural network, and the deep belief network. We compare these approaches and demonstrate the importance of key features exploited: the generic confidence-score, the application-dependent word distribution, and the rule coverage ratio. We demonstrate the effectiveness of confidence calibration on a variety of tasks with significant normalized cross entropy increasement and equal error rate reduction.


Publication typeArticle
Published inIEEE Transactions on Audio, Speech and Language Processing
> Publications > Calibration of confidence measures in speech recognition