Pawel Swietojanski, Jinyu Li, and Jui-Ting Huang
We explore the use of maxout neuron in various aspects of acoustic
modelling for large vocabulary speech recognition systems; including
low-resource scenario and multilingual knowledge transfers.
Through the experiments on voice search and short message dictation
datasets, we found that maxout networks are around three times
faster to train and offer lower or comparable word error rates on
several tasks, when compared to the networks with logistic nonlinearity.
We also present a detailed study of the maxout unit internal
behaviour suggesting the use of different nonlinearities in different