Non-conjugate Variational Message Passing for Multinomial and Binary Regression

Variational Message Passing (VMP) is an algorithmic implementation of the Variational Bayes (VB) method which applies only in the special case of conjugate exponential family models. We propose an extension to VMP, which we refer to as Non-conjugate Variational Message Passing (NCVMP) which aims to alleviate this restriction while maintaining modularity, allowing choice in how expectations are calculated, and integrating into an existing message-passing framework: Infer.NET. We demonstrate NCVMP on logistic binary and multinomial regression. In the multinomial case we introduce a novel variational bound for the softmax factor which is tighter than other commonly used bounds whilst maintaining computational tractability.

KnoMin11.pdf
PDF file
knowles2011ncvmp_supp.pdf
PDF file

In  Advances in Neural Information Processing Systems 24

Details

TypeInproceedings
> Publications > Non-conjugate Variational Message Passing for Multinomial and Binary Regression