Approximate Bayesian Inference

Bayesian inference provides a powerful mechanism for data analysis and learning. However, in real-world situations it is rarely possible to perform exact inference; in fact, exact Bayesian inference is in general NP hard. One of the most successful approaches to address this problem is to exploit a factorial structure of both the sampling distribution and the prior. Then, there are a variety of methods that exploit the factorisation for efficient approximations in inference. The Expectation-Propagation (EP) algorithm is a powerful tool for inference in such factor graphs. For discrete distributions, the EP algorithm is also known as Belief Propagation.

References

• Ralf Herbrich. On Gaussian Expectation Propagation. 2005. (PDF)
• Ralf Herbrich. Minimising the Kullback-Leibler Divergence. 2005 (PDF)