A roadmap to research on EP

Tutorials Expectation Propagation: CUED Tutorial slides A family of algorithms for approximate Bayesian inference How to construct EP algorithms, illustrated with a variety of different approximations and factor grouping schemes. From Belief Propagation to Expectation Propagation K. Murphy, 2001 A quick skim of thesis Chapter 3, with some more derivations. EP Summary E. Sudderth, 2002 An even shorter summary. EP: A quick reference A list of equations useful for constructing Gaussian EP algorithms. Notes on Minka's Expectation Propagation for Gaussian Process classification M. Seeger, 2002 EP in practice Extending expectation propagation for graphical models Y. Qi, 2004 An overview of techniques used to implement EP in practice. Expectation Propagation for Exponential Families M. Seeger, 2007 Theoretical developments The impact of different divergence measures Divergence measures and message passing Raising factors to powers Power EP Kikuchi approximation Structured Region Graphs: Morphing EP into GBP Convergence control Damping and skipping: Expectation-Propagation for the Generative Aspect Model Double-loop: Expectation propagation for approximate inference in dynamic Bayesian networks T. Heskes and O. Zoeter, UAI'2002 EP within EM Expectation-Propagation for the Generative Aspect Model Predictive Automatic Relevance Determination by Expectation Propagation Sparse approximation Gaussian Processes - Iterative Sparse Approximations L. Csato, 2002 Gaussian quadrature based expectation propagation O. Zoeter and T. Heskes, AISTATS 2005 The objective function Expectation Propagation for approximate Bayesian inference The EP energy function and minimization schemes TAP Gibbs Free Energy, Belief Propagation and Sparsity L. Csato, M. Opper, and O. Winther, NIPS'2001 Expectation propagation for approximate inference in dynamic Bayesian networks T. Heskes and O. Zoeter, UAI'2002 Approximate Inference Techniques with Expectation Constraints T. Heskes, M. Opper, W. Wiegerinck, O. Winther and O. Zoeter, Journal of Statistical Mechanics: Theory and Experiment, 11015 (2005)

Uses of EP Perceptrons, Gaussian process classifiers Assessing Approximate Inference for Binary Gaussian Process Classification M. Kuss and C. E. Rasmussen, JMLR 2005 Predictive Automatic Relevance Determination by Expectation Propagation Fast Sparse Gaussian Process Methods: The Informative Vector Machine N. Lawrence, M. Seeger, and R. Herbrich, NIPS'2002 Gaussian Processes - Iterative Sparse Approximations L. Csato, 2002 A family of algorithms for approximate Bayesian inference Gaussian Processes for Classification: Mean Field Algorithms M. Opper and O. Winther, Neural Computation 12: 2655-2684, 2000 Bayesian Conditional Random Fields Discrete Bayes nets and Markov random fields Expectation Consistent Approximate Inference M. Opper and O. Winther, JMLR, to appear Tree-structured approximations by expectation propagation Density estimation with Gaussian processes Gaussian Processes - Iterative Sparse Approximations L. Csato, 2002 Neural networks, Multilayer perceptrons Computing with Finite and Infinite Networks O. Winther, NIPS'2001 (ADATAP) Independent Components Analysis (ICA) TAP Gibbs Free Energy, Belief Propagation and Sparsity L. Csato, M. Opper, and O. Winther, NIPS'2001 Mean Field Approaches to Independent Component Analysis P. A.d.F.R. Højen-Sørensen, O. Winther, and L. K. Hansen, Neural Computation 14: 889-918 (2002) Text modeling, latent Dirichlet allocation Expectation-Propagation for the Generative Aspect Model Hybrid dynamic systems (continuous + discrete state) Expectation propagation for approximate inference in dynamic Bayesian networks T. Heskes and O. Zoeter, UAI'2002 Window-based expectation propagation for adaptive signal detection in flat-fading channels (Fixed-lag smoothing with EP) Nonlinear dynamic systems Iterated extended Kalman smoothing with Expectation-Propagation A. Ypma and T. Heskes, NNSP'2003 Expectation Propagation for Continuous Time Bayesian Networks U. Nodelman, D. Koller, and C. R. Shelton, UAI 2005 Visualization of time-series data Hierarchical visualization of time-series data using switching linear dynamical systems O. Zoeter and T. Heskes, PAMI 2003 Infinite Mixture Models Expectation Propagation for Infinite Mixtures