Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Inference algorithms and learning theory for Bayesian sparse factor analysis

Magnus Rattray, Oliver Stegle, Kevin Sharp, and John Winn

Abstract

Bayesian sparse factor analysis has many applications; for example, it has been applied to the problem of inferring a sparse regulatory network from gene expression data. We describe a number of inference algorithms for Bayesian sparse factor analysis using a slab and spike mixture prior. These include well-established Markov chain Monte Carlo (MCMC) and variational Bayes (VB) algorithms as well as a novel hybrid of VB and Expectation Propagation (EP). For the case of a single latent factor we derive a theory for learning performance using the replica method. We compare the MCMC and VB/EP algorithm results with simulated data to the theoretical prediction. The results for MCMC agree closely with the theory as expected. Results for VB/EP are slightly sub-optimal but show that the new algorithm is effective for sparse inference. In large-scale problems MCMC is infeasible due to computational limitations and the VB/EP algorithm then provides a very useful computationally efficient alternative.

Details

Publication typeInproceedings
Published inInternational Workshop on Statistical-Mechanical Informatics 2009, Journal of Physics: Conference Series
> Publications > Inference algorithms and learning theory for Bayesian sparse factor analysis