The Hidden Life of Latent Variables: Bayesian Mixed Graph Models in Supervised and Unsupervised Learning

Hidden common causes are often the explanation behind the observed association of our recorded variables. In many cases, however, we are interested in modeling only a few or even none of the hidden variables: the rest could as well be marginalized, if they play no role in our learning problem. The reward is not having to make any further assumptions about variables one does not care, but this might result in a set of independence constraints that cannot be represented by standard graphical notations, such as directed acyclic graphs or factor graphs.

Directed mixed graphs are graphical representations that include directed and bidirected edges. Such a class is motivated by dependencies that arise when hidden common causes are marginalized out of a distribution. In this talk, we will show how to perform Bayesian inference on a variety of different mixed graph models for learning parameters and making predictions. We will discuss models, priors, and Monte Carlo, variational and expectation-propagation algorithms. Applications include modeling social and economical data, and performing classification with relational data.

Joint work with Zoubin Ghahramani and Wei Chu.