Infer.NET

Sharing variables between models

On occasion you might want to split up your inference so that it runs on different sub-models (or different copies of the same model) where the participating sub-models have variables in common. Here are some scenarios where you might want to do this:

You have very large amounts of data and a single model won't fit into memory
You are doing online inference and the data is not all available ahead of time
You want more control over the schedule by splitting your model into smaller chunks and scheduling them manually
You want to parallelise your inference code
You want to run different algorithms on different parts of your model

All these scenarios have a common pattern. You will need to

Run inference to convergence on one of the sub-models, say model A
Extract the shared variable messages output from model A
Initialise the next model, model B, say, by providing as input a product of all the output messages from all models except B

Infer.NET provides a SharedVariable class which makes it easy to implement this pattern. It supports sharing between models of different structures and also across multiple data batches. Also you can update all shared variables with a single function call.

The following example extends the simple Gaussian example so that the data is divided into two chunks. This first code fragment shows the model. In this fragment, the unknown mean and precision of the Gaussian are defined as shared variables. These are shared between the two chunks. The other variables and factors defined in the code are not shared.

Because both chunks share the same structure, a single Model object is created. Each Model object represents a model structure. In this case, we have two datasets ('chunks') using the same model structure. Notice that the data count for a given chunk is provided as a variable, since it will vary from chunk to chunk.

The shared variables are turned into ordinary variables in the context of a given Model by calling GetCopyFor. In this case, we create ordinary variables corresponding to the shared mean and precision, and use them to generate the data.

// The data double[][] dataSets = new double[][] { new double[] { 11, 5, 8, 9 }, new double[] { -1, -3, 2, 3, -5 } }; int numChunks = dataSets.Length; // The model Gaussian priorMean = Gaussian.FromMeanAndVariance(0, 100); Gamma priorPrec = Gamma.FromShapeAndScale(1, 1); SharedVariable<double> mean = SharedVariable<double>.Random(priorMean); SharedVariable<double> precision = SharedVariable<double>.Random(priorPrec); Model model = new Model(numChunks); Variable<int> dataCount = Variable.New<int>(); Range item = new Range(dataCount); VariableArray<double> data = Variable.Array<double>(item); data[item] = Variable.GaussianFromMeanAndPrecision( mean.GetCopyFor(model), precision.GetCopyFor(model)).ForEach(item);

This next code fragment shows the inference. During inference, we cycle through each chunk, setting the appropriate chunk-specific variables (in this case, data and dataCount). The data and dataCount are set using the ObservedValue property. We then infer the variables using the InferShared method on the Model instance which takes care of extracting output messages to the shared variables from one batch, and creating the correct input messages to the shared variables for the next batch. For convergence, we perform multiple cycles through the batches.

// Set the inference algorithm InferenceEngine engine = new InferenceEngine(new VariationalMessagePassing()); for (int pass = 0; pass < 5; pass++) { // Run the inference on each data set for (int c = 0; c < numChunks; c++) { data.ObservedValue = dataSets[c]; dataCount.ObservedValue = dataSets[c].Length; model.InferShared(engine, c); } }

We get the final marginal distribution over the shared variables by using the Marginal method. Note this does not initiate any inference. It simply returns the stored result from the computations above.

// Retrieve the posterior distributions Gaussian marginalMean = mean.Marginal<Gaussian>(); Gamma marginalPrec = precision.Marginal<Gamma>(); Console.WriteLine("mean=" + marginalMean); Console.WriteLine("prec=" + marginalPrec);

Last modified at 12/1/2008 6:17 PM by John Guiver