Infer.NET

Infer.NET user guide : Tutorials and examples

Multi-class Bayes Point Machine using shared variables

Page 1 | Page 2

In the previous example , we presented details for implementing a multiclass Bayes point machine. In that example, the weight vectors for the train model were maintained independently of test model. Hence, during testing, we explicitly set the posterior distribution (inferred in the training phase) to be the prior distribution for the weights of the test model. Infer.Net provides an elegant solution to overcome this manual setting through the use of shared variables. As we illustrate using this example, shared variables are also useful when we want to learn from large amounts of data that can not fit into memory, and hence can only be accessed in chunks.

Here, we share the weight vector between the train and the test model. As we will see later, each of these models can obtain a copy of the shared variables. We specify standard VectorGaussian priors over the weights. Currently, in Infer.Net, we can not use a Variable to specify the prior distribution for SharedVariable

SharedVariable<Vector>[] w = new SharedVariable<Vector>[nClass];
VectorGaussian wPrior = new VectorGaussian(new Vector(nFeatures),
PositiveDefiniteMatrix.Identity(nFeatures))
for (int c = 0; c < nClass; c++)
{
w[c] = SharedVariable<Vector>.Random(wPrior);
}

Model for training

During training, all examples are labeled. So, we group training examples by their label. The number of items, the range operator for these items, and the feature vectors (represented using Vector) for examples from each class are declared as follows:

Variable<int>[] nItem = new Variable<int>[nClass];
Range[] item = new Range[nClass];
VariableArray<Vector>[] xValues = new VariableArray<Vector>[nClass];

To access the weight vectors, we first create a Model object, with parameter nChunks, indicating the number of chunks of data. The shared variables are turned into ordinary variables in the context of a given Model by calling GetCopyFor. Thus, in the snippet below, wModel will function in the same way as w in the model without shared variables.

Model model = new Model(nChunks).Named("model"+s);

// The weight vector within a submodel
Variable<Vector>[] wModel = new Variable<Vector>[nClass];
for (int c = 0; c < nClass; c++)
{
// Get a copy of the shared weight vector variable for the submodel
wModel[c] = w[c].GetCopyFor(model).Named("wModel_" + c + s);
}

As before, we can specify the model, using wModel as the variables for the weight vectors for each class:

for (int c = 0; c < nClass; c++)
{
nItem[c] = Variable.New<int>().Named("nItem_" + c + s);
item[c] = new Range(nItem[c]).Named("item_" + c + s);
xValues[c] = Variable.Array<Vector>(item[c]);

using (Variable.ForEach(item[c]))
{
Variable<double>[] score = BPMUtils.ComputeClassScores(wModel, xValues[c][item[c]]);
BPMUtils.ConstrainArgMax(c, score);
}
}

Training the model

The above model can be nicely encapsulated as trainModel, which can then be used to train. During training, for a particular chunk of data, chunkNo, we set the observed values for the variables, and then perform inference for w by making the InferShared call. InferShared method on the Model instance takes care of extracting output messages to the shared variables from one batch, and creating the correct input messages to the shared variables for the next batch. The posterior marginal can be subsequently obtained using the call to Marginal:

trainModel.model.InferShared(trainModel.ie, chunkNo)
VectorGaussian[] wInferred = new VectorGaussian[nClass];
for (int c = 0; c < nClass; c++)
{
wInferred[c] = w[c].Marginal<VectorGaussian>();
}

Next, we see how to specify the model for testing and use it to perform prediction.

Page 1 | Page 2

Last modified at 12/3/2008 3:00 PM by John Guiver