Infer.NET user guide : Tutorials and examples : Multi-class classification

Multi-class Bayes Point Machine using shared variables

Page 1 | Page 2

In the previous example, we presented details for implementing a multiclass Bayes point machine. In that example, the weight vectors for the train model were maintained independently of test model.  Hence, during testing, we explicitly set the posterior distribution (inferred in the training phase) to be the prior distribution for the weights of the test model. Infer.Net provides an elegant solution to overcome this manual setting through the use of shared variables. As we illustrate using this example, shared variables are also useful when we want to learn from  large amounts of data that can not fit into memory, and hence can only be accessed in  chunks.

Here, we share the weight vector (for each class) between the train and the test model. As we will see later, each of these models can obtain a copy of the shared variables. We specify standard VectorGaussian priors over the weights. Currently, in Infer.Net, we can not use a VariableArray to specify the prior distribution for SharedVariableArray. As with the non-shared version, we set the prior for one set of weights to a VectorGaussian point mass in order to remove an unnecessary degree of freedom.

var wPrior = (VectorGaussianArray)Distribution<Vector>.Array(Util.ArrayInit(
  nClass, cls => cls == 0 ?
  VectorGaussian.PointMass(Vector.Zero(nFeatures)) :
c = newRange(nClass).Named("c");
w = SharedVariable<Vector>.Random(c,wPrior).Named("w");

Model for training

During training, all examples are labeled. So, we group training examples by their label. The number of items, the range operator for these items, and the feature vectors (represented using Vector) for examples from each class are declared  as follows:

var nItem = Variable.Array<int>(c);
Range item = new Range(nItem[c]).Named("item");
var xValues = Variable.Array(Variable.Array<Vector>(item),c).Named("x");

To access the weight vectors, we first create a Model object, with parameter nChunks, indicating the number of chunks of data. The shared variables are turned into ordinary variables in the context of a given Model by calling GetCopyFor.  Thus, in the snippet below, wModel will function in the same way as w in the model without shared variables.

Model model = newModel(nChunks).Named("model"+s);
VariableArray<Vector> wModel = w.GetCopyFor(model).Named("wModel_" + s);

As before, we can specify the model, using wModel as the variables for the weight vectors for each class:

using(var cBlock = Variable.ForEach(c))
  var currC = cBlock.Index;
  using (Variable.ForEach(item))
    var score = BPMUtils.ComputeClassScores(wModel,xValues[c][item], NoisePrec);
    BPMUtils.ConstrainArgMax(currC, score);

Training the model

The above model can be nicely encapsulated as trainModel, which can then be used to train. During training, for a particular chunk of data, chunkNo, we set the observed values for the variables, and then perform inference for w by making the  InferShared call.  InferShared method on the Model instance takes care of extracting output messages to the shared variables from one batch, and creating the correct input messages to the shared variables for the next batch. The posterior marginal can be subsequently obtained using the call to Marginal:

trainModel.model.InferShared(, chunkNo);
var wInferred = w.Marginal<VectorGaussian[]>();

Next, we see how to specify the model for testing and use it to perform prediction.

Page 1 | Page 2

©2009-2013 Microsoft Corporation. All rights reserved.  Terms of Use | Trademarks | Privacy Statement