Infer.NET user guide : Tutorials and examples : Multi-class classification
Multi-class Bayes Point Machine using shared variables
Page 1 | Page 2
In the previous example, we presented details for implementing a multiclass Bayes point machine. In that example, the weight vectors for the train model were maintained independently of test model. Hence, during testing, we explicitly set the posterior distribution (inferred in the training phase) to be the prior distribution for the weights of the test model. Infer.Net provides an elegant solution to overcome this manual setting through the use of shared variables. As we illustrate using this example, shared variables are also useful when we want to learn from large amounts of data that can not fit into memory, and hence can only be accessed in chunks.
Here, we share the weight vector (for each class) between the train and the test model. As we will see later, each of these models can obtain a copy of the shared variables. We specify standard VectorGaussian priors over the weights. Currently, in Infer.Net, we can not use a VariableArray to specify the prior distribution for SharedVariableArray. As with the non-shared version, we set the prior for one set of weights to a VectorGaussian point mass in order to remove an unnecessary degree of freedom.
Model for training
During training, all examples are labeled. So, we group training examples by their label. The number of items, the range operator for these items, and the feature vectors (represented using Vector) for examples from each class are declared as follows:
To access the weight vectors, we first create a Model object, with parameter nChunks, indicating the number of chunks of data. The shared variables are turned into ordinary variables in the context of a given Model by calling GetCopyFor. Thus, in the snippet below, wModel will function in the same way as w in the model without shared variables.
As before, we can specify the model, using wModel as the variables for the weight vectors for each class:
Training the model
The above model can be nicely encapsulated as trainModel, which can then be used to train. During training, for a particular chunk of data, chunkNo, we set the observed values for the variables, and then perform inference for w by making the InferShared call. InferShared method on the Model instance takes care of extracting output messages to the shared variables from one batch, and creating the correct input messages to the shared variables for the next batch. The posterior marginal can be subsequently obtained using the call to Marginal:
Next, we see how to specify the model for testing and use it to perform prediction.
Page 1 | Page 2