Improving the speed and accuracy of inference
There are often multiple ways of using Infer.NET to solve a particular problem. The list below suggests ways in which you can change your Infer.NET code to improve its speed and/or the accuracy of the solution given.
- Use observed random variables rather than constants
Avoid using constants for data and/or priors, and use observed variables instead - this means that your model can be run multiple times with different data and priors without recompiling the model. See the truncated Gaussian tutorial for an example of this.
- Use variable arrays and ranges rather than .NET arrays.
Where possible, use variable arrays made with Variable.Array() instead of arrays of Variable instances (see Learning a Gaussian tutorial).
- Use sparse messages when using large Discrete or Dirichlet distributions.
The messages needed to perform inference in models with large Discrete or Dirichlet distributions often have sparse structure. This can be exploited by using sparse messages to significantly reduce memory consumption and increase speed.
- Specify the exact set of variables to be inferred
Use the OptimiseForVariables property on the inference engine to specify exactly which variables you want to infer. This is more efficient than the Infer() method which opportunistically computes all marginals that may be needed (see Mixture of Gaussians for an example).
- Reuse memory for the marginals
Set Compiler.ReturnCopies to false on the inference engine. This causes the engine to modify the previous marginal distributions in place. However, be warned that you need to copy the marginal distributions yourself if you want to save them.
- Reuse memory for internal messages
Set Compiler.FreeMemory to false on the inference engine. This causes the engine to keep internal messages allocated between inference runs. However, be warned that this can significantly increase the memory consumption of your program.
- Reduce the number of factors
In general, the fewer factors in your factor graph, the faster and more accurate inference can be. Try to rewrite your model to reduce the number of factors if possible. The ShowFactorGraph option on the inference engine should help.
- Profile the inference code
You can profile the generated inference code to find bottlenecks, as described in debugging inference.
- Use release libraries
Both debug and release libraries are provided - check you are using the release libraries for maximum performance.
- Reduce the number of iterations of inference
The default number of iterations is 50 - your model may need fewer iterations to converge. Modify NumberOfIterations in the inference engine settings or use a custom convergence criterion (see Controlling how inference is performed).
- Ensure that you are using optimised inference code
Make sure that the compiler property OptimiseInferenceCode is set to true (it should normally be true and is so by default).
- Avoid the 64-bit JIT compiler
If you compile your executable in Visual Studio with platform target = x64 or "Any CPU", then on a 64-bit OS you will be using the 64-bit JIT compiler at runtime, which is very inefficient on large methods such as the ones that Infer.NET generates. This causes a significant memory and computation spike on the first iteration of inference. To avoid this, change the platform target to x86.
- Avoid server-mode and concurrent garbage collection on a 64-bit machine
If you are running on a 64-bit machine, configure the garbage collector so that it does not run concurrently and so that it does not run in server mode. You can do this by adding the following lines to your application's app.config file:
<gcServer enabled="false" />
<gcConcurrent enabled="false" />
- Use parallel for loops (experimental feature)
You can get the model compiler to emit parallel for loops, allowing multiple cores to be used when running inference. Set the compiler UseParallelForLoops property to true.