Probabilistic Programming the Future of Machine Learning?

In today’s world of connectivity and ubiquitous data, we need to be able to solve ever more complex machine-learning problems. Ideally, we would like the solutions to be provided automatically, given a definition of the problem. But how do we even describe a complex and intricate problem in a way a machine can understand?

Probabilistic programming gives us a powerful and flexible way to express such machine-learning problems. It uses the rich expressivity of modern programming languages to describe a problem as an inference query. Once you have a probabilistic program, the idea is to use an existing inference ‘engine’ to actually run the program and so answer the inference query automatically. This allows the program to be tested rapidly and, if necessary, quickly modified and improved to meet the needs of the application.

In this lecture, I will describe efforts going on at Microsoft Research Cambridge to create a fast, general purpose programming language, called probabilistic C#. As an example, I will show how probabilistic C# can be used to produce compact, complex models of unstructured text. To do inference in probabilistic C# (to run the program), we use the Infer.NET inference engine. I will discuss some of the (many) issues arising in trying to build a general purpose inference engine and describe approaches we are investigating to make running probabilistic C# programs both fast and accurate. The aim is to make probabilistic C# an ideal language for solving the next generation of machine learning problems.