Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Fast Variational Inference for Large-scale Internet Diagnosis

John Platt, Emre Kıcıman, and Dave Maltz

Abstract

Web servers on the Internet need to maintain high reliability, but the cause of intermittent failures of web transactions is non-obvious. We use approximate Bayesian inference to diagnose problems with web services. This diagnosis problem is far larger than any previously attempted: it requires inference of 104 possible faults from 105 observations. Further, such inference must be performed in less than a second. Inference can be done atfl this speed by combining a mean-field variational approximation and the use of stochastic gradient descent to optimize a variational cost function. We use this fast inference to diagnose a time series of anomalous HTTP requests taken from a real web service. The inference is fast enough to analyze network logs with billions of entries in a matter of hours.

Details

Publication typeProceedings
Published inThe Conference on Neural Information Processing Systems (NIPS)
> Publications > Fast Variational Inference for Large-scale Internet Diagnosis