Minimising the Kullback-Leibler Divergence
- Ralf Herbrich
In this note we show that minimising the KullbackLeibler divergence over a family in the class of exponential distributions is achieved by matching the expected natural statistic. We will also give an explicit update formula for distributions with only one likelihood term.