Sep 30, 2004: Learning to Rank

1) Ralf Herbrich and Thore Graepel and Klaus Obermayer. Large Margin Rank Boundaries for Ordinal Regression.,   Advances in Large Margin classifiers, pages 115-132, 2000. Sections 1,2 and 4, skipping the learning theory section 3. The results in section 5 are optional.
The author himself, Ralf Herbrich, will be available to answer our questions.

2) W. Chu and Z. Ghahramani (2004), Gaussian processes for ordinal regression. Technical Report 2004.

Leader: Ralf Herbrich

Sep 2, 2004:  Factor model for Discrete data

GaP: A Factor Model for Discrete Data
John Canny - UC Berkeley
SIGIR 2004

Leader: Phil Cowans

Aug 12, 2004:  Boostmap

BoostMap: A Method for Efficient Approximate Similarity Rankings, Vassilis Athitsos, Jonathan Alon, Stan Sclaroff, and George Kollios, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2004.
This paper attempts to learn an embedding to be used for fast retrieval in large collections (images here).
 

Leader: Biswajit Bose

Aug 5, 2004: Kernelized CRFs

Kernel conditional random fields: Representation and clique selection
John Lafferty, Xiaojin Zhu, and Yan Liu
Machine Learning: Proceedings of the Twenty-First International Conference (ICML), 2004
Leader: Sanjiv Kumar

July 29, 2004:  Object recognition and segmentation

We look at the combination of object recognition (classification) and segmentation. The first paper is simple to read. The second and third are more difficult.
1. Appearance Based Qualitative Image Description for Object Class Recognition, Thureson, Carlsson
2. Combined Object Categorization and Segmentation with an Implicit Shape Model:  Leibe, Leonardis, Sciele, (Workshop Statistical Learning in CV)
3. Learning to Segment: Borenstein Ullman
(There is also an earlier version of the first paper from BMVC 03)

Leader: Carsten Rother

July 15, 2004

Sharing features: efficient boosting procedures for multiclass object detection
Antonio Torralba, Kevin Murphy and William Freeman
CVPR'04 (Computer Vision and Pattern Recognition).

Leader: Fei-Fei Li

June 24, 2004:

Zhuowen Tu and Alan L. Yuille
Shape Matching and Recognition - Using Generative Models and Informative Features
ECCV 2004

Leader: Martin Szummer

May 20, 2004

Hierarchical Dirichlet Processes.
Y.W. Teh, M.I. Jordan, M.J. Beal and D.M. Blei. Technical Report 653, UC Berkeley Statistics, 2004.
www.cs.toronto.edu/~ywteh/research/npbayes/report.pdf

Leader: Phil Cowans and Hugo Zaragoza

May 6, 2004

The Concave-Convex Procedure (CCCP)
A.L. Yuille and A. Rangarajan.
Proceedings NIPS'00. 2001
http://www.stat.ucla.edu/~yuille/pubs/optimize_papers/cccp_nips01.ps.gz

There are also longer journal versions of the paper on Alan Yuille's site.
The second paper is:

Stable fixed points of loopy belief propagation are minima of the Bethe free energy.
Tom Heskes (2003).
Proceedings NIPS 15
ftp://ftp.mbfys.kun.nl/pub/snn/pub/reports/Heskes.nips2002.ps.gz

Leader: Alan Yuille will lead the discussion. He may mention the information-geometric interpretation of BP and CCCV proposed by Shiro Ikeda (http://www.ism.ac.jp/~shiro/publications/index.html).

April 22, 2004

Box Sampling
by John Skilling
http://www.inference.phy.cam.ac.uk/bayesys/  (select the paper on the right).

Leader: David MacKay

April 8, 2004

Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms Citation: Yedidia, J.S., Freeman, W.T., and Weiss, Y.,
August 2002
http://www.merl.com/papers/TR2002-35/

And to accompany it, an exchange of emails written up as:

A Conversation about the Bethe Free Energy and Sum-Product
David J C MacKay, Jonathan S. Yedidia, William T. Freeman & Yair Weiss
http://www.inference.phy.cam.ac.uk/mackay/abstracts/bethe.html

Tom writes: The paper describes a new graphical notation ("region graphs") for expressing propagation algorithms, which generalizes junction trees. This is contrast to factor graphs, which are good at expressing a distribution, but not propagation algorithms. Because they include both exact and approximate inference, I think region graphs are currently the best method for expressing propagation algorithms.

Leaders: David MacKay and Tom Minka

April 1, 2004:  Factor Graphs

F. R. Kschischang, B. J. Frey and H.-A. Loeliger 2001.
Factor graphs and the sum-product algorithm.
IEEE Transactions on Information Theory 47:2, February, 498-519.

http://www.research.microsoft.com/~szummer/reading-group/papers/Kschischang-factor-graphs.pdf

The plan is to cover especially section 5.

March 25, 2004: Graphical models, exponential families, and variational inference.

M. J. Wainwright, and M. I. Jordan. Graphical models, exponential families, and variational inference. UC Berkeley, Dept. of Statistics, Technical Report 649. September, 2003.
Sections 4-5.

http://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJorVariational03.ps

Leader: Ralf Herbrich and Thore Graepel

Mar 18, 2004: Graphical models, exponential families, and variational inference.

M. J. Wainwright, and M. I. Jordan. Graphical models, exponential families, and variational inference. UC Berkeley, Dept. of Statistics, Technical Report 649. September, 2003.    Sections 1-3.

http://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJorVariational03.ps

Leader: Ralf Herbrich and Thore Graepel

Feb 24, 2004: What energy functions can be minimized via graph cuts?

Kolmogorov, V.; Zabih, R.;
"What energy functions can be minimized via graph cuts?"
Pattern Analysis and Machine Intelligence, IEEE Transactions on , Volume: 26 , Issue: 2 , Feb 2004
Pages: 147 - 159
http://www.research.microsoft.com/~szummer/reading-group/papers/Kolmogorov-energy-func-pami-04.pdf

Leader: Vladimir Kolmogorov

Feb 19, 2004: More conditional random fields

"Dynamic Conditional Random Fields for Jointly Labeling Multiple Sequences."
Andrew McCallum, Khashayar Rohanimanesh and Charles Sutton. 
NIPS*2003 Workshop on Syntax, Semantics, Statistics, 2003. at: http://www.cs.umass.edu/~mccallum/papers/dcrf-nips03.pdf
The first paper addresses some shortcomings of Kumar and Herbert papers; notably, it does not use the pseudolikelihood approximation for learning.

"An Alternate Objective Function for Markovian Fields."
Sham Kakade, Yee Whye Teh & Sam Roweis.
International Conference on Machine Learning 19 (ICML'02). pp. 275—282 http://www.cs.toronto.edu/~roweis/papers/newcost_draft.ps.gz

Leaders: Martin Szummer and Tom Minka

Feb 12, 2004: Conditional Random Fields

“Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification”
Sanjiv Kumar and M Hebert
ICCV in 2003
http://www.ri.cmu.edu/pub_files/pub4/kumar_sanjiv_2003_4/kumar_sanjiv_2003_4.pdf
 

The NIPS 2003 paper by Kumar and Hebert is a bit more recent than the ICCV one, and uses a slightly different model. It is available at:
http://books.nips.cc/papers/files/nips16/NIPS2003_VM06.pdf

Leader: Chris Bishop
 

Feb 5, 2004: DjVu Image Segmentation

Leaders:  Phil Cowans and John Winn

We will continue the discussion about DJVU, focussing on image segmentation - how DjVu segments foreground (text etc.) from background.

Patrick Haffner, Léon Bottou, Yann Le Cun, and Luc Vincent, "A General Segmentation Scheme for DjVu Document Compression," in Proceedings of the International Symposium on Mathematical Morphology (ISMM'02), (Sydney, Australia), Apr. 2002.

Paper 45 at http://leon.bottou.org/publications/index.html

If we have time, we will also look at how DjVu uses this segmentation to improve its compression of the background image:
L. Bottou and S. Pigeon, "Lossy Compression of Partially Masked Still Images," in Proceedings of IEEE Data Compression Conference, (Snowbird, UT), Apr. 1998.

which is paper 32 at http://leon.bottou.org/publications/index.html
 

Jan 29, 2004: DjVu coding algorithm

Leaders:  Phil Cowans and John Winn

Paper 33 at http://leon.bottou.org/publications/index.html

[Bottou and Howard and Bengio, 1998]
L. Bottou, P. Howard, and Y. Bengio, "The Z-Coder Adaptive Binary Coder," in Proceedings IEEE Data Compression Conference 1998, (Snowbird), Apr. 1998. [10 pages]

Jan 24, 2004:  DjVu overview

Leader: David MacKay

L. Bottou, P. Haffner, P. G. Howard, P. Simard, Y. Bengio, and Y. Le Cun, "High Quality Document Image Compression with DjVu," Journal of Electronic Imaging, vol. 7, no. 3, pp. 410-425, 1998. [25 pages]
(paper 31 on http://leon.bottou.org/publications/index.html).
 

I'd like to suggest that everyone try out using djvu also.
The decoder is free software and there is a free encoder too. You can get the encoder to talk to you about how many bits it is using for each aspect of the image.
http://www.djvuzone.org/

Jan 8, 2004  Microsoft-Gatsby-Inference group meeting

Nov 24, 2003  Microsoft-Gatsby-Inference group meeting

10:40 Shiro Ikeda (Gatsby) "Belief propagation from information geometrical viewpoint"
11:20 Phil Cowans (Cambridge) "Bayesian Language Modeling"
12:00 Martin Szummer (Microsoft) "Information regularization"
1:40 Mark Andrews (Gatsby) "Sequential pattern learning with nonlinear dynamical systems"
2:20 Yuan Qi (Microsoft/MIT) "Tree-structured approximations by expectation propagation"
3:00 Discussion: Open Problems and Common Interests in Machine Learning
4:00 Seb Wills (Cambridge) "Dynamic memories with spiking neurons"

Oct 16, 2003

Reading: John Winn's thesis chapter 5 on Structured Variational distributions.

Leader: John Winn

He writes:
Rather than read the rest of the thesis for background, you can instead read the parts 1-4 of the AI Stats paper “Structured Variational Distributions in VIBES” (abstract below), which you can get from:

http://www.inference.phy.cam.ac.uk/jmw39/Papers/VIBES_AIStats2003.ps

Oct 9, 2003

Oct 2, 2003

Bayes meets Bellman: The Gaussian Process Approach to Temporal Difference Learning
Y. Engel, S. Mannor and R. Meir, To appear in Proc. of the 20th International Conference on Machine Learning 2003 (ICML-03)
Download it from: http://www.ee.technion.ac.il/~rmeir/Publications/Icml03Gptd.pdf

Leader: Thore Graepel has volunteered to introduce the paper.

Sep 25, 2003

A generalized mean field algorithm for variational inference in exponential families.
E.P. Xing, M.I. Jordan, and S. Russell, Uncertainty in Artificial Intelligence, 2003. (UAI2003).
Leader: Markus Svensen

Sep 18, 2003.  Microsoft-Gatsby-Inference group meeting

Sep 12, 2003

Efficiently Inducing Features Of Conditional Random Fields By Andrew McCallum 

http://www.inference.phy.cam.ac.uk/pjc51/reading_group/mccallum03crfs.pdf

Leader: Martin Szummer

Solving MAP Exactly Using Systematic Search By James Park and Andrew Darwiche

 http://www.inference.phy.cam.ac.uk/pjc51/reading_group/park03exactmap.pdf

Leader: Phil Cowans

April 24, 2003

Learning Probabilistic Models of Relational Structure with Lise Getoor, Ben Taskar, and Daphne Koller.
In Eighteenth International Conference on Machine Learning, 2001