Ranking SVM on LETOR 



Learning parameters




Introduction to Ranking SVM

The basic idea of Ranking SVM is to formalize learning to rank as a problem of binary classification on instance pairs, and then to solve the problem using Support Vector Machines.

The details of Ranking SVM can be found from http://www.research.microsoft.com/~rherb/papers/herobergrae99.ps.gz and http://svmlight.joachims.org/.


Learning Parameters

We used a code based on primal optimization of the objective function as described in this paper. The multiplicator in front of the loss term is C/n where n is the number of preference pairs and C is chosen on the validation set in the set

{0.01, 0.1, 1, 10, 100}.
The Matlab script used to run these experiments can be found here.


C (from Fold1 to Fold5)


0.01, 10, 100, 10, 10


100, 100, 100, 100, 100


100, 100, 100, 100, 100


100, 1, 10, 10, 100


100, 100, 100, 100, 10


10, 100, 100, 100, 100


100, 1, 100, 0.01, 0.01

Papers & Docs

 R. Herbrich, T. Graepel, and K. Obermayer. Large Margin Rank Boundaries for Ordinal Regression. Advances in Large Margin Classifiers, 115-132, Liu Press, 2000.

T. Joachims. Optimizing Search Engines Using Clickthrough Data, Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD), ACM, 2002.

O. Chapelle and S. S. Keerthi. Efficient algorithms for ranking with SVMs. Information Retrieval Journal, Special Issue on Learning to Rank, 2009. to appear.

    author = {Herbrich, Ralf   and Graepel, Thore   and Obermayer, Klaus  },
    booktitle = {Advances in Large Margin Classifiers},
    citeulike-article-id = {477598},
    editor = {Smola and Bartlett and Schoelkopf and Schuurmans},
    keywords = {ml\_for\_ir, reranking, svm, um},
    priority = {2},
    publisher = {MIT Press, Cambridge, MA},
    title = {Large margin rank boundaries for ordinal regression},
    url = {http://citeseer.ist.psu.edu/contextsummary/1891774/0},
    year = {2000}

    author = {Thorsten Joachims},
    title = {Optimizing search engines using clickthrough data},
    booktitle = {KDD '02: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining}, 
    year = {2002},
   isbn = {1-58113-567-X},
   pages = {133--142},
   location = {Edmonton, Alberta, Canada},
   doi = {http://doi.acm.org/10.1145/775047.775067},
   publisher = {ACM},
   address = {New York, NY, USA},


This document was written by Olivier Chapelle, and the experiments were conducted by Olivier Chapelle. If any problem, please contact letor@microsoft.com or chap@yahoo-inc.com