Machine Learning Approaches to Ordinal Regression

In contrast to the standard machine learning tasks of classification and metric regression we investigate the problem of predicting variables of ordinal scale, a setting referred to as ordinal regression. This problem arises frequently in the social sciences and in information retrieval where human preferences play a major role. Whilst approaches proposed in Statistics rely on a probability model of a latent (unobserved) variable we present a distribution independent risk formulation of ordinal regression which allows us to derive uniform convergence bounds. Applying these bounds we present a large margin algorithm that is based on a mapping from objects to scalar utility values though classifying pairs of objects. We give experimental results for an Information Retrieval task which show that our algorithm outperforms more naive approaches to ordinal regression such as Support Vector Classification and Support Vector Regression in the case of more than two ranks

References

• Ralf Herbrich, Thore Graepel, and Klaus Obermayer. Large Margin Rank Boundaries for Ordinal Regression. . Advances in Large Margin Classifiers, pages 115-132, 2000. (Gzipped PostScript).
• Ralf Herbrich, Thore Graepel, Peter Bollmann-Sdorra, and Klaus Obermayer. Learning a Preference Relation in IR. . In Proceedings Workshop Text Categorization and Machine Learning, International Conference on Machine Learning 1998, pages 80-84, 1998. (Gzipped PostScript)