A Test Collection of Preference Judgments
- Ben Carterette ,
- Paul Bennett ,
- Olivier Chapelle
In Proceedings of the SIGIR 2008 Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments Workshop |
We describe an initial release of a set of binary preference judgments over a subset of the LETOR data. These judgments are meant to serve as a starting point for research into questions of evaluation and learning over non-binary, multi-item assessments.