Ranking Mechanisms for Twitter-Like Forums

We study the problem of designing a mechanism to rank

items in forums by making use of the user reviews such as

thumb and star ratings. We compare mechanisms where fo-

rum users rate individual posts and also mechanisms where

the user is asked to perform a pairwise comparison and state

which one is better. The main metric used to evaluate a

mechanism is the ranking accuracy vs the cost of reviews,

where the cost is measured as the average number of reviews

used per post. We show that for many reasonable prob-

ability models, there is no thumb (or star) based ranking

mechanism that can produce approximately accurate rank-

ings with bounded number of reviews per item. On the

other hand we provide a review mechanism based on pair-

wise comparisons which achieves approximate rankings with

bounded cost. We have implemented a system, shoutveloc-

ity [5], which is a twitter-like forum but items (i.e., tweets

in Twitter) are rated by using comparisons. For each new

item the user who posts the item is required to compare

two previous entries. This ensures that over a sequence of

n posts, we get at least n comparisons requiring one review

per item on average. Our mechanism uses this sequence of

comparisons to obtain a ranking estimate. It ensures that

every item is reviewed at least once and winning entries are

reviewed more often to obtain better estimates of top items.

In  Proc. of Third ACM International Conference on Web Search and Data Mining (WSDM)

Publisher  Association for Computing Machinery, Inc.
Copyright © 2007 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or permissions@acm.org. The definitive version of this paper can be found at ACM’s Digital Library --http://www.acm.org/dl/.


> Publications > Ranking Mechanisms for Twitter-Like Forums