Ranking and Matchmaking
The TrueSkill ranking system is a skill based ranking system for Xbox Live developed at Microsoft Research. The purpose of a ranking system is to both identify and track the skills of gamers in a game (mode) in order to be able to match them into competitive matches. The TrueSkill ranking system only uses the final standings of all teams in a game in order to update the skill estimates (ranks) of all gamers playing in this game. Ranking systems have been proposed for many sports but possibly the most prominent ranking system in use today is ELO. For more details, please see the TrueSkill project page.
- Ralf Herbrich, Tom Minka, and Thore Graepel, TrueSkill(TM): A Bayesian Skill Rating System, in Advances in Neural Information Processing Systems 20, MIT Press, January 2007
- Ralf Herbrich and Thore Graepel, TrueSkill(TM): A Bayesian Skill Rating System, no. MSR-TR-2006-80, 2006
- Thore Graepel and Ralf Herbrich, Ranking and Matchmaking, in Game Developer Magazine, October 2006
Computer Go
The game of Go is an ancient Chinese game of strategy for two players. By most measures of complexity it is more complex than Chess. While Deep Blue (and more recently Deep Fritz) play Chess at the world champion's level no Go-playing program has yet even reached the level of play of an average amateur Go player. The reason for the failure to reproduce the impressive results in chess for the game of Go appear to lie in its greater complexity, both in terms of the number of different positions and in the difficulty of defining an appropriate evaluation function for Go positions.
We take the view, that only an automated way of acquiring Go knowledge - machine learning - can radically improve on the current situation in computer Go. Numerous Go servers in the internet offer thousands of game records of Go played by players that are very competent as compared to today's computer Go programs. The great challenge is to build machine learning algorithms that extract knowledge from these data-bases such that it can be used for playing Go well.
- David Stern, Ralf Herbrich, and Thore Graepel, Learning To Solve Game Trees, in Proceedings of the International Conference of Machine Learning, January 2007
- David Stern, Ralf Herbrich, and Thore Graepel, Bayesian Pattern Ranking for Move Prediction in the Game of Go, in Proceedings of the International Conference of Machine Learning, January 2006
- Thore Graepel, Mike Goutrie, Marco Krüger, and Ralf Herbrich, Learning on Graphs in the Game of Go, in Proceedings of the Ninth International Conference on Artificial Neural Networks, January 2001
Learning to Fight
We apply reinforcement learning to the problem of finding good policies for a fighting agent in a commercial computer game. The learning agent is trained using the SARSA algorithm for on-policy learning of an action-value function represented by linear and neural network function approximators. Importance aspects include the selection and construction of features, actions, and rewards as well as other design choices necessary to integrate the learning process into the game. The learning agent is trained against the built-in AI of the game with different rewards encouraging aggressive or defensive behaviour.
- Thore Graepel, Ralf Herbrich, and Julian Gold, Learning to Fight, in Proceedings of the International Conference on Computer Games: Artificial Intelligence, Design and Education, January 2004
Learning to Race
Drivatars are a novel form of learning artificial intelligence (AI) developed for Forza Motorsport. The technology behind the Drivatar concept is exploited within the game in two ways:
- as an innovative new learning game feature: create your own AI driver!
- as the underlying model for all the AI competitors in the Arcade and Career modes.
Our original goal in developing Drivatars was to create human-like computer opponents to race against. For more details please visit the Drivatar project page.
