Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Machine Learning and Games - MALAGA

NIPS2007 Workshop - 8 December 2007, Whistler BC This workshop explored open directions in applying machine learning to games.


Computer games sales are three time larger than industry software sales, and on par with Hollywood box office sales; Halo 3 has become the biggest entertainment launch in history, with $170 Million in sales in the US alone in the first 24 hours! Modern computer games are often based on extremely complex simulations of the real world and constitute one of the very few real fields of application for artificial intelligence encountered in everyday life. Surprisingly, machine learning methods are not present in the vast majority of computer games. There have been a few recent and notable successes in turn-based two-player, discrete action space games such as Backgammon, Checkers, Chess and Poker. However, these successes are in stark contrast to the difficulties still encountered in the majority of computer games, which typically involve more than two agents choosing from a continuum of actions in complex artificial environments. Typical game AI is still largely built around fixed systems of rules that often result in implausible or predictable behaviour and poor user experience. The purpose of this workshop is to involve the NIPS community in the exciting challenges that games - ranging from traditional table top games to cutting-edge console and PC games - offer to machine learning.



Opening. Organizers


Decision Making and Knowledge Representation in Halo 3
Max Dyckhoff, Bungie Studios


Computer (and Human) Perfection at Checkers
Jonathan Schaeffer, University of Alberta


Coffee break


State-of-the-Art in Computer Poker
Michael Bowling, University of Alberta


Monte-Carlo Planning for the Game of Go
Olivier Teytaud, TAO(Inria, Univ. Paris-Sud, UMR CNRS-8623)




Lunch Break


Planning in Real-Time Strategy Games
Michael Buro, University of Alberta


Learning Driving Tasks in TORCS Using Reinforcement Learning
A. Lazaric, D. Loiacono, A. Prete, M. Restelli, P.L. Lanzi,
Politecnico di Milano


Modelling Go Positions with Planar CRFs
D. Kamenetsky, N. Schraudolph, S.Günter, S.V.N. Vishwanathan, NICTA


Opportunities for Machine Learning to Impact Interactive Narrative
David L. Roberts, Mark Riedl, Charles L. Isbell, Georgia Institute of Technology


Coffee break


XNA Game Studio: Next-Gen Game Development at your Fingertips
Chris Satchell, Microsoft


Panel discussion
Invited panelist: Luis von Ahn, Carnegie Mellon University

Invited Speakers

  • Max Dyckhoff, Bungie Studios. Programmed the AI of Halo 3, the most highly anticipated computer game ever. Within 24 hours of release, Halo 3 generated $170m in sales in the United States alone.
  • Chris Satchell, XNA Group. The man behind “XNA Game Studio”, a development environment for game development intended for students, hobbyist and independent game developers. A number of academic institutions have already agreed to include XNA Game Studio in their course offerings.
  • Jonathan Schaeffer, University of Alberta. Lead-creator of Chinook, the world man-machine Checkers champion: the first program to win a human championship in any popular board game. More recently he solved the game of Checkers.
  • Michael Bowling, University of Alberta. Head of the University of Alberta computer Poker group and part of the team that won the bankroll and the series competitions at the 2006 AAAI Computer Poker Competition, and the limit online and limit equilibrium series at the 2007 AAAI Computer Poker Competition.
  • Sylvain Gelly, Google. Co-creator of the Go program MoGo, winner of six Kiseido Go Server (KGS) tournaments and considered one of the best programs in the world.
  • Michael Buro, University of Alberta. Expert in Real-Time Strategy (RTS) Game AI, creator of the ORTS Game Engine. Michael is also the author of the computer Othello player Logistello, unbeaten between 1993 and 1997.


  • Learning to drive by Reinforcement Learning. Phillip Trelford, MSRC.
    Interactive demo showing the AMPS reinforcement learning framework integrated into the state of the art car racing game, Project Gotham Racing 3 (PGR3).
  • Checkers. Jonathan Schaeffer, University of Alberta.
    Playable demo of the perfect checkers player, which is the result of the recent solution of the game after almost 20 years of computation!
  • MoGo: Monte Carlo Go. Sylvain Gelly, Google.
    Opportunity to witness and play against one of the strongest currently available Go programs based on Monte-Carlo search.
  • Polaris: Computer Poker. Michael Bowling, University of Alberta.
    Chance to match your wits against the first poker computer player ever to challenge top human professional players.
  • Halo 3 AI. Max Dyckhoff, Bungie Studios.
    Display of the inner workings of the Artificial Intelligence driving the Earth threatening covenants in the blockbuster game Halo 3.

Panel Discussion

We devoted the last hour of the workshop to a panel discussion. The panel was composed of the invited speakers, plus as a special guest, the speaker of the NIPS 2007 opening night banquet:

  • Luis von Ahn, Carnegie Mellon University. Expert in human computation; inventor of games such as the ESP game for image labelling to make use of human brain cycles.