Machine Learning and Games - MALAGA

NIPS2007 Workshop - 8 December 2007, Whistler BC This workshop explored open directions in applying machine learning to games.

Description

Computer games sales are three time larger than industry software sales, and on par with Hollywood box office sales; Halo 3 has become the biggest entertainment launch in history, with $170 Million in sales in the US alone in the first 24 hours! Modern computer games are often based on extremely complex simulations of the real world and constitute one of the very few real fields of application for artificial intelligence encountered in everyday life. Surprisingly, machine learning methods are not present in the vast majority of computer games. There have been a few recent and notable successes in turn-based two-player, discrete action space games such as Backgammon, Checkers, Chess and Poker. However, these successes are in stark contrast to the difficulties still encountered in the majority of computer games, which typically involve more than two agents choosing from a continuum of actions in complex artificial environments. Typical game AI is still largely built around fixed systems of rules that often result in implausible or predictable behaviour and poor user experience. The purpose of this workshop is to involve the NIPS community in the exciting challenges that games - ranging from traditional table top games to cutting-edge console and PC games - offer to machine learning.

Programme

9:00am	Opening. Organizers

9:15am	Decision Making and Knowledge Representation in Halo 3 Max Dyckhoff, Bungie Studios

9:45am	Computer (and Human) Perfection at Checkers Jonathan Schaeffer, University of Alberta

10:15am	Coffee break

10:30am	State-of-the-Art in Computer Poker Michael Bowling, University of Alberta

11:00am	Monte-Carlo Planning for the Game of Go Olivier Teytaud, TAO(Inria, Univ. Paris-Sud, UMR CNRS-8623)

11:30am	Demos

12:00am	Lunch Break

2:00pm	Planning in Real-Time Strategy Games Michael Buro, University of Alberta

2:30pm	Learning Driving Tasks in TORCS Using Reinforcement Learning A. Lazaric, D. Loiacono, A. Prete, M. Restelli, P.L. Lanzi, Politecnico di Milano

2:50pm	Modelling Go Positions with Planar CRFs D. Kamenetsky, N. Schraudolph, S.Günter, S.V.N. Vishwanathan, NICTA

3:10pm	Opportunities for Machine Learning to Impact Interactive Narrative David L. Roberts, Mark Riedl, Charles L. Isbell, Georgia Institute of Technology

3:30pm	Coffee break

3:45pm	XNA Game Studio: Next-Gen Game Development at your Fingertips Chris Satchell, Microsoft

4:15pm	Panel discussion Invited panelist: Luis von Ahn, Carnegie Mellon University

Invited Speakers

Max Dyckhoff, Bungie Studios. Programmed the AI of Halo 3, the most highly anticipated computer game ever. Within 24 hours of release, Halo 3 generated $170m in sales in the United States alone.
Chris Satchell, XNA Group. The man behind “XNA Game Studio”, a development environment for game development intended for students, hobbyist and independent game developers. A number of academic institutions have already agreed to include XNA Game Studio in their course offerings.
Jonathan Schaeffer, University of Alberta. Lead-creator of Chinook, the world man-machine Checkers champion: the first program to win a human championship in any popular board game. More recently he solved the game of Checkers.
Michael Bowling, University of Alberta. Head of the University of Alberta computer Poker group and part of the team that won the bankroll and the series competitions at the 2006 AAAI Computer Poker Competition, and the limit online and limit equilibrium series at the 2007 AAAI Computer Poker Competition.
Sylvain Gelly, Google. Co-creator of the Go program MoGo, winner of six Kiseido Go Server (KGS) tournaments and considered one of the best programs in the world.
Michael Buro, University of Alberta. Expert in Real-Time Strategy (RTS) Game AI, creator of the ORTS Game Engine. Michael is also the author of the computer Othello player Logistello, unbeaten between 1993 and 1997.

Demos

Learning to drive by Reinforcement Learning. Phillip Trelford, MSRC.
Interactive demo showing the AMPS reinforcement learning framework integrated into the state of the art car racing game, Project Gotham Racing 3 (PGR3).
Checkers. Jonathan Schaeffer, University of Alberta.
Playable demo of the perfect checkers player, which is the result of the recent solution of the game after almost 20 years of computation!
MoGo: Monte Carlo Go. Sylvain Gelly, Google.
Opportunity to witness and play against one of the strongest currently available Go programs based on Monte-Carlo search.
Polaris: Computer Poker. Michael Bowling, University of Alberta.
Chance to match your wits against the first poker computer player ever to challenge top human professional players.
Halo 3 AI. Max Dyckhoff, Bungie Studios.
Display of the inner workings of the Artificial Intelligence driving the Earth threatening covenants in the blockbuster game Halo 3.

Panel Discussion

We devoted the last hour of the workshop to a panel discussion. The panel was composed of the invited speakers, plus as a special guest, the speaker of the NIPS 2007 opening night banquet:

Luis von Ahn, Carnegie Mellon University. Expert in human computation; inventor of games such as the ESP game for image labelling to make use of human brain cycles.

People

Thore Graepel