Extensions of Bayesian Optimization for Real-World Applications

Bayesian Optimization (BO) is a popular approach in statistics and machine learning for the global optimization of expensive blackbox functions. It has strong theoretical foundations and also yields state-of-the-art empirical results for optimizing functions with few all-continuous inputs. However, many blackbox optimization problems in real-world applications do not fit into this scope. For example, the “algorithm configuration” problem of identifying the best instantiation of a parametric algorithm poses various challenges to BO, including: high dimensionality, mixed discrete/continuous optimization, function evaluations of varying costs, partial function evaluations that only yield a bound on the true function value, and computational efficiency with tens of thousands of function evaluations. In this talk, I discuss recent work at UBC that extends BO to handle these challenges. Empirical results demonstrate that the resulting methods achieve state-of-the-art performance for the configuration of algorithms for solving hard combinatorial problems and for the configuration of machine learning classifiers.

Based on joint work with Holger Hoos, Kevin Leyton-Brown, and Nando de Freitas and his machine learning group.

Speaker Details

Frank Hutter is a Postdoctoral Research Fellow at the Computer Science Department of the University of British Columbia (UBC) in Vancouver. His research is at the intersection of machine learning and optimization, with a particular focus on reasoning about and improving the performance of algorithms for solving hard computational problems. Frank obtained his PhD from UBC in 2009 and his MSc from Darmstadt University of Technology (Germany) in 2004. He worked as a summer intern at MSR Cambridge in 2005 and at NASA Ames in 2002 & 2003. Frank received the 2010 CAIAC doctoral dissertation award, and together with his co-authors, several best paper awards (including the 2010 IJCAI/JAIR Best Paper Prize) and numerous prizes in recent international SAT competitions.

Date:
Speakers:
Frank Hutter
Affiliation:
University of British Columbia
    • Portrait of Jeff Running

      Jeff Running

Series: Microsoft Research Talks