Automatic differentiation and machine learning
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD) is a technique for calculating derivatives efficiently and accurately, established in fields such as computational fluid dynamics, nuclear engineering, and atmospheric sciences. Despite its advantages and use in other fields, machine learning practitioners have been little influenced by AD and make scant use of available tools. We survey the intersection of AD and machine learning, cover applications where AD has the potential to make a big impact, and report on some recent developments in the adoption of this technique. We also aim to dispel some misconceptions that we would contend have impeded the use of AD within the machine learning community.
Speaker Details
Gunes Baydin is a postdoctoral researcher at the Hamilton Institute and the Department of Computer Science of the National University of Ireland Maynooth. His current work, with Prof. Barak Pearlmutter, involves automatic differentiation (AD) and its potential applications to machine learning. Our research aims to add exact first-class differentiation operators to the lambda calculus, allowing numeric algorithms and scientific computations to be expressed in a very clear and succinct way. We are designing and implementing languages embodying compositionality, where gradient optimization can be performed efficiently and concurrently on many nested levels in a system.
- Date:
- Speakers:
- Gunes Baydin
- Affiliation:
- Maynooth University
-
-
Jeff Running
-
