Instructor  Manik Varma 
Coordinator  K. K. Biswas 
Student Volunteer  Rahul Kumar 
Credits  1 
Classroom  201, IIA, Bharti Building 
Timings  2:00  3:30 PM on Tuesdays and Fridays. 
Kaggle Competitions  Data Set 1 Data Set 2 Data Set 3 Data Set 4 Data Set 5 Data Set 6 
Support Vector Machines (SVMs) are one of the most popular tools used to tackle supervised machine learning problems. They have defined the stateoftheart on multiple benchmark tasks and are easy to use requiring relatively little machine learning expertise. In this course, we will take a detailed look at the area of supervised machine learning from an SVM perspective. We will start by introducing machine learning and SVM basics and will then go into four current areas of SVM research: (a) formulations; (b) kernels and kernel learning; (c) optimization and (d) prediction. Emphasis will be placed on both the mathematical treatment of SVMs as well as the practical aspects of optimization and implementation. Students are expected to be comfortable with linear algebra, probability/statistics and coding in C and Matlab. By the end of the course, students should be able to use SVMs to solve real world machine learning problems as well as take up research projects extending SVMs in novel directions.
Lectures
Lecture 1 (30072013) Introduction to ML, features, over fitting and generalization, noise and prior knowledge 
Lecture 1 notes Chapter 1 of DHS 
Lecture 2 (02082013) Bayesian vs MAP vs ML approaches 
Lecture 2 notes Chapter 1 of PRML Chapter 3 of DHS 
Lectures 3 (06082013) and 4 (13082013) Generative vs Discriminative approaches 
Lecture 3 notes Lecture 4 notes Chapter from Mitchell Ng & Jordan NIPS 01 
Lectures 5 (16082013) and 6 (23082013) Linear SVMs Primal optimization  gradient descent and stochastic gradient descent 
Lecture 5 notes Lecture 6 notes Tutorial by Chris Burges Pegasos 


Lecture 7 (15102013) Linear SVMs continued Dual optimization  dual coordinate ascent 
Lecture 7 notes Liblinear 


Lecture 8 (18102013) Linear SVMs continued Cutting plane optimization 
Lecture 8 notes Joachims KDD 06 


Lectures 9 (29102013) and 10 (01012013) Kernels Block coordinate ascent optimization 
Lecture 9 notes Lecture 10 notes SMO  Fan, Chen and Lin JMLR 05 
Code
Most code was written five minutes before the start of each lectureand comes with no guarantees, comments or documentation. In particular, no attempt has been made to bulletproof the code. For instance, if there is no feasible solution for your parameter settings then the figure plotting subroutines will crash (the LR and SVM learning routines should be stable). In any case, run the code at your own peril.
Recommended Reading