Adaptive Margin Machines for Classification Learning

We propose a new learning algorithm for kernel classifiers. Former approaches like Quadratic Programming Machines (SVMs), and Linear Programming Machines were based on minimization of a regularized margin loss where the margin was treated equivalently for each training pattern. We propose a reformulation of the minimization problem such that adaptive margins (AMM) for each training pattern are utilized. Furthermore, we give bounds on the generalization error of AMMs which justify their robustness against outliers. We show experimentally that the generalization error of AMMs is comparable to QP- and LP-Machines on benchmark datasets from the UCI repository.

References

• Ralf Herbrich and Jason Weston. Adaptive Margin Support Vector Machines for Classification Learning. . In Proceedings of the Ninth International Conference on Artificial Neural Networks, pages 880-885, 1999. (Gzipped PostScript)
• Jason Weston and Ralf Herbrich. Adaptive Margin Support Vector Machines. . Advances in Large Margin Classifiers, pages 281-296, 2000. (Gzipped PostScript).