Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Orthant-Wise Limited-memory Quasi-Newton Optimizer for L1-regularized Objectives

The Orthant-Wise Limited-memory Quasi-Newton algorithm (OWL-QN) is a numerical optimization procedure for finding the optimum of an objective of the form {smooth function} plus {L1-norm of the parameters}. It has been used for training log-linear models (such as logistic regression) with L1-regularization. The algorithm is described in "Scalable training of L1-regularized log-linear models" by Galen Andrew and Jianfeng Gao. This implementation includes built-in capacity to train logistic regression or least-squares models with L1 regularization. It is also possible to use OWL-QN to optimize any arbitrary smooth convex loss plus L1 regularization by defining the function and its gradient using the supplied "DifferentiableFunction" class, and passing an instance of the function to the OWLQN object. For more information, please read the included file README.txt. Also included in the distribution are the ICML paper and slide presentation.

Details

TypeDownload
File NameOWLQN.zip
Version1.1.2
Date Published11 December 2007
Download Size0.64 MB

Note By installing, copying, or otherwise using this software, you agree to be bound by the terms of its license. Read the license.

Overview

The Orthant-Wise Limited-memory Quasi-Newton optimizer (OWL-QN) implements an algorithm for optimizing L1-regularized objectives. It works with any smooth convex loss plus the L1 norm of the parameters, such as log-linear models (e.g. logistic regression and Markov random fields) and squared error (e.g. regression with the LASSO).

This package contains C++ source code allowing you to plug in any differentiable, convex loss you choose, just by defining a method to return the function’s value and gradient at any point. In addition, there is a standalone main method for optimizing the weights of logistic regression or least-squares models. For usage and other information, please see the enclosed file README.txt.

New Features

Here’s what’s new in this release:

  • Implementation of the OWL-QN optimization algorithm
  • Standalone trainer for logistic regression or least-squares (with compiled win32 executable)
  • Related paper and ICML presentation

System Requirements

To run this software, you’ll need the following:

  • A C++ compiler (if you want to define your own loss function or compile the standalone trainer yourself)
  • or, a computer running MS Windows (if you only want to run the standalone logistic-regression or least-squares trainer using the included executable)

Installation Instructions

To install OWL-QN, do the following:

  1. Download and unzip the package.
  2. Read README.txt for usage and other information.
  3. Run the standalone trainer, or link the source into your application and compile.