Doubly Stochastic Primal-Dual Coordinate Method for Empirical Risk Minimization
We proposed a doubly stochastic primal-dual coordinate optimization algorithm for regularized empirical risk minimization that can be formulated as a saddle-point problem using convex conjugate functions. Different from the existing coordinate methods, the proposed method randomly samples both primal and dual coordinates to update solutions. The convergence of our method is established in both the solution’s distance to optimality and the primal-dual objective gap. When applied to the data matrix factorized as a product of two smaller matrices, we show that the proposed method has a lower overall complexity than other coordinate methods, especially, when data size is large. Furthermore, we also generalize the method for convex saddle-point problem with a block-wise decomposable structure and give a lower bound for the iteration complexity of a family of primal-dual coordinate method for solving such a problem. This is joint work with Qihang Lin (UIowa) and Tianbao Yang (UIowa).
- Series:
- Microsoft Research Talks
- Date:
- Speakers:
- Wei Yu
- Affiliation:
- Carnegie Mellon University
-
-
Lin Xiao
Senior Principal Researcher
-
-
Series: Microsoft Research Talks
-
-
-
-
Galea: The Bridge Between Mixed Reality and Neurotechnology
Speakers:- Eva Esteban,
- Conor Russomanno
-
Current and Future Application of BCIs
Speakers:- Christoph Guger
-
Challenges in Evolving a Successful Database Product (SQL Server) to a Cloud Service (SQL Azure)
Speakers:- Hanuma Kodavalla,
- Phil Bernstein
-
Improving text prediction accuracy using neurophysiology
Speakers:- Sophia Mehdizadeh
-
-
DIABLo: a Deep Individual-Agnostic Binaural Localizer
Speakers:- Shoken Kaneko
-
-
Recent Efforts Towards Efficient And Scalable Neural Waveform Coding
Speakers:- Kai Zhen
-
-
Audio-based Toxic Language Detection
Speakers:- Midia Yousefi
-
-
From SqueezeNet to SqueezeBERT: Developing Efficient Deep Neural Networks
Speakers:- Sujeeth Bharadwaj
-
Hope Speech and Help Speech: Surfacing Positivity Amidst Hate
Speakers:- Monojit Choudhury
-
-
-
-
-
'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project
Speakers:- Peter Clark
-
Checkpointing the Un-checkpointable: the Split-Process Approach for MPI and Formal Verification
Speakers:- Gene Cooperman
-
Learning Structured Models for Safe Robot Control
Speakers:- Ashish Kapoor
-
-