ArtIAMAS Seminar Series
Co-organized by UMBC, UMCP, and Army Research Lab

Iterative Preconditioning for
Accelerating Machine Learning Problems

Nikhil Chopra
Mechanical Engineering, UMCP

12-1 ET Wed. 27 April 2022, WebEx

We study a new approach to accelerating machine learning problems in this talk. The system comprises multiple agents, each with a set of local data points and an associated local cost function. The agents are connected to a server, and there is no inter-agent communication. The agents’ goal is to learn a parameter vector that optimizes the aggregate of their local costs without revealing their local data points. We propose an iterative preconditioning technique to mitigate the deleterious effects of the cost function’s conditioning on the convergence rate of distributed gradient-descent. Unlike the conventional preconditioning techniques, the pre-conditioner matrix in our proposed technique updates iteratively to facilitate implementation on the distributed network. In the particular case when the minimizer of the aggregate cost is unique, our algorithm converges superlinearly. We demonstrate our algorithm’s superior performance in machine learning, distributed estimation, and beamforming problems, thereby demonstrating the proposed algorithm’s efficiency for distributively solving nonconvex optimization problems.

Dr. Nikhil Chopra is a Professor in the Department of Mechanical Engineering at the University of Maryland, College Park. He received a Bachelor of Technology (Honors) degree in Mechanical Engineering from the Indian Institute of Technology, Kharagpur, India, in 2001, an M.S. degree in General Engineering in 2003, and a Ph.D. degree in Systems and Entrepreneurial Engineering in 2006 from the University of Illinois at Urbana-Champaign. His current research interests are in the areas of nonlinear control, robotics, and machine learning. He is the co-author of the book Passivity-Based Control and Estimation in Networked Robotics. He is currently an Associate Editor of Automatica and was previously an Associate Editor of IEEE Transactions on Control of Network Systems and IEEE Transactions on Automatic Control.