Abstract: In many emerging applications, it is of paramount interest to learn hidden parameters from data. For example, self-driving cars may use onboard cameras to identify pedestrians, highway lanes, or traffic signs in various light and weather conditions. Problems such as these can be framed as classification, regression, or risk minimization in general, at the heart of which lies stochastic optimization and machine learning. In many practical scenarios, distributed and decentralized methods for optimization and machine learning are preferable as they benefit from a divide-and-conquer approach towards storage and computation at the expense of local (short-range) communication. In this talk, I will present our recent work that develops a novel algorithmic framework to address various aspects of distributed stochastic first-order optimization methods for non-convex problems. A major focus will be to characterize regimes where distributed solutions outperform their centralized counterparts and lead to optimal convergence rates and guarantees. Moreover, I will characterize certain desirable attributes of the distributed methods including the notions of linear speedup and network-independent convergence rates. Throughout the talk, I will demonstrate such key aspects with the help of provable theoretical results and numerical experiments on real data.
Usman A. Khan is a Professor of Electrical and Computer Engineering (ECE) at Tufts University with a secondary appointment in Computer Science. His research interests include signal processing, stochastic systems, optimization and control, and machine learning. He has published extensively in these topics with more than 120 articles in journals and conference proceedings and holds multiple patents. Recognition of his work includes the prestigious National Science Foundation (NSF) Career award, several federally funded projects and NSF REU awards, an IEEE journal cover, and several news articles including two in IEEE Spectrum. His work received the 2022 EURASIP Best Paper Award for articles published in the EURASIP Journal on Advances in Signal Processing over 2017-2021, and Best Student Paper awards at the IEEE International Conference on Networking, Sensing and Control (2014), and at the IEEE Asilomar Conference (2014 and 2016).
Dr. Khan received his B.S. degree in 2002 from University of Engineering and Technology, Pakistan, M.S. degree in 2004 from University of Wisconsin-Madison, USA, and Ph.D. degree in 2009 from Carnegie Mellon University, USA, all in ECE. He was a postdoctoral researcher in the GRASP lab at the University of Pennsylvania and also has held a Visiting Professor position at KTH, Sweden. Dr. Khan is an IEEE Senior Member and is an elected full member of the Sensor Array and Multichannel Technical Committee with the IEEE Signal Processing Society since 2019, where he was an Associate member from 2010 to 2019. He was an elected full member of the IEEE Big Data Special Interest Group from 2017 to 2019, and has served on the IEEE Young Professionals Committee and on the IEEE Technical Activities Board. He has served as an Associate Editor on several publications: IEEE Transactions on Smart Grid (2014-2017); IEEE Control System Letters (2018-2020), IEEE Transactions on Signal and Information Processing over Networks (2019-current); IEEE Open Journal of Signal Processing (2019-current); and IEEE Transactions on Signal Processing (2021-current). He served as the Chief Editor for the Proceedings of the IEEE special issue on Optimization for Data-driven Learning and Control (Nov. 2020), and as a Guest Associate Editor for the IEEE Control System Letters special issue on Learning and Control (Nov. 2020). He served as the Technical Area Chair for the Networks track in 2020 IEEE Asilomar Conference on Signals Systems and Computers and for the Signal Processing for Self Aware and Social Autonomous Systems track at the 1st IEEE International Conference on Autonomous Systems (Aug. 2021).