Prof. Osvaldo Simeone

Title

Channel Noise as Monte Carlo Sampling: Efficient Bayesian Distributed Learning in Wireless Systems

Abstract

Conventional frequentist learning, as assumed by existing federated learning protocols, is limited in its ability to quantify uncertainty, incorporate prior knowledge, guide active learning, and enable continual model updates. Bayesian learning provides a principled approach to address all these limitations, at the cost of an increase in computational complexity. A standard approach to implement Bayesian learning is through Monte Carlo sampling, whereby the learner generates samples (approximately) drawn from the posterior distribution to enable Gibbs or ensemble predictors. Focusing on wireless distributed Bayesian learning, this talk introduces the idea of channel-driven MC sampling: Rather than treating channel noise as a nuisance to be mitigated, channel-driven sampling utilizes channel noise as an integral part of the MC sampling process.
Two specific settings are studied: a wireless data center system encompassing a central server and multiple distributed workers, and a federated system with an edge access point and distributed agents. For the first setting, the talk investigates for the first time the design of distributed one-shot, or "embarrassingly parallel", Bayesian learning protocols via consensus Monte Carlo (CMC), while for the second we consider Langevin MC sampling schemes based on multiple communication rounds. In both cases, uncoded transmission is introduced not only as a means to implement "over-the-air" computing but also as a way to enable channel-driven sampling. Simulation results demonstrate that, if properly accounted for, channel noise can indeed contribute to MC sampling, and does not necessarily decrease the accuracy level.

Biography

Osvaldo Simeone is a Professor of Information Engineering with the Centre for Telecommunications Research at the Department of Engineering of King's College London, where he directs the King's Communications, Learning and Information Processing lab. He received an M.Sc. degree (with honors) and a Ph.D. degree in information engineering from Politecnico di Milano, Milan, Italy, in 2001 and 2005, respectively. From 2006 to 2017, he was a faculty member of the Electrical and Computer Engineering (ECE) Department at New Jersey Institute of Technology (NJIT), where he was affiliated with the Center for Wireless Information Processing (CWiP). His research interests include information theory, machine learning, wireless communications, and neuromorphic computing. Dr Simeone is a co-recipient of the IEEE Veh. Techn. Society 2021 Jack Neubauer Memorial Award, 2019 IEEE Communication Society Best Tutorial Paper Award, the 2018 IEEE Signal Processing Best Paper Award, the 2017 JCN Best Paper Award, the 2015 IEEE Communication Society Best Tutorial Paper Award and of the Best Paper Awards of IEEE SPAWC 2007 and IEEE WRECOM 2007. He was awarded a Consolidator grant by the European Research Council (ERC) in 2016. His research has been supported by the U.S. NSF, the ERC, the Vienna Science and Technology Fund, as well as by a number of industrial collaborations. He currently serves in the editorial board of the IEEE Signal Processing Magazine and is the vice-chair of the Signal Processing for Communications and Networking Technical Committee of the IEEE Signal Processing Society. He was a Distinguished Lecturer of the IEEE Information Theory Society in 2017 and 2018, and he is currently a Distinguished Lecturer of the IEEE Communications Society. Dr Simeone is a co-author of two monographs, two edited books published by Cambridge University Press, and more than 150 research journal papers. He is a Fellow of the IET and of the IEEE.