Faeze Moradi Kalarde
ABSTRACT
Combining over-the-air uplink transmission and multi-antenna beamforming can improve the efficiency of federated learning (FL). However, to mitigate the significant aggregation error due to communication noise and signal distortion, pre-processing of device signals and post-processing at the server are required. In this paper, we study the optimization of receiver beamforming and device transmit weights in over-the-air FL, to minimize the total transmit power in each communication round while guaranteeing the convergence of FL. We establish sufficient convergence conditions based on the analysis of gradient descent with error and formulate a power minimization problem. An alternating optimization approach is then employed to decompose the problem into tractable subproblems, and efficient solutions are developed for these subproblems. Our proposed method is evaluated through simulation on standard image classification tasks, demonstrating its effectiveness in achieving substantial reductions in transmit power compared with existing alternatives.
- Abdullatif Albaseer, Mohamed Abdallah, Ala Al-Fuqaha, and Aiman Erbad. 2022. Fine-Grained Data Selection for Improved Energy Efficiency of Federated Edge Learning. IEEE Transactions on Network Science and Engineering 9, 5 (2022), 3258--3271.Google Scholar
Cross Ref
- Mohammad Mohammadi Amiri and Deniz Gündüz. 2020. Federated learning over wireless fading channels. IEEE Trans. Wireless Commun. 19, 5 (2020), 3546--3557.Google ScholarCross Ref
- Mohammad Mohammadi Amiri and Deniz Gündüz. 2020. Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air. IEEE Trans. Signal Process. 68 (2020), 2155--2169.Google ScholarCross Ref
- Mingzhe Chen, Zhaohui Yang, Walid Saad, Changchuan Yin, H Vincent Poor, and Shuguang Cui. 2020. A joint learning and communications framework for federated learning over wireless networks. IEEE Trans. Wireless Commun. 20, 1 (2020), 269--283.Google ScholarDigital Library
- Min Dong and Qiqi Wang. 2020. Multi-group multicast beamforming: Optimal structure and efficient algorithms. IEEE Trans. Signal Process. 68 (2020), 3738-- 3753.Google ScholarDigital Library
- Michael P Friedlander and Mark Schmidt. 2012. Hybrid deterministic-stochastic methods for data fitting. SIAM Journal on Scientific Computing 34, 3 (2012), A1380--A1405.Google ScholarDigital Library
- Jochen Gorski, Frank Pfeuffer, and Kathrin Klamroth. 2007. Biconvex sets and optimization with biconvex functions: a survey and extensions. Mathematical Methods of Operations Research 66, 3 (2007), 373--407.Google ScholarCross Ref
- Huayan Guo, An Liu, and Vincent KN Lau. 2020. Analog gradient aggregation for federated learning over wireless networks: Customized design and convergence analysis. IEEE Internet of Things Journal 8, 1 (2020), 197--210.Google ScholarCross Ref
- Kun Guo, Zihan Chen, Howard H. Yang, and Tony Q. S. Quek. 2022. Dynamic Scheduling for Heterogeneous Federated Learning in Private 5G Edge Networks. IEEE Journal of Selected Topics in Signal Processing 16, 1 (2022), 26--40.Google ScholarCross Ref
- Wei Guo, Ran Li, Chuan Huang, Xiaoqi Qin, Kaiming Shen, and Wei Zhang. 2022. Joint Device Selection and Power Control for Wireless Federated Learning. IEEE J. Sel. Areas Commun. 40, 8 (Aug. 2022), 2395--2410.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- Yun Ji, Zhoubin Kou, Xiaoxiong Zhong, Hangfan Li, Fan Yang, and Sheng Zhang. 2022. Client Selection and Bandwidth Allocation for Federated Learning: An Online Optimization Perspective. In Proc. IEEE Global Commun. Conf. (GLOBECOM).Google ScholarCross Ref
- Tao Jiang and Yuanming Shi. 2019. Over-the-air computation via intelligent reflecting surfaces. In Proc. IEEE Global Commun. Conf. (GLOBECOM).Google ScholarDigital Library
- Minsik Kim, A. Lee Swindlehurst, and Daeyoung Park. 2023. Beamforming Vector Design and Device Selection in Over-the-Air Federated Learning. IEEE Trans. Wireless Commun. 54, 6 (2023), 2239--2251.Google Scholar
- Alex Krizhevsky and Geoffrey E. Hinton. 2009. Learning multiple layers of features from tiny images. Master's thesis. Univ. of Toronto (UofT), Toronto, Canada.Google Scholar
- Premchandra Kumar, Bhushan Patil, and Suraj Ram. 2015. Selection of radio propagation model for long-term evolution (LTE) network. International Journal of Engineering Research and General Science 3, 1 (2015), 373--379.Google Scholar
- Yann LeCun, Corinna Cortes, and Christopher J. Burges. 1998. The MNIST Database of Handwritten Digits. [Online]. Available: http://yann.lecun.com/exdb/mnist/ (1998).Google Scholar
- Yangchen Li, Ying Cui, and Vincent Lau. 2021. Optimization-Based GenQSGD for Federated Edge Learning. In Proc. IEEE Global Commun. Conf. (GLOBECOM).Google ScholarDigital Library
- Hang Liu, Xiaojun Yuan, and Ying-Jun Angela Zhang. 2021. Reconfigurable intelligent surface enabled federated learning: A unified communication-learning design approach. IEEE Trans. Wireless Commun. 20, 11 (2021), 7595--7609.Google ScholarDigital Library
- Bing Luo, Xiang Li, ShiqiangWang, Jianwei Huang, and Leandros Tassiulas. 2021. Cost-Effective Federated Learning in Mobile Edge Networks. IEEE Journal on Selected Areas in Communications 39, 12 (2021), 3606--3621.Google ScholarCross Ref
- Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Proc. Int. Conf. on Artificial Intelligence and Statistics (AISTATS).Google Scholar
- Faeze Moradi Kalarde, Min Dong, Ben Liang, Ho Ting Cheng, and Yahia Ahmed. 2023. Joint beamforming and device selection in federated learning with overthe- air aggregation. arXiv preprint arXiv:2302.14336 (2023).Google Scholar
- Boris T. Polyak. 1987. Introduction to optimization. Optimization Software.Google Scholar
- Tomer Sery and Kobi Cohen. 2020. On analog gradient descent learning over multiple access fading channels. IEEE Trans. Signal Process. 68 (2020), 2897--2911.Google ScholarCross Ref
- Tomer Sery, Nir Shlezinger, Kobi Cohen, and Yonina C Eldar. 2021. Over-the-air federated learning from heterogeneous data. IEEE Trans. Signal Process. 69 (2021), 3796--3811.Google ScholarDigital Library
- N. D. Sidiropoulos, T. N. Davidson, and Z.-Q. Luo. 2006. Transmit beamforming for physical-layer multicasting. IEEE Trans. Signal Process. 54, 6 (Jun. 2006), 2239--2251.Google ScholarCross Ref
- Yuxuan Sun, Sheng Zhou, and Deniz Gündüz. 2020. Energy-Aware Analog Aggregation for Federated Learning with Redundant Data. In Proc. IEEE Int. Conf. on Communications (ICC).Google ScholarCross Ref
- Yuxuan Sun, Sheng Zhou, Zhisheng Niu, and Deniz Gündüz. 2022. Dynamic Scheduling for Over-the-Air Federated Edge Learning With Energy Constraints. IEEE Journal on Selected Areas in Communications 40, 1 (2022), 227--242. https: //doi.org/10.1109/JSAC.2021.3126078Google ScholarDigital Library
- Shuo Wan, Jiaxun Lu, Pingyi Fan, Yunfeng Shao, Chenghui Peng, and Khaled B. Letaief. 2021. Convergence Analysis and System Design for Federated Learning Over Wireless Networks. IEEE Journal on Selected Areas in Communications 39, 12 (2021), 3622--3639.Google ScholarCross Ref
- Juncheng Wang, Min Dong, Ben Liang, Gary Boudreau, and Hatem Abou-zeid. 2022. Online Model Updating with Analog Aggregation in Wireless Edge Learning. In Proc. IEEE Conf. on Computer Communications (INFOCOM).Google ScholarDigital Library
- Jie Xu and Heqiang Wang. 2021. Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective. IEEE Transactions on Wireless Communications 20, 2 (2021), 1188--1200.Google ScholarDigital Library
- Kai Yang, Tao Jiang, Yuanming Shi, and Zhi Ding. 2020. Federated learning via over-the-air computation. IEEE Trans. Wireless Commun. 19, 3 (2020), 2022--2035.Google ScholarDigital Library
- Kai Yang, Yuanming Shi, Yong Zhou, Zhanpeng Yang, Liqun Fu, and Wei Chen. 2020. Federated machine learning for intelligent IoT via reconfigurable intelligent surface. IEEE Network 34, 5 (2020), 16--22.Google ScholarCross Ref
- Jingjing Zheng, Kai Li, Eduardo Tovar, and Mohsen Guizani. 2021. Federated learning for energy-balanced client selection in mobile edge computing. In Proc. IEEE Int. Wireless Commun. Mobile Comput. Conf. (IWCMC).Google ScholarCross Ref
- Guangxu Zhu, Li Chen, and Kaibin Huang. 2018. MIMO over-the-air computation: Beamforming optimization on the Grassmann manifold. In Proc. IEEE Global Commun. Conf. (GLOBECOM).Google ScholarDigital Library
- Guangxu Zhu, Yong Wang, and Kaibin Huang. 2019. Broadband analog aggregation for low-latency federated edge learning. IEEE Trans. Wireless Commun. 19, 1 (2019), 491--506.Google ScholarCross Ref
Index Terms
- Power Minimization in Federated Learning with Over-the-air Aggregation and Receiver Beamforming
Recommendations
Adaptive MMSE receiver with beamforming for DS/CDMA systems
The minimum mean-squared error (MMSE) receiver is a linear filter which can suppress multiple access interference (MAI) effectively in direct-sequence code-division multiple-access (CDMA) communications. An antenna array is also an efficient scheme for ...
Quantifying the power loss when transmit beamforming relies on finite-rate feedback
Transmit beamforming achieves optimal performance in systems with multiple transmit antennas and a single receive antenna from both the capacity and the received signal-to-noise ratio (SNR) perspectives but ideally requires perfect channel knowledge at ...
Transmit beamforming with power adaptation in downlink multi-user systems
This paper addresses power adaptation in the context of beamforming in the downlink of a cellular system. Specifically, we consider a system consisting of a base station (BS) with multiple antennas and multiple users each with a single antenna, in which ...
Comments