Mohak Chadha
ABSTRACT
Federated learning (FL) enables resource-constrained edge devices to learn a shared Machine Learning (ML) or Deep Neural Network (DNN) model, while keeping the training data local and providing privacy, security, and economic benefits. However, building a shared model for heterogeneous devices such as resource-constrained edge and cloud makes the efficient management of FL-clients challenging. Furthermore, with the rapid growth of FL-clients, the scaling of FL training process is also difficult.
In this paper, we propose a possible solution to these challenges: federated learning over a combination of connected Function-as-a-Service platforms, i.e., FaaS fabric offering a seamless way of extending FL to heterogeneous devices. Towards this, we present FedKeeper, a tool for efficiently managing FL over FaaS fabric. We demonstrate the functionality of FedKeeper by using three FaaS platforms through an image classification task with a varying number of devices/clients, different stochastic optimizers, and local computations (local epochs).
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16). 265--283.Google Scholar
Digital Library
- Ioana Baldini, Paul Castro, Kerry Chang, Perry Cheng, Stephen Fink, Vatche Ishakian, Nick Mitchell, Vinod Muthusamy, Rodric Rabbah, Aleksander Slominski, et al. 2017. Serverless computing: Current trends and open problems. In Research Advances in Cloud Computing. Springer, 1--20.Google Scholar
- Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz. 2019. Cirrus: A serverless framework for end-to-end ml workflows. In Proceedings of the ACM Symposium on Cloud Computing. 13--24.Google ScholarDigital Library
- Ryan Chard, Yadu Babuji, Zhuozhao Li, Tyler Skluzacek, Anna Woodard, Ben Blaiszik, Ian Foster, and Kyle Chard. 2020. FuncX: A Federated Function Serving Fabric for Science. In Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing (Stockholm, Sweden) (HPDC '20). Association for Computing Machinery, New York, NY, USA, 65--76. https://doi.org/10.1145/3369583.3392683Google ScholarDigital Library
- Mingzhe Chen, Ursula Challita, Walid Saad, Changchuan Yin, and Mérouane Debbah. 2019. Artificial neural networks-based machine learning for wireless networks: A tutorial. IEEE Communications Surveys & Tutorials 21, 4 (2019), 3039--3071.Google ScholarDigital Library
- Mung Chiang and Tao Zhang. 2016. Fog and IoT: An overview of research opportunities. IEEE Internet of Things Journal 3, 6 (2016), 854--864.Google ScholarCross Ref
- Bart Custers, Alan M Sears, Francien Dechesne, Ilina Georgieva, Tommaso Tani, and Simone Van der Hof. [n.d.]. EU Personal Data Protection in Policy and Practice. Springer.Google Scholar
- Google Cloud Functions. 2020. https://cloud.google.com/functions. Accessed on 09/24/2020.Google Scholar
- Joseph M Hellerstein, Jose Faleiro, Joseph E Gonzalez, Johann Schleier-Smith, Vikram Sreekanti, Alexey Tumanov, and Chenggang Wu. 2018. Serverless computing: One step forward, two steps back. arXiv preprint arXiv:1812.03651 (2018).Google Scholar
- Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the cloud: Distributed computing for the 99%. In Proceedings of the 2017 Symposium on Cloud Computing. 445--451.Google ScholarDigital Library
- AWS Lambda. 2020. https://aws.amazon.com/lambda/. Accessed on 09/24/2020.Google Scholar
- Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.Google ScholarCross Ref
- Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine 37, 3 (2020), 50--60.Google ScholarCross Ref
- X. Li, Y. Li, T. Liu, J. Qiu, and F. Wang. 2009. The Method and Tool of Cost Analysis for Cloud Computing. In 2009 IEEE International Conference on Cloud Computing. 93--100.Google Scholar
- Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics. PMLR, 1273--1282.Google Scholar
- Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML.Google Scholar
- Hai Duc Nguyen, Chaojie Zhang, Zhujun Xiao, and Andrew A. Chien. 2019. Real-Time Serverless: Enabling Application Performance Guarantees. In Proceedings of the 5th International Workshop on Serverless Computing (Davis, CA, USA) (WOSC '19). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3366623.3368133Google ScholarDigital Library
- Nvidia. 2020. The performance of modern AI for millions of devices NVIDIA Jetson Nano. https://www.nvidia.com/de-de/autonomous-machines/embedded-systems/jetson-nano/. Accessed on 09/24/2020.Google Scholar
- OpenFaaS. 2020. https://www.openfaas.com/. Accessed on 09/24/2020.Google Scholar
- Apache OpenWhisk. 2020. https://openwhisk.apache.org/. Accessed on 09/24/2020.Google Scholar
- rancher/k3s: Lightweight Kubernetes. 2020. https://github.com/rancher/k3s. Accessed on 09/24/2020.Google Scholar
- Vaishaal Shankar, Karl Krauth, Qifan Pu, Eric Jonas, Shivaram Venkataraman, Ion Stoica, Benjamin Recht, and Jonathan Ragan-Kelley. 2018. Numpywren: Serverless linear algebra. arXiv preprint arXiv:1810.09679 (2018).Google Scholar
- CNCF Serverless WG. March 2018. Cncf wg-serverless whitepaper v1.0. https://gw.alipayobjects.com/os/basement_prod/24ec4498-71d4-4a60-b785-fa530456c65b.pdfGoogle Scholar
Index Terms
- Towards Federated Learning using FaaS Fabric
Recommendations
SeBS: a serverless benchmark suite for function-as-a-service computing
Middleware '21: Proceedings of the 22nd International Middleware ConferenceFunction-as-a-Service (FaaS) is one of the most promising directions for the future of cloud services, and serverless functions have immediately become a new middleware for building scalable and cost-efficient microservices and appli cations. However, ...
Architectural Implications of Function-as-a-Service Computing
MICRO '52: Proceedings of the 52nd Annual IEEE/ACM International Symposium on MicroarchitectureServerless computing is a rapidly growing cloud application model, popularized by Amazon's Lambda platform. Serverless cloud services provide fine-grained provisioning of resources, which scale automatically with user demand. Function-as-a-Service (FaaS)...
Migrating from microservices to serverless: an IoT platform case study
WoSC '22: Proceedings of the Eighth International Workshop on Serverless ComputingMicroservice architecture is the common choice for developing cloud applications these days since each individual microservice can be independently modified, replaced, and scaled. As a result, application development and operating cloud infrastructure ...
Comments