Skip to main content
SpringerLink
Log in

Use of edge resources for DNN model maintenance in 5G IoT networks

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Internet-of-Things (IoT) services become closely coupled with machine learning and cloud computing, where the 5G network provides the connectivity for the IoT devices. The 5G network can be used not only for connecting the IoT devices to the cloud servers, but also for providing computing resources for ’edge computing’. In this paper, we propose to use the edge node resources of the 5G network for ’inferencing’ and ’training’ the deep neural network (DNN) models for massive IoT services. More specifically, two types of 5G edge nodes are utilized to this end: (i) the ‘IoT controller’, which functions as a 5G-UE (user equipment), (ii) the ‘edge controller’, which is collocated with 5G-UPF (user plane function) in the 5G core network. In the proposed scheme, the downsized DNN models are executed and trained at the IoT controllers. At the edge controller, a deep reinforcement learning (DRL) algorithm is executed to determine the downsizing configuration and the training configuration of the DNN models. The resource constraints of the IoT controllers are considered in these decisions. Extensive evaluations with various DNN models show the effectiveness of the proposed scheme. We show that the proposed scheme achieves proper load balancing even when the resource capacity of individual IoT controllers is very low. For example, fairly complex DNN models for computer vision can be effectively supported by using IoT controllers equipped with the resource capacity of NVIDIA Jetson Nano.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. RedCap stands for a reduced capability. The expected use cases include wearable devices, industrial wireless sensors and video surveillance. It cuts down the device bandwidth, the antenna configuration, the supported number of downlink (DL) MIMO layers and etc. In this way, RedCap can substantially extend the battery time and reduce the power consumption of IoT devices.

  2. It is a control plane function which deals with a PDU session like a PDN connection in 4 G core networks).

  3. A VM is a complete isolated space for processing a maintenance request in terms of computing resources. It requires a VM provisioning process. It means the entire process of procuring a VM image and the complete pre-trained model from the cloud, allocating resources for the VM and booting-up(initializing) the VM.

  4. A low-cost low-powered GPU-attached microcomputer of NVIDIA [13].

  5. An open-source hypervisor https://www.virtualbox.org/

  6. A high-level wrapper of Tensorflow at https://keras.io/

References

  1. Qi, X., Liu, C.: Enabling deep learning on iot edge: Approaches and evaluation. In: 2018 IEEE/ACM Symposium on Edge Computing (SEC), pp. 367–372 (2018). IEEE

  2. Calo, S.B., Touna, M., Verma, D.C., Cullen, A.: Edge computing architecture for applying ai to iot. In: 2017 IEEE International Conference on Big Data (Big Data), pp. 3012–3016 (2017). IEEE

  3. Zhu, G., Liu, D., Du, Y., You, C., Zhang, J., Huang, K.: Toward an intelligent edge: Wireless communication meets machine learning. IEEE Commun. Mag. 58(1), 19–25 (2020)

    Article  Google Scholar 

  4. Wang, X., Wang, C., Li, X., Leung, V.C., Taleb, T.: Federated deep reinforcement learning for internet of things with decentralized cooperative edge caching. IEEE Int. Things J. 7(10), 9441–9455 (2020)

    Article  Google Scholar 

  5. Zhou, P., Chen, X., Liu, Z., Braud, T., Hui, P., Kangasharju, J.: Drle: decentralized reinforcement learning at the edge for traffic light control in the iov. IEEE Trans. Intell. Transp. Syst. 22(4), 2262–2273 (2020)

    Article  Google Scholar 

  6. Xu, H., Chen, M., Meng, Z., Xu, Y., Wang, L., Qiao, C.: Decentralized machine learning through experience-driven method in edge networks. IEEE J. Sel. Areas Commun. 40(2), 515–531 (2021)

    Article  Google Scholar 

  7. Zhou, Q., Qu, Z., Guo, S., Luo, B., Guo, J., Xu, Z., Akerkar, R.: On-device learning systems for edge intelligence: A software and hardware synergy perspective. IEEE Internet Things J. 8(15), 11916–11934 (2021)

    Article  Google Scholar 

  8. Team, T.A.: About Arduino. https://www.arduino.cc/en/about

  9. Foundation, R.P.: Raspberry Pi Foundation About Us. https://www.raspberrypi.org/about/

  10. Pena, D., Forembski, A., Xu, X., Moloney, D.: Benchmarking of cnns for low-cost, low-power robotics applications. In: RSS 2017 Workshop: New Frontier for Deep Learning in Robotics, pp. 1–5 (2017)

  11. Baller, S.P., Jindal, A., Chadha, M., Gerndt, M.: Deepedgebench: Benchmarking deep neural networks on edge devices. In: 2021 IEEE International Conference on Cloud Engineering (IC2E), pp. 20–30 (2021). IEEE

  12. Feng, H., Mu, G., Zhong, S., Zhang, P., Yuan, T.: Benchmark analysis of yolo performance on edge intelligence devices. Cryptography 6(2), 16 (2022)

    Article  Google Scholar 

  13. NVIDIA.: NVIDIA Jetson. https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/

  14. NVIDIA.: Jetson Benchmarks. https://developer.nvidia.com/embedded/jetson-benchmarks (2022)

  15. Süzen, A.A., Duman, B., Şen, B.: Benchmark analysis of jetson tx2, jetson nano and raspberry pi using deep-cnn. In: 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1–5 (2020). IEEE

  16. 3GPP.: System architecture for the 5G System (5GS). Technical Specification (TS) 23.501, 3rd Generation Partnership Project (3GPP) (June 2022). Version 17.5.0. https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3144

  17. 3GPP.: NR; NR and NG-RAN Overall description; Stage-2. Technical Specification (TS) 38.300, 3rd Generation Partnership Project (3GPP) (July 2022). Version 17.1.0. https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3191

  18. Moloudi, S., Mozaffari, M., Veedu, S.N.K., Kittichokechai, K., Wang, Y.-P.E., Bergman, J., Höglund, A.: Coverage evaluation for 5g reduced capability new radio (nr-redcap). IEEE Access 9, 45055–45067 (2021)

    Article  Google Scholar 

  19. Veedu, S.N.K., Mozaffari, M., Hoglund, A., Yavuz, E.A., Tirronen, T., Bergman, J., Wang, Y.-P.E.: Toward smaller and lower-cost 5g devices with longer battery life: An overview of 3gpp release 17 redcap. arXiv preprint arXiv:2203.05634 (2022)

  20. Molchanov, P., Tyree, S., Karras, T., Aila, T., Kautz, J.: Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440 (2016)

  21. Wang, N., Choi, J., Brand, D., Chen, C.-Y., Gopalakrishnan, K.: Training deep neural networks with 8-bit floating point numbers. In: Advances in Neural Information Processing Systems, pp. 7675–7684 (2018)

  22. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)

  23. Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937 (2016)

  24. Ignatov, A., Timofte, R., Chou, W., Wang, K., Wu, M., Hartley, T., Van Gool, L.: Ai benchmark: Running deep neural networks on android smartphones. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 0–0 (2018)

  25. Wang, S., Tuor, T., Salonidis, T., Leung, K.K., Makaya, C., He, T., Chan, K.: When edge meets learning: Adaptive control for resource-constrained distributed machine learning. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications, pp. 63–71 (2018). IEEE

  26. Valerio, L., Passarella, A., Conti, M.: Accuracy vs. traffic trade-off of learning iot data patterns at the edge with hypothesis transfer learning. In: 2016 IEEE 2nd International Forum on Research and Technologies for Society and Industry Leveraging a Better Tomorrow (RTSI), pp. 1–6 (2016). IEEE

  27. Zhang, X., Wang, Y., Shi, W.: pcamp: Performance comparison of machine learning packages on the edges. In: \(\{\)USENIX\(\}\) Workshop on Hot Topics in Edge Computing (HotEdge 18) (2018)

  28. Devarakonda, A., Naumov, M., Garland, M.: Adabatch: adaptive batch sizes for training deep neural networks. arXiv preprint arXiv:1712.02029 (2017)

  29. Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P.T.P.: On large-batch training for deep learning: generalization gap and sharp minima. arXiv preprint arXiv:1609.04836 (2016)

  30. You, Y., Gitman, I., Ginsburg, B.: Large batch training of convolutional networks. arXiv preprint arXiv:1708.03888 (2017)

  31. Bengio, Y.: Practical recommendations for gradient-based training of deep architectures. In: Neural Networks: Tricks of the Trade, pp. 437–478. Springer, New York (2012)

  32. Domhan, T., Springenberg, J.T., Hutter, F.: Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)

  33. Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)

  34. Mohammadi, M., Al-Fuqaha, A., Sorour, S., Guizani, M.: Deep learning for iot big data and streaming analytics: a survey. IEEE Commun. Surv. Tutor. 20(4), 2923–2960 (2018)

    Article  Google Scholar 

  35. Google.: Tensorflow Lite. https://www.tensorflow.org/lite

  36. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  37. 3GPP.: 5G System Enhancements for Edge Computing; Stage 2. Technical Specification (TS) 23.548, 3rd Generation Partnership Project (3GPP) (June 2022). Version 17.3.0. https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=3856

  38. Lai, L., Suda, N.: Enabling deep learning at the iot edge. In: Proceedings of the International Conference on Computer-Aided Design, p. 135 (2018). ACM

  39. Ba, J., Caruana, R.: Do deep nets really need to be deep? In: Advances in Neural Information Processing Systems, pp. 2654–2662 (2014)

  40. Denil, M., Shakibi, B., Dinh, L., Ranzato, M., De Freitas, N.: Predicting parameters in deep learning. In: Advances in Neural Information Processing Systems, pp. 2148–2156 (2013)

  41. Inc., D.: Kubernetes - Docker. https://www.docker.com/products/kubernetes/

  42. LeCun, Y., Cortes, C., Burges, C.J.: The mnist database of handwritten digits, 1998. URL http://yann. lecun. com/exdb/mnist 10, 34 (1998)

  43. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011)

  44. Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Activity recognition using cell phone accelerometers. ACM SIGKDD Explor. Newsl. 12(2), 74–82 (2011)

    Article  Google Scholar 

  45. Kachuee, M., Fazeli, S., Sarrafzadeh, M.: Ecg heartbeat classification: A deep transferable representation. In: 2018 IEEE International Conference on Healthcare Informatics (ICHI), pp. 443–444 (2018). IEEE

  46. He, J., Zhang, Z., Wang, X., Yang, S.: A low power fall sensing technology based on fd-cnn. IEEE Sens. J. 19(13), 5110–5118 (2019)

    Article  Google Scholar 

  47. Konda, V.R., Tsitsiklis, J.N.: Actor-critic algorithms. In: Advances in Neural Information Processing Systems, pp. 1008–1014 (2000)

  48. Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems, pp. 1057–1063 (2000)

  49. Sung, J., Han, S.-J., Kim, J.-W.: Cloning-based virtual machine pre-provisioning for resource-constrained edge cloud server. Clust. Comput. (2023). https://doi.org/10.1007/s10586-023-04045-3

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the IITP grant funded by the Korean government (MSIT) (No. 2021-0-00155).

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, Korea

    Jungwoong Sung & Seung-jae Han

Authors
  1. Jungwoong Sung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seung-jae Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seung-jae Han.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sung, J., Han, Sj. Use of edge resources for DNN model maintenance in 5G IoT networks. Cluster Comput (2024). https://doi.org/10.1007/s10586-023-04236-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10586-023-04236-y

Keywords

Search

Navigation