| 284 | 0 | 95 |
| 下载次数 | 被引频次 | 阅读次数 |
在边缘计算中，微服务架构可以提升数据处理效率与应用响应速度，适用于快速响应和频繁交互的各类应用场景。然而现有研究忽视了微服务之间不同交互频率对通信开销的影响。针对该问题，提出了一种基于强化学习的多目标微服务最优部署方法，以提升边缘环境中的微服务性能。先建立一种考虑降低微服务交互通信开销和平衡边缘节点资源的双重优化目标模型，在此基础上，设计了基于改进奖励机制的深度Q学习算法。为了适应微服务部署过程中共享资源的特性，引入共享奖励机制，使算法拥有更好的收敛性。实验结果表明，与现有DIM方法和Kubernetes默认部署方法相比，所提出的算法更能均衡微服务交互感知和节点资源利用率，拥有更短的响应时间。
Abstract：In edge computing, microservice architecture could improve data processing efficiency and application response speed, which was suitable for various application scenarios with fast response and frequent interactions. However, existing studies neglected the impact of different interaction frequencies between microservices on the communication overhead. To address this problem, a multi-objective microservice optimal deployment method based on reinforcement learning to improve the performance of microservices in edge environments was proposed. A dual optimization objective model that considers reducing the communication overhead of microservice interactions and balancing the resources of edge nodes was established. Then a deep Q-learning algorithm based on an improved reward mechanism was designed. In order to adapt to the characteristics of shared resources in the process of microservice deployment, a shared reward mechanism was introduced so that the algorithm had better convergence. The experimental results showed that the proposed algorithm could balance the microservice interaction perception and node resource utilization better, and had shorter response time compared with the existing DIM method and Kubernetes default deployment method.
[1] SHI W,ZHANG X,WANG Y F,et al.Edge computing:state-of-the-art and future directions[J].Journal of computer research and development,2019,56(1):69-89.
[2] SHI W S,CAO J,ZHANG Q,et al.Edge computing:vision and challenges[J].IEEE Internet of Things journal,2016,3(5):637-646.
[3] 陈玉平，刘波，林伟伟，等.云边协同综述[J].计算机科学，2021,48(3):259-268.CHEN Y P,LIU B,LIN W W,et al.Survey of cloud-edge collaboration[J].Computer science,2021,48(3):259-268.
[4] DRAGONI N,GIALLORENZO S,LAFUENTE A L,et al.Microservices:yesterday,today,and tomorrow[M]//Present and Ulterior Software Engineering.Cham:Springer International Publishing,2017:195-216.
[5] TAN B X,MA H,MEI Y.A NSGA-II-based approach for multi-objective micro-service allocation in container-based clouds[C]//2020 20th IEEE/ACM International Symposium on Cluster,Cloud and Internet Computing.Piscataway:IEEE Press,2020:282-289.
[6] DENG S G,XIANG Z Z,TAHERI J,et al.Optimal application deployment in resource constrained distributed edges[J].IEEE transactions on mobile computing,2021,20(5):1907-1923.
[7] GUERRERO C,LERA I,JUIZ C.Genetic algorithm for multi-objective optimization of container allocation in cloud architecture[J].Journal of grid computing,2018,16(1):113-135.
[8] LIN M,XI J Q,BAI W H,et al.Ant colony algorithm for multi-objective optimization of container-based microservice scheduling in cloud[J].IEEE access,2019,7:83088-83100.
[9] BROGI A,FORTI S,GUERRERO C,et al.How to place your apps in the fog:state of the art and open challenges[J].Software:practice and experience,2020,50(5):719-740.
[10] SMET P,DHOEDT B,SIMOENS P.Docker layer placement for on-demand provisioning of services on edge clouds[J].IEEE transactions on network and service management,2018,15(3):1161-1174.
[11] SAMANTA A,TANG J H.Dyme:dynamic microservice scheduling in edge computing enabled IoT[J].IEEE Internet of Things journal,2020,7(7):6164-6174.
[12] JOSEPH C T,CHANDRASEKARAN K.IntMA:Dynamic Interaction-aware resource allocation for containerized microservices in cloud environments[J].Journal of systems architecture,2020,111:101785.
[13] WANG S G,GUO Y,ZHANG N,et al.Delay-aware microservice coordination in mobile edge computing:a reinforcement learning approach[J].IEEE transactions on mobile computing,2021,20(3):939-951.
[14] CHEN L L,XU Y C,LU Z H,et al.IoT microservice deployment in edge-cloud hybrid environment using reinforcement learning[J].IEEE Internet of Things journal,2021,8(16):12610-12622.
[15] FU K H,ZHANG W,CHEN Q,et al.Adaptive resource efficient microservice deployment in cloud-edge continuum[J].IEEE transactions on parallel and distributed systems,2022,33(8):1825-1840.
[16] CHEIKHROUHOU O,MAHMUD R,ZOUARI R,et al.One-dimensional CNN approach for ECG arrhythmia analysis in fog-cloud environments[J].IEEE access,2021,9:103513-103523.
[17] PALLEWATTA S,KOSTAKOS V,BUYYA R.Microservices-based IoT application placement within heterogeneous and resource constrained fog computing environments[C]//Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing.New York:ACM Press,2019:71-81.
[18] FAN J,WANG Z,XIE Y,et al.A theoretical analysis of deep Q-learning[C]//Learning for Dynamics and Control.New York:PMLR,2020:486-489.
[19] MAEI H R.Gradient temporal-difference learning algorithms[D].Edmonton:University of Alberta,2011.
[20] PETERS J.Policy gradient methods[J].Scholarpedia,2010,5(11):3698.
[21] SCHAUL T,QUAN J,ANTONOGLOU I,et al.Prioritized experience replay[EB/OL].(2015-11-08)[2024-06-20].https://arxiv.org/abs/1511.05952.
[22] HENDERSON P,ISLAM R,BACHMAN P,et al.Deep reinforcement learning that matters[C]// Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2018:3207-3214.
基本信息:
DOI：10.13705/j.issn.1671-6841.2024139
中图分类号:TP18;TP393.09
引用信息:
[1]张璊瑶,张盈希,郑文祺,等.基于强化学习的多目标微服务部署方法[J].郑州大学学报(理学版),2026,58(02):33-39+47.DOI:10.13705/j.issn.1671-6841.2024139.
基金信息:
群集适应性系统建模与宏观行为分析方法研究基金项目(62272126)
2024-08-04
2024
2024-08-08
2024
1
2024-10-25
2024-10-25
2024-10-25