Deep Deterministic Policy Gradient Algorithm Dynamic Task Scheduling in Edge-Cloud Environment Using Reinforcement Learning
Main Article Content
Abstract
In contemporary era, cloud computing is helping high performance computing applications by providing scalable and affordable computing resources. However, the latency of cloud resources is relatively less when compared with edge computing. In this context, usage of edge cloud for task scheduling became indispensable to reap benefits of latency performance with edge cloud. However, it is not possible to assign every task to edge cloud and resource-intensive tasks should be scheduled to cloud. With edge-cloud environment, it becomes very complex and scheduling is a NP-hard problem. Many existing methods based on reinforcement learning are found to have shortcoming in dealing with large action-space in presence of state-space. In this paper we proposed an algorithm known as Deep Deterministic Policy Gradient Algorithm for Dynamic Task Scheduling (DDPGA-TS). Our algorithm has a novel pruning strategy that continuously monitors action space and reduces it to improve overall performance in task scheduling. We used three scales of environments with our experiments. Number of performance indicators are used to evaluate performance of the proposed algorithm. Experimental results revealed that the proposed algorithm outperforms existing methods such as DDPG-NN and DDPG-CNN.
Downloads
Article Details
Section
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
References
Jaber Almutairi and Mohammad Aldossary; (2021). A novel approach for IoT tasks offloading in edge-cloud environments . Journal of Cloud Computing, p1-19.
Shaoshuai Ding; Lei Yang; Jiannong Cao; Wei Cai; Mingkui Tan and Zhenyu Wang; (2021). Partitioning Stateful Data Stream Applications in Dynamic Edge Cloud Environments . IEEE Transactions on Services Computing, p1-14.
Bulej, Lubomìr; BureÅ¡, Tomáš; Filandr, Adam; HnÄ›tynka, Petr; HnÄ›tynková, Iveta; PacovskÃ, Jan; Sandor, Gabor and Gerostathopoulos, Ilias (2021). Managing latency in edgeâ“cloud environment. Journal of Systems and Software, 172, p1-15.
Xu Zhao; Guangqiu Huang; Ling Gao; Maozhen Li and Quanli Gao; (2021). Low load DIDS task scheduling based on Q-learning in edge computing environment . Journal of Network and Computer Applications, p1-12.
Yu Zhang, Bing Tang, Jincheng Luo and Jiaming Zhang. (2022). Deadline-Aware Dynamic Task Scheduling in Edge–Cloud Collaborative Computing. MDPI. 11, pp.1-24.
Abdullah Lakhan, Mazin Abed Mohammed, Karrar Hameed Abdulkareem and Mu. (2022). Delay Optimal Schemes for Internet of Things Applications in Heterogeneous Edge Cloud Computing Networks. MDPI. 22, pp.1-30.
Qihe Huang;Xiaolong Xu and Jinhui Chen; (2021). Learning-aided fine grained offloading for real-time applications in edge-cloud computing . Wireless Networks, p1-16.
Asghari, Ali; Sohrabi, Mohammad Karim and Yaghmaee, Farzin (2020). Task scheduling, resource provisioning, and load balancing on scientific workflows using parallel SARSA reinforcement learning agents and genetic algorithm. The Journal of Supercomputing.
Wang, Hongman; Li, Yingxue; Zhou, Ao; Guo, Yan and Wang, Shangguang (2020). Service migration in mobile edge computing: A deep reinforcement learning approach. International Journal of Communication Systems, e4413–.
Bassem Sellami;Akram Hakiri;Sadok Ben Yahia and Pascal Berthou; (2020). Deep Reinforcement Learning for Energy-Efficient Task Scheduling in SDN-based IoT Network . 2020 IEEE 19th International Symposium on Network Computing and Applications (NCA).
Guanjin Qu;Huaming Wu;Ruidong Li and Pengfei Jiao; (2021). DMRO: A Deep Meta Reinforcement Learning-Based Task Offloading Framework for Edge-Cloud Computing . IEEE Transactions on Network and Service Management.
Yixue Hao; Min Chen; Hamid Gharavi; Yin Zhang and Kai Hwang; (2021). Deep Reinforcement Learning for Edge Service Placement in Softwarized Industrial Cyber-Physical System . IEEE Transactions on Industrial Informatics.
Tuli, Shreshth; Ilager, Shashikant; Ramamohanarao, Kotagiri and Buyya, Rajkumar (2020). Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments using A3C learning and Residual Recurrent Neural Networks. IEEE Transactions on Mobile Computing, 1–1.
Qi Zhang; Lin Gui; Shichao Zhu and Xiupu Lang; (2021). Task Offloading and Resource Scheduling in Hybrid Edge-Cloud Networks . IEEE Access.
Yicen Liu;Yu Lu;Xi Li;Wenxin Qiao;Zhiwei Li and Donghao Zhao; (2021). SFC Embedding Meets Machine Learning: Deep Reinforcement Learning Approaches . IEEE Communications Letters.
Miaojiang Chen; Tian Wang; Shaobo Zhang and Anfeng Liu; (2021). Deep reinforcement learning for computation offloading in mobile edge computing environment . Computer Communications.
M. S. Mekala; Alireza Jolfaei; Gautam Srivastava; Xi Zheng; Amjad Anvari-Moghaddam and P. Viswanathan; (2022). Resource Offload Consolidation Based on Deep-Reinforcement Learning Approach in Cyber-Physical Systems . IEEE Transactions on Emerging Topics in Computational Intelligence.
Bing Lin;Kai Lin;Changhang Lin;Yu Lu;Ziqing Huang and Xinwei Chen; (2021). Computation offloading strategy based on deep reinforcement learning for connected and autonomous vehicle in vehicular edge computing . Journal of Cloud Computing,.
Qiang Liu;Tao Han and Ephraim Moges; (2020). EdgeSlice: Slicing Wireless Edge Computing Network with Decentralized Deep Reinforcement Learning . 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS),.
Yiwei Zhang; Min Zhang; Caixia Fan; Fuqiang Li and Baofang Li; (2021). Computing resource allocation scheme of IOV using deep reinforcement learning in edge computing environment . EURASIP Journal on Advances in Signal Processing, .
Xiaokang Zhou, Wei Liang, Ke Yan, Weimin Li, Kevin I-Kai Wang, Jianhua Ma and Qun Jin. (2023). Edge-Enabled Two-Stage Scheduling Based on Deep Reinforcement Learning for Internet of Everything. IEEE. 10(4), pp.3295-3304. https://creativecommons.org/licenses/by/4.0/
Bassem Sellami, Akram Hakiri, Sadok Ben Yahia and Pascal Berthou. (2022). Energy-Aware Task Scheduling and Offloading using Deep Reinforcement Learning in SDN-enabled IoT Network. Elsevier, pp.1-20.
Bassem Sellami, Akram Hakiri and Sadok Ben Yahia. (2022). Deep Reinforcement Learning for Energy-Aware Task Offloading in Join SDN-Blockchain 5G massive IoT Edge Network. Elsevier, pp.1-19.
Jiwei Huang, Jiangyuan Wan, Bofeng Lv, Qiang Ye and Ying Chen. (2023). Joint Computation Offloading and Resource Allocation for Edge-Cloud Collaboration in Internet of Vehicles via Deep Reinforcement Learning. IEEE. 17(2), pp.2500-2511. http://doi:10.1109/JSYST.2023.3249217
Lixiang Zhang, Chen Yang, Yan Yan and Yaoguang Hu. (2022). Distributed Real-Time Scheduling in Cloud Manufacturing by Deep Reinforcement Learning. IEEE. 18(12), pp.8999-9007. http://doi:10.1109/TII.2022.3178410