PPO-RR: virtual machine scheduling algorithm and deep reinforcement learning method for green computing

Abstract

The ever-growing number of energy requirements needed in cloud data centers has led to the rising use of intelligent, sustainable and environmental scheduling algorithms. The study proposes a new hybrid algorithm that is a fusion of Proximal Policy Optimization (PPO) and Round Robin (RR), and is called PPO-RR, to enhance the efficiency of virtual machine (VM) scheduling and reduce the amount of energy consumption in a virtualized cloud system. The suggested approach follows the strategy of reinforcement learning: using the feedback about the system state, it can actively optimize the scheduling decisions to improve resource utilization and eliminate unnecessary waste of power in idle conditions.Experiments were carried using an artificially created cloud environment dataset. The PPO-RR model was seen to have shown significant performance gains over the conventional RR and heuristic scheduling techniques. To be more precise, PPO-RR achieved the accuracy of 95.3%, precision of 94.6%, recall of 95.4, and F1-score of 94.9. Such findings demonstrate the superiority of the model in its capacity to assign the VMs to the hosts appropriately during different workloads.With regard to energy measures, PPO-RR saved 1520 kWh to 1235 kWh translating to about 56.3 percent of energy saved as compared to conventions practices. The PPO-RR confusion matrix shows that the True Positives (TP) are 420, True Negatives (TN) are 360, False Positives (FP) are 15 and False Negatives (FN) are 25 which means that the accuracy of the classification of the decisions in scheduling is high.The model offers dynamic scheduling with performance and energy benefits compared to the traditional schedulers because the PPO-RR model adapts to the work load profile and system conditions in a real-time fashion. Despite the fact that this study shows PPO-RR to positively perform in various metrics, no formal statistical test of significance (e.g., t-test) was conducted and will be done in subsequent research.In sum, the PPO-RR approach provides a flexible and extensible mechanism to implement sustainable cloud computing, since it contributes to satisfying the objectives of green computing with the help of well-organized schedules.

Keywords

proximal policy optimization virtual machine scheduling green computing round robin energy efficiency deep reinforcement learning

Get full access to this article

View all access options for this article.

References

Buyya

Ilager

Arroba

. Energy-efficiency and sustainability in new generation cloud computing: a vision and directions for integrated management of data centre resources and workloads. Softw - Pract Exp Aug. 2023; 54: 24–38.

Yang

CenkGursoy

. Deep reinforcement learning and optimization based green Mobile edge computing. In: 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC), Jan. 2021, pp.1–2. 10.1109/ccnc49032.2021.9369566

, et al. Virtual machine migration techniques for optimizing energy consumption in cloud data centers. IEEE Access 2023; 11: 86739–86753.

Yang

, et al. Deep reinforcement learning based green resource allocation mechanism in edge computing driven power internet of things. In: 2022 International Wireless Communications and Mobile Computing (IWCMC), Jun. 2020, pp.388–393. https://doi.org/10.1109/iwcmc48107.2020.9148169

Abdelaziz

Santos

Dias

, et al. A hybrid model of self-organizing map and deep learning with genetic algorithm for managing energy consumption in public buildings. J Cleaner Prod Jan. 2024; 434: 140040–140040.

Wang

, et al. Research on virtual machine consolidation strategy based on combined prediction and energy-aware in cloud computing platform. J Cloud Comput Sep. 2022; 11: 50. 10.1186/s13677-022-00309-2

Yuan

Zhou

. Energy-Efficient and QoS-optimized adaptive task scheduling and management in clouds. IEEE Trans Autom Sci Eng Apr. 2022; 19: 1233–1244.

Luo

, et al. A resource scheduling algorithm of cloud computing based on energy efficient optimization methods. In: 2012 International Green Computing Conference (IGCC), Jun. 2012. https://doi.org/10.1109/igcc.2012.6322251

Dinesh Reddy

Gangadharan

Rao

GSVRK

. Energy-aware virtual machine allocation and selection in cloud data centers. Soft COmput Nov. 2017; 23: 1917–1932.

10.

Aldossary

Alharbi

. Towards a green approach for minimizing carbon emissions in fog-cloud architecture. IEEE Access 2021; 9: 131720–131732.

11.

Wang

Zhang

, et al. RLSchert: an HPC job scheduler using deep reinforcement learning and remaining time prediction. Appl Sci Oct. 2021; 11: 9448.

12.

Shukla

Chaturvedi

. Design of energy optimization algorithm for virtual machine scheduling in cloud computing. Lect Notes Electr Eng 2024; 1189: 291–304.

13.

Robaglia

B-M

Coupechoux

Tsilimantos

. Deep reinforcement learning for uplink scheduling in NOMA-URLLC networks. arXiv (Cornell University), Jan. 2023, https://doi.org/10.48550/arxiv.2308.14523

14.

, et al. TDP-43 represses cryptic exon inclusion in the FTD–ALS gene UNC13A. Nature Mar. 2022; 603: 124–130. Available: https://www.nature.com/articles/s41586-022-04424-7

15.

Gao

, et al. Real-Time virtual machine scheduling in industry IoT network: a reinforcement learning method. IEEE Trans Ind Inf Oct. 2022; 19: 2129–2139.

16.

Siddesha

Jayaramaiah

Singh

. A novel deep reinforcement learning scheme for task scheduling in cloud computing. Cluster Comput Jun. 2022; 25: 4171–4188.

17.

Hou

AgosJawaddi

Ismail

. Energy efficient task scheduling based on deep reinforcement learning in cloud environment: a specialized review. Future Gener Comput Syst Feb. 2024; 151: 214–231. Available: https://www.sciencedirect.com/science/article/abs/pii/S0167739X23003771. [Accessed: Dec. 18, 2023].

18.

Rjoub

Bentahar

Wahab

, et al. Deep Smart Scheduling: A Deep Learning Approach for Automated Big Data Scheduling Over the Cloud. In: Conference on the Future of the Internet, Aug. 2019, https://doi.org/10.1109/ficloud.2019.00034

19.

Chandrasiri

Meedeniya

. Energy-Efficient dynamic workflow scheduling in cloud environments using deep learning. Sensors Feb. 2025; 25: 1428.

20.

Naik

Sooda

. Efficient job scheduling in cloud environments using reinforcement learning actor-critic models. Eng Technol Appl Sci Res Oct. 2024; 14: 16559–16564.