FBQ-LA: Fuzzy based Q-Learning approach for elastic workloads in cloud environment

Abstract

Cloud computing relates to the storage and accessing of data as a service from Internet for any organizational infrastructure at-any-time. The delivery of some of the services related to computing such as servers, networking, storage, software, etc., is made possible with the use of cloud computing. Companies offer these services in terms of cloud service providers (CSPs) who charge for the services they provide to the users. When a request is made to use the services, the service provider allocates a feasible number of virtual machines (VMs). Determining optimum amount of resources required at runtime to satisfy the user’s request is not a trivial task. Therefore, in cloud ecosystem the cardinal issue is the management of resource allocation to an application in order to abide by the service level agreements (SLAs). The fundamental objective of cloud service management is to design a self-adjustable auto-scalar to respond to elastic workload and optimizing the allocation of resources with reduced cost. The notable issue is how and at what time resources are to be allocated/de-allocated in order to follow agreed SLAs. In this paper, we propose a resource provisioning framework based on the integrated concepts of autonomic computing with Fuzzy Q Learning and Chebyshev’s Inequality principle. The concept of auto-scaling mechanism is commonly implemented in four phases of proposed autonomic MAPE loop framework: Monitoring, Analysis, Planning and Execution. The proposed framework follows the control MAPE loop structure with the inclusion of Chebyshev’s inequality for prediction in the analysis phase and fuzzy Q-learning in planning phase, where human intervention in the form of fuzzy rules ensures efficacious provisioning of VMs. A comparative analysis has been performed with a different combination of (i) LRM in the analysis phase with FBQ-LA in planning phase, ii) Chebyshev’s Inequality in the analysis phase with FBQ-LA in planning phase, and iii) Chebyshev’s Inequality in the analysis phase with Q-Learning in planning phase. Experimental results prove that the proposed autonomic model based on Chebyshev’s inequality and FBQ-LA outperforms the existing model in terms of improved VM provisioning, minimized costs as well as reduction in response time.

Keywords

Resource provisioning autonomic computing MAPE loop fuzzy Q-learning Q-learning

Get full access to this article

View all access options for this article.

References

Buyya ,

Vecchiola and

S.T.

Selvi , Mastering cloud computing: Foundations and applications programming, Newnes (2013).

Singh and

Chana , Resource provisioning and scheduling in clouds: QoS perspective, The Journal of Supercomputing 72(3) (2016), 926–960.

Singh and

Chana , QRSF: QoS-aware resource scheduling framework in cloud computing, The Journal of Supercomputing 71(1) 2015, 241–292.

Singh and

Chana , A survey on resource scheduling in cloud computing: Issues and challenges, Journal of grid computing 14(2) (2016), 217–264.

R.R.

Bane ,

Annappa and

K.C.

Shet , Survey of dynamic resource management approaches in virtualized data centers, In Proceeding of IEEE International Conference on Computational Intelligence and Computing Research, 2013, pp. 1–7.

Buyya ,

S.K.

Garg and

R.N.

Calheiros , SLA-oriented resource provisioning for cloud computing: Challenges, architecture, and solutions, In Proceeding of IEEE International Conference on Cloud and Service Computing, 2011, pp. 1–10.

Chaisiri ,

B.S.

Lee and

Niyato , Optimization of resource provisioning cost in cloud computing, IEEE Transactions on Services Computing 5(2) (2012), 164–177.

Maurer ,

Breskovic ,

V.C.

Emeakaroha and

Brandic , Revealing the MAPE loop for the autonomic management of cloud infrastructures, In Proceeding of IEEE Symposium on Computers and Communications, 2011, pp. 147–152.

M.C.

Huebscher and

J.A.

McCann , A survey of autonomic computing—degrees, models, and applications, ACM Computing Surveys (CSUR) 40(3) (2008), 7.

10.

Buyya ,

R.N.

Calheiros and

Li . Autonomic cloud computing: Open challenges and architectural elements, In Proceeding of IEEE International Conference on Emerging Applications of Information Technology, 2012, pp. 3–10.

11.

Jacob ,

Lanyon-Hogg ,

D.K.

Nadgir and

A.F.

Yassin , A practical guide to the IBM autonomic computing toolkit, IBM Redbooks 4 (2004), 10.

12.

Bahrpeyma ,

Haghighi and

Zakerolhosseini , A bipolar resource management framework for resource provisioning in Cloud's virtualized environment, Applied Soft Computing 46 (2016), 487–500.

13.

Moreno and

Xu . Neural network-based overallocation for improved energy-efficiency in real-time cloud environments, In proceeding of IEEE 15th International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing, 2012, 119–126.

14.

S.K

Garg ,

A.N

Toosi ,

S.K.

Gopalaiyengar and

Buyya , SLA-based virtual machine management for heterogeneous workloads in a cloud datacenter, Journal of Network and Computer Applications 45 (2014), 108–120.

15.

A.F.

Antonescu ,

Robinson and

Braun , Dynamic SLA management with forecasting using multi-objective optimization, In Proceeding of IFIP/IEEE International Symposium on Integrated Network Management, 2013, pp. 457–463.

16.

M.M.

Al-Sayed ,

Khattab and

F.A.

Omara , Prediction mechanisms for monitoring state of cloud resources using Markov chain model, Journal of Parallel and Distributed Computing 96 (2016), 163–171.

17.

P.Y.

Glorennec , Reinforcement learning: An overview, In Proc of ESIT, 2000, 17–35.

18.

R.S.

Sutton and

A.G.

Barto , Reinforcement learning: An introduction, MIT press, 1998.

19.

C.J.

Watkins and

Dayan , Q-learning. Machine learning 8(3–4) (1994), 279–292.

20.

Xu ,

Zhao ,

Fortes ,

Carpenter and

Yousif , Autonomic resource management in virtualized data centers using fuzzy logic-based approaches, Cluster Computing 11(3) (2008), 213–227.

21.

Maurer ,

Brandic and

Sakellariou , Simulating autonomic SLA enactment in clouds using case based reasoning, In European Conference on a Service-Based Internet. Springer, Berlin, Heidelberg, 2010, pp. 25–36.

22.

Mao and

Humphrey , Auto-scaling to minimize cost and meet application deadlines in cloud workflows, In Proceeding of IEEE International Conference on High Performance Computing, Networking, Storage and Analysis, 2011, pp. 1–12.

23.

Ritter ,

Mitschang and

Mega , Dynamic provisioning of system topologies in the cloud, Enterprise Interoperability V. Springer, London, 2012, pp. 391–401.

24.

Frey ,

Lüthje ,

Reich and

Clarke , Cloud QoS scaling by fuzzy logic, In Proceeding of IEEE International Conference on Cloud Engineering, 2014, pp. 343–348.

25.

Jamshidi ,

Ahmad and

Cl.

Pahl , Autonomic resource provisioning for cloud-based software, In Proceedings of ACM 9th international symposium on software engineering for adaptive and self-managing systems, 2014, pp. 95–104.

26.

Jamshidi ,

Sharifloo ,

Pahl ,

Metzger and

Estrada , Self-learning cloud controllers: Fuzzy q-learning for knowledge evolution, In Proceeding of IEEE International Conference on Cloud and Autonomic Computing, 2015, pp. 208–211.

27.

Amiri ,

M.R.

Feizi-Derakhshi and

Mohammad-Khanli , IDS fitted Q improvement using fuzzy approach for resource provisioning in cloud, Journal of Intelligent & Fuzzy Systems 32(1) (2017), 229–240.

28.

Singh and

Chana , EARTH: Energy-aware autonomic resource scheduling in cloud computing, Journal of Intelligent & Fuzzy Systems 30(3) (2016), 1581–1600.

29.

Ghobaei-Arani ,

Jabbehdari and

M.A

Pourmina , An autonomic resource provisioning approach for service-based cloud applications: A hybrid approach, Future Generation Computer Systems 78 (2018), 191–210.

30.

Arabnejad ,

Pahl ,

Jamshidi and

Estrada , A comparison of reinforcement learning techniques for fuzzy cloud auto-scaling, In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2017, pp. 64–73.

31.

M.S.

Aslanpour ,

Ghobaei-Arani and

A.N.

Toosi . Auto-scaling web applications in clouds: A cost-aware approach. Journal of Network and Computer Applications 95 (2017), 26–41.

32.

P.Y.

Glorennec , Fuzzy Q-learning and dynamical fuzzy Q-learning, In fuzzy system, Proceedings of the Third IEEE Conference World Congress on Computational Intelligence, 1994, pp. 474–479.

33.

H.R.

Berenji , Fuzzy Q-learning for generalization of reinforcement learning, In Proceedings of the Fifth IEEE International Conference on Fuzzy Systems, Vol. 3, 1996, pp. 2208–2214.

34.

C.H.

Oh ,

Nakashima and

Ishibuchi , Initialization of Q-values by fuzzy rules for accelerating Q-learning, In Neural Networks Proceedings, IEEE World Congress on Computational Intelligence. IEEE International Joint Conference, Vol. 3, 1998, pp. 2051–2056.

35.

Jouffe , Fuzzy inference system learning by reinforcement methods, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 28(3) (1998), 338–355.

36.

Bonarini ,

Lazaric ,

Montrone and

Restelli . Reinforcement distribution in fuzzy Q-learning, Fuzzy sets and systems 160(10) (2009), 1420–1443.

37.

Boumehraz ,

Benmahammed ,

M. L.

Hadjili and

Werzz , Fuzzy Inference Systems Optimization by Reinforcement Learning, Courrier du Savoir 1(1) (2001), 9–15.

38.

M.J.

Er and

Deng , Online tuning of fuzzy inference systems using dynamic fuzzy Q-learning, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34(3) (2004), 1478–1489.

39.

F.J.

Cabrerizo ,

Chiclana ,

Al-Hmouz ,

Morfeq ,

A.S.

Balamash and

Herrera-Viedma , Fuzzy decision making and consensus: challenges, Journal of Intelligent & Fuzzy Systems 29(3) (2015), 1109–1118.

40.

A.A.

Jaoude , The paradigm of complex probability and Chebyshev’s inequality, Systems Science & Control Engineering 4(1) (2016), 99–137.

41.

Y.L.

Tong , Relationship between stochastic inequalities and some classical mathematical inequalities, Journal of Inequalities and Applications 1(1) (1997), 85–98.

42.

R.N.

Calheiros ,

Ranjan ,

Beloglazov ,

AF De Rose and

Buyya , Cloud Sim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms, Software: Practice and experience 41(1) (2011), 23–50.

43.

Nasa, one day http data log http://ita.ee.lbl.gov/html/contrib/NASA-HTTP.html (Accessed on 17.11.17).