Skip to main content
Log in

Dynamically hierarchical resource-allocation algorithm in cloud computing environment

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The development of big data challenges the computing power and communication capability of cloud architecture, but traditional resource-allocation algorithms perform poorly due to the large-scale communication among cloud nodes. In this paper, a dynamically hierarchical, resource-allocation algorithm is proposed for multiple cloud nodes collaborating in big data environment. Using fuzzy pattern recognition, the algorithm dynamically divides tasks and nodes into different levels based on computing power and storage factors. Thus a dynamically adjusted mapping is generated between tasks and nodes. When a new task arrives, only the nodes corresponding to the task level join in the bid. The algorithm takes advantages of dynamical hierarchy to reduce the communication traffic during resource allocation. Both theoretical and experimental results illustrate that the proposed algorithm outperforms the MinMin algorithm in terms of communication traffic and makespan.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Michael A et al (2010) A view of cloud computing. Commun ACM 53(4):50–58

    Article  MathSciNet  Google Scholar 

  2. Seref S, Duygu S (2013) Big data: a review. In: 2013 international conference on collaboration technologies and systems, San Diego, CA, USA, pp 42–47

  3. McAffe A, Brynolfsson E (2012) Strategy & competition big data: the management revolution. Harv Bus Rev 90(10):60–66, 68, 128

  4. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616

    Article  Google Scholar 

  5. Amazon EC2 (2015). http://aws.amazon.com/cn/ec2/. Accessed 7 Feb 2015

  6. IBM SmartCloud (2015). http://www.ibm.com/cloud-computing/cn/zh/index.html. Accessed 7 Feb 2015

  7. Seokho S, Gihun J, Sung CJ (2013) An SLA-based cloud computing that facilitates resource allocation in the distributed data centers of a cloud provider. J Supercomput 64:606–637

    Article  Google Scholar 

  8. Nathuji R, Kansal A, Ghaffarkhah A (2010) Q-clouds: managing performance interference effects for QoS-aware clouds. In: 5th ACM European conference on computer systems (EuroSys 2010), Paris, April 13–16, 2010

  9. Marx V (2013) The big challenges of big data. Nature 498(7453):255–260

    Article  Google Scholar 

  10. Sam M (2012) From databases to big data. IEEE Intenet Comput 12:089–7801

    Google Scholar 

  11. JiSu P, Hyongsoon K, Young-Sik J, Eunyoung L (2014) Two-phase grouping-based resource management for big data processing in mobile cloud computing. Int J Commun Syst 27:839–851

    Article  Google Scholar 

  12. Hassan MM, Song B, Hossain MS, Alamri A (2014) QoS-aware resource provisioning for big data processing in cloud computing environment. In: 2014 international conference on computational science and computational intelligence, Las Vegas, NV, USA, March 10–13, 2014

  13. Simon SW, Jelena M (2014) Optimal application allocation on multiple public clouds. Comput Netw 68:138–148

    Article  Google Scholar 

  14. Liang Q, Zhang J, Zhang YH, Liang JM (2014) The placement method of resources and applications based on request prediction in cloud data center. Inf Sci 279:735–745

    Article  MathSciNet  Google Scholar 

  15. Yin C, Huang BQ, Liu F et al (2011) Common key technology system of cloud manufacturing service platform for small and medium enterprises. Comput Integr Manuf Syst 17:495–503

    Google Scholar 

  16. Amit N, Sanjay C, Gaurav S (2012) Policy based resource allocation in IaaS cloud. Future Gener Comput Syst 28:94–103

    Article  Google Scholar 

  17. Christian V, Rodrigo NC, Dileban K, Rajkumar B (2012) Deadline-driven provisioning of resources for scientific applications in hybrid clouds with Aneka. Future Gener Comput Syst 28:58–65

    Article  Google Scholar 

  18. Guiyi W, Athanasios V, Vasilakos YZ, Naixue X (2010) A game-theoretic method of fair resource allocation for cloud computing services. J Supercomput 54:252–269

    Article  Google Scholar 

  19. Anton B, Jemal A, Rajkumar B (2012) Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Gener Comput Syst 28:755–768

    Article  Google Scholar 

  20. Young CL, Albert YZ (2012) Energy efficient utilization of resources in cloud computing systems. J Supercomput 60:268–280

    Article  Google Scholar 

  21. Wu CM, Chang RS, Chan HY (2014) A green energy-efficient scheduling algorithm using the DVFS technique for cloud datacenters. Future Gener Comput Syst 31:141–147

    Article  MATH  Google Scholar 

  22. Xiao Z, Song WJ, Chen Q (2013) Dynamic resource allocation using virtual machines for cloud computing environment. IEEE Trans Parallel Distrib Syst 24(6):1107–1117

    Article  Google Scholar 

  23. Daniel W, Odej K (2011) Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Trans Parallel Distrib Syst 22(6):985–997

    Article  Google Scholar 

  24. Ahmed S, Kenli L, Aijia O, Zhiyong L (2014) Proactive workload management in dynamic virtualized environments. J Comput Syst Sci 80:1504–1517

    Article  MATH  Google Scholar 

  25. Haluk T, Salim H, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  26. Wang ZJ, Liu JJ (2011) Research on multi-agent-based distributed task scheduling mechanism with multi-objective. J Dalian Univ Technol 51(5):755–760

    MATH  Google Scholar 

  27. Wang ZJ, Fang T (2014) Task scheduling model based on multi-agent and multi-objective dynamical scheduling algorithm. J Netw 9(6):1588–1595

    Google Scholar 

  28. Grekovs R (2002) Methods of fuzzy pattern recognition. In: Scientific proceedings of RIGA Technical University

  29. Pedrycz W (1990) Fuzzy sets in pattern recognition: methodology and methods. Pattern Recognit 23(1–2):121–146

    Article  Google Scholar 

  30. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York

    Book  MATH  Google Scholar 

  31. Zhao RH, Lu XL, Wang M (2011) Safety evaluation of urban flood control system based on variable fuzzy pattern recognition. Adv Mater Res 159:264–269

    Article  Google Scholar 

  32. Cui XJ, Cao BY (2010) Using two-level fuzzy pattern recognition in the classification of convex quadrilateral. In: 2nd international conference on quantitative logic and soft computing, vol 82, pp 527–534

  33. Fatma AO, Rasha MZ (2010) Dynamic task scheduling using fuzzy logic in distributed memory systems. In: 2010 7th international conference on informatics and systems, Cairo, Egypt, vol 5

  34. Sepideh A, Ali M, Amir MR, Hamid B, Hengameh DT (2013) A new fuzzy negotiation protocol for grid resource allocation. J Newt Comput Appl. doi:10.1016/j.jnca.2012.12.030

    Google Scholar 

  35. Guo FY, Yu L, Tian S, Yu J (2014) A workflow task scheduling algorithm based on the resources’ fuzzy clustering in cloud computing environment. Int J Commun Syst. doi:10.1002/dac.2743

  36. Akihiro K, Nathan S (2008) Optimized algorithms for multi-agent routing. In: AAMAS ’08 proceedings of the 7th international joint conference on autonomous agents and multiagent systems, New York, pp 1585–1588

  37. Rodrigo NC, Rajiv R, Anton B, César AFDR, Rajkumar B (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithm. Softw Pract Exp 41:23–50

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by the National Natural Science Foundation of China (Grant No. 61173160), Intel Industrial Liaison and Comprehensive Reform Program of Intel Semiconductor (US) LLC in 2014 of Ministry of Education of the People’s Republic of China. We would like to thank the anonymous reviewers for their attentive reading and for their constructive comments that have helped to further strengthen this paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xianxian Su.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Su, X. Dynamically hierarchical resource-allocation algorithm in cloud computing environment. J Supercomput 71, 2748–2766 (2015). https://doi.org/10.1007/s11227-015-1416-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-015-1416-x

Keywords

Navigation