A Taxonomy and Survey of Energy-Efficient Data Centers and Cloud Computing Systems

doi:10.1016/B978-0-12-385512-1.00003-7

Advances in Computers

Volume 82, 2011, Pages 47-111

https://doi.org/10.1016/B978-0-12-385512-1.00003-7 Get rights and content

Abstract

Traditionally, the development of computing systems has been focused on performance improvements driven by the demand of applications from consumer, scientific, and business domains. However, the ever-increasing energy consumption of computing systems has started to limit further performance growth due to overwhelming electricity bills and carbon dioxide footprints. Therefore, the goal of the computer system design has been shifted to power and energy efficiency. To identify open challenges in the area and facilitate future advancements, it is essential to synthesize and classify the research on power- and energy-efficient design conducted to date. In this study, we discuss causes and problems of high power/energy consumption, and present a taxonomy of energy-efficient design of computing systems covering the hardware, operating system, virtualization, and data center levels. We survey various key works in the area and map them onto our taxonomy to guide future design and development efforts. This chapter concludes with a discussion on advancements identified in energy-efficient computing and our vision for future research directions.

Introduction

The primary focus of designers of computing systems and the industry has been on the improvement of the system performance. According to this objective, the performance has been steadily growing driven by more efficient system design and increasing density of the components described by Moore's law [1]. Although the performance per watt ratio has been constantly rising, the total power drawn by computing systems is hardly decreasing. Oppositely, it has been increasing every year that can be illustrated by the estimated average power use across three classes of servers presented in Table I[2]. If this trend continues, the cost of the energy consumed by a server during its lifetime will exceed the hardware cost [3]. The problem is even worse for large-scale compute infrastructures, such as clusters and data centers. It was estimated that in 2006 IT infrastructures in the United States consumed about 61 billion kWh for the total electricity cost about 4.5 billion dollars [4]. The estimated energy consumption is more than double from what was consumed by IT in 2000. Moreover, under current efficiency trends, the energy consumption tends to double again by 2011, resulting in 7.4 billion dollars annually.

Energy consumption is not only determined by hardware efficiency, but it is also dependent on the resource management system deployed on the infrastructure and the efficiency of applications running in the system. This interconnection of the energy consumption and different levels of computing systems can be seen in Fig. 1. Energy efficiency impacts end-users in terms of resource usage costs, which are typically determined by the total cost of ownership (TCO) incurred by a resource provider. Higher power consumption results not only in boosted electricity bills but also in additional requirements to a cooling system and power delivery infrastructure, that is, uninterruptible power supplies (UPS), power distribution units (PDU), and so on. With the growth of computer components density, the cooling problem becomes crucial, as more heat has to be dissipated for a square meter. The problem is especially important for 1U and blade servers. These types of servers are the most difficult to cool because of high density of the components, and thus lack of space for the air flow. Blade servers bring the advantage of more computational power in less rack space. For example, 60 blade servers can be installed into a standard 42U rack [5]. However, such system requires more than 4000 W to supply the resources and cooling system compared to the same rack filled by 1U servers consuming 2500 W. Moreover, the peak power consumption tends to limit further performance improvements due to constraints of power distribution facilities. For example, to power a server rack in a typical data center, it is necessary to provide about 60 A [6]. Even if the cooling problem can be addressed for future systems, it is likely that delivering current in such data centers will reach the power delivery limits.

Apart from the overwhelming operating costs and the total cost of acquisition (TCA), another rising concern is the environmental impact in terms of carbon dioxide (CO₂) emissions caused by high energy consumption. Therefore, the reduction of power and energy consumption has become a first-order objective in the design of modern computing systems. The roots of energy-efficient computing, or Green IT, practices can be traced back to 1992, when the U.S. Environmental Protection Agency launched Energy Star, a voluntary labeling program which is designed to identify and promote energy-efficient products in order to reduce the greenhouse gas emissions. Computers and monitors were the first labeled products. This has led to the widespread adoption of the sleep mode in electronic devices. At that time, the term “green computing” was introduced to refer to energy-efficient personal computers [7]. At the same time, the Swedish confederation of professional employees has developed the TCO certification program—a series of end-user and environmental requirements for IT equipment including video adapters, monitors, keyboards, computers, peripherals, IT systems, and even mobile phones. Later, this program has been extended to include requirements on ergonomics, magnetic and electrical field emission levels, energy consumption, noise level, and use of hazardous compounds in hardware. The Energy Star program was revised in October 2006 to include stricter efficiency requirements for computer equipment, along with a tiered ranking system for approved products.

There are a number of industry initiatives aiming at the development of standardized methods and techniques for the reduction of the energy consumption in computer environments. They include Climate Savers Computing Initiative (CSCI), Green Computing Impact Organization, Inc. (GCIO), Green Electronics Council, The Green Grid, International Professional Practice Partnership (IP3), with membership of companies such as AMD, Dell, HP, IBM, Intel, Microsoft, Sun Microsystems, and VMware.

Energy-efficient resource management has been first introduced in the context of battery-powered mobile devices, where energy consumption has to be reduced in order to improve the battery lifetime. Although techniques developed for mobile devices can be applied or adapted for servers and data centers, this kind of systems requires specific methods. In this chapter, we discuss ways to reduce power and energy consumption in computing systems, as well as recent research works that deal with power and energy efficiency at the hardware and firmware, operating system (OS), virtualization, and data center levels. The main objective of this work is to give an overview of the recent research advancements in energy-efficient computing, identify common characteristics, and classify the approaches. On the other hand, the aim is to show the level of development in the area and discuss open research challenges and direction for future work. The reminder of this chapter is organized as follows: in the next section, power and energy models are introduced; in Section 3, we discuss problems caused by high power and energy consumption; in 4 Taxonomy of Power/Energy Management in Computing Systems, 5 Hardware and Firmware Level, 5.1 Dynamic Component Deactivation, 5.2 Dynamic Performance Scaling, 5.2.1 Dynamic Voltage and Frequency Scaling, 5.3 Advanced Configuration and Power Interface, 6 Operating System Level, 6.1 The On‐Demand Governor (Linux Kernel), 6.2 ECOsystem, 6.3 Nemesis OS, 6.4 The Illinois GRACE Project, 6.5 Linux/RK, 6.6 Coda and Odyssey, 6.7 PowerNap, 7 Virtualization Level, 7.1 Virtualization Technology Vendors, 7.1.1 Xen, 7.1.2 VMware, 7.1.3 Kernel-based Virtual Machine (KVM), 7.2 Energy Management for Hypervisor-based VMs, 8 Data Center Level, we present the taxonomy and survey of the research in energy-efficient design of computing systems, followed by a conclusion and directions for future work in Section 9.

Section snippets

Power and Energy Models

To understand power and energy management mechanisms, it is essential to clarify the terminology. Electric current is the flow of electric charge measured in amperes. Amperes define the amount of electric charge transferred by a circuit per second. Power and energy can be defined in terms of work that a system performs. Power is the rate at which the system performs the work, while energy is the total amount of work performed over a period of time. Power and energy are measured in watts (W) and

Problems of High Power and Energy Consumption

The energy consumption by computing facilities raises various monetary, environmental, and system performance concerns. A recent study on the power consumption of server farms [2] has shown that in 2005 the electricity use by servers worldwide—including their associated cooling and auxiliary equipment—costed 7.2 billion dollars. The study also indicates that the electricity consumption in that year had doubled compared to the consumption in 2000. Clearly, there are environmental issues with the

Taxonomy of Power/Energy Management in Computing Systems

A large volume of research has been done in the area of power and energy-efficient resource management in computing systems. As power and energy management techniques are closely connected, from this point we will refer to them as power management. As shown in Fig. 5, the high-level power management techniques can be divided into static and dynamic. From the hardware point of view, SPM contains all the optimization methods that are applied at the design time at the circuit, logic,

Hardware and Firmware Level

As shown in Fig. 6, DPM techniques applied at the hardware and firmware level can be broadly divided into two categories: dynamic component deactivation (DCD) and dynamic performance scaling (DPS). DCD techniques are built upon the idea of the clock gating of parts of an electronic component or complete disabling during periods of inactivity.

The problem could be easily solved if transitions between power states would cause negligible power and performance overhead. However, transitions to

Operating System Level

In this section, we discuss research works that deal with power-efficient resource management at the OS level. The taxonomy of the characteristics used to classify the works is presented in Fig. 7. To highlight the most important characteristics of the works, they are summarized in Table II (the full table is given in Appendix A).

Virtualization Level

The virtualization level enables the abstraction of an OS and applications running on it from the hardware. Physical resources can be split into a number of logical slices called VMs. Each VM can accommodate an individual OS creating for the user a view of a dedicated physical resource and ensuring the performance and failure isolation between VMs sharing a single physical machine. The virtualization layer lies between the hardware and OS and, therefore, a virtual machine monitor (VMM) takes

Data Center Level

In this section we discuss recent research efforts in the area of power management at the data center level. Most of the approaches to dealing with the energy-efficient resource management at the data center level are based on the idea of consolidating the workload into the minimum of physical resources. Switching off idle resources leads to the reduced energy consumption, as well as the increased utilization of resources; therefore, lowering the TCO and speeding up Returns On Investments (ROI).

Conclusions and Future Directions

In recent years, energy efficiency has emerged as one of the most important design requirements for modern computing systems, ranging from single servers to data centers and Clouds, as they continue to consume enormous amounts of electrical power. Apart from high operating costs incurred by computing resources, this leads to significant emissions of CO₂ into the environment. For example, currently, IT infrastructures contribute about 2% of the total CO₂ footprints. Unless energy-efficient

Acknowledgments

We would like to thank Adam Wierman (California Institute of Technology), Kresimir Mihic (Stanford University), and Saurabh Kumar Garg (University of Melbourne) for their constructive comments and suggestions on improving the Chapter.

References (70)

R. Buyya et al.
Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility
Future Generation Comput. Syst.
(2009)
G.E. Moore
Cramming more components onto integrated circuits
Proc. IEEE
(1998)
J.G. Koomey
Estimating Total Power Consumption by Servers in the US and the World
(2007)
L. Barroso
The Price of Performance
(2005)
R. Brown et al.
Report to Congress on Server and Data Center Energy Efficiency: Public Law 109–431
(2008)
L. Minas et al.
Energy Efficiency for Information Technology: How to Reduce Power Consumption in Servers and Data Centers
(2009)
P. Ranganathan et al.
Ensemble-level power management for dense blade servers
S. Rowe
V. Venkatachalam et al.
Power reduction techniques for microprocessor systems
ACM Comput. Surv. CSUR
(2005)
L.A. Barroso et al.
The case for energy-proportional computing
Computer
(2007)

X. Fan et al.

Power provisioning for a warehouse-sized computer

M. Blackburn

Five Ways to Reduce Data Center Server Power Consumption

(2008)

Thermal Guidelines for Data Processing Environments

(2004)

G. Dhiman et al.

A system for online power prediction in virtualized environments using gaussian mixture models

G. Koch

Discovering multi-core: extending the benefits of Moore's law

Technology

(2005)

F. Petrini et al.

What are the future trends in high-performance interconnects for parallel computers?

C. Pettey

S. Devadas et al.

A survey of optimization techniques targeting low power VLSI circuits

V. Tiwari et al.

Technology mapping for low power

V. Pallipadi et al.

The ondemand governor

E. Elnozahy et al.

Energy-efficient server clusters

Power Aware Comput. Syst.

(2003)

E. Pinheiro et al.

Load balancing and unbalancing for power and performance in cluster-based systems

L. Benini et al.

A survey of design techniques for system-level dynamic power management

IEEE Trans. VLSI Syst.

(2000)

S. Albers

Energy-efficient algorithms

Commun. ACM

(2010)

M.B. Srivastava et al.

Predictive system shutdown and other architectural techniques for energy efficient programmable computation

IEEE Trans. VLSI Syst.

(1996)

C.H. Hwang et al.

A predictive system shutdown method for energy saving of event-driven computation

ACM Trans. Des. Autom. Electron. Syst.

(2000)

F. Douglis et al.

Adaptive disk spin-down policies for mobile computers

Comput. Syst.

(1995)

G. Buttazzo

Scalable applications for energy-aware processors

M. Weiser et al.

Scheduling for reduced CPU energy

Mobile Comput.

(1996)

K. Govil et al.

Comparing algorithm for dynamic speed-setting of a low-power CPU

A. Wierman et al.

Power-aware speed scaling in processor sharing systems

L.L. Andrew et al.

Optimality, fairness, and robustness in speed scaling designs

A. Weissel et al.

Process cruise control: event-driven clock scaling for dynamic power management

K. Flautner et al.

Automatic performance setting for dynamic voltage scaling

Wireless Netw.

(2002)

S. Lee et al.

Run-time voltage hopping for low-power real-time systems

Cited by (561)

Sustainable computing across datacenters: A review of enabling models and techniques
2024, Computer Science Review
The growth rate in big data and internet of things (IoT) is far exceeding the computer performance rate at which modern processors can compute on the massive amount of data. The cluster and cloud technologies enriched by machine learning applications had significantly helped in performance growths subject to the underlying network performance. Computer systems have been studied for improvement in performance, driven by user’s applications demand, in the past few decades, particularly from 1990 to 2010. By the mid of 2010 to 2023, albeit parallel and distributed computing was omnipresent, but the total performance improvement rate of a single computing core had significantly reduced. Similarly, from 2010 to 2023, our digital world of big data and IoT has considerably increased from 1.2 Zettabytes (i.e., sextillion bytes) to approximately 120 zettabytes. Moreover, in 2022 cloud datacenters consumed $\sim$ 200TWh of energy worldwide. However, due to their ever-increasing energy demand which causes ${CO}_{2}$ emissions, over the past years the focus has shifted to the design of architectures, software, and in particular, intelligent algorithms to compute on the data more efficiently and intelligently. The energy consumption problem is even greater for large-scale systems that involve several thousand servers. Combining these fears, cloud service providers are presently facing more challenges than earlier because they fight to keep up with the extraordinary network traffic being produced by the world’s fast-tracked move to online due to global pandemics. In this paper, we deliberate the energy consumption and performance problems of large-scale systems and present several taxonomies of energy and performance aware methodologies. We debate over the energy and performance efficiencies, both, which make this study different from those previously published in the literature. Important research papers have been surveyed to characterise and recognise crucial and outstanding topics for further research. We deliberate numerous state-of-the-art methods and algorithms, stated in the literature, that claim to advance the energy efficiency and performance of large-scale computing systems, and recognise numerous open challenges.
Intent-driven orchestration of serverless applications in the computing continuum
2024, Future Generation Computer Systems
Following the emergence of Internet of Things (IoT) and edge computing technologies and the development of distributed applications with strict Quality of Service (QoS) requirements, a transition is underway from centralized computing models to distributed computing continuum systems. The latter provide abstractions of the available resources, while they lead to the development of orchestration mechanisms with increased distributed intelligence and autonomy characteristics. These characteristics make the computing continuum suitable for managing serverless computing applications, considering their loose coupling with the resources and the need for reactiveness and efficiency to serve dynamic workloads in the edge and cloud part of the infrastructure. This paper presents an intent-based approach for the orchestration of a function chain over an edge and cloud infrastructure. We propose a Reinforcement Learning (RL) based autoscaling method that incorporates a reward function that aims to provide optimal utilization of resources and timely autoscaling of the functions, taking advantage of methods for workload prediction. Based on a high-level intent, the proposed optimization problem dictates the scheduling of the functions, minimizing the communication overhead and transformation cost while also considering the energy efficiency of the infrastructure. The proposed solution is evaluated with other state-of-the-art techniques for autoscaling and scheduling of serverless functions in a small edge infrastructure. Our solution provides 1.52% QoS violations with a slight increase in the deployed resources, while also significantly reducing the total cost and power consumption for the deployment of the function chain.
Imitation learning enabled fast and adaptive task scheduling in cloud
2024, Future Generation Computer Systems
Studies of resource provision in cloud computing have drawn extensive attention, since effective task scheduling solutions promise an energy-efficient way of utilizing resources while meeting diverse requirements of users. Deep reinforcement learning (DRL) has demonstrated its outstanding capability in tackling this issue with the ability of online self-learning, however, it is still prevented by the low sampling efficiency, poor sample validity, and slow convergence speed especially for deadline constrained applications. To address these challenges, an Imitation Learning Enabled Fast and Adaptive Task Scheduling (ILETS) framework based on DRL is proposed in this paper. First, we introduce behavior cloning to provide a well-behaved and robust model through Offline Initial Network Parameters Training (OINPT) so as to guarantee the initial decision-making quality of DRL. Next, we design a novel Online Asynchronous Imitation Learning (OAIL)-based method to assist the DRL agent to re-optimize its policy and to against the oscillations caused by the high dynamic of the cloud, which promises DRL agent moving towards the optimal policy with a fast and stable process. Extensive experiments on the real-world dataset have demonstrated that the proposed ILETS can consistently produce shorter response time, lower energy consumption and higher success rate than the baselines and other state-of-the-art methods at the accelerated convergence speed.
Progressive-fidelity computation of the genetic algorithm for energy-efficient virtual machine placement in cloud data centers
2023, Applied Soft Computing
Energy efficiency is a critical issue in the management and operation of data centers, which form the backbone of cloud computing. Virtual machine placement has a significant impact on the energy efficiency of virtualized data centers. Among various methods to solve the virtual-machine placement problem, the genetic algorithm has been well accepted for its quality of solution. However, it is computationally demanding largely due to the complex form of fitness function, limiting its applications in data centers. To enhance the computational efficiency of the genetic algorithm while maintaining its quality of solution, a progressive-fidelity approach is developed in this paper for genetic-algorithm computation. It starts with a low-fidelity genetic algorithm with a simple fitness function. Then, for solution refinement, it switches to a medium-fidelity genetic algorithm with a more complicated fitness function. Finally, it progresses to the fine tuning of solution through a high-fidelity genetic algorithm with the energy consumption of data centers as fitness function. Heuristics are presented for the adaptive switching of genetic-algorithm computation from low fidelity to medium fidelity and finally to high fidelity. Experiments show that compared with the standard genetic algorithm of high fidelity, our progressive-fidelity approach of genetic algorithm computation is 50% faster for large-scale data centers while maintaining similar quality of solution in terms of the energy consumption of data centers.
An optimal solution to reduce virtual machine migration SLA using host power
2023, Measurement: Sensors
The growing demand for cloud services from users, organizations are planning to build more and more data centers to meet the demand. This is expected to increase energy consumption, which in turn will lead to an increase in operating costs and CO2 emissions. Virtual machine live migration technology can be used to reduce the amount of energy consumed by virtual machines. By doing so, virtual machines can be migrated from one server to another and therefore cause a delay in delivering the services to the users. Service Level Agreement (SLA) violation is one of the important parameters to be considered for better utilization of cloud services. By applying the proposed algorithm, SLA violations will be reduced through the process of selecting the host to which the Virtual Machine will be migrated according to the remaining power of the host, and thus the SLAs will be more enforced in the long run. The proposed algorithm has been implemented in CloudSim. In the comparison of the results achieved with the existing algorithms and the proposed algorithm, it is found that the proposed algorithm is capable of reducing the violation of the SLA by 13.6% when compared to the existing algorithms.
Optimizing the use of renewable energy to minimize operational costs in distributer green data centers
2023, Renewable Energy Production and Distribution: Solutions and Opportunities: Volume 2
Green data centers are more and more deployed worldwide. They integrate many renewable sources to provide clean power and decrease their operating cost. GDCs are typically deployed over multiple locations where renewable energy availability, bandwidth prices, and grid electricity cost have high geographical diversity. This chapter focuses on delay-bounded applications in distributed GDCs (DGDCs) and performs cost and energy-effective scheduling of multiple heterogeneous applications verifying delay-bound constraints of different tasks. DGDCs' operational cost minimization problem is formulated and successfully optimized using multiple optimization approaches. Real-life data trace-driven experiments are conducted to assess and compare the effectiveness of the different approaches at solving this problem. High-performance task scheduling results are obtained. The operational cost of each GDC is minimized, and the utilization of solar and wind renewable energy from the different geographical locations is maximized while delay-bound constraints of all tasks are strictly met.

View all citing articles on Scopus

View full text

Chapter 3 - A Taxonomy and Survey of Energy-Efficient Data Centers and Cloud Computing Systems

Abstract

Introduction

Section snippets

Power and Energy Models

Problems of High Power and Energy Consumption

Taxonomy of Power/Energy Management in Computing Systems

Hardware and Firmware Level

Operating System Level

Virtualization Level

Data Center Level

Conclusions and Future Directions

Acknowledgments

Future Generation Comput. Syst.

Cramming more components onto integrated circuits

Proc. IEEE

Estimating Total Power Consumption by Servers in the US and the World

The Price of Performance

Report to Congress on Server and Data Center Energy Efficiency: Public Law 109–431

Energy Efficiency for Information Technology: How to Reduce Power Consumption in Servers and Data Centers

Ensemble-level power management for dense blade servers

Power reduction techniques for microprocessor systems

ACM Comput. Surv. CSUR

The case for energy-proportional computing

Computer

Power provisioning for a warehouse-sized computer

Five Ways to Reduce Data Center Server Power Consumption

Thermal Guidelines for Data Processing Environments

A system for online power prediction in virtualized environments using gaussian mixture models

Discovering multi-core: extending the benefits of Moore's law

Technology

What are the future trends in high-performance interconnects for parallel computers?

A survey of optimization techniques targeting low power VLSI circuits

Technology mapping for low power

The ondemand governor

Energy-efficient server clusters

Power Aware Comput. Syst.

Load balancing and unbalancing for power and performance in cluster-based systems

A survey of design techniques for system-level dynamic power management

IEEE Trans. VLSI Syst.

Energy-efficient algorithms

Commun. ACM

Predictive system shutdown and other architectural techniques for energy efficient programmable computation

IEEE Trans. VLSI Syst.

A predictive system shutdown method for energy saving of event-driven computation

ACM Trans. Des. Autom. Electron. Syst.

Adaptive disk spin-down policies for mobile computers

Comput. Syst.

Scalable applications for energy-aware processors

Scheduling for reduced CPU energy

Mobile Comput.

Comparing algorithm for dynamic speed-setting of a low-power CPU

Power-aware speed scaling in processor sharing systems

Optimality, fairness, and robustness in speed scaling designs

Process cruise control: event-driven clock scaling for dynamic power management

Automatic performance setting for dynamic voltage scaling

Wireless Netw.

Run-time voltage hopping for low-power real-time systems