Tianshu Wei
ABSTRACT
Buildings account for nearly 40% of the total energy consumption in the United States, about half of which is used by the HVAC (heating, ventilation, and air conditioning) system. Intelligent scheduling of building HVAC systems has the potential to significantly reduce the energy cost. However, the traditional rule-based and model-based strategies are often inefficient in practice, due to the complexity in building thermal dynamics and heterogeneous environment disturbances. In this work, we develop a data-driven approach that leverages the deep reinforcement learning (DRL) technique, to intelligently learn the effective strategy for operating the building HVAC systems. We evaluate the performance of our DRL algorithm through simulations using the widely-adopted EnergyPlus tool. Experiments demonstrate that our DRL-based algorithm is more effective in energy cost reduction compared with the traditional rule-based approach, while maintaining the room temperature within desired range.
- E. Barrett and S. Linder. Autonomous HVAC Control, A Reinforcement Learning Approach. Springer, 2015.Google Scholar
Cross Ref
- L. Bottou. Large-scale machine learning with stochastic gradient descent. Proceedings of COMPSTAT. 2010.Google ScholarCross Ref
- G. T. Costanzo and et al. Experimental analysis of data-driven control for a building heating system. CoRR, abs/1507.03638, 2015.Google Scholar
- EnergyPlus. https://energyplus.net/.Google Scholar
- D. Ernst and et al. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 2005. Google ScholarDigital Library
- P. Fazenda and et al. Using reinforcement learning to optimize occupant comfort and energy usage in hvac systems. Journal of Ambient Intelligence and Smart Environments, pages 675--690, 2014. Google ScholarDigital Library
- K. He and et al. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. IEEE International Conference on Computer Vision, 2015. Google ScholarDigital Library
- G. Hinton, N. Srivastava, and K. Swersky. Lecture 6a overview of mini---batch gradient descent. http://www.es.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf.Google Scholar
- B. Li and L. Xia. A multi-grid reinforcement learning method for energy conservation and comfort of HVAC in buildings. pages 444--449, 2015.Google Scholar
- Y. Ma and et al. Model predictive control for the operation of building cooling systems. IEEE Transactions on Control Systems Technology, 20(3):796--803, 2012.Google ScholarCross Ref
- M. Maasoumy and et al. Model-based hierarchical optimal control design for HVAC systems. DSCC, 2011.Google Scholar
- V. Mnih and et al. Human-level control through deep reinforcement learning. Nature 518.7540, 2015.Google Scholar
- National Solar Radiation Data Base. http://rredc.nrel.gov.Google Scholar
- D. Nikovski, J. Xu, and M. Nonaka. A method for computing optimal set-point schedules for HVAC systems. REHVA World Congress CLIMA, 2013.Google Scholar
- F. Oldewurtel and et al. Energy efficient building climate control using stochastic model predictive control and weather predictions. ACC, 2010.Google Scholar
- S. J. Olivieri and et al. Evaluation of commercial building demand response potential using optimal short-term curtailment of heating, ventilation, and air-conditioning loads. Journal of Building Performance Simulation, 2014.Google Scholar
- D. Ormoneit and S. Sen. Kernel-based reinforcement learning. Machine Learning, 49(2):161--178, 2002. Google ScholarDigital Library
- M. Riedmiller. Neural Fitted Q Iteration -- First Experiences with a Data Efficient Neural Reinforcement Learning Method. Springer, 2005.Google Scholar
- D. Silver and et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587), 2016.Google Scholar
- SCE. https://www.sce.com/NR/sc3/tm2/pdf/CE281.pdf.Google Scholar
- A. Standard. Standard 55-2004-thermal environmental conditions for human occupancy. ASHRAE Inc., 2004.Google Scholar
- D. Urieli and P. Stone. A learning agent for heat-pump thermostat control. AAMAS, 2013. Google ScholarDigital Library
- U.S. DoE. Buildings energy data book.Google Scholar
- C. J. Watkins and P. Dayan. Q-learning. Machine learning, 8(3-4):279--292, 1992. Google ScholarDigital Library
- T. Wei, Q. Zhu, and M. Maasoumy. Co-scheduling of HVAC control, EV charging and battery usage for building energy efficiency. ICCAD, 2014. Google ScholarDigital Library
- M. Wetter. Co-simulation of building energy and control systems with the building controls virtual test bed. Journal of Building Performance Simulation, 2011.Google ScholarCross Ref
- L. Yang and et al. Reinforcement learning for optimal control of low exergy buildings. Applied Energy, 2015.Google Scholar
Recommendations
MARCO - Multi-Agent Reinforcement learning based COntrol of building HVAC systems
e-Energy '20: Proceedings of the Eleventh ACM International Conference on Future Energy SystemsOptimal control of building heating, ventilation, air-conditioning (HVAC) equipment has typically been based on rules and model-based predictive control (MPC). Challenges in developing accurate models of buildings render these approaches sub-optimal and ...
Multi-zone Residential HVAC Control with Satisfying Occupants’ Thermal Comfort Requirements and Saving Energy via Reinforcement Learning
Parallel and Distributed Computing, Applications and TechnologiesAbstractResidential HVAC system control has been focused on thermal comfort and energy consumption. Due to the complexity of the dynamic building thermal model, weather conditions and human activities, traditional methods such as rule-based control (RBC) ...
An online reinforcement learning approach for HVAC control
AbstractHeating, Ventilation and Air Conditioning (HVAC) optimization for energy consumption reduction is becoming ever more a topic of the utmost environmental and energetic concerns. The two most employed methodologies for optimizing HVAC systems are ...
Highlights- HVAC optimization with reinforcement learning algorithms.
- Assessment and comparison of three different approaches.
- Online approach with imitation learning provide reliable and inexpensive solution.
Comments