Artificial and Human Intelligence with Digital Twins
robotic applications. 12 13 Beside the simple
RL applications, there are still few real-world
applications of RL to increase efficiency. We
studied and did some research to extend an
RL algorithm for controlling the heating,
ventilation and air conditioning (HVAC)
systems. HVAC includes all the components
that are supposed to maintain a certain
comfort level in the building. Buildings
consume 30% to 40% of all consumed energy
in the world, so any improvement could
result in huge savings in energy consumption
and carbon dioxide release. 14 Advances in
new technologies in recent years have
improved the efficiency of most components
in the HVAC systems. Nevertheless, there
are still several directions to reduce the
energy consumption by controlling different
decisions on these systems.
We considered a multizone HVAC system
and selected the amount of air flow as the
main control decision. Using the obtained
data from an SAS building in Cary, NC, we
trained an environment and used it to train
an RL algorithm when there are 10 zones in
the system with a set-point of 72 with ±3
allowance. The figure below shows the
results of 50 cases with different initial
temperatures. The upper figure is the
temperature and the lower figure is taken
actions over 150 minutes in which every
three minutes a decision is taken. We
compared this result to the commonly used
rule-based algorithm (in which the system is
turned on/off at 69/75), and RL obtained
47% improvement on combination of
obtained comfort and energy consumption.
12
Hao, Karen. 2019. “The Rise of Reinforcement Learning” in “We Analyzed 16,625 Papers to Figure Out Where AI Is Headed
Next.” Available: https://www.technologyreview.com/s/612768/we-analyzed-16625-papers-to-figure-out-where-ai-is-headed-
next/
13
Burda, Yuri. 2018. “Reinforcement Learning with Prediction-Based Rewards.” Available:
https://blog.openai.com/reinforcement-learning-with-prediction-based-rewards/
14
U.S. Department of Energy, 2008, “Energy Efficiency Trends in Residential and Commercial Buildings” Available:
https://www1.eere.energy.gov/buildings/publications/pdfs/corporate/bt_stateindustry.pdf
IIC Journal of Innovation
- 67 -