Learn to Stay Cool: Online Load Management for Passively Cooled Base Stations

Passively cooled base stations (PCBSs) are highly relevant for achieving better efficiency in cost and energy. However, dealing with the thermal issue via load management, particularly for outdoor deployment of PCBS, becomes crucial. This is a challenge because the heat dissipation efficiency is subject to (uncertain) fluctuation over time. Moreover, load management is an online decision-making problem by its nature. In this paper, we demonstrate that a reinforcement learning (RL) approach, specifically Soft Actor-Critic (SAC), enables to make a PCBS stay cool. The proposed approach has the capability of adapting the PCBS load to the time-varying heat dissipation. In addition, we propose a denial and reward mechanism to mitigate the risk of overheating from the exploration such that the proposed RL approach can be implemented directly in a practical environment, i.e., online RL. Numerical results demonstrate that the learning approach can achieve as much as 88.6% of the global optimum. This is impressive, as our approach is used in an online fashion to perform decision-making without the knowledge of future heat dissipation efficiency, whereas the global optimum is computed assuming the presence of oracle that fully eliminates uncertainty. This paper pioneers the approach to the online PCBSs load management problem.