Enhancing Multi Energy System Operations Under Uncertainty

The growing awareness of environmental concerns and the rising demand for energy necessitate a transformation in energy systems. The electricity sector is increasingly adopting more renewable energy sources (RES) to achieve sustainable decarbonization targets, despite the challenges posed by integrating uncertain and intermittent power generation. Multi energy systems (MESs) present a promising solution by enabling the integration of various energy sources and optimizing system performance. Effective management of MESs is crucial to create flexibility, both economically and technically, given the inherent uncertainties in these systems. In this context, accurate estimations of uncertain inputs is essential for the comprehensive management of MESs. The advancements and progress in machine learning (ML) offer significant potential for improving the management of MESs by addressing their stochastic nature. This thesis explores how to enhance the operation of various MESs operations under uncertainties by addressing three primary research questions, grounded in the fundamentals of ML and MESs to ensure a clear understanding: (i) “How to model the uncertainties from the operation of distributed energy resources?”, (ii) “How to incorporate uncertainty into short-term power system operation studies?”, and (iii) “How to implement real-time operational algorithms applicable to different multi-energy use cases?”.
The first research question addresses managing uncertainties by focusing on stochastic solutions, as most energy system decisions involve a significant level of uncertainty. Deterministic solutions often fall short in characterizing uncertainty and making realistic decisions. In MESs, various aspects involve randomness. This research focuses on addressing uncertainty through the forecasting of renewable energy generation and demand, as well as scenario generation. In a stochastic approach, a discrete representation of the uncertain variable using a finite set of scenarios is employed for decision-making. These scenarios are generated using variational autoencoders (VAEs) and are later solved through stochastic programming. Generative models like VAEs attempt to approximate the probability distribution of a dataset, not to predict exact outcomes, but to create representative samples that effectively cover the probability space, especially in time series data with multiple features. In this thesis, scenarios were generated for the 24-hour charging schedules of workplace electric vehicles (EV) parking lots, each with corresponding probabilities. Given the high randomness and independent preferences in EV owner behaviors, the focus is not on accurately forecasting the future, but on covering the probability space with realistically assigned scenarios and probabilities. While scenario generation aims to understand potential outcomes, another approach emphasizes predicting uncertain inputs such as power generation and load demand. deep learning (DL) models are increasingly advantageous for forecasting time series data as they can capture inherent nonlinear features and complex structures within datasets. The results of hybrid DL models for day-ahead energy forecasting demonstrate that combining various DL architectures and optimizing hyperparameters can enhance computational efficiency and provide additional room for improvement.
The second research question focuses on the operation of MESs under uncertainties by utilizing previously generated day-ahead generation and load demand forecasts, as well as EV charging scenarios. The day-ahead forecasts are used as input for the reinforcement learning (RL) agent, which interacts with its environment to explore and develop a policy that maximizes cumulative reward by the end of the day. This approach gives the agent a broader perspective for future decisions rather than simply focusing on immediate actions. Additionally, the generated scenarios are employed by the aggregator to determine a charging schedule using two-stage stochastic programming within a rolling horizon framework. The problem is formulated as two-stage stochastic model and solved deterministicmixed integer linear programming (MILP) equivalent. Both approaches incorporate uncertainties and represent intelligent control strategies, even though they may not be entirely new methods.
The final research question explores various energy system use cases, including EV charging with an aggregator, home energy management, and district heating systems, to demonstrate different approaches and solutions for managing operations and uncertainties in real time from a stochastic perspective. Each system is managed individually, although they could potentially be combined into a single case. The first use case involves the EV charging schedule for a workplace parking lot. This scenario investigates the transition toward more sustainable transportation options, with a focus on managing EV charging facility by an aggregator. The study employs generated scenarios and a two-stage stochastic model, comparing the results with worst-case scenarios and those obtained under perfect information conditions. The second use case focuses on the home energy management system (HEMS) and the scheduling of its storage unit. Day-ahead forecasts of renewable generation and electric load demand for a household are used by a RL agent, which then produces a storage unit schedule using five different algorithms and three electricity pricing scenarios. The primary goal is to investigate the differences between various deep reinforcement learning (DRL) algorithms and pricing scenarios, revealing several factors that affect the agent’s performance and highlighting areas for further improvement. The final use case focuses on the district heating system (DHS), specifically modeling an RL-based controller for the representation of Bornholm DHS. The DQN algorithm is employed to control the set points of the electric and straw boilers in a DHS inspired by the system in Bornholm. The RL agent’s decisions are examined under both low and high generation-consumption scenarios, demonstrating that physical systems can also be effectively represented and managed in Python using an RL agent. These three use cases are autonomously controlled using intelligent DL models to evaluate their potential and applicability. Although these problems could be solved without such advanced techniques, the goal is to show the feasibility of using these methods to address larger, more complex problems in a similar way.
Overall, MESs offer significant flexibility to the power system, facilitating the integration of various DL technologies and contributing to the achievement of a low-carbon energy future.