Skip to main navigation Skip to search Skip to main content

Graph Reinforcement Learning for Network Control via Bi-Level Optimization

  • Stanford University
  • Google Research
  • National University of Singapore

Research output: Contribution to conference › Paper › Research

19 Downloads (Orbit)

Abstract

Optimization problems over dynamic networks have been extensively studied and widely used in the past decades to formulate numerous real-world problems. However, (1) traditional optimization-based approaches do not scale to large networks, and (2) the design of good heuristics or approximation algorithms often requires significant manual trial-and-error. In this work, we argue that data-driven strategies can automate this process and learn efficient algorithms without compromising optimality. To do so, we present network control problems through the lens of reinforcement learning and propose a graph networkbased framework to handle a broad class of problems. Instead of naively computing actions over high-dimensional graph elements, e.g., edges, we propose a bi-level formulation where we (1) specify a desired next state via RL, and (2) solve a convex program to best achieve it, leading to drastically improved scalability and performance. We further highlight a collection of desirable features to system designers, investigate design decisions, and present experiments on real-world control problems showing the utility, scalability, and flexibility of our framework.
Original languageEnglish
Publication date2023
Number of pages24
Publication statusPublished - 2023
Event40th International Conference on Machine Learning - Honolulu, United States
Duration: 23 Jul 2023 → 29 Jul 2023

Conference

Conference40th International Conference on Machine Learning
Country/TerritoryUnited States
CityHonolulu
Period23/07/2023 → 29/07/2023

Fingerprint

Dive into the research topics of 'Graph Reinforcement Learning for Network Control via Bi-Level Optimization'. Together they form a unique fingerprint.

Cite this