Feasibility Studies in Multi-GPU Target Offloading

Anton Rydahl*, Mathias Gammelmark, Sven Karlsson

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


Many of the largest supercomputers are based on heterogeneous architectures with multiple general-purpose graphics processing units (GPGPUs) per compute node. While many APIs for GPU programming are vendor-specific, OpenMP offers a portable alternative. Therefore OpenMP target offloading is advantageous in terms of long-term code sustainability. Further, many applications have already been parallelized with OpenMP. Hence the amount of work needed to port the code to GPUs may be limited. However, the support for the OpenMP 5.x specification is not equally mature across different compilers. Additionally, the multi-GPU support in the OpenMP 5.x specification is limited. We explore what is possible with the Nvidia NVC compiler. We present a case study of solving the Poisson equation on multiple GPGPUs to outline which approaches for multi-target offloading give good results. We find that a task-based multi-GPU implementation leads to better performance than generating deferrable tasks with the clause. We demonstrate that data transfers and computations can be fully overlapped by using only the subset of the OpenMP specifications, which is supported in the 22.3 release of the Nvidia NVC compiler. For compute nodes with multiple Nvidia A100 or V100, we obtain close to ideal strong scaling when increasing the number of accelerators.

Original languageEnglish
Title of host publicationOpenMP in a Modern World : From Multi-device Support to Meta Programming - 18th International Workshop on OpenMP, IWOMP 2022, Proceedings
EditorsMichael Klemm, Bronis R. de Supinski, Jannis Klinkenberg, Brandon Neth
Publication date2022
ISBN (Print)9783031159213
Publication statusPublished - 2022
Event18th International Workshop on OpenMP - Chattanooga, United States
Duration: 27 Sept 202230 Sept 2022


Conference18th International Workshop on OpenMP
Country/TerritoryUnited States
SeriesLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13527 LNCS


  • GPGPU programming
  • Heterogeneous computing
  • Target offloading
  • Tasks


Dive into the research topics of 'Feasibility Studies in Multi-GPU Target Offloading'. Together they form a unique fingerprint.

Cite this