A Min-Heap-Based Accelerator for Deterministic On-the-Fly Pruning in Neural Networks

Zuzana Jelcicova, Evangelia Kasapaki, Oskar Andersson, Jens Sparsø

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

This paper addresses the design of an area and energy efficient hardware accelerator that supports on-the-fly pruning in neural networks. In a layer of N neurons, the accelerator selects the top K neurons in every timestep. As K is fixed, the runtime of the pruned network is deterministic, which is an important property in real-time systems such as hearing aids. As a first contribution, we propose to use a min-heap for the top K selection due to its efficient data structure and low time complexity. As a second contribution, we design and implement a hardware accelerator for dynamic pruning that is based on the min-heap algorithm. The heap memory storing the top K neurons and their index is realized as a 3-port standard cell-based memory implemented with latches. As a third contribution, we evaluate the energy savings from pruning of a gated recurrent unit used in a neural network for speech enhancement (regression task). Our experiments demonstrate energy savings of ~78% without degrading the SNR improvement, and up to ~93% while reducing the SNR improvement by 0.1 - 1.11 dB. Moreover, the overhead of the hardware accelerator constitutes negligible ~0.5% of the total energy. The accelerator is implemented in a 22nm CMOS process.
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE International Symposium on Circuits and Systems (ISCAS)
Number of pages5
PublisherIEEE
Publication date2023
ISBN (Print)978-1-6654-5110-9
ISBN (Electronic)978-1-6654-5109-3
DOIs
Publication statusPublished - 2023
Event2023 IEEE International Symposium on Circuits and Systems - Monterey, United States
Duration: 21 May 202325 May 2023

Conference

Conference2023 IEEE International Symposium on Circuits and Systems
Country/TerritoryUnited States
CityMonterey
Period21/05/202325/05/2023

Keywords

  • Hardware accelerator
  • Min-heap
  • Top K elements
  • Determinism
  • Neural networks
  • Dynamic pruning
  • Hearing aids

Fingerprint

Dive into the research topics of 'A Min-Heap-Based Accelerator for Deterministic On-the-Fly Pruning in Neural Networks'. Together they form a unique fingerprint.

Cite this