Data Aware Embedded Machine Learning

Over the past decade, Machine Learning (ML) systems have revolutionized the world by turning vast datasets into intelligent systems. In doing so, they have enabled groundbreaking applications, such as human-like chatbots and autonomous vehicles, making a profound impact on everyday lives. Perhaps even more important, ML models have revolutionized fields like drug discovery, medical diagnostics, and crop pest detection — advancements with the potential to save thousands, if not millions, of lives.

However, as ML systems grow in capability and prevalence, their darker sides have come under increased scrutiny. Large language models, for instance, consume orders of magnitude more energy than traditional internet searches, contributing to global environmental challenges. Furthermore, the hardware powering these systems is often prohibitively expensive for smaller organizations and developing economies, exacerbating global inequalities. Moreover, their reliance on centralized processing raises significant concerns around data privacy, security, and fairness. These issues highlight a broader problem: the resource-intensive nature of contemporary ML systems and the societal trade-offs made in pursuit of ever-higher performance.

These challenges are not intrinsic to ML but reflect a paradigm that prioritizes performance over sustainability and accessibility. Embedded ML offers a compelling alternative, focusing on designing lean ML systems that run efficiently on low-power devices. By leveraging advanced optimization techniques — such as efficient hardware architectures, model quantizing or pruning, and automated model design using AutoML — Embedded ML aims to deliver a new generation of ML systems that are private, energy-efficient and reliable.

Despite its promise, Embedded ML adoption in the industry remains slow, largely due to the deep technical expertise required to implement its optimizations effectively. This thesis addresses these barriers by presenting new insights into the development of Embedded ML systems for Predictive Maintenance (PdM) applications, a new large and highquality dataset for advancing Embedded ML research, and Data Aware Neural Architecture Search (NAS) — a novel AutoML technique enabling unprecedented tiny ML models.

Through these contributions, this thesis advances Embedded ML toward its transformative vision: a future of sustainable, accessible, and ubiquitous machine learning.