Distributing Inference in the User Plane of Complex Network Topologies with DUNE

Abstract

The deployment of Machine Learning (ML) models in the user plane has emerged as a promising approach to enable line-rate in-network inference, thereby reducing end-to-end latency and improving the scalability of network functions such as network telemetry. Nevertheless, integrating ML models into programmable switches remains challenging due to stringent memory and computational constraints. Prior work has predominantly focused on deploying monolithic ML models into individual programmable network devices, an approach fundamentally limited by hardware resources, particularly for complex classification tasks. In this paper, we introduce DUNE, a novel framework that, for the first time, enables distributed user-plane inference across multiple programmable network devices. DUNE employs original, fully automated techniques to (i) decompose large ML models into lightweight sub-models that retain inference accuracy while reducing resource consumption, and (ii) determine the design, sequencing, and placement of these sub-models to support efficient, distributed joint packet- and flow-level inference. We deploy P4 implementations of DUNE both in an experimental testbed with industry-grade programmable switches and in an emulation environment. We leverage the former setup to demonstrate the practical viability of the solution with real-world hardware in a simple linear topology, and the latter to evaluate DUNE’s scalability to larger and more complex network topologies coexisting with routing strategies. In both cases, we evaluate the framework using real-world traffic on two challenging classification tasks: our results show that DUNE not only reduces per-switch resource usage compared to traditional monolithic ML deployments but also improves inference accuracy by up to 7.5%.