Demonstrating Distributed Inference in the User Plane with DUNE

Abstract

Deploying Machine Learning (ML) models in the user plane enables low-latency and scalable in-network inference, but integrating them into programmable devices faces stringent constraints in terms of memory resources and computing capabilities. In this demo, we show how the newly proposed DUNE, a novel framework for distributed user-plane inference across multiple programmable network devices by automating the decomposition of large ML models into smaller sub-models, mitigates the limitations of traditional monolithic ML designs. We run experiments on a testbed with Intel Tofino switches using measurement data and show how DUNE not only improves the accuracy that the traditional single-device monolithic approach gets but also maintains a comparable per-switch latency.