A Hierarchical Framework for Fair and Efficient Distributed Inference at the Edge

Abstract

To fully leverage the potential of artificial intelligence in distributed environments, it is essential to understand and control the use of shared resources so that all users can benefit fairly. This paper introduces the Hierarchical Inference Framework (HIF), a decentralized mechanism that enhances the collective performance of multiple edge devices without relying on explicit inter-device communication. Each device independently adjusts its inference parameters using local proxies of network state, enabling implicit coordination that balances accuracy and fairness utilization. The framework prevents network saturation and mitigates dominance by individual nodes, promoting equitable participation across heterogeneous devices. Experiments on low-power edge nodes demonstrate that HIF improves global inference accuracy and fairness while maintaining efficient use of communication resources. These results highlight the potential of hierarchical, proxy-based adaptation as a scalable strategy for fairness-aware distributed inference in future edge and mobile AI systems.