Label-Aware Aggregation for Improved Federated Learning

Abstract

Federated Averaging (FedAvg) is the most common aggregation method used in Federated learning, which performs a weighted averaging of the updates based on the sizes of the individual datasets of each client. A raising discussion in the research community suggests that FedAvg might not be the optimal method since, for instance, it does not fully take into account the variety of the client data distributions. In this paper, we propose a label-aware aggregation method FedLA, that addresses the biased models issue by considering the variety of labels in the weighted averaging. It combines two main properties of the client data, namely data size and label distribution. Through extensive experiments, we demonstrate that FedLA is particularly effective in several heterogeneous data distribution scenarios. Especially when only a small group of the clients is participating in the Federated Learning process. Furthermore, we argue that accurately describing the data distribution is crucial in selecting the appropriate aggregation method. In this regard, we discuss various properties that can be used to describe data distribution and illustrate how these properties can guide the choice of an aggregation method for specific data distributions.