This European patent describes a method for training a machine learning model, particularly in the context of communication networks like 5G. The primary focus is to improve the accuracy of machine learning models trained using federated learning (FL) by addressing the issue of data heterogeneity among the FL clients. The proposed method involves determining the similarity between the federated learning server’s test data and the clients’ training data and using this determination to decide whether or not to incorporate a client’s model update into the global model.
Key Themes
Machine Learning for Communication Network Analytics: The patent highlights the importance of machine learning models for network statistical analysis and prediction in communication networks, such as 5G mobile networks. These models are used to monitor network conditions (load, resource usage, component status) and predict potential issues like overload, enabling measures to maintain service quality.
Federated Learning (FL) in Communication Systems: The document describes federated learning as a decentralized ML technique where a global model is trained across multiple clients using their local datasets, coordinated by a central server. It outlines the typical iterative process of FL (server selecting clients, distributing models, clients training locally, and server aggregating updates). The application of FL within the 5G system is discussed, specifically mentioning its use for NWDAF (Network Data Analytics Function) analytics.
Network Data Analytics Function (NWDAF): The patent explains the role of the NWDAF in providing network analysis and prediction information to other network functions (NFs) in a 5G system. It introduces the decomposition of the NWDAF into two logical functions: NWDAF(AnLF) for analytics and inference, and NWDAF(MTLF) for model training.
Data Heterogeneity in Federated Learning: A significant challenge in FL, particularly in communication network contexts, is data heterogeneity among clients. The patent explains that “if a local training dataset of a FL client is quite different from the local training data of other clients,” it can lead to decreased accuracy of the global model due to detrimental updates from clients with “ill-fitting” data. This heterogeneity can arise from temporal or geographical differences, faulty behavior, or malicious attacks.
Addressing Data Heterogeneity: The core of the invention lies in methods to mitigate the negative impact of data heterogeneity. The proposed approaches involve comparing the statistical characteristics of the FL server’s test data (G-Data) and the FL clients’ local training data (L-Data) to determine if they meet a “predetermined similarity criterion.” Based on this similarity, and potentially in conjunction with accuracy comparisons, decisions are made about whether to include a client’s update.
Most Important Ideas and Facts
Problem: Training accurate ML models in communication networks using federated learning is challenging due to data heterogeneity among clients.
Solution: A method is proposed to determine if a client’s training data is sufficiently similar to the server’s test data before incorporating the client’s model update into the global model.
Similarity Criterion: The similarity criterion can be based on comparing statistical information (e.g., mean, quantile, variance) about the server’s test data and the client’s training data.
Decision Making: If the data are sufficiently similar, the client’s update is used to update the global model. If not, the update may be avoided, and/or the training or test data may be updated.
Accuracy Consideration: The method may also involve comparing the accuracy of the updated local model on local test data (L-Accuracy) with the accuracy of the current global model on the global test data (G-Accuracy).
- If L-Accuracy < G-Accuracy and data is statistically different, the client may not participate in training and may obtain new local data.
- If L-Accuracy > G-Accuracy and data is statistically different, the client may request the server to update its global test data.
- If data is not statistically different, the client sends its model update.
Implementation Location of Heterogeneity Detection: The data heterogeneity detection logic can reside on the FL clients, the FL server, or both.
NWDAF Context: The method is particularly relevant for training ML models within the NWDAF for network analytics, including predictions of communication resource needs.
Improved Performance: By improving the accuracy of FL-trained models, the method leads to more accurate predictions, better decision-making by NFs and AFs (Application Functions), and ultimately “improved performance” in the communication network.
Data Privacy: The similarity check can be performed by exchanging characteristics (statistics, distribution, sparsity) rather than the actual datasets, which is crucial for maintaining data privacy in FL.
System Implementation: The method can be implemented in a data processing system comprising data processing devices (mobile terminals, servers) with communication interfaces, memory, and processing units.