PaperlayerView exact revision on arXiv

Source-linked AI summary

Sample Selection Using Multi-Task Autoencoders in Federated Learning with Non-IID Data

Emre Ardıç, Yakup Genç

arXiv:2604.26116v1cs.CVcs.LG

TL;DR

Federated learning needs better sample selection for heterogeneous, noisy client data. This paper uses multitask autoencoders with server-managed outlier detection and federated SVDD loss, improving accuracy by up to 7.02% on CIFAR10 with OCSVM.

  • Problem

    Sample contribution estimation and effective selection remain underexplored for large-scale, non-IID federated learning, where heterogeneous data can bias training and slow convergence.

  • Method

    A multitask autoencoder analyzes losses and features for server-managed OCSVM, IF, and adaptive-threshold selection, augmented by federated multi-class SVDD loss.

  • Results

    Across CIFAR10 and MNIST experiments with varying clients and noise, sample selection improved global accuracy, with gains up to 7.02% on CIFAR10 using OCSVM.

  • Takeaways & Limitations

    The methods improve federated image-classification accuracy under non-IID and noisy conditions without preprocessing or pre-trained models.

  • Takeaways & Limitations

    OCSVM and IF impose computational complexity, while their contamination parameters require tuning to the dataset’s noise level.

Abstract

from arXiv · show

Federated learning is a machine learning paradigm in which multiple devices collaboratively train a model under the supervision of a central server while ensuring data privacy. However, its performance is often hindered by redundant, malicious, or abnormal samples, leading to model degradation and inefficiency. To overcome these issues, we propose novel sample selection methods for image classification, employing a multitask autoencoder to estimate sample contributions through loss and feature analysis. Our approach incorporates unsupervised outlier detection, using one-class support vector machine (OCSVM), isolation forest (IF), and adaptive loss threshold (AT) methods managed by a central server to filter noisy samples on clients. We also propose a multi-class deep support vector data description (SVDD) loss controlled by a central server to enhance feature-based sample selection. We validate our methods on CIFAR10 and MNIST datasets across varying numbers of clients, non-IID distributions, and noise levels up to 40%. The results show significant accuracy improvements with loss-based sample selection, achieving gains of up to 7.02% on CIFAR10 with OCSVM and 1.83% on MNIST with AT. Additionally, our federated SVDD loss further improves feature-based sample selection, yielding accuracy gains of up to 0.99% on CIFAR10 with OCSVM. These results show the effectiveness of our methods in improving model accuracy across various client counts and noise conditions.

1. Introduction

The introduction identifies communication, device, and statistical heterogeneity as core federated-learning challenges and motivates filtering abnormal samples to improve training. It presents a multi-task autoencoder and server-managed outlier detection to estimate sample contributions through loss and feature analysis.

  • Challenges: Federated learning faces communication overhead, device heterogeneity, and statistical heterogeneity because clients differ in capabilities and locally generated data.These differences make reducing communication rounds and transmitted data important while complicating model training.
  • Motivation: Filtering abnormal client samples can produce more accurate model updates, faster convergence, and reduced communication costs.The approach addresses federated-learning challenges by removing abnormal samples before their updates affect training.
  • Motivation: Sample-contribution estimation supports anomaly detection, robust learning, and client valuation by identifying low-quality data and assessing client sample quality.The introduction describes sample valuation as a fundamental yet underexplored federated-learning problem.
  • Motivation: Existing valuation methods are inefficient for large-scale, non-IID federated learning, because Shapley Value is computationally costly and loss- or gradient-based estimates remain underexplored.The stated setting involves a high number of clients, up to 1000.
  • Proposed approach: The proposed Multi-Task Autoencoder jointly supports Image Classification and Reconstruction, estimating sample contributions from IR and IC losses and feature-based outlier detection.The architecture is designed to filter abnormal samples on clients and improve model accuracy.
  • Contributions: The contributions include a server-managed unsupervised outlier-detection strategy for non-IID federated learning and a multi-class federated SVDD loss for feature-based selection.The stated contribution summary also includes a new variant of FedBalancer and evaluation across client counts and noise types.

2. Related Works

Related work estimates client and sample contributions in federated learning using Shapley Value, loss or gradient norm, while anomaly detection uses autoencoders to identify noisy and malicious samples. Shapley Value methods are accurate but slower for resource-constrained clients, whereas loss-based methods require a single forward pass.

  • Contribution valuation: Federated learning valuation research using Shapley Value remains limited and treats clients or data sources as players in a cooperative game.Federated Shapley Value computes total gains generated by players.
  • Contribution valuation: Shapley Value and deletion methods estimate grouped feature and instance importance in vertical and horizontal federated learning, respectively.Other approaches estimate client sample contributions using leave-one-out and Shapley Value methods, requiring multiple local-model retrainings.
  • Contribution valuation: Loss- and gradient-norm methods estimate sample importance through a forward pass and are faster than Shapley Value methods for resource-constrained federated learning clients.Li et al. aggregate local sample importance to compute client importance.
  • Anomaly detection: Anomaly detection is receiving increasing attention in federated learning, particularly for cybersecurity, healthcare, and industrial applications.Autoencoders help identify noisy and malicious samples through IR loss in decentralized and federated settings.

3. Methods

The methods define a synchronous federated-learning pipeline, characterize noisy data, and use a multi-task autoencoder with adaptive sample-selection strategies to filter problematic samples.

  • Federated-learning process: Federated learning uses a server and N clients that exchange model updates, aggregate them with FedAvg, and iteratively update the global model.The process includes client selection, broadcasting, local updates, aggregation, and global updating.
  • Noise characterization: Clients are trusted but may contain noisy, redundant, or abnormal samples that cannot be centrally inspected because their datasets remain inaccessible.The study distinguishes closed-set label noise from open-set noise and evaluates both on MNIST and CIFAR10.
  • MTAE architecture: The MTAE combines an encoder, decoder, and classifier to produce embedded features, reconstructed inputs, and predicted labels for each sample.Its reconstruction loss is measured using Mean Squared Error, while the architecture supports image reconstruction and classification.
  • Sample contribution estimation: Sample contribution is estimated from per-sample loss computed in one client-side forward pass, with high loss indicating potentially valuable or problematic samples.The proposed loss-based approaches use these loss values as sample contribution estimates for identifying malicious, abnormal, or redundant data.
  • Adaptive sample selection: Adaptive threshold selection combines reconstruction and classification losses, while a server-adjusted threshold separates presumed-normal samples from outliers or important samples.The threshold increases early in training and adapts as the loss distribution changes.

2 CalculateLt(LLowR, LHighR, ltr) · 7 ControlLtr(U, LossSumR, LR, ltr, lss, tw, R)

The provided excerpt states a control condition that compares two sums over adjacent ranges of R, using a strict greater-than test.

  • 7 ControlLtr(U, LossSumR, LR, ltr, lss, tw, R): The condition is evaluated as line 11 of ControlLtr.The supplied passage labels the expression as line 11.
  • 7 ControlLtr(U, LossSumR, LR, ltr, lss, tw, R): The test compares ∑U over the range R −2tw : R −tw with ∑U over R −tw : R.These are the two exact ranges shown in the condition.
  • 7 ControlLtr(U, LossSumR, LR, ltr, lss, tw, R): The first summed interval is R −2tw : R −tw.This interval appears on the left side of the comparison.
  • 7 ControlLtr(U, LossSumR, LR, ltr, lss, tw, R): The second summed interval is R −tw : R.This interval appears on the right side of the comparison.
  • 7 ControlLtr(U, LossSumR, LR, ltr, lss, tw, R): Both interval sums apply U to the corresponding range.The notation ∑U is used for both terms.
  • 7 ControlLtr(U, LossSumR, LR, ltr, lss, tw, R): The condition holds only when the first sum is greater than the second.The comparison operator between the two sums is >.

13 else · 17 SelectSample(Di, lossi, ltR, p) · 22 else

The method centrally detects and filters abnormal samples using loss- or feature-based outlier models, with adaptive threshold control and warm-up scheduling. Federated multi-class SVDD further shapes class-specific feature hyperspheres through server-computed centroids and radii.

  • 22 else: UR compares the selected samples’ total loss with their count to guide adaptive threshold updates every tw rounds.If recent average loss is lower than past rounds, ltr increases; otherwise, it decreases to separate normal and noisy samples.
  • 3.6. Unsupervised Outlier Detection: OCSVM and IF identify outliers in either a weighted CE-MSE 2D loss space or the feature space.The methods support loss-based and feature-based sample selection.
  • 3.6. Unsupervised Outlier Detection: The central server trains the outlier detector from client-collected losses or features and broadcasts it with the global model.Clients then compute losses or embeddings and remove detected outliers before local training.
  • 3.6. Unsupervised Outlier Detection: Retraining the outlier detector every tw rounds reduces computational cost, while feature-based selection replaces losses with sample embeddings.The feature detector operates in feature space rather than the 2D loss space.
  • 3.6. Unsupervised Outlier Detection: Sample selection begins after a predefined warm-up round ts so the model learns normal samples before abnormal-sample memorization.Excessive warm-up can cause overfitting in the CE-MSE loss space, whereas feature selection requires sufficient embedding-space learning.
  • 3.7. Federated Multi-Class SVDD Loss: Multi-class deep SVDD loss regularizes the autoencoder to make embedded points more suitable for outlier detection across classes.The combined objective is L′ = L + λregLreg, with λreg controlling feature-space distortion.
  • 3.7. Federated Multi-Class SVDD Loss: Class-specific hyperspheres map most normal points inside while pushing anomalous points outside, using centroid µi and radius Ri for each class.The objective learns a minimum hypersphere for every class in the embedding space.
  • 3.7. Federated Multi-Class SVDD Loss: The server computes class centroids and radii from a public test dataset after a predefined target, distributes them to clients, and updates radii using the q-th distance quantile.Here q = 1 −ν controls the fraction of data points treated as outliers.

4. Experiments and results

Experiments on noisy, non-IID CIFAR10 and MNIST data evaluated server-managed sample selection across client counts and noise types. Loss-based methods improved accuracy in several settings, while feature-based selection generally underperformed unless federated SVDD loss helped specific scenarios.

  • Experimental setup: Experiments introduced 40% open-set or closed-set noise before non-IID partitioning and evaluated OCSVM, IF, and AT across CIFAR10, MNIST, noise types, and client counts.The methods were evaluated in a FedML-based federated learning environment, with the primary goal of improving server-computed global test accuracy.
  • Noise impact: 38.59% was the lowest CIFAR10 accuracy, occurring with 50 clients under closed-set noise; MNIST accuracy fell to 85.58% for 50 clients under closed-set noise.Closed-set noise caused the largest accuracy reductions, while EMNIST affected MNIST more than ImageNet32 and ImageNet32 affected CIFAR10 more than SVHN.
  • Loss-based selection: 1.83% was the largest MNIST closed-set improvement, achieved by AT for 100 clients; OCSVM achieved a 1.12% improvement in EMNIST experiments for 50 clients.IF improved accuracy for 1000, 200, and 100 clients in MNIST closed-set experiments but failed across all client counts in EMNIST experiments.
  • Loss-based selection: 7.02% was the highest CIFAR10 improvement, achieved by OCSVM for 50 clients under closed-set noise; IF reached a 6.65% increase in the same setting.OCSVM and IF improved accuracy across all client counts in the CIFAR10 closed-set experiments, while AT produced mixed results.
  • Evaluation metrics: 5.55% precision gain and 5.64% F1-score gain were achieved by OCSVM over the baseline on CIFAR10 across 1000 clients, with OCSVM outperforming the other methods.Macro-averaging gave equal weight to each class, providing a balanced view of classification performance.
  • Feature-based selection with SVDD loss: 0.99% was the highest improvement from federated SVDD loss, achieved by OCSVM for 200 clients in the CIFAR10 ImageNet32 open-set experiments; feature-based selection otherwise reduced accuracy across scenarios.Federated SVDD loss also produced a 0.59% OCSVM improvement for 200 clients with SVHN open-set noise, but its effectiveness was limited in some client-count settings.

5. Limitations

The methods are limited by the computational complexity of server-side outlier detection and by sensitivity to contamination settings. Future improvements include adaptive contamination adjustment and class weighting in federated SVDD for non-IID imbalance.

  • Computational complexity: Server-side OCSVM and IF outlier detection introduce computational complexity, with OCSVM requiring O(n^2 · d) to O(n^3 · d) and IF requiring O(t · ψ log ψ).OCSVM complexity depends on convergence speed and the number of support vectors.
  • Parameter sensitivity: IF and OCSVM contamination parameters must match dataset noise levels; overly high settings can misclassify high-quality samples as outliers and reduce model accuracy.Adaptive methods are proposed to adjust contamination parameters.
  • Federated SVDD: Adding class weights on clients to federated SVDD could address non-IID class imbalance and improve separation of outliers from normal samples in feature-based selection.The proposed improvement targets the unbalanced nature of non-IID distributions.

6. Conclusion

The paper introduces training-time sample selection for federated learning without preprocessing or pre-trained models, using centrally managed outlier detection and adaptive thresholds. Experiments in a FedML simulation show improvements in global evaluation metrics, while future work targets computational overhead and adaptive tuning.

  • Contributions: The proposed methods select high-quality client samples during training without preprocessing or pre-trained models, using centrally managed unsupervised outlier detection and adaptive thresholds.Sample contributions are estimated through loss and feature analysis with a multi-task autoencoder architecture.
  • Evaluation: FedML-based multi-process simulations evaluate OCSVM, IF, and AT methods for periodically updated loss-based sample selection on non-IID datasets.The methods use losses or features collected from clients during training.
  • Results: The experiments show significant improvements in global test accuracy, precision, recall, and F1 score.These results support the effectiveness of the evaluated sample-selection methods.
  • Future work: Future work will develop lightweight, scalable solutions to reduce computational overhead and federated-learning resource demands in large-scale real-world scenarios.The authors also plan to optimize activation strategies and adapt hyperparameters such as OCSVM and IF contamination rates without manual tuning.

Declaration of Generative AI and AI-assisted technologies in the writing process

The authors used OpenAI’s ChatGPT-3.5/4o to improve readability and language during preparation of the work, then reviewed and edited the content as needed while retaining full responsibility.

  • Use of generative AI: The authors used OpenAI’s ChatGPT-3.5/4o during preparation of the work.The tool was used as part of the writing process.
  • Use of generative AI: The stated purpose of using ChatGPT-3.5/4o was to improve readability and language.The declaration identifies language and readability improvement as the tool’s role.
  • Author responsibility: After using the tool, the authors reviewed and edited the content as needed and took full responsibility for the publication’s content.The declaration places final review, editing, and responsibility with the authors.
Loading 2604.26116v1…