PaperlayerView exact revision on arXiv

Source-linked AI summary

A Topological Filter for Learning with Label Noise

Pengxiang Wu, Songzhu Zheng, Mayank Goswami, Dimitris Metaxas, Chao Chen

arXiv:2012.04835v2cs.CVcs.LG

TL;DR

Label noise can impair deep networks, and common clean-data selection approaches often rely on noisy-classifier predictions. TopoFilter uses latent-space topology to select clean data while jointly learning representations; the authors report theoretical guarantees and empirical advantages across datasets and noise conditions.

  • Problem

    Label noise can impair deep networks, while clean-data selection heuristics based on noisy-classifier predictions often lack guarantees for unseen datasets or noise patterns.

    Source: §1 · p. 1

  • Method

    TopoFilter uses latent-space topology to select the core of the largest connected component while jointly learning representations and collecting clean data.

    Source: §1 · p. 1, §2 · p. 3, §4 · p. 9

  • Results

    TopoFilter is theoretically guaranteed to select most clean data with high confidence, and experiments report advantages in robustness across datasets.

    Source: §1 · p. 2, §4 · p. 9

  • Takeaways & Limitations

    The paper concludes that spatial patterns and group behavior in latent representations can inform model training under label noise.

    Source: §1 · p. 2

    TopoFilter focuses on connected components in latent representations; extending it to a differentiable topological loss is left for future work.

    Source: §4 · p. 9

Abstract

from arXiv · show

Noisy labels can impair the performance of deep neural networks. To tackle this problem, in this paper, we propose a new method for filtering label noise. Unlike most existing methods relying on the posterior probability of a noisy classifier, we focus on the much richer spatial behavior of data in the latent representational space. By leveraging the high-order topological information of data, we are able to collect most of the clean data and train a high-quality model. Theoretically we prove that this topological approach is guaranteed to collect the clean data with high probability. Empirical results show that our method outperforms the state-of-the-arts and is robust to a broad spectrum of noise types and levels.

1 Introduction

Label noise can impair deep networks, while common clean-data selection heuristics rely on noisy-classifier predictions and lack guarantees. TopoFilter uses latent-space topology to select clean data, with theoretical guarantees and reported empirical advantages across datasets and noise settings.

1 Introduction

  • Many clean-data selection methods trust confident noisy-classifier predictions without theoretical guarantees, while prior spatial approaches use local information rather than global topology.The introduction also notes that some selection methods train multiple networks and are computationally expensive to tune.Source: §1 · pp. 1, 3
  • With ideal features, clean samples cluster while mislabeled samples are isolated, but noisy training skews the representation and undermines that intuition.Figure 1 visualizes ideal, skewed, and learned representations for a 40%-uniformly corrupted CIFAR-10 dataset.Source: §1 · pp. 1–2
  • TopoFilter selects the core of the largest connected component, using topological structure rather than classifier confidence alone.The algorithm peels the outer layer; the largest-component-only baseline, TopoCC, is insufficient with imperfect representations.Source: §1 · pp. 1–2
  • The algorithm is guaranteed high-probability purity and majority-clean-data coverage under compact-support, continuous-label-distribution, and connected Bayes-region assumptions.The authors describe these guarantees as enabling careful and aggressive collection, including on skewed representations from noisy classifiers.Source: §1 · p. 2
  • TopoFilter consistently outperforms existing methods across CIFAR-10, CIFAR-100, and Clothing1M under a wide range of noise types and levels.The method jointly learns representations and selects clean data during training.Source: §1 · p. 2
  • Unlike noise-transition modeling methods, TopoFilter does not rely on a noise model and is independent of the loss function.The authors state that it can be combined with noise-robust losses.Source: §1 · p. 2

2 Method

TopoFilter selects data through connected structure in latent features and trains the network on the selected data, refining the largest classwise components with ζ-filtering. Its analysis gives purity and retained-data guarantees under stated assumptions.

2 Method

  • The algorithm begins with an early-stopped noisy classifier, then jointly updates the representation and selected data, training the network only on collected samples.At each epoch, selection uses the current network's latent-space topology.Source: §2 · p. 3, §2.1 · p. 4
  • TopoCC uses only the largest connected component, while TopoFilter additionally peels away its outer layer to retain the core.The extra filtering is described as essential to the method's success.Source: §2 · p. 3
  • TopoFilter keeps classwise largest connected components, then retains a point only if at least a fraction ζ of its neighbors in the union of components share its label.The method adds this filtering because outer component layers may contain corrupted data when the representation is imperfect.Source: §2 · p. 3

2.1 Theoretical Guarantee of the Algorithm

  • The proof assumes compact support, feature-independent label flips, and a condition bounding mass near the Bayes decision boundary.The cited condition on η_i is described as weaker than conditions used in many prior KNN-classifier works.Source: §2.1 · pp. 4–5
  • After ζ-filtering, the final selected set contains no points from the stated excluded region with large probability.The result describes the filtering step's role in removing uncertain points.Source: §2.1 · pp. 5–6
  • The retained-data count concentrates around the measure of L(ζ), while increasing ζ trades a smaller retained set for higher purity.The count guarantee holds for sufficiently large samples and a specified range of neighbor counts.Source: §2.1 · p. 6

3 Experiments

Experiments evaluate TopoFilter on synthetic label noise and the real-world Clothing1M dataset. Across synthetic noise settings it outperforms competitive methods, while analyses report stable filtering and robustness to parameter choices.

Experimental setup

  • Experiments use CIFAR-10 and CIFAR-100 with uniform and pair-flipping noise; labels are corrupted using transition matrices.The synthetic experiments use ResNet-18 and clean test data.Source: §3 · pp. 6–7

Results

  • TopoFilter consistently outperforms competitive methods across CIFAR-10/100 noise settings, with statistically significant test-accuracy gains over state-of-the-art methods.Table 1 reports averages and standard deviations over five trials.Source: §3 · p. 7, Table 1 · p. 7
  • TopoCC and IterNLD are less effective: connected components can retain noise, while noisy clusters may evade spatial outlier detection.The authors attribute these weaknesses to imperfect connectivity and the possibility that noisy samples form a compact cluster.Source: §3 · pp. 7–8

Behavior analysis

  • The selected-data pool maintains high purity during training as its size steadily approaches the clean-data limit, while validation accuracy does not drop.Figure 3(a) reports validation accuracy, and panels (b–e) show selected-data size and purity.Source: §3 · p. 8

Parameter analysis

  • TopoFilter is robust to validation-set size and purity, the kc and ko parameters, and feature-space dimension.The reported parameter analysis finds almost unchanged performance with noisy or small clean validation data and low sensitivity to kc and ko over a wide range.Source: §3 · p. 8, Figure 4 · p. 9

Real-world corrupted dataset

  • On Clothing1M, TopoFilter achieves the best performance among the compared baseline methods.The dataset has 1 million clothing images with labels estimated to be 61.54% accurate; evaluation uses 10,000 manually verified test images.Source: §3 · p. 8, Table 2 · p. 8

4 Conclusion

The conclusion presents TopoFilter as a method that uses latent-space topology to learn representations and collect clean data during training. The paper also states a scope limitation: it focuses on connected components.

4 Conclusion

  • TopoFilter uses feature-space topology while jointly learning data representations and collecting clean data during training.The conclusion reports theoretical guarantees and empirical advantages across datasets for robustness to label noise.Source: §4 · p. 9
  • The paper reports that TopoFilter can select most clean data with high confidence, supported by theoretical and cross-dataset empirical results.The conclusion describes improved robustness of deep models to label noise.Source: §4 · p. 9
  • The method focuses only on connected components in latent representational space; extending it to a differentiable topological loss is left for future work.The proposed future direction is based on persistent homology.Source: §4 · p. 9

Broader Impact

The broader-impact discussion frames label noise as a practical concern because corrupted labels can worsen deep-model performance. It presents data topology as an underused perspective and argues that clean-data selection may support steadier training.

Broader Impact

  • Label noise can come from imperfect human or online annotations and intentional data poisoning, and can worsen deep-model performance through memorization and overfitting.The passage describes limiting the adverse influence of noisy labels as practically significant.Source: Broader Impact · p. 9
  • The paper contrasts topology-based filtering with approaches that inspect sample losses or predicted posteriors, arguing that spatial behavior has been largely ignored.It presents the approach as supplementary to existing work.Source: Broader Impact · p. 9
  • The authors state that high-probability selection of clean data can keep the network away from corrupted labels and promote healthier, steadier training.Source: Broader Impact · p. 9

– Supplementary Material –

The supplementary material provides proofs and additional technical and experimental material.

Supplementary Material

A Proofs of Purity and Abundancy of TopoFilter

The supplement gives proofs for the paper’s two guarantees: purity of selected data and collection of a sufficient amount of clean data.

Background and Setting

The analysis models label flips and uses a KNN graph to establish connectivity of high-posterior regions, then filters points to exclude regions outside a threshold set.

  • The label-noise model assumes a flip from label i to j with probability τij, independently of the feature x.Source: Background and Setting · p. 13
  • The graph connects two feature points when either is among the other’s k nearest neighbors, then induces class-specific subgraphs using observed labels.Source: Background and Setting · p. 13
  • The proof uses connectivity of the induced KNN subgraph on points in a high-posterior region as a step toward the guarantees.For sufficiently large n and k > c1(t) log^q(n), Xi(t) is connected in Gi(X, k) with probability at least 1−δ.Source: Background and Setting · pp. 14, 18
  • Filtering points without enough neighbors of the desired label is used to remove points outside L(ζ′) from the final set.Source: Background and Setting · pp. 17–18

A.1 Proof of the Purity Theorem

The purity proof combines its lemmas to establish high-probability guarantees, including a general minimum-purity result and an average-purity result.

  • The average-purity conclusion also holds when the minimum class posterior is nonzero, although the proof expressions become more involved.Source: §A.1 · pp. 18, 21
  • With probability at least 1−δ, the proof establishes the stated purity guarantee when ζ > (1+|τ10−τ01|)/2 and n is sufficiently large.Source: §A.1 · p. 19
  • The general minimum-purity theorem gives a guarantee for admissible k values between c1(ζ) log^q(n) and c2n.Source: §A.1 · p. 21

A.2 Proof of the Abundancy Theorem

The proof treats both the abundance of retained data and agreement with the Bayes-optimal classifier. It gives a high-probability abundance bound and states that retained labels agree with Bayes-optimal predictions with large probability.

A.2 Proof of the Abundancy Theorem

  • The proof establishes a high-probability lower bound on retained-data abundance for sufficiently large n and an admissible range of k.Source: §A.2 · pp. 21–22

B On the Consistency with the Bayes Optimum

C.1 Analysis of Hyper-Parameter ζ

The ζ threshold controls how selectively TopoFilter collects clean data, and performance remains robust across tested ζ values and other hyper-parameter settings.

C.1 Analysis of Hyper-Parameter ζ

  • A sample is retained only when at least a ζ fraction of its k-nearest neighbors share its label, filtering the largest class component for purity.Source: §C.1 · p. 22
  • Binary classification requires ζ ≥ 1/2 + ε; multiclass problems can use a lower bound, and raising ζ early then relaxing it is a possible schedule.Source: §C.1 · p. 22
  • ζ values 0.25, 0.5, and 0.75 all yield reasonably good performance across datasets and noise patterns, and a constant ζ suffices.The main-paper experiments use ζ = 0.5.Source: §C.1 · pp. 22–23, Figure 2 · p. 23

C.2 Other Hyper-Parameters

  • Validation-set size and purity, k_c, k_o, and feature dimension have little effect across tested settings; experiments use clean 10k validation, k_c = 4, k_o = 32, and 512 features.Source: §C.2 · p. 23, Figure 3 · p. 23, Figure 4 · p. 23

C.3 Additional Experiments on the Point Cloud Data Domain

The point-cloud experiment extends evaluation to ModelNet40 and reports similar advantages over baselines.

C.3 Additional Experiments on the Point Cloud Data Domain

  • TopoFilter shows similar advantages over baselines on ModelNet40 point-cloud classification.Source: §C.3 · p. 24, Table 1 · p. 24
  • ModelNet40 contains 12,311 CAD models across 40 categories; experiments use 9,843 training and 2,468 test models, 20% validation, 1,024 sampled points, and PointNet.Source: §C.3 · p. 24

Related papers

Loading 2012.04835v2…