PaperlayerView exact revision on arXiv

Source-linked AI summary

Safeguarding Mutual Correction in Source-Free Domain Adaptation via Cut Statistics

Seongjun Lee, Changhee Lee

arXiv:2610.02981v1cs.AI

TL;DR

SFDA must adapt to unlabeled target data without source data, while single-model refinement and one-way ViL supervision leave mutual correction between models underused. SafeCut gates cross-model supervision by cut-based reliability, and the paper reports state-of-the-art performance across diverse domain adaptation benchmarks.

  • Problem

    SFDA methods lack a label-free way to determine which of two models is correct on each sample, despite their distinct failure modes and potential for mutual correction.

    Source: §1 · p. 2

  • Method

    SafeCut uses the cut statistic as a label-free reliability measure to dynamically control the direction and strength of per-sample cross-model supervision.

    Source: §1 · p. 2, §4.2 · p. 6

  • Results

    SafeCut achieves state-of-the-art performance across diverse domain adaptation benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support exploiting complementary failure modes for mutual correction in SFDA.

    Source: §1 · p. 2

    The approach assumes that the two models have complementary correctness patterns, including nonempty regions where only one model is correct.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Source-Free Domain Adaptation (SFDA) aims to adapt a source-pretrained model to an unlabeled target domain without access to the original source domain. While early single-model approaches rely on self-refinement, they are inherently susceptible to confirmation bias and struggle to correct their own systematic errors. To overcome this limitation, recent methods introduce Vision-Language (ViL) models as external knowledge sources. However, these approaches operate in a largely unidirectional paradigm, using the ViL model primarily to supervise the source-pretrained model. This overlooks a key structural property: the two models exhibit distinct failure modes -- where one produces an incorrect prediction, the other may produce a correct one, creating a natural opportunity for mutual correction within the target domain. Yet, without ground-truth labels, identifying which model is correct on any given sample is non-trivial, and naively exchanging predictions risks propagating errors across models. To address this challenge, we propose SafeCut, a novel approach that leverages the cut statistic as a label-free measure of prediction reliability to gate cross-model supervision. Our approach dynamically controls both the direction and strength of supervision based on relative reliability, selectively amplifying true corrections while suppressing miscorrections on a per-sample basis. We further provide theoretical justification showing that this reliability-gated mechanism guarantees a net-positive correction signal. Extensive experiments across diverse SFDA benchmarks demonstrate that SafeCut achieves state-of-the-art performance, highlighting the effectiveness of safeguarding mutual correction in SFDA via cut statistics.

1 Introduction

SFDA adapts a source-pretrained model using unlabeled target data without access to source data, but single-model self-refinement struggles with confirmation bias. SafeCut uses cut-based reliability to gate mutual supervision and reports state-of-the-art performance across domain adaptation benchmarks.

1 Introduction

  • Single-model self-refinement is vulnerable to confirmation bias and struggles to correct systematic errors distributed across the target domain.Local proximity or consistency-based supervision may not correct these errors.Source: §1 · p. 2
  • Vision-Language supervision is usually one-sided, despite distinct failure modes and sample-varying reliability that create opportunities for mutual correction.Unconditional reliance can limit gains, particularly in domains where ViL models are less reliable.Source: §1 · p. 2
  • SafeCut uses cut statistics as a label-free reliability measure to control supervision direction and strength, reporting state-of-the-art results across diverse benchmarks.It selectively amplifies corrective signals while suppressing miscorrections.Source: §1 · p. 2

2 Related Works

Prior SFDA work either refines the source model using its own predictions or uses ViL models as auxiliary knowledge. Related cross-model methods do not address the fully unsupervised SFDA setting, which SafeCut targets with cut-based routing.

Source-Free Domain Adaptation

  • SFDA methods use self-supervised pseudo-labeling, neighborhood structure, or estimated reliability, but remain constrained by the source model’s blind spots and confirmation bias.ViL-based approaches adapt ViL models or denoise their pseudo-labels.Source: §2 · p. 2
  • Existing ViL-based SFDA methods do not explore more general counterpart models and leave effective knowledge transfer between models largely unaddressed.Source: §2 · p. 2

Cross-Model Supervision

  • Mutual learning and co-teaching exchange predictions or select reliable samples, but rely on labeled or noisy-label supervision; SafeCut extends cross-model correction to label-free SFDA using cut statistics.Source: §2 · p. 3

3 Mutual Correction in Source-free Domain Adaptation

SFDA lacks target labels for deciding which of two complementary models is correct on a sample, making mutual correction vulnerable to miscorrection. SafeCut uses cut-based reliability differences to route supervision, with theoretical results linking reliability margins to correction dominance.

3.1 Problem Setting

  • The target model starts from the source model, while a peer model is assumed to have complementary correctness patterns with nonempty regions where only one model is correct.This asymmetry is the basis for mutual correction.Source: §3.1 · p. 3
  • Cross-model pseudo-labeling can correct an erroneous prediction or propagate a peer’s error, but ground-truth-free SFDA cannot directly identify which model is correct.Oracle selection would accept supervision only from the model that is correct on each sample.Source: §3.1 · p. 3

3.2 When to Rely on Which Model: A Cut Statistic Approach

  • Across evaluated settings, prediction accuracy is approximately monotonically decreasing with each model’s cut statistic, as illustrated by Figure 2.The statistic measures prediction consistency in a local neighborhood.Source: §3.2 · p. 4, Figure 2 · p. 4
  • The cut margin Δ(x) = zv(x) − zt(x) indicates relative reliability by its sign, while its magnitude reflects the strength of the reliability gap.Thresholds use positive and negative margins to approximate the two one-sided correctness regions.Source: §3.2 · p. 4
  • Under baseline reliability margins, the theorem establishes correction dominance; sufficiently stricter margins increase the true-correction-to-miscorrection ratio over baseline.The result motivates margin-based selection of supervision regions.Source: §3.2 · p. 5
  • Because optimal hard thresholds are infeasible without labels, the method uses soft supervision weights that increase with the absolute cut margin.The formulation uses a symmetric margin criterion.Source: §3.2 · p. 5

4 Training with Reliability-Gated Supervision

SafeCut estimates per-sample reliability from local target-domain neighborhoods, then uses reliability-weighted cross-model supervision and temporal self-anchoring. Batch-level regularizers add global structure to the sample-wise updates.

4.1 Estimating Prediction Reliability via Cut Statistics

  • SafeCut builds local neighborhood graphs from each model’s features, using mutual nearest neighbors to mitigate spurious links in low-density regions.Experiments use penultimate-layer features.Source: §4.1 · p. 5
  • Distance-weighted cut statistics are normalized across regions, mapped to reliability scores by an exponential function, then stabilized using cached features and an epoch-wise moving average.Source: §4.1 · p. 6

4.2 Reliability-Gated Mutual Correction

  • A sigmoid gate assigns complementary supervision weights in each direction, conserving the total supervisory budget and giving greater weight to the more reliable model.The temperature controls how sharply weights respond to the reliability gap.Source: §4.2 · p. 6
  • Temporal self-anchoring preserves predictions according to each model’s reliability, while batch-level mutual-information and diversity regularizers align prediction distributions and discourage class collapse.Anchoring addresses residual errors that soft gating alone may allow to accumulate.Source: §4.2 · pp. 6–7

5 Experiments

SafeCut is evaluated across natural-image and medical SFDA settings, with dual-ResNet experiments, training dynamics, and peer-complementarity analyses. The experiments also examine ablations and sensitivity to gating sharpness.

5.1 Experimental Setup

  • Evaluation uses closed-set SFDA on Office-31, Office-Home, DomainNet-126, and VisDA, mostly with ResNet-50 and CLIP ViT-B/32 as the peer.Additional settings replace CLIP with a non-CLIP peer and include Camelyon17 and EyePACS-to-APTOS 2019 medical adaptation.Source: §5.1 · p. 7

5.2 Performance Evaluation

  • Both models improve across all 12 dual-ResNet source-domain pairs, and adapted performance exceeds non-CLIP SFDA baselines in most combinations.The peer models have distinct failure modes, the condition identified as necessary for SafeCut.Source: §5.2 · p. 7, Table 4 · p. 8

5.3 Additional Analysis

  • Table 5(c) indicates that adapted ViL is essential relative to zero-shot CLIP, while Table 5(d) reports robustness to gating sharpness β with minimum tuning.An oracle partition provides an upper bound, suggesting that better identification of relative expertise could yield further improvement.Source: Table 5 · p. 8, §5.3 · p. 9
  • Jointly correct samples rise from 56.6% to 91.7%, while one-sided-correct samples nearly disappear and jointly incorrect samples decrease from 9.8% to 7.1%.Across training, jointly correct samples maintain low cut values, while jointly incorrect samples maintain high values.Source: §5.3 · p. 9, Figure 3 · p. 9
  • Higher pre-adaptation prediction agreement is associated with smaller adaptation gains (Pearson r = −0.81), yet the highest-agreement pair still gains +10.4%p without degradation.The highest observed agreement is 69.6%.Source: Figure 4 · p. 9, §5.3 · p. 9
  • On Camelyon17, zero-shot CLIP underperforms the source-pretrained target model (50.01% vs. 64.14%), while complete SafeCut reaches 94.96%, significantly outperforming ProDe through reliability-gated mutual correction.Source: §5.3 · pp. 9–10
  • 46.0% and 45.9%: SafeCut raises target-model and peer accuracy, respectively, on the five-class EyePACS-to-APTOS 2019 task, versus 35.2% target accuracy with ProDe.Zero-shot CLIP reaches 25.4% accuracy on the task.Source: §5.3 · p. 10

6 Conclusion

SafeCut uses cut statistics to gate mutual correction between models with different failure modes. Experiments report state-of-the-art performance across multiple benchmarks and generalization to diverse peers and medical imaging.

6 Conclusion

  • SafeCut uses the cut statistic as a label-free reliability proxy to control the direction and strength of cross-model supervision.Temporal self-anchoring complements this mechanism to amplify valid corrections while mitigating miscorrections and error accumulation.Source: §6 · p. 10
  • SafeCut achieves state-of-the-art performance across multiple benchmarks and generalizes across diverse peer models, including in challenging medical-imaging domains.The conclusion characterizes it as a general-purpose correction framework.Source: §6 · p. 10
  • The framework exploits differing failure modes between target and peer models rather than unconditionally trusting external models.Its reliability-gated mutual correction is presented as theoretically grounded.Source: §6 · p. 10

A Proofs

The proofs establish correction dominance in margin-separated regions by relating cut-statistic ordering to conditional prediction correctness. Under noisy monotonicity, the guarantee holds when the margin exceeds the noise-to-decay ratio.

A Proofs

  • In a margin-separated one-sided region, lower cut corresponds to higher correctness probability, so the integrand for the reliability gap is positive; the symmetric result holds for the opposite region.Theorem A.1 names this result Correction Dominance.Source: A Proofs · p. 14

A.1 Theoretical Justification under the Noisy Scenario

  • Noisy monotonicity allows sample-wise deviations from the ideal cut-margin relationship while bounding the reliability gap in margin-separated regions.The condition uses decay rate η and noise level ϵ.Source: §A.1 · p. 15
  • When γ > ϵ/η, true correction exceeds miscorrection by at least ηγ − ϵ in the positive-margin region.Theorem A.2 states the guarantee under noisy monotonicity.Source: §A.1 · p. 16

B Evaluation Datasets

The evaluated datasets span natural-object domain shifts, synthetic-to-real transfer, hospital histopathology, and retinal imaging.

B Evaluation Datasets

  • Office-Home contains about 15,500 images across 65 everyday-object categories and four visually distinct domains.The domains are Artistic, Clip Art, Product, and Real-World.Source: B Evaluation Datasets · p. 16
  • VisDA-C evaluates synthetic-to-real classification across 12 object categories, with 152,397 synthetic source images and 55,388 real target images.Its synthetic-to-real shift is described as substantial.Source: B Evaluation Datasets · p. 17
  • DomainNet-126 uses four domains and 126 classes from a curated subset of DomainNet, totaling roughly 145,000 images.The domains are Clipart, Painting, Real, and Sketch.Source: B Evaluation Datasets · p. 17
  • Camelyon17-WILDS evaluates binary tumor classification on 96 × 96 tissue patches across five hospital domains.The dataset contains 455,954 labeled examples.Source: B Evaluation Datasets · p. 17
  • EyePACS supplies the retinal-image source domain, while APTOS 2019 is the five-class target domain for diabetic-retinopathy severity classification.EyePACS images vary in appearance and quality due to diverse imaging conditions.Source: B Evaluation Datasets · p. 17

C Implementation Details

The implementation caches target features for epoch-level reliability estimation and uses smoothed, lagged reliability scores for gating. The extended evaluations report benchmark comparisons, computational costs, and empirical validation of cut-statistic properties.

C.1 Epoch-Indexed Formulation

  • Cached feature banks avoid a separate full target-domain forward pass for cut computation, and frozen CLIP features are computed once during warmup.The banks are populated during standard optimization forward passes.Source: §C.1 · p. 17
  • EMA smoothing uses low momentum early and high momentum later, while gating uses the previous epoch’s reliability scores to prevent circular dependence.The momentum schedule increases monotonically.Source: §C.1 · p. 18

D.1 Comparison Against Paper-Reported Best Results

  • SafeCut gains +0.6% on Office-31, +2.2% on Office-Home, and +0.8% on DomainNet-126 over ProDe-V; on VisDA-C it records 90.9% vs. 91.0%.It wins most per-task comparisons on the first three benchmarks and remains competitive on VisDA-C.Source: §D.1 · p. 18, Table 8 · p. 19, Table 9 · p. 19

D.2 VisDA Full Results

  • On VisDA-C, SafeCut averages 90.9%, exceeding ProDe-V by 0.3%p despite lower results on some individual categories, including truck (−7.6%p).It gains +5.0%p on car and +3.4%p on bus over ProDe-V.Source: §D.2 · p. 20, Table 12 · p. 20

D.3 Computational Analysis

  • Despite maintaining two models, SafeCut adapts faster than ProDe across all four benchmarks; cached-feature KNN costs remain below 0.5 s per epoch.Cached CLIP features avoid repeated backbone invocations during training.Source: §D.3 · p. 20, Table 13 · p. 20

D.4 Extended Empirical Validation of Cut Statistic Properties

  • Across Office-Home, DomainNet-126, and dual-ResNet settings, accuracy decreases approximately monotonically with each model’s cut statistic, even when peer accuracy gaps differ.CLIP peers retain a large accuracy advantage in the evaluated settings, whereas dual-ResNet curves are close and sometimes cross.Source: §D.4 · pp. 20–21, Figure 5 · p. 21, Figure 6 · p. 21
  • Complementary correction-region modes fall on opposite sides of Δ=0 across the evaluated settings, though their separation compresses when Art is the target domain.The ordering remains intact in the compressed regime.Source: §D.4 · pp. 21–22, Figure 8 · p. 23, Figure 9 · p. 23
  • Across four checkpoints on three Office-Home tasks, cut–accuracy monotonicity and the relative ordering of target and CLIP curves persist throughout adaptation.The target curve shifts upward and tightens toward the peer curve as training progresses.Source: §D.4 · p. 25, Figure 11 · p. 25

D.5 Extended Correctness Pattern Transition Results

Across settings, adaptation increases joint correctness and reduces asymmetric errors; the gains also vary with peer complementarity. The appendices describe uniform benchmark-level hyperparameter choices and a limitation of ProDe on Camelyon17.

D.5 Extended Correctness Pattern Transition Results

  • Across Office-Home and DomainNet-126 with CLIP peers and Office-Home with dual-ResNet peers, jointly correct samples increase and one-sided-correct sets nearly vanish.The jointly incorrect set also decreases modestly across most tasks.Source: §D.5 · pp. 26–27, Figure 14 · p. 27
  • In dual-ResNet settings, one-sided-correct sets often fall below 0.1% after adaptation, despite lower initial joint correctness proportions.The absolute gain in joint correctness is comparably large to the CLIP setting.Source: §D.5 · p. 27, Figure 14 · p. 27

D.6 Effect of Peer Complementarity

  • Pre-adaptation agreement is negatively related to SafeCut gain (Pearson r = −0.81), but even the highest-agreement pair (69.6%) improves by +10.4 percentage points.No performance degradation is observed across the 12 evaluated pairs.Source: §D.6 · p. 28, Table 14 · p. 28

E.1 Implementation and Hyperparameter Details

  • Hyperparameters use one configuration per dataset across its tasks, without per-task search or adjustment.The EMA schedule is constant across benchmarks; K and β vary slightly with dataset scale, while λdiv varies more substantially.Source: §E.1 · p. 29, Table 15 · p. 29

E.2 ProDe Hyperparameter Search on Camelyon17

  • On Camelyon17, ProDe’s best target-model result is 76.27%, while CLIP remains near its 50.00% zero-shot performance despite hyperparameter search.The authors attribute this ceiling to ProDe’s one-directional update structure, which cannot bootstrap a weak peer model.Source: §E.2 · pp. 29–30

E.3 Full Per-Task Ablation Results

The ablations favor SafeCut’s reliability-gated correction design, while variant comparisons highlight the role of adapting the peer model and the remaining gap to an oracle partition.

  • SafeCut consistently outperforms all ablated variants across three benchmarks, supporting complementary roles for reliability-aware cross-model correction and temporal anchoring.The supplied table caption identifies Table 17 as the full per-task component ablation results.Source: §E.3 · p. 30, Table 17 · p. 31
  • A frozen peer causes a consistent performance drop; its supervision becomes increasingly stale as the target model evolves.The design-variant comparison identifies joint peer adaptation as essential.Source: §E.3 · p. 30, Table 18 · p. 32
  • SafeCut’s 86.7% vs. 93.3% on Office-Home gap to Oracle S establishes an upper bound and leaves more precise oracle-partition approximations as future work.Source: §E.3 · p. 30
  • +13.4% on Office-31 (79.4% →92.8%), +7.7% on Office-Home (79.0% →86.7%), and +3.8% on DomainNet-126 (80.2% →84.0%) with SafeCut peer adaptation.Table 19 compares zero-shot CLIP with CLIP adapted as SafeCut’s peer model.Source: §E.3 · p. 31, Table 19 · p. 32

F Limitations and Future Directions

The discussion identifies cases that challenge mutual correction, outlines possible extensions, and describes broader deployment implications and asset-use terms.

F Limitations and Future Directions

  • Similar predictions provide fewer opportunities for cross-model correction, while samples where both models are wrong remain comparatively challenging.Low-margin samples could benefit from complementary reliability cues, and pseudo-label strategies could provide richer supervisory signals.Source: F Limitations and Future Directions · p. 33

G Broader Impacts

  • The authors identify privacy-sensitive and data-scarce deployments as potential beneficiaries, while advising human oversight and validation for high-stakes uses.The discussion notes that shared errors may go undetected in applications such as medical diagnosis.Source: G Broader Impacts · p. 33

H Asset Licenses and Terms of Use

  • The listed datasets are used under varied terms, including academic or non-commercial research restrictions; Camelyon17 is CC0 1.0.CLIP and CoOp are identified as MIT-licensed, while the torchvision ResNet implementations are BSD 3-Clause licensed.Source: H Asset Licenses and Terms of Use · p. 33
Loading 2610.02981v1…