PaperlayerView exact revision on arXiv

Source-linked AI summary

Hard, Yet Reducible: Controlled Forward Transfer for Synthetic Degradation Curation

Chunming He, Kailai Zhou, Jiaming Zuo, Hanqi Liu, Fengyang Xiao, Youwei Pang, Xiaofeng Liu, Weisi Lin, Xiaoqi Zhao

arXiv:2610.09849v1cs.CV

TL;DR

Selecting synthetic degradations for dense prediction is difficult because realism or difficulty does not establish their training utility under a finite budget. The paper uses matched clean-control probes to estimate that utility and curate data; its approach improves tested predictors across tasks while the useful severity range shifts with model state and budget.

  • Problem

    With finite training budgets, visual fidelity and difficulty are unreliable proxies for whether training on a synthetic degradation improves generalization.

    Source: §1 · pp. 1–2

  • Method

    cRDG compares held-out degraded-image gain from a degradation probe with gain from equally long clean-control training, subject to a separate clean-harm constraint.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    CuRB improves semantic-segmentation and salient-object-detection predictors under matched budgets while preserving clean performance, and the useful severity range shifts with model state and training budget.

    Source: §1 · p. 3, §6 · p. 11

  • Takeaways & Limitations

    The correctable severity band is not fixed: its location depends on the predictor, starting checkpoint, and training budget.

    Source: §1 · p. 3, §5 · p. 11

    cRDG is an empirical surrogate tested at three budgets on one backbone, and newly trained models are single runs without reported seed variance.

    Source: §5 · p. 11

Abstract

from arXiv · show

Selecting synthetic degradations for dense prediction requires an estimate of their training utility, the generalization gain they bring under a finite training budget. Clean and degraded twins share content and labels, suggesting a score based on how much short training reduces the excess error caused by degradation. However, this gap can also shrink when clean performance deteriorates. Measuring the improvement on degraded images alone avoids that confound, but it still credits progress that the same amount of clean training would have produced. We propose the \textbf{controlled Reducible Degradation Gap} (cRDG) for regions defined by degradation type and severity. From a common checkpoint, cRDG runs two budget-matched probes that differ only in one augmentation slot, which holds either a synthetic degradation or a clean augmentation. The score is the gain on held-out degraded images relative to the clean-control probe. Clean harm is a separate feasibility constraint. cRDG reveals a correctable severity band in which training on the degradation yields high controlled gain under the available budget, and the band moves with the predictor, the starting checkpoint, and the training budget. \textbf{Curation of Reducible Bands} (\method) uses cRDG to select synthetic data without changing the predictor. On semantic segmentation and salient object detection, \method{} improves representative predictors under matched synthetic-data budgets and training schedules, extends to existing data-generation pipelines, and preserves clean performance. Code and supporting materials will be publicly released.

1 Introduction

With a finite training budget, synthetic degradations should be selected by the extra generalization they yield beyond clean training, not by realism or difficulty alone. The paper introduces matched-control probes and CuRB to estimate and use that utility, which varies with severity, predictor, checkpoint, and budget.

1 Introduction

  • Visual fidelity and difficulty do not reliably indicate training utility; hard degradations may lack learnable signal, making loss- or uncertainty-based selection waste budget or cause negative transfer.The finite-budget alternative is a clean augmentation, so the relevant value is a degradation’s extra generalization relative to that control.Source: §1 · pp. 1–2
  • cRDG measures held-out degraded gain beyond equally long clean-control training, while a separate clean-harm constraint excludes damaging regions.Matched probes start from the same checkpoint and differ only in whether the designated augmentation slot contains a degradation or clean augmentation.Source: §1 · p. 2
  • Paired gap reduction can be inflated by clean forgetting, while degraded-loss improvement alone also credits progress equally long clean training would produce.The matched-control formulation measures degraded gain beyond that clean trajectory and checks clean harm separately.Source: §1 · p. 2, Figure 1 · p. 2
  • Each degradation type has a correctable severity band, and its centroid shifts with probe budget, starting checkpoint, and predictor setting.Per-region retraining at three final-training budgets shows the same shift in real-target gains.Source: §1 · pp. 2–3
  • CuRB ranks feasible positive-score regions to fill a fixed synthetic-data budget, adding degradation-aware curation to existing predictors without changing their architectures.It uses validity gating and k-center sampling for the final partial region; under matched protocols, it outperforms region-level adaptations of existing selection methods across semantic segmentation and salient object detection while preserving clean performance.Source: §1 · p. 3

2 Related Work

Prior work develops synthetic degradations, adapts predictors, or estimates sample value, but this paper focuses on choosing which degradations merit a finite training budget. CuRB adds region-level, twin-based, clean-control curation between existing generators and predictors.

Synthetic Degradation and Degradation-Aware Training

  • Synthetic-data and restoration approaches generate or use degraded images, whereas CuRB is a curation layer between an existing generator and predictor that changes neither.The evaluation uses ISSA, procedural weather corruption, and Gen4Seg under their original settings.Source: §2 · p. 3

Data Valuation and Learnability-Guided Selection

  • Valuation methods score samples or guide generation, while CuRB estimates utility at the region level using content-matched twins, clean-control probes, and held-out transfer.Diffusion Curriculum is closest in spirit, but does not compare against an equally long clean-control probe.Source: §2 · p. 3
  • CuRB ranks degradation-severity regions by matched-probe utility under a clean-harm constraint and fixed synthetic-data budget.The overview depicts generation and validity filtering, matched probes, region ranking, and final training from the common starting checkpoint.Source: Figure 2 · p. 4

3 Controlled Reducibility for Degradation Curation

CuRB estimates region-level degradation utility with matched probes, then selects feasible regions by cRDG under a fixed synthetic-data budget without changing the predictor. Its correctable severity band is a diagnostic that may shift with the predictor, checkpoint, and probe budget.

3.1 Candidate Regions and Validity

  • Four appearance degradations are divided into eight severity levels, forming 32 regions; a validity gate rejects candidates only when generation changes scene structure.Regions group candidates by degradation family and severity bin rather than by individual scenes.Source: §3.1 · p. 4

3.2 Matched-Control Probes

  • Gap shrinkage can reward clean forgetting, so cRDG compares degradation training with an equally long clean-control probe from the same checkpoint.The probes share source IDs, steps, batch composition, optimizer, schedule, data order, and initialization; the probe budget scales with the final schedule.Source: §3.2 · p. 5

3.3 Matched-Control Reducibility

  • cRDG measures held-out degraded-image gain over the matched control, while clean harm is a separate feasibility constraint on selection.The constraint bounds probe-time harm but does not guarantee final clean accuracy; cRDG is a short-horizon, source-side estimate.Source: §3.3 · pp. 5–6

3.4 Curation and Final Training

  • CuRB removes infeasible and non-positive regions, ranks the rest by cRDG, and fills the synthetic-data budget in descending order.Final training restarts from the common checkpoint on clean data plus selected synthetic data; changing the predictor or schedule requires recomputing cRDG, not changing the architecture.Source: §3.4 · p. 6

3.5 The Diagnostic Severity Band

  • The correctable band contains feasible positive bins scoring at least half the family maximum; its severity centroid is diagnostic, and the band-migration hypothesis predicts shifts with budget, checkpoint, predictor, and degradation type.Selection itself ranks regions directly by cRDG.Source: §3.5 · p. 6

4 Experiments

Under matched synthetic-data budgets, CuRB improves results across SOD benchmarks and existing data-generation settings. Ablations separate the value of controlled, held-out utility estimation from clean-retention feasibility, while analysis finds that useful severity bands shift with training conditions.

4.1 Setup

  • The evaluation uses official SOD splits and published recipes, plus Cityscapes-to-ACDC/Dark Zurich protocols with fog, night, rain, and backlight candidates.Snow is held out for unseen-family evaluation, and selectors share the candidate pool, synthetic budget, initialization, objective, and schedule.Source: §4.1 · pp. 6–7, Table 3 · p. 8

4.2 Main Results

  • At matched augmentation budgets, cRDG selection improves all four metrics for every base model on both SOD benchmarks, outperforming uncurated twins.The margin over uncurated twins remains positive even for CSSAM, where uncurated data provide little benefit.Source: Table 1 · p. 7, §4.2 · p. 8
  • +1.2 to +1.5 ACDC mIoU and +1.2 to +1.7 Dark Zurich mIoU over corresponding originals are reported when CuRB is added across generation settings.The settings span style synthesis, procedural weather corruption, diffusion-based editing, and a composite host; Cityscapes performance is maintained or slightly improved.Source: Table 2 · p. 7, §4.2 · p. 8
  • CuRB reaches 51.97 in-family mIoU versus 50.50 for the strongest control and 50.04 for the strongest literature baseline, with 51.38 on held-out snow.The shared recipe isolates selection policies; snow is excluded from the candidate pool, curation, and final training.Source: §4.2 · pp. 8–9, Table 3 · p. 8

4.3 Ablation Study

  • Removing clean-harm gating leaves ρglob at 0.79 but lowers Cityscapes accuracy from 70.13 to 67.18, showing that reducibility and clean feasibility differ.Removing clean-control subtraction lowers ρglob from 0.79 to 0.70; in SOD, the H-free variant lowers every clean DUTS-TE metric, including Sα from .921 to .908.Source: §4.3 · p. 9, Table 4 · p. 9, Table 5 · p. 9

4.4 Analysis: Training Utility and the Correctable Band

  • Against held-out real retraining utility, cRDG improves ρglob from 0.70 to 0.79 and in-family mIoU from 51.02 to 51.97 over the held-out loss drop.Its ρDZ also rises from 0.62 to 0.72.Source: §4.4 · p. 10, Table 6 · p. 10
  • Empirical severity centroids shift with training budget, while scaled-K cRDG predictions remain within 0.03 of empirical centroids at every tested budget.From 0.5× to 2× final budget, centroids shift +0.20 for fog and +0.19 for low light; probe-budget, checkpoint, and predictor settings also move centroids.Source: §4.4 · p. 10, Figure 4 · p. 10

5 Discussion and Limitations

cRDG separates clean forgetting and clean-training progress from degradation-specific gain, and its correctable band shifts with the learner and budget. The authors note empirical and procedural limits to the evidence.

5 Discussion and Limitations

  • cRDG best tracks signed retraining utility by subtracting progress obtainable from matched clean training, while paired gap reduction is affected by clean forgetting.The correctable band also depends on the learner and budget, not severity alone.Source: §5 · p. 11
  • The procedure transfers unchanged across tested predictors, but scores are recomputed for each model.Source: §5 · p. 11
  • cRDG is an empirical surrogate tested at three budgets on one backbone, and the probe horizon follows a fixed budget-scaling rule.Other stated limits include possible mild label noise near the validity threshold, cross-family-only benefit for unseen families, external clean images for the SOD twin pool, and single-run models without seed variance.Source: §5 · p. 11

6 Conclusion

The paper studies synthetic-degradation curation under a finite training budget and uses cRDG to select training data. It reports improvements across tested predictors and generation settings, while useful severity ranges vary with model state and budget.

6 Conclusion

  • cRDG compares degradation training with an equally long clean-control probe on held-out twins and treats clean harm separately.Source: §6 · p. 11
  • Using cRDG, CuRB selects a fixed-budget training set and improves salient object detection and semantic segmentation across tested predictors and generation settings.Source: §6 · p. 11
  • The useful severity range changes with model state and training budget rather than remaining fixed.Source: §6 · p. 11

A Implementation Details

Implementation keeps host training schedules and optimizers intact, generates and filters structure-preserving degradation twins, and ranks feasible positive-score regions to fill a fixed sample budget.

A Implementation Details

  • CuRB leaves each host's optimizer, resolution, augmentation, and final-training schedule unchanged; probes use matched slot ratios and a 0.25× base learning rate.For Cityscapes, the selection budget is 5,950 samples over 32 regions; other plug-in experiments retain each host's synthetic budget.Source: A Implementation Details · p. 14

B Generator and Validity Gate

  • Generation uses image-to-image diffusion with depth- and Canny-conditioned guidance, and the validity gate checks structural correspondence and boundary displacement.The gate accepts 90–96% of candidates at severe levels in the manual audit, so it does not simply remove stronger degradations.Source: B Generator and Validity Gate · p. 14, Table S1 · p. 15
  • The validity gate uses thresholds calibrated on clean source pairs, with additional object-presence checks for salient object detection.A uniformly darker or foggier image can pass when its geometry remains intact.Source: B Generator and Validity Gate · p. 14
  • CuRB splits source IDs, generates gated degradation regions, then ranks feasible positive-cRDG regions and fills the sample budget in descending order.Source: B Generator and Validity Gate · p. 15

C Probe Protocol and Estimator Details

The probe protocol compares treatment and clean-control branches from a common checkpoint under matched training conditions, evaluates held-out transfer, and recomputes cRDG for each host's region definitions.

C Probe Protocol and Estimator Details

  • Treatment and control probes share the checkpoint, budget, batch composition, clean-replay ratio, optimizer, schedule, views, and data order; evaluation uses the held-out half.Operational curation uses one fixed source split and seed; alternate partitions and probe horizons are reserved for stability and sensitivity analyses.Source: C Probe Protocol and Estimator Details · p. 15
  • Treatment views use generated twins while controls use weak photometric augmentation; failed twins are resampled, with control views retained if no valid training twin is found.Invalid evaluation twins are dropped from both branches.Source: C Probe Protocol and Estimator Details · p. 15
  • cRDG is reported signed to preserve negative transfer, with paired bootstrap statistics and partition-based standard deviations used for uncertainty reporting.Source: C Probe Protocol and Estimator Details · pp. 15–16
  • Degraded-domain performance is stable across clean-retention tolerances of 0.5–2 standard errors, while a looser tolerance permits a small clean-accuracy drop.Source: C Probe Protocol and Estimator Details · p. 16, Table S2 · p. 16

D Plug-and-Play Framework Comparisons

  • Plug-in comparisons follow each host's published pipeline and synthetic-data budget, recompute cRDG per host, and define regions using that host's degradation categories and levels.Examples include style-exemplar groups for ISSA and corruption types with levels 1–5 for Hendrycks-Weather.Source: D Plug-and-Play Framework Comparisons · p. 16

E SOD Benchmark Protocols

The salient object detection benchmarks use official splits and metrics, with each base model following its published training recipe. Augmented variants add synthetic twins under matched data budgets and final-training schedules.

E SOD Benchmark Protocols

  • WXSOD [26] and CSOD10K use their official splits and metrics, and each base model follows its published training recipe.The augmented variants add twins synthesized from clean DUTS-TR using four degradation families and eight severity levels, subject to Appendix B’s validity gate.Source: E SOD Benchmark Protocols · p. 16
  • Uncurated and CuRB variants use the same synthetic-data budget and base-model final-training schedule, differing in uniform sampling versus cRDG selection.Selection budgets are 4,000 on WXSOD and 3,750 on CSOD10K.Source: E SOD Benchmark Protocols · p. 16
  • Synthetic twins are generated from clean DUTS-TR across four degradation families and eight severity levels, with Appendix B’s validity gate applied.Source: E SOD Benchmark Protocols · p. 16

F Compute and Cost Accounting

The reported SegFormer configuration uses 66 probe updates, costing approximately 0.59 times one final-training run under the stated scaling rule; several selection costs are excluded.

F Compute and Cost Accounting

G Baseline Adaptation

Baseline comparisons hold the gated regions, selection budget, within-region k-center step, and final-training recipe constant. The baselines differ in their scoring or region-ranking procedures.

G Baseline Adaptation

  • All selection baselines rank the same 32 gated regions with the same selection budget and within-region k-center step, and use identical final-training recipes.Candidate-level scores are averaged within each region before ranking.Source: G Baseline Adaptation · p. 16
  • RHO-Loss-style selection scores a candidate’s reducible loss relative to a small model trained on held-out source data, while gradient contribution uses alignment with the mean source-validation gradient.Source: G Baseline Adaptation · pp. 16–17
  • Operational selection uses one A|B partition; the region-independent control adds two probes, with normalized probe-update cost of approximately 0.59 of one final-training run.The estimate is (66/80) × 0.71 ≈ 0.59 under the K = T/80 scaling rule.Source: Table S3 · p. 17
  • The DisCL-style baseline derives a severity preference per family from the current model’s source-validation response, then ranks regions once before final training.It does not alter the training schedule.Source: G Baseline Adaptation · p. 17
Loading 2610.09849v1…