PaperlayerView exact revision on arXiv

Source-linked AI summary

Supervision-Adaptive Curriculum and Guarded Anti-Curriculum Learning for Dense Prediction

Chunming He, Dingming Zhang, Fengyang Xiao, Zhiwen Cao, Sina Farsiu

arXiv:2610.29519v1cs.CVcs.LG

TL;DR

Dense-prediction curricula can mistake difficulty for supervision reliability, even though easy samples may expose shortcuts and hard samples may be informative or ambiguous. PaceShift combines supervision-adaptive curriculum selection with guarded spectral post-training. Across the evaluated settings, it improves accuracy, with targeted evaluations also reporting better pooled calibration and corruption robustness.

  • Problem

    Difficulty alone does not establish supervision reliability in dense prediction, where easy samples can expose shortcuts and hard samples can be informative or ambiguous.

    Source: §I · p. 1, 1) Easy samples amplify a… · p. 3

  • Method

    PaceShift combines a supervision-adaptive curriculum with guarded multi-radius spectral post-training, using preference learning to refine predictions against a Phase-I reference.

    Source: §I · p. 2

  • Results

    Across CECS, incomplete supervision, foundation-model adaptation, and broader dense prediction, PaceShift improves accuracy; targeted evaluations report improved pooled calibration and corruption robustness in evaluated settings.

    Source: §VI · p. 16

  • Takeaways & Limitations

    The framework supports full, weak, semi-supervised, and model-derived supervision while preserving the deployed architecture and adding no inference computation.

    Source: §I · p. 2

    Spectral attenuation may remove useful target evidence when objects occupy only a few pixels, and broader dense-prediction experiments do not isolate each phase’s contribution.

    Source: B. Scope and Limitations · p. 16

Abstract

from arXiv · show

Curriculum learning orders samples by difficulty, but difficulty alone does not establish supervision reliability for dense prediction. Easy samples can expose shortcuts, hard samples can be informative, and noisy or ambiguous samples can mislead training. Building on our conference method CurriSeg, we present PaceShift, a stabilize-then-challenge paradigm combining supervision-adaptive curriculum selection with guarded spectral post-training. A learning-dynamics view motivates separating supervision reliability from temporal update value and applying spectral challenges after stabilization. The first phase estimates reliability from available supervision, augmentation consistency, model disagreement, and prediction certainty. Temporal statistics guide sample weighting, while pixel-uncertainty weights reduce the early influence of uncertain regions. The second phase ranks predictions from multiple spectral views using supervision agreement, robustness, reference consistency, and penalties. A pairwise preference objective measured against the frozen Phase-I reference, together with an anchor, refines the prediction. The framework supports full, weak, semi-supervised, and model-derived supervision. Experiments across context-entangled content segmentation (CECS), broader dense-prediction tasks, and foundation-model and cross-modal adaptations show accuracy gains. Targeted diagnostics indicate improved pooled calibration and corruption robustness in the evaluated settings. PaceShift preserves the deployed architecture and adds no inference computation. Code will be publicly released.

I. INTRODUCTION

Dense prediction is shaped by supervision reliability and training order, not only model capacity. PaceShift extends CurriSeg with supervision-adaptive selection and guarded spectral post-training, then evaluates the framework across dense-prediction settings.

I. INTRODUCTION

  • In CECS, objects share visual patterns with backgrounds, making supervision reliability especially consequential in camouflaged, transparent, and concealed-defect segmentation.Polyp segmentation is included as a related low-boundary-contrast, ambiguous task.Source: §I · p. 1
  • Easy-first curricula can prioritize residual texture cues, while high error may indicate either informative difficulty or unreliable supervision.Confidence also does not guarantee correctness, and uncertain pixels can dominate early gradients.Source: §I · pp. 1–2
  • PaceShift replaces dense-mask difficulty with supervision-adaptive selection and adds guarded multi-radius post-training while retaining CurriSeg’s two-phase schedule.The framework is intended to support full, weak, semi-supervised, and model-derived supervision without changing the deployed architecture.Source: §I · p. 2
  • The journal extension broadens evaluation to clinical, dichotomous, salient, vision-language, and reference-guided tasks and adds paired-run tests and targeted mechanism diagnostics.Analyses cover supervision reliability, temporal update value, guard selection, phase ordering, frequency sensitivity, robustness, and calibration.Source: §I · p. 2

II. RELATED WORK

  • Prior dense-prediction work largely emphasizes representation, while curriculum methods often assume easy samples are reliable and difficulty tracks learning utility.PaceShift instead separates reliability from temporal update value and extends weighting to pixels before a post-stabilization challenge.Source: §II · pp. 2–3

A. Problem Formulation and Overview

  • PaceShift trains in two phases: Phase I selects reliable yet informative samples and damps uncertain pixels; Phase II challenges predictions spectrally and refines them with a guarded preference objective.Both phases use supervision evidence and source trust; the deployed model is unchanged.Source: A. Problem Formulation and Overview · p. 3

B. A Learning-Dynamics View

PaceShift separates supervision reliability from temporal learning value and delays spectral challenge until a stable representation exists. Its curriculum combines evidence-conditioned reliability cues with temporal sample weighting and pixel uncertainty control.

1) Easy samples amplify a high-frequency shortcut:

  • In CECS, easy samples can reveal high-frequency texture shortcuts, so the spectral bottleneck is applied only after a stable representation forms.The relevant frequency is in the image, unlike function-space spectral bias.Source: 1) Easy samples amplify a… · pp. 3–4

2) Temporal dynamics reveal update value:

  • Temporal difficulty patterns distinguish potentially informative variation from persistent failure, motivating a mid-variance preference and down-weighting of high-mean, low-variance samples.The authors treat these patterns as heuristics and separately ablate their terms.Source: 2) Temporal dynamics reveal update value · p. 4

3) SBFT is a post-stabilization frequency bottleneck:

  • SBFT attenuates the high-frequency image component after stabilization; applying it directly after warm-up falls below standard training on every reported metric.The challenge is intended to retain target information without relying on nuisance frequencies.Source: 3) SBFT is a post-stabilization… · p. 4

C. Supervision-Adaptive Robust Curriculum Selection

  • SA-RCS combines warm-up selection, supervision-adaptive difficulty, temporal sample weighting, and pixel uncertainty weighting without requiring complete masks.It targets both sample-level noise-ambiguity and spatial uncertainty.Source: C. Supervision-Adaptive Robust… · p. 4

1) Supervision-adaptive difficulty:

  • The reliability score combines supervision agreement, augmentation consistency, model agreement, and prediction certainty, with difficulty defined as one minus reliability.The cues are oriented so larger values indicate greater reliability; supervision evidence varies with the available regime.Source: 1) Supervision-adaptive difficulty · pp. 4–5
  • A source-trust prior sets the supervision cue’s weight, while a fixed template allocates the remaining weight to auxiliary cues; this construction is intended to reduce self-training confirmation bias.The paper describes reliability as evidence-conditioned support for learning, not annotation correctness alone.Source: 1) Supervision-adaptive difficulty · p. 5

2) Warm-up curriculum:

The curriculum begins with all samples, then progressively admits lower-reliability samples according to a percentile schedule.

2) Warm-up curriculum:

  • For the first 10 epochs, all samples are used; afterward, selection expands from high-reliability samples as the percentile rises.The stated minimum percentile is 0.6.Source: 2) Warm-up curriculum · p. 5

3) Temporal statistics-based sample weighting:

Temporal statistics weight samples by their learning dynamics, while pixel uncertainty weighting reduces uncertain pixels’ early influence without masking them from training.

3) Temporal statistics-based sample weighting:

  • Intermediate temporal variation is treated as potentially informative hardness, while high-mean, low-variance samples are down-weighted as persistent failures.The parameter-free variance weight peaks at the midpoint and suppresses both stable and highly unstable samples; a nonzero floor maintains gradient flow.Source: 3) Temporal statistics-based sample… · p. 5

4) Pixel uncertainty weighting (PUE):

  • Pixel uncertainty weighting reduces the early loss contribution of high-entropy pixels, then returns weights to one by the end of Phase I.Pixels remain in training, and a shared nonzero floor preserves gradient flow.Source: 4) Pixel uncertainty weighting (PUE) · p. 6

5) Unified curriculum loss:

The unified curriculum loss applies supervision-appropriate targets and losses across supervision regimes. The framework also extends to multi-class, instance, and conditioned segmentation by adapting the reliability cues to each output or conditioning source.

5) Unified curriculum loss:

  • The unified loss applies the regime-appropriate weighted segmentation loss to a supervision-adaptive target, with unsupervised pixels contributing zero.Dense masks, weak labels, and unlabeled samples use corresponding loss forms; unlabeled predictions still contribute consistency and entropy cues.Source: 5) Unified curriculum loss · p. 6

6) Beyond binary masks:

  • Multi-class and instance segmentation reuse the framework with normalized entropy, class- or mask-level agreement, and output-specific collapse penalties.The trust and loss coefficients are shared, and the curriculum equations operate on a scalar difficulty value rather than output format.Source: 6) Beyond binary masks · p. 6

7) Cross-modal conditioning:

  • Language conditioning uses paraphrase consistency and vision-text disagreement, while reference conditioning uses reference-swap consistency and vision-reference disagreement.Conditioned predictions provide reliability evidence rather than fixed labels; trust remains tied to the primary supervision source.Source: 7) Cross-modal conditioning · p. 6

D. Guarded Anti-Curriculum Promotion

After SA-RCS stabilizes, PaceShift challenges predictions with multiple spectral views and a supervision-adaptive guard. It then uses guarded preferences and a frozen Phase-I reference to refine predictions while limiting drift.

D. Guarded Anti-Curriculum Promotion

  • After stabilization, SBFT-P replaces single-filter fine-tuning with multi-strength spectral challenges, guarded candidate scoring, and a guarded preference objective.The guard prevents the post-training stage from reducing to confidence maximization.Source: D. Guarded Anti-Curriculum Promotion · p. 6

1) Multi-radius spectral challenge:

  • Radii 0.90, 0.95, and 1.00 progressively remove high-frequency texture, and candidate-mask changes across views provide a robustness signal.The identity view is the clean prediction; thresholded masks enter overlap and area checks, while soft probabilities enter entropy and preference terms.Source: 1) Multi-radius spectral challenge · p. 6

2) Supervision-adaptive guard:

  • The guard scores candidates using supervision-based correctness, cross-radius robustness, frozen-reference consistency, degeneration penalties, and entropy as an uncertainty term.As supervision becomes less verifiable, positive weights shift toward robustness and consistency; confidence is only a weak penalty, and selection does not inflate confidence in the reported setting.Source: 2) Supervision-adaptive guard · pp. 6–7

3) Guarded preference objective:

  • The post-training objective combines SBFT, pairwise preference, and an anchor; the clean-view preference is measured against the frozen Phase-I reference to limit drift.Without dense masks, the guarded preferred mask supplies the target; under full supervision, single-radius SBFT alone recovers the conference objective.Source: 3) Guarded preference objective · p. 7

E. Overall Training Pipeline

PaceShift preserves the deployed architecture and adds no inference computation. Its additional computation and memory are confined to training, with fixed shared constants and no per-dataset tuning.

E. Overall Training Pipeline

  • PaceShift leaves the architecture unchanged and adds no inference cost, while incurring modest overhead during training.The overhead comes from periodic Phase-I predictions and Phase-II spectral-candidate and frozen-reference evaluations.Source: E. Overall Training Pipeline · p. 7
  • Phase-I active-subset reduction offsets part of the added training computation, while Phase-II spectral views are processed sequentially with the frozen reference resident.Temporal history stores one scalar per sample per refresh checkpoint.Source: E. Overall Training Pipeline · p. 7
  • Global constants are reused across tasks, backbones, and regimes, and the procedure uses no per-dataset tuning.The mid-variance map is parameter-free, preference and temperature scales are fixed to one, and the stabilization trigger can be adaptive.Source: E. Overall Training Pipeline · p. 7

IV. EXPERIMENTS

The experiments evaluate PaceShift across CECS and broader dense-prediction tasks under controlled comparisons. On COD, PaceShift improves or matches every detector on every dataset, with reported gains also extending across the other CECS tasks.

A. Experimental Setup

  • The evaluation spans CECS, polyp, and broader dense-prediction tasks, with task-specific metrics and pooled ECE plus complementary confidence–accuracy diagnostics.Comparisons share baseline augmentations; cross-task experiments preserve each host's resolution, optimizer, and training budget, scaling phase boundaries proportionally.Source: A. Experimental Setup · pp. 7–8

B. Comparative Evaluation on CECS

  • 3.3%–5.3% mean relative gains: PaceShift improves or matches every detector on every COD dataset.Table II compares baseline, CurriSeg, and PaceShift variants across detectors and backbones.Source: Table II · p. 8, B. Comparative Evaluation on CECS · p. 8
  • On PIS and TOD, PaceShift improves task-specific detectors and RUN and exceeds CurriSeg; zero-shot CDD gains are roughly 7.7% to 7.9%.CDD transfers a COD-trained model to CDS2K without adaptation.Source: B. Comparative Evaluation on CECS · pp. 8–9
  • Fig. 4 shows better recovery of concealed structures and boundaries, with fewer false positives in texture-rich backgrounds.The visual comparison covers COD, PIS, TOD, and CDD using RUN, CurriSeg-trained, and PaceShift-trained variants.Source: Figure 4 · p. 9

C. Generalization across Architectures

PaceShift improves across architectures and broader dense-prediction families, while its supervision-adaptive online strategy outperforms prior data-centric scores and its coreset diagnostic performs best at intermediate retention.

C. Generalization across Architectures

  • DINO V2 and the frozen SAM adapter gain 4.3% and 4.7%, respectively, without changes to their adapters or frozen weights.UGTR and FSEL also benefit despite modeling uncertainty and frequency, respectively.Source: C. Generalization across Architectures · p. 9
  • Across thirteen detector configurations, PaceShift improves every row by 3.8–6.5% and exceeds CurriSeg throughout.The comparison changes the training recipe while retaining each baseline's architecture, optimizer, and augmentation.Source: C. Generalization across Architectures · p. 9, Table VIII · p. 10

E. Comparison with Data-Centric Training and Selection

  • 40–80% retention gives the Phase-I-weight diagnostic its largest Fβ gain over random selection; it is competitive at 20%, while PaceShift uses the full set.The diagnostic retains top fractions and retrains from scratch; PaceShift targets accuracy, robustness, and calibration rather than subset reduction.Source: Figure 5 · p. 10, E. Comparison with Data-Centric… · p. 11

D. Generalization to Broader Dense Prediction Families

  • Gains extend from ambiguous low-contrast detection and clinical prediction to standard segmentation, under native budgets and without task-specific constants.On RETOUCH, PaceShift recovers CurriSeg's marginal loss below baseline and further improves performance.Source: D. Generalization to Broader Dense… · p. 10, Table IX · p. 10

E. Comparison with Data-Centric Training and Selection

F. Learning under Incomplete Supervision

PaceShift adapts reliability cues to weak and semi-supervised settings, foundation-model adapters, and vision-language or reference-guided segmentation. It improves the evaluated baselines, while pseudo-label triage shows that combined reliability cues outperform individual signals.

F. Learning under Incomplete Supervision

  • In the 1/16 semi-supervised regime, PaceShift improves both segmenters on every metric despite severely limited labeled supervision, and its mean relative gain more than doubles CurriSeg’s conference gain.Source: F. Learning under Incomplete Supervision · p. 11
  • For corrupted pseudo-label detection, ρi beats confidence, entropy, small-loss, and Dyn-Unc on detection and retained quality; it also exceeds direct SAM3 agreement on all four measures.The 20% corruption diagnostic leaves the external SAM3 proposal uncorrupted; ρi without that proposal also beats every nonexternal baseline.Source: Table XIII · p. 11, F. Learning under Incomplete Supervision · p. 12

G. Foundation-Model Adaptation

  • PaceShift improves SAM-, SAM2-, and SAM3-Adapters and consistently exceeds CurriSeg under the canonical full-supervision COD protocol.The frozen foundation model is a teacher, but its mask is not treated as ground truth.Source: Table XIV · p. 11, G. Foundation-Model Adaptation · p. 12

H. Results on VLM- and Reference-Guided Segmentation

  • PaceShift improves both CLIP-based weakly supervised models on every evaluated split, both open-vocabulary camouflaged segmenters, and both reference-guided models on every metric.The VLM-based weak-supervision setting uses CAM agreement and vision-language cues; reference-target agreement is computed in an independent feature space.Source: H. Results on VLM- and… · p. 12, Table XV · p. 12, Table XVI · p. 12

I. Ablation Study and Hyperparameter Sensitivity

On COD10K, the ablations support complementary Phase-I mechanisms, reliability-based ranking, strict stabilize-then-challenge ordering, and source-specific trust; guarded pairwise post-training improves corruption and calibration measures.

Curriculum components and complementarity

  • Selection is the strongest standalone Phase-I mechanism; temporal weighting adds a smaller consistent gain, and pixel weighting improves all four metrics when combined with the other mechanisms.The full factorial isolates these effects without post-training.Source: §I · p. 12, Table XVIII · p. 13

Curriculum ordering

  • Reliability-based rankings improve all four metrics, while online tracking retains a modest margin over frozen reliability ranking and random ordering stays near no selection.Loss- and confidence-based rankings give inconsistent and limited gains under the matched budget.Source: §I · p. 13, Table XIX · p. 13

Guarded anti-curriculum

  • 0.0058 ± 0.0018 higher Sαc, 0.0010 ± 0.0003 lower ECE, and 0.0027 ± 0.0009 lower GT-CAG with the reference-relative pairwise objective across five runs.Compared with guarded distillation, it improved all three reported measures in every run.Source: §I · p. 13, Table XXI · p. 14

Phase ordering

  • Challenge-first schedules fall substantially below standard training, while strict two-phase ordering performs better than alternating losses, especially on Sαc and GT-CAG.An adaptive validation-plateau trigger performs on par with the fixed transition epoch.Source: §I · p. 13, Table XXII · p. 14

Trust parameterization

  • Source-specific trust outperforms uniform cue weights, a shared trust value, and mismatched presets across semi-supervised and point-supervision settings.Ten order-preserving ±20% perturbations stay close to default, and validation tuning improves Sα by only 0.001.Source: §I · p. 13, Table XXIII · p. 14

Hyperparameter sensitivity

  • Joint perturbations of five non-transition constants keep mean Sα within 0.002 at ±20% and produce no abrupt collapse at ±40%.The largest reported sensitivity is in GT-CAG when either ε or λanc is set to zero.Source: §I · p. 14

Training dynamics

  • After the challenge begins, PaceShift drops less than CurriSeg, recovers earlier, and ends higher; ten extra epochs change every curve by at most 0.001.The curves are means over five runs with one-standard-deviation bands.Source: §I · p. 14, Figure 6 · p. 14

J. Robustness and Calibration

Across the evaluated corruption suite, PaceShift retains more performance than CurriSeg as severity increases. On COD10K, it also lowers pixel-pooled ECE and improves the reported confidence diagnostics, though post-hoc temperature scaling reaches slightly lower ECE and GT-CAG.

Robustness under diverse corruptions

  • PaceShift retains more than CurriSeg at every nonzero corruption severity, with the margin widening as corruption strengthens.Retention is each recipe’s own clean Sα ratio, averaged within four corruption families and then equally across families.Source: J. Robustness and Calibration · p. 15, Figure 7 · p. 15

Calibration

  • On COD10K, PaceShift reduces pixel-pooled ECE from 0.019 to 0.011, below CurriSeg at 0.016, and improves all three confidence diagnostics.Post-hoc temperature scaling reaches slightly lower ECE and GT-CAG but requires a separate labeled calibration split.Source: J. Robustness and Calibration · p. 15, Table XXIV · p. 15

K. Mechanism Analysis

PaceShift’s diagnostics examine supervision reliability, update value, candidate-mask selection, and frequency reliance. They report improved accuracy across the evaluated settings and identify scope boundaries for spectral intervention and reliability-aware scheduling.

K. Mechanism Analysis

  • The mechanism analysis tests whether ρ_i tracks supervision quality, ω_i reflects update value, Q(m) ranks candidates, and spectral post-training reduces high-frequency reliance.Source: K. Mechanism Analysis · p. 15

1) Does reliability track supervision quality:

  • In pseudo-label settings, ρ_i ranks supervision quality: quality declines across its five quintiles, with a 32.2-point top-to-bottom IoU gap and 0.64 sample-level Spearman correlation with true IoU.Source: 1) Does reliability track supervision… · p. 15

2) Does the temporal weight align with useful updates:

A. Why the Paradigm Works

  • Across diagnostics, the proposed order outperforms reversed and challenge-first schedules, and the guard selects low-regret candidates that confidence alone would not.Source: A. Why the Paradigm Works · p. 16

B. Scope and Limitations

  • Spectral intervention is most useful when targets and backgrounds share low-level statistics; it may remove useful target evidence when objects occupy only a few pixels.Source: B. Scope and Limitations · p. 16

VI. CONCLUSION

  • PaceShift improves accuracy across CECS, incomplete-supervision, foundation-model, and broader dense-prediction settings, with targeted evaluations showing better pooled calibration and corruption robustness.Source: §VI · p. 16

Related papers

Loading 2610.29519v1…