TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models

1 Introduction

• Few-shot noisy-label methods often discard or downweight data, while aggressive correction risks changing clean labels and worsening confirmation bias.The introduction argues that conservative, high-precision correction is needed to use limited supervision while managing mis-correction risk.Source: §1 · p. 2
• Under 50% random noise, noisy samples often had long stable OT runs matching ground truth, while clean samples rarely had long stable incorrect runs.The authors use this contrast to motivate temporal consistency as a criterion for deciding when to correct labels.Source: §1 · p. 2, Figure 1 · p. 2
• A sufficient margin under bounded epoch-wise perturbations can preserve the OT candidate argmax, and OT label flips connect to forgetting-style dynamics.This analysis supports a K-epoch stability gate and suggests choosing K to improve correction precision.Source: §1 · pp. 2–3
• TecoPrompt combines globally consistent OT label candidates with K-epoch stability checks and EMA confidence gating before hard rewriting.Corrected labels return to prompt optimization through a tri-group objective, with cross-entropy for clean and rectified samples and noise-robust losses for uncertain samples.Source: §1 · p. 3
• 66.6% vs. 55.3% on Flowers102 under 75% asymmetric noise is the reported example of TecoPrompt’s gains over prior approaches.The paper also reports substantial gains across diverse datasets and noise settings.Source: §1 · p. 3

Prompt Learning for Vision-Language Models

• Prompt learning ranges from continuous text contexts and cross-modal prompts to domain adaptation, test-time tuning, and annotation-light approaches.The section names CoOp and CoCoOp, visual and joint prompting, TPT and C-TPT, and other directions.Source: Prompt Learning for Vision-Language… · p. 3

Learning with Noisy Labels

• Noisy-label defenses include robust training and regularization, noise-robust losses, objective correction, and data selection or refinement.The section frames these as responses to memorization and biased optimization under noisy supervision.Source: Learning with Noisy Labels · p. 3
• Earlier prompt-learning approaches purify or partition noisy data, whereas TecoPrompt checks OT-label stability over K epochs and uses a tri-group loss.The section contrasts TecoPrompt with JoAPR and NLPrompt’s PromptOT approach.Source: Learning with Noisy Labels · p. 4

Optimal Transport for Global Consistency

• Unlike OT prompt methods focused on purification, assignment, or loss reweighting, TecoPrompt uses global OT candidates plus sample-wise temporal consistency to trigger hard rewriting.The section situates this design alongside prior OT methods and OT in prompt learning.Source: Learning with Noisy Labels · p. 4

3 Formulation of Temporal-Consistency OT Label Rewriting

• The theoretical analysis characterizes when a temporally stable OT assignment can serve as a reliable candidate and argues that stronger consistency requirements yield a more conservative, reliable rewrite set.It builds on temporal consistency in semi-supervised and teacher–student learning.Source: §3 · p. 4

3.1 Notation.

• The notation distinguishes each sample’s latent clean label from its observed, possibly noisy label, and denotes its epoch-specific OT pseudo-label.These quantities support the section’s theoretical clean/noisy analysis and trajectory definitions.Source: §3.1 · p. 4
• A rewrite candidate must remain unchanged across the K-epoch window, disagree with the observed label, and maintain confidence above threshold θ throughout that window.The candidate event combines temporal consistency and reliability requirements.Source: §3.1 · p. 5

3.2 OT-Forgettability and a 2 × 2 Decomposition.

• The OT flip rate φ_i measures trajectory instability: smaller values indicate more stable assignments, and exceeding threshold φ_0 marks a sample OT-forgettable.The indicator is observable from the OT trajectory, unlike classic forgetting defined by correctness against the latent clean label.Source: §3.2 · p. 5
• The clean/noisy-by-forgettable/unforgettable partition identifies noisy OT-unforgettable samples as primary rewrite targets, while requiring stable disagreement and high confidence on clean data.These samples have incorrect observed labels but consistent OT pseudo-labels across training.Source: §3.2 · p. 5, Rewriting objective · p. 5
• The framework combines global OT candidates, confidence-weighted label blending and grouping, tri-group training, and a parallel temporal gate for hard rewriting.The overview describes confidence scores as EMA-smoothed and the stability gate as K-epoch.Source: Figure 2 · p. 6

3.3 Temporal-Consistency Rewriting.

• A sample is rewritten only when its OT label is temporally consistent, disagrees with the observed label, and stays high-confidence over the history window.This rule makes multi-epoch stability and confidence prerequisites for correction.Source: §3.3 · p. 6
• Under the theorem’s assumptions, increasing K improves rewritten-label reliability, which can become arbitrarily high for sufficiently large K.Longer consistent OT trajectories progressively exclude unstable samples from correction.Source: §3.3 · p. 6

4 Method

• TecoPrompt combines global OT-based candidate generation with temporal verification and ternary grouping for loss-specific training.Source: §4 · p. 6

4.1 Text & Image Encoder.

• Frozen image and text encoders provide image and class-text features; learnable context prompts adapt text, and class scores use feature inner products.Source: §4.1 · p. 7

4.2 OT-based Purification

• Entropically regularized OT matches image features to class-text prototypes, producing balanced soft pseudo-labels used for selection, confidence estimation, and refinement.Source: §4.2 · p. 7

4.3 EMA-Smoothed OT Confidence

• EMA smooths each sample's OT confidence, while a short history of labels and confidences supports temporal consistency checks.Source: §4.3 · p. 8

4.4 Temporal Consistency Label Rewriting

• Hard rewriting requires a stable, sufficiently confident OT candidate; a K-epoch warmup collects assignments before rewriting begins at t ≥ K.Source: §4.4 · p. 8, warmup · p. 8

4.5 Ternary Grouping and Training Objectives

• Samples are grouped by OT-label agreement and EMA confidence; CE trains clean samples, GCE handles the mid group, and an MAE-style loss addresses noisy samples.Source: Loss functions · p. 9, Clean set · p. 9, Mid set · p. 9

5.1 Benchmarks and Baselines; 5.2 Implementation Details

• Experiments use seven synthetic-noise datasets plus Food101N, compare four baselines, and follow matched 16-shot splits with frozen CLIP encoders.Synthetic noise is applied to training labels; official test splits remain unchanged.Source: §5.1 · p. 10, §5.2 · p. 10

5.3 Results on Synthetic Noisy Labels

• Across most datasets and noise levels, TecoPrompt performs strongly, with a more pronounced performance gap under severe label corruption.Table 1 covers synthetic symmetric and asymmetric label noise across seven datasets.Source: §5.3 · p. 10, Table 1 · p. 11

5.4 Results on real-world Noisy Labels

• TecoPrompt achieves the highest accuracy among compared approaches on Food101N, a real-world dataset with naturally noisy labels.The reported result indicates that its robustness transfers to naturally corrupted supervision.Source: §5.4 · p. 11, Table 2 · p. 12

5.5 Few-shot Learning Analysis

• Accuracy rises with more shots for all methods, while TecoPrompt leads across four datasets, especially at very low shot counts.The analysis varies shots per class from 1 to 16 while holding noise at 50%.Source: §5.5 · pp. 11–12, Figure 4 · p. 12

5.6 Comparison of Supervision Strategies

• TecoPrompt performs best across Flowers102 noise settings, with larger margins under severe corruption than soft-supervision alternatives.The comparison covers symmetric and asymmetric corruption at rates from 20% to 80%.Source: §5.6 · p. 12, Table 3 · p. 13

5.7 Ablation Study

• Rewriting drives robustness under heavy noise; EMA adds smaller gains but steadies rewriting, while moderate K balances precision against correction frequency.Small K permits noisy corrections, whereas large K makes rewriting overly conservative; K=8 is the stated default.Source: §5.7 · pp. 12–13, Table 4 · p. 13, Figure 5 · p. 14

5.8 Qualitative Analysis of Label Rewriting

• On OxfordPets at 50% noise, 222 of 234 rewrites corrected labels to ground truth (94.87%), covering 75.00% of noisy samples.Six correct labels became wrong, and six wrong labels were changed to another wrong label; illustrated errors include visually similar fine-grained classes.Source: §5.8 · p. 14, Figure 6 · p. 15

5.9 Limitations

• TecoPrompt intentionally leaves some noisy labels uncorrected; frequent-flip examples fail verification, K needs dataset-specific tuning, and history and EMA add modest overhead.On Caltech101 in the 16-shot setting, the reported overhead is 0.20 seconds more per epoch than NLPrompt on one RTX 3080 Ti.Source: §5.9 · p. 14

• TecoPrompt rewrites OT-based label candidates only when they remain consistent within a K-epoch window and pass EMA confidence gating, reducing confirmation bias and pseudo-label drift.Source: §6 · p. 15
• A group-specific objective applies tailored losses to clean, mid, and noisy subsets.Source: §6 · p. 15
• Experiments across diverse noise types and severities show consistent robustness gains for noise-tolerant prompt tuning.Source: §6 · p. 15

A.1 Generalization of TecoPrompt

• TecoPrompt is not specific to CoOp and can be extended to PromptSRC and MaPLe.Source: §A.1 · p. 21
• The generalization evaluation integrates TecoPrompt with both PromptSRC and MaPLe on UCF101.Source: §A.1 · p. 21
• The reported UCF101 results show excellent performance with both frameworks, suggesting applicability across prompt-learning paradigms.Source: §A.1 · p. 21, Table 5 · p. 21

A.2 Experiments on SUN397

• TecoPrompt delivers the best performance on SUN397 under both symmetric and asymmetric label noise.Source: §A.2 · p. 21, Table 6 · p. 21
• On SUN397, TecoPrompt shows clear improvements across all noise rates.Source: §A.2 · p. 21, Table 6 · p. 21
• The SUN397 results further verify TecoPrompt's robustness and effectiveness on challenging noisy-label settings.Source: §A.2 · p. 21, Table 6 · p. 21

A.3 Additional Ablation Study

• On EuroSAT, combining EMA and temporal consistency significantly improves robustness, with β = 0.8 and K = 8 achieving the best average accuracy.Source: §A.3 · pp. 21–22, Table 7 · p. 22
• A moderate temporal window better balances correction precision and coverage than smaller or larger K values.Source: §A.3 · pp. 21–22, Table 7 · p. 22
• The tri-group loss outperforms CE, GCE, and MAE on EuroSAT.Source: §A.3 · p. 22, Table 7 · p. 22

B.1 Notation and Definitions

• The analysis distinguishes noisy labels from OT-forgettable samples, which are identified from pseudo-label trajectories rather than ground-truth correctness.OT-unforgettable samples retain stable OT pseudo-labels; the analysis also defines noise by disagreement between observed and ground-truth labels.Source: §B.1 · pp. 22–23
• A rewrite candidate must mismatch the observed label consistently across K epochs and pass the confidence gate throughout that window.The candidate event combines the stable-mismatch event with the K-epoch confidence gate.Source: §B.1 · p. 23

B.2 Assumptions

• The guarantees assume OT-unforgettable samples are more stable than forgettable ones, confidence filtering retains stable candidates, and noisy-unforgettable rewrites have residual error at most ε⋆.The stability bounds apply across K−1 successive transitions; the correctness condition is conditional on the noisy OT-unforgettable block and rewrite candidacy.Source: §B.2 · pp. 23–24

B.3 Main Results and Proofs

• Across K epochs, OT-unforgettable persistence is at least p_u^(K−1), while OT-forgettable persistence is at most p_f^(K−1).The lemma derives these bounds from successive one-step stability probabilities.Source: §B.3 · pp. 24–25
• Rewrite precision is at least (1−ε⋆) times the candidate set's noisy OT-unforgettable share.The result follows by retaining the target-block contribution in the partition of rewrite candidates and applying the residual-correctness assumption.Source: §B.3 · p. 25
• The noisy OT-unforgettable share of the candidate set has a lower bound nondecreasing in K and converging to 1.The bound's non-target contamination terms decrease as K grows while its leading target-block term remains constant.Source: §B.3 · pp. 25–26
• Theorem 3 lower-bounds rewrite precision by an expression nondecreasing in K that converges to 1−ε⋆.The bound combines the residual-correctness factor with a lower bound on the noisy OT-unforgettable share of the candidate set.Source: §B.3 · p. 26