PaperlayerView exact revision on arXiv

Source-linked AI summary

DVLA-RL++: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learning

Wenhao Li, Xianjing Meng, Qiangchang Wang, Zhongyi Han, Yilong Yin, Liqiang Nie

arXiv:2610.12095v1cs.CVcs.LG

TL;DR

Few-shot prototypes can be contaminated by incidental context when textual agreement reflects both intrinsic object properties and background cues. DVLA-RL++ combines selective evidence aggregation with nuisance-aware, reinforcement-learning-based semantic fusion. Across standard, fine-grained, and cross-domain benchmarks, it reports state-of-the-art accuracy and a 1.4% average gain over DVLA-RL.

  • Problem

    Image-text agreement can reflect intrinsic object properties and incidental context, making it difficult to determine which support evidence should define few-shot prototypes.

    Source: §I · p. 1

  • Method

    DVLA-RL++ uses CSP to select support evidence against intrinsic and nuisance descriptions and CRG to learn layer-wise semantic fusion with a nuisance-aware reward and paired reference trajectory.

    Source: §I · p. 2

  • Results

    Across nine FSL benchmarks, DVLA-RL++ achieves state-of-the-art results with an average accuracy gain of 1.4% over DVLA-RL.

    Source: §I · p. 2, §V · p. 12

  • Takeaways & Limitations

    The analysis relates retained nuisance mass and anchor quality to prototype stability and establishes conditions for unbiased on-policy gradient estimation.

    Source: §I · p. 2, §V · p. 12

Abstract

from arXiv · show

Few-shot learning aims to recognize novel categories from limited labeled examples. Recent studies incorporate textual semantics to compensate for limited visual observations and improve class representations. However, high image-text agreement may reflect both intrinsic object properties and incidental context, making support prototypes susceptible to contextual contamination. To address this problem, we propose DVLA-RL++, which extends DVLA-RL with complementary semantic purification (CSP) and counterfactual reinforcement-learning gating (CRG). Specifically, CSP generates intrinsic and nuisance descriptions from labeled supports and compares their agreement with each support token. An ambiguity-dependent rejection margin guides sparse evidence allocation, while an intrinsic semantic anchor fills the unassigned mass to provide a fallback when visual evidence is unreliable. CRG learns layer-wise semantic fusion strengths using a reward that balances recognition performance and nuisance exposure. An independently executed reference trajectory on the same episode provides a paired learning signal. Theoretical analysis relates retained evidence and anchor quality to prototype stability and establishes conditions for unbiased on-policy gradient estimation. Experiments on standard, fine-grained, and cross-domain benchmarks show state-of-the-art accuracy, with an average gain of 1.4% over DVLA-RL. The project page is available at https://peacelwh.github.io/TPAMI27-DVLA-RLpp/.

I. INTRODUCTION

Few-shot prototypes can absorb incidental context as well as class-defining evidence, even when both match language descriptions. DVLA-RL++ addresses this with selective support-evidence aggregation and adaptive semantic fusion, reporting state-of-the-art accuracy across nine benchmarks with a 1.4% average gain over DVLA-RL.

I. INTRODUCTION

  • Support prototypes can include incidental context because image-text agreement may reflect both object properties and contextual cues.The introduction frames evidence selection as distinct from semantic enrichment: reliable prototypes must distinguish class-defining evidence from context.Source: §I · p. 1

B. Complementary Evidence and Selective Aggregation

  • Unlike spatial foreground suppression, semantic evidence selection addresses incidental cues that may occur within or alongside the target object.Nuisance descriptions characterize incidental visual cues within labeled support images, which may coexist with the class label.Source: §I · p. 1, B. Complementary Evidence and… · pp. 2–3
  • CSP compares support tokens with intrinsic and nuisance descriptions, rejects tokens below an ambiguity-dependent margin, and assigns unallocated mass to an intrinsic anchor.The anchor provides a fallback when no visual token is retained; the analysis relates retained nuisance mass and anchor quality to prototype stability.Source: §I · p. 2, B. Complementary Evidence and… · p. 3
  • CRG learns layer-wise semantic fusion strengths using a nuisance-aware reward and paired utilities from independently executed policy and reference trajectories.The paired learning signal is used to establish conditions for unbiased on-policy gradient estimation.Source: §I · p. 2

I. INTRODUCTION

  • Across nine FSL benchmarks, DVLA-RL++ achieves state-of-the-art results with an average accuracy gain of 1.4% over DVLA-RL.The evaluation spans standard, fine-grained, and cross-domain scenarios.Source: §I · p. 2

A. Visual and Semantic Few-Shot Learning

  • Visual few-shot methods improve matching and transfer, while semantic methods add class knowledge through names, prompts, attributes, and descriptions.DVLA-RL combines local attributes and global descriptions, but does not explicitly select which support features define a class.Source: A. Visual and Semantic Few-Shot Learning · p. 2

C. Adaptive Fusion and Policy Learning

DVLA-RL++ combines selective support-prototype construction with adaptive semantic fusion. Its gated attention uses support-derived semantic information to enrich image tokens while propagating only image tokens between transformer blocks.

C. Adaptive Fusion and Policy Learning

  • CRG builds on reference-based policy learning by pairing sampled and reference rollouts on the same episode, with each branch propagating its own states.The analysis specifies conditions under which the reference utility is an action-independent baseline for unbiased on-policy gradient estimation.Source: C. Adaptive Fusion and Policy Learning · p. 3

III. METHOD

  • DVLA-RL++ combines dual-level alignment, CSP for support-evidence selection, and CRG for learning semantic-fusion strengths from recognition and evidence-reliability signals.The method retains DVLA-RL’s alignment while adding selective support aggregation and a nuisance-aware gate-learning objective.Source: §III · p. 3, A. Problem Formulation · p. 4

A. Problem Formulation

  • An N-way K-shot episode uses labeled supports and separate queries, and support-evidence contamination means incidental context enters prototypes despite correct support labels.The base, validation, and test categories are disjoint; class names provide semantic supervision.Source: A. Problem Formulation · p. 4

B. Dual-Level Semantic Alignment

  • At each selected layer, a shared episode-level gate balances image-guided and text-guided semantic responses before joint transformer processing propagates only image tokens.Larger gate values emphasize image-conditioned responses, while smaller values emphasize text-conditioned responses.Source: Figure 3 · p. 4, B. Dual-Level Semantic Alignment · p. 4

C. Complementary Semantic Purification

CSP compares support tokens with intrinsic and nuisance semantic descriptions, then allocates visual evidence selectively and uses an intrinsic semantic anchor for unallocated mass.

C. Complementary Semantic Purification

  • CSP scores final-layer visual tokens against intrinsic and nuisance description banks, with positive signed scores indicating stronger intrinsic agreement.Responses use size-normalized log-sum-exp aggregation with fixed weights for local and global semantics.Source: C. Complementary Semantic Purification · p. 5
  • The required intrinsic advantage rises with maximum cross-bank similarity, reflecting greater difficulty distinguishing semantically overlapping explanations.The overlap is evaluated only when the nuisance bank is nonempty.Source: C. Complementary Semantic Purification · p. 5
  • Sparse allocation retains at most unit visual mass per support, and a token receives positive weight only when its score exceeds the rejection margin.Unallocated mass is not redistributed to rejected tokens.Source: C. Complementary Semantic Purification · p. 5
  • The intrinsic anchor fills prescribed semantic share and unallocated visual mass, so a support with no retained token contributes exactly the anchor.The prototype aggregates support representations and is normalized with a numerical safeguard.Source: C. Complementary Semantic Purification · p. 5
  • The pre-allocation evidence deficit penalizes tokens below the rejection margin as a surrogate for nuisance exposure, while also reflecting ambiguous or insufficient intrinsic evidence.It is computed on final-layer image tokens.Source: C. Complementary Semantic Purification · p. 5

D. Counterfactual Reinforcement-Learning Gating

CRG learns layer-wise semantic-fusion gates using recognition and nuisance-exposure signals, with a paired reference rollout providing a learning signal. Its on-policy gradient result depends on fixed environment and baseline parameters and an independent reference rollout.

D. Counterfactual Reinforcement-Learning Gating

  • CRG pairs a sampled gate trajectory with an independently executed reference trajectory on the same episode to guide gate-policy updates.The representation remains fixed during these paired evaluations.Source: D. Counterfactual… · p. 6, Figure 5 · p. 6
  • The reward balances recognition, an evidence-deficit surrogate for nuisance exposure, and deviation from the reference schedule; the reference has zero deviation cost.The policy observes incoming tokens, while the evidence-deficit cost uses post-action tokens.Source: D. Counterfactual… · p. 6
  • The reference utility can be subtracted without changing the expected on-policy gradient when it is independent of sampled actions conditional on the episode and shared randomness.This result assumes fixed environment and baseline parameters, regular policy densities, and integrable score-weighted rewards.Source: D. Counterfactual… · pp. 6–7
  • After policy parameters change, the clipped PPO surrogate is not generally an unbiased estimator of the on-policy reward gradient.At behavior-policy parameters, its surrogate gradient agrees with the negative on-policy reward gradient.Source: D. Counterfactual… · p. 7

E. Prototype Stability and Inference

The analysis bounds prototype error through retained intrinsic evidence, nuisance evidence, and anchor quality. It also gives conditions for preserving a fixed query’s prediction and describes per-episode cached inference.

E. Prototype Stability and Inference

  • Reducing retained nuisance mass improves the prototype-error bound when the replacement anchor is more reliable than the rejected nuisance evidence.The bound separates contributions from retained intrinsic evidence, retained nuisance evidence, and the semantic anchor.Source: E. Prototype Stability and Inference · p. 7
  • Under negative-mean sub-Gaussian nuisance scores, expected retained nuisance mass decreases exponentially with squared combined separation, without requiring token independence.A larger rejection margin suppresses nuisance allocation but increases reliance on the anchor, making anchor quality central to the stability guarantee.Source: E. Prototype Stability and Inference · p. 7
  • If the prototype error bound is below 1 − εn, normalization yields a prediction-preservation condition based on the ideal query’s cosine margin.The guarantee concerns prototype perturbations for a fixed query representation.Source: E. Prototype Stability and Inference · p. 7
  • At inference, support-conditioned semantic banks, deterministic mean gates, and prototypes are cached per episode, and each query is classified independently in one forward pass.Inference uses no reference rollout or parameter update.Source: E. Prototype Stability and Inference · p. 7

IV. EXPERIMENTS

The experiments evaluate DVLA-RL++ across standard, fine-grained, and cross-domain few-shot recognition, then examine its components and supporting diagnostics.

IV. EXPERIMENTS

  • Evaluation covers standard, fine-grained, and cross-domain few-shot recognition.The experiments then examine component ablations and diagnostics of evidence reliability and policy learning.Source: §IV · p. 7
  • Component ablations and evidence-reliability diagnostics follow the benchmark evaluation.Source: §IV · p. 7
  • Policy-learning diagnostics are also included.Source: §IV · p. 7

A. Experimental Settings

Across standard, fine-grained, and cross-domain few-shot settings, DVLA-RL++ reports higher mean accuracy than DVLA-RL. Model analyses examine context robustness, evidence selection, semantic fusion, and policy learning.

A. Experimental Settings

  • Experiments use nine benchmark settings across eight datasets, with 2,000 5-way episodes per setting and separate query images.The standard, fine-grained, and cross-domain scenarios use disjoint training, validation, and test categories.Source: A. Experimental Settings · pp. 7–8

B. Main Results

  • DVLA-RL++ has the highest mean accuracy in all six standard settings, improving over DVLA-RL by 1.2%–1.7%.Table I covers miniImageNet, tieredImageNet, and CIFAR-FS in 5-way 1-shot and 5-shot settings.Source: B. Main Results · p. 8, Table I · p. 8
  • DVLA-RL++ has the highest mean accuracy in all six fine-grained settings, with gains over DVLA-RL of 2.18%–2.41% in 1-shot and 0.55%–1.25% in 5-shot recognition.The comparison covers CUB, Stanford Cars, and Stanford Dogs.Source: B. Main Results · p. 8, Table II · p. 9
  • DVLA-RL++ has the highest reported mean accuracy in all six cross-domain settings and improves over DVLA-RL by 1.0% on average.Table III evaluates transfer from miniImageNet to CUB, Places, and ChestX.Source: B. Main Results · p. 8, Table III · p. 9

C. Model Analysis

  • With both CSP and CRG, Shift accuracy rises by 5.4% and 7.0%, while Clean accuracy rises by 1.7% and 2.4% on miniImageNet and CUB, respectively.The Clean-to-Shift drop falls from 5.9% to 2.2% on miniImageNet and from 6.9% to 2.3% on CUB; each component alone also improves Shift accuracy over DVLA-RL.Source: C. Model Analysis · pp. 9–10, Table IV · p. 9
  • Every internal design control reduces Shift accuracy more than Clean accuracy; uniform allocation causes the largest reported Shift degradation, 3.6% on miniImageNet and 4.8% on CUB.Removing nuisance descriptions costs 2.2% and 2.9%, while removing the neutral option or intrinsic fallback costs 1.5%–2.2%.Source: C. Model Analysis · p. 10, Table V · p. 10
  • The default τs = 0.2 balances sparse and broader token allocation, while miniImageNet accuracy stays within 0.3% of default across sampled λ values in [0.5, 0.9].Across sampled γ values in [0.1, 0.75], accuracy stays within 0.2% of default; signed-margin overlap falls from 0.74 to 0.16 on miniImageNet and from 0.75 to 0.12 on CUB.Source: C. Model Analysis · pp. 10–11, Figure 6 · p. 10, Figure 7 · p. 10
  • The paired estimator reduces normalized gradient-covariance trace to 0.60 and nuisance-cost increases from 24% to 9%; displayed examples show broader object coverage with weaker contextual responses.The audit reports 24% token retention, 94% intrinsic precision among unambiguously annotated retained tokens, and 0.08 neutral mass; the t-SNE projections show more compact class neighborhoods and clearer separation.Source: C. Model Analysis · pp. 11–12, Figure 8 · p. 11

V. CONCLUSION

DVLA-RL++ addresses support-evidence contamination in few-shot recognition through selective evidence aggregation and adaptive semantic fusion. Experiments report consistent gains over DVLA-RL across benchmark types, with an average accuracy gain of 1.4%.

V. CONCLUSION

  • DVLA-RL++ combines complementary semantic purification to select reliable support tokens with an intrinsic semantic anchor, and counterfactual reinforcement-learning gating to adapt semantic fusion using a nuisance-aware reward.Source: §V · p. 12

A. Feature, Semantic, and Policy Interfaces

The feature, semantic, and policy interfaces define how support evidence is scored, allocated, and used for queries. A projection-based rejection rule permits unassigned visual mass, while sensitivity analysis relates changes in scores and anchors to prototype changes.

A. Feature, Semantic, and Policy Interfaces

  • Layer-specific image tokens are scored through a projection into the text-embedding space, and an anchor projection maps semantic information into the final feature space.The scorer maps are calibrated during warm-up and then frozen, while representation updates train the encoder and anchor map.Source: A. Feature, Semantic, and Policy… · p. 14
  • The gate state pools support visuals and semantic summaries, including signed evidence, retained and neutral masses, and a layer indicator.The policy consumes a finite-dimensional summary; the proofs retain the complete support history and preceding actions.Source: A. Feature, Semantic, and Policy… · pp. 14–15
  • Queries replay the support-derived action schedule and use all candidate classes’ intrinsic tokens, without requiring the unknown query class or allowing one query to alter another’s prediction.Source: A. Feature, Semantic, and Policy… · p. 15

B. Projection, Rejection, and Sensitivity

  • The allocation is a Euclidean projection onto nonnegative weights with total mass at most one, so visual tokens below the rejection margin can receive zero weight.When all scores fall below the margin, the prototype is exactly the intrinsic anchor; no division by retained mass is needed.Source: B. Projection, Rejection, and… · p. 15
  • Candidate deficit is not an independent audit of retained contamination: it can be positive after nuisance rejection or small when a misgrounded bank scores nuisance tokens as intrinsic.The sensitivity statement concerns changes in scoring and anchoring for fixed image tokens.Source: B. Projection, Rejection, and… · p. 15

C. Retained Nuisance Mass and Prototype Error

The analysis bounds retained nuisance mass and prototype error, relating their stability to evidence quality and the intrinsic anchor. It also identifies conditions and trade-offs: larger rejection margins reduce nuisance mass but can increase anchor reliance.

C. Retained Nuisance Mass and Prototype Error

  • The retained-mass result bounds allocation to erroneous nuisance tokens, controlling their total weight beyond the familywise probability bound.When σ = 0, every nuisance score is below the margin and nuisance mass is zero.Source: C. Retained Nuisance Mass and… · p. 15
  • The prototype error analysis combines bounded intrinsic-token, nuisance-token, and anchor errors to bound deviation from the ideal class direction.The comparison with the anchor-only upper bound is conditional; the supplied passage does not include its condition.Source: C. Retained Nuisance Mass and… · p. 15
  • When the fallback is active, the prototype equals the intrinsic anchor and its error is bounded by the anchor error; when inactive, the support estimate stays above the stated norm threshold.The inactive case follows when Ec < 1 − εn.Source: C. Retained Nuisance Mass and… · p. 15
  • If normalization holds for every class, the margin condition preserves the ideal label, while larger rejection margins reduce nuisance mass but can increase anchor reliance.An inaccurate anchor raises BA, and incorrectly assigned nuisance phrases can violate the score-gap assumption.Source: C. Retained Nuisance Mass and… · p. 16

D. Paired Policy Gradients and Clipping

The section establishes an unbiased on-policy gradient under conditional-independence and regularity conditions, then delineates what paired returns and clipped surrogates do not guarantee. In particular, positive covariance can reduce paired-return variance without ensuring lower gradient variance.

D. Paired Policy Gradients and Clipping

  • The paired reference return and action-independent value baseline yield an unbiased policy-gradient estimate under the stated conditional-independence and differentiation conditions.The proof conditions on the episode and exogenous randomness; the separate reference rollout avoids an action-dependent baseline that could erase a nonzero gradient.Source: D. Paired Policy Gradients and Clipping · p. 16
  • Positive covariance reduces paired-return variance relative to an independently sampled reference, but does not ensure lower variance than using R alone or lower gradient variance.The paper therefore reports gradient-variance diagnostics directly rather than using paired reward variance as a substitute.Source: D. Paired Policy Gradients and Clipping · p. 16
  • At θ = θold, the clipped-surrogate gradient has the on-policy expectation, but after updates it need not equal the objective gradient or correct changed state visitation.Neither this result nor clipping alone establishes monotonic improvement.Source: D. Paired Policy Gradients and Clipping · p. 16
  • The stated clipping counterexample shows that an excess-ratio expression cannot bound the clipping-gradient error.For the example, the expected gradient gap is 0.242, while the proposed excess-ratio bound would give 0.002.Source: D. Paired Policy Gradients and Clipping · p. 16

E. Training Algorithm

Training alternates policy and representation updates after warm-up with a reference gate, combining CSP prototype purification with CRG layer-wise fusion gating. At inference, the current mean policy produces cached episode-level gates and prototypes without the reference gate or value baseline.

E. Training Algorithm

  • After reference-gate warm-up, episodic training alternates policy and representation updates using CSP and CRG to construct purified prototypes.CSP allocates support tokens against intrinsic and nuisance semantics and anchors unassigned mass; CRG compares sampled gates with an independent reference rollout under a nuisance-aware reward.Source: E. Training Algorithm · pp. 16–17
  • For policy updates, training computes a reference schedule and return, samples layer-wise gates, constructs prototypes, calculates reward and advantages, then updates the policy and value baseline.The algorithm draws a fresh episode for representation updates and stops gradients through the gate actions.Source: E. Training Algorithm · p. 17
  • Representation updates freeze gate actions and optimize the representation with classification loss on episode queries.The policy trajectories are discarded after the representation update.Source: E. Training Algorithm · p. 17
  • At inference, the current mean policy generates and caches the episode’s gate schedule and class prototypes; the reference gate and value baseline are unnecessary.Queries are encoded individually after the support state is updated following each mean action.Source: E. Training Algorithm · p. 17
Loading 2610.12095v1…