PaperlayerView exact revision on arXiv

Source-linked AI summary

Local Gains and Fixed-Assignment Set Losses in Shared Set Decoders

Ze Zhang, Yang Zhang

arXiv:2608.14717v1cs.CVcs.LG

TL;DR

In shared set decoders, an edit can improve its local slot while reducing the utility of the prediction set, raising the question of how its effects propagate across the set and readouts. We compare selected deletions with matched active controls and find local-positive but fixed-assignment-negative effects in both checkpoints, with persistence differing after rematching and native selection.

  • Problem

    In shared set decoders, object queries are updated jointly and outputs depend on assignment and selection, motivating evidence on how one edit affects the set across readouts.

    Source: §1 · p. 1

  • Method

    We compare selected target deletions with matched active controls across local, fixed-assignment, rematched, and native readouts, plus a fixed-map hard-deletion versus mass-preserving edit.

    Source: §1 · p. 2, §3.2 · p. 3

  • Results

    Target-minus-control DETR local effect +0.095621 and fixed-assignment effect −0.005741; DINO local effect +0.036633 and fixed-assignment effect −0.008833; DETR rematched/native intervals crossed zero, while DINO remained negative.

    Source: §1 · p. 2, §4.1 · p. 4

  • Takeaways & Limitations

    Selected relations exhibit deletion sensitivity whose observed consequence depends on the readout and intervention operator.

    Source: §7 · p. 9

    Results are conditional on selected image–relation populations from two related checkpoints and do not estimate arbitrary-relation prevalence or an architecture-family effect.

    Source: §6 · p. 9

Abstract

from arXiv · show

A query-relation deletion can improve the edited slot while reducing the utility of the prediction set that contains it. We study this tension in two related ResNet-50 DETR-family checkpoints using recorded, selection-conditional evidence from 710 paired image-relation units per checkpoint. The primary comparison subtracts a matched active control, which deletes the same leader source at a different recorded recipient, from the selected target deletion. It is therefore a composite contrast rather than a same-recipient placebo. The target-minus-control contrast is locally positive and fixed-assignment negative in both checkpoints. The opposite-sign pattern occurs within 302/710 DETR units and 460/710 DINO units. After rematching, the corresponding counts are 285/710 and 433/710. Rematching and native selection absorb enough of the mean loss for DETR intervals to cross zero, whereas DINO intervals remain negative, so persistence across readouts differs by checkpoint. A fixed-map comparison between hard deletion and a mass-preserving edit also differs before rematching. That comparison is conditional on the outcome-blind map and does not establish same-dose transport. Local intervention success therefore does not determine the consequence for a jointly decoded set. The supported conclusion is selection-conditional deletion sensitivity whose persistence depends on the readout and intervention operator. We do not identify an intervention-invariant edge mechanism, detector-level degradation, population prevalence, or the value of a training-time regularizer.

1 Introduction

The introduction frames query-relation deletion as a jointly decoded response: a local gain can coexist with loss in fixed prediction-set utility. It argues that what this response establishes depends on the comparator, intervention operator, selected population, and readout stage.

1 Introduction

  • The local, set-level, and post-matching responses are linked observations of a jointly decoded output, not interchangeable verdicts about whether a relation is helpful or harmful.Reassignment can recover utility without undoing the underlying state change, while native selection can expose another response.Source: §1 · pp. 1–2
  • The evidence is selection-conditional: fixed-assignment loss is not detector average precision, selected responses are not prevalence estimates, and they do not establish training-design value.The paper also distinguishes observed sensitivity from stronger claims about intervention transport or mechanism.Source: §1 · p. 2
  • The analysis uses a target-minus-matched-active-control comparison across local, fixed, rematched, and native readouts, alongside a separate fixed-map operator comparison.The operator comparison contrasts hard deletion with a mass-preserving edit; supporting dose, fresh-context, and design-authorization analyses test whether stronger interpretations are warranted.Source: §1 · p. 2
  • In both checkpoints, target deletion improves the local slot but reduces fixed-assignment set utility; after rematching and native selection, persistence differs by checkpoint.DETR intervals cross zero, whereas DINO intervals remain negative; opposite-sign responses occur in many selected units.Source: §1 · p. 2
  • The mapped hard-minus-mass difference appears before rematching, but the evidence does not calibrate operator transport or identify a unique mechanism.The findings support an intervention-specific interpretation rather than an intervention-invariant edge mechanism.Source: §1 · p. 2

2 Related Work

Related work motivates caution in interpreting intervention responses and frames query interaction as a decoder-design concern; this study instead examines selected relation interventions across assignment-sensitive readouts at fixed checkpoints.

2.1 Intervention responses and mechanism claims

2.2 Query interaction as a decoder design problem

2.3 Matching-aware readout

  • Set-prediction readouts separate original-assignment correspondence, rematching, and native selection; the contribution is an empirical comparison of selected deletion, active control, and an indexed alternative operator, not a new matching procedure.These readouts locate where an intervention response persists as the set can reorganize.Source: §2.3 · p. 2

3 Study Design and Estimands

The study uses baseline-selected image–relation units to compare target deletion with a matched active control, then evaluates local and set-level responses under fixed-assignment and rematched readouts. Its estimands are conditional on the selected populations, and the operator and readout definitions constrain their interpretation.

3.1 Datasets, checkpoints, and evaluated populations

  • 710 complete paired units per checkpoint form the primary discovery populations, drawn from a validation-derived partition of 1,500 images.The checkpoints are DETR-R50-500 and DINO-R50 4-scale 12-epoch; units require target and control feasibility in both checkpoints, and estimates are conditional on baseline-only selection.Source: §3.1 · p. 3

3.2 Relation selection and matched active control

  • The target arm deletes the competitor’s leader relation, while the active control deletes the same leader source at a separately matched recipient, making the contrast composite rather than a same-recipient placebo.Selection uses baseline quantities: eligible queries must have IoU ≥ 0.30, and the leader and competitor are the top two by quality; the matched control recipient uses target-class score, layer-3 hidden-state norm, and leader-to-source attention.Source: §3.2 · p. 3

3.3 Intervention operators

  • Hard deletion removes a layer-3 pre-softmax attention logit, whereas the mass-preserving operator exchanges target and donor weights after softmax.The hard-minus-mass comparison uses an outcome-blind map based on target-weight change and immediate message norm; it is conditional on that fixed map, not a same-dose or transport estimate.Source: §3.3 · p. 3

3.4 Utility and matching-aware readouts

  • A deterministic Hungarian assignment defines image-set utility, with fixed-assignment, rematched, and native-selection effects distinguished from the local recipient-slot endpoint.The decomposition separates focal, spillover, matching, and selection contributions; the target-set endpoint is the maximum quality among queries assigned to the same object, and these readouts are not interchangeable.Source: §3.4 · p. 4

3.5 Statistical uncertainty and multiplicity

  • Checkpoint analyses use 10,000 paired-image bootstrap resamples and within-model simultaneous 95% intervals, while sign-event rates use two-sided Wilson 95% intervals.All effects and event rates describe selected populations; the COCO assay’s decision bands do not support a mechanism claim because its hard-deletion sensitivity prerequisite fails.Source: §3.5 · p. 4

3.6 Reproducibility artifacts

  • The companion package reconstructs reported aggregates from analysis-ready rows and configurations but excludes pixels and weights, so it supports aggregate rather than end-to-end inference replication.Its CPU-only reproduction script does not require model inference.Source: §3.6 · p. 4

4 Results

In selected 710-unit checkpoint populations, target deletion is locally positive but fixed-assignment negative against a matched active control. Reassignment changes the persistence of that response by checkpoint, while the mapped-operator, transport, and regularizer analyses do not support broader upgrades.

4.1 Target deletion improves the local slot but reduces fixed-assignment set utility

  • Both checkpoints show a local gain alongside a fixed-assignment set loss under the selected target-minus-matched-active-control contrast.DETR local effect: +0.095621, simultaneous 95% interval [+0.070160, +0.121082]; fixed-assignment effect: −0.005741, interval [−0.010889, −0.000592]. DINO local effect: +0.036633, interval [+0.030623, +0.042643]; fixed-assignment effect: −0.008833, interval [−0.013130, −0.004536]. This is a composite contrast using different recipients, and the set-level readout is not detector average precision; the downstream mechanism is unresolved.Source: §4.1 · pp. 4–5, Table 1 · p. 5, Figure 2 · p. 6

4.2 Reassignment shows checkpoint-specific recovery

  • Opposite-sign local-positive/target-set-negative events occur in 302/710 DETR units and 460/710 DINO units, or 285/710 and 433/710 after rematching.Events use the declared deadzone and occur within selected units. DETR rematched and native means are −0.001575 and −0.001686, with both intervals crossing zero; DINO means are −0.007424 and −0.007429, with both intervals remaining negative.Source: §4.2 · p. 6

4.3 A mapped operator difference appears before rematching

  • Under the fixed outcome-blind map, hard-minus-mass differences appear before rematching, but the comparison is not same-dose and does not establish operator transport.DETR mapped local, spillover, and fixed-assignment means are +0.081169, −0.005091, and −0.004981, with intervals excluding zero; its rematched and native intervals cross zero. DINO local, spillover, fixed-assignment, rematched, and native means are +0.034817, −0.007747, −0.007701, −0.006561, and −0.006571, with all five intervals excluding zero. Possible mechanisms remain unresolved.Source: §4.3 · pp. 6–7, Figure 3 · p. 7

4.4 Dose and fresh-context evidence do not support a transport upgrade

  • Neither the legacy dose analysis nor the fresh assay supports upgrading the findings to calibrated transport or a mechanism claim.The legacy analysis has all 1,420 expected records and passes integrity checks, but its full conjunction is not met at any intermediate nominal dose; nominal attenuation is not calibrated realized dose. In the fresh assay, pairwise intervals fall within specified equivalence bands, but the hard-sensitivity prerequisite fails in both checkpoints, and differences in population and ground-truth formation prevent establishing equivalence to the discovery effect.Source: §4.4 · pp. 7–8

4.5 Diagnostic sensitivity does not authorize a regularizer study

  • No prespecified candidate met the fixed-checkpoint selection rule, so no regularizer was trained or evaluated.The design analysis covered 256 images per checkpoint and provides no evidence that static or adaptive regularizers generally succeed or fail.Source: §4.5 · p. 8

5 Discussion

The discussion shows that local improvement can coexist with set-level loss, and that rematching and intervention choice affect which response is observed. These findings motivate careful reporting but do not identify an intervention-invariant mechanism or establish training-time design benefits.

5.1 Local improvement can coexist with set-level redistribution

  • Opposite signs across local and fixed-assignment readouts show that a selected-recipient improvement can coexist with set-level redistribution, without identifying a harmful relation or unique causal path.In a jointly decoded system, responses can redistribute through other slots and later states.Source: §5.1 · p. 8

5.2 Matching is part of the measured response

  • DETR’s later-readout intervals cross zero while DINO’s remain negative, so rematching acts as a recovery stage rather than neutral relabeling.The fixed-assignment result remains; the rematched readout instead tests whether reassignment can recover utility.Source: §5.2 · p. 8

5.3 Intervention versions define different scientific questions

  • The hard-versus-mass-preserving fixed-map contrast appears before rematching, but does not identify its source or resolve transport.Nominal dose was not connected to realized dose in the recorded outcomes; the fresh assay does not close the gap because its prerequisite fails and its population and baseline differ.Source: §5.3 · p. 8

5.4 Diagnosis and design are separate empirical steps

  • The prespecified design analysis selects no stable continuation and includes no training, so fixed-checkpoint diagnosis does not establish regularizer performance.Intervention studies should report the edited location, comparator, selected population, realized-dose contract, and readout stage; local, fixed, rematched, and native outcomes should remain separate.Source: §5.4 · p. 8

6 Limitations

The findings are conditional on selected populations from two related checkpoints and do not establish population-wide prevalence, an architecture-family effect, or detector-level degradation. Missing dose, mechanism, transport, and subgroup evidence further limits interpretation of the reported sensitivity and recovery patterns.

6 Limitations

  • The results concern selected image–relation populations from two related checkpoints; they do not estimate arbitrary-relation prevalence or an architecture-family effect.The active control edits a different recipient, making the primary contrast composite rather than a same-recipient comparison.Source: §6 · p. 9
  • Set utilities are not detector average precision, and the study does not establish detector-level degradation.There is no AP bridge or qualitative image case study, limiting claims about detector-level relevance.Source: §6 · p. 9
  • Legacy outcomes lack co-recorded per-head realized dose and immediate message displacement; the mapped hard-minus-mass comparison uses a fixed outcome-blind map without mapping-uncertainty propagation.The fresh assay changes context and fails its hard-sensitivity prerequisite; evidence therefore does not identify a unique mechanism or establish calibrated operator transport or population-wide non-transport.Source: §6 · p. 9
  • Reported populations and sign-event rates remain conditional on the 0.30 and 0.05 IoU admission thresholds and 10^-4 event deadzone; threshold-sensitivity analyses were not performed.The study does not decompose responses by class, object size, crowding, attention head, or decoder layer, limiting claims about heterogeneity and when checkpoint-specific recovery occurs.Source: §6 · p. 9
  • The study does not evaluate a learned regularizer, and materials reserved for prospective validation remain sealed and unread.Source: §6 · p. 9

7 Conclusion

Across two related shared set-decoder checkpoints, deleting a selected query relation improves its local target slot but reduces fixed-assignment set utility. The observed deletion sensitivity depends on readout and intervention operator, without establishing an intervention-invariant mechanism or training-time design claim.

7 Conclusion

  • Deleting a selected query relation versus a matched active control improves the local target slot while reducing fixed-assignment set utility.The within-unit reversals show this disagreement is not only an aggregate effect.Source: §7 · p. 9
  • Rematched and native readouts show checkpoint-specific persistence, while the mapped hard-minus-mass difference before rematching does not establish operator transport.The evidence supports deletion sensitivity whose observed consequence depends on readout and intervention operator.Source: §7 · p. 9
  • The evidence does not establish an intervention-invariant mechanism or a training-time design claim.Source: §7 · p. 9

8 Data and Code Availability

The source code and frozen reproduction package are public and support exact reconstruction of reported aggregate statistics. They do not enable end-to-end model-inference replication, and prospective-validation materials are not included.

8 Data and Code Availability

  • The public repository provides a frozen, CPU-only package for reproducing the reported aggregate analyses.It includes per-image rows, analysis configurations, population and pair manifests, integrity records, and expected aggregates; the same package is arXiv ancillary material.Source: §8 · p. 9
  • Image pixels and model weights are not redistributed, so the package supports aggregate reconstruction but not end-to-end model-inference replication.The pixels and weights must be obtained from their original sources and are not required for aggregate reproduction.Source: §8 · p. 9
  • The repository’s separately scoped detection demonstration does not generate or replace paper results, and prospective-validation materials were neither accessed nor included.Source: §8 · p. 9

9 Broader Impact · A Supporting Analyses and Provenance

Precise reporting of internal interventions helps prevent positive ablations and fixed-assignment utility changes from being overinterpreted. The study reports no specific funding, no competing interests, and no role for Wuhan United Imaging Surgical Co., Ltd. in the research or submission decision.

9 Broader Impact

  • Keeping the comparator, selected population, and readout stage attached to each result helps prevent fixed-assignment utility changes from being misread as detector-level degradation.The broader impact concern is that this misreading is the main risk, alongside overclaiming from positive ablations.Source: §9 · p. 9
  • The research received no specific grant, Wuhan United Imaging Surgical Co., Ltd. had no role in the study or submission, and the authors declared no competing interests.The company had no role in study design, data analysis, or interpretation of results.Source: §9 · p. 9

A.1 Operator and context checks · A.2 Native endpoint provenance note · B Operator and Population Details

The operator checks do not establish calibrated transport: the legacy dose analysis fails its prespecified criteria, and the separate assay’s hard-deletion sensitivity check fails in both checkpoints. The native-endpoint note and population details further limit interpretation to T0-defined, selection-conditional estimands.

A.1 Operator and context checks

  • Across 1,420 legacy intervention records, controls are exactly zero, but no intermediate mass-preserving condition passes the prespecified local, set-level, and joint-event criteria.The records lack per-head realized dose and immediate message displacement, so this result does not establish calibrated operator transport.Source: §A.1 · p. 11
  • The input-only replay supports the fixed map, but the separate 128-unit-per-checkpoint assay fails the hard-deletion sensitivity check in both checkpoints, leaving operator transport unresolved.Although its pairwise operator intervals fall inside the assay-specific bands, the required sensitivity check fails.Source: §A.1 · p. 11
  • None of the complete prespecified regularizer candidates passes on 256 images per checkpoint, so no training study is undertaken.This is a design-stopping result, not evidence about regularizer performance.Source: §A.1 · p. 11

A.2 Native endpoint provenance note

  • H4-D and T0 are historical provenance aliases, not canonical endpoints; H4-D’s native readout differs from the T0 common-estimand endpoint.Their numerical values are not combined, compared, or substituted; primary quantitative claims use T0-defined estimands.Source: §A.2 · pp. 11–12

B Operator and Population Details

  • Hard deletion sets the selected layer-3 pre-softmax attention logit to −∞, changing the denominator and allowing deleted mass to redistribute across remaining sources.The mass-preserving arm edits post-softmax target/donor weights at the same layer while preserving their two-cell mass.Source: B Operator and Population Details · p. 12
  • Table 2 maps the layer-3 intervention arms, with target and active-control arms using different recipients.The target-pair rule is shared across discovery and supporting assays, but their populations, datasets, ground-truth formation, ranking, and admission contexts differ.Source: Table 2 · p. 12, B Operator and Population Details · p. 12
  • Because discovery and supporting assays differ in population and admission context, the denominator is selection-conditional.The distinct dataset, primary-ground-truth formation, and ranking contexts also condition interpretation.Source: B Operator and Population Details · p. 12

C Readout and Reproducibility Contract

The section distinguishes fixed-assignment, rematched, and native endpoints, and keeps two native summaries separate. It also specifies the reproducibility records and reports supporting operator and context checks.

C Readout and Reproducibility Contract

Related papers

Loading 2608.14717v1…