PaperlayerView exact revision on arXiv

Source-linked AI summary

Reversion, Retention, and Semantic Intervention: Measuring Functional Content in CLIP Fine-Tuning Deltas

Chih-Ting Liao, Kezhen Bao, Xin Cao

arXiv:2610.34144v1cs.CVcs.LG

TL;DR

The paper asks how much task performance is supported by semantically identified fine-tuning updates when other updates may support the same predictions. It pairs reversion and retention with controls, finding that retained recovery and semantic-removal effects provide complementary evidence about update function within the evaluated CLIP recipe.

  • Problem

    Weight-space descriptions of fine-tuning deltas leave open how much performance depends on the identified updates, especially when other updates may support the same predictions.

    Source: §1 · p. 2

  • Method

    The paper pairs component reversion and retention across accuracy, loss, and margin with random-retention, co-correction, anchor-substitution, and matched-random removal controls.

    Source: §1 · p. 3

  • Results

    Across five settings, VO retention recovers 0.428–0.869 of the accuracy improvement, and readout-guided removal causes larger recovery collapse than matched-random removal in all five.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Paired removal and retention, with explicit parameter and intervention controls, provide complementary evidence about readable updates’ functional role within this CLIP fine-tuning recipe.

    Source: §8 · p. 12

    Evidence covers five CLIP classification setups under a shared recipe tuning the final six visual blocks; broader training conditions and model families remain untested.

    Source: §7 · p. 11

Abstract

from arXiv · show

Fine-tuning adapts pretrained models to new tasks, but identifying which weight changes support the resulting behavior remains difficult. Weight-space interpretability assigns semantic descriptions to these changes, called fine-tuning deltas. Such descriptions leave open how much task performance depends on the identified updates, particularly when other updates can support the same predictions. We address this gap by pairing reversion, which restores selected parameters to their pretrained values, with retention, which keeps only selected fine-tuning updates. We evaluate Contrastive Language-Image Pretraining (CLIP) models on synthetic financial charts, Flowers102, EuroSAT, and FGVCAircraft with ViT-L/14, and on Flowers102 with ViT-B/16, using a shared recipe that tunes the final six visual blocks. Our contributions are threefold. First, attention value-output (VO) updates alone recover 43-87% of the accuracy improvement, although reverting them leaves chart accuracy unchanged. Cross-entropy loss preserves this contrast on Charts and EuroSAT, while correct-class score margins reveal additional contributions. Second, VO weights recover more loss improvement than all ten equally sized, layer-matched random subsets in each setting. VO and multilayer perceptron (MLP) retentions jointly correct 25-81% of pretrained errors, demonstrating overlapping recovered behavior, which we call operational redundancy. Third, removing semantically selected directions reduces VO-retained recovery more than matched-random removal in all five settings. Together, these results distinguish semantic alignment, removal cost, and retained recovery as complementary evidence about the functional role of fine-tuning updates within the evaluated CLIP recipe.

1 Introduction

Semantic descriptions of fine-tuning deltas identify task-aligned structure, but do not establish how much behavior depends on those updates when other updates may support the same predictions. The paper pairs reversion and retention across five CLIP classification setups to measure necessity and recovered performance, finding substantial component recovery alongside metric- and task-dependent effects.

1 Introduction

  • Weight-space interpretability describes structures in fine-tuning deltas, the parameter differences between pretrained and fine-tuned models, including task-aligned semantic directions in attention VO operators.The paper frames the functional question as how much acquired performance is supported by updates identified by semantic readouts.Source: §1 · pp. 1–2
  • Removing updates alone is incomplete evidence because remaining updates may support the same predictions, and accuracy can conceal changes in prediction confidence.Attention and MLP updates are jointly trained and act in series through the residual stream.Source: §1 · p. 2
  • Reversion restores selected parameters to pretrained values to measure necessity under that edit, whereas retention keeps selected updates to measure recovered fine-tuning improvement.The paper compares both interventions using accuracy, cross-entropy loss, and correct-class margin.Source: §1 · p. 2
  • The five setups use CLIP on synthetic financial charts, Flowers102, EuroSAT, and FGVCAircraft with ViT-L/14, plus Flowers102 with ViT-B/16.All runs fine-tune the final six visual blocks while keeping class-text embeddings and the visual projection fixed; charts control training concepts, while standard datasets test whether the contrast extends beyond that construction.Source: §1 · pp. 2–3
  • On Charts, accuracy rises from 0.136 to 0.985 after tuning; VO reversion leaves accuracy unchanged, while VO-only retention recovers 0.808 of the improvement.Across five settings, VO retention recovers 0.428–0.869 and MLP retention 0.682–0.989 of accuracy improvement.Source: §1 · p. 3, Figure 2 · p. 3
  • VO and MLP retentions jointly correct 0.250–0.814 of pretrained errors, while VO retention outperforms ten equally sized random delta subsets in every setting.Charts and EuroSAT preserve the reversion/retention contrast in cross-entropy, and VO reversion reduces correct-class margin in all five settings.Source: §1 · p. 3
  • Readout-guided removal produces larger recovery collapse than matched-random removal in all five settings; replacing financial text anchors with generic anchors reduces coherent chart heads from 41 to zero.The readout acts on per-head effective VO deltas, while component retention tests recovery at the update-group level.Source: Figure 1 · p. 2, §1 · p. 3
  • The three contributions pair reversion with retention, test specificity and overlap using random subsets and shared pretrained-error corrections, and evaluate semantic readouts through anchor substitution and matched-random direction removal.Together, these experiments distinguish task-aligned structure, removal cost, and recoverable performance.Source: §1 · p. 3

2 Related Work

Prior work offers methods for describing weight deltas, intervening on models, and restricting adaptation. This paper focuses on relating semantic descriptions of weight changes to edited-checkpoint behavior through component retention and direction removal.

Semantic weight reading

  • Weight-based SITH [13] reports semantic alignment in fine-tuning deltas despite limited rotation of absolute weights, alongside other approaches to describing learned representations.The paper focuses on the relationship between delta descriptions and edited-checkpoint behavior.Source: §2 · p. 4

Localization and intervention

  • Intervention studies motivate validating a readout against a specified edit, since causal-tracing localization does not necessarily predict editing success.The paper distinguishes removing an update from the improvement recovered by retaining it.Source: §2 · p. 4

Redundancy and self-repair

  • Backup heads and the Hydra effect illustrate how ablation effects depend on surrounding computation [36] [26] [32].Here, downstream computation remains active after weight edits, and measurements characterize behavior recovered by retained updates in the shared network.Source: §2 · p. 4

Sparse adaptation and merging

  • The paper connects semantic VO readouts to architectural-component retention and tests whether an equally sized random subset accounts for observed recovery.This frames the relationship between semantic descriptions of weight deltas and behavior of edited checkpoints.Source: §2 · p. 4

3 Reading and Testing Weight Deltas

This section defines semantic readouts for effective VO deltas and pairs component reversion and retention with direction-removal tests to measure edit costs and recovered performance.

Setting

  • The audit covers the final six visual blocks of CLIP ViT-L/14 and ViT-B/16 while keeping the text tower and several visual components fixed.The audit includes 96 heads for ViT-L/14 and 72 for ViT-B/16.Source: §3 · p. 4

Semantic readout

  • A semantic readout maps leading singular directions of each effective VO delta through frozen gain and projection, then tests dictionary selection against random-direction alignments.A head is coherent if either mapped direction passes the stated criterion.Source: §3 · p. 4, Figure 3 · p. 5

Direction intervention

  • Direction interventions project selected unit directions out of the full effective operator and compare recovery against matched-random edits.The VO-retained experiment recomputes the delta SVD after each selected-rank edit.Source: §3 · p. 5

Component interventions

  • Reversion restores selected parameters to pretrained values, while retention keeps only selected updates; VO retention for random-subset tests restores biases and both LayerNorms.The VO reversion and standard VO retention use different bias conventions.Source: §3 · pp. 5–6

Operational redundancy

  • Operational redundancy describes overlapping recovered behavior when distinct retained update groups recover performance despite low reversion costs.The study uses an accuracy-necessity threshold of 0.05; its five-point criterion holds in the first three settings.Source: §3 · p. 6

Continuous metrics and uncertainty

  • Cross-entropy and correct-class margin complement accuracy, while paired bootstrap intervals quantify evaluation-example uncertainty conditional on the checkpoints.Random-mask ranges separately describe variation across mask seeds.Source: §3 · p. 6

4 Experimental Design

The design tests component recovery, overlap, and semantic direction removal on fixed checkpoints and evaluation populations. It combines a controlled chart-anchor comparison with standard-dataset and matched-random controls.

Design questions

  • The experiments address whether small reversion costs conceal retained recovery, whether recovery depends on component and overlaps across examples, and whether semantic selection guides effective direction removal.Each comparison uses fixed checkpoints and evaluation populations.Source: §4 · p. 6

Controlled semantic writing

  • The chart experiment uses 13 financial concepts and compares chart-caption anchors with generic anchors while holding images and training hyperparameters fixed.The null tune isolates anchor choice under the recipe.Source: §4 · p. 6

Standard datasets

  • Standard-dataset evaluations cover Flowers102, EuroSAT, and Aircraft with the same adaptation surface, plus a Flowers102 ViT-B/16 condition.Evaluation uses 2,000 test examples drawn by uniform random sampling with a fixed seed.Source: §4 · p. 6

Matched random retention

  • Random retention matches VO weight parameter count and layer allocation, using ten fixed seeds without rescaling or retraining.The retained delta norm is allowed to vary.Source: §4 · pp. 6–7

Example-level comparisons

  • Example-level comparisons measure prediction agreement and shared corrections of pretrained errors across VO retention, MLP retention, and the full model.Shared correct answers separate recovered behavior from agreement on an incorrect label.Source: §4 · p. 7

Protocol history

  • All five VO recoveries exceed 0.25, so the hypothesized dissociation of low VO recovery and stronger MLP recovery does not hold.The observed result is substantial component retention with setting-dependent necessity and overlap.Source: §4 · p. 7

5 Results

Across the five settings, component retentions recover substantial accuracy improvement, but reversion costs, metric dependence, and semantic interventions show that necessity, recoverability, and semantic alignment are distinct properties. The results also find overlapping recovered behavior and stronger recovery collapse from readout-guided than matched-random removal.

5.1 Low Accuracy Necessity Coexists with Substantial Recovery

  • VO-only recovery ranges from 0.428 to 0.869 and MLP-only recovery from 0.682 to 0.989; MLP retention recovers more accuracy gain in every setting.The low-necessity criterion holds for all tested groups in Charts, Flowers102 on ViT-L/14, and EuroSAT, while Aircraft and ViT-B/16 also show substantial retained recovery alongside larger component costs.Source: §5.1 · pp. 7–8, Table 1 · p. 7

5.2 The Contrast Extends to Loss, While Margins Reveal Additional Contributions

  • VO-only loss recovery is 0.486–0.822 and MLP-only loss recovery is 0.698–0.988; Charts and EuroSAT retain the reversion/retention contrast in cross-entropy.VO reversion reduces correct-class margin in all five settings, while VO-only margin recovery is 0.225–0.283, compared with 0.470–0.687 for MLP.Source: §5.2 · p. 8, Table 3 · p. 8

5.3 Recovery Is Component-Specific under Count Matching

  • Weights-only VO exceeds every one of ten sampled random masks in loss recovery in every setting, including unsaturated Aircraft.On Aircraft, weights-only VO accuracy is 0.4930, versus at most 0.4035 across random masks, with full-model accuracy 0.7430.Source: §5.3 · pp. 8–9, Table 4 · p. 8

5.4 Shared Corrections Quantify Operational Redundancy

  • Both retentions correct 0.2502–0.8143 of the pretrained model’s fixed errors, with the largest overlap on EuroSAT and the smallest on Aircraft.Joint correction exceeds the product of marginal correction rates in all five settings; this association can also reflect example difficulty.Source: §5.4 · p. 9, Table 5 · p. 9

5.5 Readout-Guided Removal Reduces Recovery More Than Random Removal

  • Readout-guided removal causes greater normalized-recovery collapse than matched-random removal in all five settings, but only EuroSAT meets the specified majority-contribution criterion.Charts loses about 0.37 normalized recovery, below its required 0.405; the intervention is sequential and also acts on pretrained mass.Source: Table 6 · p. 9, §5.5 · pp. 9–10

5.6 Semantic Alignment and Its Attribution

  • Coherent delta heads number 41/96 on Charts, 74/96 on Flowers102, 71/96 on EuroSAT, 61/96 on Aircraft, and 16/72 on ViT-B/16.Charts counts range from 40 to 43 across three seeds, and replacing financial anchors with generic anchors reduces coherent heads from 41 to zero under fixed images and hyperparameters.Source: §5.6 · p. 10, Figure 4 · p. 10

5.7 Pretrained Readouts and Polysemy Controls

  • After adding explicit competitors, the pretrained chart audit finds only one vector passing both semantic criteria, and its financial-versus-portrait sense remains ambiguous.Under the specified dictionary, leading VO directions in the evaluated OpenAI checkpoints provide little unambiguous evidence of the target financial concepts.Source: §5.7 · pp. 10–11

6 Discussion

The discussion argues that reversion and retention answer different questions, and that matched controls and shared-error analysis clarify component-specific recovery. Semantic readouts can target weight structure for intervention, but their behavioral implications depend on the intervention substrate and implementation.

6 Discussion

  • A small reversion cost measures necessity for that edit in its surrounding network, not absence of functional contribution when retention recovers substantial improvement.Charts demonstrates the distinction in accuracy and loss, while its margin reduction shows why the evaluation metric must be specified.Source: §6 · p. 11
  • Count-matched random retention tests whether recovery depends on update selection, while co-correction on pretrained errors quantifies overlap in recovered behavior.The shared correct answers arise through the pretrained residual network, where attention and MLP are serial, jointly trained components.Source: §6 · p. 11
  • Semantic readouts identify task-aligned weight structure and provide a target for controlled intervention, while component retention measures recovery on a broader substrate.Readout-guided removal connects semantic selection to behavior at a finer scale; these substrates should remain explicit in editing, activation steering, and capability-removal studies.Source: §6 · p. 11
  • A practical evaluation should specify masks and projection choices, pair reversion with defined retention, report raw performance and normalized recovery, and compare structured with matched-random controls.For direction interventions, state whether directions are fixed or recomputed and how pretrained mass is affected.Source: §6 · p. 11

7 Limitations

The evidence is bounded to five CLIP classification setups using one shared fine-tuning recipe. Interpretation also depends on the evaluation metric and on which updates each intervention changes.

7 Limitations

  • The findings cover five CLIP classification setups under a recipe that fine-tunes the final six visual blocks with frozen text anchors and readout.Learning-rate changes, broader trainable surfaces, other model families, and per-layer retention remain untested.Source: §7 · p. 11
  • Operational redundancy varies by task and metric: paired-bootstrap intervals reflect evaluation-example variation conditional on each checkpoint, and random-retention controls do not match delta norm.Charts and EuroSAT show low VO necessity alongside substantial retention in accuracy and cross-entropy; margin reductions reveal contributions in every setting.Source: §7 · p. 11
  • Component retention measures recovery through the shared pretrained network, while shared corrections indicate overlapping behavior rather than identical internal computation.Readout-guided sequential projections can remove pretrained mass and are not direction-only retention, limiting the scope of those conclusions.Source: §7 · p. 11

8 Conclusion

Paired removal and retention show that small reversion costs can coexist with substantial retained recovery in the evaluated CLIP recipe. The paper also frames capability audits around edited-model behavior and provides a reproduction package with stated limits.

8 Conclusion

  • A small accuracy drop after reversion can coexist with substantial recovery when the same component’s updates are retained.The conclusion recommends evaluating readable updates through paired removal and retention, with explicit controls for selected parameters and intervention.Source: §8 · p. 12

Broader Impact Statement

  • Capability-removal audits should evaluate edited-model behavior and continuous scores directly, alongside retention controls where appropriate.A capability may persist after selected updates are removed, and a small measured accuracy change does not establish that those updates carried no useful adaptation.Source: Broader Impact Statement · p. 12

Reproducibility Statement

  • The reproduction package includes chart generation, training and intervention scripts, environment specifications, saved results, and per-example outputs.Checkpoint hashes are supplied, but checkpoint weights are not; model-level reruns require datasets and pretrained weights plus training or corresponding fine-tuned checkpoints.Source: Reproducibility Statement · p. 12

AI Assistance Statement

  • AI tools assisted with language and drafting; the authors state that experimental design, controls, analyses, and scientific judgments were their own.Source: AI Assistance Statement · p. 12

A Instruments, Data, and Intervention Accounting

The appendices specify model training and evaluation populations, semantic-readout sensitivity checks, intervention operators, and weight-only random controls. They also report five weights-only VO loss-recovery values.

A Instruments, Data, and Intervention Accounting

  • The primary model is OpenAI CLIP ViT-L/14, with the final six visual blocks trainable; the text tower, final visual LayerNorm, visual projection, and logit scale remain fixed.Training uses learning rate 10^-5 and batch size 48; Charts runs three epochs and standard datasets five.Source: A Instruments, Data, and Intervention… · p. 15
  • Chart intervention accuracy uses an independently rendered seed-4242 set of 780 images, on which the full model scores 98.46%.The training-time validation set is separately rendered with seed 999 and reports 98.3%.Source: A Instruments, Data, and Intervention… · p. 15
  • Standard evaluation sets are uniform random samples of 2,000 test examples; the fixed pretrained-error counts are 674, 426, 1,061, 1,379, and 606 in setting order.Source: A Instruments, Data, and Intervention… · p. 15
  • The semantic-readout sensitivity checks test the top two left singular vectors of each effective VO delta against ten greedy dictionary atoms and a 5,000-direction empirical null.The generic dictionary contains 1,076 ImageNet-derived and attribute atoms; financial probes include 41 chart phrases and 21 competing confounds.Source: A Instruments, Data, and Intervention… · p. 15
  • VO reversion restores V rows and input-projection biases plus O weight columns, while leaving the output-projection bias tuned; standard VO retention keeps that bias.The named VO reversion and retention operators therefore differ in output-bias scope.Source: A Instruments, Data, and Intervention… · p. 15
  • Weights-only VO loss recoveries are 0.745, 0.629, 0.820, 0.504, and 0.484 across the five settings.Random masks restore eligible updates before adding selected scalar weight deltas; ten seeds determine the masks.Source: A Instruments, Data, and Intervention… · p. 15
  • Direction surgery sequentially projects readout-selected directions out of O columns, with matched-random controls using the same number and head allocation of projections.The experiment starts from VO retention with LN1 kept, unlike the no-LayerNorm retention in Table 1.Source: A Instruments, Data, and Intervention… · p. 15

B Additional Results and Complete Continuous Metrics

The appendix reports chart intervention results, delta structure, and cross-corpus semantic-readout checks. Its figures organize component retention and reversion, semantic-readout counts, and the original chart intervention battery.

B Additional Results and Complete Continuous Metrics

  • Figure 5 presents VO and MLP accuracy recoveries alongside VO-reversion accuracy costs; Table 2 contains the complete VO, QK, and MLP necessity measurements.The caption notes larger MLP costs on Aircraft and ViT-B/16 but gives no values here.Source: Figure 5 · p. 16
  • Removing 56 readout-selected directions across 41 chart heads costs 0.029 accuracy, versus approximately zero for three matched-random controls; full VO reversion costs 0.000 on the primary seed.The targeted-versus-untargeted concept selectivity test passes for zero of six eligible concepts, and joint VO+QK reversion costs 0.009.Source: B Additional Results and Complete… · p. 16, Figure 7 · p. 17
  • Chart component weights have mean relative Frobenius changes of 0.006–0.009, an MLP-to-VO relative-write ratio of 1.03, and an effective VO relative change of 0.0105.The 56 hit directions cover eight trained concepts; leading matches are double top (19), volatility spike (13), and rising volume (11).Source: B Additional Results and Complete… · p. 17
  • The additional ViT-B/32 chart run reaches 0.986 accuracy but fails the 0.20 coherent-fraction replication criterion.It has six coherent heads of 72 and a zero-head null tune; its width-512 null threshold is approximately 1.22× the width-768 threshold in these runs.Source: B Additional Results and Complete… · p. 17
  • The four LAION-2B hits have alignments of 0.158–0.231 against a 0.151 null threshold, but missing per-step competitor traces limit semantic disambiguation.Figure 6 contrasts selected and null-passing vector counts across chart audits and technically coherent pretrained heads across corpora and architectures.Source: Figure 6 · p. 16, B Additional Results and Complete… · p. 17

B.1 Continuous metrics and recovery intervals

The continuous-metric analyses keep each checkpoint’s logit scale fixed and use paired bootstrap intervals conditional on that trained checkpoint. Agreement is assessed on pretrained errors, separating joint corrections from shared incorrect predictions.

B.1 Continuous metrics and recovery intervals

  • Cross-entropy and margin costs are reported in raw units, while normalized recovery divides by the corresponding base-to-full improvement.Raw margins should be compared within an architecture’s fixed-scale evaluation, not across architectures.Source: §B.1 · p. 17
  • Paired bootstrap intervals quantify evaluation-example variation conditional on each trained checkpoint through joint resampling.The analysis conditions on the fixed pretrained-error subset before resampling agreement.Source: §B.1 · p. 17
  • 345 of 1,379 Aircraft pretrained errors are corrected by both VO and MLP retentions, while 448 produce the same incorrect prediction.The joint-correction fraction uses the fixed pretrained-error subset as its denominator.Source: §B.1 · p. 17

C Analysis Protocol and Decision Record

The protocol record distinguishes original decision rules from revision-stage analyses, while documenting how the numerical tables and model-level results can be reproduced. The records describe saved-output CPU checks, but model-level replication requires training because checkpoint weights are not included.

C Analysis Protocol and Decision Record

  • The original chart audit’s direction-necessity gate required a drop of at least 15 accuracy points, twice the matched-random drop, and twofold targeted-concept selectivity; the observed 2.9-point drop failed it.The semantic-attribution checks passed: 41 versus zero coherent heads, cosine 0.175, and coverage of eight concepts.Source: C Analysis Protocol and Decision Record · p. 19
  • The original standard-data dissociation gate failed because every VO recovery exceeded 0.25; Aircraft replaced CUB-200 for availability, and EuroSAT replaced Pets for insufficient headroom.The required pattern was VO-revert cost at most 0.05, VO-only recovery at most 0.25, and MLP-only recovery at least 0.50 in at least two standard datasets.Source: C Analysis Protocol and Decision Record · p. 19
  • Revision-stage direction-removal criteria were met only by EuroSAT, and their confidence intervals were conditional on three controls.The original five-point bound was extended to the component matrix during revision; it was not an original all-component threshold.Source: C Analysis Protocol and Decision Record · p. 19
  • The follow-up configuration records that earlier accuracy results had been seen and specifies cross-entropy, margin, ten count-matched random masks, and direct VO-only retention analyses.Agreement was computed by post-processing saved predictions.Source: C Analysis Protocol and Decision Record · p. 19

D Reproducing the Numerical Tables

  • Saved prediction arrays support recomputation of accuracy contrasts, normalized recovery, paired-bootstrap intervals, and per-class counts; other continuous-metric estimates and agreement summaries are read from recorded files.The workflow regenerates eight component-analysis tables and associated accuracy and class-count JSON files.Source: D Reproducing the Numerical Tables · p. 19
  • CPU checks use distributed saved outputs, but model-level replication requires training because checkpoint weights are not supplied, and repeated training may vary with the compute environment.GPU training and intervention runs were not repeated during packaging.Source: D Reproducing the Numerical Tables · p. 20

Related papers

Loading 2610.34144v1…