PaperlayerView exact revision on arXiv

Source-linked AI summary

The Tokens Remember: When Tokenization Bypasses Knowledge Editing and Unlearning

Manit Baser, Aditya Nawal, Dinil Mon Divakaran, Mohan Gurusamy

arXiv:2609.29045v1cs.CRcs.AI

TL;DR

Open-weight model editing and unlearning face evaluation gaps: prior approaches can require the pre-edit model or auxiliary resources, and commonly assume canonical tokenization. Toketive searches alternative tokenizations using only the released model to detect modifications and reconstruct pre-edit responses; across evaluated models, datasets, and techniques, it reports a 38.6% average bypass rate.

  • Problem

    Existing evaluations of editing and unlearning can require the pre-edit model or auxiliary resources and assume canonical tokenization, leaving alternative representations unexamined.

    Source: §I · pp. 1–2, A. Limitations of Existing Methods · p. 4

  • Method

    Toketive searches alternative tokenizations of a query using the released model to detect modified facts and reconstruct pre-edit responses without pre-edit models or auxiliary training.

    Source: §I · p. 2

  • Results

    Across five models, six datasets, and six editing and unlearning techniques, the average bypass rate is 38.6%.

    Source: §VIII · p. 13

  • Takeaways & Limitations

    Tokenization variance is a previously overlooked attack surface for post-release knowledge control in open-weight models.

    Source: §I · p. 2

    Existing auditing methods may require the pre-edit model or shadow and auxiliary training, which post-hoc auditors may not possess.

    Source: A. Limitations of Existing Methods · p. 4

Abstract

from arXiv · show

Open-weight LLMs give downstream users control over the inference stack, but this flexibility can undermine post-release guarantees that sensitive knowledge has been modified or removed. Model editing and machine unlearning are used to modify or remove targeted knowledge without retraining models from scratch. However, existing security evaluations of these techniques face two critical limitations. First, they typically require access to either the original pre-edit/unlearning model or auxiliary classifiers to detect modifications or reconstruct pre-edit behavior. Second, they evaluate modifications under the canonical tokenization of an input, implicitly treating tokenization as a benign preprocessing step. We show that this assumption creates a security gap: the same input string can be represented by alternative valid tokenizations that induce different computational trajectories, allowing an adversary to bypass localized modifications and recover information intended to be suppressed. We introduce Toketive, a simple yet powerful reference-free attack that exploits the tokenization-based side channel to (i) detect modified knowledge and (ii) reconstruct the corresponding pre-edit response. It operates solely on the released model and requires neither the pre-edit model, training data, shadow models, nor auxiliary classifiers. Across five LLMs, six datasets, and six editing and unlearning techniques, we find that 38.6% of alternative tokenizations bypass the modification and recover the pre-edit response. Toketive detects modified facts with an F1 score of 84.2%, a 26.2% relative gain over the strongest baseline, and reconstructs pre-edit responses with 74.5% top-5 accuracy, 21.7% higher than the best baseline. Our results show that localized modifications should not be treated as robust knowledge-control boundaries without adversarial evaluation over alternative representations.

I. INTRODUCTION

Open-weight models enable targeted knowledge changes, but existing evaluations assume canonical tokenization and often require unavailable pre-edit models or auxiliary resources. Alternative valid tokenizations can follow different computational paths and bypass localized edits, motivating Toketive’s reference-free detection and reconstruction attack.

I. INTRODUCTION

  • Open-weight releases give downstream users control over the inference stack, while editing and unlearning offer targeted alternatives to costly retraining.Such modifications can remove personal information, correct outdated knowledge, or mitigate unsafe behavior.Source: §I · p. 1

A. Model Editing and Machine Unlearning

  • Editing and unlearning target specific knowledge without retraining, seeking localized changes that suppress or replace a fact while preserving unrelated behavior.Editing replaces an association; unlearning suppresses it, and both aim to limit collateral effects.Source: §I · p. 1, A. Model Editing and Machine Unlearning · pp. 2–3

I. INTRODUCTION

  • Existing security evaluations often require the pre-edit model or auxiliary resources and assume queries use the canonical tokenization applied during editing.This leaves alternative representations of the same input outside the assumed evaluation setting.Source: §I · p. 2

B. Noncanonical tokenization

  • Different valid tokenizations of the same string induce different computational trajectories, and noncanonical paths can bypass an edit and recover the original response.A tokenization preserves the input string while dividing it into a valid sequence of subword tokens; the canonical form is only one such representation.Source: §I · p. 2, B. Noncanonical tokenization · p. 3

I. INTRODUCTION

  • Across five models, six datasets, and six editing and unlearning techniques, Toketive reaches 84.2% detection F1 and 74.5% top-5 accuracy for pre-update response recovery.Detection is a 26.2% relative gain over the strongest baseline; reconstruction is 21.7% higher than the best baseline.Source: §I · p. 2

C. Representational Entanglement

The paper describes representational similarity as a signal for finding useful alternative tokenizations, then frames an attacker with access to a released open-weight model. Toketive unifies edit detection and pre-edit response reconstruction through search over tokenizations.

C. Representational Entanglement

  • At the probe layer, representational entanglement is cosine similarity between prompt representations; Toketive seeks an intermediate similarity range to balance bypassing updates with retaining the target association.Tokenizations too close to the canonical representation may remain affected by the update, while very distant ones may lose the relevant association.Source: C. Representational Entanglement · p. 3

III. THREAT MODEL

  • The threat model assumes access to the released model’s parameters and activations, and excludes fine-tuning-based modification approaches from scope.The attacker may query arbitrary token sequences and inspect activations, but lacks the pre-edit response, knowledge of the modification technique, and shadow models or auxiliary training facilities.Source: §III · pp. 3–4

IV. TO K E T I V E: A NEW ADVERSARIAL ATTACK ON EDITING & UNLEARNING

  • Toketive searches noncanonical tokenizations that preserve the input string but follow alternate computational paths to detect and recover modified knowledge.The detection and reconstruction goals are coupled in one search procedure.Source: §IV · p. 4, §III · p. 4

A. Limitations of Existing Methods

Existing auditing methods are constrained by access to the pre-edit model or auxiliary training, and some cannot reconstruct pre-edit responses. Toketive uses alternative valid tokenizations of the same prompt to detect and recover modified knowledge, with adaptive sampling guided by representational entanglement.

A. Limitations of Existing Methods

  • Some auditing methods require the original pre-edit model, which may be unavailable to post-modification auditors or adversaries.Examples include methods that use the original model to obtain training data or analyze parameter differences.Source: A. Limitations of Existing Methods · p. 4
  • Methods avoiding the pre-edit model may still require shadow models, auxiliary classifiers, training data distributions, or knowledge of the modification algorithm.These requirements impose training overhead and may be unavailable to post-hoc auditors.Source: A. Limitations of Existing Methods · p. 4
  • Among methods without pre-edit model access, the cited approaches detect or identify modifications but do not reconstruct the suppressed response.For deletion-request audits, verifying that a modification occurred does not establish what was suppressed or whether it remains accessible.Source: A. Limitations of Existing Methods · p. 5

B. The Toketive Attack

  • Toketive searches alternative tokenizations of the exact same prompt, exploiting distinct computational paths that can retain the original association after modification.It takes the post-edit model and a target prompt and returns a detection label plus a reconstructed response when the fact is judged modified.Source: B. The Toketive Attack · p. 5

1) Adaptive Tokenization Sampler:

  • The adaptive sampler varies tokenization aggressiveness and uses representational entanglement to route valid candidates, without relying on a particular theory of knowledge localization.Candidates must decode to the original string; the sampler extracts the subject’s final-token representation at layer L = ⌊depth/3⌋, and entanglement is used as an algorithmic routing heuristic rather than a causal account.Source: 1) Adaptive Tokenization Sampler · pp. 5–6

2) Phase ①(Edit/Unlearning Detection):

The detection phase tests whether alternative tokenizations still produce the canonical post-modification answer. If the modification is detected, reconstruction filters out those answers and uses the remaining bypass continuations to recover the pre-edit response.

2) Phase ①(Edit/Unlearning Detection):

  • Toketive tests sufficiently divergent but informative tokenizations for the canonical post-modification answer; a low bypass rate indicates that alternatives often avoid it.The detection band is intended to differ enough from the canonical tokenization to potentially bypass patched circuits without losing semantic signal.Source: 2) Phase ①(Edit/Unlearning Detection) · pp. 6–7

3) Phase ②(Pre-Edit Reconstruction):

  • For reconstruction, Toketive samples moderately entangled tokenizations, discards completions containing the post-update answer, and plurality-votes the remaining object spans.Top-k accuracy measures whether the ground-truth pre-edit object appears among the most frequent candidates.Source: 3) Phase ②(Pre-Edit Reconstruction) · p. 7

V. EXPERIMENTS

The experiments examine predictability, tokenization invariance, and adversarial detection and reconstruction across varied models, datasets, and editing or unlearning techniques. The setup standardizes prompts and uses deterministic decoding to isolate tokenization effects.

V. EXPERIMENTS

  • The experiments ask whether entanglement predicts response convergence, whether modifications generalize across tokenizations, and whether an adversary can detect and reconstruct without pre-edit access.The questions are evaluated across five language models, six benchmarks, and six editing or unlearning techniques.Source: §V · p. 8

A. Experimental Setup

  • The setup spans six benchmarks for factual editing and unlearning and evaluates five language models with six modification techniques.The models include Llama3, Llama3.1, OLMo2, Tulu3, and Tulu3.1; the techniques include AlphaEdit, RECT, MEMIT, PRUNE, CoME, and LTU.Source: A. Experimental Setup · p. 8
  • Evaluation uses standardized prompt formats and deterministic greedy decoding, and retains only instances answered correctly under canonical tokenization across experiments.CounterFact uses its first 1,000 samples; MQuAKE and RippleEdits each contribute 1,000 factual triples.Source: A. Experimental Setup · p. 8

B. Q1: Predictability of Tokenization Convergence

This section asks whether internal signals predict which alternative tokenizations preserve the canonical factual object. Representational entanglement predicts convergence more consistently than token-level edit distance.

B. Q1: Predictability of Tokenization Convergence

  • For each prompt, the study samples 30 alternative segmentations that decode to the same string and tests whether the original factual object appears in each response.It evaluates representational entanglement and normalized token-sequence Levenshtein distance as convergence predictors.Source: B. Q1: Predictability of Tokenization… · pp. 8–9
  • AUC 0.684–0.850 and |rb| 0.368–0.699 for representational entanglement, versus 0.526–0.674 and 0.052–0.348 for edit distance; Fig. 4 averages results across five models.Across model-dataset combinations, even the lowest entanglement AUC exceeds the highest edit-distance AUC.Source: Figure 4 · p. 9, Table II · p. 9, B. Q1: Predictability of Tokenization… · p. 9
  • For Llama3.1 and Tulu3 on Real Authors, convergent tokenizations show clearer separation by representational entanglement, while edit-distance distributions overlap substantially.The entanglement ordering indicates that internal representational similarity tracks factual retrieval better than token-level surface similarity.Source: Figure 5 · p. 9, B. Q1: Predictability of Tokenization… · p. 9
  • On RippleEdits, edit distance approaches random performance while representational entanglement remains reliable; the entanglement score is Toketive’s adaptive-sampler routing criterion.Source: B. Q1: Predictability of Tokenization… · pp. 9–10

C. Q2: Tokenization Invariance of Editing and Unlearning

Alternative tokenizations frequently bypass editing and unlearning, and Toketive detects modified facts and reconstructs pre-update responses. Its evaluation reports substantial bypass rates and stronger results than the compared baselines.

C. Q2: Tokenization Invariance of Editing and Unlearning

D. Q3: Adversarial Efficacy of Toketive

  • Evaluation restricts detection and reconstruction to successful edits; detection compares random, edit-distance, and post-edit gradient baselines, while reconstruction excludes the gradient baseline.Successful edits satisfy P(new fact) > P(old fact).Source: D. Q3: Adversarial Efficacy of Toketive · p. 11
  • 84.2% average F1 makes Toketive the strongest edit detector, versus 66.7% for edit distance across the evaluated models and datasets.The FUMA-adapted gradient baseline averages 60.8% F1.Source: D. Q3: Adversarial Efficacy of Toketive · p. 11
  • 74.5% top-5 accuracy for Toketive reconstructs pre-edit responses, outperforming other baselines by at least 21.7%.The advantage over edit distance is most pronounced on MQuAKE and CounterFact.Source: D. Q3: Adversarial Efficacy of Toketive · pp. 11–12

C. Q2: Tokenization Invariance of Editing and Unlearning

VI. RELATED WORK

Prior work frames editing and unlearning as targeted knowledge modification, but security evaluations largely assume canonical tokenization. The discussion and conclusion show that alternative tokenizations expose residual knowledge, while adaptive updates trade locality for robustness.

VI. RELATED WORK

  • Editing updates targeted factual associations through locate-then-edit, while unlearning removes training-data influence and is also framed as targeted editing,,.Prior work also evaluates reliability, generalization, locality, unlearning efficacy, utility, and hallucination avoidance.Source: §VI · p. 12
  • Security evaluations of editing and unlearning have largely used canonical tokenization, while recent data-centric metrics still omit tokenization-aware adversaries.Prior work examined membership inference,, gradient signals, training-data extraction, update reverse engineering [39], and text reconstruction.Source: §VI · p. 12

VII. DISCUSSION: TOWARD TOKENIZATION-ROBUST EDITING AND UNLEARNING

  • Adaptive MEMIT reduces bypass rate from 0.32 to 0.04 after four iterations, while ripple effect rises from 1.0× to 8.6×.Valid tokenization space grows with input length, and repeated updates may require broader parameter changes that undermine locality,,.Source: §VII · p. 13

VIII. CONCLUSION

  • Across five models, six datasets, and six techniques, Toketive finds a 38.6% average bypass rate, with 84.2% detection F1 and 74.5% top-5 reconstruction accuracy.The conclusion reports gains over strongest baselines of 26.2% for detection and 21.6% for reconstruction.Source: §VIII · p. 13

ETHICS CONSIDERATIONS

  • The study limits its threat model to querying public models with alternative tokenizations and uses public benchmarks without sensitive real-world data.It focuses on structural vulnerabilities and frames the work around evaluation and defense rather than attack deployment.Source: ETHICS CONSIDERATIONS · p. 14

OPEN SCIENCE AND RESPONSIBLE DISCLOSURE

  • The artifact repository provides Toketive source code, experiment scripts, configurations, workflow documentation, and dataset information for reproducibility.The evaluated models and datasets are publicly available.Source: OPEN SCIENCE AND RESPONSIBLE DISCLOSURE · p. 17

APPENDIX A ADAPTIVE EDITING AND UNLEARNING AGAINST

Adaptive MEMIT incorporates discovered bypass tokenizations into later unlearning updates, reducing bypasses while increasing collateral changes. Fine-tuning also lowers bypasses, but its broader updates come with a larger ripple effect.

TOKENIZATION-BASED BYPASSES

  • Adaptive MEMIT first unlearns the canonical prompt, then searches alternative tokenizations and reapplies the same unlearning target when one still elicits the original response.The experiment evaluates Llama3 on 1,000 edit-control pairs with representational entanglement greater than 0.9, and stops when no bypass is found or the iteration limit is reached.Source: TOKENIZATION-BASED BYPASSES · pp. 17–19
  • 0.32 to 0.04: adaptive MEMIT lowers bypass rate after four iterations, while ripple effect rises from 1.0× to 8.6×.The reported intermediate bypass rates are 0.14 after two iterations and 0.09 after three; corresponding ripple effects are 3.3× and 6.7×.Source: Table VIII · p. 19, TOKENIZATION-BASED BYPASSES · p. 19

APPENDIX B FINE-TUNING FOR TARGETED UNLEARNING AND EDITING

  • 0.32 to 0.08: fine-tuning lowers bypass rate relative to MEMIT, but produces a 5.2× larger ripple effect.The experiment uses 1,000 highly entangled edit-control pairs on Llama3; the authors note collateral effects make fine-tuning less suitable when locality is a primary objective.Source: Table IX · p. 19, Appendix B · p. 19

APPENDIX C TOKENIZATION SAMPLING PROCEDURE

The appendices specify Toketive’s sampling constraints, computational cost, and fixed hyperparameter calibration. Sampling requires exact round-trip validity, and the calibrated settings are applied without per-model or per-dataset retuning.

APPENDIX C TOKENIZATION SAMPLING PROCEDURE

  • Candidate segments are retained only when they round-trip exactly to the original text; sampling allows up to 300 attempts per requested tokenization.The quotas are 20 samples for detection and 50 for reconstruction; validated out-of-band proposals can fill a shortfall when the attempt budget is exhausted.Source: Appendix C · pp. 19–20

APPENDIX D COST AND EFFICIENCY ANALYSIS OF TOKET I V E

  • Toketive’s per-fact cost is O(M(ndet + nrec) · F + (ndet + nrec) · G), where G typically exceeds F.F is one forward-pass cost and G is autoregressive generation cost; the random and edit-distance baselines omit the scoring F-term.Source: Appendix D · p. 20

APPENDIX E HYPERPARAMETER CALIBRATION

  • τdet is fixed at 0.10 after calibration on Llama3 with MEMIT on Known-1000, where F1 peaks at 82.9%.The threshold and band boundaries are held fixed across all 180 experiments without per-model or per-dataset tuning; the detection entanglement band targets [0.30, 0.50).Source: Appendix E · p. 20

APPENDIX F MEAN SCORE ANALYSIS

Convergent tokenizations consistently score higher than non-convergent ones on representational entanglement, whereas edit distance separates them only weakly. The detection threshold’s F1 peaks at 0.10.

APPENDIX F MEAN SCORE ANALYSIS

  • 0.124–0.278: convergent tokenizations score higher on representational entanglement in all 30 model-dataset combinations, with the largest gaps exceeding 0.20 for most models on Real Authors and MQuAKE.The table reports mean scores for convergent and non-convergent tokenizations and their difference.Source: Appendix F · pp. 20, 22, Table XI · p. 21
  • 82.9%: detection F1 peaks at τdet = 0.10 and declines monotonically at larger thresholds.The figure averages performance across models, datasets, and techniques; its high-threshold plateau reflects the degenerate case where all facts are labeled edited.Source: Figure 9 · p. 22
  • −0.011 to −0.094: edit-distance differences are negative and small, typically 0.03–0.07, between non-convergent and convergent tokenizations.The reported pattern indicates that edit distance does not reliably separate the two groups.Source: Appendix F · p. 22
  • Representational entanglement provides a consistent directional signal for factual retrieval behavior, supporting its use as Toketive’s adaptive-sampler routing criterion.The difference is positive across all 30 model-dataset combinations, while edit-distance separation is marginal.Source: Appendix F · p. 22

APPENDIX G DISTRIBUTIONAL VISUALIZATIONS

Across datasets, representational entanglement separates convergent from non-convergent tokenizations better than edit distance. Its calibration is generally stronger, and qualitative examples show that alternative tokenizations can recover a pre-edit answer even when the canonical path returns the edited answer.

APPENDIX G DISTRIBUTIONAL VISUALIZATIONS

  • Representational entanglement shifts higher for convergent tokenizations, while edit-distance distributions overlap substantially; ROC curves also favor entanglement across the evaluated settings.The distributions have clearly separated interquartile ranges in most settings, whereas edit-distance medians are near-identical.Source: Appendix G · p. 22

APPENDIX H CALIBRATION ANALYSIS

  • On TOFU-World Facts, edit-distance ECE exceeds 0.4 across all models, while representational-entanglement ECE remains below 0.25.Across all five models and six datasets, representational entanglement has lower ECE than edit distance.Source: Appendix H · pp. 22–23
  • Representational entanglement has lower MCE than edit distance in most model-dataset combinations, with OLMo2 exceptions on Real Authors and CounterFact.For OLMo2, edit-distance MCE is 0.330 vs. 0.436 on Real Authors and 0.250 vs. 0.263 on CounterFact; entanglement retains lower MCE on the other four datasets.Source: Appendix H · p. 23

APPENDIX I QUALITATIVE EXAMPLES OF TOKENIZATION-AWARE

  • In the Llama3 AlphaEdit examples, a noncanonical tokenization recovers George Orwell while the canonical tokenization produces the edited answer, George R.R. Martin.Another noncanonical tokenization instead returns Michael Moorcock, showing that not every departure from the edited behavior recovers the original answer.Source: Appendix I · p. 23, Table XII · p. 23

INVARIANCE OF EDITING AND UNLEARNING

Across models, datasets, and techniques, noncanonical tokenizations often recover pre-update answers, while the post-edit answer appears for only a minority of them in most settings. The appendix reports these outcomes as New, Old, and Other responses.

INVARIANCE OF EDITING AND UNLEARNING

  • No technique achieves near-zero bypass rates across the evaluated settings, and noncanonical tokenizations produce the post-edit answer only in a minority of settings.The Other response fraction is largest for LTU across most settings.Source: INVARIANCE OF EDITING AND UNLEARNING · p. 23
  • Table XIII breaks noncanonical-tokenization responses into New, Old, and Other fractions; Old (%) is the bypass rate because it counts pre-update answers.New (%) counts post-edit answers, while Other (%) counts responses matching neither answer.Source: INVARIANCE OF EDITING AND UNLEARNING · p. 23, Table XIII · p. 24
Loading 2609.29045v1…