PaperlayerView exact revision on arXiv

Source-linked AI summary

Sleeping Secrets: How Fine-Tuning Reawakens Privacy Risks in Language Models

Jianhong Li, Jiahao Chen, Yuwen Pu, Chunyi Zhou, Oubo Ma, Zhou Feng, Hangtao Zhang, Jichao Bi, Chunqiang Hu

arXiv:2610.01365v1cs.CR

TL;DR

Prior fine-tuning recovery attacks relied on genuine private supervision, leaving open whether model-generated supervision alone can recover previously learned associations. ReGap generates candidates, selects them by answer-token likelihood, and adapts the target model; across six settings, it improves recovery over the post-training target model. The results also show that prior target exposure matters and that recovery is limited by association structure and explicit NPO unlearning.

  • Problem

    Prior fine-tuning recovery attacks require genuine private supervision, leaving open whether model-generated supervision can recover previously learned private associations without it.

    Source: §1 · p. 2

  • Method

    ReGap generates task-consistent candidates from the target model, selects them by answer-token likelihood under that frozen model, and trains a fresh LoRA adapter without target answers or genuine private records.

    Source: §1 · p. 2

  • Results

    6–21 percentage points: one ReGap round improves target-association recovery over the target model across six GPT-2, OPT, and Qwen3 settings.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Recovery can persist with disjoint adaptation identities and no exact target answers in the supervision, while a matched prior-exposure control shows no gain on a checkpoint that never encountered the targets.

    Source: §1 · p. 2, §7 · p. 10

    Recovery weakens sharply when structured Enron-derived associations are replaced by opaque random bindings, and explicit NPO unlearning yields almost no additional recovery.

    Source: §6 · p. 9

Abstract

from arXiv · show

Beyond adapting Large Language Models (LLMs) to specialized applications, fine-tuning has recently been shown to recover private information that is no longer accessible through direct queries. Previous fine-tuning recovery attacks, however, require genuine private supervision drawn from the same distribution, i.e., the previous training dataset. We argue that such recovery remains possible without such impractical knowledge. We show that LLM-generated candidates can provide sufficient supervision to recover previously learned private associations. Based on this, we propose ReGap, a data-free attack that recovers private associations using task structure, filters them by answer-token likelihood, and updates the target model via low-rank adaptation. Specifically, ReGap requires neither target answers nor auxiliary genuine private supervision. Across six GPT-2, OPT, and Qwen3 models, ReGap improves target-association recovery by 6-21 percentage points over the post-training target model. Recovery remains substantial even when the adaptation identities are disjoint from all memorized and evaluation identities, with no exact target answers appearing in the generated or selected supervision. Moreover, the same trained adapters increase recovery from 42\% to 63\% on a previously exposed checkpoint, but produce no gain on a matched checkpoint that never encountered the targets. This contrast shows that adaptation alone is insufficient to explain the observed recovery and that prior target exposure strongly affects post-adaptation recoverability. Our findings highlight that routine model customization can reawaken latent privacy risks, warranting urgent attention from the academic and industrial communities.

1 INTRODUCTION

ReGap asks whether previously learned private associations can become more recoverable using model-generated supervision alone, without target answers or genuine private records. Across six model settings, the paper reports improved recovery and evidence that prior target exposure matters, while identifying limits tied to association structure and unlearning.

1 INTRODUCTION

  • Earlier recovery work used genuine private association pairs, leaving recovery without authentic private records as an unexplored attacker capability.The attacker considered here has the released post-training model but not the original model or genuine private records.Source: §1 · p. 2
  • ReGap generates task-consistent candidates, selects them by answer-token likelihood under the frozen target model, and trains a fresh LoRA adapter without target answers or genuine private records.The paper also provides a local gradient-based analysis of updates without identity or answer overlap.Source: §1 · p. 2
  • 6–21 percentage points: one ReGap round improves target-association recovery over the target model across six GPT-2, OPT, and Qwen3 settings.The paper also reports substantial recovery with disjoint adaptation identities and no exact target answers in generated or selected supervision.Source: §1 · p. 2
  • Recovery weakens sharply for opaque random bindings and after explicit NPO unlearning, delimiting when model-generated supervision supports recovery.The reported characterization spans GPT-2, OPT, and Qwen3 models.Source: §1 · p. 2

2 RELATED WORK

Related work distinguishes memorized information from what a fixed checkpoint directly reveals, and describes privacy extraction through inference-time interaction. Prior studies also show that adaptation can affect privacy; ReGap addresses recovery using supervision constructed from the target model itself.

2.1 MEMORIZATION AND BEHAVIORAL ACCESSIBILITY

  • Memorized information includes entity–attribute associations, while reduced direct accessibility establishes neither deletion nor intact internal storage.The paper uses “Hidden” as a behavioral notion rather than a claim about internal information removal.Source: §2.1 · p. 3

2.2 PRIVACY EXTRACTION FROM LANGUAGE MODELS

  • Fixed-checkpoint extraction studies elicit memorized text or PII through prompting, prompt optimization, or large-scale querying, while lexical cues can also support attribute reconstruction.Successful prediction of a private attribute does not necessarily establish item-specific memorization.Source: §2.2 · p. 3

2.3 POST-ADAPTATION PRIVACY RISKS

  • Prior work finds that fine-tuning can restore access to PII using genuine private pairs, while later updates can also affect privacy and unlearning outcomes.ReGap instead constructs supervision from the target model without target answers or genuine private association pairs.Source: §2.3 · p. 3

3 PROBLEM FORMULATION

The problem is whether fine-tuning can make previously learned private associations more accessible after direct recovery has declined. The attacker receives only the post-training target model and constructs auxiliary supervision from its own outputs under a known task schema.

3 PROBLEM FORMULATION

  • The target associations are query-key and private-attribute pairs, with a fixed task template converting each key into an evaluation query and target sequence.Recovery is assessed through generated continuations under a fixed evaluation protocol.Source: §3 · p. 3
  • The post-training target model has lower direct recovery than the original model, but this behavioral difference establishes neither deletion nor intact internal retention.The attacker receives only the post-training model.Source: §3 · p. 4
  • The attacker knows the association schema and answer format, can query and fine-tune the target model, and uses generated answers without genuine private pairs or target answers.Ground truth is reserved for evaluation and post-hoc analysis, not attack construction or model selection.Source: §3 · p. 4
  • Auxiliary identities may include known target keys or, in the Unseen-Identity Control, be disjoint from memorization and evaluation keys.No distributional match between externally supplied auxiliary data and targets is required.Source: §3 · p. 4

4 FROM RECOVERY GEOMETRY TO REGAP

The section motivates ReGap by analyzing how auxiliary updates may affect target loss and how candidate selection can use observable model support. It then specifies a Generate–Select–Adapt procedure using model-generated candidates and a fresh adaptation step.

4 FROM RECOVERY GEOMETRY TO REGAP

  • ReGap is a Generate–Select–Adapt pipeline that constructs fine-tuning supervision from the target model itself.The overview figure presents the three stages as a procedure for recovering hidden associations.Source: §4 · p. 4, Figure 1 · p. 5

4.1 LOCAL GEOMETRY OF FINE-TUNING

  • Auxiliary updates can decrease target loss without reproducing target answers when their gradients align sufficiently and the alignment outweighs the update penalty.The paper cautions that lower target loss alone does not imply higher recovery.Source: §4.1 · pp. 4–5

4.2 FROM SELF-GENERATED ANSWERS TO AN OBSERVABLE SELECTOR

  • Exact on-policy sampling with matched complete-sequence likelihood has zero expected gradient at the generating model, so ReGap uses decoding and filtering choices to shape its update.A nonzero update benefits targets only when it is aligned with them.Source: §4.2 · p. 5
  • Under the conditional support–alignment ordering assumption, selecting the K largest support scores maximizes expected target–auxiliary gradient alignment among score-based selectors.This is a conditional justification, not a guarantee that support ranks alignment in every model; target-loss improvement also requires alignment sufficient to dominate the update penalty.Source: §4.2 · p. 6

4.3 REGAP: GENERATE–SELECT–ADAPT

  • ReGap generates greedy and temperature-controlled top-p answers, removes malformed or duplicate candidates, and does not check candidates against true private attributes.The same candidate construction applies to target-aware and unseen-identity auxiliary keys.Source: §4.3 · p. 6
  • The method retains up to a prescribed budget of candidates with the smallest selection loss, then fine-tunes from the target model on selected answer tokens.Generation and scoring use the unchanged target model, and budgets are fixed without target-answer validation or recovery feedback.Source: §4.3 · p. 6

5 EVALUATION

The evaluation tests ReGap across six model settings, then probes how recovery depends on target-specific information, candidate selection, and prior target exposure. ReGap improves recovery over unadapted models, including when adaptation identities and answers do not reveal the targets.

5.1 EXPERIMENTAL SETUP

  • The benchmark uses 3,000 Enron-derived person–email associations and six open-source GPT-2, OPT, and Qwen3 models.The evaluation reserves 100 associations as targets, uses one Generate–Select–Adapt round with K = 50 unless noted, and compares against baselines for direct recovery, random candidate selection, and genuine-private supervision.Source: §5.1 · pp. 6–7

5.2 RQ1: RECOVERY WITH MODEL-GENERATED SUPERVISION

  • One ReGap round improves recovery over unadapted models by 6–21 percentage points across six settings.Figure 2 summarizes the six settings; on GPT-2 Medium, recovery rises from 42/100 to 63/100. Genuine private supervision achieves higher recovery overall, but ReGap comes within 3.3 and 5.0 percentage points on GPT-2 Medium and Large.Source: §5.2 · p. 7, Figure 2 · p. 8

5.3 RQ2: HOW MUCH TARGET-SPECIFIC INFORMATION IS REQUIRED?

  • On GPT-2 Medium and Large, recovery improves with disjoint adaptation identities and no exact target answers in generated candidates or selected examples.Across evaluated seeds, the gains are 15.3 and 14.0 percentage points, respectively; the audits find zero target-answer hits in candidate pools and selected examples.Source: Table 1 · p. 7, §5.3 · p. 8
  • Recovery benefits from task- and entity-conditioned queries, remains substantial under several English prompt reformulations, and is sensitive to prompt wording.Shuffled entity-conditioned queries perform nearly identically to original queries, whereas generic queries reduce recovery to pre-adaptation levels; the Chinese prompt is also near that level.Source: §5.3 · p. 8, Table 2 · p. 8

5.4 RQ3: DOES CANDIDATE SELECTION MATTER?

  • ReGap exceeds Matched Random by 0.7–12.0 percentage points, but Matched Random also improves every setting, making selection an amplifier rather than the sole source of recovery.The selection benefit varies by model, and the authors do not claim universal or statistically significant superiority over random selection.Source: §5.4 · pp. 8–9

5.5 MATCHED PRIOR-EXPOSURE CONTROL

  • The same adapters raise recovery from 42.0% to 63.0% on a previously exposed checkpoint but produce no gain on the matched unexposed checkpoint, which remains at 13.0%.The fixed-adapter comparison indicates that adaptation alone is insufficient to explain recovery; prior target exposure strongly affects post-adaptation recoverability.Source: §5.5 · p. 9

6 DISCUSSION

The discussion interprets post-adaptation recovery as dependent on prior target exposure, while emphasizing that the evidence is behavioral and that recovery has important boundaries. The experiments focus on structured associations and do not establish general robustness to unlearning.

6 DISCUSSION

  • Applying the same adapters increases recovery on a previously exposed checkpoint but produces no gain on a matched checkpoint that never encountered the targets.The comparison supports a role for training history, but does not show that memories remain internally intact or unchanged.Source: §6 · p. 9, Table 3 · p. 9
  • Recovery weakens sharply for opaque random bindings and after explicit NPO unlearning, but this does not establish general robustness to unlearning.The main evaluation focuses on structured Enron-derived person–email associations with exploitable lexical and compositional regularities.Source: §6 · p. 9
  • The study assumes knowledge of the association task and expected answer format, and evaluates open-source models that allow token-level scoring and parameter-efficient adaptation.The authors identify less structured information, additional defenses, and restricted fine-tuning interfaces as future research settings.Source: §6 · p. 9

7 CONCLUSION

The conclusion reports that model-generated supervision can restore access to previously learned private associations without genuine private records. The results depend on the associations and updates involved, and motivate evaluating both direct disclosure and later customization.

7 CONCLUSION

  • Across GPT-2, OPT, and Qwen3 settings, one ReGap round improves target-association recovery by 6–21 percentage points.Recovery also persists with disjoint adaptation identities and no exact target answers in candidate pools or selected supervision.Source: §7 · pp. 9–10
  • Recovery weakens sharply for opaque random bindings and after explicit NPO unlearning, while matched prior-exposure results support a role for training history.The conclusion describes ReGap’s effectiveness as dependent on the associations being recovered and the updates the model has undergone.Source: §7 · p. 10
  • The absence of externally supplied private records is insufficient to rule out recovery, so privacy evaluation should also assess whether customization makes previously learned information accessible.The conclusion includes customization data generated by the model itself as a setting to assess.Source: §7 · p. 10

AI USE STATEMENT

The appendix details the Enron-derived benchmark and its controls, then specifies the post-training, candidate-selection, and adaptation procedures used in the experiments.

AI USE STATEMENT

  • Generative AI tools assisted research, design feedback, coding, interpretation, translation, and manuscript preparation; authors manually reviewed outputs and checked results and citations.The authors take responsibility for the final paper content.Source: AI USE STATEMENT · p. 10

A BENCHMARK AND DATA CONSTRUCTION

A.3 UNSEEN-IDENTITY CONTROL

  • The Unseen-Identity Control uses identities disjoint from memorized and evaluation identities while preserving the person-email task structure.Audits found no exact target answers in generated candidates or selected examples.Source: §A.3 · p. 16

A.4 OPAQUE RANDOM-BINDING CONTROL

  • The opaque control pairs synthetic identities with random 12-character email local parts, removing the lexical relationship between names and answers.It is a structural boundary, not a strictly matched causal ablation, because it uses independently trained checkpoints.Source: §A.4 · p. 16

A.5 ADDITIONAL ENRON CONTROLS

  • Additional Enron variants include independently extracted clean name-address pairs and 714 non-Enron-domain associations that may retain lexical relationships.These are descriptive distributional controls, not the headline results or an unrelated-domain benchmark.Source: §A.5 · pp. 16–17

B.1 POST-TRAINING MODEL CONSTRUCTION

  • The post-training model is produced by continual training on WikiText-2 without target associations, and a matched target-unexposed checkpoint excludes targets from its entire training history.The same trained adapters are transferred to the matched checkpoint without further candidate generation, selection, or training.Source: §B.1 · p. 17

B.2 CANDIDATE GENERATION; B.3 ORIGINAL-LOSS CANDIDATE SELECTION

  • ReGap generates candidates with the frozen target model, filters invalid outputs, and ranks candidates by answer-token average negative log-likelihood under that model.The selector uses no target-answer labels, target gradients, or recovery-set performance.Source: §B.2 · p. 17, §B.3 · p. 17

B.4 LORA ADAPTATION; B.6 RANDOM BASELINE AND FAIRNESS CONTROLS

  • Each run trains a fresh LoRA adapter on generated answer tokens; the matched Random Selection baseline differs only in its candidate-selection rule.The main paper reports one Generate–Select–Adapt round, while additional rounds do not consistently improve recovery across models.Source: §B.4 · p. 18, §B.6 · p. 18

B.7 EVALUATION PROTOCOL

Recovery is measured on held-out target associations using normalized substring matching. The prior-exposure comparison shows an increase after adaptation only for the checkpoint that previously encountered the targets.

B.7 EVALUATION PROTOCOL

  • A target counts as recovered when its normalized, nonempty reference answer appears as a substring in the normalized generated continuation.Email-format matching is computed separately and does not affect the primary recovery metric.Source: §B.7 · p. 19
  • Stochastic adaptation results are reported as the mean and sample standard deviation over seeds 42, 43, and 44; target answers are reserved for evaluation and audits.They are not used for candidate generation, scoring, selection, or LoRA adaptation.Source: §B.7 · p. 19
  • 42.0% to 63.0%: recovery increased after ReGap adaptation on the previously exposed checkpoint, while the target-unexposed checkpoint remained at 13.0%.Both checkpoints were evaluated on the same 100 target associations with the same decoding and recovery procedure.Source: §B.7 · p. 19

B.8 MAIN-PROTOCOL SUMMARY; C FULL RECOVERY RESULTS

  • Additional adaptation rounds do not uniformly benefit models, and the full-results section reports per-seed one-round outcomes, multi-round dynamics, and LoRA versus full-parameter fine-tuning.Table 9 summarizes per-seed one-round recovery under the Original-Loss protocol; Random Selection and ReGap share pools and budgets but differ in selection.Source: Table 10 · p. 20, C FULL RECOVERY RESULTS · p. 20, Table 9 · p. 20

C.1 PER-SEED RESULTS ACROSS SIX MODELS

Supplementary analyses examine the consistency and boundaries of recovery across seeds, adaptation rounds, supervision baselines, and gradient diagnostics. They report model-dependent selector benefits, no consistent gains from repeated updates or full fine-tuning, and a positive but non-causal association between candidate compatibility and target-gradient alignment.

C.1 PER-SEED RESULTS ACROSS SIX MODELS

  • ReGap exceeds the target model in every evaluated seed across all six settings; Matched Random Selection improves over Hidden in most settings.The per-seed results indicate the gains are not driven by a single favorable run.Source: §C.1 · p. 20
  • ReGap-Random gains are model dependent: 1.3 points on GPT-2 Small, 10.0 on GPT-2 Medium, 12.0 on GPT-2 Large, 6.7 on OPT-1.3B, 0.7 on Qwen3-1.7B, and 3.0 on Qwen3-4B.The authors interpret loss-based selection as an additional filtering mechanism, not the sole source of recovery.Source: §C.1 · p. 20

C.2 MULTI-ROUND RECOVERY DYNAMICS

  • Additional rounds do not consistently improve recovery; GPT-2 Medium, GPT-2 Large, and OPT-1.3B peak after Round 1, motivating the one-round protocol.GPT-2 Small and Qwen3-1.7B show small Round 2 changes but no sustained improvement by Round 3.Source: §C.2 · pp. 20–21, Figure 3 · p. 21

C.3 LORA VS. FULL FINE-TUNING

  • Full fine-tuning improves mean recovery for GPT-2 Small and Medium but is substantially worse for GPT-2 Large and OPT-1.3B under the historical ReGap-Abs protocol.This diagnostic is not directly comparable to the current one-round results; more trainable parameters do not guarantee stronger recovery.Source: Figure 4 · p. 21, §C.3 · p. 21, Table 11 · p. 22

C.4 RANDOM-SEED CONSISTENCY

  • On GPT-2 Medium, Matched Random Selection yields 53.0±2.0% recovery across the three pre-specified seeds.The three seed-level values are 55.0%, 51.0%, and 53.0%.Source: §C.4 · p. 22

C.5 GENUINE-PRIVATE AND ICL BASELINES

  • Genuine-private supervision exceeds ReGap by 3.3 percentage points on GPT-2 Medium and 5.0 on GPT-2 Large, while ICL adds no recovery on either model.The ICL comparison uses genuine private associations as demonstrations on GPT-2 Medium and Large.Source: §C.5 · p. 22

D.1 CANDIDATE COMPATIBILITY AND GRADIENT ALIGNMENT

  • On GPT-2 Medium, candidate compatibility is positively associated with target-gradient alignment, but the post-hoc target gradient is unavailable to ReGap and the correlation does not establish causation.The attack's selector uses answer-token negative log-likelihood from the frozen target model.Source: §D.1 · p. 23, Figure 5 · p. 24

D.2 SET-LEVEL GRADIENT COMPARISON

  • Randomly selected candidates induce nearly orthogonal aggregate updates, while historical compatibility-based selection produces stronger positive alignment; genuine-private supervision is a reference point.This is a post-hoc diagnostic under an earlier selector-analysis protocol, not an optimization performed by ReGap.Source: §D.2 · pp. 23–24, Table 14 · p. 24

D.3 ADDITIONAL SELECTOR ANALYSIS

Historical selector analyses found no reliable advantage from more elaborate ranking strategies over simpler compatibility-based alternatives. Boundary tests also found that recovery weakens sharply for opaque random bindings, under an earlier protocol and independently trained checkpoints.

D.3 ADDITIONAL SELECTOR ANALYSIS

  • Automatic selector search does not reliably improve on simpler compatibility-based scores, and more elaborate selectors show no consistent advantage over the best manually specified alternatives.The historical experiments motivated the final paper's simpler frozen-model answer-token-loss selector.Source: §D.3 · p. 24, Table 15 · p. 25

E BOUNDARY EXPERIMENTS

  • Boundary experiments use earlier ReGap-Abs protocols and independently trained checkpoints, so they are descriptive rather than strictly matched extensions of the main protocol.They vary the amount and source of exploitable structure in the associations.Source: E BOUNDARY EXPERIMENTS · p. 25

E.1 OPAQUE RANDOM BINDINGS

  • Previously exposed opaque bindings fall from 95.0% recovery to 1.0% after non-target continual training and only 2.0% after ReGap-Abs, with no exact target answers in generated or selected supervision.The result contrasts with larger gains in structured person-email settings, but does not establish that structure is the only factor in recoverability.Source: §E.1 · p. 25

E.2 CONTROLLED ENRON ASSOCIATIONS

This section examines the conditions and limits of answer-free recovery through boundary tests of association structure and accessibility reduction, alongside a theoretical analysis of finite-step decreases in conditional expected target loss. It also clarifies that maximizing support does not guarantee beneficial alignment.

E.2 CONTROLLED ENRON ASSOCIATIONS

  • 27.0% rose to 77.7±1.5% under historical three-round ReGap-Abs; for targets absent from candidate pools, recovery rose from 0.7% to 66.0%, without identifying the underlying memory mechanism.The Controlled Enron result suggests recovery is not limited to identifiers mechanically derived from email local-parts, but does not show that all real-world person-email distributions support similar recovery.Source: §E.2 · pp. 25–26

E.3 NON-ENRON-DOMAIN ASSOCIATIONS

  • 46.0% target recovery versus 46.7±3.2% with ReGap-Abs on external-domain associations; matched random reached 50.3±2.1%, Original-Loss 42.0±9.9%, and genuine-private supervision 80.7±1.2%.This descriptive contrast suggests that removing the dominant corporate-domain pattern weakens answer-free recovery, but the independently constructed checkpoints do not provide a matched causal comparison.Source: §E.3 · p. 26, Table 18 · p. 27

E.4 EXPLICIT NPO UNLEARNING STRESS TEST

  • 71.0% changed to 71.3 ± 0.6% under ReGap on the Moderate NPO checkpoint, while Strong remained at 32.0%; the stress test does not establish general robustness to unlearning defenses.This used one-round Original-Loss adaptation on GPT-2 Medium checkpoints produced by explicit NPO, a different accessibility-reduction mechanism from the main setting.Source: Table 19 · p. 27, §E.4 · p. 27

F CURRENT-PROTOCOL BOUNDARY CONTROLS

  • ReGap did not improve recovery on opaque random bindings when no exact target answers appeared, whereas current-protocol Controlled Enron gains coincided with 49–51 generated and 25–27 selected exact answers per seed.The latter GPT-2 Large post-hoc target result is target-supervision replay, not fully answer-free recovery.Source: §F.1 · p. 28, Table 20 · p. 28, §F.2 · p. 28

G LEAKAGE AND REPRODUCIBILITY AUDITS

  • Unseen-Identity adaptation reused neither target nor memorized identities, and no exact target answers appeared in generated or selected examples across three seeds for GPT-2 Medium and Large.This rules out identity reuse and exact-answer replay in that control, but not in the primary target-identity-aware setting, where exact matches can occur.Source: §G.1 · p. 28, §G.2 · pp. 28–29

G.4 SEED AND PROTOCOL CONSISTENCY

  • 60.0%, 64.0%, and 65.0% ReGap recoveries averaged 63.0 ± 2.6%, versus 53.0 ± 2.0% for matched random; on the target-unexposed control, both remained at 13.0%.The audits also verify fresh LoRA adapters and a matched selection baseline with the same candidate pool, example count, target model, and optimization configuration.Source: §G.4 · p. 29, §G.3 · p. 29, Table 23 · p. 30

H ANALYTICAL RESULTS

  • Identity-disjoint examples can positively align with a target gradient when shared task structure dominates residual effects, but initial alignment need not persist along actual optimizer updates.The analysis holds candidate texts and selection decisions fixed and does not differentiate through generation or selection; identity disjointness alone does not guarantee alignment.Source: §H.1 · p. 30, §H.3 · p. 31, §H.4 · p. 31

H.5 SELF-SAMPLING AND ANSWER-LENGTH WEIGHTING

  • Matched self-sampling has zero expected unweighted loss gradient, but length weighting can make it nonzero; under the stated score-alignment assumption, top-K selection maximizes expected alignment.Neither a nonzero length-weighted gradient nor biased generation alone establishes positive alignment with the target gradient.Source: §H.5 · pp. 31–32, §H.6 · p. 32

H.7 FROM CONDITIONAL RANKING TO A FINITE-STEP LOSS BOUND

  • When A_K(s)>0 and the stated learning-rate condition holds, conditional expected target loss decreases under the finite-moment and segment-smoothness assumptions.Corollary 8 assumes B_K(s)<∞ and a common β>0 bounding target-loss smoothness on the specified update segments.Source: §H.7 · pp. 32–33
  • Proposition 3 maximizes A_K, not the complete bound, and high support alone does not guarantee beneficial alignment.In the binary example, candidate 1 has higher support than target answer 0, but I_T(1)=−0.09 while I_T(0)=0.81.Source: §H.7 · p. 33
  • The moments in the analysis are not inputs to an attack-side learning-rate or model-selection rule.Source: §H.7 · p. 33

H.8 CONNECTING LOSS CHANGES TO DECODED RECOVERY

This section separates changes in target loss from changes in decoded recovery. It gives a sufficient condition under which an update makes greedy decoding recover a target, while clarifying the condition’s limits.

H.8 CONNECTING LOSS CHANGES TO DECODED RECOVERY

  • Recovery changes equal newly recovered targets minus targets that cease to be recovered after the update.The identity also holds in expectation over evaluation randomness.Source: §H.8 · p. 33
  • A lower target loss can still change greedy decoding from correct to incorrect, so loss reduction alone does not determine recovery transitions.In the example, target probability rises from 0.40 to 0.41 while a competitor rises from 0.35 to 0.45.Source: §H.8 · p. 33
  • Under the stated smoothness and decoding assumptions, a sufficient token-margin condition ensures greedy decoding produces a continuation beginning with the target attribute.The generation budget must cover the target tokens, and no external stopping rule may truncate a correct prefix.Source: §H.8 · p. 33
  • The greedy-recovery condition is sufficient but not necessary, and average teacher-forced target-loss reduction alone guarantees neither new nor retained recoveries.If a set of previously unrecovered targets satisfies the condition and previously recovered targets remain recovered, the gain is at least |J|/N.Source: §H.8 · p. 34

Related papers

Loading 2610.01365v1…