PaperlayerView exact revision on arXiv

Source-linked AI summary

Background Gradients Shape Memorization in Flow Matching

Xuanhua Yin, Boyu Wei, Shuyi Zhang, Shunqi Mao, Chuanzhi Xu, Weidong Cai

arXiv:2609.33210v1cs.CV

TL;DR

How same-class training images affect target retention and copying remains unclear, even though repetition is associated with memorization. This paper uses paired trajectories and reciprocal response transfer to study the background gradient response to target-induced parameter changes. Distinct backgrounds reduce extraction at fixed target repetition and row count, while reciprocal transfer changes copying in both directions and its post-removal effects reproduce across tested datasets and architectures.

  • Problem

    How other training images affect target retention and copying remains unclear, despite evidence linking repetition with memorization.

    Source: §1 · p. 2

  • Method

    Paired trajectories isolate target-induced parameter displacement and its background gradient response, which the authors characterize through integrated curvature and test using reciprocal response transfer.

    Source: §1 · pp. 2–3

  • Results

    With 400 same-class background rows, replacing 25 repeated images with 400 distinct images reduces target extraction from 80.7% to 18.0%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The background gradient response is a mechanism shaping target retention and copying in class-conditioned flow matching, with transfer effects reproduced after target removal across tested datasets and architectures.

    Source: §1 · pp. 2–3, §5 · p. 10

    The mechanism experiments concern jointly introduced targets in low-resolution, class-conditioned flow models; isolating responses induced by individual targets remains future work.

    Source: §5 · p. 10

Abstract

from arXiv · show

Repetition is closely associated with memorization in generative models, but how other training images affect the retention and copying of targets remains unclear. We study this question in class-conditioned flow matching, where images outside the target set form the background. At fixed target repetition and same-class background row count, replacing repeated same-class images with distinct images reduces the target extraction rate from 80.7% to 18.0%. To explain this effect, we develop a paired-trajectory framework that isolates target-induced parameter displacement and the background gradient response to it. This response has an exact path-integrated curvature representation, connecting background loss geometry to target learning. Reciprocal response transfer between repeated and distinct backgrounds changes target retention and copying in both directions, establishing the response's causal role. After target removal, the response correction parallel to the target-induced displacement preserves approximately 90% of the copying effects of full response transfer. Directly scaling the displacement also changes copying without further training. The post-removal copying effects of reciprocal transfer are reproduced across datasets and architectures. Together, these results identify the background gradient response as a mechanism through which same-class training data shape the retention of target learning and the reproduction of target images.

1 INTRODUCTION

The introduction asks whether distinct same-class background images reduce memorization when target repetition is fixed, and finds that they reduce target extraction. It proposes paired trajectories and reciprocal response transfer to connect background gradient response to target retention and copying.

1 INTRODUCTION

  • With 400 same-class background rows, replacing 25 repeated images with 400 distinct images reduces target extraction from 80.7% to 18.0%.The much larger change comes from replacing repeated images with distinct ones, rather than changing whether the 25 images are repeated.Source: §1 · p. 2
  • Paired trajectories isolate target-induced parameter displacement and define the background gradient response as the background-gradient difference between paired states.The response equals the background Hessian applied to the displacement, integrated along the segment between the paired states.Source: §1 · pp. 2–3
  • Reciprocal response transfer changes target copying in both directions, and post-removal effects reproduce across datasets and architectures.The response correction parallel to the target-induced displacement preserves 90.8% and 88.9% of the full post-removal copying effects; directly scaling the displacement also changes copying without further training.Source: §1 · pp. 2–3
  • The authors frame reciprocal response transfer and response-component interventions as establishing a causal role for background response in target retention and copying.The framework connects background loss geometry to target learning through the response to target-induced parameter displacement.Source: §1 · p. 3

2 RELATED WORK

Prior work connects memorization to data duplication, dataset properties, and training dynamics, while leaving open how background loss geometry acts on target-induced parameter changes. This paper studies that action through paired trajectories and transferable background responses.

Data Composition and Memorization

  • Prior diffusion-model studies link replication to duplication, dataset size and distribution, and local data coverage; this paper isolates distinct same-class content at fixed target repetition and background row count.It replaces repeated background images with distinct images, then investigates the training-time mechanism in class-conditioned flow matching.Source: §2 · p. 3

Geometry and Training Dynamics of Memorization

  • Geometric accounts describe memorization through manifold dimensions and probability-landscape sharpness, but leave unresolved how background loss geometry acts on target-induced changes.The paper represents that action as the finite background gradient response between paired target-on and target-off states, equal to the integrated background Hessian action along their connecting segment.Source: §2 · p. 3

Training-Data Influence and Causal Interventions

  • Attribution and intervention methods do not test the background response as a separately transferable component; this work exchanges full responses at recipient paired states while retaining the recipient’s target-off background gradient.Prior approaches include influence functions, checkpoint gradients, optimization unrolling, data attribution, and interventions on training or trained models.Source: §2 · p. 3

3 METHODOLOGY

The methodology controls same-class background composition, tracks target-induced parameter displacement with paired trajectories, and intervenes on background gradient responses. It then evaluates how these choices affect target retention and copying.

3.1 CONTROLLED DATA INTERVENTIONS

  • The balanced cyclic design varies target repetition and distinct same-class image count while holding total target and background row counts fixed.To isolate distinct content, it compares 400 different images with 25 repeated 16 times at 400 background rows; a separate comparison measures the effect of repeating the same 25 images.Source: §3.1 · p. 4
  • For mechanism experiments, repeated, intermediate, and distinct backgrounds use nested sets of 25, 100, and 400 images per class, repeated 16, 4, and 1 times, preserving labels and background row count.Each panel contains ten target images, one per class; background and target losses are weighted by their proportions in the full training set, and removing target loss leaves the background weight unchanged.Source: §3.1 · p. 4

3.2 PAIRED TRAINING TRAJECTORIES AND BACKGROUND GRADIENT RESPONSES

  • Paired target-on and target-off trajectories share starting model and optimizer states, background data, and stochastic inputs; their parameter difference captures target-induced displacement.The difference in background loss gradients defines the response, which measures the effect of that displacement on the background gradient.Source: §3.2 · p. 4
  • For a twice continuously differentiable background loss, the response has an exact path-integral curvature representation along the segment connecting paired parameters.Its projection is the average directional curvature along that segment; target-direction curvature at the shared target-free checkpoint is also measured, and is distinct from response along the realized displacement.Source: §3.2 · pp. 4–5

3.3 RECIPROCAL RESPONSE TRANSFER

  • Reciprocal transfer replaces the response using an alternative background evaluated at the recipient’s current paired states, while retaining the recipient’s baseline background gradient.The recipient’s baseline gradient is recomputed at its current target-off state; each arm recomputes its direction and correction at its own current pair while retaining the complete native gradient.Source: §3.3 · pp. 5–6
  • Random-direction and preclip gradient-norm controls replace the transferred correction or match the transferred gradient’s norm while using the recipient’s current pair.The complete gradient is accumulated, unscaled, globally clipped, and applied with AdamW using branch-specific momentum and adaptive preconditioning.Source: §3.3 · p. 5
  • The response correction can be decomposed into components parallel and orthogonal to the recipient’s target-induced displacement.The parallel component isolates the change in finite-response projection; the orthogonal component has zero projection onto the displacement.Source: §3.3 · p. 6

4 EXPERIMENTS AND RESULTS

Across controlled flow-matching experiments, distinct same-class backgrounds reduced target extraction, and reciprocal background-response transfers changed target retention and copying. Scaling target-induced parameter displacement also changed copying, while post-removal transfer effects appeared across tested datasets and architectures.

4.1 EVALUATION AND STATISTICAL SETUP

  • The balanced design crossed target repetition and distinct-background image count; response-transfer experiments used paired panels and evaluated both post-removal and interleaved regimes.Extraction used nearest-reference target hits with class-specific thresholds; mechanism experiments used paired 95% confidence intervals across independent panels.Source: §4.1 · p. 6

4.2 DISTINCT SAME-CLASS IMAGES REDUCE TARGET EXTRACTION RATE

  • At fixed target repetition and background row count, replacing 25 images repeated 16 times with 400 distinct images reduced extraction from 80.7% to 18.0%.Repeating the same 25 images instead changed extraction from 86.0% to 80.7%.Source: §4.2 · p. 7

4.3 BACKGROUND COMPOSITION CHANGES THE RETENTION OF TARGET LEARNING

  • After target removal, distinct backgrounds reduced normalized benefit from 0.3517 to 0.1428 and copying from 2.685% to 0.523%.Across native R, M, and U backgrounds under interleaved training, both normalized target-specific benefit and copying decreased; target-direction curvature increased in the opposite ordering.Source: §4.3 · pp. 7–8, Table 1 · p. 8

4.4 RECIPROCAL RESPONSE TRANSFER CHANGES RETENTION AND TARGET COPYING

  • After target removal, transferring the U response to R lowered copying from 2.685% to 1.732%, while transferring R to U raised it from 0.523% to 1.622%.The transfers also shifted normalized benefit from 0.3517 to 0.1954 for R and from 0.1428 to 0.2922 for U.Source: Figure 3 · p. 7, §4.4 · p. 8
  • Under interleaved target training, U-to-R transfer reduced copying from 2.520% to 1.601%, while R-to-U transfer raised it from 0.439% to 1.249%.Both-direction copying effects exceeded random-direction and gradient-norm controls; conditional accuracy changes ranged from −0.524 to +0.688 percentage points.Source: §4.4 · p. 8
  • After target removal, parallel-only response correction preserved 90.8% and 88.9% of full-transfer copying effects for R and U, respectively.Its copying effects exceeded orthogonal-only correction by 0.750 points in R and 0.860 points in U.Source: §4.4 · p. 8, Table 2 · p. 9

4.5 TARGET-INDUCED PARAMETER DISPLACEMENT CONTROLS GENERATED COPYING

  • Without further training, scaling the target-induced displacement raised copying from 0.33% to 1.30%, 2.520%, and 3.71% across four scales.The random and non-target directions yielded 0.38% and 0.44% copying, respectively.Source: §4.5 · pp. 8–9, Table 3 · p. 9

4.6 POST-REMOVAL RESPONSE TRANSFER ACROSS DATASETS AND ARCHITECTURES

  • Post-removal reciprocal transfers lowered copying for R and raised it for U on both CINIC-10/DiT and CIFAR-10/U-Net, exceeding matched random-direction controls.Each setting used six independent panels, matched background row counts, and targets absent throughout continuation.Source: §4.6 · p. 9, Table 4 · p. 9

5 CONCLUSION

The paper identifies the background gradient response to target learning as a mechanism shaping memorization in class-conditioned flow matching. Its mechanism experiments establish a causal role in target retention and copying, while the studied setting is limited to low-resolution, class-conditioned flow models.

5 CONCLUSION

  • Reciprocal transfer establishes a causal role for the background response in target retention and copying, and displacement-aligned correction preserves most full-transfer copying effects after target removal.Scaling the target-induced displacement also changes copying without further training.Source: §5 · p. 10
  • The mechanism experiments concern jointly introduced targets in low-resolution, class-conditioned flow models; isolating individual-target responses remains future work.The authors propose studying whether target-background relationships predict retention differences and generated copying.Source: §5 · p. 10

A BACKGROUND RESPONSE, TARGET BENEFIT, AND GENERATION QUALITY

  • The finite-response projection uses the realized target-induced displacement, whereas hS uses the fixed direction u at the common initial checkpoint.At step 2,500, native directional Hessian values were 0.03834 for R, 0.05200 for M, and 0.06499 for U.Source: A BACKGROUND RESPONSE, TARGET BENEFIT,… · p. 15
  • At step 6,000, all four reciprocal-transfer comparisons agree in direction between Bspec and R across all eight panels.Pulse-chase ratios measure retention after target training stops; interleaved ratios combine continued learning and retention.Source: Table 5 · p. 15
  • Table 6 reports native interleaved generation quality and LPIPS copying at step 6,000, with rates in percentages and KID based on unscaled Inception pool3 features.The R row also describes the shared γ = 1 dose endpoint.Source: Table 6 · p. 15
  • Inference treats each of eight data panels as an independent unit, weights classes equally, and reports 95% t intervals over panel summaries.Contrast intervals use paired panel differences; primary copying contrasts are Holm-adjusted within each regime.Source: A BACKGROUND RESPONSE, TARGET BENEFIT,… · p. 15

A.1 SHARED-START MECHANISM PROTOCOL AND CONTROLS

The section describes paired response-transfer and parameter-displacement procedures, along with tests of copying effects, curvature orientation, and target-loss improvement from output-layer updates. It also presents matched native-training controls that measure copying under background-gradient response transfer during continued flow-matching training.

A.1 SHARED-START MECHANISM PROTOCOL AND CONTROLS

  • Across all 200 panel-target pairs, target-benefit denominators were positive and above the exclusion threshold, with no condition differences; every initial normalized benefit was 1.The shared-start protocol cloned model, optimizer, and random-number-generator states into paired branches at step 2,900.Source: §A.1 · pp. 15, 18

A.3 PAIRED MECHANISM CONTRASTS

  • Response transfer reduced pulse-chase copying by 0.953 pp for receiver R and exceeded the matched random correction by 0.676 pp (95% interval [0.489, 0.863]; Holm-adjusted p = 1.21 × 10−4).The random correction reduced copying by 0.277 pp (95% interval [−0.062, 0.617]) relative to native training.Source: Table 12 · p. 19, §A.3 · p. 19
  • Transfer fractions were 44.1% [34.7, 51.7] for R and 50.8% [44.0, 58.7] for U in pulse-chase, versus 44.2% [38.5, 49.6] and 38.9% [35.3, 42.3] in interleaved training.These are intervention-effect fractions relative to the native R–U copying gap, not natural mediation proportions.Source: §A.3 · p. 19

A.5 ADDITIONAL VISUAL SUMMARIES

  • Replacing 25 relabeled other-class images repeated 16 times with 400 distinct relabeled images reduced target extraction from 82.0% to 20.0%.The images retained the target conditioning label; their conditional accuracies were 5.8% and 8.3%, respectively.Source: Figure 4 · p. 21, §A.5 · p. 21

B CONTROLLED-GEOMETRY EXPERIMENTS

  • The controlled-geometry experiment tests curvature orientation at fixed eigenvalues, unlike response transfer, which changes the full finite response of a true background loss.This separate experiment uses background supervision and an eight-step SGD schedule distinct from the natural-response experiments.Source: B CONTROLLED-GEOMETRY EXPERIMENTS · p. 23

B.1 ISOLATING TARGET-SPECIFIC LEARNING

  • To isolate target-specific learning, target rows used their true flow-matching velocity while background rows used fixed initial-model predictions, making background gradients initially zero.The experiment used two independently trained initial models with the same ten new targets, one per class.Source: §B.1 · p. 23

B.2 CHANGING CURVATURE ORIENTATION AT FIXED EIGENVALUES

  • The high- and low-alignment penalties shared eigenvalues, trace, and Frobenius norm but differed in curvature alignment with the residual target gradient.At the initial weights, both penalties and their gradients were zero, so the treatments began with the same target gradient.Source: §B.2 · p. 24

B.3 PREDICTING TARGET-LOSS IMPROVEMENT

  • The target-loss prediction uses an exact fixed-feature output-layer recurrence and compares its pre-saved predictions with full-network treatment effects across two evaluation noise banks.The recurrence’s predictions are evaluated using slope, correlation, and paired target-label permutations.Source: §B.3 · p. 24

B.4 NATIVE-TRAINING CONTROLS

  • From three target-free checkpoints, training continued for 3,000 AdamW updates with ten new targets per checkpoint and matched 4,160-row distinct- and repeated-background conditions.Distinct backgrounds used 400 different images; repeated backgrounds used 25 images repeated 16 times per class. Both conditions included 16 copies of each target.Source: §B.4 · p. 25
  • At each update, conditions exchanged the background-gradient component along the current target gradient, with both gradients evaluated at the recipient model.Equal-norm orthogonal controls tested directional specificity, while a repeated-background baseline halved target loss weight without changing background weight; comparisons shared generated noise and a fixed reference bank. Outcomes were generated-copy rate and conditional accuracy.Source: §B.4 · p. 25

B.5 CONTROLLED-GEOMETRY RESULTS

In the controlled-geometry experiment, high alignment reduces eight-step target-loss improvement; supplementary analyses specify the balanced experiment’s design and distinguish training exposure from target-row multiplicity.

B.5 CONTROLLED-GEOMETRY RESULTS

  • 15.47–18.14%: high alignment reduces eight-step target-loss improvement across two initial models and two noise banks.The equal first update and fixed penalty spectrum isolate orientation in this controlled objective.Source: §B.5 · p. 25

C.1 TRAINING CONFIGURATION FOR THE BALANCED DATA EXPERIMENT

  • The balanced experiment uses a DiT for 15,000 steps, with the 10,000-step EMA checkpoint as its primary endpoint.The model has 12 transformer blocks, width 384, six attention heads, and 2 × 2 patches; secondary checkpoints are evaluated at 5,000, 7,500, 12,500, and 15,000 steps.Source: §C.1 · p. 25
  • Composition models train for 10,000 steps, while the eight-panel response experiments evaluate raw parameters at step 6,000.Generation and quality random streams are separate from training.Source: §C.1 · pp. 25–26
  • Target multiplicity counts dataset rows, whereas expected exposure under row sampling and realized presentations can differ when the final incomplete batch is dropped.These quantities are distinguished from exact target-draw counts in the stratified mechanism schedule.Source: §C.1 · p. 26

D NOTATION

  • Table 17 distinguishes flow time, which describes noise-to-data interpolation, from the update index, which describes optimization.It provides additional notation for the earlier data and controlled-geometry experiments.Source: Table 17 · p. 26

E MEASUREMENT UNITS

The measurement section defines extraction against calibrated holdout thresholds and reports matched untrained-reference rates; it also documents predictor checks and analysis safeguards.

E MEASUREMENT UNITS

  • 3.0%, 2.7%, and 3.3%: matched untrained-reference extraction baselines for the balanced, composition, and photometric experiments, respectively.The decoy check uses each experiment’s detector, reference policy, threshold, and M = 2,000 generation budget; per-generation hit probability is approximately 1.5 × 10−5.Source: E MEASUREMENT UNITS · pp. 26–27, Table 18 · p. 27
  • Extraction marks a target hit when its nearest same-class reference meets a threshold calibrated from independent holdout-to-reference distances.Pixel ℓ2 is the primary distance; LPIPS-Alex is calibrated separately on the same holdout set and evaluated on the same generated pool.Source: E MEASUREMENT UNITS · p. 26
  • Data experiments use a common pool of 2,000 generated images per class and model, while mechanism experiments use 4,096 images per class from two shared noise banks.Target-loss improvement is measured as the decrease in mean flow-matching loss under fixed evaluation noise.Source: E MEASUREMENT UNITS · p. 26
  • 15.3, 78.0, and 83.3 percentage points: trained-minus-decoy extraction differences for composition arms 400 × 1, 25 × 16, and 25 × 1, respectively.The differences subtract the matched 2.7% decoy baseline from trained-target rates of 18.0%, 80.7%, and 86.0%.Source: E MEASUREMENT UNITS · p. 27

F PREDICTOR AND INTERVENTION CHECKS

  • Predictor analysis saves predictions before accessing updated parameters or post-intervention losses, and uses ten paired targets as independent units across models and noise banks.Initial/updated feature and output-layer recombinations check reconstruction and component contributions; predictive accuracy is evaluated separately.Source: F PREDICTOR AND INTERVENTION CHECKS · p. 27

G NEAR-DUPLICATE PROTOCOL

The near-duplicate protocol compares exact copies with brightness, contrast, and noise variations under multiple reference policies; estimated effective repetition remains within the prespecified equivalence range.

G NEAR-DUPLICATE PROTOCOL

  • The experiment compares exact copies, light variations, and strong variations across five paired seeds, three training arms, and five saved checkpoints.Variations alter brightness, contrast, and pixel noise while preserving spatial arrangement; every target occurrence is transformed in variation arms.Source: G NEAR-DUPLICATE PROTOCOL · p. 27
  • Three reference policies—training-span, canonical-only, and matched-one—control how training variants are represented during extraction evaluation.The two one-reference policies hold matching opportunities fixed; matched-one uses 16 detector replicates so variants are selected equally often at each repetition level.Source: G NEAR-DUPLICATE PROTOCOL · p. 27
  • Effective repetition is the exact-copy count predicted to yield the same extraction probability as a variation condition.The fitted log-odds shift is relative to exact copies, and the repetition coefficient describes the effect of doubling repetition.Source: G NEAR-DUPLICATE PROTOCOL · p. 28
  • All four effective-repetition intervals lie within [0.8, 1.25], with point estimates of 1.054 and 1.073 for light and strong variations under the all-variants policy.The separate seed-level sensitivity interval is [0.93, 1.05].Source: G NEAR-DUPLICATE PROTOCOL · p. 28, Table 19 · p. 28

H STATISTICAL ESTIMATION DETAILS

The estimation details use bias-reduced and clustered analyses for balanced extraction and composition contrasts, with separate block-level sensitivity results and a figure summarizing balanced-design endpoints.

H STATISTICAL ESTIMATION DETAILS

  • Adding support to repetition alone reduces held-cycle log-loss by 0.0064, 0.2088, and 0.2061 across the three held cycles.The balanced extraction model uses Firth/Jeffreys bias reduction, two-way target/run cluster-robust coefficient intervals, and Fieller exponent intervals.Source: H STATISTICAL ESTIMATION DETAILS · p. 28, Table 20 · p. 28
  • Figure 8 presents balanced-design pixel and LPIPS extraction, endpoint-specific support exponents with 95% Fieller intervals, and held-cycle prediction.Its gray exponent band is the prespecified [0.4, 0.6] reference range, not an uncertainty interval; each heatmap cell contains 30 targets.Source: Figure 8 · p. 29
  • The first three composition contrasts satisfy the Holm-adjusted significance criterion in the post hoc conditional Firth analysis.The ordinary conditional maximum-likelihood estimate is nonfinite because of quasi-complete separation; the analysis conditions on each target’s hit total and retains identifiable run effects.Source: H STATISTICAL ESTIMATION DETAILS · pp. 28–29, Table 21 · p. 29
  • −5.3333 percentage points: the mean block-level extraction difference for 25 × 16 versus 25 distinct same-class images, with interval [−8.6692, −1.9974].All three leave-one-block-out contrasts retain their pooled direction, and all three unadjusted 95% t intervals exclude zero.Source: H STATISTICAL ESTIMATION DETAILS · pp. 29–30, Table 22 · p. 30

I NATIVE-TRAINING INTERVENTION PROTOCOL

The protocol compares target retention and copying under controlled background interventions, then evaluates models that predict target-specific outcomes across held-out panels. Its later sections report aggregate prediction performance and specify the paired measurement design.

I NATIVE-TRAINING INTERVENTION PROTOCOL

  • Each native-training condition continues a target-free checkpoint for 3,000 updates, using 240 background draws and 16 target draws per update.Background and target loss weights match their training-row proportions, and generation uses the raw continued network.Source: I NATIVE-TRAINING INTERVENTION PROTOCOL · p. 30
  • The intervention exchanges a donor-background gradient response while retaining recipient-background and target gradients; its control applies an equal-norm correction orthogonal to the target-gradient direction.The exchange is tested in both directions, and the simple baseline uses the recipient background gradient plus half-weighted target gradient.Source: I NATIVE-TRAINING INTERVENTION PROTOCOL · p. 30

J BALANCED ASSIGNMENT

  • The balanced assignment gives every target all 25 combinations of background support and target repetition, with 1,550 background and 62 target rows per model.The design spans five background-support levels and five repetition levels.Source: J BALANCED ASSIGNMENT · p. 31

K ADDITIONAL QUANTITATIVE RESULTS

  • The appendix reports target-loss improvement differences under high versus low alignment and full-network predictions calibrated by leaving one target out.The table captions specify the comparison and calibration design but do not provide outcome values in these passages.Source: Table 24 · p. 31, Table 25 · p. 31

L FINITE-STEP PREDICTOR DETAILS

  • The fixed-feature recurrence is exact for its quadratic objective, while the study tests its accuracy on a full network whose features change during training.Leave-one-target-out calibration evaluates recurrence predictions; its fitted coefficients do not enter the recurrence.Source: L FINITE-STEP PREDICTOR DETAILS · p. 33

M PANEL-LEVEL PREDICTION SUMMARIES

  • Eight outer folds hold out all ten targets from one panel, while ridge regression predicts retention and logistic regression predicts copying from nested feature sets.The feature blocks range from exposure, initial losses, and generation quality (P0) through AdamW-state dynamics (P4).Source: M PANEL-LEVEL PREDICTION SUMMARIES · p. 33, Table 26 · p. 33
  • On held-out panels, P3 lowers error versus P2 by 0.0213 in retention MAE (0.1163 to 0.0950), 0.0464 in copy log-loss (0.3578 to 0.3114), and 0.0329 in open-loop error (0.1831 to 0.1502).Inputs are measured at step 2,900; open-loop error is evaluated over 256 steps and normalized by initial target-specific benefit.Source: M PANEL-LEVEL PREDICTION SUMMARIES · p. 33, Table 26 · p. 33

N.1 COMPOSITION UNDER A LONGER TRAINING BUDGET

The extended analyses test whether composition effects persist under longer training and compare full response transfer with projection-based and component-level interventions. They also describe paired-panel calculations used to quantify copying effects.

N.1 COMPOSITION UNDER A LONGER TRAINING BUDGET

  • At 20,000 steps, extraction is 30.00% for the same-class 400 × 1 background and 78.00% for the 25 × 16 background, a 48.00-point gap.The corresponding conditional accuracies are 80.00% and 94.00%, respectively; the 400-row comparison is at equal training steps.Source: Table 27 · p. 34, §N.1 · p. 34

N.2 BENEFIT TRAJECTORIES AND THEIR CONSTITUENT LOSSES

  • At step 6,000, interleaved minus pulse-chase differences are −0.0128 in Bspec, −0.0311 in R, and −0.165 copying points for R.For U, the corresponding differences are −0.0049 in Bspec, −0.0123 in R, and −0.084 copying points; both regimes share 640 common-learning target draws, then differ in continuation exposure.Source: Table 28 · p. 34, §N.2 · p. 34
  • The four step-6,000 losses recover signed target-specific benefit, with target-off, decoy-off, and decoy-on entries all 0.5000, 0.4200, and 0.4000 across conditions.The benefit is calculated from the target on/off and decoy on/off loss differences.Source: Table 29 · p. 35, §N.2 · p. 35

N.3 SAME-PROTOCOL COMPARISON OF RESPONSE INTERVENTIONS

  • Full response transfer changes copying by 0.953 percentage points downward in R and 1.099 points upward in U, exceeding projection exchange after operation-specific controls.The control-adjusted differences between operations are 0.448 points for R and 0.652 points for U.Source: Table 30 · p. 35, §N.3 · p. 35

N.4 RESPONSE-COMPONENT ASSAY

  • The along-displacement correction changes copying by 0.865 percentage points in R and 0.977 points in U, while orthogonal corrections change it by 0.115 and 0.117 points.The along-displacement effects are 0.088 and 0.122 points smaller than full-transfer effects, respectively.Source: §N.4 · p. 35, Table 31 · p. 36, Table 32 · p. 36

N.5 PANEL-LEVEL COPYING INPUTS AND PAIRED CALCULATIONS

  • The panel-level copying effects are paired within each comparison; the reported intervals are ordinary two-sided 95% t intervals over eight panels.The effect is native minus intervention copying for R and intervention minus native copying for U.Source: §N.5 · p. 36

O CONNECTIONS TO MEMORIZATION, OPTIMIZATION, AND EVALUATION

The paper situates its experiments among work on memorization, training-data attribution, optimization geometry, generative modeling, and sample-quality evaluation. It distinguishes its background-response intervention and copying endpoint from related approaches and metrics.

Repetition, Extraction, and Generalization

  • Prior work links repetition to memorization and studies deduplication, while this paper separates repeated background rows from distinct content at a fixed row count.The cited literature also distinguishes fitting examples, generalization, and recovering examples from model outputs (Zhang et al., 2017; Feldman, 2020; Feldman & Zhang, 2020).Source: O CONNECTIONS TO MEMORIZATION,… · p. 37

Measuring and Attributing Image Memorization

Curvature and the Optimization Trajectory

  • The intervention changes a background-loss gradient component before AdamW, while the finite-response projection describes its correction along the target-induced displacement.The optimizer state still determines how that correction becomes a parameter update.Source: O CONNECTIONS TO MEMORIZATION,… · p. 38

Generative Formulations and Model Settings

  • The study uses flow matching with a diffusion transformer and an additional U-Net setting, and distinguishes reference-based copying from KID and conditional accuracy.Copying uses a calibrated nearest-reference rule, whereas KID and conditional accuracy characterize generation quality alongside copying.Source: O CONNECTIONS TO MEMORIZATION,… · p. 38

Related papers

Loading 2609.33210v1…