PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Does Train-Validation Separation Emerge? Update-Pressure Density Dynamics in Pretrained Backbones

Yuchen Li, Mingyu Du, Zongqi Fan, Ken-Tye Yong, Nguyen H. Tran

arXiv:2610.01425v1cs.LG

TL;DR

The paper asks how the train–validation gap evolves during pretrained-model adaptation and proposes that continued fitting can shift toward narrower support with weaker held-out transfer. It measures update-pressure allocation using fixed training probes and tests its relationship with the gap in a controlled hierarchy and real NLP and vision settings. The real-model readouts show positive gap associations across the 90 NLP runs and in the tested vision datasets, while not identifying a unique latent support decomposition.

  • Problem

    The paper asks how the empirical train–validation gap evolves during pretrained-backbone fine-tuning and what internal structural changes accompany it.

    Source: §1 · pp. 1–2

  • Method

    The paper combines a conditional support-transfer account with fixed-training-probe measurements and tests them in a constructed hierarchy and real-data experiments.

    Source: §1 · p. 4

  • Results

    In the 90-run NLP cohort, both dispersion readouts have positive raw and smoothed accuracy-gap level associations in all runs, and both extend to the tested ResNet-18 vision datasets.

    Source: §6 · p. 22

  • Takeaways & Limitations

    The controlled hierarchy and real-model associations support an observable structural account of gap formation across the tested settings.

    Source: §6 · p. 22

    The local mechanism depends on explicit assumptions, and empirical readouts are specific to the monitored backbone and setting, so their magnitudes and association strengths require within-setting interpretation.

    Source: §7 · p. 23

Abstract

from arXiv · show

Train-validation separation is the evolving difference between performance on observed training examples and a finite held-out validation set. We propose a dynamic structural account of how this gap develops during adaptation of pretrained models: continued fitting can shift update demand from broadly reusable support toward narrower support with weaker held-out transfer. A conditional local model links this shift to increasing heterogeneity in gradient allocation and train-validation separation. Fixed training probes make this structural evolution observable without validation examples entering the readouts; held-out performance is used separately to evaluate its relation to the gap. In a constructed hierarchy implemented with a residual multilayer perceptron (ResMLP), increasing the target share of example-private features from $p=.3$ to $.5$ to $.7$, while preserving the relative mixture $1{:}2{:}3{:}4$ among the four shared feature levels, increases the final mean accuracy gap from $.185$ to $.331$ to $.527$ across five runs per condition. Masked-input losses measured separately on training and validation examples expose the corresponding transfer asymmetry. The natural language processing (NLP) analysis uses 10-epoch runs of RoBERTa, DeBERTa, and Qwen on six datasets (90 runs): the training-probe-weighted within-class and overall dispersion readouts each have positive raw and smoothed level correlations with the accuracy gap in all 90 runs. Raw changes paired at approximately one-epoch intervals remain positively associated in 86/90 and 87/90 runs, respectively. A 40-epoch ResNet-18 study tests both readouts on three vision datasets. Together, controlled simulation, NLP, and vision support the dynamic structural account across settings, with real-model evidence testing its observable predictions under the specified monitors.

1 Introduction

The paper asks how the empirical train–validation gap evolves during pretrained-backbone adaptation and whether changing update-pressure structure accompanies it. It proposes a shift from broadly reusable support toward narrower, lower-transfer support, observed through fixed training probes and tested in controlled and real-data settings.

1 Introduction

  • The gap is the time-evolving difference between performance on a training subset and a finite held-out validation set.Training accuracy may rise while validation accuracy improves more slowly, saturates, or decreases.Source: §1 · p. 1
  • Continued fitting may shift from reusable, high-transfer support toward subset- or example-specific patterns that benefit training more than validation.The face-recognition analogy contrasts broadly shared topology, community patterns, and private marks.Source: §1 · p. 2, Figure 1 · p. 3
  • Fixed training probes measure internal update-pressure structure without using validation examples, while held-out performance separately tests its relation to the gap.The monitored coordinates are fixed at task-adjacent backbone interfaces.Source: §1 · pp. 3–4
  • The evaluation strategy pairs a trackable five-level ResMLP hierarchy with tests of observable relationships across pretrained NLP backbones and vision models.The NLP cohort spans three backbones, six datasets, and 90 runs; the vision study uses ResNet-18 on three datasets.Source: §1 · pp. 4–5
  • A conditional local theory links faster consumption of shared demand and increasing residual heterogeneity to greater dispersion and local surrogate-risk separation.The relation holds under explicit sufficient conditions.Source: §1 · p. 4

2 Related Work

Prior work addresses memorization, gradient interactions, transferability, validation monitoring, and representation structure, but the paper identifies a gap in explaining train–validation separation through evolving internal update pressure. It combines a constructed support intervention with train-probe measurements while treating real-model findings as associations rather than latent-feature identification.

2 Related Work

  • The study targets a time-evolving train–validation gap through internal update-pressure allocation, a question not jointly addressed by the closest cited approaches.Its real-model evidence does not identify a unique latent support decomposition.Source: §2 · pp. 5–6
  • Example-dynamics and memorization studies characterize fitting at the output or example-score level, not through internal update-pressure distributions over monitored coordinates.The cited work motivates asking what is fitted at each training stage.Source: §2 · p. 5
  • Validation monitoring selects models using held-out performance, whereas this work uses validation accuracy as an external reference rather than an input to internal observables.The paper frames its question as mechanistic rather than evaluative.Source: §2 · p. 5
  • Gradient coherence and stiffness study cross-sample direction or benefit; this work instead measures nonnegative pressure allocation across monitored coordinates.The magnitude-based view abstracts away from signed gradient alignment.Source: §2 · p. 5
  • Transferability, gradient starvation, and fine-tuning distortion motivate the support-transfer account, but do not explain gap formation as declining validation-side efficiency of continued training-loss reduction.The paper tests that account using a controlled-initialization protocol and train-probe dispersion.Source: §2 · pp. 5–6
  • Representation-similarity tools do not directly measure the backpropagated update-pressure distributions induced by fixed training probes.The local nonnegative mixture is explanatory, not an algorithm for uniquely recovering its factors.Source: §2 · p. 6

3 Train-Probe Update-Pressure Framework

The framework uses fixed training probes to track normalized magnitude allocation over monitored coordinates and compares its dispersion with held-out behavioral outcomes. It distinguishes allocation heterogeneity from sign changes and treats accuracy gap as the primary endpoint.

3 Train-Probe Update-Pressure Framework

  • Fixed training probes preserve sample identity across checkpoints, allowing per-sample pressure vectors to retain heterogeneity that minibatch averaging would suppress.Real datasets do not reveal which effective features are globally shared, subgroup-specific, or sample-specific.Source: §3 · p. 6

3.1 Fixed Probes and Monitored Coordinates

  • The monitor selects fixed task-adjacent coordinates in the final k backbone blocks and records each probe’s first-order pressure before optimizer transformations.The readout is not an exact AdamW update and remains sensitive to optimizer-induced changes in parameter states.Source: §3.1 · p. 7

3.2 Density Dispersion

  • Overall and training-probe-weighted within-class dispersion summarize how heterogeneous normalized pressure allocations are across probes.Within-class dispersion measures allocation heterogeneity among examples sharing a label, rather than movement only between class centers.Source: §3.2 · p. 8

3.3 Why Absolute Gradients Are Used

  • Magnitude normalization makes sign-equivalent gradients share an allocation representation, while genuine redistribution across coordinates remains visible.This avoids treating label-conditioned polarity or local sign reversals as allocation changes, but discards directional identity.Source: §3.3 · pp. 8, 10, Figure 2 · p. 9

3.4 External Behavioral References

  • Validation examples do not enter pressure vectors or their dispersion summaries; validation accuracy and loss are external behavioral references.The primary target is the accuracy-gap level.Source: §3.4 · p. 10

3.5 Why Accuracy Gap Is the Primary Endpoint

  • Accuracy gap is primary because cross-entropy can change with confidence and logit scale even when argmax predictions, and accuracy, remain unchanged.Loss gap is retained as an external diagnostic rather than defining the internal pressure state or headline empirical claim.Source: §3.5 · p. 11, Figure 3 · p. 11

4 Local Dynamics of Support Narrowing and Train–Validation Separation

The local account links support narrowing to increasing dispersion in training-probe pressure and greater train–validation separation. It separates signed gradients, which motivate learning order, from magnitude-normalized pressure, which measures allocation.

4 Local Dynamics of Support Narrowing and Train–Validation Separation

  • The local account proposes that coherent directions dominate early updates, broad demand is reduced faster, and weaker held-out transfer from residual support raises separation pressure.The resulting local surrogate-separation growth requires the stated positivity condition.Source: §4 · p. 12, §4.1 · p. 13

4.1 Notation and Claim Boundary

  • The notation distinguishes signed per-sample gradients from normalized nonnegative pressure allocations and labels shared and residual demand separately.The local model also tracks residual-shape heterogeneity, approximation error, and transfer quantities.Source: §4.1 · p. 12

4.2 Batch Coherence and Learning Order

  • Signed batch means retain repeated directions while attenuating approximately zero-mean sample-specific directions at the usual m^-1/2 scale.This motivates earlier learning of broad support only when it yields a task-aligned signed component after label conditioning; magnitude sharing alone is insufficient.Source: §4.2 · p. 13

4.3 Continuous-Time Consumption of Shared Support

  • Shared demand is consumed faster than heterogeneous residual demand, so residual demand can become relatively dominant without increasing in absolute size.This rate ordering provides a local route from coherent shared updates to support narrowing.Source: §4.3 · pp. 13–14

4.4 Gradient Footprints and Residual Heterogeneity

  • Sample-dependent residual processing yields nonidentical normalized magnitude footprints across probes only under a testable nondegeneracy condition.If residual paths induce identical footprints, residual heterogeneity is zero and the theory predicts no residual contribution to ideal pooled dispersion.Source: §4.4 · p. 14

4.5 Observable Barycentric Geometry

  • The cross-probe pressure barycenter is an exact observable template, but real-data dispersion around it does not identify a unique semantic feature set.The barycenter is also the best single pressure allocation representing all probes in squared error.Source: §4.5 · p. 14

4.6 Pressure Mixture and Observable Dispersion

  • A local support-narrowing interval has an increasing residual mixture weight, but dispersion rises only when residual pressure is heterogeneous.The model’s residual fraction is latent; the recorded dispersion tests the associated trajectory under its aggregation convention.Source: §4.6 · p. 15

In the noiseless two-component case e(t)

Under stated local conditions, increasing residual demand can accompany rising pressure dispersion and surrogate train–validation separation. The analysis distinguishes this conditional mechanism from the accuracy-gap measurements and from claims of unique latent identification.

In the noiseless two-component case e(t)

  • Dispersion grows in the noiseless two-component case when residual-weight growth outweighs any relative contraction in residual heterogeneity.With approximation error, the two-time growth condition requires the increase in λσv to exceed the combined error margins.Source: In the noiseless two-component case e(t) · pp. 15–16

4.7 Validation-Transfer Efficiency and Gap Dynamics

  • When τu > τv, increasing residual weight raises local separation pressure per unit train-side reduction, but separation itself grows only under the stated positivity condition.The transfer efficiencies represent held-out surrogate-risk reduction per unit train-side reduction; negative values are allowed when an update worsens held-out risk.Source: §4.7 · p. 16

4.8 Combined Mechanism and Falsification Conditions

  • Under the combined conditions, pressure dispersion and surrogate separation grow, while separation pressure per unit train-side improvement rises with residual weight.The result assumes faster shared-demand depletion, sufficiently persistent residual heterogeneity, lower residual transfer, and effective held-out transfer below one.Source: §4.8 · p. 17
  • The account would be weakened if higher constructed private-feature share failed to enlarge the accuracy gap or masked private burden, or if real-data associations disappeared.These are empirical failure conditions, not claims that the latent components are uniquely identified.Source: §4.8 · p. 17

4.9 Accuracy-Gap Interpretation

  • Because accuracy is discontinuous, experiments test co-evolution with the accuracy gap rather than exact equality to local surrogate separation.Cross-entropy validation loss is not asserted to be a calibrated measure of classification correctness or to map one-to-one to accuracy.Source: §4.9 · p. 17
  • The real-data readout tests association with the gap but does not identify a unique decomposition into latent support components or equal the ideal pooled functional exactly.The controlled hierarchy instead uses feature-level interventions and split-specific masked-loss trajectories to diagnose the predicted transfer ordering.Source: §4.9 · p. 17

5 Experiments

The experiments move from a controlled ResMLP hierarchy to NLP and vision cohorts, testing the proposed support-transfer mechanism and whether training-probe readouts track the accuracy gap. The hierarchy intervention enlarges the gap as private support increases, while real-model studies test the predicted associations under specified monitors.

5 Experiments

  • The study combines a controlled five-level ResMLP intervention, a 10-epoch NLP cohort testing temporal and loss-gap associations, and a 40-epoch vision cohort.In real models, semantic support is latent, so readout associations do not establish the latent support transition.Source: §5 · pp. 17–18

5.1 Mechanism Study: Five-Level Hierarchical Feature Mixture

  • .185, .331, and .527 were the final mean accuracy gaps at private shares p=.3, .5, and .7, respectively, with the non-private relative ratio fixed at 1:2:3:4.The intervention changes the full support composition; each condition used five 25-epoch runs.Source: §5.1 · pp. 18–19, Table 2 · p. 19
  • Adjacent-epoch increment associations in the hierarchy were mixed, although all 15 runs increased in both unnormalized dispersion quantities from epoch 1 to 25.The unnormalized measures capture gradient magnitude as well as allocation, complementing normalized correlations.Source: §5.1 · p. 19
  • Validation-side private masked-loss share rose more than training-side share at every tested private share, widening their disparity most at p=.7.Figure 5 compares the split-specific private shares; Table 4 quantifies their change from epoch 1 to 25.Source: §5.1 · p. 20, Table 4 · p. 20, Figure 5 · p. 20

5.2 10-Epoch NLP Results: Overall and Temporal Associations

  • Both NLP readouts had positive raw and LOWESS accuracy-gap level associations in all 90 runs; raw one-epoch changes were positive in 86/90 weighted within-class and 87/90 overall runs.The cohort used 10-epoch runs of three backbones across six datasets.Source: §5.2 · p. 21, Table 6 · p. 21
  • Across the 90 NLP runs, direct Dden level correlations with the validation-minus-training loss gap averaged .818, and raw one-epoch difference correlations averaged .480.Accuracy remains the primary behavioral outcome; loss and accuracy need not share identical local trajectories.Source: §5.2 · p. 22

5.3 40-Epoch All-Class Vision Results

  • The 40-epoch ResNet-18 measurements extend both readout associations to three vision datasets and all observed training-probe classes.The readouts are paired with the overall accuracy gap.Source: §5.3 · p. 22, Table 7 · p. 22

6 Conclusion

The paper presents a conditional structural account of the evolving empirical train–validation gap and makes update-pressure evolution observable with fixed training probes. A controlled hierarchy and real-model NLP and vision results support the account’s observable predictions, while the real-model evidence does not uniquely identify latent support structure.

6 Conclusion

  • In the constructed hierarchy, increasing private-support mixture while preserving non-private relative composition enlarged the accuracy gap and validation-side private masked-input loss burden.This provides a controlled test of the proposed mechanism.Source: §6 · p. 22
  • In NLP, both readouts had positive raw and LOWESS accuracy-gap level associations in all 90 runs, and vision results extended the associations to tested ResNet-18 datasets.Real-model measurements support an observable relationship consistent with the controlled mechanism.Source: §6 · p. 22
  • Neither positive correlation nor a particular monitor depth uniquely identifies the latent shared/private decomposition.The structural readouts are observable from fixed training probes without validation data entering their construction.Source: §6 · p. 22

7 Limitations and Scope

The structural account is conditional, and real-data associations are interpreted as corroboration rather than identification of latent support transitions. Readout magnitudes and association strength should be interpreted within each specified setting.

7 Limitations and Scope

  • The local mechanism depends on explicit assumptions about shared-support consumption, residual heterogeneity, and relative transfer efficiency.Readout magnitudes and association strength are specific to the backbone and monitor.Source: §7 · p. 23
  • NLP and vision test predicted observable relationships, but semantic support remains latent; nonlinear or autocorrelated common influences can remain despite temporal checks.Real-data results are therefore treated as corroboration of observable predictions.Source: §7 · p. 23
  • Learning from random initialization, cross-model calibration, prospective monitoring, and training interventions remain directions for further work.Source: §7 · p. 23

A 10-Epoch NLP Dataset-Level Results

The dataset-level NLP tables report within-run associations between dispersion and accuracy or loss gaps, including paired changes at roughly one-epoch intervals. A separate depth check finds that association strength generally rises with monitor depth, so it is not invariant to the monitor.

A 10-Epoch NLP Dataset-Level Results

  • The tables pair class-specific D0 and D1 with class-specific gaps, while W and direct Dden are paired with the overall gap.Correlations are computed within runs and averaged equally across five runs per backbone–dataset cell.Source: A 10-Epoch NLP Dataset-Level Results · p. 25, Table 8 · p. 25
  • Across the displayed datasets, paired ∆Dden and ∆G associations are positive, though some are less consistent across runs, including Qwen3-1.7B RTE at .276 (4/5).The table pairs each checkpoint with the available checkpoint nearest one epoch later; ∆ is later minus earlier.Source: A 10-Epoch NLP Dataset-Level Results · pp. 25–26, Table 9 · p. 25

B.1 Monitor-Depth Sensitivity

  • Mean association and directional consistency generally rise across tested depths, showing that empirical association strength depends on K.The depth check uses 15 ResMLP trajectories at five depths; Table 10 reports the sensitivity results.Source: §B.1 · p. 26, Table 10 · p. 26

B.2 Training-Probe Count Sensitivity

The probe-count grid is a descriptive sensitivity analysis across recorded settings, not repeated measurements of one unchanged trajectory.

B.2 Training-Probe Count Sensitivity

  • In Table 11’s ResMLP sensitivity grid, the p = .3, N = 128 direct raw mean is .894, versus .911 in the main intervention in Table 2.Source: §B.2 · p. 26

B.3 Near-Zero Mass and Precision

The appendix checks numerical stability and run-level displays, then qualifies the theoretical links among pressure dispersion, support narrowing, and transfer. These results remain conditional on the monitors, assumptions, and local regime described.

B.3 Near-Zero Mass and Precision

  • Near-zero monitored masses motivate filtering: the global minimum is 4.47 × 10^-33, while FP32 controls record zero invalid probes among 9,600 evaluations.The K = 10 cohort excludes 460 evaluations (0.9583%); FP32 control associations remain positive across the three private shares.Source: §B.3 · pp. 26–27
  • The BF16-AMP check has 26 of 9,600 evaluations at or below εnum, all at p = .3; its settings do not support matched comparisons across private shares.The reported check concerns ResMLP; corresponding monitored-mass diagnostics are unavailable for BF16 LoRA NLP runs.Source: §B.3 · p. 27

B.4 Run-Level Dispersion Values and Displays

  • Across the 15 controlled-hierarchy runs, all four epoch-25 readouts exceed their epoch-1 values; Figure 6 plots checkpoint dispersion against the gap by private-share condition.Figure 7 shows corresponding run trajectories; the companion unnormalized measure retains gradient magnitude.Source: §B.4 · p. 29, Figure 6 · p. 29, Figure 7 · p. 30

C.1 Normalization Stability

  • Away from near-zero monitored pressure mass, the stabilized logged vector differs from the exact simplex-normalized vector by a controlled perturbation.The implementation details are introduced in the normalization-stability subsection.Source: §C.1 · p. 31

C.2 Finite-Probe Estimation

  • The iid finite-probe bound is a fixed-state reference, not a simultaneous confidence band for the experiment’s fixed, stratified time series.The measurements primarily compare structural evolution within a run.Source: §C.2 · p. 31

C.3 Relation to Signed-Gradient Controls and the Role of the Absolute Value

  • The magnitude-based readout tracks cross-probe allocation heterogeneity, not signed direction or a unique semantic split; pooled and class-balanced calculations need not coincide.The signed mean–residual decomposition is arithmetic, and the ideal pooled decomposition does not establish equality with the directly recorded class-balanced readout when probe counts differ.Source: §C.3 · pp. 32–33

C.4 A Sufficient Local Dynamic for Increasing Residual Weight

  • In the sufficient local model, shared demand contracts faster than residual demand, raising residual weight and—when λ(t)σv(t) increases—ideal pooled dispersion.This is a mechanism bridge, not an optimizer-specific convergence claim for every nonconvex trajectory.Source: §C.4 · pp. 33–34

C.5 First-Order Local Surrogate-Risk Changes and the Transfer Error

A first-order expansion relates local train and validation surrogate-risk changes to the parameter step, with a remainder. The tracking algorithm records training-probe pressure readouts alongside separate performance references.

C.5 First-Order Local Surrogate-Risk Changes and the Transfer Error

  • The first-order local surrogate-risk relation is meaningful for sufficiently small steps, but does not establish that absolute validation-loss scale measures classification correctness or that accuracy is additive.The remainder absorbs Taylor terms, minibatch noise, optimizer effects, omitted interactions, and changing transfer efficiencies; latent coefficients are not directly observable.Source: §C.5 · p. 34

D Pseudocode

  • The tracking routine fixes probes and monitored groups, trains normally, then logs normalized absolute-gradient allocations and evaluates train/validation metrics without feeding validation information into internal metrics.The fixed probe branch has cost tied to probe count and monitored groups, not dataset size, and does not update parameters.Source: D Pseudocode · p. 35

E Backbone-Specific Monitoring and Metric Computation Protocol

The protocol uses fixed training probes and backbone-specific interfaces to track structural pressure while keeping validation signals outside internal readouts. Probe computation is observational and does not alter training updates.

E Backbone-Specific Monitoring and Metric Computation Protocol

  • The probe branch is observational only and does not modify the loss, optimizer, sampling, curriculum, parameter-freezing policy, or model update rule.Structural summaries and external reference signals are recorded separately.Source: E Backbone-Specific Monitoring and… · p. 35

E.1 Probe Set Construction

  • Fixed probes come exclusively from the training split, with NLP probes stratified to training-class proportions and ResMLP and vision probes using specified class-balanced counts.The fixed probe count makes per-record monitoring cost independent of full dataset size.Source: §E.1 · p. 35

E.2 Common Interface-Pressure Vector

  • The monitor forms pressure over selected shared-output interfaces by concatenating absolute gradient coordinates, with activation-side pressure included when that interface is monitored.Empirical tables retain the directly recorded Dden,t summary under its stated aggregation convention.Source: §E.2 · pp. 35–36

E.3 Backbone Monitors

  • Backbone monitors select final blocks and common attention-output and feed-forward-output interfaces, excluding query, key, and value tensors to reduce sensitivity to architecture-specific topology.Raw Dden,t magnitudes are interpreted within a fixed backbone, parameterization, and monitor, not as aligned coordinates across backbone families.Source: §E.3 · p. 36

E.4 Metric Construction Boundary

  • Internal pressure readouts use only fixed training probes, while validation accuracy and loss provide separate external trajectories such as GapAcc(t) and GapLoss(t).The probe identities remain fixed throughout each run.Source: §E.1 · p. 35, §E.4 · p. 37

E.5 Probe Computation and Logging

  • Probe computation uses exact per-sample gradients in evaluation mode, clears gradients between samples, makes no parameter updates, and restores training mode before optimization resumes.The run logs step-level training and validation metrics, gaps, and density summaries using the same probe identities and monitored-coordinate rule throughout.Source: §E.5 · p. 37

F Reproducible Experimental Configuration

The experiments use fixed cohort-specific run lengths, probe settings, and monitoring backends. Task hyperparameter selection is separated from monitor-depth calibration, and selected monitor depths remain fixed across datasets and runs.

F Reproducible Experimental Configuration

  • The NLP cohort uses 10 epochs, vision uses 40, and ResMLP studies use 25; conditions have five independent runs unless a precision control specifies one.NLP uses 128 fixed training probes, and the main ResMLP intervention uses 128 class-balanced probes.Source: F Reproducible Experimental… · p. 37
  • Task hyperparameters are selected per dataset and backbone using a limited validation-accuracy sweep over learning rate and dropout, without density readouts as selection criteria.The five non-calibration NLP datasets receive their own planned hyperparameter optimization rather than inheriting BoolQ settings.Source: F Reproducible Experimental… · p. 37
  • BoolQ calibrates monitor depth for each NLP backbone, after which the selected depths—4 for RoBERTa-base, 6 for DeBERTa-base, and 2 for Qwen3-1.7B—remain fixed across six datasets and runs.Calibration compares candidate depths for stable execution, acceptable probe cost, and a non-degenerate monitored-gradient time series.Source: F Reproducible Experimental… · pp. 37–38
  • Reported NLP runs use a fixed shared-output parameter-side monitor, while controlled ResMLP uses a fixed hybrid parameter/activation monitor; neither backend is tuned using density-gap correlations.These choices specify the pressure coordinateization rather than selecting the task hyperparameters.Source: F Reproducible Experimental… · p. 38
  • The synthetic hierarchy has no label noise and feature-noise standard deviation 0.01; RoBERTa and DeBERTa use full fine-tuning, while Qwen uses frozen-backbone LoRA.Qwen3-1.7B uses LoRA rank 32 in BF16; the other two backbones use FP32.Source: F Reproducible Experimental… · p. 38
Loading 2610.01425v1…