PaperlayerView exact revision on arXiv

Source-linked AI summary

Conditioned Direct Feedback Alignment via Activity and Error Geometry

Houman Safaai, Varun Reddy, Bernardo L. Sabatini

arXiv:2607.18574v2cs.LGcs.NEq-bio.NC

TL;DR

DFA’s fixed random feedback can provide useful credit while unequal activity or error scales still distort local updates. This paper studies inverse-moment conditioning of either factor and finds practical activity-conditioning benefits under tested settings, alongside important limits.

  • Problem

    Even when fixed random feedback provides a useful descent component, high-variance activity directions without task information can dominate DFA’s local update; the paper asks whether reweighting activity or error directions helps while feedback stays fixed.

    Source: §1 · p. 1

  • Method

    The paper studies established inverse-second-moment conditioning operators applied to the activity side, error side, or both sides of DFA updates.

    Source: §1 · p. 1, §2 · p. 2

  • Results

    Activity conditioning improves tuned DFA at matched measured learning time, centered correlations contribute beyond the tested mean-preserving control, and early-only conditioning offers an accuracy–work tradeoff.

    Source: §1 · p. 1, §6 · p. 9

  • Takeaways & Limitations

    Established conditioning operators can improve approximate credit with fixed random feedback, but this is not a new matrix inverse or a general advantage over backpropagation.

    Source: §6 · p. 9

    The main practical evidence concerns MLPs and fixed data splits; equal measured learning time excludes validation and export and does not imply equal FLOPs.

    Source: §6 · p. 9

Abstract

from arXiv · show

Direct feedback alignment (DFA) trains hidden layers with fixed random projections of the output error. Even when this feedback provides useful credit, unequal scales across activity or error directions can distort the local update. We study conditioned DFA (nDFA), a family that adapts established inverse-moment preconditioning to either side of this update. An aligned linear analysis describes how activity conditioning changes spectral learning rates and early-stopping risk. Synthetic experiments show gains from both stabilization and changes in update direction. Confirmatory CIFAR-10 experiments show that activity conditioning, including its FOOF formulation, improves tuned DFA at matched measured training time. Controls support a role for centered correlations beyond the tested mean and diagonal alternatives, while early-only conditioning retains most of the benefit with less training work. Conditioning also benefits backpropagation, and timing interventions do not establish a mechanism specific to random feedback. Error conditioning improves short-budget models and changes update directions, but its small additional improvement under stable training does not survive correction for multiple comparisons. Predicted benefits on image-background benchmarks are not confirmed. These findings establish practical benefits and important limits of conditioning learning with fixed random feedback.

1 Introduction

DFA uses fixed random feedback instead of backpropagating errors, but useful feedback can still yield updates dominated by nuisance activity. This study tests whether conditioning activity or error directions improves learning while feedback remains fixed.

1 Introduction

  • High-variance presynaptic directions without task information can dominate DFA’s local update with nuisance structure, even when feedback provides a useful descent component.The update is an outer product with presynaptic activity.Source: §1 · p. 1
  • Conditioning reweights directions in local activity or error signals while leaving the feedback fixed.Activity anisotropy concerns presynaptic inputs; error anisotropy concerns teaching signals delivered to hidden units.Source: §1 · p. 1
  • The study separates activity and error factors, tests activity statistics, and compares conditioning schedules with tuned baselines at matched measured work.The activity operator applies FOOF to DFA errors; its matrix inverse is not new.Source: §1 · p. 1
  • Joint confirmation supports activity conditioning and an early-only efficiency tradeoff, while limiting additional error-side benefit and transfer of synthetic nuisance predictions.The study distinguishes stabilization, faster optimization, and improved generalization.Source: §1 · p. 1

2 Conditioned DFA

Conditioned DFA reweights activity, error, or both sides of the local update using inverse-second-moment operators. An aligned linear analysis characterizes activity conditioning’s spectral effect and the conditions under which it may help.

2 Conditioned DFA

  • The method estimates uncentered activity and error moments and uses separate positive damping values to stabilize their inverses.Error moments use per-example signals; comparisons also use independently selected damping and layerwise norm matching.Source: §2 · pp. 2–3
  • nDFA applies activity-side conditioning, while K-nDFA applies both activity- and error-side factors to the DFA pseudo-gradient.The activity operator shares FOOF’s core operator; the error moment is not a Fisher factor.Source: §2 · p. 2
  • The updates use minibatch-wide layer statistics and damped solves, not backpropagated hidden errors, and are not synaptically local.Exact sample-space identities can replace dense solves when a minibatch has fewer examples than units.Source: §2 · p. 2
  • Activity-side conditioning is right multiplication in weight-gradient coordinates, while error-side conditioning is left multiplication.The terminology refers to inverse-second-moment preconditioning with power-1 updates, except for the ZCA control.Source: §2 · p. 2

2.1 Linearized analysis: inverse-second-moment conditioning

  • In the post-alignment linear-Gaussian regime, activity conditioning changes each input eigendirection’s factor from λ_i to λ_i/(λ_i+λ_A).The predicted advantage is conditional on task orientation; realized learning also depends on residual, initialization, step size, and sampling noise.Source: §2.1 · p. 3
  • The aligned analysis does not explain DFA alignment acquisition or establish a general early-stopping principle; a separate construction varies factor orientations at fixed raw update.The separate construction examines how activity or error orientations can improve or worsen approximate credit at the same moment spectrum.Source: §2.1 · p. 3

3 Experimental setup

The experiments separate directional effects from stabilization before testing conditioning against stronger baselines. The practical study includes CIFAR-10 MLP comparisons at matched measured learning time and a joint confirmation protocol.

3 Experimental setup

  • An eight-class synthetic suite varies task geometry, label noise, and sample size, while historical MNIST and Fashion-MNIST experiments separate activity and error factors at short horizons.The historical protocols and test-selection limitations are reported in Appendices D–E.Source: §3 · p. 3
  • The primary practical study compares tuned DFA and BP with FD-DFA, activity conditioning, early-only conditioning, and FOOF on CIFAR-10 ReLU MLPs with batch normalization.Core arms use per-method validation selection and equal measured learning time on the same accelerator.Source: §3 · p. 3
  • Additional interventions examine centered covariance, conditioning windows, temporary error conditioning, and transfer predictions on image-background tasks.FOOF uses averaged activity moments and periodic inverse updates without norm matching; A uses current minibatch moments and norm matching.Source: §3 · p. 3
  • Figure 1D uses population linear simulations at condition number 50 and fixed alignment, with separately selected learning rates and three realizations.The caption notes that updates to target loss depend on task orientation.Source: Figure 1 · p. 4
  • The joint protocol fixes settings and endpoints, uses ten paired seeds, and applies global Holm correction across 21 paired tests in the primary family.Intervals are conditional on the data split and selection procedure; equal measured learning time excludes validation and export and does not imply equal FLOPs.Source: §3 · p. 4

4 Results

Across synthetic and vision experiments, activity conditioning improved tuned DFA on CIFAR-10 at matched measured learning time, while its benefits varied with task and training regime. Error conditioning helped short-budget training, but its incremental benefit under stable training was not confirmed after multiple-comparison correction.

4.1 Synthetic controls separate stabilization from geometry

  • In the focused synthetic ablation, activity-only nDFA minus DFA was +20.9 pp in nuisance, +6.5 pp in low-sample/noisy, +1.1 pp in mixed, and −0.3 pp in clean task-aligned cells.Across stressed regimes, shared-rate raw DFA instability contributed to the gaps; a clean, norm-matched control showed no activity benefit.Source: §4.1 · p. 4, Figure 2 · p. 5

4.2 Error conditioning: short-budget gains and a stable-training limit

  • Error conditioning improved raw DFA and the addition of an error factor improved short-budget endpoints in historical studies, but these fixed-rate tests did not compare tuned, converged baselines.The ReLU/softmax MNIST replication preserved the effects; the original MNIST seeds had already been evaluated on test data before validation development.Source: §4.2 · p. 5, Figure 3 · p. 6
  • Under stable CIFAR-10 training, adding error conditioning changed update directions substantially, but its small favorable accuracy and loss effects did not survive global correction.It also failed required performance comparisons with diagonal error conditioning, so the prespecified criterion for an additional error-side benefit was not met; this does not show that the factor is exactly ineffective.Source: §4.2 · p. 5

4.3 Controls separate scale, covariance, and normalization

  • Norm-matched controls support directional benefit beyond scalar step-size change, while the gain grew with designed nuisance-to-task energy only as a descriptive association.The same activity operator helped BP in stressed synthetic conditions; inverse-square-root conditioning captured most input-side gain, and full inversion added more in nuisance-dominant and low-sample/noisy regimes.Source: §4.3 · p. 6, Figure 4 · p. 7
  • Full and centered activity conditioning outperformed a mean-preserving diagonal-covariance control in every confirmation seed, supporting a role for centered correlations beyond the tested mean and diagonal alternatives.The result does not exclude other coordinatewise methods.Source: §4.3 · p. 6, Figure 5 · p. 8

4.4 Activity gains survive stronger baselines and matched work

  • Activity, early activity, and FOOF-DFA improved tuned DFA on CIFAR-10 at matched measured learning time, with every paired seed favoring each method.All three passed the prespecified practical and corrected statistical criteria; FOOF-DFA had the highest mean accuracy among tested DFA methods, while activity conditioning had lower mean CE.Source: §4.4 · p. 7, Figure 5 · p. 8
  • Early-window conditioning outperformed late-window conditioning for both DFA and BP; early-only activity conditioning retained accuracy within the prespecified margin while using about one-third less measured work.The timing interaction did not establish a difference between BP and DFA, and the work tradeoff applies to the separate fixed-epoch recipe.Source: §4.4 · pp. 7–8, Figure 4 · p. 7, Figure 5 · p. 8
  • Predicted background-task gains were not confirmed, and activity conditioning worsened corrected cross-entropy on both constructed background tasks.A separate fixed-recipe intervention found that batch normalization restored ordinary loss scales for raw and conditioned DFA; reducing feedback scale alone did not.Source: §4.4 · p. 8

5 Related Work

Related work positions conditioned DFA as update preconditioning that complements approaches focused on feedback pathways. The paper distinguishes its current-batch activity and local-error factors from related normalization, gradient-alignment, and accumulated-gradient methods.

5 Related Work

  • Conditioning targets DFA’s local update, whereas GrAPE aligns feedback using forward-mode Jacobian estimates and BP anchors.The paper presents these as complementary axes.Source: §5 · p. 9
  • Shampoo uses accumulated gradient-derived factors, unlike the current-batch activity and local-error moments used here [Gupta et al., 2018].The paper’s contribution is a controlled factor study under imperfect fixed feedback.Source: §5 · p. 9
  • Related approaches include representation normalization and decorrelation, task-dependent preconditioning, and spectral-gradient analyses; this work applies conditioning factors under fixed DFA feedback [Desjardins et al., 2015; Huang et al., 2018, 2019; Amari et al., 2021; Heckel and Yilmaz, 2021; Yoshida and Nitanda, 2025; Braun et al., 2026].FA uses fixed random feedback instead of exact weight symmetry, and DFA directly broadcasts output error to hidden layers [Nøkland, 2016; Refinetti et al., 2021].Source: §5 · pp. 8–9

6 Discussion and Limitations

The discussion finds robust practical support for activity conditioning but more limited evidence for error conditioning and transfer beyond the tested settings. It also identifies boundaries on the scope and interpretation of the results.

6 Discussion and Limitations

  • Activity conditioning, including early-only conditioning and FOOF, improves tuned DFA; centered correlations contribute beyond the tested mean-preserving control, and early-only use trades some accuracy for less work.These are benefits of established conditioning operators, not a new matrix inverse or a general advantage over backpropagation.Source: §6 · p. 9
  • Error conditioning changes update directions and improves short-budget models, but its small stable-training gain fails the corrected criterion; background-task predictions also fail.The aligned theory does not explain alignment acquisition, and timing interventions do not establish a DFA-specific mechanism.Source: §6 · p. 9
  • Practical evidence is mainly from MLPs and fixed data splits; training-seed uncertainty excludes dataset and hyperparameter-selection uncertainty, and measured-time matching does not imply equal FLOPs.Standard test sets had historical project exposure, and the main operations are layer-wide rather than synaptically local.Source: §6 · p. 9

Supplementary material

The supplement documents the joint confirmation, implementation, statistical criteria, and activity and timing studies. It also records the protocol boundaries and complete primary evidence, including failed predictions.

A.1 Design, selection and replication

  • The final inventory contains 880 models in eight cohorts with ten selected-setting seeds; real-data uncertainty is conditional on fixed splits and choices, and no test score selected a setting.The standard CIFAR-10, MNIST, and Fashion-MNIST test sets had historical project exposure.Source: §A.1 · p. 13

A.2 Architecture, operators and work accounting

  • A uses minibatch moments and norm matching; FOOF uses averaged moments and periodic inverse updates without norm matching, while measured time excludes validation and export and is not FLOPs.Table 2 distinguishes synchronized learning-work budgets from total elapsed experiment time and separates the efficiency endpoint from the validation-selected endpoint.Source: §A.2 · p. 14, Table 2 · p. 14

A.3 Hardware deviation and interruption handling

  • The selected CIFAR cohort moved from H200 to RTX PRO 6000 Blackwell before confirmation; its average update count rose about 35% versus development, not as a controlled hardware speedup.The main H100 CIFAR comparison was unaffected.Source: §A.3 · p. 14

A.4 Statistical criteria fixed before testing

  • The primary family used 21 paired tests with global Holm correction; secondary benchmark equivalence used individual 90% intervals and a ±1-point margin, which MNIST label noise did not meet.Table 3 lists all primary contrasts and notes that individual intervals are not simultaneous.Source: §A.4 · p. 15, Table 3 · p. 15
  • Early-only conditioning passes its fixed-epoch noninferiority criterion: the accuracy difference is −0.164 points, its lower bound is −0.401, and the work ratio is 0.658.This is a conditional accuracy–work tradeoff, not equal performance or a schedule-independent result.Source: §A.4 · p. 15

A.5 Interpretation of the selected comparisons

  • In the ten-seed CIFAR-10 confirmation, A, early A, and FOOF-DFA favor tuned DFA in accuracy without increasing mean CE; every paired seed favors each method.All three pass the prespecified practical and corrected statistical criteria; Table 5 gives the complete practical cohort.Source: §A.5 · p. 15, Table 5 · p. 16

A.6 Activity statistics

  • The activity geometry controls support a role for centered correlations beyond the tested mean-preserving diagonal-covariance control, not a universal failure of diagonal or mean-based methods.Table 6 reports final fixed-epoch geometry endpoints.Source: §A.6 · p. 16, Table 6 · p. 16

A.7 Timing and efficiency

  • Timing results show the early-window DFA effect survives correction, but its extra benefit over BP does not; shared-state interventions also find gains after late forks.The late-fork gains contradict an irreversible-capture interpretation.Source: §A.7 · p. 17, Table 9 · p. 17

A.8 Temporary error conditioning

Early error conditioning changed update directions, but its small favorable CIFAR-10 performance estimates did not meet the prespecified corrected-evidence criterion. The accompanying image-benchmark predictions were not confirmed, and the secondary synthetic interaction does not establish a general DFA-specific mechanism.

A.8 Temporary error conditioning

  • Early K had favorable mean CIFAR-10 accuracy and CE versus A, but neither performance comparison survived Holm correction, so the prespecified error-conditioning claim failed.Only the directional criterion passed; early error factors ran during the first quarter, with activity conditioning throughout, and endpoints were selected at 200 seconds of measured learning work.Source: §A.8 · pp. 17–18, Table 10 · p. 18

A.9 Image-benchmark predictions

  • Neither constructed image-background condition met the full prediction, CE worsened significantly after correction in both, and the original background task showed no corrected benefit.The constructed variants were paired conditions, not the original MNIST-variations dataset; the image-benchmark tables retain all six prespecified families and conditions.Source: §A.9 · p. 18, Table 12 · p. 19, Table 13 · p. 19

A.10 Secondary synthetic factorial

  • The secondary FOOF interaction was 11.04 points without BN versus 3.84 with BN, but it was outside the primary family and does not establish a general DFA-specific mechanism.The individual 95% intervals were [9.84, 12.24] without BN and [−0.06, 7.73] with BN.Source: §A.10 · p. 20, Table 15 · p. 20

A.11 Seed-level primary contrasts

  • Tables 16–17 report paired seed-level contrasts on fixed real-data splits; these are training replications, not independent datasets.Display rounding did not enter the statistical calculations.Source: §A.11 · p. 20, Table 16 · p. 21, Table 17 · p. 21

B Linear analysis and simulation protocols

This section analyzes and simulates conditioning, spanning an explicitly aligned linear model and the conditioning solves, experimental protocols, and synthetic evidence. The linear results concern idealized models, while mechanism checks are limited by replication, non-prospective diagnostics, and evaluation scope.

B Linear analysis and simulation protocols

  • Conditioning replaces each aligned BP/DFA input-direction factor λ_i with λ_i/(λ_i + λ_A), reducing the input-side condition number without changing the feedback direction.This exact population identity assumes fixed readout and perfect alignment; it is not a guarantee of faster nonlinear training, and isotropic inputs yield no condition-number improvement.Source: B Linear analysis and simulation… · pp. 22–23

B.1 Finite-sample damping: scope of the implemented estimator

  • With a full-rank minibatch activity moment, zero-damping conditioning cancels that same empirical moment exactly; damping remains useful for singularity, noise, nonlinearities, and numerical effects.The jointly estimated gradient and factor do not imply a universal interior damping optimum, so damping is selected empirically.Source: §B.1 · p. 24

B.2 Mode timing: generalization dynamics under conditioning

  • Under the population-spectrum approximation, BP and aligned DFA fit high-variance modes first, while nDFA compresses timing gaps and all three rules reach the same least-squares destination.Figure 6 illustrates compression to ρ_nDFA=5.5 at λ_A=0.1 while preserving mode order; its simulations check the model rather than independently validate its assumptions.Source: §B.2 · p. 25, Figure 6 · p. 27
  • In the two-block model, conditioning lowers the best trajectory risk when task modes are slower than nuisance modes, but raises it when task modes are faster.The Figure 6 task-aligned control reverses the gain; the crossing depends on task location, noise, and damping, and the result is not a guarantee for nonlinear DFA.Source: §B.2 · pp. 25–26, Figure 6 · p. 27

C Implementation, statistics and study provenance

  • The implementation uses fixed random feedback and minibatch activity/error factors to form conditioned pseudo-gradients, with study-specific normalization, damping, and optimizer choices.The supplied protocol distinguishes centered diagnostic covariances from the uncentered moments inverted during training; FOOF averages are an exception to minibatch recomputation.Source: C Implementation, statistics and study… · pp. 27–28

C.1 Diagnostic definitions

  • The projected BP-step ratio measures instantaneous descent in BP units but depends on update magnitude as well as direction, so it is not a training objective.Feedback rank and activity spectra characterize the geometry on which feedback acts.Source: §C.1 · p. 28

C.2 Moments, damping, and optimization

  • Moment scale determines inverse-update strength, so loss normalization and damping conventions are specified; CIFAR-10 A/E/K studies use trace-scaled damping with a 10^-6 floor.Moments are uncentered unless a control specifies centering, and activity/error conditioning affects hidden weight matrices while the output classifier receives its exact supervised gradient.Source: §C.2 · p. 28

C.2 Moments, damping, and optimization; C.3 Exact solves in minibatch space; C.7 Computational scope

  • Conditioning rescales each nonzero conditioned gradient to the raw DFA gradient’s Frobenius norm, while exact minibatch-space solves preserve current-batch moments and the learning rule using smaller systems when advantageous.When both factors use minibatch space, solve cost is O(b^3 + b^2(din + dout) + b dindout); the dense gradient still requires O(din dout) storage, and moving averages need a separate representation or approximation. The direct dense solve costs O(d^2b + d^3) time and O(d^2) storage.Source: §C.2 · p. 28, §C.3 · p. 29, §C.7 · p. 31

C.4 Replication units, selection, and statistical comparisons

  • CIFAR-10 replicates jointly varied applicable randomness and used paired seed-level comparisons, while the declared 21-test family distinguishes the joint confirmation from earlier comparisons.The paired intervals are conditional on the benchmark, reused split, and development procedure; they do not represent dataset or hyperparameter-selection uncertainty.Source: §C.4 · p. 29

C.5 Final evaluation and reproducibility scope

  • Final model selection preceded testing, but the official test set had been used in earlier project studies, and compact prediction records cannot establish hardware timing or original test-access order.Prediction-based reproduction checks verify reported accuracy and losses without retraining or checkpoint inference.Source: §C.5 · p. 30

D.1 Predicted anisotropy gain

  • Across 128 synthetic cells, the designed nuisance/task energy ratio correlated with activity-nDFA gain over DFA (Spearman ρ = 0.62), but this was a descriptive mechanism check.The ratio is derived from generator ground truth and uses exploratory, test-selected feedback-rank summaries, so it is neither an independent-task significance test nor a validated predictor for new tasks.Source: §D.1 · p. 32

D.2 A prospective estimate of the mechanism variable does not transfer

  • The prospective pre-training diagnostic was no better than task-blind κ(C) on synthetic data and reversed association on held-out vision, so it did not provide a reliable decision rule.It estimates activity covariance and class-conditional mean activations from an untrained network; the tested initialization statistics did not capture enough subsequent learning dynamics to predict benefit.Source: §D.2 · p. 33, Table 23 · p. 33

D.3 Conditioning and feedback-alignment dynamics

  • Historical synthetic alignment trajectories do not establish an alignment-rescue mechanism because shared learning rates did not separate stabilization from useful-credit acquisition.The post-alignment model assumes useful credit; gradient alignment alone is insufficient for good learning performance (Chu and Bacho, 2023).Source: §D.3 · p. 33

D.4 Reduced variability across feedback and order seeds

  • Across 156 synthetic, vision-MLP, and MLP-Mixer cells, the median DFA-to-nDFA feedback/order-seed accuracy-SD ratio was 6.1, but the analysis does not isolate feedback variability.Feedback seeds also controlled minibatch order, and the reused-grid summaries are descriptive rather than independent-cell inference.Source: §D.4 · p. 33, Figure 7 · p. 35, §D.5 · p. 35

D.5 Regularization, scalar scaling and moment controls

  • With BatchNorm under a shared learning rate and 14-epoch horizon, conditioning improved accuracy in nuisance and low-sample regimes but reduced it in the other two.The comparison does not establish the ordering after independent learning-rate tuning or convergence.Source: Table 28 · p. 36

D.6 Transfer to a spurious-correlation task

The spurious-correlation task tests conditioning under a training-to-test reversal in color–label correlation. Conditioning reduced seed variability, but the accuracy gains over DFA did not survive Holm correction.

D.6 Transfer to a spurious-correlation task

  • Conditioning reduced seed variability and brought mean accuracy close to BP, but nDFA–DFA accuracy gains did not survive Holm correction.The comparison used eight paired training seeds; the table reports descriptive differences.Source: §D.6 · p. 36, Table 30 · p. 36
  • Training color–label correlation was 85%, versus 15% at test, and the grayscale probe removed color.Labels remained clean; this differs from the two training environments and label noise of Arjovsky et al. (2019).Source: Table 30 · p. 36
  • Earlier repeated MNIST test evaluations preclude interpreting that comparison as test-blind confirmation.The disclosed prior test exposure limits the confirmation interpretation.Source: Figure 8 · p. 37

E Independent activity and error confirmations

The factor studies find benefits from activity and error conditioning in several MNIST settings, while bounding what the comparisons establish. They then motivate CIFAR-10 scale studies using a shared forward and data protocol.

E Independent activity and error confirmations

  • On original MNIST, A-nDFA gained +13.32 pp and E-nDFA +4.76 pp over DFA; K-nDFA gained +0.40 pp over A-nDFA, but prior test exposure limits confirmation.Each contrast had the same sign across five seed-level pairs; the two-sided Wilcoxon test was floor-limited at p = 0.0625.Source: Table 31 · p. 37, E Independent activity and error… · pp. 37–38

E.1 Preregistered Fashion-MNIST replication and source-swap diagnosis

  • On frozen Fashion-MNIST, E-nDFA gained +1.77 pp over DFA and K-nDFA gained +0.50 pp over A-nDFA; the BP-source comparator did not isolate an error-source effect.The BP-source moment was much smaller than the local DFA-error moment, making it nearly scalar under shared damping and norm matching.Source: Table 32 · p. 38, §E.1 · p. 38

E.2 Fresh-seed ReLU/softmax architectural replication

  • In fresh-seed ReLU/softmax MNIST, error nDFA improved DFA by +7.53 ± 0.86 pp, while K-nDFA improved A-nDFA by +0.90 ± 0.04 pp.The two-sided rule gave the best accuracy and loss, but exploratory Fashion-MNIST development did not preserve its incremental gain, so ReLU dataset generality is not claimed.Source: §E.2 · p. 38, Table 33 · p. 39

E.3 Post-hoc seed extensions

  • Post-hoc seed extensions retained the signs of the factor contrasts but did not remove earlier test exposure or establish tuned-baseline superiority.The extensions improve descriptive precision; the original cohorts remain the basis of the original tables and figures.Source: §E.3 · p. 39, Table 34 · p. 39

F CIFAR-10: activity geometry and transfer across scale

F.1 Shared forward and data protocol

  • The shared-BN CIFAR-10 protocol uses a 2,000-image validation split and either a nested 10,000-image pool or the remaining 48,000 training identities.It uses flattened-image ReLU MLPs with hidden widths 1024–512 or 2048–1024, paired augmented views, and frozen BN running statistics at validation and test.Source: §F.1 · p. 39

F.2 Larger training pools, wider models, and longer baselines

Activity conditioning retained an equal-update advantage across the tested CIFAR-10 pool sizes and widths, although longer raw-DFA training reduced it. A full activity moment also outperformed a diagonal control in one tested recipe.

F.2 Larger training pools, wider models, and longer baselines

  • Across the tested pool sizes and widths, A favored accuracy and cross-entropy over DFA at equal updates; longer raw-DFA training reduced the advantage, and longer BP outperformed A.Increasing width enlarged the A–DFA gap by degrading DFA more than A, not by improving A’s absolute accuracy.Source: §F.2 · p. 40, Table 37 · p. 41, Table 36 · p. 41
  • The four equal-update accuracy comparisons had exact sign-flip p = 0.0078125 and Holm-adjusted p = 0.03125, but these tests concern seed variation within fixed conditions.The longer raw baselines still used somewhat less measured work than A, so the extension was not an exact equal-time comparison.Source: §F.2 · p. 40

F.3 Full versus diagonal activity moments

  • 46.51 ± 1.04% test accuracy for full conditioning exceeded 42.10 ± 1.34% for the diagonal control, a 4.41-point full-minus-diagonal contrast.The contrast supports useful off-diagonal uncentered moment structure in this recipe, but does not distinguish centered correlations from mean-activity effects.Source: §F.3 · p. 42

G CIFAR-10: transfer of the error factor

Activity conditioning improved DFA at matched measured work, while the added error-factor benefit varied across settings. Longer-budget tests retained the activity gain but did not establish convergence or transfer beyond the tested architecture and dataset.

G CIFAR-10: transfer of the error factor

H CIFAR-10: comparisons at matched measured work

H.1 Common optimizer and work budget

  • The work budget charges sampling, augmentation, gradient construction, conditioning or decorrelation, and optimization, while excluding validation and export.The compared budgets were separate training runs, not checkpoints along one trajectory.Source: §H.1 · pp. 43–44

H.2 Equal learning-rate search for BP, DFA, and A

  • At matched measured work, A improved DFA in test accuracy and cross-entropy across paired seeds despite completing fewer updates; BP remained stronger.The final-test primary comparison remained significant under the declared two-comparison Holm adjustment; boundary learning-rate selections leave tuned convergence unresolved.Source: §H.2 · p. 44

H.3 Equal-budget comparison with forward decorrelation

  • A again improved DFA at matched work, but its primary contrast with FD-DFA spanned zero on validation and final test; neither an A advantage nor equivalence follows.BP remained stronger.Source: §H.3 · p. 45, Table 45 · p. 45

I Targeted tests of work, damping and normalization

  • The targeted follow-ups used five paired seeds per condition, completed all 202 planned cases, and evaluated each of 170 final models once on test.They were follow-up analyses rather than additions to an earlier confirmatory family; individual t intervals do not provide simultaneous coverage.Source: I Targeted tests of work, damping and… · p. 45

I.1 Transfer to longer measured work

  • At 60- and 120-second budgets, activity conditioning improved raw DFA, its observed gap to BP narrowed, and BP remained ahead.A–FD intervals spanned zero at both budgets, establishing neither superiority nor equivalence; these results do not establish converged performance or transfer beyond this architecture and dataset.Source: §I.1 · pp. 45–46, Table 46 · p. 46

I.2 Width-specific error damping and a diagonal control

The width-specific follow-up found persistent activity-conditioning gains, while the selected error-side operators stayed close to DFA and K stayed close to A. A separate fixed-recipe intervention found that batch normalization restored ordinary predictive-loss scales, but reducing feedback scale alone did not prevent extreme losses.

I.2 Width-specific error damping and a diagonal control

  • Activity gains persisted at both widths, while selected full and diagonal error conditioning stayed close to DFA and K stayed close to A.Strong damping made full E nearly scalar: its final-update cosine with the raw update exceeded 0.999 in every recorded layer and seed, with damped condition number below 1.10.Source: §I.2 · p. 46, Table 47 · p. 46
  • The width-specific study does not establish a robust error-side gain: strong selected damping largely suppressed the error transformation, without proving equivalence or excluding gains elsewhere.The fixed relative damping need not preserve conditioning across widths when error-moment rank is batch-limited, and the recorded spectra do not establish this as the sole cause of earlier transfer failure.Source: §I.2 · p. 46

I.3 Crossing normalization and feedback scale

  • With batch normalization, all four rules had ordinary predictive-loss scales, and activity conditioning retained an accuracy gain; lowering feedback scale without batch normalization did not prevent extreme losses.This fixed-recipe intervention identifies a normalization effect under one common recipe, not the cause of every difference between historical protocols.Source: §I.3 · p. 47, Table 48 · p. 47, Figure 10 · p. 47

J Predictive stability under a separate selected protocol

A separate CIFAR-10 protocol tested whether method ordering persisted under different normalization and training choices, without isolating any one changed choice. In that protocol, activity conditioning did not improve raw DFA, error conditioning performed worse, and activity conditioning also improved BP; predictive-loss audits confirmed instability beyond a small tail.

J Predictive stability under a separate selected protocol

  • The comparison used a separate no-BN, 30-epoch CIFAR-10 protocol with a fixed train/validation split, and its outcomes cannot be attributed to any single changed training choice.The study selected settings by validation accuracy from an initial grid and a declared adaptive lower-rate grid, then evaluated five fresh paired replicates on the test set.Source: J Predictive stability under a… · pp. 47–48
  • The completed development grids had 199 finite endpoints and 41 numerical failures; selected configurations sometimes had extreme finite losses but remained eligible under accuracy-first selection.The completed rerun, rather than the interrupted attempt, entered the development comparison.Source: J Predictive stability under a… · p. 48, Table 50 · p. 49
  • In the separate protocol, A did not improve raw DFA, E performed worse, K exceeded A with different activity damping, and forward-normalized DFA methods outperformed update conditioners.Activity conditioning also improved BP, while extreme finite cross-entropies showed that norm matching did not guarantee stable predictive losses.Source: J Predictive stability under a… · p. 48
  • Large median losses for A and E showed that instability extended beyond a small tail, while the highest-loss one percent contributed only a minority of total loss.The float64 audit agreed with the original evaluation for all 50 endpoints; it confirms predictive failure but not its cause.Source: J Predictive stability under a… · p. 48, Table 52 · p. 50

K Moment orientation and imperfect credit

An engineered fixed-feedback diagnostic varied activity and error moment orientations while holding the raw update fixed, showing that conditioning can change its alignment with the true gradient. Yet better initial alignment need not yield a better endpoint, and the paper connects conditioning to methods that alter credit source or routing while noting implementation limits.

K Moment orientation and imperfect credit

  • The diagnostic prescribed compatible activity and error moments in one fixed-feedback network while keeping the raw cross moment fixed, so it isolates operator effects rather than discovering useful credit.The construction varied activity and error orientations independently and used the desired true gradient explicitly.Source: K Moment orientation and imperfect… · p. 50
  • At equal damping, signal-low conditioning improved update direction while exchanging its eigenspaces worsened it; the moment spectrum alone did not determine the benefit’s sign.Two favorable factors reinforced one another, while opposing orientations canceled at equal damping.Source: K Moment orientation and imperfect… · p. 51, Figure 11 · p. 51
  • For a positive raw-gradient alignment bound, conditioning guarantees descent only when the update rotation satisfies c > sin θ; local covariance estimates do not certify the required gradient margin.The bound is sharp, and the K-factor condition number is the product of its two factor condition numbers.Source: K Moment orientation and imperfect… · p. 52
  • A directional guarantee does not certify a finite step: at zero derivative, the quadratic loss term can still raise loss.The boundary construction exhibits ascent, zero derivative, and descent around the threshold while raw DFA remains a descent direction.Source: K Moment orientation and imperfect… · p. 52
  • The stationary mismatch relies on a fixed bottleneck readout and its unlearnable output complement, and the analysis does not establish stationary structure for nonlinear networks or trained readouts.The comparison does not make the mismatch a general property of full-rank linear DFA.Source: K Moment orientation and imperfect… · pp. 52–53
  • With both signal-low factors, K initially aligned closest to the true gradient among local rules but approached a mismatched stationary point faster and ended with higher reducible loss.This engineered trajectory shows that improved initial direction need not imply a better endpoint.Source: K Moment orientation and imperfect… · p. 53, Figure 11D

L Connections to other credit-assignment rules

  • Conditioning changes how a credit signal is used, complementing methods that change its source or routing; its layer-wide statistics and solves leave synaptic locality open.The implementation also does not resolve every limitation of approximate credit.Source: L Connections to other… · p. 53
Loading 2607.18574v2…