PaperlayerView exact revision on arXiv

Source-linked AI summary

Omega-S: A Functional Resilience Index for LLM Fine-Tuning

Alberto Acedo

arXiv:2608.03887v1cs.LGcs.NEq-bio.MN

TL;DR

Large language models can forget previously learned capabilities during fine-tuning. Omega-S is a weight-only graph-based regularizer that improves retention over unregularized fine-tuning and standard baselines in the reported Llama-3-8B LoRA experiment.

  • Problem

    Fine-tuning large language models can cause catastrophic forgetting, degrading capabilities learned before adaptation.

  • Method

    Omega-S is a graph-based regularizer computed from weight matrices without previous-task data, Fisher information, or stored old weights.

  • Results

    0.173 → 0.238 absolute HumanEval pass@1, with Omega-S outperforming no regularization on 9 of 10 seeds and retaining 61.9% → 84.1% over tuned weight decay.

  • Takeaways & Limitations

    In this formulation, Omega-S’s measured utility comes from reducing node-degree variance rather than activating its intended clustering channel.

  • Takeaways & Limitations

    The main retention result is demonstrated on Llama-3-8B with LoRA, while controlled structural experiments were conducted on a small Split-MNIST MLP.

Abstract

from arXiv · show

Fine-tuning a large language model on new data degrades what it previously learned. We present Omega-S, a drop-in penalty computed from the weight matrix alone: it needs no previous-task data, no Fisher matrix and no stored copy of the old weights. It is three lines in an existing training loop and adds under 4% to the cost of a step. Retention. On Llama-3-8B with LoRA, fine-tuned from code to prose and measured by HumanEval over ten seeds, Omega-S retains more of the original capability than no regularisation on 9 of 10 seeds (0.173 -> 0.238 absolute pass@1; sign test one-sided p=0.011, Wilcoxon p=0.006), as a retention ratio, 62.9% -> 84.1%. It also beats tuned weight decay on 10 of 10 seeds (p=0.002) and tuned EWC on 8 of 10 (p=0.014), every arm re-measured in the same session. Mechanism, measured rather than asserted. Omega-S is topological by construction, its objective built from Tr(A^3), but we measured which of its four factors actually moves and three do not: their elasticity with respect to the weights is at or below 1e-4, against 9e-3 for the degree-variance term. As implemented, the composite reduces to a penalty on the variance of node degrees, which means row magnitude in square modules and directional alignment in non-square ones. We report this because a method whose name promises one thing and whose gradient does another should say so. We also enumerate the open design choices, including a contrast-preserving construction that does what it was designed to do and makes retention worse on all ten seeds. Repeating an identical configuration, same seed and same hardware, gives a standard deviation of 0.104 in retention ratio. We have not found this quantified for low-rank fine-tuning of language models, and it bounds every seed-paired comparison in this literature, ours included. Code, per-seed results and the full record of negative results are available.

1. Introduction

Omega-S addresses catastrophic forgetting in LLM fine-tuning with a graph-based regulariser that requires no previous-task data, Fisher matrix, or stored old weights. On Llama-3-8B, it improves code retention, while measurements show its gradient effectively reduces to degree-variance regularisation and remains subject to substantial run-to-run variation.

  • Motivation: Omega-S targets weight monopolies, which standard weight decay and SAM do not directly address.The paper links these monopolies to overwritten representations during fine-tuning.
  • Method: Under 4% is the added training-step cost for Omega-S, which needs no previous-task data, Fisher matrix, or stored old weights.The penalty is built from Tr(A^3) and estimated with Hutchinson’s method at O(N^2) rather than O(N^3).
  • Retention: 0.173 → 0.238 absolute HumanEval shows Omega-S improving retention over no regularisation on 9 of 10 seeds.The comparison has sign test p = 0.011 and Wilcoxon p = 0.006; Omega-S also beats tuned weight decay on 10 of 10 seeds and tuned EWC on 8 of 10.
  • Mechanism: 10−4 or below are the elasticities of three Omega-S factors, versus 9 × 10−3 for degree variance.As implemented, the objective therefore reduces in practice to a penalty on the variance of node degrees, despite being topological by construction.
  • Limitations: 0.104 is the standard deviation in retention ratio when an identical configuration is repeated.The paper reports this as a bound on interpreting seed-paired comparisons in low-rank language-model fine-tuning, including its own.

2. Method

Omega-S is defined in raw-trace and log-ratio forms, with the reported Llama-3-8B retention experiments using the log-ratio composite. The composite removes weight-scale sensitivity but, through its bounded logistic map, can collapse contrast and immobilize clustering.

  • Forms: Omega-S has two non-interchangeable forms: retention experiments use the log-ratio composite, while earlier unreported MLP pruning experiments use the raw penalty.The paper distinguishes these forms because they produce different results, with the raw form performing worst on retention.
  • Raw trace penalty: The raw trace penalty is homogeneous of degree six in W, making its gradient degree five and causing it to explode with growing weights or vanish as weights shrink.This scale dependence can make the penalty stop acting when another training pressure reduces weight magnitude.
  • Log-ratio composite: The log-ratio composite uses scale-independent quantities and logarithms, making its gradient proportional to ∇f/f rather than ∇f.The logistic map sends entries to [0.5, 1), while the logarithm removes dependence on the penalty value's size.
  • Log-ratio composite: |Wcorr| spanning several orders of magnitude makes σ collapse most entries to nearly one value, pinning the contrast-dependent clustering term C near a constant.The bounded map solves scale sensitivity but creates this contrast-collapse cost.
  • Implementation cost: 79% reduction in computation time is achieved by the sequential decomposition versus explicit A^3 for m = 4096 and np = 3; amortised overhead is below 4% at K = 10.Omega-S is applied every K optimisation steps rather than necessarily every step, with the penalty gradient calibrated as a fixed fraction of the task gradient.

Quick Start

Omega-S is a drop-in PyTorch penalty requiring three added lines, with recommended support for LoRA fine-tuning through HuggingFace and PEFT. Its reported configuration fixes the graph construction, objective form, modularity orientation, update frequency, and gradient scale after testing alternative choices.

  • Quick Start: Three lines of code add Omega-S to an existing PyTorch training loop as a penalty applied during optimization.The loop adds omega(model) to the task loss before backpropagation.
  • Quick Start: The recommended LoRA setup works with any PEFT model using StochasticOmegaS(lambda_omega=0.05, k=10).The example preserves the existing model setup and adds the penalty inside the training loop.
  • Graph construction: 0.547 against 0.841, Wilcoxon p = 0.002: the cosine composite loses on all ten seeds despite increasing median triadic excess.The default bounded Gram-matrix construction returns C/D = 1.0000 in every tested module, matching its permutation null.
  • Modularity orientation: 76.6% retention against 84.1%, losing on 8 of 10 seeds: the opposite modularity orientation performs worse, but the +0.075 difference is below the 0.104 run-to-run standard deviation.The modularity term is numerically inert under either orientation, so the comparison is reported primarily for mechanism rather than performance.
  • Final configuration: The adopted configuration uses a bounded Gram-matrix map, log-ratio objective, M = 1/λ2, ten-step penalty updates, and a coefficient calibrated to 3% of the task-gradient magnitude.The objective choice, graph construction, and modularity orientation are treated as open design choices, with alternatives retained where they serve different purposes.

3. Experiments

Across ten seeds, Omega-S improved retention over no regularisation, tuned weight decay, and tuned EWC, while its measured mechanism reduced in practice to degree-variance regularisation rather than a broadly active topological composite. The experiments also show that absolute capability and retention ratio can diverge, and that plasticity on the new task was not measured.

  • Retention: 84.1% retention versus 61.9% for tuned weight decay, with Omega-S winning on 10 of 10 seeds; against tuned EWC, it won on 8 of 10 seeds with a +17.3 pp advantage.The weight-decay comparison had p = 0.002; the EWC comparison had +17.3 pp.
  • Limitations: Plasticity on the prose task was not measured: retention used HumanEval throughout, and assessing new-task performance would require a Wikitext perplexity measurement.The experiment therefore does not establish whether Omega-S costs plasticity on the second task.
  • Retention: 0.238 HumanEval pass@1 after prose versus 0.173 with no regulariser, with Omega-S ahead on 9 of 10 seeds by the absolute measure.The exact sign test was one-sided p = 0.011.
  • Retention: Omega-S and no regularisation were statistically indistinguishable on post-code HumanEval, with means of 0.291 and 0.268, Omega-S ahead on 5 of 10 seeds with one tie, and p = 0.50.The mean paired difference was +0.023 with sd = 0.050; post-code baselines were arm-specific because the penalty was applied during both tasks.
  • Controls: 17.5% retention for the row-norm variance control versus 84.1% for Omega-S, with the control losing on all ten seeds.The control used Omega-S’s calibration target and had p = 0.002, so the comparison tests spending the same penalty budget on norms rather than degree variance.
  • Mechanism: 0.0091 elasticity for Coex versus 0.0000, 0.0000, and 0.0001 for C, D, and M at base weights; during training, Coex fell 5.87% while the other factors stayed under 0.1%.Thus, as implemented under these conditions, the composite objective reduces in practice to a penalty on the variance of node degrees, despite being defined from Tr(A^3).

4. Conclusion

Omega-S is introduced as an efficient graph-based regulariser that improves sequential fine-tuning retention, while measurements show its current penalty acts through node-degree variance rather than clustering. The conclusion also identifies untested single-task generalisation and frames the method as a structural regularisation principle inspired by biological systems.

  • Method: Omega-S is presented as a graph-based regulariser whose objective uses Hutchinson trace estimation from the clustering coefficient of WW ⊤.The conclusion describes the method as computationally efficient for large-scale neural networks.
  • Retention and efficiency: 0.173 →0.238 absolute HumanEval: Omega-S leaves more code capability than no regularisation on 9 of 10 Llama-3-8B fine-tuning seeds.The method adds +1.5–3.7% overhead at K=10, has O(N^2) per-layer cost, and requires zero inter-GPU communication in LoRA+FSDP settings.
  • Mechanism: Node-degree variance, not clustering, is the measured mechanism through which Omega-S currently acts.The clustering channel is saturated by the logistic adjacency map and does not move, leaving Omega-S best understood as a degree-variance regulariser with an inert topological objective.
  • Limitations: Whether degree-variance control improves single-task generalisation beyond retention remains untested.Weight decay is used as a retention baseline here but is primarily a standard single-task generalisation tool, so this comparison is left as follow-up work.
  • Broader contribution: Structural regularisation principles observed in biological systems are proposed as productive priors for artificial neural network design.The work connects soil microbiome network observations to a mathematically grounded and empirically validated regulariser for LLMs.

Code Availability and Licensing

Omega-S’s complete PyTorch implementation is publicly available, with AGPL-3.0 covering specified non-commercial uses and separate commercial licensing required for for-profit deployment and related uses.

  • Code availability: The complete PyTorch implementation of Omega-S is publicly available at https://github.com/BiomeMakers/OmegaS-LLM.
  • Licensing: AGPL-3.0 permits free use, modification, and redistribution with attribution for non-commercial academic research, education, and non-profit experimentation.Network deployment of a modified version requires public disclosure of the modified source code.
  • Licensing: Commercial licensing is required for production deployment, commercial training-pipeline integration, or any use within a for-profit organisation, covering software and patent rights.Commercial licensing, research partnership, and collaborative-development inquiries should be directed to the author via the repository.

Appendices

The appendices contain supplementary framing, related work, secondary-supporting experiments, and the complete record of unsuccessful attempts. They are retained because the negative results are part of the paper’s report, although they are not needed to use Omega-S or judge its results.

  • Appendix scope: The appendices preserve framing, related work, secondary experiments, and negative results that are supplementary to using or evaluating Omega-S.The paper retains the full record of what was tried and did not work because those negative results are part of its report.

Note on the appendices · Figures that differ between the two configurations are dated by which appendix

The appendices mix an earlier M = λ2 orientation with the paper’s main M = 1/λ2 configuration, so differing figures must be interpreted by appendix. Table 1’s arms were re-measured together using the paper’s configuration and previously selected hyperparameters.

  • Note on the appendices: The main text uses the log-ratio composite with M = 1/λ2, and every arm was measured in a single session.Several appendices predate this configuration and instead use M = λ2.
  • Note on the appendices: The calibration sweep predates the main configuration and reports the alternative orientation M = λ2.It selected the penalty strength but was not repeated under the paper’s final configuration.
  • Note on the appendices: The row-norm strength sweep is separate from Table 1’s row-norm arm and tunes that control on its own grid.This appendix experiment also predates the main configuration.
  • Note on the appendices: No regulariser, weight decay, EWC, row-norm control, or cosine composite in Table 1 belongs to the predated appendix set.All five arms were re-measured together in one session using the paper’s configuration.
  • Note on the appendices: The earlier hyperparameter sweeps were not repeated; their selected hyperparameters were reused for evaluation in the common Table 1 session.Thus, what changed was selection, not evaluation.
  • Figures that differ between the two configurations are dated by which appendix: 76.6% mean retention for Omega-S in an appendix reflects M = λ2, whereas the paper’s configuration gives 84.1%.The distinction is stated once because appendix figures are dated by the configuration they use.

A. The row-norm control, swept on its own grid … B.3. Hutchinson Trace Estimator

The row-norm control does not reliably improve retention over no regularisation and is weaker than Omega-S, while the paper situates Omega-S in weight-graph topology and uses Hutchinson estimation to reduce trace-computation cost. These results are qualified by selection limitations and by the distinction between the effective log-ratio form and weaker alternatives.

  • A. The row-norm control, swept on its own grid: 6 of 10 seeds: the tuned row-norm control exceeded no regularisation, but mean absolute HumanEval was only 0.174 versus 0.168 (p = 0.38).Its retention standard deviation was 0.216 versus 0.145 for Omega-S, with one seed collapsing to 0.254.
  • A. The row-norm control, swept on its own grid: 8 of 10 seeds: Omega-S exceeded the row-norm control, although the one-sided comparison was not conventionally significant (p = 0.055).The control was selected using seeds 42 and 123, which were its two worst evaluation seeds, limiting confidence in the comparison.
  • A. The row-norm control, swept on its own grid: 53.9% mean retention: the raw Tr((WW ⊤)3) penalty was weaker than the 63.1% no-regularisation baseline and won only 1/10 against weight decay.The paper attributes this gap to scale sensitivity causing the raw sextic penalty’s gradient to vanish as weights shrink; the log-ratio form delivers the reported results.
  • B.1. Topological Properties of Weight Graphs: Weight matrices can be viewed as bipartite graphs, with neurons as nodes and weights as edge strengths; highly skewed degree distributions create weight monopolies.The paper distinguishes concentrated structure through degree variance rather than treating a high clustering coefficient alone as its signature.
  • B.1. Topological Properties of Weight Graphs: The clustering coefficient is proportional to Tr(A^3) and describes local connectivity density, but degree variance distinguishes distributed from concentrated triadic structure.This motivates using graph-topological quantities while avoiding the interpretation that high clustering alone indicates a weight monopoly.
  • B.2. Scale-Free Networks in Biological Systems: Scale-free networks follow P(k) ∼k−γ, combining resilience to random node failures with vulnerability to targeted attacks on hubs.Related work describes agricultural and soil microbial networks in which environmental pressures can disturb network properties and produce structural monopolies.
  • B.3. Hutchinson Trace Estimator: The Hutchinson estimator gives an unbiased stochastic approximation of a matrix trace and, for M = A3 = (WW ⊤)3, avoids materialising A.The matrix-vector product is decomposed to estimate the trace without explicitly constructing A^3.
  • B.3. Hutchinson Trace Estimator: O(6npmn) per layer: Hutchinson estimation costs this total versus O(m3) for explicit A3, while Hutch++ provides variance reduction for ill-conditioned matrices.Each step requires O(mn) operations.

B.4. Related Regularisation Methods

Omega-S differs from related regularisers by targeting weight-matrix structure rather than uniformly shrinking weights, seeking flat minima, or directly eliminating neurons. Its relationship to graph-spectral and group-sparsity methods remains limited by an inert λ2 channel and no demonstrated compression advantage.

  • Weight decay uniformly shrinks weights regardless of structural role and produces unstructured sparsity unsuitable for structural-pruning FLOPs reduction.
  • SAM targets flat loss minima and is blind to weight-matrix topology, making it complementary to Omega-S rather than a competing method.
  • Fiedler regularisation is the closest antecedent, penalising graph algebraic connectivity λ2 and equivalently applying a structurally weighted L1 penalty.The authors report that their λ2 channel is numerically inert and do not claim improvement over Fiedler regularisation; that comparison remains unrun.
  • The composite penalty combined with group lasso does not exceed group lasso alone in compression achieved, so Omega-S makes no compression claim here.

B.5. Geometric and Spectral Approaches to Continual Learning … B.10. Structural Control: Omega-S vs. Weight Decay

Across continual-learning and structural-control approaches, Omega-S is positioned as a training-time regulariser on whole-matrix topology rather than individual-weight importance, projection directions, or pruning budgets. Its measured effect is concentrated in degree variance, while comparisons remain bounded by setting differences and open objections about function-space interference.

  • B.5. Geometric and Spectral Approaches to Continual Learning: Natural orthogonality was already high in seven of eight directions, so orthogonalisation has limited room; Omega-S instead acts on degree distributions, making the families complementary.The comparison concerns row-space geometry and the weight-graph degree distribution, not the same quantity.
  • B.5. Geometric and Spectral Approaches to Continual Learning: Omega-S differs from per-weight regularisation by penalising whole-matrix degree-sequence variance, while whether polysemanticity extends to such distributional properties remains open.Its reported retention results were obtained in the weight basis, which the cited objection would not predict.
  • B.5. Geometric and Spectral Approaches to Continual Learning: Head-to-head comparison with projection methods is not reported because this study uses single-adapter LoRA and a held-out capability after a task pair, unlike multi-task streams of five to fifteen tasks.The cited settings and metrics are therefore not commensurable.
  • B.6. Non-Uniform Structured Pruning of LLMs: Omega-S operates during training for knowledge retention, whereas OWL and AlphaPruning allocate post-training compression budgets for inference efficiency.Omega-S is not claimed as a pruning-allocation signal, and the methods can be applied sequentially.
  • B.7. Conceptual Origin: From Soil Microbiomes to Neural Networks: The biological motivation is that disturbance reorganises network structure, but the cited soil studies do not support pooled directional claims, degree-distribution claims, or scale-free claims.The paper explicitly distinguishes the studies’ estimators, units of analysis, and clustering definitions.
  • B.8. Thermodynamic and Information-Theoretic Framing: Triangle counts alone do not distinguish evenly distributed connectivity from hub concentration; degree variance separates these configurations without assigning clustering a direction.The clustering coefficient is proportional to Tr(A^3).
  • B.9. Relation to Spectral Quantities: Tr(A^3) is a spectral functional, but Omega-S is used as a training objective whose gradient differs from other functionals and is tractable with matrix–vector products and Hutchinson estimation.Entropy-based alternatives require eigendecomposition at every optimisation step.
  • B.10. Structural Control: Omega-S vs. Weight Decay: 8× greater reduction in degree variance was achieved by Omega-S versus weight decay (0.136 vs. 1.12), with 0.54 pp accuracy cost versus 2.90 pp.Weight decay reduced the maximum hub more aggressively (69.00 vs. 116.83), while Omega-S redistributed degree concentration while preserving legitimate hubs.

B.11. How the hyperparameters were selected, and the anomaly this explains

The apparent anomaly—both regularised baselines retaining less than no regularisation—reflects selection on only two evaluation seeds rather than disjoint seeds. The conservative comparison is Omega-S versus no regularisation, while Omega-S’s own boundary optimum also requires broader selection.

  • Anomaly and interpretation: 59.3% and 59.3% retention for weight decay and EWC fell below 63.1% without regularisation, so the +17.3 pp margin should not be treated as definitive.The baselines represent reasonable fixed settings, not necessarily each method’s best achievable performance.
  • Selection protocol: Weight decay returned λ = 0.05 from {0, 0.01, 0.05, 0.1}, while EWC returned 103, the highest-scoring and most stable value with standard deviation 0.011.The weight-decay grid included zero but did not select it; EWC alternatives had standard deviations of 0.06 to 0.08.
  • Selection protocol: Two selection seeds overlapped with the ten evaluation seeds, yielding selection retention of 0.69 for weight decay and 0.72 for EWC versus 0.59 and 0.59 across all seeds.The selection set was therefore favourable and non-disjoint, producing the expected regression on the larger evaluation set.
  • Conservative comparison: 63.1% →76.6% (+13.5 pp) was Omega-S’s mean improvement over no regularisation across ten seeds, winning 8/10 with one-sided p = 0.055.This comparison is considered conservative because no regulariser has no hyperparameter to select; Omega-S also won 9/10 against weight decay and 8/10 against EWC.
  • Omega-S selection: 0.03 was Omega-S’s selected value at the edge of its swept range, with retention falling monotonically as the target rose.The first replication changes are disjoint-seed selection and extending the Omega-S grid below its current floor.

B.12. Testing the contrast-preserving reformulation: a negative result

The contrast-preserving row-wise cosine construction activates and steers the clustering channel, but it worsens retention relative to the original formulation on all ten seeds. The result supports interpreting the original method as effectively a degree-variance regulariser, with conclusions limited to LoRA catastrophic forgetting on this model and task pair.

  • Construction: The row-wise cosine affinity removes saturation, is invariant to row scaling, and spans the full [0, 1] range.This makes node degree mathematically independent of row magnitude, unlike the original construction.
  • Channel validation: 0.9999 to 0.802 in q_proj and 0.99999 to 0.512 in v_proj: the cosine construction makes the clustering channel live.The channel was also steerable: penalty sign separated movement direction in 16 of 16 measurements.
  • Retention outcome: Mean retention was 0.537 versus 0.766, with mean paired difference −0.229, as the cosine formulation lost on all ten seeds.The comparison had Wilcoxon signed-rank W = 0 and two-sided p = 0.002; every cosine-arm value fell below every original-arm value.
  • Limits: The conclusion is specific to catastrophic forgetting under LoRA on this model and task pair, and the uniform cosine construction differs from the original shape-conditional implementation.The clustering-only arm also had standard deviation 0.35 and range 0.09 to 1.33, indicating large but unreliable seed-specific gains.

B.13. First run: five-seed replication … D. Additional tables

Across replication, profiling, mechanism, sparsity, applications, and supplementary analyses, Omega-S shows stronger retention than weight decay, modest overhead, and a present effect best characterized as degree-variance regularization. Several broader applications remain hypotheses, while the paper documents methodological limits, negative findings, and language-model assistance.

  • B.13. First run: five-seed replication: Omega-S beat tuned weight decay on 4 of 5 seeds by +14.1 pp mean, while the five-seed comparison with EWC was not separable.Omega-S had higher mean retention than EWC, 71.7% vs. 60.1%, but EWC won three of five seeds; the ten-seed tuned run led EWC on 8/10.
  • B.13. First run: five-seed replication: 0.223 was Omega-S’s highest absolute post-prose HumanEval, versus 0.168 with no regulariser and 0.162 with weight decay, without architectural or previous-data access.The reported training overhead was +1.5%.
  • B.14. Infrastructure Overhead: Single-GPU and FSDP: +1.5% was Omega-S’s relative overhead at K=10 under two-GPU FSDP, with zero all-gather communication because LoRA adapter weights were not sharded.Single-GPU profiling measured +3.7% latency and +13 MB (+0.06%) VRAM overhead; the high absolute FSDP latency reflects PCIe limitations, while relative overhead is hardware-independent.
  • B.16. Structured Sparsity as an Emergent Property: Omega-S alone does not produce structured sparsity; group-lasso supplies the explicit pressure to collapse neurons to zero, while Omega-S shifts which neurons become redundant.The marginal contribution over group-lasso alone remains unisolated.
  • B.17. Biological Analogy and Its Limits: The microbial-network analogy provides structural intuition rather than mathematical equivalence because biological and neural networks arise under different timescales and pressures.The paper presents empirical validation of this intuition as its contribution.
  • B.18. Broader Applications: Potential applications include quantisation, federated learning, and MoE routing, but quantisation remains a hypothesis after small-scale probing found reduced degree variance without confirmed benefit.The paper also proposes uses in computational biology and quantitative finance.
  • C. Declaration of LLM Usage: Large language models assisted manuscript clarity, coding and debugging, and related-work formalization; the author developed, verified, and approved the technical content and interpretations.This declaration covers the preparation of the manuscript, not the paper’s experimental findings.

A. A contrast-preserving reformulation of the clustering term

The implemented clustering channel is inert because sigmoid affinities saturate near an all-ones matrix, leaving Omega-S to act through degree variance. Contrast-preserving reformulations were proposed and tested, but the row-wise cosine variant lost to the original formulation on all ten seeds.

  • Diagnosis: The clustering channel is inert: F≈0.9997 regardless of weight structure, so the penalty acts only through degree variance.The saturation arises from A = σ(|WW^T|), which compresses affinities into [0.5, 1).
  • Reformulations: Three contrast-preserving alternatives were outlined: temperature-scaled, thresholded, and rank-normalised affinities.These are proposed as open directions for testing genuinely clustering-driven variants, not as main-paper results.
  • Open design choice: The clustering term’s sign remains unresolved, so clustering-only experiments should test both minimising and increasing Tr(A^3)/∥A∥^3.The resilience index and implemented objective imply opposite directions, while the saturated channel provides no evidence to arbitrate.
  • Empirical outcome: p = 0.002: a row-wise cosine affinity that removed saturation and made clustering steerable lost to the original formulation on all ten seeds.A clustering-only arm performed worse still, overturning the earlier conjecture that restoring the topological channel would improve retention.
Loading 2608.03887v1…