PaperlayerView exact revision on arXiv

Source-linked AI summary

PRAXIS: Learning Dynamics of Self-Improving Models with Symbolic Archives

Venkat Margapuri, Mustafa Teber

arXiv:2610.11803v1cs.LG

TL;DR

Self-improving learning adapts data selection and symbolic signals, creating nonstationary objectives that fixed or independently updated components do not capture well. PRAXIS models generators, learners, and symbolic archives as a coupled dynamical system. Its theory analyzes drift, learner stationarity, and archive suppression, while experiments show stable optimization and archive concentration with domain-dependent predictive-performance effects.

  • Problem

    Self-improving systems adapt data, tasks, and inductive signals, but these components are often fixed or updated independently, limiting analysis of objective drift and adaptation.

    Source: §1 · p. 1

  • Method

    PRAXIS models adaptive universe selection, gradient-based learning, and symbolic archive regularization as coupled processes, with KL-controlled Generator updates.

    Source: §1 · p. 1, §4.4 · p. 4

  • Results

    Experiments across visual robustness, relational graph reasoning, and algorithmic graph reasoning show stable optimization and archive concentration, while predictive-performance effects vary by domain.

    Source: §8 · p. 9

  • Takeaways & Limitations

    The framework separates Generator- and archive-induced objective drift and provides analyses of learner stationarity and archive suppression under stated conditions.

    Source: §1 · p. 1, §8 · p. 9

    The formulation uses finite universes and simple deterministic archive programs; extensions to continuous or latent universes and richer archives remain future directions.

    Source: §8 · pp. 9–10

Abstract

from arXiv · show

Self-improving learning systems adapt data selection, optimization, and auxiliary symbolic components, inducing nonstationary objectives outside standard learning assumptions. We introduce \textsc{PRAXIS}, a co-evolutionary framework that models generators, learners, and symbolic archives as interacting dynamical processes. We prove that KL-constrained generator updates and controlled archive-weight movement bound one-step objective drift, that archive updates suppress a program relative to any fixed comparator with a persistent cumulative utility advantage under sub-Gaussian noise, and that stochastic gradient descent achieves an average-stationarity guarantee whose degradation is governed by cumulative objective drift. Experiments across visual robustness, relational graph reasoning, and algorithmic graph reasoning exhibit generator stabilization, decreasing learner loss, and archive concentration consistent with these theoretical mechanisms.

1 Introduction

PRAXIS frames self-improving learning as a coupled process in which universe selection, learner optimization, and symbolic archives evolve together. It formalizes how these components interact under nonstationarity.

1 Introduction

  • PRAXIS models the Generator, Learner, and symbolic Archive as interacting processes, enabling analysis of objective drift, learner tracking, and suppression of non-competitive programs.The framework addresses settings where data distributions and inductive signals are often fixed or updated independently of learner dynamics.Source: §1 · p. 1

2 Preliminaries

The preliminaries define a finite universe space and probability distributions over it, then specify the learner, generator, and archive components used by PRAXIS.

2 Preliminaries

  • A universe space U indexes supervised data-generating distributions, and Δ(U) denotes distributions over those universes.Each universe may differ in data volume, bias, augmentation strength, or noise; π0 is a fixed reference prior.Source: §2 · p. 2
  • The Learner maps inputs to predictive label distributions, while the Generator selects a distribution over universes for sampling training data.The predictive output is the interface exposed to symbolic components.Source: §2 · p. 2
  • The finite archive contains deterministic programs that map learner predictive distributions to hard labels, and its scores induce soft archive weights.Programs encode heuristic or structural biases and are not assumed to be semantically correct.Source: §2 · p. 2

3 Problem Formulation

PRAXIS formulates self-improving learning as adaptive universe selection combined with archive-regularized learner training. The components evolve jointly rather than solving the objective to equilibrium.

3 Problem Formulation

  • The learner’s mixture risk averages population loss across universes, while the Generator adapts the universe distribution to emphasize learning-relevant regions.Deviations from a fixed reference prior are regularized through a KL penalty.Source: §3 · p. 2
  • Archive programs add a nonnegative consistency penalty using hard predictions from the learner’s predictive distribution.The hard program output is a stop-gradient pseudo-label; gradients flow through the learner distribution, not the discrete program output.Source: §3 · p. 2
  • The archive penalty is weighted by current program weights and may be modulated by an intervention gate that is not differentiated.The resulting augmented loss combines supervised loss with symbolic regularization.Source: §3 · pp. 2–3
  • The regularized min–max objective induces coupled closed-loop dynamics; the archive influences learning through time-varying weights rather than direct optimization.The objective is not solved to equilibrium.Source: §3 · p. 3

4 Solution Approach

PRAXIS updates its Learner, Archive, and Generator online using signals from the current coupled state. Sampling, learner regularization, archive scoring, and KL-controlled generator adaptation form the closed-loop process.

4.1 Sampling and Estimation

  • The Generator samples universes and aggregates their empirical datasets into a shared signal for Learner, Archive, and Generator updates.The components update online from the current state rather than solving to equilibrium.Source: §4.1 · p. 3, §4 · p. 3

4.2 Learner Update

  • The Learner descends the current mixture objective using minibatch gradients, with symbolic programs contributing through a weighted consistency regularizer.Program outputs become stop-gradient pseudo-labels, so gradients propagate only through the learner’s predictive distribution.Source: §4.2 · p. 3

4.3 Archive Update

  • Archive utilities combine fitness with a penalty for program disagreement on examples the Learner classifies correctly, and accumulated noisy utility updates scores and weights.Persistent utility accumulation increases influence for sustained performers and downweights persistently uninformative or contradictory programs.Source: §4.3 · p. 4

4.4 Generator Update

  • The Generator evaluates sampled universes after the learner update and uses a KL-regularized objective to favor universes that remain informative under adaptation.One KL term limits step-to-step movement, while another discourages collapse onto degenerate subsets; accepted updates satisfy KL(qt+1∥qt) ≤ κt.Source: §4.4 · p. 4

5 Theoretical Analysis of PRAXIS Dynamics

PRAXIS analyzes coupled Generator, Learner, and Archive dynamics through bounds on objective drift, archive suppression, and learner stationarity. These guarantees depend on controlled Generator and archive movement, persistent utility advantages, and standard stochastic-gradient assumptions.

5.1 Generator-induced objective drift

  • KL-constrained Generator movement and controlled archive-loss changes jointly bound one-step objective drift; Generator stabilization alone suffices only when archive-induced drift is also controlled.When programs and gates are fixed, archive-loss drift is bounded by archive-weight movement.Source: §5.1 · p. 5
  • Exponential Generator stabilization and exponentially decreasing archive-loss drift yield exponentially decreasing uniform objective drift and bound the along-iterate drift.The bound combines the Generator and archive contributions, so stabilization of only one component does not establish this joint rate.Source: §5.1 · p. 5

5.2 Archive suppression under persistent utility advantage

  • Learner-gated blame gives a learner-dependent lower bound on pairwise utility gaps, but does not alone show that learner improvement enlarges those gaps.That stronger conclusion requires directional control of the difference in blame between programs.Source: §5.2 · p. 6
  • With high probability, a program persistently less useful than a fixed comparator loses relative archive weight according to its cumulative utility advantage, despite stochastic estimation noise.The suppression guarantee holds under the stated sub-Gaussian utility-noise assumptions and does not require an optimal or converged Learner.Source: §5.2 · p. 6
  • A persistent signed utility margin makes non-elite program weights decay exponentially, up to sublinear stochastic fluctuations.The decay rate depends on the learning rate, inverse temperature, and persistent utility gap.Source: §5.2 · p. 6

5.3 Average learner stationarity under objective drift

  • Under the stated stochastic-gradient and smoothness assumptions, average SGD stationarity is bounded by the usual optimization and noise terms plus a cumulative objective-drift penalty.The drift contribution vanishes when cumulative uniform drift is sublinear in the horizon.Source: §5.3 · pp. 6–7
  • Vanishing drift contribution to average stationarity does not establish parameter-space tracking of a moving minimizer.This is a boundary of the guarantee, not a claim about whether such tracking occurs.Source: §5.3 · p. 7

6 PRAXIS Experimental Setup

PRAXIS is evaluated across visual robustness, relational reasoning, and algorithmic graph reasoning using the same Generator–Learner–Archive process. The domains vary in their universes, tasks, and held-out transfer settings.

6 PRAXIS Experimental Setup

  • The evaluation covers MIT Indoor visual robustness, GraphLog relational reasoning, and CLRS-style algorithmic graph reasoning.The CLRS task tests shortest-path threshold prediction, not the full CLRS benchmark suite.Source: §6 · pp. 7–8
  • MIT Indoor universes vary training draw count, augmentation, corruption type, and severity; held-out universes use disjoint settings.The setup defines 300 training and 27 evaluation universes.Source: §6 · p. 7
  • GraphLog uses rule-world and target-relation universes, with 43 training and 24 held-out universes.The task is binary relational query answering over labeled graph structures.Source: §6 · pp. 7–8
  • CLRS universes vary graph size, family, density, and edge-weight regime, with a hard split transferring to larger weighted graphs.The split has 16 training and 18 held-out universes.Source: §6 · p. 8
  • Baselines include fixed-mixture ERM, adaptive selection without archive regularization, DRO-style sampling, and bandit-style samplers.The comparisons contextualize predictive performance; the main empirical question is whether the predicted dynamics recur across domains.Source: §6 · p. 8

7 Results

Across the three domains, the archive concentrates, the Generator adapts with bounded drift, and learner training loss declines. The strength of concentration and the loss trajectory vary by domain.

RQ1: Does the Generator adapt while remaining stable?

  • Maximum Generator drift proxies remained at or below 0.0173 across domains as entropy fell and maximum universe probability rose.Entropy changed from 4.68 to 3.91 on MIT Indoor, 3.14 to 0.38 on GraphLog, and 2.77 to 2.31 on CLRS; maximum drift proxies were 0.0128, 0.0173, and 0.0016, respectively.Source: §7 · p. 8, Figure 1 · p. 8

RQ2: Does learner optimization remain stable under the induced nonstationarity?

  • Training cross-entropy decreased from 1.007 to 0.080 on MIT Indoor, 0.702 to 0.109 on GraphLog, and 0.617 to 0.421 on CLRS, with a CLRS minimum of 0.361.Training loss is not a direct measurement of the gradient-norm quantity bounded in Theorem 5.14.Source: §7 · p. 8, Figure 2 · p. 9

RQ3: Does the symbolic archive concentrate and suppress non-elite programs?

  • Archive maximum weights rose from 0.120 to 0.181 on MIT Indoor, 0.141 to 0.987 on GraphLog, and 0.164 to 0.964 on CLRS, while entropy decreased across domains.With elite size k = 2, median non-elite ratios fell from −0.95 to −26.19 on MIT Indoor, −0.49 to −8.03 on GraphLog, and −1.04 to −18.29 on CLRS; the suppression theorem requires persistent signed utility advantage over a named comparator.Source: §7 · pp. 8–9, Figure 3 · p. 9, Figure 4 · p. 9

8 Conclusion and Future Work

PRAXIS models self-improving learning as coupled evolution of data selection, gradient-based learning, and symbolic archive regularization. Its controlled formulation motivates future extensions, while its related-work positioning and archive composition clarify the framework’s scope.

8 Conclusion and Future Work

  • PRAXIS couples adaptive universe selection, gradient-based learning, and symbolic archive regularization, analyzing drift, average SGD stationarity, and fixed-comparator suppression.The formulation uses finite universes and simple deterministic archive rules; continuous or latent universes and richer archives are future directions.Source: §8 · pp. 9–10

A Related Work

  • PRAXIS complements self-improving agent work by providing a learning-theoretic model of coupled data selection, optimization, and symbolic structure rather than a new agent architecture.Its guarantees address objective drift, stability, and long-horizon tracking.Source: A Related Work · p. 12
  • Prior adaptive-selection methods include curricula, DRO, and bandit approaches, while PRAXIS treats resulting nonstationarity as a coupled dynamical quantity.The related-work discussion contrasts this with objectives typically treated as exogenous, adversarial, or heuristic.Source: A Related Work · p. 12
  • Many symbolic-learning approaches assume fixed or trusted programs or focus on task accuracy; PRAXIS instead dynamically modulates fallible symbolic programs.The comparison spans differentiable logic, symbolic program induction, and programmable neurosymbolic systems.Source: A Related Work · p. 12

B Symbolic Archive Composition

  • Archive programs provide auxiliary predictions or structural preferences and are weighted through utility-based updates, not manually selected at evaluation.MIT Indoor uses five families of seeded rules; GraphLog and CLRS each use eight programs with domain-specific heuristics.Source: B Symbolic Archive Composition · p. 13

C Implementation and Reproducibility Details

The multidomain implementation compares adaptive methods on shared train and held-out universe splits, with 50 PRAXIS iterations for the main diagnostics.

C Implementation and Reproducibility Details

  • All adaptive methods use the same domain-specific train and held-out universe splits; main multidomain diagnostics run for 50 PRAXIS iterations.Methods include Uniform/ERM, GroupDRO-style sampling, bandit-style sampling, Generator-only, and Full PRAXIS.Source: C Implementation and Reproducibility… · p. 13

C.1 Component Ablation Analysis

Component ablations show that PRAXIS’s strongest configuration depends on the domain: it leads on MIT Indoor and CLRS, while adaptive universe selection without archive regularization is strongest on GraphLog.

C.1 Component Ablation Analysis

  • The ablation compares five configurations, from fixed-mixture ERM without an archive to full PRAXIS, to identify component roles across domains.The compared roles include adaptive generator training without an archive, frozen archive or generator variants, and full PRAXIS.Source: §C.1 · p. 13
  • Full PRAXIS leads MIT Indoor and CLRS on mean and CVaR10, but adaptive universe selection without archive regularization is strongest on GraphLog.Table 3 reports Mean, Min, CVaR10, and Std across held-out universes.Source: §C.1 · pp. 13–14, Table 3 · p. 14
  • Adaptive universe selection carries the clearest benefit on GraphLog, while archive adaptation contributes more clearly to MIT Indoor robustness and more weakly to CLRS.On CLRS, PRAXIS's margin over the no-archive and frozen-archive variants is modest.Source: §C.1 · p. 14

D.1 Proofs for Section 5.1

These proof fragments use divergence control and expectation-based arguments to establish objective-drift and along-iterate claims.

D.1 Proofs for Section 5.1

  • The proof takes a supremum over θ and applies Pinsker’s inequality; a later step takes expectations and applies Lemma 5.6 for the along-iterate claim.Source: §D.1 · p. 14
  • When only archive weights change, the archive contribution is bounded by λ∥w_t+1 − w_t∥_1, and convergence follows if both terms on the right tend to zero.The preceding objective-drift term is not included in the supplied passage.Source: §D.1 · p. 14
  • Corollary 5.8 is obtained by applying Theorem 5.4 and the two exponential component bounds.Source: §D.1 · p. 14

D.2 Proofs for Section 5.2

The appendix proofs bound archive utility gaps using learner-gated blame and sub-Gaussian noise, then use smoothness and objective drift in the learner’s stationarity argument.

D.2 Proofs for Section 5.2

  • Blame is learner-gated and bounded by the learner’s correctness probability, yielding 0 ≤ B_k,t ≤ 1 − e_t.This bound is used to lower-bound pairwise utility gaps through fitness differences and the blame penalty.Source: §D.2 · p. 15
  • A conditional-mean and martingale-noise decomposition, followed by a union bound over program pairs and horizons, yields the simultaneous sub-Gaussian bound Γ_subG.Source: §D.2 · p. 15
  • When μ = 0, Γ_subG(t, δ) grows as O(√t log T) for fixed horizon T, while the signed signal grows linearly in t.Source: §D.2 · p. 15

D.3 Proof for Section 5.3

  • For the learner proof, the gradient coefficient is at least η/2 when η ≤ 1/L, and the next objective is bounded using the drift δ_t.Source: §D.3 · p. 16
  • Summing the learner inequalities, telescoping, and using F_T(θ_T) ≥ F_min completes the stationarity-claim argument.Source: §D.3 · p. 16
Loading 2610.11803v1…