PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Evaluation for Principled Generalization Using a Kripkean Challenge to Meaning as Use

Niyar R Barman, Omar Naim, Nicholas Asher

arXiv:2610.35111v1cs.CL

TL;DR

Finite use and success on fixed OOD cases do not by themselves establish that a model has acquired a rule for future extension. The paper proposes evaluating abstraction and recursion through tasks that separate direct performance from forming, applying, and iterating rules; its diagnostics indicate that models have not fully learned these principles.

  • Problem

    Finite observed usage and success on finite OOD extensions do not establish whether a model has acquired the rule needed for further cases, limiting what fixed-case benchmarks show about principled generalization.

    Source: §1 · p. 1

  • Method

    The paper applies Kripke’s rule-following argument and evaluates abstraction and recursion by separating direct performance, abstract-function formation and application, composition, and iterative execution.

    Source: §1 · p. 2

  • Results

    The diagnostics indicate that models have not fully learned abstraction or recursion, and their abstraction and application abilities fall below their ability to answer even simple problems correctly.

    Source: §7 · p. 8, §8 · p. 8

  • Takeaways & Limitations

    Evaluations of principled generalization should test whether models preserve rules across new instances rather than treating correct expressions or benchmark answers as sufficient evidence.

    Source: §1 · p. 1, §7 · p. 8

    The diagnostics are intentionally narrow, and the supplied abstraction paradigm gives models variables and bindings rather than asking them to choose what to abstract over.

    Source: §9 · p. 9

Abstract

from arXiv · show

Large language models operationalize an intuitive, view of meaning as use. We argue that it faces a rule-following challenge. Using Kripke's interpretation of Wittgenstein, we argue that finite histories of use do not determine a unique rule for future extension. This reframes the debate about OOD generalization but also about NLP evaluation; benchmarks that measure performance on fixed use cases or local robustness, we argue, do not by themselves establish semantic competence. To evaluate whether models preserve rules and concepts under principled domain extension, we propose to evaluate two capacities, central to this ability: abstraction and recursion; and we offer an experimental paradigm for testing both.

1 Introduction: LLMs and skepticism

The paper asks whether mastery of observed usage justifies extending patterns to novel cases, arguing that finite performance alone does not establish rule acquisition. It proposes evaluating abstraction and recursion as capacities for principled extension.

1 Introduction: LLMs and skepticism

  • Meaning-as-use treats patterns of expression use as fixing meaning, with words, sentences, and texts gaining content through their contexts.The paper frames LLMs as operationalizing this view.Source: §1 · p. 1
  • Finite training examples or finite OOD success do not by themselves establish that a model has acquired the rule governing further cases.The paper argues that evaluation should test whether models identify and apply relevant abstractions as the domain is systematically extended.Source: §1 · p. 1
  • Abstraction forms general functions from concrete instances, while recursion extends finite procedures systematically beyond encountered examples.Such functions can apply to novel inputs and compose into more complex structures.Source: §1 · pp. 1–2
  • The proposed evaluation separates direct task performance, abstract-function formation and application, function composition, and iterative addition.For abstraction, ASDiv and GSM8K distinguish direct problem solving from forming and applying a function to new instances; recursion tests reliability as application depth grows.Source: §1 · p. 2
  • The paper presents a Kripkean reframing of OOD evaluation, argues for abstraction and recursion as principles of generalization, and tests 10 models across arithmetic, geometry, and NLP reasoning.Its empirical paradigm distinguishes direct performance, abstraction, application, composition, and iteration.Source: §1 · p. 2

2 A Kripkean Challenge

The section applies Kripke’s rule-following challenge to meaning-as-use: finite histories of use do not uniquely determine how a rule should extend. It then considers objections and explains why controlled rule-governed tasks are useful for evaluation.

2.1 The rule-following argument

  • The rule-following challenge asks how a finite history of use could determine one rule rather than another for future application.Kripke’s skeptical point is not that addition is mathematically invalid, but that finite use alone does not justify one extension over another.Source: §2.1 · p. 2
  • Any semantic concept tied to linguistic use invokes future uses, so a model based only on finite past uses cannot capture it fully.The paper says finite observed patterns can capture task-essential information but do not justify preserving meaning under unbounded OOD extension.Source: §2.1 · p. 2

2.2 Objections and replies

  • Addition and universal quantification serve as controlled testbeds because their reference procedures permit evaluation of abstraction, application, and iterative execution without task ambiguity.The same condition can be tested over increasingly long sequences for universal quantification.Source: §2.2 · p. 3
  • Dispositions or probability distributions do not justify assuming that an unbounded test domain follows the training distribution’s patterns.The argument asks what distinguishes a plus disposition from a quus disposition beyond the domain on which it was formed.Source: §2.2 · p. 3
  • Computational capacity and learning in the limit do not, by themselves, establish general rule-following ability or answer Kripke’s critique.The text notes that computable procedures can be programmed, while distinguishing this from learning new rule-following behavior.Source: §2.2 · p. 3

3 Related work

The related work situates the paper among research on meaning, distribution shift, and rule generalization. It distinguishes the paper’s concern—what determines a target rule and its extension—from tests of particular fixed rules or other accounts of semantic grounding.

Related work

4 Lessons from Kripkenstein: OOD Generalization as Rule-Following Failure

The section uses arithmetic and OOD evaluations to illustrate how performance on finite extensions can differ from rule-following ability. It reports persistent collapse beyond the training range despite interventions, and motivates tests that distinguish far-OOD extrapolation from near-ID robustness.

4. Lessons from Kripkenstein: OOD Generalization as Rule-Following Failure

  • Finite OOD tests and augmented synthetic data may fail to isolate rule-following, and benchmarks near the training distribution may measure robustness to small shifts instead.The section raises the question of how to design benchmarks that test extrapolation beyond the training distribution.Source: §4 · p. 4
  • Prior reports describe cliff-like OOD degradation in regression and quantification, alongside expectations of graceful degradation from representation, memory, or integer-precision limits.The passage cites small GPT-style models and pretrained or fine-tuned LLMs as examples of reported failures.Source: §4 · p. 4
  • Addition provides a controlled illustration because increasing digit length changes input size without changing the rule, separating rule extension from distribution-bound regularities.The section predicts that distribution-bound performance can remain high in-distribution yet collapse on longer inputs.Source: §4 · pp. 4–5
  • Six literature-based interventions extended OOD performance, some significantly, but did not eliminate the cliff-like collapse in accuracy.Figure 1 reports near-ceiling in-distribution exact match on addition and collapse beyond each model’s maximum training operand length n.Source: §4 · p. 5, Figure 1 · p. 5

5 An empiricist response: two principles

The paper argues that abstraction, composition, and recursion can support generalization beyond observed examples. It describes how these principles turn concrete instances into rules that can be systematically applied and extended.

5 An empiricist response: two principles

  • The authors propose abstraction, composition, and recursion as principles for forming general rules, rather than relying on targeted data, favorable representations, explicit rule descriptions, or benchmark success alone.They argue that those approaches do not suffice to establish rule-like semantic generalization.Source: §5 · p. 5
  • Abstraction replaces details in concrete structures with variables to form functions, while composition combines functions.In logic, abstraction on a formula ϕ(x) yields λxϕ; if rules are functions, combining concepts can be treated as function composition.Source: §5 · p. 5
  • A general schema can extend beyond observed instances: abstraction represents every(P, Q) as equivalent to P ⊆ Q, while recursion defines the natural numbers from 0 and successor.The authors describe using these definitions to provide content for universal claims and to define addition algorithms over the natural numbers.Source: §5 · p. 5
  • The proposed progression runs from observing concrete regularities to selecting properties, abstracting them into a function, and examining its behavior as inputs vary, including limiting cases.The authors argue that abstraction and recursion can yield concepts whose application exceeds the finite data that motivated them.Source: §5 · p. 6

6 How to evaluate abstraction capability

The authors evaluate abstraction by separating direct problem solving, function formation, and transfer to altered instances, and test application and recursion in additional controlled settings. Their designs examine whether models can preserve a rule when inputs or the depth of its application change.

6 How to evaluate abstraction capability

  • On 214 GSM8K problems paired with variants that preserve functional structure but change scenario and quantities, models are asked to infer a solution function and apply it to new values.For the abstraction step, the model receives quantity-to-variable mappings and is checked for recovering the function; in the example, f(a, b, c, d) = (a − b − c)d.Source: §6 · p. 6
  • Table 1 distinguishes direct target solving (Eval 1), abstraction (Eval 2), and transfer through the induced abstraction (Eval 3).The transfer step gives the model its generated expression and asks it to solve a paired problem with changed setting and quantities but the same relationships.Source: Table 1 · p. 6, §6 · p. 6
  • The diagnostic differs from prior compositional-generalization tests by asking whether models can learn to use abstraction and application to find and use generalizations.The cited prior work uses abstraction to generate test data for the model.Source: §6 · p. 6
  • The authors also assess abstraction on 190 ASDiv addition and subtraction problems, comparing numerical-answer accuracy with symbolic-expression accuracy.Source: §6 · p. 6
  • A zero-shot geometry test covers 500 problems across ten families, supplies each rule explicitly, and varies ambient dimension from 4 to 4096 in two families.The prompts use exact rational inputs and ask models to show substitutions and calculations before answering.Source: §6 · p. 6
  • For recursion, models apply a fixed tail-recursive addition procedure to 100 pairs of n-digit integers at each tested length from 5 to 35, with both sums and call-state sequences scored.Only the number of calls varies; the procedure continues until both input registers are zero.Source: §6 · p. 6

7 Results

Across the reported tests, models often performed better on direct answers than on abstraction or reliable rule application, and performance weakened as inputs or procedures extended. These results support evaluating whether models preserve rules across new instances, not treating correct answers alone as evidence of that capacity.

ASDiv abstraction

  • In ASDiv addition and subtraction, zero-shot original-to-abstracted gaps were 25.3 points for Qwen3-8B and 31.1 for Llama-3.1-8B-Instruct.Five-shot reduced the gaps to 15.3 and 25.3 points, respectively; Figure 2 contrasts numerical-answer accuracy with formula exact match.Source: §7 · pp. 6–7, Figure 2 · p. 7

GSM8K abstraction and transfer

  • On 214 GSM8K-derived pairs, Qwen3-32B and Qwen3-8B scored 91.6% and 90.2% on direct solving, but 86.9% and 72.0% on transfer.The direct-solving gap was 1.4 points and the transfer gap was 15.0 points; Figure 3 compares direct solving, abstraction, and transfer.Source: Figure 3 · p. 7, §7 · p. 7

Numerical scaling

  • From baseline to extended scale, transfer fell from 86.9% to 43.5% for Qwen3-32B and from 72.0% to 30.4% for Qwen3-8B.At extended scale, Llama-3.1-70B-Instruct had 2.3% transfer and OLMo-2-0325-32B-Instruct had 0.0%; source-abstraction accuracy remained comparatively stable as source problems were unchanged.Source: Figure 4 · p. 7, §7 · p. 7
  • At baseline, Qwen3-32B correctly applied its abstraction to 181 of 191 correctly abstracted problems, versus 90 of 191 at extended scale.The authors treat application to new values as a more direct test of rule following than producing the symbolic form alone.Source: §7 · p. 7

Geometrical concepts

  • With explicit geometric rules, verified accuracy was 69.0% for Llama-3.1-70B-Instruct, 17.6% for OLMo-2-0325-32B-Instruct, and 8.6% for Qwen3-0.6B.Only Qwen3-8B and Qwen3-32B produced sound derivations for nearly all their correct answers, so correct answers alone did not establish rule application.Source: §7 · p. 8, Figure 5 · p. 8
  • From d = 4 to d = 4096, EM fell from 95% to 55% for Llama-3.1-70B-Instruct and from 55% to 5% for OLMo-2-0325-32B-Instruct, despite a fixed criterion.For OLMo, EM stayed at 50% while rule application collapsed; Figure 5 reports EM and EM versus dimension d.Source: Figure 5 · p. 8, §7 · p. 8

Recursion

  • With a fixed supplied recursive rule, accuracy fell as recursive calls increased: Qwen3-32B declined from 90% at 5 digits to 39% at 35 digits (100 problems per point).Source: §7 · p. 8, Figure 6 · p. 8

Takeaways

  • Models had difficulty fully learning abstraction and recursion; abstract structure and its application to new data lagged behind correct answers, even on simple problems.The authors argue that abstraction should be evaluated by whether a rule is preserved across new instances.Source: §7 · pp. 7–8

8 Conclusion

The authors argue that abstraction and recursion complement experience in supporting stable rule and concept extension. Their evaluations of ten models across math problems and geometrical concepts, and of recursion in arithmetic and universal quantification, indicate that models have not fully mastered these principles.

Conclusion

  • The paper argues that abstraction and recursion complement experience and are needed for stable rule and concept extension.The authors present them as essential for out-of-distribution generalization.Source: §8 · p. 8
  • Across ten models, evaluations of abstraction on grade-school math and geometry, and recursion on arithmetic and universal quantification, indicate that models have not fully mastered these principles.The authors state that the evaluations provide a way for models to acquire such mastery.Source: §8 · p. 8

9 Limitations

The paper’s empirical diagnostics are intentionally narrow, and its theoretical discussion connects finite observations to limits on justified rule extension. Across the supplied sections, experiments show that added structure can extend performance beyond training conditions without establishing reliable extension indefinitely.

9 Limitations

  • The experiments are narrow diagnostics using arithmetic, geometric criteria, and grade-school problems; they do not establish the full scope of natural-language OOD failure.The abstraction paradigm supplies variables and bindings, and the 214 GSM8K-derived pairs may have appeared in pretraining data.Source: §9 · p. 9

A Learning in the limit

  • Limit computability is equivalent to being both computably enumerable and co-computably enumerable; under Kripkean justification patterns, Π1 and Σ1 sets are not learnable with one hypothesis change.The learnability claim depends on objective functions with Kripkean justification patterns.Source: A Learning in the limit · p. 12

B.1 Experimental Setup

  • Addition diagnostics use digit-length shifts across several pretrained models and structural interventions, scoring exact matches on full generated answers.The setup includes numerical representations, positional scaffolding, rule prompts, and procedural supervision.Source: §B.1 · p. 12

B.2 Structured numerical interfaces

  • The column-state decoder replaces numbers in the text stream and uses a dedicated number decoder, while the auxiliary method preserves the text pathway and adds numerical structure.The interfaces are intended for different model settings: replacement for smaller models and a lighter auxiliary interface for stronger pretrained models.Source: Table 2 · p. 13, §B.2 · p. 13

B.3 Addition under digit-length shift

  • Structured numerical interfaces improve in-distribution and near-OOD accuracy, but far-OOD addition remains largely unsolved; positional scaffolding extends the range yet leaves a substantial gap.The Abacus-style scaffold supplies digit-column alignment rather than a carry algorithm or column-update procedure.Source: §B.3 · p. 13, Table 3 · p. 13, Table 4 · p. 13

B.4 Good representation̸ = rule following

  • Numerical-auxiliary training makes the sum residue more decodable, yet exact-match accuracy still collapses at greater digit lengths; the probes do not isolate representation from procedure.The authors therefore do not claim that numeral representation itself has been solved.Source: §B.4 · p. 14

B.5 Giving the rule doesn’t yield rule following

  • A rule-guided prompt improves short and moderate sequence performance but does not prevent exact-match accuracy from collapsing as sequence length increases.The prompt experiment distinguishes giving a rule description from applying it reliably under domain extension.Source: §B.5 · p. 14

B.6 Providing structure in training

Procedural trace supervision extends the range over which a model applies a rule compared with direct input-output supervision, but it does not eliminate the OOD cliff. The supplied appendices describe the interfaces, diagnostics, and implementation details used to examine that pattern.

B.6 Providing structure in training

  • Trace supervision moves the OOD cliff farther beyond the training boundary than direct final-answer supervision, but does not eliminate it.The gain is attributed to additional procedural structure rather than ordinary input-output use alone.Source: §B.6 · p. 15, Figure 7 · p. 15

C.1 Data and evaluation protocol; C.5 Training and evaluation

  • Arithmetic experiments use synthetic addition examples with operands extending beyond the training digit range, and score exact matches on full generated answers.Headline arithmetic runs use deterministic decoding; the setup reports 50,000 training examples and 2,000 examples per evaluation split.Source: §C.1 · p. 15, §C.5 · p. 16

C.2 Numerical encoder

  • The numerical encoder combines periodic, log-magnitude, and sign features into a whole-number embedding projected to the language model’s hidden dimension.Its periodic features make decimal-scale information explicitly available.Source: §C.2 · p. 15

C.3 Column-state decoder

  • The column-state decoder substitutes a whole-number embedding for each special number token and decodes answers separately, using aligned operand-column memories.Its dedicated decoder emits the answer least-significant digit first.Source: §C.3 · pp. 15–16

C.4 Auxiliary supervised text method

  • The auxiliary text method preserves ordinary text tokens while adding gated numerical embeddings; its training makes sum residues more decodable even though far-OOD exact-match generalization remains limited.Auxiliary heads supervise numerical structure, while answers are still generated through the ordinary language-model vocabulary.Source: §C.4 · p. 16

D Residue Probes for Arithmetic Geometry

  • Residue probes use ridge readouts of modulo 2, 5, 10, and 100 information from operand and answer-context states to test what arithmetic information is linearly decodable.Held-out accuracy is computed by mapping predicted Fourier vectors back to residue classes.Source: D Residue Probes for Arithmetic Geometry · p. 17, Table 6 · p. 18

E Algorithms

  • The appendix specifies addition as right-to-left register updates, carrying values across columns and concatenating the final digit registers.Source: E Algorithms · p. 18

F Quantification Trace Diagnostic: Implementation Details

  • The quantification diagnostic trains on prefix truth values or intermediate state traces, then evaluates longer signed-integer sequences against the running claim that all numbers are positive.Training sequence lengths range from 1 to 40, while evaluation includes lengths up to 100.Source: F Quantification Trace Diagnostic:… · p. 18

G GSM8K transfer: dataset variants and complete results

The transfer evaluation pairs source and target problems that preserve the reference expression while changing scenarios and quantities, and separates direct solving, source abstraction, and transfer. Results distinguish generating an expression from applying it under the evaluated conditions.

G GSM8K transfer: dataset variants and complete results

  • The 214 paired problems change target scenarios and bound quantities while preserving the reference expression and mathematical relationships across four target conditions.Variables denote quantity roles rather than their order of occurrence.Source: G GSM8K transfer: dataset variants and… · p. 19
  • The fractional intervention changes only 66 pairs, so its aggregate score averages changed and unchanged problems; abstraction scores across conditions come from separate executions on unchanged source prompts.Scaled and perturbed conditions reuse generated abstractions, whereas baseline and extended-scale evaluations generate them separately.Source: G GSM8K transfer: dataset variants and… · p. 19
  • Llama-3.1-70B-Instruct transfer falls from 59.3% at baseline to 2.3% at extended scale, while OLMo-2-0325-32B-Instruct has 78% abstraction accuracy but no correct extended-scale transfers.These results distinguish generating an appropriate expression from reliably executing it under the evaluated conditions.Source: G GSM8K transfer: dataset variants and… · p. 19
  • At extended scale, Qwen3-32B scores 45.3% with a direct control using the same bindings, 43.5% on transfer, and 36.4% on unassisted direct solving.The apparent advantage over unassisted solving does not isolate a benefit from generating the expression.Source: G GSM8K transfer: dataset variants and… · p. 19
  • The sources are curated and may have appeared in pretraining, target scenarios lack independent human review, and extended-scale quantities need not be realistic.The results characterize exact-answer performance under the stated construction and generation budgets, not unrestricted generalization or failure under every computational regime.Source: G GSM8K transfer: dataset variants and… · p. 19

H Composition of symbolic functions

These experiments examine composition of explicitly supplied symbolic rules, ASDiv addition and subtraction scores, and rule application in geometry. They distinguish task performance from evidence about intermediate reasoning or execution.

H Composition of symbolic functions

  • The 50 algebraically distinct rules cover one to five operations, with 1,000 problems per condition and explicitly supplied procedures rather than hidden-operation inference.Each operation count contains different rules, primitive choices, branching structures, and output magnitudes, so the operation-count curves are descriptive rather than an isolated effect of adding an operation.Source: H Composition of symbolic functions · pp. 19–20
  • Qwen3-8B reaches 99.8% symbolic accuracy with provided decomposition and 99.3% with model-generated decomposition, while Llama-3.1-8B-Instruct scores 83.8% symbolically but 27.5% numerically under direct prompting.Llama achieves 99.6% accuracy on 1,029 isolated primitive calculations, so composition difficulty is not always reducible to constituent calculations.Source: H Composition of symbolic functions · p. 20, Figure 10 · p. 21
  • At five operations, Qwen3-8B scores 99.0% and 100.0% symbolic accuracy with provided and model-generated decomposition, versus 72.5% and 50.0% for Llama-3.1-8B-Instruct.Performance is not uniformly monotonic in operation count.Source: H Composition of symbolic functions · p. 20, Figure 11 · p. 21

I Complete ASDiv addition and subtraction results

  • Across the same 190 ASDiv addition and subtraction problems, some smaller models match or exceed their numerical scores on abstracted problems under five-shot prompting.The comparison uses numeric equality for numerical answers but formula exact match, rather than algebraic equivalence, for abstracted answers.Source: I Complete ASDiv addition and… · p. 21, Table 8 · p. 21

J Geometry rule-application experiment

  • Geometry examples cover ten families of matched yes/no pairs, and labels use exact rational arithmetic; every correct final answer is audited for whether its derivation applies the rule adequately.Verified EM counts only correct derivations, while unverified responses include correct answers with an error or no adequate derivation.Source: J Geometry rule-application experiment · p. 22, Table 9 · p. 22

K Recursive addition: setup and complete results

The recursive-addition evaluation supplies a fixed procedure and increases operand length to test its repeated application. Results show that the prompt's effects vary across models and that correct final answers can diverge from correct execution traces.

K Recursive addition: setup and complete results

  • The evaluation samples 700 addition problems across operand lengths of 5 to 35 digits and compares direct answers, chain-of-thought, and a supplied recursive rule with recorded states.Each n-digit problem requires n recursive calls and n + 1 recorded states.Source: K Recursive addition: setup and… · p. 22
  • Exact match scores the final answer, while verified EM also requires the recorded states to match the reference sequence; truncated or unparseable responses count as incorrect.The evaluation additionally checks each recursive call's state transitions locally.Source: K Recursive addition: setup and… · pp. 22–23
  • For Qwen3-32B, recursive prompting gives 39% EM at 35 digits versus 26% direct and 14% CoT; at 50 digits it gives 28% versus 13% direct.For other models it remains below the better baseline at every length except Qwen3-8B at 10 digits.Source: Figure 12 · p. 23, K Recursive addition: setup and… · p. 23
  • Qwen3-32B removes digits correctly in more than 99.9% of calls, but its carry is correct in 49.4% and output digit in 14.2%; it continues past the base case in 941 of 1,000 responses.Only 28 of its 594 correct sums equal the value returned by its final recorded state.Source: K Recursive addition: setup and… · p. 23

Related papers

Loading 2610.35111v1…