PaperlayerView exact revision on arXiv

Source-linked AI summary

Language Is an Insufficient Substrate for Quantitative Reasoning, and Consequential Domains Need Large Quantitative Models

Reuben Vandeventer, David Imrem, David J. Wild

arXiv:2609.12105v1cs.AIcs.LG

TL;DR

Language-based models learn from human descriptions that can irreversibly omit quantitative information needed for consequential decisions. The paper formalizes requirements for a model class trained on native quantitative records; in a network-defense deployment, its structured policy achieved reported precision of 0.91±0.02 and recall of 0.87±0.03.

  • Problem

    Human descriptions are lossy representations of quantitative records, raising a gap for consequential decisions whose outputs must be grounded in those records.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    The paper formalizes representation and map requirements and proposes a quantitative model class that reasons over native records using explicit structure and computable lineage.

    Source: §1 · p. 1, §6.1 · p. 7, §7.2 · p. 9

  • Results

    0.91±0.02 precision and 0.87±0.03 recall: the structured policy had the highest precision among compared graph-based lateral-movement detectors, at recall within the strongest detectors’ range.

    Source: §8 · p. 10

  • Takeaways & Limitations

    The paper concludes that the quantitative core of consequential decisions calls for models capable of refining, structuring, and learning from the underlying record.

    Source: §11 · p. 13

    Building the proposed class is impossible without first solving autonomous refinement, privacy enforcement, and lineage preservation for quantitative data.

    Source: §9 · p. 12

Abstract

from arXiv · show

The prevailing assumption in applied machine learning is that progress on consequential quantitative decisions such as pricing risk, allocating capital, triaging patients, or containing a network intrusion will follow from progress in large language models (LLMs). A language model is trained on a representation of the world that was produced by human description; description is a lossy encoding of the quantitative record, and the loss is irreversible: no downstream model, at any scale, can recover from a description what the description did not encode. We formalize this as a property of the representation on which a model is trained rather than of the model capacity, and we identify three further properties that consequential settings demand of a model and that a language substrate cannot supply by construction: reproducibility, lineage from every output back to the source records that produced. it, and calibrated uncertainty. We argue that these properties define a distinct model class, which we call the Large Quantitative Model (LQM).

1 Introduction

Human description loses quantitative information needed for consequential decisions, and scaling a language model cannot repair that loss or supply other required properties. The paper proposes a distinct model class trained on quantitative records, with language models retained for interface tasks.

1 Introduction

  • Compressing quantitative records into language irreversibly loses fidelity before modeling, and increasing model scale cannot restore fidelity, reproducibility, lineage, or calibration.The paper frames these as properties of the representation and map, not parameter count or training budget.Source: §1 · p. 2
  • A Large Quantitative Model is substrate-native, structurally explicit, lineage-complete, and uncertainty-calibrated, including an abstention state.The proposed class trains on native quantitative records and maps every output to source records.Source: §1 · p. 2
  • In the proposed hybrid, a language model translates intent and narrates conclusions, while a model trained on the domain’s record performs the reasoning.The paper does not argue that language models are unhelpful or irrelevant in these domains.Source: §1 · p. 2
  • The paper argues that prioritizing language-model safety for quantitative decisions is misdirected relative to building models safe by construction.It presents this as a claim about research priorities.Source: §1 · p. 2

2 Preliminaries: four properties of a representation

The preliminaries define a domain record, its representation, and four requirements for consequential models. Fidelity depends on the queries at issue, while reproducibility, lineage, and calibration have distinct operational meanings.

2 Preliminaries: four properties of a representation

  • A domain record is the primitive quantitative data a domain generates; a representation transforms that record, and a model maps a representation and query to an output.The four properties are framed as properties of a representation or map, independent of model class.Source: §2 · p. 2
  • Fidelity is query-relative: a representation preserves a query when its answer is recoverable from that representation, but may fail for other queries about the record.A summary that preserves whether loss development was adverse may not preserve the development triangle itself.Source: §2 · pp. 2–3
  • Reproducibility means identical outputs across independent executions when input, configuration, and release version are fixed.A stochastically trained model can qualify if its deployed map is frozen, versioned, and applied deterministically.Source: §2 · p. 3
  • Lineage completeness requires a computable map from every output to contributing source records, preserved through every pipeline transformation.Unlike post-hoc attribution, lineage returns source records by query rather than estimating which inputs influenced an output.Source: §2 · p. 3
  • Calibration requires stated uncertainty to have its claimed frequency interpretation, prediction-set coverage of at least 1 − α, and abstention below a stated evidence threshold.The definition gives nominal 1 − α coverage as the probability bound.Source: §2 · p. 3
  • The paper says the four requirements reflect what institutions overseeing consequential decisions already ask for, including replicability, documented data flows, and characterized uncertainty.It cites examples from financial, insurance, AI, and medical-device guidance.Source: §2 · p. 3

3 Why language is an insufficient substrate

The section argues that language is an insufficient substrate for consequential quantitative decisions: descriptions irreversibly lose record information, while sampling, implicit representations, and generated confidence fail other required properties.

3.1 Fidelity: description is an irreversible compression

  • If a description leaves more than one bit of query uncertainty, no model can answer with certainty; the error bound depends on the description and query, not model scale.The proof applies the data-processing and Fano inequalities.Source: §3.1 · pp. 3–4
  • The paper quantifies a long-standing encoding problem: descriptions reduce and select information, so expertise at the receiving end cannot restore what was discarded.It connects this loss to quantitative domains, where the missing meaning is a well-defined mathematical object.Source: §3.1 · p. 4
  • Providing raw records in context can help with small, self-contained tasks, but consequential records may exceed context windows and in-context access leaves reproducibility, structure, and lineage unresolved.The passage gives enterprise authentication graphs as an example of records too large for a context window.Source: §3.1 · p. 4

3.2 Reproducibility: sampling is the mechanism, not a setting

  • Sampling makes language-model outputs non-reproducible by construction, while deterministic settings depend on implementation and exposed reasoning is not a faithful computation trace.Wrapping the stochastic core with retrieval, verifiers, logging, or review can supply safeguards, but the paper argues these are properties a quantitative substrate should provide natively.Source: §3.2 · p. 4

3.3 Lineage: attribution is not provenance

  • Attribution estimates which inputs mattered, whereas lineage returns contributing records; retrieval-augmented generation returns documents, not the records behind a reported quantity.The computation from retrieved documents to output still uses the generation mechanism.Source: §3.3 · p. 5

3.4 Calibration: verbalized confidence is generated text

  • Verbalized confidence is generated text that can be overconfident or degrade under shift; external calibration can be imposed, but abstention is not natural to text continuation.The passage names temperature scaling and conformal procedures as external calibration approaches.Source: §3.4 · p. 5

3.5 Empirical corroboration

  • Empirical evidence is consistent with substrate limits: numerical perturbations and compositional depth hurt language models, while purpose-built tabular models outperform general-purpose deep models.The paper characterizes the enterprise generative-AI survey as practitioner evidence, not a controlled, peer-reviewed finding.Source: §3.5 · p. 5

4 What consequential settings require

Consequential settings govern model form as well as outcomes, and the paper frames governance requirements as a specification for models built on quantitative records.

4 What consequential settings require

  • Four regimes—banking, insurance, medical devices, and EU AI regulation—engage the four representational properties, which the paper treats as necessary but not sufficient for compliance.Emergency management and public-safety decision support face similar demands informally, through after-action review and liability.Source: §4 · pp. 5–6, Table 1 · p. 6
  • Governance requirements expose a mismatch between consequential decisions and next-token prediction, including questions of conceptual soundness, replication, and source-data lineage.The passage poses these as challenges for risk, claim severity, and network-movement decisions.Source: §4 · p. 6
  • The paper argues that models trained on quantitative records can carry reproducibility, lineage, calibrated uncertainty, and inspectable representation natively, making governance a specification for the model class.It contrasts building such models with making language models compliant through an exoskeleton.Source: §4 · p. 6

5 The precedent: domain-native substrates elsewhere in machine learning

The paper presents domain-native substrates as an established pattern elsewhere in machine learning and argues that consequential quantitative domains warrant the same shift.

5 The precedent: domain-native substrates elsewhere in machine learning

  • World-model research trains agents on observations containing objects, forces, and causality, then models environment dynamics rather than descriptions [5] [20].The paper asks why the same substrate logic should not apply to portfolios, patient populations, or enterprise networks.Source: §5 · p. 6
  • Protein structure prediction advanced through models trained on protein sequences and structures rather than literature about proteins.The paper presents this as an example of a domain-native quantitative substrate.Source: §5 · p. 7
  • Tabular and time-series foundation models address data properties such as non-stationarity, heavy tails, heterogeneous features, and small samples [21] [2] [12] [46].The paper adds that substrate-nativeness alone does not ensure lineage or an explicit representation, since a model's domain representation may live in its weights.Source: §5 · p. 7
  • Cybersecurity systems increasingly remove language models from consequential decision loops and confine them to the interface [43].The paper describes this as structured models replacing language-based agents that failed in environments they could describe.Source: §5 · p. 7

6 The position: Large Quantitative Models

The paper defines LQMs as foundation-class models native to a domain’s quantitative record, with explicit structure, complete lineage, and calibrated uncertainty. It argues for composing domain-slice models with language models at the interface.

6.1 Definition

  • An LQM is a foundation-class system trained across a domain’s native quantitative record, with explicit structure, complete lineage, calibrated uncertainty, and an abstention state.Individual tasks become conditioned inferences over a structure the model already holds.Source: §6.1 · p. 7
  • Explicit structure makes the domain representation inspectable apart from predictions, but constrains learning; the authors argue that cost is worth paying in consequential settings.A representation can have explicit structure while learning its parameters, such as annotations on a graph of record sets.Source: §6.1 · p. 7
  • Lineage must remain computable through aggregation, projection, dimension reduction, and learned encoding, so learned components consume rather than replace explicit structure.Preserving provenance through ingestion alone is insufficient if it is lost at a learned layer.Source: §6.1 · p. 7

6.2 Neighbouring classes

  • Unlike narrow, hand-built single-task quantitative models, LQMs cover a domain’s record and condition tasks on a shared structural representation.Tabular and time-series foundation models share native data, but as a class do not commit to explicit structure or prediction-to-training-row provenance.Source: §6.2 · p. 8

6.3 Composition, not monolith

  • The proposed end state is many domain-slice foundational models satisfying P1–P4, composed and orchestrated by language models at the interface—not one monolith.The paper treats the composable class, rather than a monolithic model, as the appropriate unit.Source: §6.3 · p. 8

7 One instantiation: refine, structure, learn

The authors present one existence-argument instantiation of a Large Quantitative Model, organized as refinement, explicit structure, and learning over that structure. Its stages preserve record lineage, make structure inspectable, and keep the language model at the interface.

7 One instantiation: refine, structure, learn

  • The three-stage approach is offered as one way to build a Large Quantitative Model, not as part of its definition.The authors describe it at the level of each stage’s contribution and rationale, leaving specific operators for separate treatment.Source: §7 · p. 8

7.1 Refine: from record to lineage-complete substrate

  • The refinement stage turns fragmented, private, semantically undocumented records into entity-level data with exhaustive profiling, column-level privacy enforcement, and source-field provenance.Lineage is recorded at the first data touch because it cannot be reconstructed later; the authors identify refinement as a common point of enterprise deployment failure.Source: §7 · pp. 8–9, §7.1 · p. 9

7.2 Structure: an explicit representation with computable inverse images

  • The structure stage builds a Mapper-derived complex from entity descriptors, then annotates outcomes separately with overlap-corrected estimates and an effective-sample-size abstention threshold.Each structural element has a computable inverse image, and reportable multiscale parameters allow coarse portfolio or fine entity views without remodelling.Source: §7.2 · p. 9
  • The authors do not claim the computed complex recovers an underlying continuous object’s homotopy type, because available nerve guarantees concern covers of spaces, not finite samples.They use topology as a local-to-global organizing device with computable provenance; stability guarantees for general lens dimensions remain partial.Source: §7.2 · p. 9

7.3 Learn: dynamics over the structure, with the language model at the interface

  • Learning proceeds by predicting structure evolution before predicting outcomes conditioned on structure; the observed-structure proposition bounds uncertainty, but does not guarantee gains when future structure is predicted.Calibration uses held-out probability calibration and conformal intervals, while the language model selects structural views and narrates computed results constrained to the computed object.Source: §7.3 · pp. 9–10

8 Evidence: decoupling the language model from the decision

A cyber-operations deployment illustrates a division of labor in which structured models make decisions and the language model supplies a constrained narrative. The reported system combined competitive detection performance with fast decision execution and structured audit controls, but it was not a full Large Quantitative Model.

8 Evidence: decoupling the language model from the decision

  • 0.91±0.02 precision and 0.87±0.03 recall were achieved on held-out red-team events, with the highest precision among compared graph-based detectors and recall within the strongest detectors’ range.The decision pipeline ran in under 100 milliseconds, while language-model narrative synthesis accounted for 60% of the 6.3-second median end-to-end latency.Source: §8 · p. 10
  • An immutable decision ledger, two-source evidence requirement, and tiered autonomy supplied audit controls through the structured policy rather than the language model.Read-only reconnaissance was automatic, reversible containment required critic validation, and irreversible intervention required a human.Source: §8 · p. 10
  • The deployment is not a full Large Quantitative Model: it is single-task, and decision-to-event lineage passes through a learned encoder rather than a computable inverse image.The authors present it as evidence for their proposed division of labor in a domain where language models have been proposed as drop-in analysts.Source: §8 · pp. 10–11
  • In chemogenomics and emergency management, the authors report that explicit, provenance-preserving structures supported defensible inference and made recommendations’ rationale accessible under incomplete data and time pressure.They describe these examples as consistent with the position, not as broader demonstrations.Source: §8 · p. 11

9 Alternative views

The authors address objections that scale, fine-tuning, tools, existing quantitative models, or explicit structure make their position unnecessary. They distinguish language-model capabilities from the substrate and tool design needed for consequential quantitative reasoning, while acknowledging unresolved trade-offs and data-engineering constraints.

9 Alternative views

  • Greater scale can recover more information present in a training distribution, but cannot recover quantitative details omitted by its descriptions.The authors also report that fluency about quantitative domains has improved faster than fidelity across model generations.Source: §9 · p. 11
  • Fine-tuning on serialized tables changes a language model’s behavior, not its text-trained substrate, so sampling, absent lineage, and implicit domain representations remain concerns.The authors cite tabular-model research arguing that architectures and objectives should reflect data’s statistical character.Source: §9 · p. 11
  • Tools and retrieval can delegate computation, but a provenance-preserving quantitative tool is the reasoning core, while generated code leaves reproducibility and lineage bounded by its generation step.The authors regard a tool querying an explicit, provenance-preserving representation with calibrated outputs as an LQM by another name.Source: §9 · p. 11
  • The proposed class differs from classical quantitative models by being foundation-scale, representing a whole domain, and treating tasks as conditioned inferences rather than hand-built single tasks.The authors characterize this as a difference in kind, not degree.Source: §9 · p. 11
  • Explicit structure may sometimes underperform end-to-end learning on a single benchmark metric, and the accuracy–structure trade-off remains an open problem.The authors argue consequential settings require accuracy alongside reproducibility, lineage, and calibration, while noting that structured representations can support expressive learning.Source: §9 · pp. 11–12
  • Fragmented, private, regulated data with undocumented semantics makes model-building dependent on solving refinement, privacy enforcement, and lineage preservation first.The authors frame this as a sequencing constraint rather than evidence that the model class is impossible.Source: §9 · p. 12

10 A research agenda

The research agenda prioritizes foundations for representations and lineage, learning over evolving structure, stability and calibration, and evaluation and interface design. It also identifies autonomous data refinement as a binding constraint on building the proposed model class.

A theory of representational adequacy and lineage as a learning constraint

  • Develop theory of query-relative representational adequacy and information loss, alongside architectures and formal constraints that preserve lineage through learned components.The agenda also calls for principled bounds when lineage passes through learned encoders.Source: §10 · p. 12

Structure-first learning and stability of explicit representations

  • Determine when prediction over future structure improves on direct prediction, and establish stability results for explicit topological representations beyond the currently understood cases.The authors identify general lens dimensions and heterogeneous local reducers as an unresolved weak point in the instantiation.Source: §10 · p. 12

Calibration under drift and benchmarks for the four properties

  • Develop guarantees combining calibration and abstention under drift, including characterization of the gap between observed- and predicted-structure conditioning.The authors also call for benchmarks that score reproducibility, lineage completeness, and calibrated abstention alongside outcomes.Source: §10 · p. 12

Interface protocols between language and quantitative models

  • Specify interfaces that translate intent into queries over explicit structures and constrain generated narration to computed results with auditable record identifiers.The authors identify interface protocols as a research problem where language-model researchers can contribute.Source: §10 · p. 12

Autonomous refinement

  • Treat autonomous refinement of fragmented, private, semantically undocumented records into lineage-complete training data as a research topic and foundational constraint.The authors argue this work deserves attention comparable to model architecture.Source: §10 · p. 12

11 Conclusion

The conclusion argues that consequential quantitative decisions need a model class built on the quantitative record, not language. It says governance demands fit that substrate and notes a first instantiation with evidence supporting the proposed architecture.

11 Conclusion

  • For consequential decisions, the quantitative record—not language—is the appropriate substrate for a model class that can show its work from source records.The authors frame this as a conclusion about the substrate of the model, not a claim that language models are useless.Source: §11 · p. 13
  • The paper argues that governance demands are natural to models built on quantitative records and unnatural to models built on language.It presents the corresponding class as defined by four properties and distinct from neighboring model classes.Source: §11 · p. 13
  • The authors report a first instantiation and evidence that a language-model interface with a structured quantitative core delivers what the position predicts.The language model sits at the interface, while the structured quantitative model is at the core.Source: §11 · p. 13

Competing interests

The authors disclose Duo Dimensio LLC and Indiana University affiliations, and state that the position is their own.

Competing interests

  • R.V., D.I. and D.J.W. are affiliated with Duo Dimensio LLC, which develops the described systems; D.J.W. is also affiliated with Indiana University.The authors state that the position advanced is their own.Source: Competing interests · p. 13
Loading 2609.12105v1…