PaperlayerView exact revision on arXiv

Source-linked AI summary

Cost Scales with Change, Not Corpus Size: Incrementally Maintaining an Evolving Semantic Substrate

Yusuke Takahashi, Kyle Wild, Asako Uraki

arXiv:2608.16621v1cs.AIcs.DBcs.IR

TL;DR

Maintaining a compiled semantic substrate raises questions about update and embedding-model migration costs as corpora evolve. The paper uses incremental low-rank updates and Procrustes alignment, finding maintenance 33.7× cheaper per update and 23.8× cheaper cumulatively than full re-SVD while tracking the full subspace to floating-point precision.

  • Problem

    Keeping a compiled semantic substrate current appears expensive, while changing the embedding model appears to require re-embedding the entire corpus.

  • Method

    The paper maintains a rank-k truncated-SVD substrate with incremental low-rank updates and uses an orthogonal Procrustes virtual-axis update for embedding-model changes.

  • Results

    33.7× cheaper per update and 23.8× cheaper cumulatively than full re-SVD, incremental maintenance tracked the full subspace below 10^-11 degrees of maximum principal-angle drift with recall@10 = 1.0.

  • Takeaways & Limitations

    The synthetic pilot supports maintaining an evolving semantic substrate because maintenance cost scales with corpus change rather than corpus size.

  • Takeaways & Limitations

    The synthetic pilot uses idealized updates, so its speedups and exact-tracking results are best-case bounds requiring validation on real corpora and live revision streams.

Abstract

from arXiv · show

Retrieval-augmented and agentic question-answering systems increasingly re-derive the meaning of a corpus at query time. Put plainly, instead of re-deriving what a corpus means on every question, the work is done once when a document arrives and is thereafter merely consulted -- a compiler, not an interpreter, of meaning. An alternative is to compile that meaning once, at ingest time, into a compact, queryable semantic substrate and maintain it as the corpus evolves. The central objection is maintenance cost: rebuilding a truncated singular value decomposition (SVD) on every change appears prohibitive, and a change of embedding model seems to force a full re-embedding. We argue and show empirically that maintenance cost scales with the amount of change, not corpus size. On a controlled synthetic pilot (dimension 256, rank 32, a corpus grown from 3,000 to 9,000 documents over 50 update events), incremental low-rank updates were 33.7 times cheaper per update than full re-SVD and 23.8 times cheaper cumulatively, while the incremental subspace tracked the full recomputation to within floating-point precision (maximum principal-angle drift below 1e-11 degrees; recall@10 = 1.0). An orthogonal Procrustes virtual axis update recovered 0.95 mean cosine to truly re-embedded vectors by re-embedding only about 10 percent of the corpus. The results support maintaining, rather than repeatedly reconstructing, a semantic substrate.

I. INTRODUCTION

The introduction contrasts query-time semantic reconstruction, which redoes meaning work per question, with ingest-time semantic compilation, which compiles a reusable semantic substrate once. It frames maintenance and embedding-model migration as practical objections and presents incremental updates and virtual-axis adaptation as solutions.

  • Query-time semantic reconstruction: Query-time semantic reconstruction retrieves passages and reconstructs relevant meaning from raw text whenever a question arrives.Recent systems can also translate natural-language questions into executable plans over heterogeneous data.
  • Ingest-time semantic compilation: Ingest-time semantic compilation performs semantic work once, compiling the corpus into a compact, queryable substrate that later queries consult.The introduction analogizes QSR to an interpreter and ISC to a compiler.
  • Problem: The central practical objection is that corpus changes may make substrate maintenance expensive, while embedding-model upgrades may appear to require re-embedding everything.The paper argues this objection is largely misplaced.
  • Contributions: Incremental low-rank updates make maintenance cost scale with change rather than corpus size and track full reconstruction to floating-point precision.The contribution is evaluated on a controlled synthetic pilot and reports an order-of-magnitude cost reduction versus full reconstruction.
  • Contributions: An orthogonal Procrustes virtual axis update absorbs an embedding-model generation change by re-embedding only a small anchor fraction.This provides a migration procedure without re-embedding the entire corpus.

II. LITERATURE REVIEW · A. Semantic spaces and orthogonality

Prior work represents documents and queries in cosine-ranked vector spaces, including low-rank SVD-based semantic spaces with orthogonal singular directions. The paper contrasts reconstructing meaning at every query with compiling it at ingest and maintaining it incrementally for cheap lookup.

  • A. Semantic spaces and orthogonality: Vector space models represent documents and queries as high-dimensional vectors ranked by cosine similarity.
  • A. Semantic spaces and orthogonality: Latent semantic indexing uses a truncated SVD to place related terms near one another in a low-rank space.
  • A. Semantic spaces and orthogonality: The low-rank space’s singular vectors are orthogonal.
  • A. Semantic spaces and orthogonality: An explicit mathematical model of meaning orthogonalizes a semantic space on a basic-vocabulary basis.
  • A. Semantic spaces and orthogonality: Orthogonality is treated as the property that allows cosine to function as a semantic distance.
  • A. Semantic spaces and orthogonality: The manuscript is accepted for publication in the 2026 International Electronics Symposium in Yogyakarta, Indonesia, August 1–3, 2026.The final published version will appear in IEEE Xplore.
  • A. Semantic spaces and orthogonality: QSR reconstructs meaning on every query, whereas ISC compiles meaning once at ingest and maintains it incrementally.Under ISC, a query becomes a cheap lookup.

B. Embedding geometry and anisotropy · C. Dense retrieval and retrieval-augmented generation · D. Indexing, maintenance, and migration

The paper frames semantic compilation as a response to anisotropic embeddings and query-time meaning reconstruction, supported by retrieval indexes, incremental factorization updates, truncation, and Procrustes-based migration. These techniques motivate a maintainable semantic structure rather than repeatedly reconstructing meaning per query.

  • B. Embedding geometry and anisotropy: Contextualized embeddings occupy a narrow cone, inflating cosine similarities between unrelated items.This anisotropy motivates post-hoc correction methods and compiling or re-orthogonalizing a semantic substrate.
  • B. Embedding geometry and anisotropy: Embedding quality varies sharply across tasks and models, motivating substrate compilation and, where useful, re-orthogonalization.The passage characterizes flow-based normalization, contrastive training, and cluster-based isotropy enhancement as partial, retrofitted re-orthogonalization.
  • C. Dense retrieval and retrieval-augmented generation: Dense retrieval learns query and passage encoders for nearest-neighbor search, with late-interaction, fusion-in-decoder, and self-reflective retrieval variants.These approaches define the QSR baseline, in which meaning is reconstructed per query.
  • C. Dense retrieval and retrieval-augmented generation: QSR, production file-search features, and retrieval-oriented MCP servers reconstruct meaning per query rather than persisting a compiled, maintainable semantic structure.The passage identifies this repeated reconstruction as the relevant limitation of these systems.
  • D. Indexing, maintenance, and migration: Approximate nearest-neighbor indexes make semantic-substrate lookups practical on the read side.The passage cites multiple approximate nearest-neighbor indexing approaches as support for practical substrate access.
  • D. Indexing, maintenance, and migration: Incremental and online SVD update factorizations as rows are added without recomputing from scratch on the write side.Nested representations also allow learned spaces to be safely truncated for rank control.
  • D. Indexing, maintenance, and migration: Nested representations support safe truncation of learned spaces for rank control.This provides a representation-management mechanism alongside incremental and online SVD updates.
  • D. Indexing, maintenance, and migration: Orthogonal Procrustes migration has a closed-form solution, while backward-compatible representation learning targets model-space transitions.The passage presents migration between representation spaces as standard.

III. METHODS · A. Preliminaries · B. Semantic substrate

The methods represent documents as vectors, organize the corpus into a matrix, and compress it into a low-rank semantic substrate using truncated SVD. Queries are projected into this subspace and ranked by cosine similarity.

  • A. Preliminaries: Document embeddings encode meaning as vectors, with similar documents represented by similar directions.The corpus is treated as a matrix whose rows are document vectors.
  • A. Preliminaries: Truncated SVD reorganizes the corpus matrix into a compact rank-k representation.Keeping only the top k components reduces the representation size.
  • B. Semantic substrate: The top-k right singular vectors V_k form an orthonormal basis for a low-rank semantic subspace.This basis captures the principal semantic axes retained by the substrate.
  • B. Semantic substrate: The corpus matrix is formed by stacking N document embeddings, each in D dimensions, as X ∈ ℝ^{N×D}.This establishes the dimensional representation used by the semantic substrate.
  • B. Semantic substrate: The substrate is defined by X ≈ U_k Σ_k V_kᵀ, with V_kᵀ V_k = I_k.The factorization is the truncated-SVD form used to construct the semantic substrate.
  • B. Semantic substrate: O(Dk) per item, a query q is projected as q̃ = q V_k and scored using cosine similarity.The projection enables query evaluation within the reduced semantic space.

C. Incremental maintenance vs. full re-SVD · D. Virtual axis update for model changes

The paper contrasts full-corpus truncated SVD recomputation with incremental low-rank maintenance whose cost depends primarily on the changed documents. For embedding-model upgrades, orthogonal Procrustes aligns old and new representations using a small anchor set, enabling virtual axis updates instead of a full rebuild.

  • C. Incremental maintenance vs. full re-SVD: Full re-SVD recomputes the truncated SVD over the whole corpus at roughly O(NDk) per iteration, so cost grows with N.
  • C. Incremental maintenance vs. full re-SVD: Incremental low-rank updates modify (U_k, Σ_k, V_k) directly using only the change, at roughly O(Dk·m + Dk²) per update.This cost is independent of N to first order.
  • C. Incremental maintenance vs. full re-SVD: Periodic re-orthogonalization bounds departure from the condition V_kᵀ V_k = I_k.
  • C. Incremental maintenance vs. full re-SVD: Incremental maintenance adjusts the catalogue using only newly added or revised documents rather than rereading the entire library.
  • D. Virtual axis update for model changes: Old and new embedding-model vectors are not comparable, so a small anchor set is embedded under both models to form M and M̃.
  • D. Virtual axis update for model changes: The orthogonal Procrustes solution MᵀM̃=WΣZᵀ yields R*, which virtually updates axes while re-embedding only the small anchor fraction.This trades a full rebuild for an alignment problem and complements backward-compatible training.

E. Rank control and break-even · F. Experimental design

Rank control frames substrate maintenance as worthwhile when amortized costs beat per-query semantic reconstruction, while the experimental design isolates maintenance and tracking quality in a controlled synthetic pilot. The pilot models gradual topic drift with 256-dimensional document vectors generated from a slowly rotating latent basis.

  • E. Rank control and break-even: The rank k trades fidelity for substrate size, and nested-representation training suggests adapting k to a target precision.Rank selection is presented as a fidelity–size tradeoff rather than a fixed setting.
  • E. Rank control and break-even: Substrate maintenance is worthwhile only when its amortized cost beats per-query reconstruction.The comparison concerns semantic reconstruction performed over raw passages, not merely retrieval.
  • E. Rank control and break-even: R* = (N·c_c + W·c_m) / (c_q − c_r) defines the break-even query count.The expression relates corpus cost, maintenance cost, query reconstruction cost, and retrieval cost.
  • E. Rank control and break-even: Because incremental maintenance keeps c_m small and independent of N, R* stays low in high-query regimes.This is the stated condition under which incremental maintenance preserves a favorable break-even point.
  • F. Experimental design: The controlled synthetic pilot isolates maintenance cost and tracking quality from confounds, while positioning real-corpus evaluation as future work.The experiment is explicitly described as a harness rather than a real-corpus study.
  • F. Experimental design: Each document is modeled as x = B_t c + ε ∈ ℝ^D with D = 256, using a latent semantic basis with R0 = 40.B_t is orthonormalized by QR and drifts through small random rotations with perturbation scale 0.03.

G. Metrics and implementation

The section evaluates maintenance using update and cumulative cost, subspace drift, retrieval recall, and reconstruction error. It compares full re-SVD with an incremental second-moment maintainer and describes Procrustes migration and controlled timing conditions.

  • Metrics: The evaluation measures per-update and cumulative wall-clock cost, maximum principal angle, recall@10 over 200 fixed random queries, and reconstruction error.Full re-SVD recomputes the top-k right singular vectors of the whole corpus at cost ∼O(N D²).
  • Metrics: 23.8–33.7× cheaper: incremental maintenance matched full re-SVD with drift below 10^-11 degrees and recall@10 = 1.0.These are synthetic-pilot results with D=256 and K=32.
  • Implementation: The incremental maintainer accumulates each new batch in a running D×D second-moment matrix and re-extracts top-k eigenvectors at cost ∼O(m D² + D³), independent of N.Migration learns an orthogonal Procrustes map from k ∈ {50, 100, 200, 400, 800} anchor pairs and applies it corpus-wide without re-embedding.
  • Implementation: Timings use standard dense linear algebra on a single CPU core, excluding embedding time shared by both methods.The setup isolates maintenance cost rather than the common embedding cost.

IV. RESULTS

Incremental maintenance remained far cheaper than full re-SVD as the corpus grew, while closely matching the full subspace. A Procrustes virtual-axis update achieved high cosine recovery by re-embedding only a small corpus fraction.

  • Cost asymmetry: 8.4 ms per update versus 283 ms for full re-SVD at N = 9,000 made incremental maintenance 33.7× cheaper and 23.8× cheaper cumulatively over 50 events.Incremental per-update cost stayed flat as the corpus grew, whereas full re-SVD rose with N.
  • Subspace tracking: Below 10⁻¹¹ degrees maximum principal-angle drift and recall@10 = 1.0 showed that incremental and full subspaces tracked to working precision.Incremental reconstruction error matched the full baseline to working precision.
  • Virtual axis update: 0.95 mean cosine to truly re-embedded vectors was recovered with an orthogonal Procrustes map while re-embedding only about 10% of the corpus.The map was learned from a small anchor set.
  • Scaling: The full re-SVD curve rose with N, whereas the incremental curve stayed flat, set by batch size m rather than N, widening the cumulative gap as the corpus grew.These scaling trends were reported in Fig. 3.

V. DISCUSSION

The discussion argues that semantic-substrate maintenance scales with corpus change rather than corpus size, making incremental compilation attractive for large, slowly changing, repeatedly queried corpora. It also frames virtual-axis alignment as a cheaper response to embedding-model upgrades while acknowledging when compilation is not worthwhile and that the synthetic pilot provides best-case evidence.

  • Maintenance economics: Maintenance cost scales with the amount of change, not corpus size: incremental updates stay flat with corpus size while full re-SVD rises with N.Incremental cost is set by batch size m, whereas full recomputation grows with corpus size.
  • Maintenance economics: 23.8× cumulatively, the incremental approach widened its maintenance-cost advantage over 50 update events as the corpus grew.The cumulative gap increased across the update sequence.
  • When compilation pays: Compilation is not always preferable: small, highly volatile, or rarely queried corpora may never reach the break-even count R*, while per-query reconstruction remains preferable.The claim is that dominant maintenance cost is change, not that compiling always wins.
  • Update fidelity: Below 10⁻¹¹ degrees throughout, incremental and full subspaces had maximum principal-angle drift, with recall@10 = 1.0.The discussion characterizes this as matching full re-SVD to floating-point precision rather than serving as a lossy shortcut.
  • Model upgrades: Re-embedding only ~10% of the corpus recovered 0.95 mean cosine to truly re-embedded vectors through a Procrustes virtual axis update.This converts an embedding-model upgrade from full re-embedding into a small alignment problem.
  • Limitations: The pilot is synthetic and idealized, so its absolute speedups and exact-tracking result are best-case bounds requiring validation on noisy real corpora, live revision streams, and real embedding APIs.Rank-one streaming variants may introduce bounded drift, and Procrustes recovery depends on anchor representativeness.

VI. CONCLUSION

The conclusion argues that an evolving semantic substrate should be maintained because maintenance scales with change rather than corpus size, while embedding-model changes can use virtual axis updates instead of full rebuilds. It identifies validation on real revision streams and production systems, plus complementary symbolic substrates, as future work.

  • Conclusion: Maintenance cost scales with corpus change, not corpus size, supporting maintenance over repeated reconstruction.The conclusion presents this claim as supported by the synthetic pilot.
  • Conclusion: Embedding-model changes can be absorbed through a virtual axis update instead of a full rebuild.This is presented as the alternative to rebuilding the semantic substrate after a model change.
  • Future work: Future work will also examine complementary symbolic substrates, including knowledge objects, for provenance and adherence.This complements evaluation of evolving semantic maintenance.
  • Future work: Future work will replay a timestamped Wikipedia revision stream through production embedding APIs and compare with vector-database re-indexing practice.The study will extend the pilot to real-world retrieval and RAG deployments.
Loading 2608.16621v1…