PaperlayerView exact revision on arXiv

Source-linked AI summary

Financial Evidence Crowding: Diagnosing and Mitigating Constraint-Induced Displacement in Retrieval-Augmented Generation

Yixi Zhou, Jiayi Yin, Fan Zhang, Xiangyi He, Haipeng Zhang

arXiv:2609.35782v1cs.IR

TL;DR

Financial retrieval benchmarks do not separate explicit constraint conflict from semantic competition, despite financial queries containing observable constraints. The paper diagnoses this displacement with matched pools and tests gated score correction on fixed candidate banks; the correction recovers covered evidence while preserving the original order when it predicts no improvement.

Abstract

from arXiv · show

Retrieval-augmented generation (RAG) retrieves candidate evidence and sends only a limited top-ranked subset, the top-k context, to a generator. In financial question answering, passages can match a query's topic while conflicting with its period, segment, metric scope, or table scope. We study the resulting set-level ordering failure, which we call financial evidence crowding. FinDeCrowd-Stress isolates this failure through matched compatible and incompatible candidate pools while fixing the query, relevant evidence, ranking model, candidate count, and retrieval budget. On a FinDER test split containing companies unseen during training, incompatible pools reduce top-10 evidence inclusion (Recall@10) by 0.147 relative to equally difficult compatible pools. This gap shows that conflicting candidates consume limited context slots and displace answer-supporting evidence. We then introduce FinDeCrowd-RAG, a learned score correction that combines a fixed relevance score with typed compatibility and local lexical competition. A query-level identity gate applies the correction only when it predicts a better order; otherwise, it preserves the original ranking. On identical controlled candidates, FinDeCrowd-RAG raises top-10 evidence inclusion from 0.757 to 0.902 by recovering evidence already present in the candidate set. On a FinDER index built without query-specific candidate insertion, gated reranking raises this inclusion rate from 0.420 to 0.492, while top-100 retrieval coverage remains 0.743 by design. With a fixed generator, the same ordering change improves answer accuracy and citation recall on FinanceBench and FinQA. These results identify constraint-induced displacement as a measurable RAG evaluation target and show that identity-gated reranking can recover evidence already covered by first-stage retrieval.

Introduction

The paper frames explicit financial-constraint conflicts as a distinct source of RAG evidence displacement, separate from semantic competition. It diagnoses this failure with matched pools and tests a fixed-bank correction with an identity gate.

Introduction

  • Figure 1 depicts topically similar candidates with observed constraint conflicts displacing answer-bearing evidence beyond a fixed top-k boundary.Source: Figure 1 · p. 1
  • Financial benchmark aggregate scores conflate semantic competition with explicit query-constraint conflict, although financial queries and candidates expose period, segment, metric, and table constraints.Source: Introduction · p. 1
  • Financial evidence crowding names displacement caused by explicit constraint conflicts; FinDeCrowd-Stress compares matched pools, while FinDeCrowd-RAG reorders a fixed candidate bank with a gated score adjustment.Source: Introduction · p. 1
  • With 30 distractors, incompatible pools lower Recall@10 by 0.147; correction raises controlled Recall@10 by 0.145 on identical candidates.The introduction also reports that, in a non-injected FinDER index, 43.5% of top-10 failures with evidence in the top 100 are orderable failures.Source: Introduction · p. 2

Related Work

Prior financial retrieval benchmarks report aggregate scores that do not isolate explicit constraint conflict from semantic competition. This paper positions its matched-pool diagnosis and fixed-bank correction as narrower tools for those questions.

Related Work

  • Financial reasoning and retrieval benchmarks report aggregate performance, but those scores do not separately identify explicit constraint conflict and semantic competition.Source: Related Work · p. 2
  • Unlike hard-negative and neighborhood-competition studies, FinDeCrowd-Stress compares matched candidate sets that differ in observable compatibility under a fixed ranker and budget.Source: Related Work · p. 2
  • Structure-aware retrieval methods optimize overall quality, whereas FinDeCrowd-RAG tests whether fixed-bank correction repairs diagnosed constraint-induced loss while preserving the bank and base score.Source: Related Work · pp. 2–3

Ordering Failure

The diagnostic treats financial evidence crowding as a set-level allocation failure under a fixed context budget. Matched compatible and incompatible pools isolate the additional displacement associated with observed constraint conflict.

Ordering Failure

  • Compatibility means observed entity, metric, period, segment, table-scope, unit, and currency fields contain no explicit conflict; missing fields remain unknown.Source: Ordering Failure · p. 3
  • FinDeCrowd-Stress compares same-size incompatible and compatible pools using a higher-is-better top-k metric; a negative delta indicates additional displacement under incompatibility.The contrast is specific to the ranker, budget, and intervention because the intervention fixes the query, relevant evidence, ranker, candidate count, construction protocol, and matching population.Source: Ordering Failure · p. 3
  • Typed interventions use candidates that conflict on one target field while preserving other jointly observed fields, and compatible hard candidates match constraints without supporting the answer.The candidate list is similarity-ordered and frozen before evaluated rankers score it; nested pools contain 10, 30, or 60 distractors.Source: Ordering Failure · p. 3
  • The primary comparison matches pools on model and lexical similarity, company, table form, length, and numeric density, with entity-disjoint splits and identical candidate IDs and relevance sets.Source: Ordering Failure · p. 3
  • A non-injected top-10 miss is labeled crowded only if relevant evidence is ranked 11–100, a conflicting candidate outranks it, and deleting conflicting candidates moves it into the top 10.Source: Ordering Failure · p. 3

Constraint-Aware Correction

FinDeCrowd-RAG adjusts scores within a fixed candidate bank using a learned residual and a query-level identity gate. The gate applies the correction only when it predicts improved ranking quality; otherwise, the base order is preserved.

Constraint-Aware Correction

  • FinDeCrowd-RAG preserves the frozen relevance score and learns a residual combining lexical relevance, typed compatibility, schema coverage, and local lexical concentration, without retrieving new evidence.A deterministic field extractor had average field-level F1 of 0.959 on a 480-record audit; missing observations contribute zero match and zero conflict.Source: Constraint-Aware Correction · p. 3
  • Local lexical concentration averages similarity to a candidate’s five nearest bank competitors; larger values indicate repeated local competition for top-k slots.Explicit conflicts alone drive crowding diagnosis; local concentration enters only the correction.Source: Constraint-Aware Correction · p. 3
  • The final score adds a standardized linear residual to a within-bank normalized base score; α sets correction scale and γ controls activation.The residual is trained on supporting-versus-distracting candidate pairs, with at most 30 high-ranked distractors per query-pool bank and unit total weight per bank.Source: Constraint-Aware Correction · p. 3
  • The identity gate predicts whether the residual improves nDCG@10, choosing either the residual order or the original order while preserving candidate membership.Its inputs include financial fields, detected conflicts, distance from training ranges, the top-score margin, and local concentration; target domains require disjoint relevance-labeled calibration data.Source: Constraint-Aware Correction · pp. 3–4

Experiments

The experiments isolate constraint conflict from semantic difficulty, test fixed-bank correction and transfer, then assess ordinary retrieval and answer support. Across these settings, conflict displaces evidence, while gated reranking improves context allocation and downstream answer support.

Evaluation Protocol

  • The evaluation uses matched FinDER pools, fixed candidate banks, external financial datasets, a non-injected FinDER index, and fixed-generator tests on disjoint FinanceBench and FinQA subsets.Recall@100 measures retrieval coverage, Recall@10 context allocation, and nDCG@10 rewards earlier supporting evidence.Source: Evaluation Protocol · p. 4

Incompatibility Adds Displacement Beyond Hardness

  • Matched incompatible pools reach 0.736 Recall@10 versus 0.883 for compatible pools, a 0.147 gap; the highest-ranked supporting unit also moves 3.34 positions lower.The comparison matches semantic similarity and surface-form characteristics; robustness checks retain gaps between 0.136 and 0.151.Source: Incompatibility Adds Displacement… · pp. 4–5, Table 1 · p. 4

Fixed-Bank Correction Recovers Covered Evidence

Correction Transfers Across Bases and Datasets

  • Across four datasets, the frozen FinDER residual recovers 0.035 to 0.052 Recall@10 under lexical hard negatives, and correction improves four base rankers after base-specific fitting.Expanded relevance reduces the measured gain from 0.145 to 0.124–0.128; with the tested conflict family withheld from training, macro Recall@10 remains 0.115 above Frozen BGE.Source: Table 4 · p. 5, Correction Transfers Across Bases and… · p. 6, Table 5 · p. 6

Crowding Appears in Non-Injected Retrieval

  • On the augmented FinDER bank, Recall@10 rises from 0.420 to 0.492 while Recall@100 remains 0.743 by design; the gate repairs 66.3% of crowded failures and lowers regression to 2.6%.Crowding accounts for 43.5% of augmented-bank and 43.3% of raw-only orderable failures; the gate cannot repair 147 queries whose evidence is absent from the top 100.Source: Crowding Appears in Non-Injected… · p. 6, Table 7 · p. 6
  • The controlled construction audit reaches 96.5% agreement; the separate non-injected audit estimates 92.5% precision for crowded labels and 88.8% for the primary conflict family.The identity gate requires a small labeled calibration set for each target domain.Source: Crowding Appears in Non-Injected… · p. 6

Better Ordering Improves Answer Support

  • In fixed-generator held-out tests (120 FinanceBench and 1,047 FinQA queries), identity-gated correction versus original order raises answer accuracy and citation recall on both; unsupported responses fall 8.3 and 2.2 points, respectively.Source: Better Ordering Improves Answer Support · p. 7, Table 9 · p. 7

Discussion: Coverage, Allocation, and Use

The discussion separates evidence availability, context allocation, and answer use, then considers how fixed-bank correction and identity gating affect those stages. It also identifies the financial scope and remaining limits of the diagnosis.

Discussion: Coverage, Allocation, and Use

  • Recall@100 measures evidence availability, Recall@10 measures allocation to generator context, and nDCG and generation metrics test placement and evidence use.The matched diagnostic holds candidate count and observed difficulty fixed while varying compatibility.Source: Discussion: Coverage, Allocation, and… · p. 7
  • Matched pools show explicit constraint conflicts contribute displacement beyond generic semantic competition, with repeated conflicts consuming more of the fixed top-10 budget.Compatible and incompatible pools share candidate count, company, table form, and observed difficulty; the contrast is stable across gold-free construction and common-support estimators.Source: Discussion: Coverage, Allocation, and… · p. 7
  • Fixed-bank correction improves allocation only among covered evidence: it retrieves no new passages and cannot repair queries lacking relevant top-100 evidence.The gate repairs 53 of 80 augmented-bank crowded failures and repairs none of the 147 queries without relevant top-100 evidence.Source: Discussion: Coverage, Allocation, and… · p. 7
  • Identity gating trades a smaller mean gain for lower regression than always-on correction, with reported regression rates of 2.6% and 10.7%, respectively.The residual risk is among already successful queries; crowded orderable queries incur no nDCG regression.Source: Discussion: Coverage, Allocation, and… · p. 7
  • The diagnosis uses auditable financial constraints, while cross-domain validation and zero-shot gate transfer remain open.Missing fields remain unknown and incur no conflict penalty; the reported evidence remains financial and fixed-bank.Source: Discussion: Coverage, Allocation, and… · p. 7

Responsible Use

The study evaluates evidence ordering rather than financial recommendation, but incorrect extracted fields can still mislead users.

Responsible Use

  • FinDeCrowd-RAG orders evidence and does not act as a financial recommender.The study uses public financial filings and released benchmark evidence.Source: Responsible Use · p. 7
  • Incorrect company, period, metric-scope, or segment values can still mislead users, and the reported regression rate leaves residual risk.The passage reports a 2.6% regression rate.Source: Responsible Use · p. 7
  • Deployments should expose citations and extracted constraints, retain identity fallback, re-audit extraction after corpus shifts, and route material decisions to human review.Source: Responsible Use · p. 7

Limitations

The diagnostic and correction have explicit scope limits, including unknown financial fields, corpus-dependent prevalence, and constrained downstream evaluation.

Limitations

  • The diagnostic covers explicit financial fields under specified rankers, budgets, and interventions; implicit periods and rare metrics may remain unknown.Restatements, unit conversion, and complex tables may also remain unknown.Source: Limitations · p. 7
  • Non-injected crowding prevalence depends on corpus composition, chunking, coverage, and reference mapping.Source: Limitations · p. 7
  • External gates use small disjoint relevance-labeled calibration sets, and the downstream study uses one generator and citation protocol.Reranking cannot recover evidence absent from the candidate bank.Source: Limitations · p. 7

Conclusion

The conclusion frames financial evidence crowding as a measurable set-level RAG failure and presents fixed-bank correction with identity gating as a way to recover covered evidence.

Conclusion

  • Matched controls isolate constraint-induced displacement beyond generic semantic competition, and FinDeCrowd-RAG recovers covered evidence while its identity gate limits collateral changes.Source: Conclusion · pp. 7–8
  • The same ordering failure appears in non-injected retrieval, and gated reranking improves answer support.Source: Conclusion · p. 8
  • The conclusion motivates constraint-aware context allocation as a distinct RAG target.Source: Conclusion · p. 8
Loading 2609.35782v1…