Source-linked AI summary
Factorized Hypothesis Search for Evidence-to-Taxonomy Retrieval
Linhai Ma, Ethan F. Wei, Xueqing Peng, Yan Wang, Lingfei Qian, Víctor Gutiérrez-Basulto
TL;DR
Indirect evidence often does not explicitly express the taxonomy concept it supports, creating a retrieval-readiness gap. Factorized Hypothesis Search represents competing interpretations as structured semantic hypotheses and achieves the strongest head-ranking and final prediction results across financial and clinical coding tasks among compared methods.
Problem
Evidence-to-taxonomy retrieval must infer a target concept from indirect contextual evidence rather than an input that explicitly expresses it.
Method
Factorized Hypothesis Search represents competing interpretations as factorized semantic hypotheses, fuses their retrieval results, and verifies candidates across semantic dimensions.
Results
Across financial tagging and clinical coding, FHS achieves the strongest head-ranking and final prediction results among the compared methods.
Takeaways & Limitations
Structured comparison among competing interpretations supports FHS, while sequential revision adds no improvement over a strong parallel round despite substantially greater cost.
Takeaways & Limitations
Applying FHS to inventories without existing taxonomy structure and metadata may require defining or inducing an appropriate semantic schema.
Abstract
from arXiv · showhide
Large-taxonomy retrieval often assumes that the input already expresses the target concept. In many settings, however, the input is indirect evidence, such as a table cell whose meaning depends on its row, column, datatype, and context. We call this mismatch the retrieval readiness gap. Our analysis shows that the current index retrieves the target reliably when its semantics are explicit, while raw evidence often leaves it deep in the ranking. We propose Factorized Hypothesis Search (FHS), which maintains multiple partial interpretations over named semantic dimensions. These hypotheses support structured query rendering, multi-hypothesis retrieval, and dimension-level candidate verification. On both financial taxonomy tagging and CodiEsp clinical coding tasks, FHS achieves the best Recall@1, MRR, and final accuracy among the non-oracle methods. Replacing the factorized hypothesis path with a free-text ensemble causes the largest drop in head-ranking performance, while sequential refinement provides no additional gain over FHS's strong parallel first round.
1 Introduction
Evidence-to-taxonomy retrieval addresses cases where indirect, contextual evidence must be interpreted before mapping to a large concept inventory. Factorized Hypothesis Search maintains partial semantic interpretations to improve retrieval and verification across financial and clinical tasks.
- Motivation: Indirect evidence creates a retrieval readiness gap because target concepts depend on contextual dimensions rather than being explicitly expressed.Financial cells require numerical values, headers, datatype, and adjacent cells; clinical coding depends on analyte, specimen type, and clinical notes.
- Motivation: Single-query rewriting is risky because evidence can support multiple plausible interpretations, and premature commitment may direct retrieval into the wrong taxonomy sub-hierarchy.The problem concerns uncertain core semantics, not merely lexical refinement.
- Method: FHS maintains explicit partial hypotheses over semantic dimensions, leaving unsupported dimensions unresolved while generating structured queries for retrieval.Dimensions include concept family, event type, qualifier, scope, and temporal context; each hypothesis produces definition-style and label-form queries.
- Results: Across financial tagging and clinical diagnosis coding, FHS improves head ranking and final accuracy over direct retrieval and strong grounding baselines.Its pipeline fuses rankings into a candidate pool and uses candidate-level verification against hypothesis commitments.
- Results: Ablations identify the factorized hypothesis path as particularly important for head ranking, with definition-form rendering and candidate-level verification providing additional gains.These findings are reported in the introduction as support for the framework’s retrieval design.
- Results: Sequential refinement adds no significant improvement over FHS’s initial parallel round despite substantially greater inference cost, supporting FHS as a general framework.The comparison is reported across the paper’s evaluation results.
2 Related Work
The related work situates the task within concept-inventory alignment across biomedical, clinical, schema, table, and XBRL domains. It also distinguishes FHS from self-correction and aggregation approaches through relative candidate verification and factorized sampling.
- Concept Inventory Alignment: Concept-inventory alignment spans biomedical entity linking, medical coding, schema linking, table-cell linking with context, and XBRL.These examples cover structured-inventory mapping across multiple application domains.
- Concept Inventory Alignment: Iteration improves over weak single-pass grounding but not over FHS’s strong parallel first round.The comparison is reported in Section 4.5.
- Evaluation Setting: The evaluation excludes methods requiring retraining or interactive user clarification because it is zero-shot and offline.These methods fall outside the stated evaluation setting.
- Self-Correction, Verification, and Aggregation: FHS uses the LLM as a relative verifier over retrieved candidates rather than an absolute self-verifier.This design responds to prior observations that intrinsic self-correction can produce wavering and that confidence can trade off against critique.
- Self-Correction, Verification, and Aggregation: Unlike majority-vote self-consistency, FHS’s sampled ensemble uses factorization so its samples disagree on identified dimensions.The supplied passage contrasts the ensemble’s sampling with factorized disagreement.
3 Problem Formulation
The problem is to transform located evidence in shared context into retrieval-ready queries because neither the context nor the evidence locus uniquely names a taxonomy concept. The retrieval readiness gap is interpretive: targets are often reachable but poorly ranked, motivating factorized hypotheses, verification, and consolidation.
- 3.1 Evidence-to-Taxonomy Retrieval: A taxonomy contains concepts with canonical labels, definitions, and structural metadata, while each input is a located fact embedded in a shared source context.Financial tables contain 21.3 facts on average, with different facts grounding to different concepts.
- 3.1 Evidence-to-Taxonomy Retrieval: Because the shared context hosts many facts and the locus may be a bare numeric cell or short mention, neither alone specifies a single retrieval target.A grounding function must construct a query from the located fact by reading its context.
- 3.2 The Gap Is Interpretive: Grounding maps located facts into retrieval-ready queries, whereas direct fact serialization may fail to express the target concept explicitly.The paper formalizes this contrast with an oracle grounding function g∗ and a raw query qraw(x).
- 3.2 The Gap Is Interpretive: The retrieval readiness gap arises from interpretive ambiguity: a fact admits multiple plausible readings, and the system must determine the correct one.This differs from query–document distribution mismatch addressed by hypothetical document generation and query expansion.
- 3.3 The Gap Is Primarily About Precision: 75.2% of raw facts retrieve the gold concept in the top 200, but only 24.0% place it in the top 10, showing that the main bottleneck is ranking position.The target is usually reachable but badly placed, so grounding primarily acts on head ranking.
- 3.3 The Gap Is Primarily About Precision: A single structured hypothesis raises top-10 recall from 24.0% to 35.3% and MRR from 0.111 to 0.195, while top-200 recall falls to 65.7%.These values are reported for grounding in Table 1.
- 3.4 Factorized Interpretations: Factorized interpretations represent concepts across M semantic dimensions, resolving some dimensions into values while leaving others unresolved.Resolved values form label-style queries, and candidate-level verification returns a verdict for each dimension.
- 3.5 Ensemble and Objective: Useful hypothesis complementarity comes from plausible differences on unresolved dimensions, not forced diversity; forced diversity reduced individual retrieval quality by 3–8 Recall@200 points.The system therefore samples an unmodified generator and lets verification and aggregation exploit naturally arising complementarity.
4 FHS: Factorized Hypothesis Search
FHS searches over multiple factorized partial hypotheses, rendering them into structured queries and combining their retrieved candidates before dimension-aware verification. Its parallel design preserves cautious uncertainty, while sequential refinement adds cost without improving retrieval or final accuracy.
- Pipeline: FHS generates J factorized hypotheses, renders queries, fuses retrieved rankings, and reranks candidates across four stages.Generation and reranking calls are fully parallelizable, while rendering, retrieval, and fusion are programmatic.
- Hypothesis generation: Unsupported dimensions remain blank, avoiding unsupported query values that could steer retrieval toward irrelevant regions.Blank dimensions are skipped during rendering and excluded from later scoring, whereas guessed errors enter the query.
- Query rendering: Each hypothesis produces LLM-written definition-style and rule-written label-style queries, with the cell identifier always included.The label query concatenates resolved dimensions canonically and is omitted when no dimension is resolved; both queries are generated in the structured call.
- Fusion and verification: FHS aggregates candidates with reciprocal-rank fusion and range-normalizes scores before passing each hypothesis and its candidate window to the verifier.The verifier window scans the top 60 candidates and retains the highest-ranked candidate from each distinct category profile, covering structurally different interpretations.
- Sequential refinement: Sequential refinement substantially changes the top-50 pool but yields no significant Recall@50 improvement and lower final accuracy at considerably greater inference cost.FHS-Seq uses a revise-and-refetch loop; its first round is identical to FHS’s parallel round, isolating the effect of later rounds.
5 Experiments
Experiments evaluate FHS across grounding accuracy, component contributions, candidate verification, and sequential revision under controlled model and retrieval settings. FHS achieves the strongest head-ranking results on Financial Tagging, with gains concentrated in structured representations and candidate ordering.
- Experimental design: The experiments test FHS against direct retrieval, single-pass, parallel, iterative, and learned baselines, while examining factorization, ensembling, rendering, fusion, verification, and sequential revision.All LLM-based operations and downstream listwise selection use Qwen3-32B, and Recall and MRR are measured after retrieval.
- Main results: 0.185 Recall@1 is FHS’s Financial Tagging result, gaining 0.044 over retrieval-feedback refinement; MRR rises from 0.238 to 0.257 and final accuracy from 0.234 to 0.255.Retrieval-feedback refinement instead reaches 0.660 Recall@50, compared with 0.543 for FHS.
- Component analysis: Replacing the factorized path with a same-size free-text ensemble reduces Recall@1 from 0.185 to 0.116 and MRR from 0.257 to 0.194, while Recall@50 increases from 0.543 to 0.583.This ablation jointly removes factorized representation and dimension-level verification.
- Component analysis: Removing definition-style rendering reduces Recall@1 by 0.034 and final accuracy by 0.036, larger losses than removing label-form rendering.The comparison identifies definition-style queries as more important to performance than label-form queries.
- Component analysis: The candidate-level verifier increases Recall@1 from 0.124 to 0.185 and MRR from 0.205 to 0.257, while Recall@50 remains 0.543.The verifier improves ordering within an existing candidate pool rather than expanding retrieval coverage.
6 Conclusion
The conclusion frames evidence-to-taxonomy retrieval as requiring interpretation before querying and presents FHS as a factorized approach for competing interpretations. Across financial and clinical coding tasks, FHS leads compared methods, with gains tied to structured hypotheses, definition-form rendering, and candidate verification rather than repeated query refinement.
- Evidence-to-taxonomy retrieval requires interpreting observed evidence before it becomes a useful retrieval query.
- FHS represents competing interpretations as factorized hypotheses, fuses retrieval results, and reranks the resulting candidates.
- FHS achieves the strongest head-ranking and final prediction results among compared methods across financial tagging and clinical coding.
- Ablations attribute FHS gains primarily to the structured hypothesis path, definition-form rendering, and candidate-level verification.
- Sequential revision provides no improvement over a strong parallel round despite substantially greater cost, supporting structured comparison over repeated refinement of one query.
Limitations
The evaluation reveals two limitations: FHS depends on taxonomy structure and metadata to define semantic dimensions, and the experiments use only Qwen3-32B, leaving robustness across model families and scales unestablished.
- Schema dependence: FHS may require defining or inducing a semantic schema when an inventory lacks existing taxonomy structure and metadata.Automating this schema induction is identified as a direction for future work.
- Model coverage: All LLM-based components use Qwen3-32B, so the controlled setup does not establish performance across model families or scales.Future evaluations should characterize the robustness–cost trade-off across these dimensions.
Ethical Considerations
The study uses public, non-user-generated data without collecting new personal information or identifying individuals, but recognizes risks from automated taxonomy tagging. It therefore positions FHS as support for expert review and preserves legacy clinical terminology for retrieval fidelity.
- Data and privacy: The study uses public financial filings and publicly released, physician-selected CodiEsp case reports, without collecting new personal data or attempting to identify individuals.Neither corpus is user-generated, and the authors found no offensive content in the evidence-side data.
- Human oversight: Automated taxonomy tagging may affect financial analysis, clinical coding, and reporting, so FHS should support candidate generation and expert decision making rather than replace human review.The passage warns that incorrect grounding may propagate into subsequent analysis, coding, and reporting.
- Terminology: The ICD-10-CM inventory retains legacy clinical terms that may be stigmatizing today, which the study indexes verbatim to preserve retrieval fidelity to official code descriptions.The authors state that they neither endorse nor promote these terms.
Artifact Use and Licensing
The study uses public, source-de-identified financial and clinical corpora without collecting or re-identifying personal data. It releases code and related artifacts while pointing to, rather than redistributing, the original corpora.
- Data sources: The financial corpus comes from public company filings, while CodiEsp contains open-access clinical case reports released under CC-BY 4.0.CodiEsp cases were selected by a practicing physician and a clinical documentalist.
- Licensing and release: The release includes code, prompts, vocabularies, and evaluation/diagnostic artifacts, but not the corpora themselves.The authors point users to the original CC-BY 4.0 release and make no attempt to re-identify individuals.
Use of AI Assistants … D Design Decisions on Development Data
The paper specifies how FHS is instantiated across financial tagging and CodiEsp coding, with controlled representations, shared retrieval infrastructure, and a carefully matched evaluation protocol. Development-set choices and ablations distinguish the effects of factorization, rendering, fusion, verification, and selection.
- Use of AI Assistants: AI assistants supported language polishing, brainstorming, and LaTeX editing, while authors verified all technical claims, results, analyses, and final writing decisions.
- A Task Instantiation Details; A.1 Financial Tagging; A.2 CodiEsp Diagnosis Coding: The financial test split contains 191 source contexts, 2509 target facts, and 388 distinct gold concepts, while CodiEsp contains 250 clinical cases, 3144 target facts, and 958 distinct gold codes.Financial datatypes are 90.0% monetary, 4.5% percent, 2.8% shares, 2.3% per-share, and 0.4% integer; CodiEsp uses relocated English diagnosis mentions.
- A.1 Financial Tagging; A.2 CodiEsp Diagnosis Coding: Financial retrieval indexes all 17 388 US-GAAP concepts, applies datatype compatibility before scoring, and averages 6919 candidates per query while retaining the gold concept on 100.0% of test facts.CodiEsp indexes 71 344 billable FY2018 diagnosis-code candidates, with the inventory restriction applied identically to every method.
- B.2 The label-form renderer; B.3 Generator output schema; A.1 Financial Tagging: The renderer and generator schema preserve unresolved dimensions, provide a definition-form retrieval query, normalize controlled vocabulary outputs, and deduplicate retrieval tokens before coverage scoring.The shared tokenizer handles function-word removal, light plural normalization, and camel-case splitting.
- B Method Specification; B.1 Pseudocode; A.1 Financial Tagging; A.2 CodiEsp Diagnosis Coding: FHS generates multiple factorized hypotheses, renders each in label and definition forms, fuses retrieved rankings, verifies profile-diverse candidate windows dimension by dimension, and reranks before selection.Financial profiles use FAMILY, QUALIFIER, SCOPE, and TEMPORAL; CodiEsp profiles add ROLE, yielding five controlled dimensions.
- B.4 Fusion and Normalization; C.4 Reading the Component Ablation: Summed reciprocal-rank fusion uses κ = 60, followed by range normalization so verifier weight β remains comparable across ensemble sizes.The paper contrasts mean RRF, summed RRF, and raw fused scores in component analysis.
- B.5 Implementation and Hyperparameters; C.2 What is held identical, what is matched, and what is not: All arms share QWEN3-32B, the taxonomy index, tokenizer, renderer, retrieval depth, and selector; FHS and parallel sampling use temperature 0.8, while every parallel arm draws J=2 samples.FHS uses four model calls because it adds two verifier calls, compared with two calls for the parallel baseline.
D.1 Development Design Study … E.3 Candidate-Level Verifier: Behavior
Development studies select stochastic sampling, summed fusion, and a cost–quality choice of J=2, while diagnostics show that retrieval readiness, coverage, and verifier-window design shape ranking quality. On frozen test data, explicit concept queries retrieve reliably, whereas raw evidence mainly suffers from shallow precision and unresolved semantic dimensions.
- D.1 Development Design Study: Plain stochastic sampling has the highest accumulated coverage, while forced-diversity variants reduce coverage by lowering single-hypothesis recall.The diversity findings are development-pilot results rather than a universal claim about directed specialization.
- D.1 Development Design Study: Fusing all J=3 hypotheses reaches 0.541 Recall@10 versus 0.440 for verifier selection, while summed RRF exceeds mean RRF by 6.6 points at Recall@10 and 3.9 points at MRR.The development contrast is treated as a selection result because the final test-set difference is within uncertainty.
- D.1 Development Design Study: J=2 is deployed as the lower-cost operating point although development performance peaks at J=3, because the third hypothesis adds little for one extra generation call.The J=3 fused system reaches 0.541 Recall@10 against a 0.581 oracle best-of-J, leaving a four-point selector headroom.
- D.2 Rerank Weight Sensitivity: Range normalization keeps rerank weight β comparable across ensemble sizes, and development Recall@10 and MRR show a broad optimum across weights and ensemble sizes.Summed RRF scales with the number of fused rankings, whereas verifier support remains in [0, 1].
- D.3 Label-Coverage Diagnostics: Coverage gains are largest for raw context and concentrate on short generic labels, while the interaction between coverage and label-form rendering is null against raw evidence.One- and two-token concepts are never retrieved within the top ten without the coverage term, whereas concepts with five or more tokens gain substantially less.
- E.1 Retrieval-Readiness Diagnostics: Gold-label queries retrieve every test fact within the top ten at MRR 0.972, showing that the retrieval-readiness gap is interpretive rather than index-side.Raw context reaches gold concepts more often at deep cutoffs, while structured grounding improves shallow recall by +0.113 [0.051, 0.178] for the definition form.
- E.1 Retrieval-Readiness Diagnostics: Canonical-label-only queries miss rank 1 for six concepts because longer compounds containing the queried term outrank exact matches, motivating label-side normalization.The six concepts are Assets, Liabilities, Revenues, Goodwill, Depreciation, and RegulatoryAssetsCurrent.
- E.2 Candidate-Level Verifier: Window, Judged Dimensions, and Sensitivity: The deployed verifier window Kv=10 balances ranking quality: Kv=5 costs Recall@10 by −0.014 [−0.022, −0.008], while Kv=20 gains +0.014 [0.005, 0.024] without improving MRR or Recall@1.The verifier evaluates six dimensions, excludes abstentions from support rates, and alternative abstention handling changes metrics by at most 0.003.
E.4 Retriever Robustness Results … F Sequential Refinement: A Negative Result
Across retrievers, FHS preserves its advantage over one-pass grounding, while evidence modality strongly affects the value of grounding. Retrieval-feedback refinement improves coverage but does not consistently improve top-1 ranking or final accuracy, and reported FHS latency is serial rather than fully parallel.
- E.4 Retriever Robustness Results: FHS retains its MRR and accuracy margin over one-pass grounding under sparse, dense, and hybrid retrieval.The retriever swaps do not close or reverse the gap, so subsequent experiments use BM25 as the cheapest and most reproducible option.
- E.4 Retriever Robustness Results: The retriever changes FHS’s margin only slightly: MRR margins are +0.087, +0.082, and +0.082, while accuracy margins are +0.056, +0.057, and +0.049.These values correspond to BM25, dense, and hybrid retrieval, respectively.
- E.5 Results by Evidence Modality: Tabular evidence is substantially harder than text: direct-query accuracy is 0.117 for tables versus 0.339 for text.A table cell distributes meaning across row and column headers, whereas a narrative sentence more often names its concept directly.
- E.5 Results by Evidence Modality: FHS raises tabular accuracy from 0.117 to 0.246, a +0.129 gain, compared with a +0.042 gain on text.The results therefore indicate a larger interpretive benefit from grounding for tabular evidence.
- F Sequential Refinement: A Negative Result: Retrieval-feedback refinement wins Recall@50 on both modalities but does not consistently convert its coverage advantage into superior top-1 ranking or final accuracy.On text, its margins over FHS are significant for Recall@50 (+0.167 [+0.090, +0.253]) and MRR (+0.084 [+0.009, +0.160]), but not for Recall@1 (+0.071 [−0.017, +0.163]) or accuracy (+0.066 [−0.013, +0.148]).
- E.6 Computational Cost: Table 19 measures per-fact LLM calls and retrieval operations alongside end-to-end wall-clock time on the CodiEsp test set.LLM counts exclude the shared final listwise selector, and wall-clock time is measured per full test-set run.
- E.6 Computational Cost: Reported FHS wall time is not optimal parallel latency because the current implementation executes hypothesis generation and verification serially.A fully parallel implementation would obtain different latency characteristics.
F.1 Sequential Refinement: Full Analysis … G.3 What the first-domain cases have in common
The analysis finds that sequential refinement improves weak one-pass grounding but offers no gain over FHS’s strong parallel first round. Qualitative cases show that factorized hypotheses and candidate verification address both ranking ambiguity and wrong semantic framing.
- F.1 Sequential Refinement: Full Analysis: Iteration improves weak single-pass grounding, but repeated revision of one weak grounding is not superior to FHS’s parallel first round.The comparison is made on the full test split; the sequential method differs only in control flow.
- F.1 Sequential Refinement: Full Analysis: 0.550 Recall@50 after FHS’s parallel round falls to 0.544 for the four-round episode, a difference of −0.006 [−0.016, 0.002].Later rounds replace 13.8% of the top-50 pool and consume 3.3 rounds on average, yet the nominal decrease is not distinguishable from zero.
- F.1 Sequential Refinement: Full Analysis: FHS achieves 0.185 Recall@1, 0.397 Recall@10, 0.543 Recall@50, 0.257 MRR, and 0.255 final accuracy.Oracle selection over the same hypotheses reaches 0.222, 0.437, 0.592, 0.295, and 0.262, respectively.
- F.2 Sequential Control: Full-Test Diagnostics: Later rounds replace 13.8% of the top-50 pool and move Recall@50 by −0.006 [−0.016, 0.002], changing the pool without improving it.The gold concept never enters the accumulated pool for 0.298 of facts, while 0.733 of instances use all four rounds.
- G Qualitative Case Studies: The qualitative appendix follows four individual facts through the deployed FHS pipeline, covering two financial cases and two CodiEsp diagnosis cases.These cases illustrate the mechanism rather than represent either domain or evidence type.
- G Qualitative Case Studies: The verifier moves the gold concept to rank one for 156 financial facts and away from rank one for 5, compared with 221 and 78 in the deployed CodiEsp run.The deployed score is Sfinal = eS + βv with β=0.6.
- G.1 A tabular fact: a share-based compensation rollforward: In the tabular case, the cell supports several taxonomy concepts; factorized hypotheses capture the intended reading, but fusion alone ranks the gold concept seventh.The verifier separates closely related labels through named dimensions, motivating a profile-diverse candidate window.
- G.2 A narrative fact: an amended securitization facility: For the narrative case, iterative methods rank the gold concept 172 and 194, while direct retrieval ranks it 20 and parallel sampling and decomposed retrieval miss the top 200.The financial cases therefore fail through different mechanisms: close-label ranking ambiguity in the table and incorrect event framing in the narrative, addressed by dimension-specific verification.
G.4 CodiEsp diagnosis cases
CodiEsp evaluates FHS on relocated diagnosis mentions mapped to ICD–10–CM codes, where lexical ambiguity and broader clinical context determine the intended diagnosis. Across gastric reflux and pulmonary metastasis, factorized hypotheses and candidate verification resolve retrieval errors by enforcing named clinical dimensions.
- Experimental setup: CodiEsp uses relocated English machine-translated diagnosis mentions, ICD–10–CM candidate codes, six verifier dimensions, and the full exact-relocation test split with wcov=1.0.Tables 26 and 27 use candidate sets from the deployed FHS run.
- Gastric reflux: For gastric reflux, lexical ambiguity with vesicoureteral reflux codes is resolved by hypotheses that state the gastric reading directly, targeting gold code K21.9.The note’s gastrectomy and Billroth II context supports gastroesophageal reflux disease without esophagitis.
- Pulmonary metastasis: For pulmonary metastasis, renal carcinoma, metastasis, and right pneumonectomy identify gold code C78.01 despite competing pulmonary hypertension, infection, and embolism codes.The required interpretation is secondary malignant neoplasm of the right lung.
- Method comparison: Direct and decomposed retrieval miss pulmonary metastasis from the top 200, whereas the separately run FHS −verifier arm ranks it at 49; verification rejects candidates conflicting with named dimensions.For gastric reflux, methods differ in whether final selectors recover the gold code even when they retrieve the correct family.