CARA: Causally Guided Retrieval Agent for Question Answering

• Text-rich knowledge-graph retrieval must balance breadth-oriented node search with depth-oriented traversal, a longstanding challenge.Source: §1 · p. 1
• Existing agentic retrievers primarily use textual similarity and graph structure, which can favor query-associated evidence that is unnecessary for answering.A clinical note listing medicines may match a question about what to avoid, while an allergy record can provide the needed evidence despite little textual overlap.Source: §1 · p. 1
• CARA defines causal relevance through consequentiality—whether adding evidence materially increases answerability—and essentiality—whether removing it materially reduces answerability.Source: §1 · p. 1
• CARA’s causal relevance concerns evidence contributions to answering, not only literal cause-effect relations.Source: §1 · p. 1
• CARA applies a training-free, prompt-based module to assess both the current hop and accumulated path, guiding whether to continue hopping, re-anchor globally, or finish.Source: §1 · p. 2
• The paper augments STaRK with ground-truth answers, extending its evaluation from retrieval alone to retrieval and question answering.Source: §1 · p. 2

2 Related Work

• Related work spans knowledge-graph retrieval, causal reasoning, and agentic retrieval.Source: §2 · p. 2

2.1 Knowledge Graph Retrieval for LLM Question Answering

• Knowledge-graph retrieval combines search with graph structure, but these approaches primarily assess textual, relational, or structural compatibility rather than whether evidence is consequential or essential to answering.Source: §2.1 · p. 2

2.2 Causal Reasoning for LLM Question Answering

• Prior causal retrieval methods often associate causality with cause-effect relations, dependencies, or mechanisms, whereas CARA defines relevance by evidence’s contribution to answering rather than causal structure in the described world.Source: §2.2 · p. 2

2.3 Agentic Retrieval for LLM Question Answering

• Agentic graph retrieval makes search adaptive, but existing approaches primarily rely on textual similarity; CARA introduces retrieval based on whether evidence presence and absence contribute to resolving the information need.Figure 1 depicts agents using GlobalSearch and Neighbors, with causal scores guiding their next action and result lists combined into one ranking.Source: §2.3 · p. 2, Figure 1 · p. 3

3 Method: CARA over Knowledge Graph

• CARA represents retrieval as an agent interacting with a knowledge graph through retrieval tools and an embedded causal assessment module.The agent can append retrieved nodes, continue searching, or finish when it reaches its step limit or chooses to stop.Source: §3 · p. 3

3.1 Retrieval Tools

• GlobalSearch finds seed nodes, while Neighbors explores one-hop candidates; the agent can continue hopping, re-anchor globally, or finish based on accumulated evidence.GlobalSearch uses BM25 over node text attributes, and neighborhood traversal can repeat to form multi-hop paths.Source: §3.1 · pp. 3–4

3.2 Causal Reasoning Module

• The causal module assesses candidates for consequentiality and essentiality, alongside path consistency, to guide subsequent retrieval decisions.Consequentiality assesses whether adding evidence advances the answer; essentiality asks whether removing it would reduce the ability to justify an answer.Source: §3.2 · p. 4

3.3 Aggregation of Each Agent Result

• Multiple agents explore independently, and reciprocal rank fusion rewards high within-agent ranks and evidence found across trajectories.The fused candidate set is formed from the union of nodes retrieved by the agents and ordered by fusion score.Source: §3.3 · p. 5

4 Experiment Setup

• The evaluation covers both entity retrieval and downstream question answering.Source: §4 · p. 5

4.1 STaRK and Ground-Truth Answer Augmentation

• STaRK’s AMAZON, MAG, and PRIME datasets provide queries and relevant ground-truth entities, while the study augments them with generated ground-truth answers.Answers are validated for entity-name completeness and manually inspected in stratified samples across the three domains.Source: §4.1 · p. 5

4.2 Baselines

• Baselines include sparse, dense, and knowledge-aware conventional retrievers, plus agentic retrievers Think-on-Graph, GraphFlow, and ARK.The listed agentic systems use a shared Qwen3.8-27B backbone; CARA runs three agents per query.Source: §4.2 · pp. 5–6

4.3 Retrieval Evaluation

• Entity retrieval is evaluated with Hit@1, Recall@20, and mean reciprocal rank, covering first-rank hits, top-20 entity recovery, and ranking of the first correct entity.Source: §4.3 · p. 6

4.4 Question Answering Evaluation

• Question-answering comparisons hold the answer generator, prompt, decoding, and context budget constant, measuring correctness with BERTScore and faithfulness with Ragas.Source: §4.4 · p. 6

5.1 Retrieval Performance

• CARA achieves the highest Hit@1 and MRR across AMAZON, MAG, and PRIME, with especially pronounced improvements over ARK on relation-heavy MAG.The reported configuration uses λ = 0.5 and α = 0.5.Source: §5.1 · p. 6, Table 1 · p. 7

5.2 Question Answering Performance

• CARA achieves the highest BERTScore across all three datasets and consistently improves faithfulness over ARK, although KAR has the highest faithfulness.KAR’s highest faithfulness accompanies substantially lower BERTScore values.Source: §5.2 · p. 6, Table 2 · p. 7

5.3 Impact of Exploration Budget on Performance

• On the STaRK test-0.1 subset, CARA’s retrieval (Recall@20) on AMAZON and PRIME gains only marginally when Tmax increases to 150, suggesting most relevant evidence is found relatively early.With three agents, BERTScore largely stabilizes beyond Tmax = 30; on PRIME, it decreases from 42.1 at Tmax = 30 to 41.6 at Tmax = 150.Source: Figure 2 · p. 7, §5.3 · p. 7

5.4 Ablation Study

• Removing causal reasoning consistently degrades retrieval and question answering, while replacing RRF with count-based aggregation causes smaller performance differences.Count-based aggregation is marginally better on PRIME Hit@1 and MAG and PRIME faithfulness.Source: §5.4 · p. 7, Table 3 · p. 8

5.5 Efficiency and Exploration Behavior

• Compared with ARK, CARA reduces GlobalSearch calls, Neighbors calls, and examined nodes across all three datasets, but causal reasoning adds latency overhead.MAG and PRIME have particularly pronounced overhead, in settings with more relation-intensive questions and more Neighbors calls.Source: §5.5 · p. 8, Table 4 · p. 8

6 Conclusion

• Across three datasets, CARA improves retrieval effectiveness, answer quality, and faithfulness compared with ARK while examining fewer nodes and requiring fewer retrieval operations.Additional causal reasoning introduces latency overhead, particularly for relation-intensive questions.Source: §6 · p. 8
• CARA evaluates retrieved evidence through consequentiality and essentiality, together with path consistency, to guide graph exploration.Reciprocal rank fusion aggregates evidence from multiple independent agents.Source: §6 · p. 8
• The authors augment STaRK with ground-truth answers to enable downstream question-answering evaluation.Source: §6 · p. 8

Limitations

• LLM-based consequentiality and essentiality assessments may be sensitive to model biases, prompting strategies, and errors in judging evidence contributions.Source: Limitations · p. 9
• CARA's causal reasoning module increases computational overhead and inference latency, particularly for relation-intensive questions, despite reducing retrieval operations and examined nodes.Source: Limitations · p. 9
• Downstream question-answering evaluation relies on automatically constructed ground-truth answers and LLM-based faithfulness assessments, which may introduce evaluation noise.Source: Limitations · p. 9

A.1 Implementation Details

• Table 5 lists the implementation settings.Source: §A.1 · p. 12

A.2 Prompts of the Causal Reasoning Module

• The prompt defines causal relevance as evidence whose presence or absence materially affects the ability to support an answer, rather than a domain cause-effect relation.Source: §A.2 · p. 12
• The module assesses candidate nodes returned by Neighbors but does not search the graph or choose the agent's next action.Source: §A.2 · p. 12

• Q denotes the natural-language query the agent is answering.Source: §1 · p. 12

2. Accumulated paths Pt+1

• The accumulated path records the nodes traversed so far, ending at the current node.This path provides the existing evidence against which candidate neighbors are assessed.Source: §2 · p. 12

3. Current node vt

• Candidates are scored independently for consequentiality, essentiality, and path consistency, using the query, existing paths, relations, and candidate descriptions.Textual similarity and adjacency alone are not evidence; every candidate must receive scores, including irrelevant ones.Source: §3 · p. 12

2. Ground-Truth Products

• Amazon answers must use only supplied ground-truth products, preserve their complete names, mention all products, and avoid unsupported product claims or rankings.The prompt requires a concise, coherent single-paragraph answer rather than a list.Source: §2 · p. 13

2. Ground-Truth Papers

• MAG answers must use only supplied ground-truth papers, preserve complete titles, mention every paper, and avoid unsupported claims or comparisons.The prompt also forbids external knowledge and requires factual claims to be supported by the provided paper information.Source: §2 · pp. 13–14

2. Ground-Truth Medical Concepts

• PRIME answers must use only supplied ground-truth medical concepts, mention each concept, and avoid unsupported medical claims or personalized advice.Concept names must be reproduced exactly, and the prompt prohibits external knowledge and unsupported inferences.Source: §2 · pp. 14–15

Case 1: Amazon (test query 116)

• The Amazon case presents a scuba-weight query, identifies Sea Pearls Vinyl Coated Lace Thru Weights as ground truth, and reports methods’ final rankings.The ranking context compares CARA’s RRF with ARK’s count-based aggregation.Source: Case 1: Amazon (test query 116) · p. 15

Case 2: MAG (test query 2057)

• The MAG case introduces a query about publications on nanocantilevers and nanoscale magnetism and identifies a ground-truth paper.The cited passage names “Room temperature magnetometry of an individual iron filled carbon nanotube acting as nanocantilever” as the ground truth.Source: Case 2: MAG (test query 2057) · p. 15
• The MAG example identifies the ground-truth paper as “Room temperature magnetometry of an individual iron filled carbon nanotube” and reports final-ranking positions as none, none, none, 1, 14, 1.The supplied passages do not identify which methods correspond to each position.Source: Case 2: MAG (test query 2057) · p. 15

Case 3: Prime (test query 6548)

• For the EPS8L2 interaction query, ACTG1 is the ground-truth entity; the reported final-ranking positions are 7, none, 1, 1, 1, 6.The listing also names HNF4A, LRRK2, ROR1, PTK2, and deafness, autosomal recessive; method-to-entry mappings are not supplied.Source: Case 3: Prime (test query 6548) · p. 16