PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Large Language Models Latently Perform Multi-Hop Reasoning?

Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, Sebastian Riedel

arXiv:2402.16837v2cs.CL

TL;DR

The paper asks whether LLMs latently perform multi-hop reasoning when the intermediate information is not explicitly provided. It evaluates bridge-entity recall and subsequent attribute use across a broad prompt dataset, finding strong but highly contextual evidence, with first-hop scaling by model size but not second-hop scaling.

  • Problem

    The paper asks whether LLMs retrieve implicit factual links and traverse them during two-hop completion rather than relying only on explicitly provided information.

  • Method

    The authors analyze bridge-entity recall and downstream attribute utilization using two proxy metrics, a causal-effect frequency framework, and the 45,595-prompt TWOHOPFACT dataset.

  • Results

    Evidence is strong for some fact-composition types, exceeding 80% pathway use, but aggregate second-hop and full-traversal evidence is moderate; first-hop evidence scales with model size while second-hop evidence does not.

  • Takeaways & Limitations

    Latent multi-hop reasoning exists in LLMs but is highly contextual, and the observed pathway effects may understate the models’ broader ability.

  • Takeaways & Limitations

    The study tracks one pathway and changes at a single layer rather than measuring multi-hop reasoning end to end, so observed effects may be a lower bound.

Abstract

from arXiv · show

We study whether Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of 'Superstition' is". We look for evidence of a latent reasoning pathway where an LLM (1) latently identifies "the singer of 'Superstition'" as Stevie Wonder, the bridge entity, and (2) uses its knowledge of Stevie Wonder's mother to complete the prompt. We analyze these two hops individually and consider their co-occurrence as indicative of latent multi-hop reasoning. For the first hop, we test if changing the prompt to indirectly mention the bridge entity instead of any other entity increases the LLM's internal recall of the bridge entity. For the second hop, we test if increasing this recall causes the LLM to better utilize what it knows about the bridge entity. We find strong evidence of latent multi-hop reasoning for the prompts of certain relation types, with the reasoning pathway used in more than 80% of the prompts. However, the utilization is highly contextual, varying across different types of prompts. Also, on average, the evidence for the second hop and the full multi-hop traversal is rather moderate and only substantial for the first hop. Moreover, we find a clear scaling trend with increasing model size for the first hop of reasoning but not for the second hop. Our experimental findings suggest potential challenges and opportunities for future development and applications of LLMs.

1 Introduction

This work asks whether LLMs latently traverse two-hop factual relations and develops measurements to test the bridge-entity recall and attribute-use steps. It finds substantial first-hop evidence, while second-hop and full-traversal evidence are weaker and highly contextual.

  • Research question: The study tests whether LLMs identify an implicit bridge entity and then use knowledge about that entity to answer two-hop prompts.The example asks whether models infer Stevie Wonder from “the singer of ‘Superstition’” and then retrieve his mother.
  • Findings: About 70% of descriptive prompt changes increase bridge-entity recall in later transformer layers.This is the strongest aggregate evidence reported for the latent reasoning pathway.
  • Findings: 60% of recall-increasing cases also show higher consistency, while both hops co-occur in about 40% versus a random 25% baseline.The full traversal is therefore less consistently evidenced than the first hop.
  • Findings: Up to 23% of fact-composition types show strong evidence occurring in more than 80% of cases, indicating highly contextual pathway use.Other composition types show weaker aggregate evidence.
  • Contributions: TWOHOPFACT contains 45,595 two- and one-hop prompts spanning 52 fact-composition types.The dataset is constructed from diverse entities, relations, and templates.
  • Contributions: The framework introduces internal entity recall and consistency scores as proxies for bridge-entity recall and use of its attribute knowledge.It also measures relative frequencies of expected causal effects to detect latent pathways that may not dominate predictions.

2 Related Works

Prior work establishes strong explicit in-context reasoning but reports failures on latent multi-hop reasoning and limited study of its internal pathways. This paper addresses that gap while connecting the problem to model editing and consistency research.

  • Reasoning research: LLMs show strong in-context reasoning when the needed information is explicitly included, but often fail latent multi-hop reasoning even when they know each single-hop answer.The contrast motivates studying reasoning from information stored in parameters rather than supplied in the prompt.
  • Reasoning research: Latent reasoning research has mainly examined simple single-hop tasks, controlled training, or the most salient pathways in synthetic tasks and toy models.The paper instead investigates a latent pathway during complex two-hop prompting.
  • Model editing: Existing model-editing approaches focused on single-fact edits often fail to propagate changes to dependent facts.This makes latent traversal relevant to understanding whether edits can affect compositional knowledge.
  • Consistency: Prior consistency work primarily quantifies or improves inconsistency post hoc, whereas this work investigates a pathway affecting consistency during inference.The paper studies whether recalling the latent hop changes answer consistency.

3 Problem Formulation

The paper formalizes two-hop facts as compositions linked by a bridge entity and studies latent reasoning through internal recall and downstream knowledge utilization. It separates first-hop bridge recall from second-hop use of that recall.

  • 3.1 Preliminaries: A fact is represented as a triplet (e, r, e′), with the relation treated as a function e′ = r(e).The analysis focuses on cases where the object is the only or best-known answer for the relation.
  • 3.1 Preliminaries: Two-hop composition uses a bridge entity e2 to connect ((e1, r1, e2), (e2, r2, e3)) as r2(r1(e1)).For the example, Stevie Wonder connects the singer relation to the mother relation.
  • 3.1 Preliminaries: Templates convert one-hop r2(e2) and two-hop r2(r1(e1)) expressions into prompts with the same answer but different reasoning requirements.The two-hop prompt requires recalling two facts rather than one.
  • 3.1 Preliminaries: The descriptive mention is the noun phrase that refers to the bridge entity through e1 and r1, such as “the singer of ‘Superstition’” for Stevie Wonder.Fact-composition types name the downstream relation and the type of this descriptive mention.
  • 3.2 Latent Multi-Hop Reasoning in LLMs: The paper investigates whether pretrained Transformers use a latent multi-hop pathway alongside shallower or other inference pathways.Because predictions may aggregate multiple distributed pathways, the analysis targets internal dynamics rather than only the dominant output pathway.
  • 3.2 Latent Multi-Hop Reasoning in LLMs: RQ1 measures how often altering a descriptive mention increases internal recall of its bridge entity.For example, replacing “Thriller” with “Superstition” tests recall of Stevie Wonder.
  • 3.2 Latent Multi-Hop Reasoning in LLMs: RQ2 measures how often enhancing bridge-entity recall improves use of the entity’s attribute knowledge in the two-hop answer.This operationalizes second-hop reasoning as utilization of first-hop recall.
  • 3.2 Latent Multi-Hop Reasoning in LLMs: Together, the two questions aim to identify evidence that LLMs leverage a latent pathway for multi-hop reasoning.The paper treats the two hops individually before assessing their connection.

4 TWOHOPFACT Dataset

TWOHOPFACT is a Wikidata-based dataset designed to evaluate latent two-hop reasoning across varied fact compositions. It pairs one-hop and two-hop prompts built from the same underlying fact pairs.

  • Dataset construction: TWOHOPFACT contains 45,595 unique pairs of one-hop and two-hop prompts spanning 52 fact-composition types.The pairs are constructed from corresponding fact-triplet pairs.
  • Dataset construction: The dataset uses well-known named entities from Wikidata and manually selected relations.This supports evaluation across diverse fact-composition settings.
  • Dataset organization: The prompts are organized by fact-composition type, with example two-hop prompts provided for each type.Additional data statistics are reported in an appendix.

5 First Hop of Multi-Hop Reasoning

The first hop is evaluated as internal recall of the bridge entity, asking whether indirect prompt substitutions increase that recall. Across models and fact-composition types, the evidence is substantial and generally strengthens with model size.

  • Internal Entity Recall Score: ENTREC measures bridge-entity recall by projecting the hidden representation at the descriptive mention onto the vocabulary and scoring the entity’s first token.Higher ENTREC is interpreted as stronger internal recall at a given Transformer layer.
  • Experiment: RQ1 tests whether entity or relation substitution increases bridge-entity recall relative to a prompt without its descriptive mention.A relative frequency above 0.5 indicates performance above random chance.
  • Results: 0.71 is the LLaMA-2 7B peak for entity substitution, while relation substitution peaks at 0.63 in layer 20.Entity-substitution evidence becomes clearer with depth, whereas relation substitution is somewhat noisier.
  • Results: The maximum relative frequency rises from 0.71 to 0.72 to 0.78 for entity substitution and from 0.63 to 0.64 to 0.76 for relation substitution as models scale from 7B to 13B to 70B.This establishes a clear model-size scaling trend for the first hop.
  • Results: Up to 73% of fact-composition types show relatively strong first-hop evidence, with maximum relative frequencies exceeding 0.8.Strong evidence is observed for 18/25/34 types under entity substitution and 21/27/38 under relation substitution across 7B/13B/70B models.

6 Second Hop of Multi-Hop Reasoning

The second hop is tested as whether stronger bridge-entity recall makes two-hop outputs more consistent with corresponding one-hop outputs. Evidence is moderate, remains near-stable with model scaling, and varies across fact-composition types.

  • Consistency Score: Consistency compares the output distributions for a two-hop prompt and its corresponding one-hop bridge-entity prompt.The score uses symmetric averaged cross-entropy to measure distributional similarity.
  • Experiment: The experiment increases bridge-entity recall by modifying the shared hidden representation and tests whether consistency increases as a result.The representation is perturbed in the steepest direction of increasing ENTREC, then the consistency derivative is evaluated.
  • Results: 0.64 is the peak LLaMA-2 7B relative frequency for cases where stronger bridge-entity recall increases consistency.Middle and late layers exceed the 0.5 random baseline with statistical significance, peaking at layer 30.
  • Results: The second-hop maximum relative frequency remains relatively stable at 0.64, 0.65, and 0.61 for 7B, 13B, and 70B models.This scaling pattern differs from the first hop and remains unchanged when using ground-truth log probability instead of CNSTSCORE.
  • Results: Up to 19% of fact-composition types show relatively strong second-hop evidence, with maximum relative frequencies exceeding 0.8.Two types demonstrate strong evidence across all model sizes: founder of person’s undergraduate university and president of anthem’s country.

7 Latent Multi-Hop Reasoning

Latent multi-hop reasoning is identified when both first-hop recall and second-hop consistency succeed for the same prompt. The evidence is moderate overall, with some relation-substitution scaling and substantial variation across fact-composition types.

  • Latent Multi-Hop Reasoning: The analysis combines RQ1 and RQ2, treating success in both as evidence of latent multi-hop reasoning.SS denotes success in both hops; FS, SF, and FF represent the other combinations.
  • Results: 0.46 for entity substitution and 0.38 for relation substitution are the LLaMA-2 7B peaks for successful multi-hop reasoning, both above the 0.25 random baseline.The likelihood of partial multi-hop reasoning exceeds 0.8 in later layers.
  • Results: Relation substitution increases the maximum successful multi-hop frequency from 0.38 at 7B to 0.43 at 70B, while entity substitution shows no increase with model size.Thus, scaling appears dependent on the type of substitution used for the first hop.
  • Results: Up to 23% of fact-composition types show relatively strong latent multi-hop evidence under the stated threshold.For the largest model, 12 types exceed the threshold with entity substitution and 9 with relation substitution.
  • Results: The anthem-of-capital’s-country composition exceeds the strong-evidence threshold across all models and both substitution types, with maxima of 0.68/0.82/0.66 and 0.74/0.82/0.68.Individual composition types exhibit patterns distinct from the overall dataset.

8 Discussion and Conclusion

The study finds that latent multi-hop reasoning is strongly context-dependent: some fact-composition types show substantial pathway use, while average second-hop and full-traversal evidence remains moderate. First-hop reasoning scales with model size, unlike second-hop reasoning, motivating further study of training and architectures.

  • Discussion and Conclusion: More than 80% of prompts use the latent reasoning pathway for certain fact-composition types, but utilization is highly contextual.Other fact-composition types show weak or almost no evidence of reasoning.
  • Discussion and Conclusion: The evidence is substantial for the first hop but moderate for second-hop reasoning and complete multi-hop traversal across prompts.The first hop identifies the bridge entity, while the second hop uses knowledge about its attribute.
  • Discussion and Conclusion: A clear scaling trend appears for first-hop reasoning with increasing model size, whereas no such trend appears for second-hop reasoning.This pattern may relate to the reported compositionality gap that does not decrease as model size increases.
  • Discussion and Conclusion: The findings suggest limitations in current scaling approaches for promoting latent multi-hop reasoning.The authors identify pretraining data, loss functions, and architectural inductive biases as areas for future study.
  • Discussion and Conclusion: Studying prompts with strong multi-hop evidence may illuminate how these abilities emerge under existing pretraining and scaling paradigms.The broader findings can guide work on strengthening latent reasoning relevant to parameter efficiency, generalization, and controllability.

9 Limitations

The study’s scope and measurements impose important boundaries: it uses a specific LLaMA-2-based dataset construction and examines only one indirect pathway with proxy metrics. Dataset ambiguity, temporal change, and metric approximation further constrain interpretation.

  • Latent Multi-Hop Reasoning Pathway: The analysis tracks one latent reasoning pathway rather than measuring multi-hop reasoning end-to-end.Other redundant inference pathways may retrieve the same information differently, and first-hop effects may propagate across layers.
  • Dataset: TWOHOPFACT may contain Wikidata noise because selected relations are not always uniquely satisfied in changing real-world knowledge.The dataset aims for a single or most famous relation object, but strict uniqueness is difficult to maintain.
  • Metrics: The internal entity recall metric is approximate because it uses only the entity’s first token and relies on logit lens representations.The authors note potential representation drift, bias, and brittleness in logit lens, while focusing on internal dynamics rather than early-layer prediction accuracy.
  • Natural Language Templates: The prompt templates pair descriptive bridge-entity mentions with relation-specific one-hop and two-hop queries.For example, the mother relation produces both “The mother of Stevie Wonder is” and “The mother of the singer of ‘Superstition’ is”.

B Dataset Statistics

TWOHOPFACT contains 45,595 two-hop prompts spanning 52 fact-composition types, with relatively balanced coverage across types. Its figures characterize descriptive bridge-entity mentions, composition types, and majority entities.

  • Dataset composition: 45,595 two-hop prompts cover 52 fact-composition types in TWOHOPFACT.The dataset is built from unique pairs of fact triplets and translated into one-hop and two-hop prompts.
  • Dataset composition: The largest fact-composition type accounts for 7.41% of the dataset, indicating relatively balanced type coverage.The largest category is “birth city of novel’s author.”
  • Entity distributions: Figure 7 reports the percentages of majority bridge entities and majority answer entities for each fact-composition type.A majority bridge entity is the bridge entity used most often for constructing the corresponding one-hop prompts.
  • Dataset composition: Figure 6 reports inner-ring percentages for descriptive bridge-entity mentions and outer-ring percentages for fact-composition types.Composition annotations show only the type(r2) of component, while expanded abbreviations appear in Table 4.
  • Dataset composition: Each fact-composition type contains at least 30 prompts to support statistically significant results.Table 3 lists the prompt counts and examples for each type.

C Justification of Internal Entity Recall Score: Appositive Generation Experiment

The appositive-generation experiment evaluates whether ENTREC reflects latent recall of a bridge entity by testing whether recall changes alter the probability of generating that entity in an appositive position. The results support ENTREC as an indirect proxy for internal bridge-entity recall.

  • Experimental rationale: ENTREC measures bridge-entity recall at the descriptive mention, where the entity name can grammatically appear as an appositive.For example, the prompt can be extended with “, Stevie Wonder” after “the singer of ‘Superstition’.”
  • Appositive-generation test: Increasing ENTREC at one token increases the relative frequency of generating the bridge entity as the next token after a comma.The experiment tests whether stronger recall makes the appositive entity more likely to be generated.
  • Appositive-generation test: In most mid-late layers, increasing latent bridge-entity recall also increases the relative frequency of generating the bridge entity as an appositive.This result provides indirect evidence that ENTREC tracks internal bridge-entity recall.
  • Experimental caveat: The analysis excludes descriptive mentions ending in punctuation patterns for which appending a comma changes LLaMA-2 tokenization.This exclusion is applied to avoid tokenization changes affecting the analysis.
  • Proxy validation: ENTREC at the n-th token shows controllability over the token generated at n + 2, supporting its use as a proxy for internal bridge-entity recall.The measured target is the first token of the appositive followed by a comma.
  • Consistency-score comparison: CNSTSCORE is evaluated by comparing ordinary two-hop prompts with Chain-of-Thought-style prompts that explicitly provide either the bridge entity or its attribute.The comparison distinguishes internal use of one-hop knowledge from copying information given in the input.

F Accuracy-based Analysis for the Second Hop of Multi-Hop Reasoning

The accuracy-based analysis tests whether second-hop evidence persists when one-hop accuracy is filtered and when log probability replaces consistency score. The findings remain similar, with second-hop evidence not strengthening as model size increases.

  • Analysis design: The analysis uses consistency-based results as its primary approach because answer correctness alone has limitations for the research questions.Accuracy-based results are presented as an additional analysis.
  • Log-probability analysis: 17,231 of 45,595 two-hop prompts have corresponding one-hop prompts completed with the ground-truth answer by all three model sizes.This subset is used for the log-probability analysis.
  • Log-probability analysis: 0.60, 0.62, and 0.59 are the corresponding maximum relative frequencies when using ground-truth-answer log probability for 7B, 13B, and 70B models.These values are slightly lower than the consistency-score values.
  • Log-probability analysis: 0.64, 0.65, and 0.62 are the maximum CNSTSCORE relative frequencies for 7B, 13B, and 70B models, respectively.The maxima occur at layers 30, 14, and 40.
  • Scaling result: Second-hop reasoning with ground-truth-answer log probability does not strengthen with increasing model size.The overall trend remains consistent with the consistency-score analysis.
  • Accuracy filtering: Filtering by one-hop accuracy compares prompts with correct and incorrect one-hop completions to test whether second-hop evidence depends on one-hop correctness.The two sets are labeled τ1H correct and τ1H incorrect.
  • Accuracy filtering: The maximum relative frequencies are similar for τ1H-correct and τ1H-incorrect prompts across model sizes.For τ1H correct, the maxima are 0.62, 0.65, and 0.62 for 7B, 13B, and 70B; for τ1H incorrect, the reported maxima begin at 0.65 for 7B and 0.65 for 13B.
Loading 2402.16837v2…