PaperlayerView exact revision on arXiv

Source-linked AI summary

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph

Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang

arXiv:2009.11692v1cs.CL

TL;DR

Pre-trained language models can struggle with generation that requires commonsense knowledge absent from the context, and triple-based approaches may ignore graph connections. GRF performs dynamic multi-hop reasoning over external commonsense paths; the paper reports better performance than existing approaches on three text generation tasks and presents inferred paths as rationale for generated results.

  • Problem

    Pre-trained language models struggle with generation tasks that require commonsense knowledge not explicitly stated in the context, while triple-based integration can ignore rich knowledge-graph connections.

    Source: §1 · pp. 1–2

  • Method

    GRF encodes an external commonsense graph and dynamically aggregates triple evidence along multi-relational paths to generate salient concepts under the context.

    Source: §1 · p. 2

  • Results

    GRF outperforms existing approaches on three commonsense-aware text generation tasks, and inferred reasoning paths provide rationale for generated results.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The paper presents multi-hop reasoning over structural and semantic commonsense knowledge as an approach to knowledge-enriched generation, with inferred paths offering rationale for generated results.

    Source: §5 · p. 9

Abstract

from arXiv · show

Despite the success of generative pre-trained language models on a series of text generation tasks, they still suffer in cases where reasoning over underlying commonsense knowledge is required during generation. Existing approaches that integrate commonsense knowledge into generative pre-trained language models simply transfer relational knowledge by post-training on individual knowledge triples while ignoring rich connections within the knowledge graph. We argue that exploiting both the structural and semantic information of the knowledge graph facilitates commonsense-aware text generation. In this paper, we propose Generation with Multi-Hop Reasoning Flow (GRF) that enables pre-trained models with dynamic multi-hop reasoning on multi-relational paths extracted from the external commonsense knowledge graph. We empirically show that our model outperforms existing baselines on three text generation tasks that require reasoning over commonsense knowledge. We also demonstrate the effectiveness of the dynamic multi-hop reasoning module with reasoning paths inferred by the model that provide rationale to the generation.

1 Introduction

Pre-trained language models struggle when generation requires commonsense knowledge not stated in the context, while prior approaches can miss the knowledge graph’s structural connections. The introduction proposes GRF, which reasons over relational paths and combines graph concepts with vocabulary generation.

1 Introduction

  • The motivation is that commonsense knowledge needed for generation may not be explicitly stated in the input context.The example uses external relational knowledge to guide key concepts in a story ending.Source: §1 · p. 1
  • Prior triple-based post-training can leave models unable to use encoded knowledge during generation fine-tuning and ignores structural relations that could provide multiple evidence chains.The introduction argues for exploiting both structural and semantic graph information for grounded generation.Source: §1 · p. 2
  • GRF performs dynamic multi-hop reasoning over relational paths in an external commonsense graph to generate salient concepts under the context.It encodes graph concepts and relations, aggregates triple evidence along paths, then gates between copying graph concepts and choosing vocabulary words.Source: §1 · p. 2
  • The paper evaluates GRF on story ending, abductive natural language, and explanation generation, reporting that it outperforms strong baselines on these tasks.The experiments include automatic and human evaluation; inferred reasoning paths are visualized as rationale for generated results.Source: §1 · p. 2

2 Related Work

Related work covers methods that add commonsense knowledge to neural text generation and methods that reason over graph structure. The paper positions its approach as using general commonsense knowledge during generation rather than knowledge from a specific domain.

2.1 Commonsense-Aware Neural Text Generation

  • Prior pre-trained generation approaches transfer commonsense knowledge through post-training on synthetic triple-to-text data or by transferring COMeT embeddings into GPT-2.The paper contrasts these approaches with using structural and semantic graph information during generation.Source: §2.1 · p. 2

2.2 Multi-Hop Reasoning on Graph Structure

  • Earlier multi-hop graph reasoning work addresses query answering, question answering, and dialogue, while this paper targets general commonsense evidence for text generation.The cited dialogue approaches model graph hops with a Markov transition matrix or select graph information in a two-stage architecture.Source: §2.2 · pp. 2–3

3 Methodology

GRF formulates commonsense-aware generation over input-grounded subgraphs, encoding graph and text context before dynamically propagating evidence along multi-hop paths. A gate combines concept and vocabulary distributions, while auxiliary losses support training.

3.1 Problem Formulation

  • Generation selects an output sequence conditioned on the input text and an extracted subgraph of inter-connected H-hop paths starting from input concepts.The formulation considers concepts with 1-gram surface texts.Source: §3.1 · p. 3

3.2.1 Static Multi-Relational Graph Encoding

  • A static graph encoder aggregates neighboring node and relation information, using a compositional operation to encode multi-relational graph context for later reasoning.The operation combines node and relation embeddings by subtraction, and the resulting node and relation embeddings support dynamic reasoning during decoding.Source: §3.2.1 · pp. 3–4

3.2.2 Context Modeling with Pre-Trained Transformer

  • GPT-2 models the concatenated source and target sequence, and its contextual hidden state at each time step feeds the multi-hop reasoning module.The input sequence includes the source, a beginning symbol, and target tokens.Source: §3.2.2 · p. 4

3.2.3 Dynamic Multi-Hop Reasoning Flow

  • At each decoding step, dynamic reasoning propagates evidence through graph paths, scoring unvisited nodes from visited neighbors and their connecting edges.A discount factor controls information flow from previous hops, while the main results use max aggregation; mean aggregation is used in an ablation.Source: §3.2.3 · p. 4

3.2.4 Generation Distribution with Gate Control

  • A soft gate linearly combines the concept distribution from reasoning with the standard-vocabulary distribution to produce the final generation distribution.The gate controls the relative weights of the two distributions.Source: §3.2.4 · pp. 4–5

3.3 Training and Inference

  • Training minimizes target-sequence negative log-likelihood and adds gate and weak-supervision losses; inference generates tokens until the ending symbol is produced.Weak supervision matches predicted triple relevance to heuristic edge labels from breadth-first search between source concepts and target concepts.Source: §3.3 · p. 5

4 Experiments

The experiments cover three commonsense-aware generation tasks and use ConceptNet subgraphs built from input concepts as knowledge grounding. The subgraphs expand through connected neighbors, prioritizing frequently connected candidate concepts.

4.1 Datasets and Metrics

  • SEG generates endings for four-sentence ROCStories contexts, αNLG generates hypotheses linking a cause and consequence, and EG explains counterfactual statements.Table 1 reports dataset statistics, counting examples with multiple references separately.Source: §4.1 · p. 5, Table 1 · p. 6
  • Abductive NLG and explanation generation use BLEU-4, CIDEr, ROUGE-L, and METEOR; story endings use BLEU-1/2 and Distinct-n.Source: §4.1 · p. 5

4.2 Extracting Sub-Graphs as Knowledge Grounding

  • ConceptNet subgraphs start from lemmatized, fuzzy-matched input concepts, restricted to verbs and nouns to reduce noise.The graph groups 42 relation types into 17; the passage also states that reversed links are added.Source: §4.2 · p. 5
  • For H hops, extraction expands the current graph with direct neighbors and keeps top-B candidates by incoming degree.Incoming degree counts current-subgraph nodes directly connected to a candidate; this prioritizes commonly visited concepts.Source: §4.2 · p. 5

4.3 Implementation Details

The implementation uses GPT-2 with a two-layer GCN and extracts two-hop subgraphs with up to 100 nodes per hop. Comparisons include general language-model baselines and task-specific systems.

4.3 Implementation Details

  • Table 2 summarizes average concept and triple counts per extracted training subgraph across the three datasets.Source: Table 2 · p. 6
  • The main configuration uses small GPT-2, a two-layer GCN, max aggregation, and subgraphs capped at two hops and 100 nodes per hop.The authors say this extraction setting balances graph coverage and noise; max aggregation performed better than alternatives.Source: §4.3 · p. 6

4.4 Compared Baselines

  • Baselines include GRU Seq2Seq with attention and copying, GPT-2 fine-tuning, and GPT-2 post-trained on OMCS before task fine-tuning.The copying mechanism supports generating out-of-vocabulary words.Source: §4.4 · p. 6
  • Task-specific comparisons use IE+GA and WriterForcing for story endings, and COMeT-Txt-GPT2 and COMeT-Emb-GPT2 for αNLG.Source: §4.4 · p. 6

4.5 Automatic Evaluation

GRF outperforms the compared baselines on the automatic evaluations across EG, αNLG, and story ending generation. Commonsense post-training alone degrades performance on EG and αNLG, while it improves story-ending diversity.

  • Simply post-training on a commonsense source degrades performance on EG and αNLG.The authors suggest triple-level post-training may not provide rich enough semantics for reasoning and explanation tasks.Source: §4.5 · p. 6
  • Commonsense post-training improves story-ending diversity, which the authors suspect reflects generation of concepts related to the story context.Source: §4.5 · pp. 6–7
  • GRF outperforms all baselines on every reported automatic metric for EG and αNLG, and on BLEU and distinct metrics for SEG.Source: §4.5 · p. 6, Table 3 · p. 7, Table 4 · p. 7

4.6 Human Evaluation

Human evaluators compared generated texts for fluency and reasonability, with reasonability judged against each task's context. GRF significantly outperforms the compared baselines on both criteria across all datasets.

  • Three annotators judged 100 test sentences per model pair for fluency and reasonability, using task-specific context criteria.The study used 1,100 sentences drawn from five models.Source: §4.6 · p. 7
  • GRF significantly outperforms the compared baselines on fluency and reasonability across all datasets.Five of six inter-rater agreement results were moderate or good; SEG fluency agreement was κ = 0.315.Source: §4.6 · p. 7
  • SEG fluency consensus was low (κ = 0.315), which the authors attribute to GPT-2 baselines already producing grammatical endings.Source: §4.6 · p. 7

4.7 Ablation Study

The ablation study tests model components and finds that removing dynamic multi-hop reasoning produces the largest performance drop. Removing graph representations also degrades performance, and mean aggregation performs worse than max aggregation.

4.7 Ablation Study

  • Removing DMRF causes the largest performance drop, indicating that dynamic multi-hop reasoning plays a major role in this task.The study also finds that all tested components contribute to final performance.Source: §4.7 · p. 7
  • Removing SMGE degrades performance because its relational graph representations benefit concept selection.Source: §4.7 · p. 7
  • The mean aggregator performs worse than the max aggregator in the reasoning module.Source: §4.7 · p. 7

4.8 Impact of the Size of Training Data

On αNLG, GRF maintains performance gains over selected baselines across training-data sizes. The reasoning analysis examines the discount factor and illustrates how evidence from multiple hops contributes to concept selection.

4.8 Impact of the Size of Training Data

  • GRF achieves consistent performance gains over GPT-2 fine-tuning and commonsense-enhanced GPT-2 across different αNLG training-data sizes.The comparison is evaluated on the original test set.Source: §4.8 · p. 8

4.9 Effectiveness of Dynamic Multi-Hop Reasoning

  • Maximum performance occurs when γ is around 0.4 and 0.6; the authors set γ = 0.5 for the main results.Values approaching 0 ignore path-accumulated evidence, while values approaching 1 make node scores increase monotonically along paths.Source: §4.9 · p. 8, Figure 4 · p. 8
  • In the illustrated αNLG case, second-hop evidence leads the module to select “play” (gt = 0.64) as more reasonable given both observations.The first hop begins from “adopt” and “puppy” and favors neighboring verbs in the generated context.Source: §4.9 · p. 8

4.10 Case Study

The case study reports that GRF generates more specific concepts by exploring plausible relations between concepts, and illustrates examples of relation-based choices and narrative transitions.

4.10 Case Study

  • GRF generates more specific concepts than baseline models, including “expensive” as the antonym of “cheap” in a story context.Source: §4.10 · pp. 8–9
  • In one case, GRF generates “seasick” as a plausible explanation for a transition from “cruise” to “beach” that baseline models sometimes miss.Source: §4.10 · p. 9
  • In the last two explanation cases, GRF generates appropriate attributes of source concepts using external commonsense knowledge.Source: §4.10 · p. 9

5 Conclusion

The conclusion presents GRF as a generation model that reasons over structured commonsense knowledge using dynamic multi-hop paths. It reports better performance than existing approaches on three generation tasks and inferred paths that provide rationales for generated results.

5 Conclusion

  • GRF performs dynamic multi-hop reasoning on external knowledge-base paths, leveraging structural and semantic information during text generation.Source: §5 · p. 9
  • GRF outperforms existing approaches that integrate commonsense knowledge into pre-trained language models on three text-generation tasks.Source: §5 · p. 9
  • Inferred reasoning paths provide rationales for the model’s generated results.Source: §5 · p. 9
  • Table 8 presents test-set cases from three datasets, marking source concepts and associated concepts generated by GRF.Source: Table 8 · p. 9

Related papers

Loading 2009.11692v1…