PunGraph: Retrieval-Enhanced Phonetic-Semantic Graph Reasoning for Pun Understanding

1 Introduction

• Small-scale LLMs struggle to capture pun-word phonetic relationships and can generate interpretations outside the intended semantic space.The introduction identifies phonological reasoning deficiency and unconstrained semantic generation as the two core limitations.Source: §1 · p. 2
• PunGraph models phonological associations and semantic relationships in a lexical knowledge graph, then retrieves candidate words or senses to constrain LLM reasoning.The candidate options are intended to address phonological reasoning deficiencies and semantic drift.Source: §1 · p. 2
• WebPun adds a new large-scale pun dataset collected and annotated from public pun websites, supplementing existing resources.The dataset is presented as a way to enrich pun benchmarks with more diverse examples.Source: §1 · p. 2
• Experiments are reported to show consistent gains over strong baselines and competitive performance against large-scale models.Source: §1 · p. 2
• The authors analyze small-scale model limitations in both heterographic and homographic pun reasoning.Source: §1 · p. 2

• Heterographic reasoning seeks a differently meaningful alternative with similar pronunciation, whereas homographic reasoning infers two senses of one word in context.Source: §2 · p. 2
• As the IPA threshold rises, below-threshold heterographic errors increase while above-threshold errors decrease, indicating many generated replacements lack phonetic similarity.Source: Figure 2 · p. 3, §2 · p. 3
• Correct homographic interpretations achieve higher WordNet-definition matching scores than incorrect predictions, whose explanations are less likely to align with intended senses.Source: Figure 3 · p. 3, §2 · p. 3
• For homographic cases, both generated senses count as correct only when each matches dictionary definitions above the predefined threshold.Source: §2 · p. 3

3 PunGraph

• PunGraph builds a lexical graph from pronunciation-dictionary relations and dictionary senses, then uses retrieval to constrain LLM reasoning and improve accuracy and controllability.The graph models phonological relationships and integrates sense definitions to supply candidate interpretations.Source: §3 · p. 4, §3.1 · p. 4

3.1 Pun Graph Construction

• The graph combines Unisyn-derived phonetic relations, IPA similarity and G2P augmentation, with WordNet-derived semantic interpretations for matched words.Word pairs are connected when their IPA-based similarity exceeds a threshold; the resulting graph is stored in Neo4j for retrieval and analysis.Source: §3.1 · p. 4

3.2 Pun reasoning

• For heterographic puns, cases below IPA similarity 0.6 trigger retrieval of near-homophones and related words as explicit candidate options for contextual selection.The threshold focuses enhancement on samples where the authors report a higher proportion of incorrect predictions.Source: §3.2 · pp. 4–5
• For homographic puns, cases below similarity 0.5 trigger retrieval of candidate senses, which the LLM receives as multiple-choice options.The 0.5 threshold is selected because it yields the largest gap between positive and negative cases in dictionary coverage.Source: §3.2 · p. 5

4.1 Data Preparation

• WebPun retains 5,730 samples from 24,880 collected public-site entries after filtering out short, non-English, incomplete, rare-word, and phrasal-pun cases.The dataset is presented as addressing scarcity of high-quality pun-reasoning data.Source: §4.1 · p. 5, §4 · p. 5
• Three LLMs classify samples lacking pun-category annotations, with inconsistent predictions sent for manual review before final labels are assigned.The classifiers are GPT-5.5, Gemini-3.5-flash, and Claude-Opus-4-1.Source: §4.1 · p. 5

4.2 Data Analysis

• WebPun contains 5,061 heterographic and 679 homographic puns; nouns, verbs, and adjectives together constitute 95.9% of its puns.Nouns account for more than half, followed by verbs and adjectives; SemEval-2017 has 92.1% nouns.Source: §4.2 · p. 6
• Compared with existing pun datasets, WebPun strengthens coverage of common noun-based puns and offers richer lexical diversity and semantic ambiguity for LLM evaluation.The paper characterizes this distribution as making WebPun a more challenging and effective benchmark.Source: §4.2 · p. 6

5.1 Dataset

• Experiments use SemEval-2017 and WebPun, and the phonetic-semantic graph contains 46,208 word entities and 760,312 phonetic-similarity relations.The authors set the phonetic-link threshold to 0.75.Source: §5.1 · p. 6

5.2 Baselines

• Baselines span large-scale LLMs, small-scale LLMs, PunIntended, and graph-based methods GCR and ReKG-MCTS.Named large-scale models include GPT-4o, Gemini 2.0 Flash, and DeepSeek-V3.2.Source: §5.2 · p. 6

5.3 Evaluation Metrics

• Heterographic alternatives are evaluated with exact-match precision, recall, and F1, while homographic senses use semantic-similarity accuracy and F1, plus Partial Matching Accuracy.A homographic prediction is correct when both sense similarities exceed the threshold; PMA counts a match when at least one does.Source: §5.3 · p. 6

6.1 Main Results

• PunGraph improves over its backbone across both datasets and pun tasks, remains competitive with proprietary LLMs, and outperforms specialized reasoning baselines.With Qwen-2.5-7B, relative F1 improvements on heterographic puns are 57.95% on SemEval and 30.45% on WebPun; PunGraph-Llama4-Maverick achieves 83.41 F1 on SemEval heterographic puns. It exceeds ReKG-MCTS by 14.44 F1 on SemEval heterographic puns and gains +15.32 and +33.90 F1 on WebPun heterographic and homographic puns, respectively.Source: Table 1 · p. 7, §6.1 · p. 7

6.2 Ablation Study

• Removing Unisyn, IPA, or G2P reduces performance; Unisyn removal causes the largest decline, while G2P removal causes the smallest across tested backbones and datasets.On SemEval-2017 with Qwen2.5-7B, the F1-score drops are 12.42% without Unisyn, 8.61% without IPA-based similarity modeling, and 5.19% without G2P augmentation.Source: Table 4 · p. 8, §6.2 · p. 8

6.3 Error Analysis

• Incorrect selection among retrieved candidates is the main error source; it accounts for 91.5% of homographic-pun failures, while retrieval errors account for 19% and 5.9% of heterographic and homographic failures.The analyzed categories also include invalid output, when the model does not follow the required format.Source: §6.3 · p. 8, Figure 6 · p. 8

7.1 Pun Interpretation

• Semantic methods model word senses and suit homographic puns, while pronunciation-aware methods target latent alternatives needed for heterographic puns; the two operate in different spaces.LLMs may still miss alternative words or phonetic links.Source: §7.1 · pp. 8–9

7.2 Retrieval-Augmented Reasoning

• Graph-based retrieval supplies structured evidence for LLM reasoning, but prior knowledge-graph methods mainly target factual or entity-relation tasks rather than puns’ joint phonological-semantic structure.PunGraph adapts retrieval-augmented reasoning by retrieving candidate words or senses from a task-specific lexical graph.Source: §7.2 · p. 9

• PunGraph integrates phonetic similarity and semantic knowledge in a lexical graph, enabling more controllable and accurate LLM reasoning for heterographic and homographic puns.WebPun contributes 5,730 annotated pun samples; experiments on SemEval-2017 and WebPun show improved small-scale LLM performance and competitive results against strong proprietary models, with retrieval-guided constraints reducing common reasoning errors.Source: §8 · p. 9

• The English-focused framework relies on resources such as Unisyn and WordNet, and graph gaps for rare words, creative expressions, or unseen patterns can cause retrieval failures.Final predictions also depend on correct candidate selection, which remains a major error source.Source: Limitations · p. 9
• WebPun’s online written-pun sources may not fully represent spoken, multimodal, or culturally specific humor.The authors leave multilingual and multimodal extensions of PunGraph for future work.Source: Limitations · p. 9

Ethical Considerations

• The human-participant study received Shanxi University ethics approval, and its procedures followed relevant ethical guidelines and regulations.Source: Ethical Considerations · p. 9

A.1 Classification Prompts

• The classification prompt distinguishes homographic semantic ambiguity from heterographic phonetic similarity and requires a single label and the exact pun word.Source: §A.1 · pp. 11–12

A.2 Data Annotation

• Three closed-source models collaboratively annotate samples; experts review agreement, correct disagreements by majority vote, and obtain Fleiss’ Kappa of 0.54.The preliminary few-shot examples use three homographic and three heterographic SemEval puns; annotations target replacement words or two sense interpretations.Source: §A.2 · p. 12

B.1 Data Statistics

• WebPun has substantially more heterographic samples than SemEval but relatively few additional unique pun words; SemEval retrieval averages 30.55 heterographic candidates and 9.76 homographic sense definitions per pun.Candidate Retrieve Number describes graph retrieval coverage, not downstream reasoning performance; larger candidate sets may add noise and make selection harder.Source: §B.1 · p. 12

B.2 Dataset Sample

• Table 5 presents representative WebPun examples with their pun words, types, and explanations.Source: §B.2 · p. 13

B.3 Dataset Novelty and Knowledge Coverage

• Unique pun-word overlap with SemEval-2017 is 18.99% for heterographic puns and 15.52% for homographic puns, while WebPun adds lexical and contextual diversity.Source: §B.3 · p. 13
• Graph coverage is 99.84% for WebPun heterographic puns and 85.95% for its homographic puns, versus 99.45% and 86.67% on SemEval-2017, respectively.Source: §B.3 · p. 13, Table 8 · p. 13

C Retrieval Strategy

• For heterographic puns, the prompt selects a likely real-word candidate; for homographic puns, it selects two definitions or infers two meanings if none fit.Source: C Retrieval Strategy · pp. 13–14

D Phonetic Similarity

• Phonetic retrieval uses IPA edit distance with articulatory-feature-based phoneme groups, treating phonemes in the same similarity group as equivalent matches.The similarity groups model an undirected relation based on shared articulatory features.Source: D Phonetic Similarity · p. 14, Table 9 · p. 14