Repair Before Reinforce: Context-Augmented Knowledge Graph Reasoning for Multi-Hop Question Answering

1 Introduction

• KGs represent facts as head-relation-tail triples, while prior systems have used them for reasoning and as sources of QA supervision.The paper motivates context augmentation because isolated triples can omit surrounding evidence needed to interpret a relation in its local clinical context.Source: §1 · pp. 1–2
• The framework augments each primary KG triple with supporting triples from the same source chunk, forming context graphs for context-grounded supervision.The paper contrasts QA items grounded in only the target triple or path with items that also include supporting context triples.Source: §1 · p. 2
• The study constructs Gastroparesis and Diabetes KG/CG datasets and generates verifiable multiple-choice QA items spanning 1-hop to 5-hop reasoning.The disease-specific KGs are built using GraphMERT.Source: §1 · p. 2
• An LLM-judged, history-aware adaptive repair stage targets unresolved one-hop failures and removes or quarantines problematic triples before RL.The framework compares RL initialized from repaired and unrepaired SFT checkpoints to examine higher-hop generalization.Source: §1 · pp. 2–3
• Across both disease settings, context-augmented SFT improves multi-hop performance, and RL from repaired checkpoints yields the strongest 3-hop, 4-hop, and 5-hop results.The introduction also reports adaptive repair strengthens the one-hop foundation and tests robustness under answer-option shuffling.Source: §1 · p. 3

2.1 Knowledge Graphs for Question-Answering

• KGs expose structured relational evidence for interpretable question-answering and can support both inference-time reasoning and the construction of training supervision.The section also identifies reliable KG construction as important because extraction errors can propagate into downstream QA.Source: §2.1 · pp. 3–4

2.2 From Isolated Triples to Context-Augmented KG Supervision

• Context augmentation pairs a primary triple with supporting triples that provide local evidence, addressing the interpretive limits of isolated facts.The paper adapts the broader context-graph idea to structured supporting triples.Source: §2.2 · pp. 4–5

2.3 Question-Answer Item Generation from KGs

• KG triples support one-hop QA and connected paths support multi-hop QA, while generated items require checks for validity, option quality, reasoning alignment, and answer correctness.The section motivates verification because synthetic QA can contain ambiguity, leakage, or misalignment with its source path.Source: §2.3 · p. 5

2.4 Supervised Fine-Tuning using Knowledge-Grounded QA Items

• SFT trains a model to produce target answers conditioned on questions and grounding evidence, but aggregate performance may leave individual KG facts unlearned.The section motivates targeted follow-up because average-risk training can leave subsets with high error.Source: §2.4 · p. 5

2.5 Targeted Repair of Missing Knowledge

• The paper calls validation failures on a primary triple or its induced reasoning pattern missing knowledge and proposes auditing failures before targeted repair.The audit separates valid missing-knowledge cases from unreliable examples that could introduce noisy supervision.Source: §2.5 · p. 6

2.6 Reinforcement Learning for Multi-Hop Reasoning

• The section frames RL as a compositional stage after SFT and motivates rewards that combine answer correctness with alignment to the KG path.Prior work cited in the section associates path alignment with a stronger compositional-reasoning signal than answer correctness alone.Source: §2.6 · p. 6

3.1 Overall Pipeline

• The pipeline constructs disease-specific KGs, attaches same-chunk context, generates KG- and CG-grounded QA items, then applies SFT, adaptive repair, and RL.The biomedical adaptation is illustrated for gastroparesis.Source: §3.1 · p. 7, Figure 2 · p. 7

3.2 Disease-Specific Knowledge Graph Construction

• GraphMERT supplies the disease-specific KGs; the diabetes graph is relation-aware sampled to 6,000 triples, yielding 5,954 unique triples after deduplication.The gastroparesis KG is built from biomedical abstracts, while diabetes uses a size-matched subset of its larger GraphMERT KG.Source: §3.2 · pp. 7–8

3.3 Context Graph Construction

• Triples sharing a source-chunk identifier become context candidates, and SapBERT cosine similarity filters them at θ = 0.65 for a two-layer context graph.The lower layer retains primary KG triples; the upper layer contains supporting triples from the same source chunks.Source: §3.3 · pp. 8–9, Figure 3 · p. 9

3.4 QA Item Generation

• QA generation requires questions to follow the sampled path, grounds answers in that path, and balances correct-answer positions while filtering low-quality distractors.CG-grounded items may use supporting context, but the primary path remains the answer-bearing evidence.Source: §3.4 · pp. 10–11
• Generated items receive path-grounded reasoning traces and correctness checks; failed items are regenerated from the same source path rather than dropped.The correctness check assesses the complete item against its KG or CG evidence.Source: §3.4 · pp. 12–13, Figure 4 · p. 13
• Full-coverage one-hop datasets train SFT and validate adaptive repair, while RL uses mixed one- and two-hop items and evaluation tests 3-, 4-, and 5-hop reasoning.The one-hop validation set identifies triples the model still answers incorrectly after SFT.Source: §3.4 · p. 13

3.5 The Supervised Fine-Tuning Setup

• KGModel and CGModel are Qwen3-14B variants trained on KG-grounded and CG-grounded QA items, respectively, with the supervision type as their only training difference.Both variants use the same base model and fine-tuning configuration for each disease setting.Source: §3.5 · p. 14

3.6 LLM-Judged and History-Aware Adaptive Triple Repair

• The pipeline uses an LLM audit to separate invalid triples, flawed validation questions, and genuine missing knowledge, repairing only failures confirmed as valid knowledge with valid questions.Bad questions are regenerated and re-evaluated; invalid triples are removed rather than used as repair supervision.Source: §3.6 · pp. 15–16, Figure 5 · p. 16
• Confirmed missing-knowledge triples receive alternative QA phrasings and reasoning traces from the same source triple, followed by continual fine-tuning and re-evaluation.Repair examples are selected by triple-level error history: persistent triples receive 7, regression triples 5, and new-error triples 3 additional QA examples.Source: §3.6 · pp. 15–16
• After three targeted repair attempts, persistent failures go to an independent second judge, which can identify noisy triples, request question regeneration, or retain valid-but-unlearnable cases separately.Quarantining follows repeated repair and review; it is not used to hide ordinary model errors.Source: §3.6 · pp. 16–17
• The final reported perfect 1-hop accuracy is measured on the cleaned retained validation set, after invalid triples and bad questions are excluded or corrected and valid repairable failures remain.Judge-2-quarantined cases are also excluded; valid-but-unlearnable triples are documented separately.Source: §3.6 · p. 17

3.7 Reinforcement Learning Setup

• GRPO is run from both initial and repaired SFT models, using answer correctness and a path-alignment reward that is gated on a correct final answer.Repaired initialization has 100% 1-hop validation accuracy, excluding quarantined triples when applicable.Source: §3.7 · pp. 19–20
• RL uses a curated 6,000-path manifest of 1-hop and 2-hop examples, yielding approximately 5,500 retained QA items per setting after quality filtering and verification.The manifest contains 1,500 1-hop and 4,500 2-hop paths; 2-hop sampling prioritizes broad node coverage and rare relation pairs.Source: §3.7 · p. 19, Table 4 · p. 19

3.8 Evaluation Setup

• Evaluation measures multiple-choice accuracy on 3-hop, 4-hop, and 5-hop QA for both KG- and CG-grounded sets across Gastroparesis and Diabetes.Training uses full-coverage 1-hop SFT and mixed 1-hop/2-hop RL; the higher-hop tests assess transfer to longer reasoning chains.Source: §3.8 · p. 22
• The design isolates context-augmentation, adaptive-repair, and repaired-RL-initialization effects through comparisons among KGModel, CGModel, their repaired variants, and RL variants.These comparisons test whether CG-grounded SFT helps over KG-grounded SFT, whether repairing unresolved one-hop gaps improves downstream performance, and whether RL benefits from repaired lower-hop initialization.Source: §3.8 · p. 22
• All model variants are evaluated on 3-hop, 4-hop, and 5-hop QA to assess transfer to increasingly longer reasoning chains.The evaluation tests whether improvements from SFT, repair, and RL generalize to these higher-hop tasks.Source: §3.8 · p. 22, Table 6 · p. 22
• Robustness is tested by randomly shuffling all four answer options for each 3-hop, 4-hop, and 5-hop multiple-choice set, then comparing original and shuffled accuracy.The question stem stays unchanged, the correct answer text is preserved with an updated label, and the accuracy change indicates sensitivity to option ordering.Source: §3.8 · pp. 22–23

4.1 Context-Augmented SFT Improves Multi-Hop QA

• CGModel outperforms KGModel across both diseases, evaluation settings, and hop lengths, including KG-grounded tests that provide no context triples.Gastroparesis gains range from 1.98 to 2.20 percentage points on KG-grounded QA and from 2.96 to 3.79 points on CG-grounded QA.Source: §4.1 · p. 23, Table 7 · p. 24

4.2 LLM-Judged Adaptive Repair Produces 100% Accuracy on Cleaned One-Hop Validation Sets

• Adaptive repair reached 100% accuracy across four cleaned retained one-hop validation settings after 9–10 rounds, retaining 5,861–5,943 examples.Initial failures included invalid triples and bad questions as well as missing knowledge; invalid or unlearnable cases were removed from the final denominator, while bad questions were regenerated.Source: Table 8 · p. 24, §4.2 · p. 24

4.3 Repair Before RL Improves Higher-Hop Generalization

• RL initialized from repaired SFT models consistently improves accuracy over original-SFT initialization across diseases, grounding settings, and 3-hop to 5-hop QA.The gains are especially pronounced for repaired CG-based models, which achieve the strongest overall higher-hop performance.Source: Figure 6 · p. 25, Table 10 · p. 26, §4.3 · p. 26

4.4 Full Comparison Across Model Variants

• Across both diseases and evaluation settings, RL from repaired checkpoints yields larger, more consistent gains than RL from unrepaired SFT checkpoints, and CGModel-Repaired-RL achieves the best accuracy on every 3-, 4-, and 5-hop task.Source: §4.4 · p. 26

4.5 Robustness to Option Shuffling

• Across settings, average absolute accuracy changes under answer-option shuffling remain below one percentage point for every model.The stress test randomly reassigned all four answer options to labels A–D under a fixed seed.Source: Table 12 · p. 28, §4.5 · p. 28

5.1 Context-Augmented Supervision Helps

• Context triples provide structured, traceable local evidence neighborhoods that can help models learn and compose relations beyond isolated facts.The authors suggest that richer, clinically meaningful context may underlie CGModel’s advantage over KGModel.Source: §5.1 · p. 29

5.2 Adaptive Repair Is Necessary

• Validation failures can reflect noisy triples, misaligned questions, or genuine knowledge gaps, so indiscriminate fine-tuning risks reinforcing unsupported supervision.The repair pipeline diagnoses failures and targets valid missing-knowledge cases.Source: §5.2 · p. 29

5.3 Repair Before RL Improves Multi-Hop Reasoning

• Gastroparesis CG-grounded QA shows average RL gains of +2.35 points over CGModel-Repaired, versus +0.29 points over CGModel for unrepaired initialization.The discussion attributes the difference to unresolved one-hop errors potentially propagating through longer reasoning paths and limiting what RL can improve.Source: §5.3 · p. 29

5.4 Combined Effect of Context, Repair, and RL

• CGModel-Repaired-RL is consistently the strongest model, with context augmentation, adaptive repair, and repaired-checkpoint RL contributing complementary benefits in the staged pipeline.The proposed sequence is to learn basic facts, repair unresolved reliable facts, and then optimize for compositional reasoning.Source: §5.4 · p. 29

5.5 Robustness to Option Shuffling

• Average absolute accuracy changes remain below one percentage point under option shuffling, suggesting gains are not primarily due to fixed answer-position artifacts.Some repaired RL models nevertheless show small negative shifts after shuffling.Source: §5.5 · p. 30

Limitations and Future Work

• The QA datasets are synthetically generated, so residual ambiguity or generation noise may remain despite multi-stage filtering and verification.The authors suggest expert review of some generated items, especially difficult repaired or quarantined triples.Source: §6 · p. 30
• Adaptive repair relies on LLM judges whose decisions may still reflect model-specific biases.The judges used are Qwen3.6-27B for the initial audit and Qwen3-32B for final quarantine review.Source: §6 · p. 30
• Evaluation covers Gastroparesis and Diabetes, so validation on additional KGs and more diverse relation sets remains needed.Diabetes is an additional setting alongside Gastroparesis, the main disease-specific case study.Source: §6 · p. 30
• Context augmentation is limited to supporting triples extracted from the same source text chunk.Proposed extensions include source passages, clinical records, laboratory findings, imaging reports, treatment histories, and guideline snippets.Source: §6 · p. 30

7 Conclusion

• The framework attaches same-source-chunk supporting triples to primary KG triples and uses the resulting context graphs for structured QA supervision across SFT, adaptive repair, and RL.The work constructs disease-specific KGs for Gastroparesis and Diabetes using GraphMERT.Source: §7 · p. 30
• Across Gastroparesis and Diabetes, CGModel-Repaired-RL consistently achieves the strongest performance, with context augmentation, adaptive repair, and repaired RL initialization providing complementary benefits.The reported findings also say context-augmented SFT improves higher-hop QA over isolated-triple SFT, adaptive repair strengthens the lower-hop foundation, and RL works best from repaired SFT checkpoints.Source: §7 · p. 30
• The conclusion argues that multi-hop reasoning needs reliable lower-hop knowledge, strengthened through context-aware supervision and targeted repair, rather than RL after SFT alone.Future work could extend the framework to additional KGs, stronger judge models, and richer context beyond supporting triples.Source: §7 · pp. 30–31

9.1.1 Paper Retrieval from PubMed Central

• The gastroparesis PMC search excluded COVID-19 and SARS-CoV-2 and returned 41,230 records: 27,102 had OA packages, 13,936 lacked files, and 192 were unresolved.The search targeted gastroparesis-related terms.Source: §9.1.1 · p. 36

9.1.2 Cleaning and Abstract Dataset Construction

• Only English abstracts at least 200 characters long with valid .nxml files were retained; PubMedBERT tokenization used 128-token sequences, splitting longer abstracts.The cleaning process excluded non-English abstracts and missing or broken files.Source: §9.1.2 · p. 36, §9.1.3 · p. 37

9.1.4 Entity Discovery and Seed KG Construction

• Qwen3-32B-FP8 handled head-entity discovery, relation assignment, and tail-token combination, while GraphMERT performed KG completion.Source: §9.1.4 · p. 37

9.1.5 GraphMERT Training

• GraphMERT was trained from scratch on the gastroparesis corpus with an injected seed KG; appendix tables document hyperparameters and stage-specific models.Source: §9.1.5 · p. 38, Table 19 · p. 38, §9.1.7 · p. 39

9.1.6 Prediction-Time Triple Generation and Filtering

• At prediction, Qwen added relations from 20 seed types, GraphMERT predicted tails, and the helper LLM combined them; triples with source similarity > β = 0.65 were retained.Source: §9.1.6 · p. 38

9.1.8 Diabetes KG Subgraph Selection

• The diabetes subset was size-matched rather than uniformly sampled; relation-aware selection preserved rare relations and connectivity, yielding 6,000 sampled rows and 5,954 unique triples.Source: §9.1.8 · p. 39

9.1.9 Relation Frequencies

• The gastroparesis KG has a broader spread of relation types, whereas the diabetes KG is dominated by associated_with, consistent with the original diabetes KG.Source: §9.1.9 · pp. 39–40, Table 22 · p. 40

9.2 Context Augmentation

• Candidate context triples are encoded with SapBERT and retained when their cosine similarity to the primary triple is at least θ.Source: §9.2 · p. 40
• The selected thresholds were θ = 0.65 for Gastroparesis and θ = 0.60 for Diabetes, giving the highest 1-hop accuracy while retaining a moderate number of context triples.Source: §9.2 · p. 41
• Each JSON record stores the primary triple, source chunk, supporting context triples, and similarity scores; QA generation retrieves those triples as local evidence.Source: §9.2 · p. 42
• For path sampling, enumeration, or curriculum generation, JSON records are materialized as a NetworkX MultiDiGraph; selected paths are mapped back to records to recover context triples.Source: §9.2 · pp. 42–43

9.3 QA Item Generation Prompts and Filtering Details

• The pipeline uses MCQ generation, option-quality filtering and uniformization, explanation generation, and triple-grounded correctness verification across all four disease and grounding settings.Accepted items are stored with a clinical-vignette question, four options, an explanation, and the correct option letter.Source: §9.3 · p. 43, Table 25 · p. 43

9.3.1 Step 1: MCQ Generation

• KG prompts provide only the target triple or path, whereas CG prompts add supporting triples from the same source chunk, but the primary relation or path must justify the answer.Context may enrich a vignette or distractors without determining the keyed answer.Source: §9.3.1 · pp. 44, 46
• Multi-hop prompts require reasoning across every path relation and intermediate concept, and prohibit questions answerable by only a direct single-hop relation.Prompts also request a clinical vignette, four options, and one clearly correct answer.Source: §9.3.1 · pp. 45–46

9.3.2 Step 2: Quality Filtering and Answer-Option Uniformization

• The option-quality filter rejects questions whose choices are near-duplicates, then approximately balances correct-answer positions across A, B, C, and D to limit position shortcuts.Later filtering can shift the final accepted distribution away from the 25% target for each option.Source: §9.3.2 · p. 47

9.3.3 Step 3: Explanation Generation

• Explanations use the primary triple or path to justify the answer; CG items may also use supporting context triples to enrich the reasoning and connect biomedical concepts.KG items receive no context triples, while CG items receive those attached to the primary triple or path.Source: §9.3.3 · p. 47

9.3.4 Step 4: Triple-Grounded Correctness Verification

• The final verifier assesses whether the question, answer, and explanation are logically correct, medically accurate, and supported by the primary source triple or path.It receives the complete QA item and the primary source triple or path.Source: §9.3.4 · p. 48
• The gastroparesis example identifies objective delayed gastric emptying, measured by scintigraphy, as the definitive diagnostic finding after symptoms and obstruction checks.The supporting context triples enrich the example, but the keyed answer is grounded in the primary triple linking delayed gastric emptying with gastroparesis.Source: §9.3.4 · pp. 48, 50

9.3.5 Answer-Option Distribution After Filtering

• After Step 4 filtering, final one-hop SFT training datasets retain answer-option distributions close to 25% per option.The passage says answer-option uniformization is largely preserved; it does not provide the individual option percentages here.Source: §9.3.5 · p. 51, Table 26 · p. 51
• Most mixed lower-hop RL dataset settings retain answer-option distributions close to uniform.The cited passage gives no exact percentages for the settings.Source: §9.3.5 · p. 51, Table 27 · p. 51
• For higher-hop evaluation datasets, each disease and grounding regime contains 8,000 QA items per hop depth, and answer-option distributions are close to uniform across A, B, C, and D.Source: §9.3.5 · p. 51

9.3.6 Validation of Higher-Hop Evaluation QA Items

• Before testing, an LLM-based check independently validates higher-hop items for validity, answerability, and alignment with their source paths.The same validation logic as the Judge-1 audit is applied independently of the evaluated models to reduce errors attributable to misaligned or noisy items.Source: §9.3.6 · p. 51
• The alignment prompt requires that answering depend on the source, the expected answer be medically correct and uniquely correct, and any vignette logically depend on the triple.It identifies Qwen3.6-27B as the LLM judge used for this validation.Source: §9.3.6 · pp. 51–52
• Items are rejected if they can be answered without the source, have a wrong or debatable key, allow multiple defensible answers, or test a different relation.For CG-grounded questions, context may enrich the vignette or distractors, but the keyed answer must remain supported by the primary source path.Source: §9.3.6 · p. 52

• SFT dynamics were tracked across epochs through training loss, gradient norm, and learning rate.Figure 10 illustrates their evolution throughout SFT.Source: §9.4 · p. 52, Figure 10 · p. 53
• Token statistics were computed over each full question_and_explanation field, including the question, options, thinking trace, and final answer.Source: §9.4 · p. 53
• Most tokens in each QA item are used by the thinking trace, providing detailed relational reasoning supervision alongside the final answer.Source: §9.4 · p. 53

9.5.1 Overview of the Repair Procedure

• The pipeline audits all initial one-hop validation failures, removes invalid triples, regenerates bad questions, and iteratively fine-tunes on examples from valid missing-knowledge triples.Each round re-evaluates the validation set and tracks triple IDs; repeated failures go to Judge 2 for review or may be marked valid but unlearnable.Source: §9.5.1 · p. 54

9.5.2 Triple-Level Error Categories

• History tracked by source triple ID distinguishes fixed, new-error, regression, and persistent cases, while triples still failing after three targeted attempts become quarantine candidates for Judge-2 review.Persistent cases remain wrong after two consecutive repair rounds; quarantine candidates may be removed from the retained set or labeled valid but unlearnable after review.Source: §9.5.2 · p. 54

9.5.3 Repair Example Allocation

• Repair-example allocation scales with failure history: persistent triples receive 7 examples, regressions 5, and new errors 3 to vary supervision strength by failure severity.Persistent failures receive the strongest repair, regressions moderate repair, and new errors light repair to reduce overfitting and additional regressions.Source: §9.5.3 · p. 54

9.5.4 Continual SFT Setup

• Continual SFT uses a 5 × 10^-5 learning rate instead of the initial 2 × 10^-4 and selects the best validation-performing epoch from each six-epoch round to initialize the next.This gentler, checkpoint-selected approach aims to repair targeted gaps while limiting overwriting and regressions from overfitting small repair sets.Source: §9.5.4 · p. 55

9.5.5 Models Repaired

• Four disease-specific checkpoints—KG-grounded and CG-grounded models for each disease—are repaired independently using the same judge, history-aware repair, and continual-SFT procedure.Table 30 identifies the four checkpoints repaired.Source: §9.5.5 · p. 55, Table 30 · p. 55

9.5.6 Repair Trace for CGModel-Gastro

• After removing 16 Judge-1 invalid triples and 80 Judge-2 invalid or valid-unlearnable cases from the original 6,018 validation examples, the retained cleaned set contained 5,922 examples and accuracy was 100.00%.Source: §9.5.6 · p. 55

9.5.7 LLM Judge Prompt Design

• Judge 1 audits every initial failure, whereas Judge 2 reviews only quarantine candidates after repeated repair failures and decides their disposition using the accumulated history.Judge 1 diagnoses the cause of initial errors; Judge 2 handles repeatedly failing cases after repair history is available.Source: §9.5.7 · p. 58

9.5.8 Judge 1 Prompt Templates

• Judge 1 applies ordered triple-validity and question-alignment checks before assigning missing knowledge; for CG-grounded items, supporting triples provide context but the answer must hinge on the primary triple.KG-grounded prompts include the primary triple and paired validation item; CG-grounded prompts additionally include supporting triples.Source: §9.5.7 · p. 58, §9.5.8 · pp. 58, 60
• Judge 1 checks triple validity, then question alignment and answer quality, labeling a case MISSING_SKILL only when both pass.The question must require the primary triple, have a medically correct and uniquely correct answer, and not conflict with supporting triples.Source: §9.5.8 · pp. 60–61

9.5.9 Judge 2 Prompt Templates

• Judge 2 Case A rechecks previously audited MISSING_KNOWLEDGE triples, while Case B triages other repair failures using the triple and paired validation question.Case A distinguishes INVALID_TRIPLE from VALID_UNLEARNABLE after repeated targeted repair; Case B can also identify a bad validation question.Source: §9.5.9 · pp. 61–62

Step 2: BAD_VAL

• For a valid triple, Judge 2 labels a misaligned or unfair validation question BAD_VAL; a fair question with continued model failure is VALID_UNLEARNABLE.The prompt advises conservative judgments and favors VALID_UNLEARNABLE when uncertain.Source: Step 2: BAD_VAL · p. 62

• The prompt instructs the model to think through the clinical problem systematically using <think>...</think> tags.Source: §9.6 · p. 63
• The model must provide a brief explanation and format its final answer as one of A, B, C, or D.Source: §9.6 · p. 63
• The prompt places the clinical vignette and four answer options inside the question and options fields.Source: §9.6 · p. 63

9.7.2 Option-Shuffling Robustness Results

• Almost all models remain relatively stable under option shuffling; CGModel-RL on Diabetes KG-grounded QA has an isolated 2.07-percentage-point drop at 3 hops.The robustness tables cover Gastroparesis KG-grounded, Diabetes KG-grounded, and Diabetes CG-grounded QA.Source: §9.7.2 · p. 63, Table 35 · p. 64, Table 36 · p. 64

9.7.1 Full Model Comparisons for Diabetes

• Table 34 reports Diabetes higher-hop QA accuracy across model variants, with ΔRL denoting per-hop gains over corresponding non-RL checkpoints.Source: §9.7.1 · p. 63, Table 34 · p. 64