Source-linked AI summary
Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement
Sekh Mainul Islam, Pepa Atanasova, Isabelle Augenstein
TL;DR
Prior work largely studied single-step, conflict-only PK–CK interactions, leaving richer interactions and their evolution across NLEs underexplored. The paper introduces rank-2 subspace disentanglement for multi-step analysis and finds it captures diverse interactions that rank-1 misses, with alignment patterns varying by interaction type.
Problem
Earlier work focused on single-step generation and modeled PK–CK interaction as a binary rank-1 choice, leaving richer scenarios and multi-step NLE dynamics uncharacterized.
Source: §1 · p. 1Method
Results
Across four QA datasets and three open-weight instruction-tuned LMs, rank-1 fails to capture diverse interactions while rank-2 explains observed variance; hallucinations align with PK.
Source: §1 · p. 2Takeaways & Limitations
The analysis shows that PK–CK interaction is multidimensional, with supportive examples aligning more with PK and conflicting examples more with CK near the final answer.
Source: §1 · p. 2, §6 · p. 9The study follows linear decomposition findings; nonlinearity in PK–CK interaction is out of scope.
Source: Limitations · p. 10
Abstract
from arXiv · showhide
Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions by drawing on external Context Knowledge (CK) and Parametric Knowledge (PK). Understanding the interaction between these sources is key to assessing NLE grounding, yet these dynamics remain underexplored. Prior work has largely focused on i) single-step generation and ii) modeled PK--CK interaction as a binary choice within a rank-1 subspace. This approach overlooks richer interactions and how they unfold over longer generations, such as complementary or supportive knowledge. We propose a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and use it for the first multi-step analysis of knowledge interactions across longer NLE sequences. Experiments across four QA datasets and three open-weight LLMs demonstrate that rank-1 subspaces struggle to represent diverse interactions, whereas our rank-2 formulation captures them effectively, highlighting PK alignment for supportive interactions and CK alignment for conflicting ones. Our multi-step analysis reveals that hallucinated generations exhibit strong alignment with the PK direction, though context-faithful generations maintain a more balanced alignment between PK and CK.
1 Introduction
Prior analyses focused on final-answer generation and rank-1 models of conflicting PK–CK, leaving richer interactions and multi-step NLE dynamics underexplored. This work uses rank-2 disentanglement to analyze these dynamics, finding that rank-1 misses diverse interactions and that alignment patterns differ across interaction types.
1 Introduction
- NLEs can reveal whether LLM decisions draw on external context or parametric knowledge, but their PK–CK dynamics across longer generations remain unclear.The introduction notes that these explanations are used in tasks such as claim verification and question answering.Source: §1 · p. 1
- Prior work focused on single-step generation and modeled conflicting PK–CK as a binary choice in rank-1, overlooking complementary and supportive interactions.It also left step-by-step PK–CK dynamics over longer NLE sequences uncharacterized.Source: §1 · p. 1
- Across four QA datasets and three open-weight instruction-tuned LMs, rank-1 subspaces fail to capture diverse interactions, whereas rank-2 explains the observed variance.The authors also report that hallucinated generations align with PK, conflicting examples align more with CK near the final answer, and supportive examples align more with PK.Source: §1 · p. 2
- Hallucinated sequences align with PK, while conflicting examples align more with CK near the final answer and supportive examples more with PK.The authors also report that CoT often reduces PK alignment and balances PK–CK interaction.Source: §1 · p. 2
2 Related Work
Related work studies stored and contextual knowledge, their interaction, and methods for analyzing representations. This paper extends rank-1 conflict modeling with rank-2 disentanglement and multi-step analysis.
2 Related Work
- Research has examined how models encode factual knowledge in parameters and combine contextual information, including probing and representation analyses.Early work elicited factual knowledge directly from parameters, while later analyses studied encoding and combination across layers (Brown et al., 2020; Tenney et al., 2019).Source: §2 · p. 2
- PK–CK interaction studies analyze suppression and conflict types, while interventions seek to improve balance through training, representation changes, decoding, or steering.Hagström et al. (2026) compare context-usage manipulation approaches and highlight trade-offs among them.Source: §2 · p. 2
- Prior work finds NLEs and CoTs can be unfaithful, while rank-1 probes model conflict but cannot disentangle jointly encoded PK and CK contributions.This paper presents rank-2 projection as an identifiable approach for token-level, multi-step analysis of PK–CK during NLE generation.Source: §2 · p. 3
3 Method
The method formalizes NLE generation as evolving PK–CK interaction, distinguishes interaction scenarios through controlled intents, and learns a rank-2 subspace to separate the two knowledge contributions.
3.1 Task Formulation
- For each query–context pair, the model generates an answer and an NLE whose tokens are analyzed as PK–CK interaction evolves across generation steps.The task formulation distinguishes a parametric answer, a context-based answer, and a final answer combining both sources.Source: §3.1 · p. 3
3.2 Identify Different PK–CK Interactions
- The method compares context-only, PK-only, and joint intents to characterize how individual knowledge sources contribute to the final answer.The joint intent reflects the model’s intrinsic behavior under the interaction scenario.Source: §3.2 · pp. 3–4
- Interactions are classified as supportive, complementary, conflicting, irrelevant, or knowledge suppression, extending beyond conflict-only cases.Supportive sources reinforce the same answer; complementary sources contribute distinct information; suppression occurs when agreeing sources yield an erroneous or unrelated prediction.Source: §3.2 · pp. 3–4
3.3 Localize Intent-Guided PK–CK Interactions
- Patching identifies layers that capture PK and CK contributions, and the method learns their directions as orthonormal axes in a rank-2 projection subspace.The learned subspace models how individual knowledge contributions mix to generate the final answer.Source: §3.3 · p. 5
- Rank-2 projections yield normalized CK and PK contributions at each NLE step, supporting identifiable comparisons of their relative influence.The method defines αic and αip as normalized contributions in [0, 1].Source: §3.3 · p. 5
4 Results
Across four QA datasets and three open-weight LLMs, rank-2 captures PK–CK interactions that rank-1 does not. The analyses trace how knowledge alignment varies by interaction type and generation step.
4.1 Experimental Setting
- Across BaseFakepedia, MultihopFakepedia, StrategyQA, and OpenBookQA, the experiments use three open-weight instruction-tuned LLMs.The models are Llama-3.1-8B, Gemma-2 9B, and Mistral-v0.3 7B.Source: §4.1 · p. 5
4.2 RQ1: Is a Rank-1 Projection Subspace Enough for Disentangling PK and CK Contribution Across All Types of Knowledge Interaction Scenarios?
- Rank-1 distributions overlap with means near zero, while rank-2 is the minimum subspace rank reported to capture interaction variance, reaching cumulative explained variance of 1.0.This supports rank-1 insufficiency and rank-2 sufficiency across the evaluated models and datasets.Source: §4.2 · p. 6, Figure 3 · p. 6
4.3 RQ2: How Do Individual PK and CK Contributions Change Over the NLE Generation for Different Knowledge Interactions?
- OpenBookQA, with the largest rank-1-to-2 explained-variance jump across models, served to learn layers and the rank-2 subspace before evaluation across all datasets.StrategyQA yields nearly identical subspaces, with directional similarity above 0.9 on both axes.Source: §4.3 · p. 6
- On OpenBookQA, BOTH→PRI patching yields larger probability gains than BOTH→CTX, indicating greater PK reliance across all three models.For Llama-3.1-8B-Instruct, the larger gain occurs at layers 13–18 for PK versus 15–17 for CK.Source: §4.3 · p. 6, Figure 4 · p. 7
- Conflicting examples align more with CK, while supportive examples align more with PK; Fakepedia datasets show higher CK contribution and more conflicting examples.StrategyQA and OpenBookQA show higher PK contribution and include supportive contexts.Source: §4.3 · p. 7, Figure 5 · p. 7
- NLE generation starts with higher CK, then uses both sources with slight PK priority; longer NLEs show greater PK–CK fluctuation.The authors associate longer NLEs with difficult, deeper multi-hop reasoning and higher token uncertainty, which leads to iterative reconciliation of PK and CK.Source: §4.3 · p. 7, Figure 6 · p. 7
4.4 RQ3: Can We Find Reasons for Hallucinations Based on PK–CK Interactions?
- Across sequence steps, the PK–CK gap is larger for examples with hallucinated spans than for other examples.Hallucination labels were available only for Llama-3.1-8B-Instruct in these datasets because their model coverage was limited.Source: §4.4 · p. 8, Figure 7 · p. 8
4.5 RQ4: How is the CoT mechanism aligned with the knowledge interaction subspace?
- CoT maintains similar CK alignment to standard prompting and reduces PK alignment on all datasets except OpenBookQA.Source: §4.5 · p. 8, Figure 8 · p. 8
5 Steering Model Behavior via Projection Subspace Intervention
The authors steer answer-token representations along learned CK and PK directions to test whether the rank-2 subspace can control knowledge use. Steering behavior matches explicit-intent behavior, and PK-direction steering reduces hallucinations relative to an unsteered model.
5 Steering Model Behavior via Projection Subspace Intervention
- The intervention adds CK and PK directional weights to the answer-token hidden representation instead of using explicit knowledge-source instructions.The settings use pi > ci, pi < ci, or pi = ci to approximate PK-only, CK-only, or joint-source intent, respectively.Source: §5 · p. 8
- Steering with CK and PK weights produced answer predictions aligned with explicit-intent predictions, behaviorally verifying disentanglement in the learned subspace.The evaluation measures whether the model predicts the same answer without the explicit intent instruction.Source: §5 · p. 8, Table 1 · p. 9
- Hallucinated RAGTruth examples had average CK and PK scores of 0.46 and 0.54, while non-hallucinated examples scored 0.49 and 0.51, respectively.These averages are reported for Llama-3.1-8B-Instruct.Source: §5 · p. 8
- With generation parameters unchanged, rank-2 PK directional steering reduced hallucinated generations by 50/450, versus Sun et al. (2025)'s 73/450; our reduction remained statistically significant against the non-steered model.Source: §5 · p. 9, Table 2 · p. 9
- The PK axis is neither a unique nor a complete causal mechanism for hallucination, and another projection subspace may also explain the mechanism.The authors note that another subspace should not be orthogonal to the learned PK direction.Source: §5 · p. 9
6 Conclusion
The paper treats PK–CK interaction as multidimensional and uses a rank-2 projection framework to disentangle the two sources across NLE generation. Its analyses report distinct alignment patterns for hallucinated, context-faithful, and CoT-prompted generations.
6 Conclusion
- Across four QA datasets and three open-weight instruction-tuned LMs, rank-1 subspaces failed to capture diverse interactions, whereas rank-2 sufficed to explain the observed variance.The framework disentangles token-level PK and CK components and enables multi-step analysis during NLE generation.Source: §6 · p. 9
- Hallucinated generations aligned strongly with the PK axis, while context-faithful generations maintained a more balanced PK–CK alignment.Source: §6 · p. 9
- Chain-of-Thought prompting shifted generations toward CK by reducing PK reliance.Source: §6 · p. 9
- Intervention in the learned rank-2 subspace controlled model behavior toward the desired intent and reduced hallucination generation.The authors present the intervention as verifying behavioral knowledge disentanglement in that subspace.Source: §6 · p. 9
Limitations
The authors identify boundaries in the rank-2 framework's assumptions, scope, and evaluation. They leave non-orthogonal directions, higher-rank analysis, nonlinear interactions, and broader-than-QA evaluation for future work or separate study.
Limitations
- The analysis restricts PK and CK to orthonormal directions; mapping their contributions to non-orthogonal directions would be complicated.The authors verify the orthonormal directions through steering-based intervention experiments.Source: Limitations · pp. 9–10
- The method is limited to rank-2, based on observed explained variance and empirical sufficiency; examining the orthogonal component could establish whether rank-3 is needed.Source: Limitations · p. 10
- PCA and ICA are treated as invalid baselines because their directions may not correspond to knowledge directions; the authors instead theoretically analyze rank-1 and empirically test rank-2.Source: Limitations · p. 10
- Identifying what triggers a shift from CK to PK requires different experiments and is outside this work's focus on disentangling and modeling contributions.Source: Limitations · p. 10
- The study follows prior findings on linear decomposition and leaves nonlinear interactions in LLM representations out of scope.Source: Limitations · p. 10
- Evaluation covers three open-weight decoder-only instruction-tuned LLMs and four QA datasets; diverse generation tasks could broaden generalization findings.The authors cite QA datasets as the available annotated datasets with different knowledge-interaction types and note QA's reliance on PK and CK.Source: Limitations · p. 10
Ethical Considerations
The authors report using Claude API for language polishing, grammatical corrections, and improving clarity and coherence; technical contributions and final decisions were made by the authors.
Ethical Considerations
- Claude API assisted with language polishing and grammatical corrections, while authors made all technical contributions and final decisions.Source: Ethical Considerations · p. 10
A.1.1 Determining PK and CK Directions in the Rank-2 Projection Subspace.
The rank-2 projection subspace is spanned by orthonormal CK and PK directions. The method then identifies each direction using hidden representations from answers generated under corresponding intents.
A.1.1 Determining PK and CK Directions in the Rank-2 Projection Subspace.
- The subspace uses orthonormal basis vectors for the context and parametric knowledge directions.The basis is written as u = [u_c, u_p] in R^(d×2).Source: §A.1.1 · p. 14
- Hidden representations of answer tokens generated under context- and parametric-intent prompts provide the representations used to identify the two directions.The passage states that successful optimization from the two pre-selected directions yields different directions for the two knowledge sources.Source: §A.1.1 · p. 14
A.1.2 Prompt Template
The appendix describes prompt templates, projection evaluation, and checks of rank-2 disentanglement robustness. It reports stable results across layers and prompt variations, context-responsive contributions, and separated PK and CK directions.
A.1.2 Prompt Template
- Prompt templates specify intent-driven answers and NLE and CoT prompting across datasets and models.The controlled answers use intents w_p, w_c, and w_b.Source: §A.1.2 · p. 14, Table 3 · p. 15
A.1.3 Hyperparameters
- PatchScope captures individual PK and CK contributions from the final answer using the layer-selection hyperparameters.The hyperparameter figure covers Llama-3.1-8B-Instruct, Gemma-2-9B-it, and Mistral-7B-Instruct-v0.3.Source: §A.1.3 · p. 14, Figure 10 · p. 16
A.1.4 Computation Overhead of PatchScope
- Rank-2 subspace learning takes less than 30 minutes for 500 examples on Llama-3.1-8B and Mistral-v0.3-7B, and slightly over 30 minutes for 200 examples on Gemma-2-9B.The measurements used a single NVIDIA B200 GPU (180 GB).Source: §A.1.4 · p. 14
A.1.5 Evaluation Metric
- The evaluation uses subspace-component distributions and cumulative explained variance to assess whether projections capture distinct interactions and how much variance rank-r encodes.For rank 1 to adequately capture PK–CK interactions, the passage says cumulative explained variance EV1 should approximate 1.0.Source: §A.1.5 · p. 14
A.2 Additional Results
- A rank-1 probe cannot uniquely disentangle PK and CK contributions when both coefficients are nonzero.Infinitely many coefficient pairs can yield the same scalar observation.Source: §A.2 · pp. 14–15
A.3 Robustness Analysis
- 0.92, 0.88, and 0.85 are the end-to-end agreement scores for paraphrase, synonym, and word-order changes; corresponding subspace consistency scores are 0.95±0.04, 0.93±0.05, and 0.89±0.08.The authors interpret these results as showing insensitivity to superficial prompt variations.Source: §A.3 · pp. 16–17
- Minimal, medium, and maximal context yield (p_i=0.82, c_i=0.24), (p_i=0.65, c_i=0.58), and (p_i=0.85, c_i=0.87), respectively.The passage links the rise in CK contribution from minimal to medium context to greater reliance on evidence as it becomes available, and says maximal context strongly activates both sources.Source: §A.3 · p. 17
- Rank-2 disentanglement is reported as stable across layers, prompt surface perturbations, context variation, and geometric stability.These are the four robustness axes assessed in the appendix.Source: §A.3 · pp. 15, 18