Source-linked AI summary
G-Refer: Graph Retrieval-Augmented Large Language Model for Explainable Recommendation
Yuhan Li, Xinni Zhang, Linhao Luo, Heng Chang, Yuxiang Ren, Irwin King, Jia Li
TL;DR
Explainable recommendation needs explicit, informative CF signals from complex interaction graphs and a way to connect those signals with LLM-generated language. G-Refer combines hybrid graph retrieval, graph translation, knowledge pruning, and retrieval-augmented fine-tuning; experiments report superior explainability and stability, including gains of up to 8.67% over baselines.
Problem
Complex graph structure makes CF information difficult to extract, while implicit representations create a modality gap when integrating it with natural-language explanations.
Method
G-Refer uses hybrid path- and node-level retrieval for structural and semantic CF signals, prunes less relevant knowledge, and fine-tunes LLMs to use the retrieved information.
Results
G-Refer demonstrates superior explainability and stability, surpassing state-of-the-art baselines by up to 8.67%.
Takeaways & Limitations
Explicit CF retrieval combined with graph translation and LLM adaptation supports more accurate, personalized, and contextually relevant recommendation explanations.
Abstract
from arXiv · showhide
Explainable recommendation has demonstrated significant advantages in informing users about the logic behind recommendations, thereby increasing system transparency, effectiveness, and trustworthiness. To provide personalized and interpretable explanations, existing works often combine the generation capabilities of large language models (LLMs) with collaborative filtering (CF) information. CF information extracted from the user-item interaction graph captures the user behaviors and preferences, which is crucial for providing informative explanations. However, due to the complexity of graph structure, effectively extracting the CF information from graphs still remains a challenge. Moreover, existing methods often struggle with the integration of extracted CF information with LLMs due to its implicit representation and the modality gap between graph structures and natural language explanations. To address these challenges, we propose G-Refer, a framework using graph retrieval-augmented large language models (LLMs) for explainable recommendation. Specifically, we first employ a hybrid graph retrieval mechanism to retrieve explicit CF signals from both structural and semantic perspectives. The retrieved CF information is explicitly formulated as human-understandable text by the proposed graph translation and accounts for the explanations generated by LLMs. To bridge the modality gap, we introduce knowledge pruning and retrieval-augmented fine-tuning to enhance the ability of LLMs to process and utilize the retrieved CF information to generate explanations. Extensive experiments show that G-Refer achieves superior performance compared with existing methods in both explainability and stability. Codes and data are available at https://github.com/Yuhan1i/G-Refer.
1 Introduction
Explainable recommendation combines LLM generation with collaborative-filtering information, but graph complexity and implicit representations make CF extraction and LLM integration difficult. G-Refer addresses these challenges with hybrid graph retrieval, graph translation, knowledge pruning, and retrieval-augmented fine-tuning, achieving stronger explainability and stability.
- Graph-based CF information captures complex user-item patterns that can support more accurate and informative natural-language explanations.
- GNNs capture CF information through hidden user and item representations, but combining them with LLM-based recommendation still faces explanation challenges.
- G-Refer retrieves explicit structural and semantic CF information through path-level and node-level retrievers, then translates it into human-understandable text.
- Knowledge pruning filters less relevant training samples, while retrieval-augmented fine-tuning improves LLM use of retrieved CF information and profile semantics.
- 8.67%: G-Refer surpasses a series of state-of-the-art baselines in experiments on public datasets.
2 Preliminaries
Explainable recommendation generates human-understandable rationales for user-item recommendations using interaction graphs, profiles, and graph-based retrieval. GraphRAG extends retrieval-augmented generation by using graph structure to retrieve information and condition language-model responses.
- The interaction graph is bipartite, with user profiles in B and item profiles in C supplying associated textual information.
- Explainable recommendation generates a clear textual explanation for a recommended user-item pair.
- The L-hop edge-centered ego-graph around a user-item pair captures local connectivity within L hops for explanation generation.
- GraphRAG retrieves structural information from a graph and generates responses conditioned on that retrieved information.
3 Methodology
G-Refer combines path-level and node-level retrieval to extract structural and semantic collaborative-filtering signals, then translates them into usable knowledge for explanation generation. Its path retriever uses GNN-based edge masking, pruning, and shortest-path search, while the node retriever uses dense profile similarity; knowledge pruning addresses redundant retrieved information.
- Hybrid graph retrieval: G-Refer combines path-level and node-level retrievers to extract structural and semantic collaborative-filtering information from user-item graphs.The retrieved information is translated into human-understandable text for explanation generation.
- Path-level retriever: The path-level retriever identifies informative user-item paths to expose structural collaborative-filtering signals for interpretable recommendations.It uses a GNN trained for recommendation and applies learned edge masks followed by Dijkstra’s shortest-path algorithm.
- Path-level retriever: m-core pruning removes nodes with degree below m, reducing spurious neighbors before explanation paths are selected.The process recursively retains nodes whose degrees are at least m, supporting concise paths that avoid high-degree nodes.
- Path-level retriever: Mask learning selects important edges for prediction and path formation, after which Dijkstra’s algorithm retrieves up to k explanation paths.The path score prioritizes shorter paths and paths without high-degree nodes.
- Node-level retriever: The node-level retriever selects semantically relevant user and item neighbors using dense profile embeddings and similarity-based retrieval.A shared text encoder maps profiles to embeddings, and top-k neighboring users and items are retrieved by semantic similarity.
- Knowledge integration: Retrieved graph paths cannot be directly combined with user and item profiles, while profile-only explanations can make additional retrieved knowledge redundant.These observations motivate converting graph knowledge into usable text and pruning less relevant training samples.
4 Experiments
G-Refer is evaluated on three public datasets using semantic explainability, stability, human preference, component, scale, hyperparameter, and efficiency analyses. It consistently improves explanation quality and stability, while results show benefits from combining retrievers and pruning noisy training samples.
- Overall Performance: G-Refer outperforms baselines in explainability and stability across three datasets, improving BERTF1 over XRec by 1.67%, 2.73%, and 2.81%.It also increases BERTRscore by 4.56%, 8.67%, and 7.48% across the datasets, with a slight BERTPscore decrease.
- Overall Performance: Human evaluators consistently favor G-Refer explanations, selecting them in over 80% of Yelp and Google-reviews cases.The evaluation used 20 test user-item pairs per dataset and five senior researchers comparing G-Refer with XRec.
- Ablation Studies: Combining path-level and node-level retrievers yields the best ablation performance, with semantic information more important for Yelp and structural knowledge more advantageous for Google-reviews.The findings indicate that retriever contributions vary with dataset characteristics and are complementary.
- Ablation Studies: A 3B Qwen 2.5 model achieves results comparable to a 7B model after retrieval-augmented fine-tuning with retrieved knowledge.Performance scales clearly from 0.5B to 3B, and fine-tuning with retrieved knowledge substantially improves explanation generation.
- Ablation Studies: Knowledge pruning is essential because full-dataset training adds samples without proportional gains and can slightly degrade performance, including on Yelp.Pruning removes self-explanatory samples whose additional CF information may introduce noise.
- Hyperparameter Study: BERTscore precision peaks at k=2, while k=1 lacks sufficient CF information and larger k values can introduce noise; recall is relatively insensitive to k.The study varies the number of retrieved paths and nodes from 1 to 5.
- Efficiency Analysis: G-Refer trains faster and performs better than XRec despite requiring more learnable parameters, while full-set training reduces efficiency without performance gains.The efficiency comparison uses one GPU and batch size 1; knowledge pruning reduces training data and XRec is slowed by adapter insertion.
5 Related Works
Explainable recommendation methods increasingly use graph-based collaborative-filtering information to produce personalized explanations, but graph complexity makes retrieval difficult. Graph retrieval-augmented generation addresses this by incorporating graph information into LLM generation.
- Recent explainable recommendation methods generate personalized natural-language explanations from user and item profiles and interactions.
- User-item interaction graphs contain collaborative-filtering information that captures complex user-item patterns for explanation generation.
- Graph retrieval-augmented generation incorporates graph information into the LLM generation process through graph retrieval and graph-augmented generation.
6 Conclusion
G-Refer combines hybrid graph retrieval, knowledge pruning, and retrieval-augmented fine-tuning to extract explicit collaborative-filtering signals and improve explanation generation. Experiments validate its effectiveness, while future work targets training-free retrieval and transferability.
- G-Refer combines hybrid graph retrieval, knowledge pruning, and retrieval-augmented fine-tuning for explainable recommendation.
- Comprehensive experiments validate G-Refer’s effectiveness and indicate potential for graph retrieval-augmented generation in recommendation scenarios.
- Future work includes fully training-free retrievers and transferability to other tasks.
A.1 Details of Datasets
The evaluation uses three public datasets representing distinct user-item interaction settings: Amazon-books, Yelp, and Google-reviews. Table 3 reports their dataset statistics.
- Table 3 lists the statistics of the three experimental datasets.
- The experiments use three public datasets: Amazon-books, Yelp, and Google-reviews.Amazon-books contains user-book ratings and reviews; Yelp contains business reviews and ratings.
A.2 Details of Metrics
The paper evaluates generated explanations with complementary automatic metrics covering contextual similarity, generation quality, and uniqueness. BERTScore specifically compares contextual embeddings between reference and generated texts.
- GPTScore evaluates text quality with a context-aware assessment based on large language models.
- BERTScore measures similarity between reference and generated texts using contextual BERT embeddings.
- BERTScore computes word-level cosine similarities between pre-normalized reference and generated embeddings.
- BARTScore treats evaluation as text generation and scores the probability of regenerating reference texts with BART.
- BLEURT assesses similarity between generated and reference texts using a language model pretrained with synthetic data.
- USR evaluates explanation uniqueness as the ratio of unique sentences to total sentences.
A.3 Details of Baselines
The baselines use varied architectures to generate recommendation explanations from user and item information, including recurrent, attention-based, and Transformer models.
- NRT jointly predicts ratings and generates recommendation tips from user and item IDs using a GRU.
- Att2Seq generates reviews with an attention-based attribute-to-sequence model using attribute information and a two-layer LSTM.
- PETER maps user and item IDs to explanations with a personalized Transformer model that bridges IDs and words.
B Case Study
The case studies show G-Refer translating retrieved collaborative-filtering signals into human-readable explanations, with node-level and path-level evidence revealing recommendation reasons absent or misstated by XRec.
- Case Study: G-Refer presents two cases comparing translated retrieved knowledge and generated explanations against ground-truth explanations and XRec.
- Case Study: Node-level signals reveal similar users’ drinking preferences, enabling G-Refer to identify crafted beers despite no explicit drinking interest in the profile.
- Case Study: XRec produces generic explanations such as delicious food and great service, missing key recommendation reasons and receiving a low Recall score.
- Case Study: Path-level signals uncover connections indicating interest in frozen yogurt, whereas XRec incorrectly attributes the recommendation to Thai food.
- Case Study: Table 4 illustrates node-level retrieval on Yelp, while Table 5 illustrates path-level retrieval on Google-review for generating better explanations.