LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations

1 INTRODUCTION

• Recommender systems analyze multi-source behavioral data, and graphs represent user-item relationships and capture complex connections.The supplied passage also names graph neural networks, random walks, matrix factorization, and PageRank as example methods.Source: §1 · pp. 1–2
• Prior work uses graph methods for link prediction or graph classification, while LLM research applies language models to recommendation tasks such as sequential recommendation, rating prediction, explanation generation, and direct recommendation.The introduction also situates work on prompt engineering, fairness, efficiency, explainability, and ethical issues in LLM-based recommendation.Source: §1 · p. 2
• The paper identifies a gap between graph-based link prediction and LLM-based autoregressive generation, and proposes prompt and attention innovations to incorporate graph edges into LLM recommendations.The prompt design encodes direct user-item and second-order item relationships; the attention mechanism models direct connectivity and indirect spatial information.Source: §1 · p. 2
• The framework treats user and item nodes as recommendation tokens and reformulates recommendations as probabilistic generation, pretraining on contextual prompts and fine-tuning on personalized predictive prompts.Fine-tuning uses interaction histories with future-tense triggers, then recommends items from test users’ histories and predictive triggers.Source: §1 · pp. 2–3
• The paper contributes a framework and new prompt and attention designs for incorporating first- and second-order graph relationships and edge connectivity information into LLM recommendations.It reports experiments across recommendation datasets to evaluate task performance and the prompt and attention mechanisms.Source: §1 · p. 3

2.1 Symbol Definition

• Table 1 is described as a reference for symbols covering user-item interactions, textual representations, model parameters, and graph structures.The supplied passages do not enumerate the table’s individual symbols.Source: §2.1 · p. 3, Table 1 · p. 3

2.2 Important Definitions

• The paper treats users and items as unique tokens in recommendations and describes embeddings as dense vectors encoding language attributes and semantic information.Prompts guide generative LLMs to produce text tokens in specific contexts or styles.Source: §2.2 · p. 4

2.3 Problem Statement

• The problem statement represents interactions between I users and J items with a binary matrix and seeks a generative recommender that uses a user ID and interaction history to produce product recommendations.The formulation also identifies user descriptions, item descriptions, reviews, and purchase-reason explanations as textual inputs.Source: §2.3 · p. 4

3.1 Framework Overview

• The graph-attentive backbone uses user and item tokens plus first-order and second-order connectivity to reformulate recommendations as probabilistic generation in response to prompts.It combines multi-source descriptions and reviews with a graph-knowledge-guided attentive LLM and a neural attentive decoder.Source: §3.1 · p. 5
• Crowd contextual prompts combine user and item descriptions, reviews, and interaction-event text, and pretrain the backbone by maximizing text generative likelihood.This stage is intended to provide contextual knowledge about recommendations.Source: §3.1 · p. 5
• Personalized fine-tuning turns a user’s interaction history into past-tense text with a future-tense trigger and minimizes recommendation errors.The optimization target is recommendation error rather than text-generation likelihood.Source: §3.1 · p. 5
• At inference, the fine-tuned backbone uses a test user’s rating or purchase history and a predictive trigger to recommend items.The four-step overview places this generative recommendation after backbone development, contextual pretraining, and personalized fine-tuning.Source: Figure 1 · p. 4, §3.1 · p. 5

3.2.1 GPT2 as LLM Base Model.

• GPT-2 provides the base model, using attention over input words to support conditional text generation.Its attention scores determine how much each word contributes when generating the next word.Source: §3.2.1 · p. 5

3.2.2 Integrating Two Structure Knowledge for Graph Attentive LLM.

• User-item interactions form graph edges, so user and item tokens are not treated as independent sequence entities.The graph also contains user-user and item-item connections.Source: §3.2.2 · pp. 5–6
• Graph-structured attention adds relationship encoding R to the attention scores, giving connected nodes additional attention.R is extracted from graph knowledge and encodes relationships between nodes.Source: §3.2.2 · p. 6
• R combines direct first-order connectivity with indirect high-order path connectivity between users and items.Direct connections are represented by R_conn, typically a binary adjacency matrix.Source: §3.2.2 · p. 6
• R_path uses normalized shortest-path scores to represent indirect relationships, with a damping factor adjusting the influence of distant nodes.Shorter paths indicate closer connections; combining direct and indirect relationships gives attention information about node positions and interconnections.Source: §3.2.2 · pp. 6–7

3.3.1 Data Collection.

• Pre-training begins with a diverse corpus of user and item descriptions, item reviews, and historical interaction events.The procedure trains the graph-attentive GPT-2 on recommendation text data.Source: §3.3.1 · p. 7, §3.3 · p. 7

3.3.2 The Structure of Crow Contextual Prompts.

• Special userID and itemID tokens make users and items the targets of representation learning and generative recommendation.Their embeddings are intended to encode user and item semantics, including profiles, preferences, descriptions, reviews, and information propagated from the other entity type.Source: §3.3.2 · p. 7
• The tokenizer is revised to preserve user and item tokens as whole units rather than decomposing them.The passage says decomposition would hinder precise representation of their rich semantic information.Source: §3.3.2 · p. 7
• Item-content prompts include attributes such as title, brand, categories, and descriptions, while user contents are removed for privacy concerns.Source: §3.3.2 · p. 7
• First-order relationship prompts use users’ reviews and purchase explanations to represent user-item interactions.These prompts provide textual accounts of interactions with items.Source: §3.3.2 · p. 7, Figure 2 · p. 8
• Second-order prompts express item associations, and interaction-event prompts incorporate purchase histories.The prompt structure also includes user/item tokens and item contents alongside these relationship and event categories.Source: §3.3.2 · p. 7, Figure 2 · p. 8
• Crowd contextual prompts enrich training context with user and item characteristics, opinions, functionality, categories, and network relationships.The graph-attentive GPT-2 is trained to predict the next token by maximizing textual-sequence generation likelihood, with the objective described as combining information sources and modeling first- and second-order connectivity.Source: §3.3.2 · p. 7, §3.3.3 · pp. 7–8

3.4 Fine-tuning Graph Attentive LLM with Personalized Predictive Prompts

• Fine-tuning shifts the pre-trained model toward personalized recommendations by combining personalized predictive prompts with recommendation-oriented loss functions.The pre-training objective emphasizes textual generation accuracy rather than personalized item recommendation.Source: §3.4 · p. 8
• Fine-tuning adapts the pre-trained LLM to personalized item recommendation rather than recommender-system world-knowledge learning.Source: §3.4 · p. 8

3.4.1 Personalized Predictive Prompts.

• User interaction histories are converted into structured past-tense text and paired with a future-tense trigger to prompt item recommendations.The history can include ratings or purchases; the trigger may ask what the user will purchase.Source: §3.4.1 · pp. 8–9, Figure 3 · p. 9
• The model generates a recommendation list from a user’s historical item-purchase events and personalized predictive prompt.Source: §3.4.1 · p. 9
• Generated recommendations are compared with the user’s actual purchase records, and the resulting loss is used to optimize fine-tuning.The generative probability measures whether recommendations are statistically close to historical purchase records.Source: §3.4.1 · p. 9

3.4.2 The Optimization Objective and Training Procedure of Fine-tuning.

• The fine-tuning objective is the negative log probability of the recommendation list conditioned on the prompt and LLM weights.The prompt is Pr_i, the recommendation list is S_i, and Θ denotes the LLM weights.Source: §3.4.2 · p. 9

• For a test user, interaction records become the prompt, and the graph-attentive LLM scores each user-item pair.Source: §3.5 · p. 9
• Offline GPU clusters batch-compute user-candidate relevance scores for recommendation scenarios such as promotional emails and notifications.Source: §3.5 · p. 9
• Online serving selects top items by comparing the recommendation probability scores for the user, item, and interaction prompt.Source: §3.5 · p. 9
• Offline precomputation and online cache warm-up are proposed to handle GPU serving pressure and reduce redundant computations.Source: §3.5 · pp. 9–10

• The experiments assess whether the method generates more accurate recommendations.Source: §4 · p. 10
• The evaluation examines contributions from technical components, second-order relationships, and item background information.Source: §4 · p. 10
• The experiments also examine attention-mechanism impacts and parameter sensitivity and robustness.Source: §4 · p. 10

4.1.1 Data Description.

• The experiments use six named public Amazon datasets and binarize user-item interactions, treating scores greater than 3 as connections.For each user, 80% of interactions are training data, with 10% each for validation and testing and at least one sample in both latter sets.Source: §4.1.1 · p. 10

4.1.2 Evaluation Metrics.

• Effectiveness is evaluated with Recall@20, Recall@40, and NDCG@100; Recall@k measures the share of relevant items among the top-k recommendations.NDCG@k is described as a position-sensitive measure of recommendation-list quality.Source: §4.1.2 · p. 10

4.1.3 Baseline Algorithms.

• Baselines include ID-based Multi-VAE and attention-based methods, alongside sequential, graph, and LLM recommender approaches.The listed comparisons include LightGCN, LLMRec, and RecMind, among others.Source: §4.1.3 · pp. 10–11

4.1.4 Hyperparameters and Settings.

• Experiments use GPT-2 with a maximum input length of 1024, embedding dimension 768, and 50 fine-tuning epochs.Pre-training runs 10 epochs on crowd contextual data and 100 rounds on user-item interaction data.Source: §4.1.4 · p. 11

4.1.5 Experimental Environment.

• The experiments run on Ubuntu 22.04.3 with Python 3.11.5, PyTorch 2.0.1, and an NVIDIA GeForce RTX 4090 GPU.Source: §4.1.5 · p. 11

4.2.1 Overall Comparison.

• Recall@20, Recall@40, and NDCG@100 are superior to baseline models, with consistent improvements reported across datasets.The authors also report strong performance on sparse datasets and compare the model with baselines in Table 3.Source: §4.2.1 · p. 12, Table 3 · p. 12

4.2.2 Ablation Studies.

• Removing pre-training, fine-tuning, graph information in attention, or complex prompts reduces performance relative to the complete model.The ablations attribute gains to recommendation-related graph and text pre-training, fine-tuning, graph-aware attention, and prompts containing second-order relationships.Source: §4.2.2 · pp. 12–13, Table 4 · p. 13

4.2.3 Study on Different Pre-training Prompt Structures; 4.2.4 Study on Different Historical Interaction Length.

• Complete prompts with second-order relationships and item background information perform best, while historical-length variations affect performance without significant differences.The authors report flexibility in choosing historical length for deployment.Source: §4.2.3 · p. 13, Figure 4 · p. 14, §4.2.4 · p. 14

4.2.5 Study on Different Attention Injection Ways.

• Meaningful graph-aware attention performs best, and both direct connectivity and indirect path information contribute, unlike fixed connection values.The comparison includes normal attention and meaningful versus meaningless injection.Source: §4.2.5 · pp. 14–15, Figure 6 · p. 15, Figure 7 · p. 15

4.2.6 Study of Parameters; 4.2.7 Study of Different LLM backbones.

• Pre-training loss trends align with Recall@20, Recall@40, and NDCG@100, while graph integration narrows the BART–GPT performance gap.With graph information integrated, GPT slightly outperforms BART in Recall@20.Source: §4.2.6 · p. 15, §4.2.7 · p. 16, Figure 9 · p. 16

4.2.8 Study of Cold-start Situation.

• Under cold-start conditions, the method consistently performs best against BERT4Rec and RecMind, with smaller reported drops, including on AM-Luxury.On AM-Luxury, Recall@20 drops 0.0022 for this method, compared with 0.0341 for BERT4Rec and 0.0380 for RecMind.Source: §4.2.8 · pp. 16–17, Table 5 · p. 17

4.2.9 Scalability Analysis.

• Training and inference times grow approximately linearly with dataset size, while peak memory remains relatively stable.Inference takes longer because it scores all candidate items and performs a complete ranking, unlike training on sampled interactions.Source: §4.2.9 · pp. 17–18, Table 6 · p. 18

4.2.10 Case Study: Graph-Aware Recommendation Reasoning.

• In a single-user case study, the recommendations include a future-interacted item and items linked through shared category or brand information.The example is presented as evidence of direct and indirect graph connections in recommendation reasoning.Source: §4.2.10 · p. 18, Figure 10 · p. 18

5 RELATED WORK

• The related-work overview distinguishes LLM-based representation learning from directly using generative LLMs to construct recommendation logic.Both approaches are presented as recent directions in recommender systems.Source: §5 · p. 19

5.1 Traditional Recommender System

• Traditional recommendation progressed from collaborative filtering and matrix factorization to neural and graph methods that capture higher-order relationships.The section also surveys multimodal, denoising, debiasing, cold-start, and feature-engineering directions.Source: §5.1 · p. 19

5.2 Discriminative LLMs for Recommender System

• Discriminative LLM recommender work uses domain data, behavior pre-training, text, and embedding strategies to improve representations and address data scarcity.Examples include U-BERT, UserBERT, BERT4Rec, UniSRec, DWSRec, LLMRec, and CoLLM.Source: §5.2 · pp. 19–20

5.3 Generative LLMs for Recommender System

• Generative LLM recommender research explores prompt templates, dialogue, sequence generation, fine-tuning, reinforcement learning, and recommendation agents.The section names examples including ChatREC, GPTRec, TALLRec, GenRec, HSTU, and RecMind.Source: §5.3 · p. 20

6 CONCLUSION

• The framework combines prompts containing user-item relationships and second-order item relationships with attention mechanisms that model graph structure.The conclusion describes this as an approach to integrating complex user-item relationships and background information.Source: §6 · p. 20
• Experiments are reported to validate effectiveness and show improved recommendation accuracy and personalization compared with traditional recommender systems.The authors characterize the approach as a new technological path for developing recommender systems.Source: §6 · pp. 20–21