Source-linked AI summary
Profiling What Matters: Context-Aware Item Profiles from Large-Scale Metadata for LLM Recommenders
Dojun Hwang, Seunghan Lee, Cheonyoung Park, Sara Yu, SeongKu Kang
TL;DR
LLM recommenders still struggle to use vast, heterogeneous item metadata, especially when relevant features are implicit and user-dependent. CAIRO structures metadata and reviews into objective features and subjective traits, then uses lightweight selectors to build user-specialized profiles efficiently. Experiments show consistent improvements over existing baselines, while subjective-trait extraction may deviate from true user intents and requires refinement.
Problem
LLM recommenders face vast, heterogeneous item metadata with buried signals, while useful feature salience varies across items and users.
Method
CAIRO structures metadata and reviews into objective features and subjective traits, then uses lightweight context-aware selectors to construct user-specialized item profiles.
Results
CAIRO consistently outperforms existing baselines across datasets and metrics.
Takeaways & Limitations
Careful organization and user-adaptive selection of item information are as important as providing valuable information for LLM-based reranking.
Takeaways & Limitations
Subjective traits may deviate from true user intents because they are extracted through LLM reasoning rather than direct value identification.
Abstract
from arXiv · showhide
While Large Language Models (LLMs) have significantly advanced reranking in recommendation, effectively leveraging item-side information remains challenging. Real-world items are described by vast, heterogeneous, and unstructured metadata, where decision-relevant signals are often implicit, noisy, or buried in long descriptions. Moreover, feature salience is highly context-dependent, varying not only across items but also across users. Existing methods often rely on item titles, fixed attributes, or static item summaries, which limit personalized and fine-grained item understanding. To bridge this gap, we propose CAIRO, a user context-aware item profiling framework for LLM-based reranking. CAIRO first structures raw metadata and reviews into objective features and subjective traits, and employs a lightweight profiler to select the most relevant information for each user-item pair with limited serving-time overhead. The resulting profiles are concise and context-specific, providing relevant item-side evidence for the LLM's ranking decision. Experiments show that CAIRO consistently improves LLM-based reranking, highlighting the importance of item profiling that effectively exploits vast item-side information.
1 Introduction
LLM reranking benefits from user-side profiling, but item-side profiling remains limited despite the scale and diversity of real-world metadata. CAIRO addresses this gap by structuring item information and selecting features adaptively for each user context.
- User-side context: User profiling provides structured representations of preferences that improve LLM-based reranking beyond directly injecting noisy interaction data.Profiles can express likes, dislikes, habits, and long- or short-term interests.
- Motivation: Item metadata spans numerous functional and aesthetic attributes, creating challenges for LLMs that must identify decision-relevant information.Real-world items may contain tens to hundreds of heterogeneous attributes.
- Limitations of prior work: Existing methods often inject titles or manually selected attributes, which provide incomplete item understanding and do not scale to vast metadata.Static summaries improve scalability but still struggle to identify item-specific salient features.
- Limitations of prior work: Static item descriptions also fail to capture user-dependent variation in feature salience.The same item information may matter differently across users and decision contexts.
- CAIRO: CAIRO structures metadata and reviews into objective features and subjective traits, then selects user-relevant information with a lightweight profiler.The profiler uses collaborative signals rather than fixed heuristics and includes a refinement process for inferred information.
2 Related Work
Prior LLM recommender research has explored prompt augmentation and textual profiling, primarily on the user side. Item-side methods remain constrained by limited attributes, simple summaries, and static profiles that do not personalize item relevance.
- Prompt augmentation: Prompt augmentation preserves pretrained LLM generalization while incorporating auxiliary user and item information into recommendation prompts.Prior work uses histories, reviews, demographics, selected attributes, and graph-derived information.
- User profiling: User profiling has progressed from static summaries toward adaptive profiles that reflect updates, evolving preferences, and tailored formats.These approaches compress historical information into textual representations for prompting.
- Item profiling: Item profiling has received comparatively less attention and commonly relies on fixed attributes or keywords summarized from descriptions and reviews.EXP3RT adds review-based preference information, while M-LLM3Rec uses additional item phrases and attributes.
- Research gap: Existing item profiles have limited ability to identify salient features from vast metadata and use one static profile for all users.This limits personalized judgments of item relevance in LLM reranking.
3 Problem Formulation
The paper formulates LLM reranking with user and item textual profiles, focusing on constructing compact item profiles specialized for each user. The formulation targets structured representation, item-specific salience, and user-context adaptation.
- Notation: Each user has an interaction history, while each item is associated with metadata and user reviews.The notation distinguishes user histories H_u, item metadata M_i, and review sets R_i.
- Problem setting: Real-world item metadata is large, inconsistently formatted, and heterogeneous, with important signals often buried in long descriptions.Attributes may be numerical, categorical, or ordinal and can span tens to hundreds of fields.
- Problem definition: The goal is to construct a compact item profile P_i|u = ϕ(M_i,u) specialized for user u.The profile should retain informative item-side evidence while fitting the recommendation context.
- Design objectives: The target profile must address limited context windows, item-specific salient features, and adaptation to user context.These are identified as challenges C1, C2, and C3.
4 Proposed framework: CAIRO
CAIRO organizes item-side knowledge into structured representations and uses a lightweight profiler to select context-relevant information online. An optional refinement stage further improves recommendation quality.
- Framework overview: CAIRO first organizes vast item-side knowledge into structured representations.This provides the foundation for selecting useful item information.
- Framework overview: A lightweight profiler selects information relevant to the current user’s context.The profiler supports context-specific item profiles with relatively negligible online latency.
- Framework overview: CAIRO includes an optional refinement stage intended to further enhance recommendation quality.The refinement follows the core profiling and online profile-generation stages.
4.1 Structuring Item Information
CAIRO structures heterogeneous item metadata and reviews into objective features and multi-faceted subjective traits. It derives domain-specific keys from clustered metadata, extracts evidence-grounded feature values, and represents diverse review perspectives in a structured form.
- Knowledge organization: CAIRO organizes item knowledge into objective features from metadata and subjective traits inferred from reviews.Objective knowledge is factual and user-invariant, whereas subjective knowledge captures perceptions that may differ across users.
- Domain-specific key exploration: Domain-specific keys are generated from metadata clusters, then reassessed and merged to cover diverse item characteristics.The process uses clustered candidate keys and merges semantically similar keys rather than relying on frequency filtering.
- Objective item features extraction: The unified schema extracts each item’s feature values from metadata, including attributes embedded in textual descriptions.The extraction process verifies supporting evidence and assigns null when evidence for a key is unavailable.
- Subjective item traits extraction: Subjective traits summarize review aspects as a multi-faceted set covering distinct perspectives such as functionality, aesthetics, and brand affinity.Each item has 3–7 traits, with supporting objective features linked to each trait.
- Subjective item traits extraction: Generated subjective traits may deviate from true user intents because they depend on LLM reasoning, motivating an auxiliary refinement process.Objective features are obtained by identifying corresponding values, whereas subjective traits are inferred from reviews and reasoning.
4.2 User Context-aware Item Profiler
CAIRO’s lightweight profiler selects item information according to the current user’s context instead of sending all structured information to the LLM. It learns objective-feature importance from collaborative signals and selects subjective traits by semantic alignment with the user profile.
- Motivation and overview: Providing all structured item information is suboptimal because decision-relevant aspects vary across items and users, while online LLM selection adds latency.CAIRO therefore prepares the profiler offline for efficient online selection without additional LLM calls.
- Motivation and overview: The profiler uses separate submodules to select objective features and subjective traits based on the current user’s information.This produces a user-specific item profile for downstream reranking.
- Objective feature selector: The backbone recommender encodes item features, the user profile, and user/item IDs into a concatenated user–item representation.The representation combines modality-specific feature embeddings with user-profile and ID embeddings.
- Objective feature selector: A controller estimates feature importance from collaborative patterns, reweights the representation, and is jointly optimized with the recommender using binary cross-entropy.The learned weights quantify which objective features are predictive for each user–item pair.
- Feature and trait selection: The objective selector uses k-max pooling to retain the top-4 important features, while the subjective selector chooses the trait most semantically aligned with the user profile.Collaboratively learned weights can capture feature combinations, and trait alignment is computed from user-profile and trait embeddings.
4.3 Online Prompt Construction
CAIRO constructs the final reranking prompt by selecting context-relevant objective features and subjective traits for each candidate item. It also includes objective features supporting the selected trait, producing profiles that vary across user–item contexts.
- Construction process: The online construction process takes user profiles, histories, candidates, objective features, and subjective traits, and outputs a final reranking prompt.The selected profiles are assembled before standard LLM-based reranking instructions are applied.
- Construction process: Objective features are selected by the profiler through row-wise pooling, while subjective traits are selected using cosine similarity with the user profile.These selections are performed for each candidate item in the batch.
- Profile assembly: CAIRO additionally includes objective features that directly support the selected subjective trait before assembling the final prompt.This links the selected subjective explanation to factual item evidence.
- Profile assembly: Each item profile contains around five objective features in total.The profile size is kept concise while remaining specialized to the user’s context.
- Context adaptation: Different users receive different subjective traits and objective features for the same item, such as compatibility and rating for one user versus age rating for another.The example illustrates adaptation of both profile components to each user–item decision context.
4.4 Feature-grounded Trait Refinement
CAIRO refines subjective item traits offline through error-case collection, defect diagnosis, and feature-grounded revision. It selects the revision most aligned with associated users, improving preference-relevant trait content while preserving objective-feature faithfulness.
- Trait refinement: CAIRO refines subjective traits offline as an auxiliary step, grounding revisions in objective item features to preserve profile faithfulness.The process addresses weaknesses in inferred traits while keeping refinement separate from serving.
- Refinement pipeline: The refinement pipeline collects proxy-task error cases, diagnoses recurring trait deficiencies, and uses those diagnoses to guide revision.Simplified next-item prediction supplies failures for each item-trait pair; multiple diagnoses capture recurring patterns.
- Candidate selection: The LLM generates multiple candidate revisions and selects the trait with highest cosine similarity to the average embedding of corresponding user profiles.The method sets N_cand = 2 and chooses the candidate most aligned with associated users.
- Outcome: Figure 3 indicates that refinement better captures performance-related aspects aligned with user preferences while retaining objective feature grounding.The example diagnoses overemphasis on general brand reliability and redirects the trait toward performance-related information.
5 Experiments
CAIRO is evaluated against traditional, feature-based, LLM-based, and retrieval-augmented baselines on heterogeneous Amazon datasets. It consistently improves reranking while reducing profiling latency relative to iterative RAG profiling.
- Datasets and setup: Experiments use Amazon Video Games, Sports and Outdoors, and Electronics datasets with heterogeneous metadata and reviews.The datasets are sampled at 200K–300K interactions and filtered to users and items with at least five interactions.
- Overall results: CAIRO consistently outperforms all baselines across datasets and metrics, with the largest gains over LLM-based rerankers at K=5.CAIRO also outperforms baselines without refinement, while CAIRO + Refine adds consistent further gains.
- Overall results: Feature-enhanced variants of existing LLM-based methods degrade across all datasets and metrics, often falling below title-only LLMRank.The result indicates that injecting raw metadata without structuring or selecting it can harm reranking.
- Efficiency: CAIRO achieves a significant profiling-time reduction relative to REAP while retaining competitive performance.The latency advantage comes from an offline-prepared, LLM-free profiler that uses lightweight matrix operations at serving time.
- Efficiency: CAIRO’s performance gains over EXP3RT incur relatively negligible additional computational cost.CAIRO has marginally more offline LLM calls, while online serving latency remains comparable to EXP3RT without online profiling.
5.3 Study of CAIRO
CAIRO’s ablations examine how feature and trait construction and selection affect recommendation, while transfer experiments test profiles across LLM families and scales. Case studies illustrate that profiles adapt selected evidence to different users.
- Trait construction: Generating only one subjective trait per item degrades performance, indicating that diverse trait candidates support context-aligned selection.The ablation describes user opinions as multi-faceted and evaluates selecting among multiple trait candidates.
- Feature/trait selection: CAIRO achieves the best performance in the reported Video Games and Electronics ablations.The comparison includes alternative feature-selection strategies, including random and all-feature variants.
- Feature/trait selection: Indiscriminate feature injection performs worse than context-aware selection because it distracts the LLM rather than merely adding redundant information.This pattern is reported across the evaluated LLM-based recommendation settings on Electronics.
- Transferability to Other LLMs: CAIRO consistently outperforms both baselines across four LLMs from the Qwen and Llama families, spanning 3B to 32B parameters.The results support transferability across model families and scales.
- Case study: For the same item, CAIRO selects performance-related information for a user criticizing functionality and brand-specific information for a user seeking ecosystem compatibility.The case study contrasts user-specific profiles with EXP3RT’s static description.
6 Conclusion
The conclusion presents CAIRO as a framework for organizing vast item information and selecting user- and item-context-relevant evidence for LLM reranking. Experiments support the value of this profiling strategy and its broader item-side focus.
- 6 Conclusion: CAIRO structures raw metadata and reviews into objective features and subjective traits.The framework addresses item profiling from vast metadata and item/user context-relevant information.
- 6 Conclusion: Lightweight selectors construct user-specialized item profiles with reduced serving-time overhead.The selectors choose item information relevant to the current context.
- 6 Conclusion: Extensive experiments show that CAIRO consistently outperforms existing baselines.The conclusion attributes the supported result to organizing and adaptively selecting item information.
GenAI Usage Disclosure
The disclosure distinguishes models used in the reported experiments from generative AI tools used during manuscript preparation and code development. The listed uses are limited to the stated settings and editing or debugging support.
- Experimental use: GPT-4o-mini, Llama3.1, Llama3.2, and Qwen2.5 were used as methodological components within the reported experiments.The disclosure states that these models were used only in the experimental settings specified in the work.
- Manuscript preparation: ChatGPT and Claude were used during manuscript preparation for grammar, spelling, and typo correction.Their manuscript role is described as minor language editing.
- Code development: Generative AI tools were also used to suggest debugging directions during code development.This use is separately identified from the experimental model components.