On Natural Language User Profiles for Transparent and Scrutable Recommendation

1 INTRODUCTION

• The paper proposes replacing difficult-to-interpret user representations with natural-language profiles that people could inspect and edit.The profiles are intended to support transparency and control, including distinguishing future preferences from past behavior.Source: §1 · p. 1
• The approach separates generating a reliable, complete, succinct user-interest summary from recommending items based exclusively on that summary.The recommendation task must capture the breadth of a person’s interests rather than match a long-form query’s set of conditions.Source: §1 · pp. 1–2
• The authors’ goal is to collate evidence for feasibility, expand on open challenges, and motivate further research rather than present an immediate solution.They argue the challenges are worth investigating for their potential larger value to society.Source: §1 · p. 2
• The paper contributes a formal proposal, a tractable task decomposition, open research challenges, and a qualitative feasibility study based on recent language-model advances.Source: §1 · p. 2

2 OVERVIEW

• The proposal makes natural-language preference summaries the basis for personalization so users can scrutinize representations, correct errors, express aspirations, and potentially bootstrap recommendations.It also frames the summaries as portable knowledge users could choose to share with other services.Source: §2 · p. 2

2.1 Main Benefits over Existing Approaches

• User-level summaries could reveal which interests inform recommendations and support item explanations grounded in specific profile statements.Editable summaries may also let users remove incorrect inferences or undesirable criteria, but meaningful adoption depends on low-effort inspection and modification.Source: §2.1 · p. 2
• Feedback on correct and incorrect summary aspects, preference changes, future aspirations, and user-written interests are proposed as potential routes to more useful recommendations.Source: §2.1 · pp. 2–3

2.2 Research Context

• Advances in natural-language interpretation, generation, and reasoning, alongside datasets pairing language with item-based user knowledge, make the proposed approach plausible to investigate.Source: §2.2 · p. 3

2.3 Main Components

• Scrutable user-model generation produces a length-bounded natural-language preference summary from interactions and item metadata, while scrutable NL-based recommendation ranks candidate items from that summary.The summary must remain short enough for meaningful user scrutiny, and recommendation can be evaluated as a top-N ranking task.Source: §2.3 · pp. 3–4
• The two components require joint development because summaries must be interpretable, modifiable, and recommendation-effective, while recommendation quality depends on the supplied summaries.Source: §2.3 · p. 4

3.1.1 Fluent Generation: From Templates to Large Language Mod-

• Recent natural-language generation advances could replace template-based summaries with more nuanced preference statements, while parameter-efficient tuning suggests a possible bridge from vector models to text.The cited work notes that tuning as little as 0.01% of a model’s parameters can provide quality comparable to full fine-tuning.Source: §3.1.1 · p. 4

3.1.2 Effective User Characterization.

• Natural-language summaries can express preference intensity and extend beyond fixed tag vocabularies, although users may find it easier to state preferences about item sets than individual items.Prior work cited here found that new users completed preference elicitation more effectively using groups of items than rating individual items.Source: §3.1.2 · p. 4

3.1.3 User Control.

• User control can occur during preference elicitation or after recommendations appear, but too many controls increase cognitive load; summary edits are intended to show results immediately.The discussed recommender still uses underlying tag weights, which are not contained in its textual summary.Source: §3.1.3 · pp. 4–5

3.2.1 Evaluating NL Summaries.

• Evaluate generated summaries by recommendation quality across different amounts of interaction history, and assess whether users can understand, edit, and correctly interpret them.Evaluation should also consider the effects of edits and whether the interface makes scrutiny effective without excessive effort or delay.Source: §3.2.1 · p. 5

3.2.2 Evaluating Explanations.

• Evaluation criteria for item-level explanations, including transparency, scrutability, effectiveness, and perceived personalization, should also be adapted to user-level summaries.Source: §3.2.2 · p. 5

3.2.3 Existing Datasets.

• Existing evaluation options include preference-elicitation studies and template-based summaries that users correct and compare by recommendation quality; narrative-driven datasets are another possibility.Source: §3.2.3 · p. 5

3.2.4 Limitations and practical considerations.

• Stated preferences may diverge from observed choices in either direction, and potential language-model biases or harms also warrant investigation.Characterizations associated with items should be checked for correctness.Source: §3.2.4 · p. 5

4.1.1 Filtering.

• Unlike early keyword-based filtering profiles, scrutable natural-language recommendation is not limited to text documents or item streams, and the authors argue that language advances warrant revisiting profile editing.Source: §4.1.1 · pp. 5–6

4.1.2 Verbose Query Retrieval.

• Verbose-query retrieval typically targets acute, focused needs, whereas a recommendation profile describes a broad disjunction of preferences and overlapping item clusters that support diversity and serendipity.Source: §4.1.2 · p. 6

4.1.3 Narrative-driven Recommendations.

• Narrative-driven recommendation uses transaction logs alongside narrative descriptions, while the proposed approach relies solely on the natural-language summary and could use conversational feedback to update it.Source: §4.1.3 · p. 6

4.1.4 Large Language Models.

• Recent language-model examples motivate treating a natural-language summary as a shared, scrutable object; the proposal seeks good recommendations that users can inspect and edit, not necessarily better performance.Related work has also generated movie recommendations from free-form descriptions and extracted relevant text such as review sentiment.Source: §4.1.4 · p. 6

4.2.1 Evaluation Criteria.

• Evaluate rankings with standard retrieval metrics such as reciprocal rank and nDCG, while accounting for recommendation utility’s dependence on prior item experience and potentially diversity.Source: §4.2 · p. 7, §4.2.1 · p. 7

4.2.2 Existing Datasets.

• Available datasets include narrative-driven book and point-of-interest recommendations, but their narratives concern ephemeral needs rather than standing preferences; another close dataset used human recommendations from 60 candidates.Source: §4.2.2 · p. 7

• The authors return to previously raised challenges and outline a research agenda toward realizing the benefits of scrutable recommendation.Source: §5 · p. 7

5.1.1 Characteristics of NL Summaries.

• Preference summaries need language that is fluent, precise, and aligned with users’ terminology, while capturing detailed tastes and possible meta-preferences.User-generated descriptions may lack specificity and often omit metadata such as actor and director names.Source: §5.1.1 · p. 7

5.1.2 Cold-start Recommendations.

• For new users, it remains unclear how to solicit useful preferences efficiently; for new items with few reviews or details, generalizing from a stated item preference is especially challenging.Source: §5.1.2 · p. 7

5.1.3 Cognitive Load.

• Text’s assumed low cognitive load is an open question; concise, consistent, or selectable phrasing may help, and multimodal representations may also preserve scrutability while improving performance or reducing effort.Source: §5.1.3 · p. 7

5.2.1 Managing Updates of Representation.

• Profiles may separate user-confirmed preferences from unvalidated inferences, then use both in recommendations and explanations to support freshness, transparency, and control.The system must recognize preference changes and either adjust the profile automatically or request confirmation.Source: §5.2.1 · p. 8

5.2.2 Recognizing New Contexts.

• A new location may indicate a short-term context, while observations over time may support a broader one; aggregating preferences across contexts remains challenging.The travel example contrasts “when in Hawaii” with “when on vacation.”Source: §5.2.2 · p. 8

5.2.3 Refining User Summaries.

• As expertise develops, users may refine generic interests into narrower tastes, but how systems should derive and represent suggested refinements remains open.The example follows music interests from a genre to a sub-genre, era, or specific artists.Source: §5.2.3 · p. 8

5.2.4 Realizing Dislikes.

• Explicit dislikes differ from preferences about unfamiliar items, raising a trade-off between negative preferences, positive detail, scrutability, and serendipity or diversity.The section asks how negative preferences trade off against more detail about positive preferences.Source: §5.2.4 · p. 8

5.3.1 Interpreting Language Robustly.

• Recommendation from natural-language summaries requires robust mapping from nuanced terms to concepts, including resilience to wording changes, ambiguity, and subjective meanings.Sparse text for an individual can make interpretation errors consequential; systems might resolve misunderstandings or state assumptions explicitly.Source: §5.3.1 · p. 8

5.3.2 Explaining Recommendations.

• User-validated language may support explanations using a person’s terminology and preferred style, while explanations can offer recourse by exposing profile details that are too broad.The section notes that interaction with explanations for recourse has received limited attention.Source: §5.3.2 · p. 8

5.3.3 Practical Utility.

• Short summaries may omit previously seen items, so systems may need side information for novelty while deciding how to use it without undermining transparency and control.An open challenge is evaluating limited observational uses, such as filtering repeated items, against recommendation use and the added scrutiny and modification complexity.Source: §5.3.3 · p. 8

5.4.1 Trading off Competing Goals.

• Profile goals such as concision and scrutability compete with recommendation goals such as accuracy, novelty, and diversity, and users may prioritize this trade-off differently.The authors hypothesize that a single learned model could allow per-person adjustment of recommendation quality versus scrutability.Source: §5.4.1 · p. 9

5.4.2 End-to-end Optimization.

• End-to-end training could potentially learn natural-language user profiles while encoding desired properties in the optimization loss, rather than requiring a two-part process.The proposal builds on work showing controllable recommendation attributes with a β-VAE.Source: §5.4.2 · p. 9

5.5.1 Task-Specific Time/Data Efficiency.

• Evaluation could compare the time and effort to produce or update a natural-language summary with the ratings needed for similar recommendation effects.The comparison may be especially relevant when preferences shift; the authors suggest summary curation could narrow a possible non-cold-start performance gap.Source: §5.5.1 · p. 9

5.5.2 Coverage versus Specificity.

• Evaluation must weigh recommendations that precisely represent part of a user’s interests against those that cover all interests less precisely.The section notes that Top-N recommendations typically favor the former, while measuring the fraction of interests represented is harder.Source: §5.5.2 · p. 9

5.5.3 Types of Utility and Cognitive Effort.

• Evaluation should consider both recommendation quality and language utility, including user cognitive effort, time to improve, and how edits affect recommendations.The system may find some description elements useful while users find other inputs easier to generate or edit; possible supports include suggested phrases, ambiguity flags, or questions.Source: §5.5.3 · p. 9

6 FEASIBILITY STUDY

• LaMDA generated recommendations from natural-language summaries using few-shot prompt templates and top-K decoding, without fine-tuning.The prompts used three summary–recommendation examples; the templates were crafted, but summaries and outputs were not cherry-picked.Source: §6 · pp. 9–10
• Table 3 shows the first four recommendations from one run for each of three example summaries, alongside explanations that reference the summaries.Source: §6 · p. 10, Table 3 · p. 10
• The model produced appropriate recommendations and personalized explanations, while search-engine results were often less specific and lacked explanations.For some genre-like queries, search-engine suggestions reasonably overlapped with the model’s recommendations.Source: §6 · p. 10
• The examples use short, single-aspect summaries rather than holistic profiles, and may not meet users’ preferences for novel recommendations.The authors note that performance with realistic descriptions of broad interests matters more.Source: §6 · p. 10
• The model may skew toward popular U.S. movies, handle precise release-decade constraints less accurately, and hallucinate details.One cited example incorrectly says the killer in Se7en is revealed at the beginning.Source: §6 · p. 10

7 CONCLUSIONS

• The paper proposes encoding systems’ knowledge of users in natural language, a direction made conceivable by recent advances in language generation and understanding.Source: §7 · p. 10
• A process for people to review and update their representations could improve transparency, user control, and recommendation quality.Source: §7 · p. 10
• The paper decomposes scrutable recommendation into more tractable steps and identifies promising advances and open challenges for the research community.Source: §7 · p. 10