Source-linked AI summary
Bootstrapping Conversational Recommendation Agents At Spotify: Synthetic Data Generation and Self-Improvement Loops
Enrico Palumbo, Alexandre Tamborrino, Victor Ode, Ben Lacker, Adrià Casas Escoda, Jeremy Hopple, Marcus Better, James Leoni, Hugo Galvão, Hugues Bouchard, Mounia Lalmas, José Luis Redondo García, Abenezer Abebe, Ann Clifton, Anton Blomberg, Henrik Lindström, Dani Doro, Christine Doig Cardet
TL;DR
Cold-start development of conversational recommendation agents lacks realistic multi-turn data and reliable evaluation signals. The paper generates synthetic conversations and uses a variance-informed self-improvement loop to improve planning; online tests report increased listening and weekly active users and reduced skips versus a session-refinement-only experience.
Problem
Method
Results
Online tests showed more than 14% additional user listening, 5% higher weekly active users, and over 5% lower skip rate versus the prior session-refinement-only experience.
Source: §5 · p. 9Takeaways & Limitations
The approach supported Spotify’s conversational-agent launch and accelerated planning iteration, with human review retained before deployment.
Source: §5 · p. 8The current taxonomy does not yet cover informational or meta-conversational intents, and real production traffic remains a planned data extension.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Conversational recommendation agents are a new paradigm for content discovery, enabling users to express complex intents through natural language (e.g., "recommend Italian indie artists I haven't heard before"). A central challenge in building such agents is optimizing agent planning -- deciding how to select, sequence, and invoke tools -- particularly in cold-start settings where real user interactions are not yet available. We introduce a pipeline for multi-turn synthetic data generation and a self-improvement loop to address this challenge. The synthetic data pipeline transforms single-turn prompts into realistic multi-turn conversations, enabling systematic evaluation before launch. The self-improvement loop combines variance-based contrastive optimization with iterative refinement through a coding agent, automatically identifying and fixing planning and tool-use errors. Our approach improves quality by +8% on top of a highly optimized manual prompt. The system has been productionized and significantly accelerated iteration cycles for the launch of a conversational recommendation agent at Spotify. Online A/B tests demonstrate its effectiveness, with +14% user listening, +5% increase in weekly active users, and a 5% reduction in skip rate compared to a prior experience supporting only session refinement. This work provides a practical framework for accelerating the development of conversational recommendation agents in industry.
1 Introduction
Conversational recommenders support complex intents and iterative refinement, but cold-start development lacks realistic multi-turn data and reliable evaluation signals. The paper combines synthetic conversation generation with a self-improvement loop to evaluate and improve agent planning.
1 Introduction
- The cold-start challenge is that multi-turn conversational data is typically unavailable before launch, while existing Spotify experiences largely support single-turn requests and simple session refinement.Conversational agents can support complex intents, follow-up refinement, content questions, and contextual explanations.Source: §1 · p. 2
- The approach combines a synthetic conversation pipeline with a coding-agent loop that uses evaluation variance and contrastive plan comparisons to improve planning.The loop also uses iterative refinement for harder failure cases.Source: §1 · p. 2
- +8% quality improvement over a highly optimized prompt accompanies human annotation scores above 90% across all quality dimensions.Evaluation also found performance varied across capabilities and degraded beyond four turns.Source: §1 · p. 2
- Online A/B tests showed significant user-engagement improvements, and the methods were instrumental in launching Spotify’s conversational recommendation agent.The approach is presented as practical and generalizable for data-scarce development.Source: §1 · p. 2
2 Related Work
Related work frames conversational recommendation as multi-turn information seeking and distinguishes agentic systems from retrieval-generation approaches. This paper addresses gaps in multi-turn data and applies self-improvement methods in a production setting.
Conversational Recommendation
- Conversational recommendation supports recommendation-related goals through multi-turn dialogue [12], including preference elicitation, recommendations, explanations, and follow-up questions [12].Source: §2 · p. 2
- LLM-based systems include generative retrieval [11] [19] [21] [22] and agentic approaches that call existing recommendation services as tools [10]; this work focuses on the latter.Earlier systems commonly coordinated specialized components with hand-crafted rules [12].Source: §2 · p. 2
Conversational Data Generation
- Existing music and playlist recommendation datasets focus on recommendation as the sole intent, while this pipeline covers refinement, follow-up questions, and intent shifts.The pipeline seeds multi-turn conversations from existing single-turn intents.Source: §2 · p. 2
Agent Self-Improvement
- This work uses synthetic multi-turn conversations to surface failure modes, analyzes variance across sampled plans, and uses a coding agent to refine prompts and tool definitions.It positions this production-setting approach as complementary to prompt and agent-harness optimization methods [24] [2] [26] [16] [1].Source: §2 · pp. 2–3
3 Approach
The approach pairs multi-turn conversation generation with agent self-improvement to support evaluation and optimization in cold-start scenarios.
3 Approach
- The approach generates multi-turn conversations for conversational recommendation agents.Source: §3 · p. 3
- It also applies agent self-improvement to conversational recommendation agents.Source: §3 · p. 3
- Together, the methods enable evaluation and optimization in cold-start scenarios.Source: §3 · p. 3
3.1 Multi-turn Conversation Generation
The pipeline turns single-turn prompts into capability-targeted multi-turn conversations, using plans to control what is exercised and live-agent interactions to ground realized turns. A capability-specific judge scores the final turn.
3.1 Multi-turn Conversation Generation
- Synthetic conversations address missing multi-turn traffic in cold-start settings; the pipeline separates the dialog plan from its realization with the live agent.Plans can be audited, reused across evaluations, and adapted from curated cases or future real conversations; Figure 2 depicts generation and judging.Source: §3.1 · p. 3, Figure 2 · p. 4
3.1.1 Taxonomy.
- The five-capability taxonomy gives each capability a name, judge rubric, and few-shot examples that guide both plan generation and capability-aligned evaluation.Table 1 describes the five tasks and illustrates them with representative dialog plans.Source: §3.1.1 · p. 3, Table 1 · p. 3
3.1.2 Single-Turn Data.
- Seed prompts combine existing Spotify traffic with synthetic prompts for underrepresented but important use cases, and their train/validation/test splits propagate to the generated data.Source: §3.1.2 · pp. 3–4
3.1.3 Dialog Plan Generation.
- For each capability, seed prompt, and target length, the DialogPlanLLM creates a turn-by-turn plan, with the final turn defining the behavior under evaluation.Conversation lengths are sampled from 2 through a maximum as a soft generation constraint, and split assignment follows the seed prompt.Source: §3.1.3 · p. 4
3.1.4 Conversation Realization.
- A UserLLM realizes the plan through closed-loop interaction with the live agent, minimally adapting planned user turns when needed to preserve the intended capability.This grounding retains actual agent responses and structured outputs rather than replaying synthetic agent turns.Source: §3.1.4 · pp. 4–5
3.1.5 LLM-as-a-Judge.
- A capability-specific LLM judge evaluates only the final agent turn using prior turns as context, returning a green or red label and an explanation for error analysis.Scoring every turn would increase cost and latency linearly with conversation length; varying lengths enables aggregate assessment across horizons.Source: §3.1.5 · p. 5, §3.1.4 · p. 5
3.2 Self-Improvement Loop
The self-improvement loop samples and evaluates plans, attributes failures, then applies different fixes for inconsistent planning and missing capabilities. It uses contrastive optimization when successful plans exist and coding-agent refinement when they do not.
3.2 Self-Improvement Loop
- The loop samples agent plans, evaluates them with an LLM judge, attributes failures through variance analysis, and applies targeted improvements through a coding agent.A coding agent can read execution and evaluation traces and edit prompt artifacts, including system prompts, examples, and tool descriptions.Source: §3.2 · p. 5
3.2.1 Sampling.
- Sampling multiple plans for one query yields judge-labeled executions; all-pass queries need no action, mixed outcomes proceed to diagnosis and contrastive optimization, and all-fail queries go to refinement.Plans are sampled at elevated temperature, τ ∈ [0.7, 1.0].Source: §3.2.1 · p. 5
3.2.2 Root Cause Detection.
- Root-cause detection compares successful and failing executions using trace parsing and LLM reasoning, separating planning differences from tool failures, tool-output variance, and judge variance.Only planning-tool, planning-argument, or action failures enter contrastive optimization; other causes are logged separately.Source: §3.2.2 · p. 6
3.2.3 Contrastive Optimization.
- For planning failures where at least one sampled plan succeeds, the coding agent compares successful and unsuccessful plans and updates the prompt to reinforce patterns linked to success.Patterns may involve tool choice, ordering, or argument construction.Source: §3.2.3 · p. 6
3.2.4 Agentic Iterative Refinement.
- When sampling never produces a correct plan, iterative refinement has the coding agent analyze the failing trace and propose a targeted, minimal prompt change.Because this stage requires reasoning about missing capabilities and is more computationally expensive, it is reserved for cases where contrastive optimization is exhausted.Source: §3.2.4 · p. 6
3.2.5 Batch editing and validation.
- Batch editing applies one update across a random query mini-batch, then verifies it at temperature τ = 0 before committing it and continuing.The process ends with a pull request for human review and a full evaluation as a regression guardrail.Source: §3.2.5 · pp. 6–7
4 Results
The results validate the synthetic conversations for evaluation, reveal capability and conversation-length differences, and show gains from the self-improvement loop and online agent experience. The online test reports increased listening and weekly activity and reduced skip rate versus the prior session-refinement experience.
4.1 Multi-turn Evaluation Results
- All four synthetic-data quality dimensions scored above 90%, including 98% task demonstration, 93% realism, 94% applicable-case adaptation quality, and 97% judge accuracy.Instruction Retention was the only capability below 95%; some generated conversations repeated constraints across turns rather than testing retention.Source: §4.1 · p. 7
- Intent Shift was easiest, Anaphora Resolution performed next best, Separate Input followed, and Content Refinement and Instruction Retention were most challenging.Instruction Retention requires enforcing constraints introduced early across turns.Source: §4.1 · p. 7, Figure 5 · p. 8
- Evaluation exposed incorrect tool-call arguments that changed track ordering, lost constraints during session creation, and missing or incomplete metadata during content refinement.The issues were addressed through manual fixes and automated self-improvement-loop changes.Source: §4.1 · p. 7
- Performance stayed stable from 2 to 4 turns but degraded from 5 turns onward, as measured by pass@1 relative to 2-turn conversations.The authors identify context compaction and history summarization as future work.Source: §4.1 · p. 7, Figure 6 · p. 8
4.2 Self-Improvement Results
- +8% relative increase in pass@1 over a highly manually optimized prompt was statistically significant on the full test set (p=0.01).The combined changes came from one run of the self-improvement loop.Source: §4.2 · p. 8
- Variance-based root-cause analysis with contrastive optimization helped avoid spurious updates from noisy signals, while coding-agent refinement enabled fixes beyond prompts, including tool descriptions.All proposed changes were reviewed before merging.Source: §4.2 · p. 8
4.3 Online Results
- 14% additional user listening, 5% increase in weekly active users, and 5% reduction in skip-rate were observed versus the prior session-refinement experience.The two-week A/B test exposed approximately 15 million users across several markets and device types.Source: §4.3 · p. 8
5 Conclusion and Discussion
The paper presents synthetic multi-turn conversations and a self-improvement loop for bootstrapping conversational recommendation agents at Spotify. It reports gains in conversation quality, agent planning, and online engagement, while identifying long conversations and unrepresented intents as areas for further work.
5 Conclusion and Discussion
- The work combines synthetic multi-turn conversation generation with an evaluation-feedback loop that iteratively improves agent planning.The paper presents this as a practical framework for evaluating and improving conversational agents without real user interactions.Source: §5 · pp. 8–9
- Human annotation scored synthetic conversations above 90% across evaluation dimensions, while list manipulation under evolving constraints remained challenging for frontier models.Performance also degrades as conversations extend beyond four turns.Source: §5 · p. 8
- The authors identify long-context limitations and motivate context compaction that preserves key conversational details, such as recommended items.The proposed compaction would compress more generic verbal information.Source: §5 · p. 8
- +8% quality improvement over a highly optimized prompt; the self-improvement loop also uncovered concrete planning and tool-use issues.Contrastive optimization with variance-based root-cause analysis improved the loop’s signal-to-noise ratio, reduced hallucinations, and increased its efficiency.Source: §5 · pp. 8–9
- More than 14% additional user listening, 5% more weekly active users, and over 5% lower skip rate versus the prior session-refinement-only experience.Online experiments measured the practical impact of the conversational recommendation agent.Source: §5 · p. 9
- Planned extensions include informational and meta-conversational intents, real production traffic, and optimization of the broader agent pipeline.The current self-improvement scope is extended beyond prompts and tool definitions in the proposed future work.Source: §5 · p. 9