PaperlayerView exact revision on arXiv

Source-linked AI summary

Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation

Serafima Lebedeva, Sumantrak Mukherjee, Ali Arshad Sadal, Ilias Ekşi, Rahul Sharma, Julia Mueller, Theresa Dombrowski, Jakob Karolus, Viktor Bengs, Eyke Hüllermeier, Sebastian Vollmer

arXiv:2609.37800v1cs.LGcs.AI

TL;DR

New cohort members often arrive without interaction history, while finite per-user catalogs can become repetitive or depleted. CohortMix-TS transfers historical structure into metadata-conditioned priors and combines Thompson sampling with slate diversity and inventory controls; evaluations report improved early recommendations in supported settings and a larger early-to-late correctness change in a restricted deployment subset.

  • Problem

    Short-lived cohorts require learning user preferences from limited feedback while recommending from finite, non-repeatable per-user catalogs.

    Source: §1 · p. 1

  • Method

    CohortMix-TS transfers historical latent structure into metadata-conditioned priors for new users and uses Thompson sampling with diversity, depletion controls, and scheduled updates.

    Source: §1 · p. 1

  • Results

    Controlled environments and the 2024–2025 data-calibrated benchmark identify settings where warm starts improve early recommendation and pacing benefits a scarce arm; the field study found a larger early–late correctness change only in a restricted subset.

    Source: §8 · p. 8

  • Takeaways & Limitations

    Warm-start transfer and inventory-aware recommendations can support personalization for short-lived cohorts with finite catalogs.

    Source: §4.2 · p. 3, §8 · p. 8

    The field correctness result is conditional on post-assignment responses and does not support a population-wide causal claim about learning, interest, satisfaction, or retention.

    Source: §7 · p. 7

Abstract

from arXiv · show

Many recommender services repeatedly encounter cold-start cohorts, where new users arrive with little or no interaction history. This creates two challenges: learning user preferences quickly from limited feedback and sustaining useful recommendations when each user has a finite catalog that can become repetitive or depleted over time. We propose CohortMix-TS, a warm-started mixture bandit that learns latent user groups from earlier cohorts and uses available metadata to construct group-informed priors for new users. Starting from these fixed priors, the model personalizes independently as feedback from each user becomes available. Session slates combine Thompson sampling with diversity and inventory-depletion controls. We evaluate CohortMix-TS through simulation, semi-synthetic experiments, and a 25-day randomized in-the-wild deployment with 713 registered participants in a Campus Games quiz application. Our evaluations show that cross-cohort transfer improves early recommendation quality and user-level regret, while inventory-aware slate construction helps prevent premature exhaustion of preferred items. In the field deployment, treatment users also showed a larger early-to-late change in correctness than users receiving random recommendations. Together, these results show how warm-start transfer and inventory-aware recommendations can support personalization for short-lived, repeatedly cold-starting cohorts.

1 Introduction

Short-lived cohorts require useful personalization before much feedback is available, while finite per-user catalogs constrain repeated recommendations. CohortMix-TS transfers historical structure into uncertain metadata-conditioned priors and combines them with sequential slate selection and inventory controls.

1 Introduction

  • Finite per-user catalogs can be depleted by repeated selection of one item type, while long-horizon exploration may consume a substantial share of a short campaign.The introduction frames recommendation as sequential decision-making under limited interaction opportunities.Source: §1 · p. 1
  • CohortMix-TS transfers historical user and item structure into soft metadata-conditioned priors, then adapts as new-user feedback arrives.The priors avoid fixed group assignments, allowing users with the same metadata to belong to different preference groups.Source: §1 · p. 1
  • The setting combines cold-start users, delayed binary feedback, user-level no-repeat constraints, finite inventories, and changing active-user sets.Recommendations must become useful within a short cohort horizon.Source: §1 · p. 1
  • CohortMix-TS combines Thompson sampling with diversity and depletion controls and scheduled approximate hierarchical updates.The paper evaluates mechanisms in two parametric environments, a data-calibrated semi-synthetic environment, and a randomized 25-day deployment with 713 registered participants.Source: §1 · p. 2

2 Related Work

Prior work studies transferred structure, contextual and clustered bandits, and constrained slates, but the paper targets their combination in short-horizon cohort recommendation. Its policy transfers uncertain cohort structure, adapts during a campaign, and constructs feasible sessions from finite per-user catalogs.

Cold-start recommendation and transfer

  • Cold-start and warm-start bandits transfer shared structure, while CohortMix-TS uses historical factors for item arms and metadata-conditioned priors revised during the target cohort.Related approaches include meta-bandits, hierarchical bandits, and empirical-Bayes methods.Source: §2 · p. 2

Online, contextual, and clustered bandit recommendation

  • Contextual models use observed features and clustered bandits share information through inferred groups; the paper contrasts categorical metadata pooling with user-specific latent state.The study motivates randomization and careful outcome definitions because offline evaluation and treatment recommendations complicate causal assessment.Source: §2 · p. 2

Constrained slate recommendation

  • Slate-aware and diversified bandit methods address list interactions, while this policy selects at arm level and treats within-arm re-ranking as complementary.Its availability constraints arise from per-user no-repeat rules rather than a globally disappearing catalog, and it uses a penalty score rather than claiming an exact constrained-bandit solution.Source: §2 · p. 2

Synthesis

  • The practical gap is combining uncertain cohort transfer, short-campaign adaptation, and feasible multi-item sessions from finite per-user catalogs.The paper positions these requirements together rather than as isolated recommendation problems.Source: §2 · p. 2

3 Problem Formulation

The formulation models cohorts whose new members have metadata but no current-cohort outcomes, and whose active membership and item availability can change over time. It evaluates policies by cumulative cohort reward, with simulated regret and field performance-change measures used where appropriate.

3.1 Cohorts, information, and indexing

  • The catalog stays fixed across rounds, but a user’s available subset changes after items have been displayed to that user.The cohort’s active set can shrink when users leave or do not complete a round.Source: §3 · p. 2, §3.1 · p. 2
  • A cohort consists of users joining in the same period; each new user has observed metadata but no outcome from the current cohort.Historical records contain displayed items and binary outcomes from earlier cohorts.Source: §3.1 · p. 2

3.2 Cross-cohort slate environment

  • Each active user receives a slate of K distinct items, with binary outcomes observed after the slate is served.The final slate is a set; display order is not part of the model.Source: §3.2 · p. 2

3.3 Objective and evaluation

  • Cumulative reward over the cohort horizon is the primary optimization metric.In synthetic environments, known generated reward probabilities also support pseudo-regret evaluation.Source: §3.3 · p. 2
  • Policies select feasible slates using user metadata, historical data, and outcomes observed before the round.The objective is useful choices for users throughout the cohort horizon.Source: §3.3 · p. 2
  • Because field reward probabilities are unavailable, the field comparison uses personalized and random groups’ changes in observed performance.Participation and completion are secondary measures; users may leave or complete only some rounds.Source: §3.3 · p. 3

4 CohortMix-TS: Warm-Started Mixture Bandits

CohortMix-TS uses earlier cohorts to initialize metadata-conditioned arm beliefs, then constructs feasible slates that balance sampled reward, within-slate variety, and remaining inventory. Feedback is incorporated at scheduled checkpoints through soft membership and belief updates.

4.1 Historical structure and cold-start priors

  • Historical response records are factorized and clustered to form item arms and latent user groups for warm-starting new users.Metadata estimates a new user’s soft membership across latent groups; group-arm historical outcomes supply smoothed Beta priors.Source: §4.1 · p. 3
  • The metadata-conditioned mixture is summarized by a single Beta belief that preserves expected reward and gives each user-arm prior fixed historical strength.This first-moment match retains one Beta draw per arm for slate construction.Source: §4.1 · p. 3

4.2 Constraint-aware slate construction

  • At each slate step, Thompson draws are penalized for repeated arm use and consumption faster than the uniform t/T pacing plan, while infeasible arms are excluded.After choosing an arm, the policy samples uniformly from its available items and removes the item before the next step; repeated arm use remains possible when unseen items remain.Source: §4.2 · p. 3

4.3 Posterior adaptation across rounds

  • The model is held fixed while a round’s slates are served, then observed outcomes are aggregated at scheduled checkpoints to refresh memberships, shared pseudo-counts, and user beliefs.In deployment, completed-day answers were batched at the daily boundary, and membership refresh began after a user answered at least 10 items.Source: §4.3 · p. 4

5 Experimental Setup

The experiments compare warm-start and cold-start policies across controlled transfer and inventory settings, a year-split semi-synthetic benchmark, and a randomized deployment of the complete policy in Campus Games.

5.1 Baselines

  • The baselines include cold-start Thompson sampling, metadata-only contextual policies, hard and static transfer variants, and slate-control ablations alongside the full CohortMix-TS policy.Cold-start TS starts each user–arm belief at Beta(1, 1); metadata-only baselines pool feedback by metadata–arm cell, while the slate-control variants isolate diversity and depletion choices.Source: §5.1 · pp. 4–5

5.2 Three environments

  • Three environments test historical transfer, inventory-aware slate construction, and transfer under a real catalog and year-to-year split.The first varies historical–target alignment with slate controls inactive; the second starts from accurate beliefs and varies scarce-arm controls; the third uses 2024 source structure and 2025 target calibration, with a separate 98-item-arm scarcity diagnostic.Source: §5.2 · p. 5

5.3 Randomized in-the-wild deployment

  • The 25-day randomized Campus Games deployment compared the complete policy with random allocation among 713 registered participants.The quiz presented 10 items daily from a 1,347-item catalog; 859 prior users supplied transferred structure, and assignment at registration allocated 356 users to treatment and 357 to control.Source: §5.3 · p. 5

6 Results and Empirical Analysis

The results indicate that cross-cohort transfer is most useful when historical preferences align with the target, while depletion controls trade early concentration for later availability. The field correctness comparison favored treatment in a restricted responder analysis, with interpretation limited by post-assignment filtering.

6.1 Cross-cohort transfer depends on prior alignment

  • With aligned historical and target preferences, CohortMix-TS had the strongest early and campaign reward and lowest high-regret tail; under severe mismatch, its high-regret-user advantage disappeared.Metadata-only policies leveled off when a faculty contained users with opposing hidden preferences, while soft memberships allowed feedback to resolve initial latent-type uncertainty.Source: §6.1 · p. 6, Table 1 · p. 6

6.2 Slate controls trade early exploitation for later feasibility

  • Depletion variants paced use of the scarce preferred arm through the final week, and the full controller had stronger later reward than Diversity only after accepting less immediate concentration.The trade-off weakens when preferred inventory is less scarce, so depletion is useful for the stated short-inventory setting rather than universally reward-maximizing.Source: §6.2 · p. 6

6.3 Year-split data-calibrated semi-synthetic benchmark

  • In the year-split benchmark, CohortMix-TS improved on Cold-start TS, while metadata-only policies could not separate individual preferences within a faculty.Hard membership removed adaptive uncertainty about type, and a global prior removed the metadata signal; the separate real-catalog diagnostic showed early reward gains followed by loss of late slate quality when the preferred arm was exhausted.Source: §6.3 · pp. 6–7, Table 2 · p. 7

6.4 Randomized in-the-wild deployment

  • The correctness comparison is not an intention-to-treat estimate for all registrants because the analysis retained only participants with at least 30 answered items after assignment.The analysis set contained 211 users: 109 treatment and 102 control.Source: §6.4 · p. 7
  • Mean early-to-late correctness change was 0.0455 for treatment and −0.0168 for control, a difference of 0.0623 (bootstrap 95% interval [0.005, 0.119]; p=0.043).Only 23 treatment and 29 control users had outcomes in both windows; broader responder analyses weakened the evidence, and post-assignment filtering limits interpretation.Source: §6.4 · p. 7

7 Discussion and Limitations

The restricted field analysis found a larger early–late correctness change for treatment users, but correctness evidence is conditional on post-assignment responses. The discussion limits the interpretation to this subset and notes constraints in prior calibration, pacing, and benchmark breadth.

7 Discussion and Limitations

  • The field correctness result does not support a population-wide causal claim about learning, interest, satisfaction, or retention.Nonresponders have no correctness value, no pre-assignment quiz outcome was available, and no pre-specified missing-data or longitudinal model was fitted.Source: §7 · p. 7
  • A fixed prior concentration requires calibration to transfer alignment, and the study evaluated only a uniform pacing target and one deployment value.Stronger priors can help under aligned transfer but harm under severe source–target mismatch; the real-catalog stress does not establish a general depletion-controller weight.Source: §7 · p. 7
  • The comparison set is mechanism-specific rather than a broad recommender benchmark, and the arm partition remained fixed.The study did not compare contextual models with common user–item features or hierarchical Bayesian TS; revised arms require principled prior transfer.Source: §7 · p. 7
  • 0.062 was the treatment–control difference in early–late correctness change in the restricted complete-window analysis (95% interval [0.005, 0.119]; p = 0.043).The analysis included 23 treatment and 29 control users; the result is conditional on post-assignment responses.Source: §7 · p. 7, Figure 2 · p. 8

8 Conclusion

The paper develops and deploys CohortMix-TS for short-horizon recommendation with finite inventories, transferring historical structure into soft priors. Controlled evaluations identify settings where warm starts improve early recommendation and pacing benefits a scarce arm, while the in-the-wild correctness change was observed only in a restricted subset.

8 Conclusion

  • CohortMix-TS transfers historical structure into soft priors for short-horizon, finite-inventory cohort recommendation and was deployed as a complete policy in a randomized campaign.The conclusion frames the application around short-lived cohorts.Source: §8 · p. 8
  • Controlled environments and a 2024–2025 data-calibrated benchmark identify settings where historical warm starts improve early recommendation and a scarce arm benefits from pacing.Source: §8 · p. 8
  • The in-the-wild study observed a larger early–late correctness change only in the reported restricted subset, with future work targeting richer representations, adaptive priors and pacing, and longer-running services.Source: §8 · p. 8

A Detailed Environments and Ablations

The evaluations vary source–target alignment and inventory scarcity in controlled and semi-synthetic environments, then examine CohortMix-TS in a randomized field deployment with incomplete follow-up.

Common protocol

  • Across generated cohorts, policies share metadata, item pools, and Bernoulli potential outcomes, with feedback observed only after each session; early and late mean sessions 1–5 and 19–25.Every user’s slate is constructed from the same pre-session state.Source: A Detailed Environments and Ablations · p. 8

Transfer environment and membership ablations

  • The transfer environment uses four metadata values, eight hidden groups, and five arms, with each metadata value placing prior mass .65 and .35 on groups with opposing preferred arms.It tests 480 target users over 25 sessions of 10 items, and compares CohortMix-TS with fixed-mixture, hard-membership, static-source, cold-start, and metadata-only alternatives.Source: A Detailed Environments and Ablations · p. 8

Sensitivity patterns

  • At κ=10, Full-minus-Cold P90 difference was +12.27 at χ=0 and −19.62 at χ=1; alignment chiefly determines whether historical structure protects high-regret users.In a separate 32-cohort high-alignment prior sweep, campaign reward improved as prior strength increased; the λ check did not identify .3 as uniquely preferred.Source: A Detailed Environments and Ablations · pp. 8–9

Finite-inventory ablations

  • Early exhaustion fell from 1.00 at δ≤1 to .27 at δ=2 when preferred-arm capacity was .10; at .20, it was zero for δ≥.5.At looser capacities, diversity-only policy can have higher campaign reward, making depletion control a pacing trade-off rather than a universal improvement.Source: A Detailed Environments and Ablations · p. 8, Table 3 · p. 9

Year-split data-calibrated semi-synthetic environment

  • In the year-split semi-synthetic benchmark, disjoint 2024 and 2025 records calibrate source groups and hidden target profiles; source–target group–arm correlation was .985.Policies do not receive the 2025 records or target labels; the transfer benchmark fixes κ=10 and λ=.3 with slate penalties inactive.Source: A Detailed Environments and Ablations · p. 9

Field deployment

  • The 25-day field deployment randomized 713 registered participants to CohortMix-TS or uniformly sampled unseen items in a daily 10-item quiz drawn from a 1,347-item catalog.The preceding cohort supplied historical records; treatment items were reserved over a rolling three-day horizon and released if unused.Source: A Detailed Environments and Ablations · p. 9

Field analysis

  • The all-faculty thresholded-responder difference was 0.034 (p=0.146), versus 0.033 (p=0.153) among complete-window responders.Complete-window response was 13.7% in control and 13.2% in treatment (p=0.913), which does not identify missing correctness outcomes.Source: A Detailed Environments and Ablations · p. 9

Historical variation conditional on metadata

  • Figure 3 shows source-only group–arm correctness residuals within metadata categories, but cannot independently validate latent grouping because the historical grouping procedure also used metadata.The figure covers three high-support metadata categories that split into multiple inferred groups; one cell has fewer than 30 observations.Source: A Detailed Environments and Ablations · p. 9, Figure 3 · p. 9

Ethical Considerations

The paper identifies fairness and privacy risks in personalization and recommends safeguards for monitoring, participation, and data sharing.

Privacy

  • Shared study data should remove identifiers, linkable dates, and small strata that could enable re-identification.The analysis uses de-identified event logs, minimizes retained metadata, and restricts access to personnel who need the data.Source: Ethical Considerations · p. 10

Fairness risks

  • Correctness optimization can favor easier items and create unequal experiences when historical metadata encode unequal access, preparation, or participation.Using metadata as an uncertain prior rather than a fixed assignment does not remove this risk.Source: Ethical Considerations · p. 10

Monitoring

  • Operators should monitor recommendation exposure, correctness, and participation by adequately sized privacy-preserving groups alongside aggregate outcomes.The authors make no subgroup-effect claims from the deployment’s restricted complete-window analysis of 52 users across four metadata strata.Source: Ethical Considerations · p. 10
Loading 2609.37800v1…