PaperlayerView exact revision on arXiv

Source-linked AI summary

TasteRoute: Personalized Routing for Video Generation

Zhi Rui Tam, Chao-Chung Wu, Sin-Han Yang, Peyton Ku, Brendan Kuang, Tzu-Ting Hsieh, Min-Fang Hsu, Fang-Ling Tsai, Yun-Nung Chen, Wei-Chiu Ma, Chieh-Yen Lin

arXiv:2610.05896v1cs.CVcs.AIcs.CL

TL;DR

Video generators vary in capability and cost, raising the question of how to route each request to an appropriate model. TasteRoute combines request information, user preferences, and budget in personalized routing, using TasteRoute-3K’s multi-model comparisons and human preference signals. From the $1.0 budget cap upward, the paper reports that TasteRoute is both the cheapest and most accurate router.

  • Problem

    Video generators differ in capability and generation cost, while the appropriate choice depends on the request and the user’s preferences.

    Source: §1 · p. 1

  • Method

    TasteRoute predicts candidate generators’ acceptability and user-aligned preference from the request and user information, then selects within budget.

    Source: §1 · p. 2

  • Results

    From the $1.0 budget cap upward, TasteRoute is simultaneously the cheapest and most accurate router, and it outperforms every personalized baseline at every budget cap.

    Source: §10 · p. 15

  • Takeaways & Limitations

    Conditioning routing on annotator identity improves over generalized routing once the budget no longer constrains the choice.

    Source: §10 · p. 15

    Results rest on nine annotators and a fixed generator catalog, and the observed cost saving reflects these annotators’ preference for cheaper generators rather than a general property.

    Source: §11 · p. 15

Abstract

from arXiv · show

Rapid progress in video generation has led to a plethora of models that differ substantially in capability and generation cost. This raises a natural question: can each request be efficiently routed to an appropriate model? We find that even when the consensus of the other annotators is used as an oracle, it agrees with each annotator's own favorite only 34-55% of the time. Motivated by this observation, we introduce TasteRoute, a personalized video-generation router that selects a generator jointly based on the input request, user preferences, and available generation budget. Across text-to-video and image-to-video settings, TasteRoute is competitive with strong simple baselines on preference routing while reducing average generation cost. The cost saving increases under higher budget caps. Finally, we release TasteRoute-3k, a human-annotated dataset containing multi-model video comparisons, quality judgments, preference rankings, and user-profile signals to facilitate future research on personalized and cost-aware video routing.

1. Introduction

Video generators vary in cost and in the qualities they deliver, making model choice dependent on the request and the user’s preferences. TasteRoute addresses this routing problem under a budget, against a backdrop of limited video-routing research and preference data.

1. Introduction

  • Generators differ in cost and in strengths such as visual detail, physical plausibility, and style, while users value qualities differently.The paper frames model choice around task requirements and user preferences, including whether a more expensive model is worth its cost.Source: §1 · p. 1
  • TasteRoute selects a generator for a text request and optional reference image, accounting for likely acceptability, user preferences, and the available budget.Its predictions include basic acceptability and a preference score for each candidate generator.Source: §1 · pp. 1–2
  • Training a useful router requires individual preference judgments, same-request comparisons across generators, and separate judgments of basic acceptability.TasteRoute-3K combines these signals, and the paper reports that personalization reduces average generation cost by 8–12% at budget caps of $1.0–$2.0 while improving routing accuracy over personalized baselines.Source: §1 · p. 2

2.1. Model Routing

  • Prior model-routing work spans classification, cost-aware routing, richer decision mechanisms, and personalization, but video-generation routing remains unexplored.The cited work includes classification routing, cascade-based routing, and approaches such as query representations [19] and personalization.Source: §2.1 · p. 2, §2.2 · p. 3

2.2. Model Routing for Generative Models

  • Generative-model routing has begun in image generation, including prompt-aware selection and verifier-guided routing, while video generators vary in capability and serving cost.The paper notes reported 1080p image-to-video prices spanning roughly $0.30–$24.00 per generated minute.Source: §2.2 · p. 3

2.3. Video Preference Datasets

  • Existing video-preference datasets are limited by older models, relatively few human annotations, or unreleased feedback data.Huang et al. [14] has 1,150 human-annotated pairs, with the remainder labeled by a calibrated Gemini 3.1 judge; Video Arena feedback is not released.Source: §2.3 · p. 3

3. Problem Formulation

The task is to rank candidate generators for each text-to-video or image-to-video request within a budget, with personalized scores also depending on the user. TasteRoute-3K is built to support this task through multi-generator comparisons, individual preferences, and separate acceptability assessments.

3. Problem Formulation

  • For each text-to-video or image-to-video request, the router scores candidate generators and selects the highest-scoring option within budget; personalized scores also depend on the user.The pool is the set of generators labeled for a request.Source: §3 · p. 3
  • TasteRoute-3K is designed around same-request multi-generator comparisons, individual-level preferences, and separate assessments of video acceptability.These are the data requirements identified for learning the router.Source: §3 · p. 3

4. TasteRoute-3K: Data Collection and Annotation

  • The dataset uses 277 prompts to form 394 requests—274 T2V and 120 I2V—and contains 3,128 videos generated from 30 generator endpoints.Professional creators wrote some prompts with requirements such as timestamp-specific motion and camera movement; prompts may be used in both settings.Source: §4 · p. 3

4.1. Annotation Pipeline

  • Nine annotators—four general and five professional video creators—were calibrated on the interface and rubric, but not on aesthetic preference.The study leaves aesthetic preference uncalibrated because it is the signal intended for personalization.Source: §4.1 · p. 4
  • A pool is one request-setting with its labeled generators, while a group is the at-most-six videos one annotator ranks at a time.Overlapping groups cover larger pools; comparisons are ordered video pairs, and pairs without a strict plurality are excluded from consensus targets.Source: §4.1 · p. 4, Table 1 · p. 4
  • Annotation separates shared-rubric quality and acceptability assessments from each annotator’s preference ranking, with acceptability retained per annotator.The quality rubric covers text integrity, motion consistency, subject realism, and instruction following; rankings allow ties and use shared anchor videos across groups.Source: §4.1 · pp. 4–5
  • For cold-start routing, annotators report video-quality preferences and complete reference-video comparisons, including preference strength and perceptual factors.The questionnaire covers five self-reported preference items, and each annotator completes four reference-video comparisons.Source: §4.1 · p. 5

4.2. General Preference Aggregation

  • General preference targets use weighted plurality outcomes and a weighted Bradley–Terry model fitted per connected component, producing 7,062 consensus training pairs.Pairs without a strict plurality are dropped; win probabilities provide smooth training targets, while evaluation uses consensus utility without fitting a model.Source: §4.2 · p. 5

5. When Does Video Routing Need Personalization?

The section finds that prompt attributes and annotator identity both affect which generator is preferred. Disagreement is especially pronounced among acceptable videos, so pooled consensus may not represent an individual’s favorite.

5. When Does Video Routing Need Personalization?

  • The leading generator changes across prompt attributes: five of twelve attributes in T2V and eight of twelve in I2V have a different leader.In T2V, Veo 3.1 Lite has the highest overall consensus utility but loses attributes to three rivals.Source: §5 · pp. 5–6, Figure 1 · p. 6
  • Agreement falls from 83–85% when a video is rejected to 64–66% for acceptable pairs with a defect and 57% in T2V or 62% in I2V for clean pairs.These figures are based on 19,338 doubly judged comparisons across 376 pools and nine annotators.Source: §5 · p. 6, Figure 2 · p. 7
  • Consensus can fail to match any annotator’s top choice, and 7% of pools contain a majority-preference cycle.In one example, every annotator ranks the consensus winner second; another pool has three favorites in a cycle.Source: §5 · p. 6
  • The consensus favorite matches an individual annotator’s top choice only 34–55% of the time, versus 19–28% for a random pick.The mismatch occurs across all nine annotators, despite consensus performing better than random.Source: §5 · pp. 6–7
  • These findings motivate routers that model prompt-dependent pooled preferences and individual preferences among acceptable outputs.The paper describes a shared architecture conditioned on the prompt and, when available, user information; pooled preferences are represented probabilistically rather than as hard winner–loser labels.Source: §5 · p. 7

6. TasteRoute: Personalized Video Routing

TasteRoute routes requests using request features, user preferences, and a generation budget. Its shared architecture supports generalized and per-user routing, with training objectives that differ by preference supervision and active ranking branch.

6.1. Architecture

  • TasteRoute scores candidate generators from request features and, for personalized routing, a user representation; one architecture supports both generalized and per-user routers.The request is encoded before generation, and the router ranks candidates without generating a video.Source: §6 · p. 7, §6.1 · pp. 7–8
  • A shared quality head predicts rubric ratings and acceptability, while separate general and personalized ranking heads produce the scores used for routing.The quality head is evaluated separately; routing ranks candidates using the ranking heads alone.Source: §6.1 · p. 8

6.2. Training Objectives

  • Both variants use pairwise ranking and ordinal quality losses, but differ in which pairs, preference targets, and ranking branches supervise them.The quality-loss weight is λ; generalized targets aggregate annotator preferences, while personalized targets use an annotator’s own outcome.Source: §6.2 · pp. 8–9
  • TasteRoute-Generic trains its general head on aggregated preference pairs and all annotators’ rubric ratings; TasteRoute-Personal trains separate models from each annotator’s own feedback.For TasteRoute-Personal, the same pairs train the general and personalized heads, and inference uses the personalized score; no model sees another annotator’s labels.Source: §6.2 · p. 9

7. Evaluation Setup

The evaluation uses prompt-disjoint general-routing splits and per-annotator personalized splits, then measures routing quality under cost caps. Baselines include retrieval, agentic and cascade routers, personalized methods, and random selection.

7. Evaluation Setup

  • General routing uses five-fold prompt-disjoint cross-validation, while personalized routing splits each annotator’s labels into 70% training, 15% validation, and 15% test.Reported personalized results average three random-seed runs.Source: §7 · pp. 9–10
  • For label-efficiency tests, training prompt groups are selected by diversity, seeded with the questionnaire embedding and then chosen farthest from the selected-unit centroid.At each cap, the matched pooled model receives the same total number of groups, balanced across annotators; held-out labels are never used for routing.Source: §7 · p. 10
  • Under budget cap B, routers select only affordable generators, but evaluation retains the full pool as ground truth, so an unaffordable best generator remains the Top-1 target.The evaluation reports tie-aware Top-1, Top-3, and NDCG, with budget caps swept from $0.5 to $2.0.Source: §7 · p. 10

7.1. Baselines

  • Baselines span kNN retrieval, agentic Octo, FrugalGPT cascades, hierarchical and graph-based personalization, TasteRoute variants, and uniform random selection.Same-user kNN restricts retrieval to one annotator’s data; Octo is also evaluated with human labels as an upper-bound verifier signal.Source: §7.1 · p. 10
  • Figure 5 reports NDCG routing quality under router-cost limits for generalized and personalized settings; TasteRoute Generic and Personal perform competitively against strong kNN, while TasteRoute-Personal scales better as training data grows.Source: Figure 5 · p. 11

8. Performance of Generalized and Personalized Routing

The section evaluates generalized routing from pooled preferences and personalized routing with annotator preferences, across budget caps and amounts of user feedback. TasteRoute-Generic is comparable to kNN, while personalization becomes more useful as budgets and feedback expand.

8.1. Generalized Routing Results

  • At the $2.0 cap, TasteRoute-Generic reaches 42.0% Top-1 versus 39.7% for kNN, 38.2% for FrugalGPT, and 33.6% for Octo, while NDCG is similar to kNN across caps.The figure plots cost against routing quality over the affordable pool, with dashed lines marking the personalized router.Source: §8.1 · p. 11, Figure 7 · p. 13
  • TasteRoute-Generic reaches 39.9% Top-1, compared with 37.0% for attribute lookup and 36.6% for the global-best generator.Across ten attributes, changing one attribute phrase while keeping the rest of the prompt unchanged flips the top-ranked generator in 5–55% of cases, averaging 25%.Source: §8.1 · pp. 11–12, Figure 6 · p. 12

8.2. Personalized Routing Results

  • At budget caps below $1.1 per video, TasteRoute-Personal performs similarly to TasteRoute-Generic, then pulls ahead as budgets increase.An annotator’s preferred generator is unaffordable in approximately 20% of pools at $0.5, compared with almost none at $1.5.Source: §8.2 · p. 12
  • At approximately 10 labeled groups TasteRoute-Personal surpasses the pooled router, and by 50 groups it surpasses Same-user kNN.With fewer than 10 groups, the Generalized router and Same-user kNN perform better; TasteRoute-Personal NDCG continues improving at the largest training size.Source: §8.2 · p. 12

9. Analysis on Personalized Router

The analysis finds that personalized routing can reduce cost while maintaining routing quality, and examines when preference-aware routing helps. Cost–accuracy trade-offs and personalization gains vary with budget, user disagreement, and prompt attributes.

9.1. Trade-offs between Cost and Preference

  • At the $1.0, $1.5, and $2.0 caps, TasteRoute-Personal is the cheapest and most accurate router, reducing cost by 8% to 12.0% versus the most expensive alternatives at those caps.Only at the $0.5 cap is TasteRoute-Generic-Pooled marginally ahead on both axes; the savings are a byproduct of learned preference ranking, without cost supervision.Source: §9.1 · p. 13, Figure 8 · p. 13
  • At the $1.5 cap, a cost-trade-off parameter of 0.3 reduces cost by 9.8% for 1.4 points of Top-1, while 1.0 reduces cost by 25.2% for 2.8 points.The table reports Top-1 accuracy and cost savings averaged over nine annotators and three split seeds on a fixed T2V/I2V subset.Source: §9.1 · p. 13, Table 2 · p. 14
  • The cost–preference trade-off runs are exploratory, using a fixed T2V/I2V subset of personalization test splits and a single seed triple.They indicate the shape of the trade-off rather than a tuned operating point.Source: §9.1 · p. 13

9.2. Understanding Personalization Gains

  • When two annotators express opposite preferences, TasteRoute-Personal reaches 55.2% accuracy, versus 52.4% for Same-user kNN and 52.2% for PersonalizedRouter-GNN; TasteRoute-Generic achieves 50% by construction.Across 54 cohorts with a shared top-ranked generator, personalization improves Top-1 from 36.7% to 50.1%; among 66 cohorts without a shared winner, it improves from 25.9% to 28.7%, not statistically conclusive.Source: §9.2 · p. 14
  • In held-out T2V analysis, personalization raises overall Top-1 from 34.1% to 36.8%, with gains ranging from −5.6 percentage points for cartoon/3D to +9.1 for quoted text.The largest gains occur in the two prompt groups with the lowest general-head accuracy.Source: §9.2 · p. 15

10. Conclusion

TasteRoute-3k captures conflicting preferences, and TasteRoute performs strongly against personalized baselines while reducing cost at higher budget caps. The authors identify a remaining oracle gap, limited conflict resolution, and scope boundaries in the annotators and generator catalog.

10. Conclusion

  • TasteRoute outperforms every personalized baseline at every budget cap and is both cheapest and most accurate from the $1.0 cap upward.The router mostly corrects the consensus rather than resolving conflicts between annotators.Source: §10 · p. 15

11. Limitation

  • At the $2.0 cap, the best router reaches 42.0% Top-1 versus 58.0% for the oracle, and the gap widens with more generators and larger budgets.On opposite-preference pairs, the personalized router reaches 55.2%.Source: §11 · p. 15
  • Results rest on nine annotators and a fixed generator catalog, while observed cost savings reflect these annotators’ preference for cheaper generators.The authors identify user-conditioned features that raise accuracy without raising generation cost as future work.Source: §11 · p. 15, §10 · p. 15

A Implementation Details

The implementation combines request, generator, and user representations, with separate generalized and personalized routing streams. Training, validation, and evaluation protocols specify how model selection, personalization support, agentic baselines, and generation costs are handled.

A. Implementation Details

  • Request embeddings come from a frozen Qwen3-VL-Embedding-2B encoder, while generator identity combines a family embedding with an endpoint-specific residual.Generalized routers project context to 32 dimensions using fold-specific PCA; personalized routers use raw embeddings, and candidate features include vectors and pairwise products.Source: A. Implementation Details · p. 19
  • Training uses AdamW, early stopping, three seeds, and lockstep general and personalized ranking streams, with learning rates and checkpoints selected by validation.Generalized-router learning rates are selected per fold; personalized routers and the pooled baseline use 3 × 10^-4.Source: A. Implementation Details · p. 19
  • Any positive quality-loss weight trains the generalized router’s quality head, but does not measurably change ranking accuracy; λ=0.25 is used for every model.Test quality AUC is 0.79 for the generalized router versus 0.51 untrained; quality prediction is claimed only for the generalized router.Source: A. Implementation Details · p. 19
  • Personalization uses identical held-out prompt groups across nine annotators and centroid-diverse support sets, with matched pooled-model support at each cap.Reported support caps are 1, 2, 5, 10, 20, 40, 50, and the full data; same-user kNN uses k=20.Source: A. Implementation Details · p. 19
  • Octo routes to the cheapest generator above a verifier threshold and may escalate, while FrugalGPT learns a budget-specific cascade that is replayed frozen at test time.Both use a verifier rubric; generation costs accumulate over attempts, and verifier costs are reported separately.Source: A. Implementation Details · p. 19
  • Generator prices are frozen at collection-time catalog prices; Table 3 counts one vote per evaluable pool, splitting ties fractionally among co-winners.The five most frequent winners are shown; nine of 30 generators win more than 5% of pools and 28 win at least one.Source: A. Implementation Details · p. 20, Table 3 · p. 20

B. Additional Dataset Analysis

The appendix expands dataset and routing analyses, reporting a broad distribution of pool winners and persistent disagreement among acceptable outputs. It also gives the full routing metrics and clarifies how the figures and baselines should be interpreted.

B. Additional Dataset Analysis

  • The pool-winner distribution is broad: the leader wins 10.1% of pools, nine of 30 generators win over 5%, and 28 win at least one pool.The appendix states that the estimators, seeds, and inputs match the corresponding main-text figures.Source: B. Additional Dataset Analysis · p. 20
  • Only cartoon or 3D prompts have an attribute-related agreement interval excluding zero, so prompt content explains little residual disagreement when both videos are acceptable.Figure 10 shows changes in agreement for 16 keyword-detected attributes among clean pairs.Source: B. Additional Dataset Analysis · p. 20, Figure 10 · p. 21
  • Majority preference cycles occur in 7% of pools; in the example, three annotators accept every video but favor different generators.The cyclic majority leaves no winner that more than one of those annotators endorses.Source: B. Additional Dataset Analysis · p. 20

C. Additional Routing Results

  • The routing appendix reproduces the full-candidate ground truth across panels, with colors and markers matching the main text.It includes every series and the metrics omitted from the main text.Source: C. Additional Routing Results · p. 20
  • Agentic variants within each family differ by less than one NDCG point, and Octo-Human-Label is a diagnostic ceiling rather than a routing method.On NDCG, the label-reading cascade sits at the bottom because it does not order candidates it did not generate.Source: C. Additional Routing Results · p. 21
  • Every conclusion in Section 8 holds on Top-1, Top-3, and NDCG; Top-1 is noisiest, while learned methods exceed 80% Top-3 at most budgets.The main text reports NDCG because Top-3 is the most saturated metric.Source: C. Additional Routing Results · p. 21

D. Showcase of Labeling UI

The labeling interface first onboards annotators to their preferences, then collects per-video quality ratings and listwise preference rankings. The workflow includes reference-video comparisons and explicit tie and acceptability options.

D. Showcase of Labeling UI

  • Onboarding begins with a five-item self-report questionnaire, followed by a comprehension check about choosing the result annotators would prefer.The questionnaire asks about valued qualities, preferred video type, motion, camera treatment, and visual treatment.Source: D. Showcase of Labeling UI · pp. 21–22
  • Four reference-video comparisons ask annotators to choose A, B, no preference, or cannot judge, with follow-up questions tailored to the choice.The follow-ups capture influencing factors and preference strength, or reasons for being unable to judge.Source: D. Showcase of Labeling UI · p. 22
  • The ranking interface collects five quality fields on a three-point scale before annotators group candidate videos by preference, allowing ties.The interface displays the prompt and highlights its key entities.Source: D. Showcase of Labeling UI · p. 22

E. Video Generators and Cost

Table 4 lists 30 video generators in the candidate pool and specifies their tasks and prices. Prices are catalog rates for 720p video without audio, frozen for all experiments; endpoint IDs listed separately count as separate candidates.

E. Video Generators and Cost

  • Prices use catalog rates frozen for all experiments; Veo and gemini-omni-flash follow Google API catalogs, while other prices follow fal.ai.Exceptions include no-720p models billed at 1080p rates, flat per-video pricing converted at 6 seconds, and base-plus-per-second pricing shown as a 6-second equivalent.Source: E. Video Generators and Cost · p. 23, Table 4 · p. 29
  • Some models appear under two endpoint IDs, and each endpoint ID is treated as a separate candidate.Source: E. Video Generators and Cost · p. 23
  • Table 4 lists 30 generators in the TasteRoute-3K candidate pool, ordered by price, and identifies the settings in which each appears in the dataset.The listed prices are in USD per second of 720p video without audio.Source: E. Video Generators and Cost · p. 23, Table 4 · p. 29
Loading 2610.05896v1…