Source-linked AI summary
Learned Cross-Task Relationships in Multi-Task Models
Victor Zhang, Yiping Yuan, Florian Raudies, Bosun Adeoti, Brian Y. C. Leung, Sanjay Surendranath Girija, Naijing Zhang
TL;DR
Multi-task models typically output marginal predictions without explicitly quantifying relationships among task labels. This paper approximates their joint distribution using targeted pairwise auxiliary heads. In YouTube live experiments, all tested deployments showed statistically significant gains in user engagement and satisfaction, while offline AUC results were mixed.
Problem
Multi-task learning may implicitly capture cross-task relationships in shared representations, but these relationships are rarely quantified as model outputs.
Source: §1 · p. 1Method
The framework adds selected auxiliary heads to estimate pairwise joint relationships and covariance, with learning signals targeted to examples where tasks co-occur.
Source: §1 · p. 1, §2.1 · p. 3, §2.2 · p. 3Results
All live experiments showed statistically significant gains in user engagement and satisfaction, while offline AUC results were generally positive but mixed.
Source: §6 · p. 6Takeaways & Limitations
Large-scale YouTube recommender deployments showed positive impacts on user engagement from auxiliary heads that approximate pairwise joint distributions.
Source: §8 · p. 8The current approach relies on empirical trial and error to select auxiliary heads.
Source: §8 · p. 8
Abstract
from arXiv · showhide
We propose a framework that learns cross-task relationships in multi-task models by approximating the joint distribution of task labels through targeted pairwise relationships. This approach improves performance via transfer learning and enhances information extraction without the intractable complexity of modeling the full joint space. Although our framework applies to any multi-task system, we demonstrate its efficacy within YouTube's production recommendation systems. Experiments across the Notifications, Homepage, and Watch Next surfaces show improvements in both accuracy and user satisfaction metrics. Finally, we propose a workflow template to facilitate broader future implementation.
1 INTRODUCTION
The introduction argues that standard multi-task learning shares representations across tasks but usually outputs only marginal predictions, leaving cross-task relationships implicit. It proposes targeted pairwise relationships as a practical approximation of the joint distribution, and situates this approach among related work and YouTube’s multi-task-to-reinforcement-learning workflow.
1 INTRODUCTION
- Multi-task learning shares a neural architecture across related tasks, combining their losses to learn representations that support transfer and prediction.The introduction also notes the shared computational budget as a resource-efficiency benefit.Source: §1 · p. 1
- Standard multi-task outputs can omit joint structure: N task labels have exponential underlying informational complexity, while marginal predictions scale linearly with N.The shared representation may capture relationships implicitly, but those relationships are rarely quantified as model outputs.Source: §1 · p. 1
- The proposed approach approximates the joint label distribution through selected pairwise interactions, such as covariance, implemented with auxiliary heads.The contribution also describes hypotheses and a testing workflow for downstream reinforcement-learning systems.Source: §1 · p. 1
1.1 Related Work
- Unlike auxiliary-head frameworks that predict supplementary marginal probabilities, this method selects pairwise targets to estimate covariance and approximate the joint distribution.The paper distinguishes this targeted approach from estimating predictive variance for an individual task [12].Source: §1.1 · pp. 1–2
- MMoE and PLE improve shared representations through gating but retain independent marginal losses; other cited approaches use relational frameworks or graph-based label mappings, [1], [5], [22].The paper contrasts these approaches with its targeted covariance approximation via auxiliary heads.Source: §1.1 · p. 2
- For sequential user conversions, ESMM and AITM model conditional or sequential dependence, while this paper adds unconditional and cross-label heads [15],.The paper presents those heads as a way to encode task interactions for downstream reinforcement-learning policies.Source: §1.1 · p. 2
- Rather than adjusting gradients after computation, the method proposes a cross-task head to structurally strengthen signals for overlapping task features.The authors hypothesize that this supports a cooperative minimum without dynamic gradient routing or weighting.Source: §1.1 · p. 2
1.2 Multi-Task Learning in YouTube Recommender Systems
- YouTube’s workflow uses a User Engagement Model to generate behavior predictions for the reinforcement-learning state, then a learned policy uses them to select an action maximizing reward.The policy may include a neural-network component that is typically smaller than the User Engagement Model [20].Source: §1.2 · p. 2
2 MATHEMATICAL FORMULATION
The section argues that separate task marginals omit interaction information and that standard losses do not specifically target task covariance. It proposes auxiliary pairwise heads to approximate joint structure with far fewer outputs than full joint modeling requires.
2.1 Marginal and Joint Distributions
- Marginal predictions omit task interactions, and standard losses do not specifically penalize failure to capture covariance.Source: §2.1 · p. 3
- The auxiliary head predicts the joint event XY, with its training label formed by the logical AND of the two task labels.Source: §2.1 · p. 3
- The cross-task head estimates E[XY], which combines with marginal estimates E[X] and E[Y] to resolve task covariance.Source: §2.1 · p. 3
- The joint-loss gradient targets data where tasks co-occur, encouraging interaction-specific features in the shared representation for downstream tasks.Source: §2.1 · p. 3
2.2 Modeling: Approximating Pairwise Relationships
- At most O(N^2) pairwise auxiliary heads approximate task relationships, avoiding the O(2^N) additional heads required to model the full joint distribution.Source: §2.2 · p. 3
- For direct click and save tasks, the added head estimates P(save ∩ click); when save is conditioned on click, the added head estimates unconditional P(save).Source: §2.2 · p. 3
2.2.1 Deriving Covariance.
- Covariance requires E[XY] alongside E[X] and E[Y]; direct-task pairs add a joint head, while conditional-task pairs add an unconditional head for Y.Source: §2.2.1 · p. 4
2.2.2 Deriving the Pairwise Joint Distribution
- For each binary task pair, the three terms E[X], E[Y], and E[XY] determine all four joint states, making their estimation a complete pairwise joint-distribution estimate.Source: §2.2.2 · p. 4
3 TWO PRIMARY HYPOTHESES
The section presents two hypotheses for modeling cross-task relationships: auxiliary heads may transfer learning to existing tasks, and their outputs may enrich signals consumed by reinforcement-learning agents. It proposes offline and live-user experiments to assess these hypotheses.
3 TWO PRIMARY HYPOTHESES
- The paper grounds two hypotheses in joint-distribution estimation and targeted gradient signals, with validation through offline model evaluation and live-user experiments.Source: §3 · p. 4
3.1 Transfer Learning Hypothesis
- Auxiliary relationship heads may improve existing heads through a better shared representation; the proposed tests compare offline AUC and live-user metrics when their outputs are ignored.Source: §3.1 · p. 4
3.2 Environment Modeling Hypothesis
- Adding relationship-head outputs as RL inputs may enrich the agent's environment model; live A/B tests assess user-metric impact.Source: §3.2 · p. 4
4 MODEL ARCHITECTURE
The section describes two auxiliary-head architectures: direct connections to the shared body and smaller heads connected to task-specific hidden layers. Both proved effective, but the shared-body design was adopted for easier implementation.
4 MODEL ARCHITECTURE
- Both architectural approaches proved effective, but the shared-body approach was ultimately chosen for easier implementation.Source: §4 · p. 4
- The adopted architecture connects auxiliary heads directly to the shared body without stopping gradients, a simple and scalable design preferred across YouTube models.Source: §4 · p. 4
- The alternative uses a smaller cross-label head connected to the top hidden layers of the two task-specific heads, without stopping gradients.Source: §4 · p. 4
5 IMPLEMENTATION AND TESTING WORKFLOW
The workflow offers a way to adopt cross-task heads in multi-task systems, from choosing useful task pairs through offline and live evaluation to iterative refinement.
5 IMPLEMENTATION AND TESTING WORKFLOW
- Select task pairs with nontrivial joint distributions, excluding nested, mutually exclusive, and analytically redundant unconditionals.Check label integrity, including whether the joint label is sparse relative to either task.Source: §5 · pp. 4–5
- Offline evaluation checks AUC gains in the original tasks involved in each cross-task relation.The workflow advises focusing on the two tasks associated with each added head.Source: §5 · p. 5
- Live testing separates transfer learning from the added value of serving cross-task predictions to the learned policy, then iterates on head selection.The first phase suppresses new outputs; the second serves them as policy inputs, using the first phase as baseline.Source: §5 · p. 5
6 DESCRIPTION AND RESULTS ON YOUTUBE SURFACES
Across YouTube surfaces, live experiments showed statistically significant gains in user engagement and satisfaction, while offline AUC results were mixed. The evaluation uses production-scale data, but proprietary metric formulations limit how precisely some choices can be specified.
6 DESCRIPTION AND RESULTS ON YOUTUBE SURFACES
- A live experiment is an A/B test on a segment of YouTube users, while a launch deploys an algorithm change to the entire user base.Experiment arms typically include tens of millions of users; launches reach billions.Source: §6 · p. 5
- Models train on billions or more examples generated daily, using several weeks of data and a 1% evaluation set.Offline metrics such as AUC are calculated on a held-out evaluation set after training.Source: §6 · p. 6
- Proprietary production metrics require abstracting some formulations, and metric definitions and design choices may vary across applications.Small reported changes such as 0.1% can still represent substantial, statistically significant absolute impacts at YouTube’s scale.Source: §6 · p. 6
- All live experiments showed statistically significant gains in user engagement and satisfaction, while offline AUC results were positive but mixed.Some original heads improved, others were neutral, and occasional minor regressions appeared in unrelated heads.Source: §6 · p. 6
6.1 Notifications
Across Notifications launches, cross-task heads produced live user-metric gains, and a follow-up comparison favored routing gradients directly to shared event base layers, especially for sparse or class-imbalanced tasks.
6.1 Notifications
- Notifications pose attribution and user-state challenges, often producing lower baseline AUC and wider live-experiment confidence intervals.Repeating experiments or bundling changes can help establish significant quality improvements or increase confidence.Source: §6.1 · p. 6
6.1.1 Initial Launch of 10 Heads
- The initial ten-head launch improved accuracy mainly in direct-interaction heads, and its combined live arm improved engagement, satisfaction, and efficiency.The experiment separated control, transfer-learning-only, and combined arms that also served the predictions to the RL policy.Source: §6.1.1 · p. 6
6.1.2 Follow-Up Launch on More Heads
- The follow-up compared gradients routed through specialized variant towers with gradients sent directly to the shared base layer.The second route bypasses variant-specific towers; the study notes further work on optimizing auxiliary-head gradient paths.Source: §6.1.2 · pp. 6–7
- Direct-to-base Path 2 significantly outperformed variant-tower Path 1, improving AUC by approximately 0.6%–4% on class-imbalanced and sparse-label tasks.The authors suggest direct regularization of event base layers yields a higher-quality, less specialized shared representation.Source: §6.1.2 · p. 6
6.2 Homepage
On Homepage, auxiliary heads yielded positive live user outcomes, while the final live test of 11 predictions added to the RL ranking model had a smaller impact than the initial transfer-learning experiments.
6.2 Homepage
- Homepage’s high user intent gives cleaner signals, but its large model and many heads create a large task-pair selection space.The model centers on P(click), with numerous heads conditioned on positive click labels.Source: §6.2 · p. 7
6.2.1 Multi-Task Model Auxiliary Heads
- The two auxiliary-head launches improved live user metrics even when heads were passive, though AUC effects were modest and heterogeneous and some existing heads regressed.The net live impact on user experience was positive.Source: §6.2.1 · p. 7
- The first launch added five unconditioned heads, including Watch Time Ratio, Save, Like, Subscribe, and Inline Playback, with substantial live engagement and quality improvements.Existing-task AUC improvements were modest and heterogeneous.Source: §6.2.1 · p. 7
- The second launch added seven passive heads, including reversed-conditioning tasks and five click-conditioned Save × Action cross-labels.The cross-label actions were Share, Download, Channel Engagement, Homepage Engagement, and Like.Source: §6.2.1 · p. 7
6.2.2 Passing Heads to RL Policy Model
- The 11 predictions were added as RL-model inputs, with a control retaining the upstream heads but suppressing their outputs to isolate their value as intent features.The selected set included conditioned and unconditioned cross-label heads.Source: §6.2.2 · pp. 7–8
- Serving the final 11 cross-task heads to the Homepage RL ranking model improved user-engagement quality, but less than the initial transfer-learning experiments.Full-scale deployment and final refinement were still in progress.Source: §6.2.2 · p. 8
6.3 Watch Next
Watch Next recommendations use the current video as their primary context and are evaluated for effects across YouTube. Adding four non-serving auxiliary heads improved offline AUC across original serving heads and produced significant live gains.
6.3 Watch Next
- Watch Next impact is measured across all of YouTube, and only Phase 1 transfer-learning benefits are complete.Watch Next uses the current video as its primary signal context and must accommodate diverse user journey entry points.Source: §6.3 · p. 8
6.3.1 Addition of Four Auxiliary Non-Serving Heads.
- The launch added unconditional Continuation, Like, Dislike, and Subscribe heads, connected directly to the shared body.These were variants of heads previously conditioned on a watch requirement.Source: §6.3.1 · p. 8
- The treatment model yielded consistent offline AUC improvements across original serving heads and translated into significant live gains.The evaluation covers the Watch Next sidebar launch.Source: §6.3.1 · p. 8
7 DISCUSSION
The discussion distinguishes cross-task auxiliary heads from simply reweighting existing tasks or crossing their predictions. It argues that explicit joint-label predictions provide a signal and information unavailable from those alternatives.
7.1 Contrasting with Scalar Loss Weighting
- Experiments found no evidence that duplicating existing heads or increasing their loss weights improved performance, unlike adding auxiliary cross-task heads.The approaches also create different learning dynamics.Source: §7.1 · p. 8
- An XY head concentrates gradient signal on examples positive for both X and Y, unlike loss weighting, which increases importance for all examples positive for either label.When both labels are sparse, the jointly positive subset can be small.Source: §7.1 · p. 8
7.2 Difference Between Crossing Task Labels and Crossing Existing Task Predictions
- In general, E[X]E[Y] does not equal E[XY]; the difference is covariance, so crossing marginal predictions cannot recover all joint-distribution information.This is an information-availability limitation, not a limitation of the policy model’s feature-crossing sophistication.Source: §7.2 · p. 8
8 CONCLUSION AND FUTURE WORK
The paper extends multi-task learning to explicitly approximate cross-task joint structure with auxiliary heads, reporting positive user-engagement impacts in YouTube deployments. Future work targets automated head selection and a unified end-to-end architecture.
8 CONCLUSION AND FUTURE WORK
- The framework extends multi-task learning by explicitly modeling or approximating task joint distributions rather than relying only on implicit feature sharing.It introduces auxiliary heads to approximate covariance and pairwise joint distributions.Source: §8 · p. 8
- Large-scale YouTube recommender deployments showed significant positive impacts on user engagement.The practical solution uses auxiliary heads to approximate covariance and pairwise joint distributions.Source: §8 · p. 8
- Future work proposes automating auxiliary-head selection and potentially unifying the User Engagement Model with the RL policy end to end.Automation would reduce reliance on empirical trial-and-error; unification could give the policy access to learned relationships through the latent representation.Source: §8 · p. 8