Source-linked AI summary
HubMixer: Progressive Latent Hub Mixing for Parameter-Efficient Feature Interaction in Recommendation
Jie Zhou, Zixian Gong, Wenhao Li, Chang Liu, Enzhao Shen, Bo Liu, Xu Guo, Fei Pan, Peng Jiang
TL;DR
Industrial recommendation requires effective interaction modeling across heterogeneous tokens whose useful relationships are sparse and sample-dependent, while direct token mixing can be parameter-inefficient. HubMixer routes tokens through learnable latent hubs for induction, high-order interaction, and token-conditioned readout. Offline experiments outperform strong baselines with fewer parameters, and online testing reports a statistically significant 5.48% improvement in resume submission conversion rate with production deployment.
Problem
Heterogeneous recommendation tokens interact sparsely and sample-dependently, making direct raw-token mixing potentially parameter-inefficient.
Method
HubMixer uses learnable latent hubs in an induction–interaction–readout pipeline to organize cross-feature interaction in a compact latent space.
Results
5.48% improvement in resume submission conversion rate was statistically significant in online A/B testing, following offline gains with fewer parameters and production deployment.
Takeaways & Limitations
Latent hub mixing provides an effective and extensible paradigm for parameter-efficient feature interaction in industrial recommendation ranking.
Abstract
from arXiv · showhide
Learning effective feature interactions is central to industrial recommendation and advertising ranking systems. Recent token-mixing architectures simplify self-attention with lightweight mixing operators, improving hardware efficiency and enabling large-scale deployment. However, recommendation tokens are fundamentally heterogeneous: user profiles, item attributes, behavioral sequences, context features, statistical signals, and business-side features live in different semantic spaces and interact in sparse, sample-specific patterns. Directly mixing all tokens in the raw heterogeneous token space may therefore be parameter-inefficient, as the model must implicitly discover which feature groups should interact and how such interactions should be routed. In the paper, we propose HubMixer, a parameter-efficient latent hub mixing architecture for feature interaction in recommendation. Instead of directly mixing raw feature tokens, HubMixer introduces a small set of learnable latent hubs to organize feature interactions through an `induction--interaction--readout` paradigm. First, hub induction summarizes heterogeneous tokens into compact latent hubs, where latent hubs query input tokens through cross-attention. Second, hub interaction performs high-order interaction in the cleaner latent hub space. Third, token-conditioned readout lets each original token selectively read from the interacted hubs, injecting global interaction semantics while preserving token-level field identity. Extensive offline experiments on industrial recommendation tasks show that HubMixer outperforms the SOTA models. Online A/B testing in the Kuaishou short-video recruitment business further shows a statistically significant 5.48% improvement in resume submission conversion rate, and HubMixer has been fully deployed in production.
1 Introduction
HubMixer addresses the parameter inefficiency of directly mixing heterogeneous, sparse, sample-dependent recommendation tokens by routing interaction through compact latent hubs. Its induction–interaction–readout design summarizes tokens, performs high-order hub interaction, and selectively returns token-specific signals, with offline and online evidence supporting its effectiveness.
- Motivation: Recommendation tokens differ in semantics, sparsity, and interaction patterns, making direct mixing of all tokens parameter-inefficient.Useful interactions are often sparse, sample-dependent, and group-structured.
- Efficiency: The architecture concentrates interaction capacity on a compact set of high-value subspaces instead of exhaustively mixing token pairs.This factorized routing uses H latent hubs rather than dense T×T token interaction, with H ≪ T.
- HubMixer: HubMixer introduces learnable latent hubs to organize feature interaction through hub induction, hub interaction, and token-conditioned readout.Hubs aggregate heterogeneous tokens, interact in a compact space, and provide token-specific global interaction signals.
- Evaluation: HubMixer is evaluated through extensive offline experiments, ablations, and online A/B testing in Kuaishou short-video recruitment.The online evaluation reports a statistically significant 5.48% improvement in resume submission conversion rate and production deployment.
2 Related Work
Related work spans classical and neural feature-interaction models, efficient token-mixing architectures, and latent query-based abstraction methods. HubMixer differs by using latent hubs not merely to compress inputs, but to mediate structured cross-feature interaction and token-conditioned information writing.
- Feature Interaction: Classical and neural recommender models address feature interaction through pairwise modeling, cross networks, attention mechanisms, and scalable model capacity.Examples include factorization machines, DCN, DCNv2, AutoInt, and Wukong.
- Feature Interaction: Industrial recommendation makes interaction modeling harder because token-token possibilities grow rapidly while useful interactions remain sparse and context-dependent.HubMixer organizes heterogeneous feature tokens through compact latent hubs to target effectiveness and parameter efficiency.
- Token Mixing: Transformer attention improves recommendation performance but has quadratic cost, motivating token-mixing architectures such as RankMixer for efficiency and scalability.HubMixer is complementary: it focuses on interaction effectiveness and parameter efficiency rather than primarily hardware utilization.
- Latent Query Architectures: Latent architectures use small sets of latent variables or queries to summarize larger input sets through compact abstraction interfaces.Cross-attention lets latent queries selectively read information from input representations.
- Latent Query Architectures: Unlike Q-Former and OneRec, HubMixer uses latent hubs as intermediate interaction centers for general recommendation tokens, beyond representation compression.Its hubs support structured cross-feature interaction followed by token-conditioned information writing.
3 Method
HubMixer organizes heterogeneous recommendation feature tokens through compact latent hubs, enabling hub-space interaction and token-specific readout while preserving field identities. Stacked blocks feed refined tokens into a multi-task ranking objective.
- Overall Architecture: HubMixer uses a four-stage pipeline: tokenization, hub induction, hub interaction, and token-conditioned readout.
- Tokenization: Semantic feature groups are transformed into structured tokens rather than flattened embeddings, preserving field semantics for interaction modeling.
- Hub Induction: Learnable hubs selectively aggregate input tokens through cross-attention, producing sample-specific latent representations that can specialize in different semantic aspects.
- Hub Interaction: Self-attention models dependencies among the induced hubs, concentrating high-order interaction modeling on compact, content-dependent summaries instead of all token pairs.
- Token-Conditioned Readout: Token-conditioned readout lets each token selectively retrieve a customized mixture of interacted hub semantics rather than receiving one pooled global vector.
- Multi-Task Prediction and Optimization: Residual injection yields refined token representations that retain field identities while gaining global cross-feature signals for downstream multi-task prediction.
4 Experiments
Experiments show that HubMixer improves ranking quality and parameter efficiency across offline comparisons, while its complete induction–interaction–readout design and latent-hub configuration are validated by ablations. Representation analyses and online testing further associate the architecture with stronger task-relevant token updates and a statistically significant 5.48% conversion-rate gain.
- Performance Comparison: HubMixer achieves the best AUC on all four ranking objectives and outperforms strong token-mixing baselines with fewer parameters.The comparison uses the same feature set and multi-task setting across models.
- Module Ablation: Removing hub interaction drops average AUC from 0.8256 to 0.8232, showing that induced hubs alone are insufficient.The degradation occurs on every ranking objective.
- Module Ablation: Token-conditioned readout outperforms pooled hub injection because tokens selectively retrieve relevant interacted-hub information while preserving token-specific semantics.Pooled injection still helps, but its degradation is smaller than removing hub interaction.
- Hub Number Analysis: 16 hubs provide a stronger accuracy–parameter trade-off than 32 hubs, whose average AUC rises only from 0.8256 to 0.8258 while parameters increase from 142.4M to 165.1M.Increasing hubs from 4 to 16 consistently improves all four task AUCs, but gains saturate beyond 16.
- Online A/B Test: 5.48% improvement in resume submission conversion rate over the base model is statistically significant in a seven-day online A/B test.The test covered 7.2% of production traffic, and HubMixer was subsequently fully deployed.
- Token Representation Enhancement: HubMixer changes token directions more strongly than TokenMixer while producing more task-relevant representations, as shown by cosine-distance and frozen-backbone probing analyses.In the first mixer layer, average cosine distance is 0.200 for HubMixer versus 0.105 for TokenMixer; HubMixer also probes better on most tokens and the aggregated representation.
5 Discussion
HubMixer organizes heterogeneous recommendation features through learnable latent hubs, using progressive induction, hub interaction, and token-conditioned readout. The design offers tunable interaction capacity and suggests extensions for adaptive routing, hub specialization, multi-task learning, lighter hub operators, and deployment optimization.
- Discussion: HubMixer first summarizes heterogeneous feature tokens into compact latent semantics, interacts among those hubs, and selectively writes the resulting information back to each token.The hubs learn soft, content-dependent organization patterns rather than imposing a fixed feature hierarchy.
- Discussion: Token-conditioned readout preserves field-specific token information while letting each token selectively absorb cross-feature semantics from the interacted hubs.This separates compact global interaction storage from token representations used by downstream prediction heads.
- Discussion: The hub number controls latent interaction-space size, while the number of HubMixer blocks controls progressive interaction-refinement depth.These provide practical knobs for fitting parameter and latency budgets.
- Discussion: Future work could generate hubs dynamically from request context, encourage complementary hub specialization, or condition hubs and readout on task objectives.The paper also identifies diversity regularization, orthogonality constraints, and task-aware routing as possible extensions.
- Discussion: The current hub interaction uses self-attention over latent hubs, while lighter MLP-based or sparse expert-style mixing could target stricter FLOPs or latency budgets.Because the number of hubs is smaller than the number of feature tokens, hub-level interaction remains controlled.
- Discussion: Hub induction and token-conditioned readout scale with T H rather than T^2 when H≪T, enabling batching-friendly and small-matrix serving optimizations.The paper highlights kernel fusion and optimized small-matrix computation as further engineering opportunities.
6 Conclusion
HubMixer is a progressive latent hub mixing architecture that routes recommendation feature interactions through compact learnable hubs. Offline experiments, ablations, and online testing support its effectiveness and production deployment, including a statistically significant 5.48% improvement in resume submission conversion rate.
- Conclusion: HubMixer progressively induces latent hubs, performs high-order interaction in hub space, and returns customized interaction signals to original tokens.Its input-conditioned, content-adaptive routing concentrates capacity on structured cross-feature patterns.
- Conclusion: The readout output becomes the block output and feeds multi-task prediction heads, separating shared feature interaction from task-specific signal extraction.Stacking blocks progressively refines token representations for increasingly abstract cross-feature patterns.
- Conclusion: Offline experiments show better ranking performance than strong token-mixing baselines with fewer parameters, while ablations validate hub induction, hub interaction, and token-conditioned readout.The ablation evidence supports the independent contributions of the three stages.
- Conclusion: 5.48% improvement in resume submission conversion rate was statistically significant in online A/B testing, after which HubMixer was fully deployed in the Kuaishou short-video recruitment business.The online evaluation provides production evidence for the proposed interaction architecture.