An Event is Worth One Token: Event Tokenization for Industrial-scale LLM Recommendation

Fan Xia; Zhaoheng Zheng; Iman Setayesh; Ruogu Lin; Yiqin Pan; Samarth Mittal; Wentao Bao; Vinti Pandey; Sachin Patil; Jianpeng Cheng; Jun Xiao; Zhuang Wang; Xiangjun Fan; Sri Reddy; Minghai Chen

Source-linked AI summary

An Event is Worth One Token: Event Tokenization for Industrial-scale LLM Recommendation

Fan Xia, Zhaoheng Zheng, Iman Setayesh, Ruogu Lin, Yiqin Pan, Samarth Mittal, Wentao Bao, Vinti Pandey, Sachin Patil, Jianpeng Cheng, Jun Xiao, Zhuang Wang, Xiangjun Fan, Sri Reddy, Minghai Chen

arXiv:2608.25546v1Aug 26, 2026cs.IR

TL;DR

Recommendation models often encode too little information per historical event, forcing a trade-off between snapshot resolution, sequence coverage, and serving cost. AMBER compresses full temporal snapshots into end-to-end learned Event Tokens that are asynchronously cached, and it improves system-level scaling across recommendation settings. The paper also identifies feature delay and representation drift as practical boundaries of this approach.

Problem
Recommendation models discard rich event signals because serving constraints force a trade-off between detailed historical snapshots, sequence length, and feature-materialization cost.
Method
AMBER uses an end-to-end trained unified Event Tokenizer to compress heterogeneous temporal snapshots into compact Event Tokens that are asynchronously pre-computed and cached.
Results
AMBER advances the compute-quality Pareto frontier across industrial ranking and retrieval, while Event Tokenizer scaling improves downstream performance and Event Tokens improve an existing non-LLM ranker.
Takeaways & Limitations
Snapshot resolution is a viable scaling dimension, and optimizing total training plus serving FLOPs is important for industrial system-level scaling decisions.
Takeaways & Limitations
Asynchronous tokenization introduces feature delay, while recurrent encoder updates can cause representation drift that makes new tokens incompatible with cached histories.

Abstract

from arXiv · show

LLM-based recommendation has scaled along model capacity and sequence length, yet each position encodes only text, semantic IDs, or a few categorical features, discarding rich user, item, context, and outcome signals available at each event. Under autoregressive modeling, this yields weak queries at each position and, since each position becomes context for the next, the degradation compounds across the sequence. We propose an event-centric paradigm that represents each interaction by its full temporal snapshot, and identify a new scaling dimension we term snapshot resolution: the amount of information encoded per event. To efficiently scale snapshot resolution, we introduce AMBER (Autoregressive Modeling via Bottlenecked Event Representation), which compresses each temporal snapshot into a compact Event Token, a new LLM input modality. The representation is learned end-to-end, while Event Tokens are pre-computed and cached for serving, decoupling snapshot resolution from real-time serving compute. On industrial-scale ranking and retrieval benchmarks, AMBER advances the compute-quality Pareto frontier relative to alternative recommendation paradigms. At sufficient capacity, a single unified tokenizer even outperforms dedicated per-entity tokenizers, demonstrating positive transfer across structurally different entity types. AMBER's Event Tokens also transfer across model architectures: when integrated into a heavily optimized non-LLM ranker as serving-time historical features, they yield statistically significant improvements. Further scaling Event Tokenizer capacity provides additional improvements.

1 Introduction

Industrial recommendation is constrained by the trade-off between detailed event snapshots and serving cost. AMBER addresses this by compressing heterogeneous temporal snapshots into cached Event Tokens, enabling snapshot-resolution scaling and broader downstream improvements.

Snapshot resolution measures how many distinct signals a model captures per event, including user, item, context, and outcome signals.
Higher snapshot resolution improves autoregressive queries and preserves more contextual detail in key-value histories.
Industrial recommendation must balance sequence length and snapshot resolution because feature storage, deserialization, and transfer can rival GPU compute.
AMBER compresses each heterogeneous temporal snapshot into a compact Event Token through an end-to-end trained unified tokenizer.Tokens are computed asynchronously per event and cached in user histories, avoiding raw-feature materialization during real-time inference.
Event Tokenizer capacity can improve downstream quality with negligible additional compute on the primary online serving path.Downstream LLM performance also scales predictably with capacity over Event Tokens.
AMBER advances the compute-quality Pareto frontier across large-scale ranking and retrieval and lets a non-LLM ranker gain statistically significant improvement from Event Tokens.The system also addresses representation drift and Event Token storage through alignment, dropout, and quantization-aware training.

2 Related Work

Related recommendation systems encode heterogeneous events using sparse histories, text, Semantic IDs, or decoupled representations, each imposing information or optimization trade-offs. AMBER instead compresses full events end-to-end into high-density Event Tokens.

Pointwise recommenders retain rich per-impression features but represent user histories sparsely, while autoregressive models reduce historical events to simpler representations under serving constraints.
Semantic IDs encode items efficiently but discard detailed event context, whereas text describes fuller events with lower information density.
Prior end-to-end item-text embedding methods discard relevant event-level signals, and LoopFM decouples distillation from downstream gradients.AMBER trains its event-centric tokenizer end-to-end to compress full heterogeneous events into high-density Event Tokens.
Caching learned representations creates challenges from recurrent representation drift and substantial storage requirements at scale.The related work motivates alignment methods for stabilizing representations across successive temporal checkpoints.

3 Methodology

AMBER uses additional offline computation to compress heterogeneous event features into compact tokens, which downstream autoregressive models consume for ranking and retrieval. A shared tokenizer, role-specific masking, staged training, and task-specific sequence designs support this event-centric representation.

AMBER architecture: The architecture deliberately shifts computation offline because online serving scores large candidate pools under stricter computational constraints.This exploits the asymmetry between offline training and real-time serving.
AMBER architecture: AMBER compresses raw per-event signals into d_z-dimensional Event Tokens that a User LLM consumes autoregressively.The Event Tokenizer and User LLM are jointly trained, while serving uses the resulting token sequence rather than materializing detailed event features.
Tokenizer architecture: The tokenizer maps categorical, embedding, and numerical features into tokens, contextualizes them with a bidirectional Transformer, and projects summarized outputs into LLM space.Learnable [CLS] tokens summarize interactions among heterogeneous feature representations.
Unified Event Tokenizer: A single shared Event Tokenizer generates tokens for different roles through dynamic feature masking.The mask controls feature visibility, allowing one tokenizer to serve all roles.
Training: Training proceeds through pre-alignment with a frozen LLM, joint end-to-end training, and recurrent retraining for distribution shifts.Recurrent training addresses drift across updates of cached Event Tokens.
Sequence design: The sequence design prioritizes historical coverage, using one token per event for retrieval and two tokens per event for ranking.Ranking interleaves context and label tokens under autoregressive BCE, whereas retrieval uses fused event tokens with InfoNCE and separately tokenized candidates.

4 Serving at Scale

AMBER separates asynchronous tokenization and caching from downstream serving, enabling cached event representations for ranking and retrieval. At scale, it addresses representation drift and storage through alignment, sharded retraining, dimensionality flexibility, and quantization.

Asynchronous serving: AMBER tokenizes events asynchronously and appends compact representations to user histories in a feature store.Downstream models consume cached Event Token sequences instead of recomputing full heterogeneous feature sets online.
Downstream modes: Cached Event Tokens support retrieval with the User LLM and plug-and-play historical features for an existing non-LLM ranker.The work demonstrates transferability of the representation while leaving end-to-end LLM ranking infrastructure challenges beyond scope.
Representation drift: Checkpoint separability rises monotonically without regularization, while EMA- and DANN-based configurations suppress representation drift.The figure defines 0.5 as indistinguishable and higher k-NN accuracy as greater drift.
Representation drift: DANN aligns new encoder outputs with the old representation using a discriminator and gradient reversal, while EMA supplies a stable cross-cycle reference.The joint objective is L_task + λL_adv, with λ approximately 5 × 10^-3; larger values can produce degenerate representations.
Recurrent training: Sharded recurrent training updates one user shard per day and cycles through all users every N days to increase fresh training signal.Once converged, this is reported to catch up to the latest trends.
Embedding compression: Matryoshka Dropout enables flexible token dimensions by truncating tokens to sampled leading prefixes and rescaling them during training.The method uses structured suffix dropout in a single pass rather than repeated downstream forward passes.
Embedding compression: INT4 serving achieves an 8× storage reduction versus FP32 while preserving approximately 80% of the full-precision embedding’s impact.Quantization-aware training exposes the encoder to simulated INT8 noise during training.

5 Experiments

AMBER is evaluated on industrial-scale ranking and retrieval using chronological petabyte-scale impression data, with both controlled quality and complementary predictive signal assessed. Results show gains from richer event representations, cached tokenization, unified tokenizers, pretrained alignment, and scaling.

Evaluation setup: Petabyte-scale impression logs and chronological splits evaluate ranking and retrieval under industrial feature, latency, and non-stationarity constraints.Main results use the full dataset; ablations use a 10% subsample, with metrics computed on unseen future dates.
Ranking: AMBER achieves the lowest NE, outperforming Incumbent (fair), Pointwise, HSTU-style input, and Semantic IDs + CU embeddings by 0.40%, 0.30%, 1.00%, and 1.60%, respectively.The comparisons use NE relative to the controlled Incumbent (fair) baseline.
Ranking: Blending AMBER with the Incumbent reduces Ensemble NE by 0.10% under a 10-minute delay and 0.16% under a 1-minute delay.Semantic IDs + CU embeddings yield smaller reductions of 0.03% and 0.05% under the same windows.
Retrieval: AMBER improves retrieval Soft Recall by 0.31–0.51% across checkpoint ages and retains a 0.31% improvement by Day 8, while CU embeddings become neutral.AMBER also outperforms CU embeddings by 0.28–0.32 percentage points.
Feature ablations: User features improve NE by 1.10%, while removing item features degrades NE by 6.65%, showing that both user-side state and item information add signal beyond behavior sequences.Semantic IDs alone remain insufficient, leaving a 2.40% gap to full AMBER; adding CU embeddings and non-CU item categorical features narrows it to 1.52%.
Tokenizer design: A shared tokenizer improves NE by 0.02% in both four-token and two-token schemas, while two tokens per event help unlimited context by 0.10% but hurt limited windows by 0.16%.These results indicate positive transfer across entity types and a trade-off between token budget and history coverage.

6 Conclusion

AMBER raises historical snapshot resolution by asynchronously compressing heterogeneous event features into cached Event Tokens, improving scaling and serving trade-offs. Its evaluation and design also highlight system-level compute considerations and a serving-time delay boundary.

6 Conclusion: AMBER asynchronously compresses heterogeneous event features into compact cached Event Tokens, enabling high-resolution histories without online feature materialization.The approach targets the serving-cost increase caused by higher historical snapshot resolution.
6 Conclusion: End-to-end joint training, total training-and-serving FLOP analysis, and drift mitigation are identified as AMBER’s three primary advantages.The drift mitigation uses domain-adversarial training combined with EMA updates during recurrent serving.
6 Conclusion: Asynchronous tokenization introduces feature delay, motivating short-retention high-resolution histories and on-demand tokenization as future work.The paper also proposes jointly modeling Event Tokens with text for personalized LLMs.

A Total Computational Cost and Sensitivity Analysis

The appendix estimates total computational cost by separating training, serving, traffic, candidate-scoring, utilization, and feature-materialization factors. AMBER’s asynchronous cached tokenizer makes event-side serving cost much smaller than user-model serving cost, while infrastructure-dependent coefficients limit precision.

A Total Computational Cost and Sensitivity Analysis: Ranking inference scores every candidate on every request without downsampling, unlike training, which uses logged impressions and heavily samples negatives.
A Total Computational Cost and Sensitivity Analysis: The serving-to-training workload amplification m_u is empirically estimated at 30–100×, with 30× used in primary plots; this range is ranking-specific.Retrieval workloads generally have more limited amplification.
A Total Computational Cost and Sensitivity Analysis: For AMBER, asynchronous cached Event Tokenization gives m_e ≪ m_u, whereas an uncached full-feature sequence sets m_e = m_u.
A Total Computational Cost and Sensitivity Analysis: Feature-materialization compute is mapped to the common FLOP axis using an infrastructure-dependent empirical power ratio, so only coarse ranges are reported.
A Total Computational Cost and Sensitivity Analysis: The first-order materialization model scales with served requests, history length, per-event feature counts, and empirically calibrated per-value costs.The calibration coefficients may differ across serving stacks.
A Total Computational Cost and Sensitivity Analysis: AMBER replaces variable-cost Semantic ID and item-feature materialization with a fixed-size cached token, affecting compute coordinates but not directly measured NE values.

B Baseline Encoder Architectures

The encoder study compares self-attentive Event Tokenizers with simpler alternatives and finds a four-layer bidirectional Transformer to offer the best practical trade-off. Reducing Transformer capacity worsens NE, while AMBER’s input schema remains separable from the downstream backbone architecture.

B Baseline Encoder Architectures: A four-layer bidirectional Transformer matches DHEN (×12) at comparable FLOPs, while PMA and Concat+FFN trail by 0.20% and 0.22% NE, respectively.
B Baseline Encoder Architectures: The Transformer is selected for greater flexibility and slightly higher practical efficiency through FlashAttention.
B Baseline Encoder Architectures: The Event Tokenizer processes projected event features with learnable [CLS] tokens and bidirectional Transformer layers before producing an Event Token.Concat+FFN instead concatenates projected features and applies a feed-forward network without self-attention.
B Baseline Encoder Architectures: Shrinking the Transformer to ×2 and ×1 increases NE by 0.09% and 0.19%, respectively.
B Baseline Encoder Architectures: The HSTU comparison changes the input schema while holding AMBER’s User LLM backbone fixed, isolating schema effects from architecture effects.

D Pointwise Baseline

The pointwise baseline reproduces pointwise information flow within the same User LLM backbone and single-pass autoregressive training. Its remaining gap to AMBER isolates the value of retaining high-resolution event information throughout history.

D Pointwise Baseline: The remaining difference supports preserving high snapshot resolution throughout history rather than only at the current query.
D Pointwise Baseline: The controlled baseline keeps the backbone, objective, compute budget, and autoregressive efficiency matched while changing whether high-resolution information persists in history.
D Pointwise Baseline: Each event uses a full-feature query token A, an item-only history token B, and an outcome token C, with masking enforcing pointwise information flow.A_i attends only to prior B and C tokens, while B and C cannot attend to A tokens.
D Pointwise Baseline: Pointwise improves over Incumbent (fair) by 0.1% NE but remains 0.3% behind AMBER.

E Robustness to Feature Materialization Latency

Feature materialization delays of 1–5 minutes are typical, but simulated delays degraded NE by 0.2%, suggesting inherent robustness to latency. Longer temporal gaps between encoder checkpoints instead produce accumulated Event Token drift.

1–5 minutes is a typical delay between event occurrence and feature materialization in realistic serving settings.
0.2% NE degradation resulted from randomly masking recent events during training, even when evaluation used the same offset.The result suggests the artificial masking removes useful training signal.
20-day gaps between encoder checkpoints produce color separation within Event Token clusters, indicating accumulated drift.With a 1-day gap, the two checkpoint distributions are well-mixed.

G Recurrent Training Stability

Recurrent training stability improves when drift mitigation is applied: DANN + EMA produces a smoother NE trajectory and more stable downstream predictions than an unmitigated run.

DANN + EMA yields a smoother NE trajectory during recurrent training than training without drift mitigation.
Without drift mitigation, NE exhibits higher variance across training steps.
The smoother DANN + EMA trajectory indicates more stable downstream predictions under identical hyperparameters.

H Pre-trained Initialization Dynamics

Pre-trained LLM initialization provides its largest NE advantage early in training, but the benefit decays to a small stable level after 20K steps.

∼2.2% NE advantage occurs early in training with pre-trained rather than random initialization.
The initialization advantage decays to a stable ∼0.2% after 20K steps.
Pre-trained initialization improves NE relative to random initialization throughout the reported training trajectory, with the advantage diminishing over time.