PaperlayerView exact revision on arXiv

Source-linked AI summary

CRAMER: Control via Request-Aware Masking for Editing Recommenders

Zhiyuan Julian Su, Naihe Feng, Zhen Luther Qin, Ga Wu

arXiv:2608.25370v1cs.IRcs.AIcs.LG

TL;DR

Sequential recommenders are inflexible to immediate user requests, while existing adaptation methods can be costly. CRAMER uses request-conditioned masks to steer frozen recommender parameters, and experiments show it outperforms strong baselines across datasets and metrics with minimal overhead.

  • Problem

    Sequential recommenders remain inflexible to immediate natural-language requests, which may conflict with historical preferences or express nuanced constraints requiring controllable adjustment.

    Source: §1 · pp. 1–2

  • Method

    CRAMER treats a natural-language request as a control input and uses learned structured parameter masks to steer a frozen sequential recommender without retraining.

    Source: §1 · p. 2, §5 · p. 9

  • Results

    CRAMER consistently outperformed strong request-aware baselines across datasets and metrics with minimal runtime and memory overhead, remaining significantly better in 41 of 48 paired tests (85.42%) after BH correction.

    Source: §4.2 · p. 6, §5 · p. 9

  • Takeaways & Limitations

    CRAMER offers a controllable, efficient approach to integrating immediate user intent into sequential recommendation.

    Source: §5 · p. 9

    Deployment may require safeguards against unintended recommendation steering or conflicts between short-term requests and users’ long-term interests.

    Source: Impact Statement · p. 9

Abstract

from arXiv · show

Sequential recommendation models, while powerful, have limited flexibility in responding to immediate user requests, making it difficult to adapt their recommendations to the user's timely interests. Unfortunately, existing user request adaptation methods often incur high computational overhead due to either 1) retraining the entire backbone network or 2) leveraging the inference ability of large language models (a.k.a. prompt engineering), limiting their applicability in large-scale recommendation services. This paper presents Control via Request-Aware Masking for Editing Recommenders (CRAMER), a framework that takes users' natural-language requests to immediately change sequential recommendation models' behavior. Specifically, inspired by the model control theory, CRAMER treats user requests as control signals to modulate frozen backbone parameters through masking, achieving instant adaptation to diverse requests while avoiding costly retraining. Experiments on multiple large-scale benchmark datasets show that CRAMER outperforms four state-of-the-art request-aware baselines across multiple recommendation metrics while achieving minimal overhead. Moreover, the proposed framework exhibits enhanced controllability and cross-domain adaptability, establishing a new paradigm for request-aware sequential recommendation.

1. Introduction

CRAMER uses request-aware parameter masking to steer a frozen sequential recommender toward immediate user intent. The paper reports stronger results than four request-aware baselines on large-scale benchmarks with minimal computational overhead.

Introduction

  • Natural-language requests can conflict with historical preferences, express negations or fine-grained constraints, and demand efficient real-time adaptation.The introduction frames these requirements as balancing immediate intent with long-term behavior, interpreting rich request semantics, and avoiding full fine-tuning.Source: §1 · p. 1
  • Input-level methods add control tokens or vectors (Li et al., 2023), while output-level methods rerank or filter candidates (Liu et al., 2024; Liao et al., 2026; Zhang et al., 2025).The paper says these approaches risk shallow intent representations, domain-specific pretraining, or costly request-time inference; CRAMER targets short-horizon adaptation between model updates, not replacement of retraining.Source: §1 · p. 1
  • Existing approaches treat the recommender backbone as a static black box, while CRAMER seeks posterior control by steering a frozen model’s parameters on the fly.This differs from traditional ante-hoc user-controllable recommendation, which conditions behavior in advance.Source: §1 · p. 2
  • CRAMER outperforms four state-of-the-art request-aware baselines on multiple large-scale benchmarks while adapting with minimal computational overhead.The framework treats a natural-language request as a control input and modulates frozen backbone parameters through masking.Source: §1 · p. 2
  • CRAMER mean-pools request-token embeddings, applies Gumbel–Top-k to produce sparse row–column gates, and converts them into masks on selected frozen Transformer matrices.It offers three masking strategies for attention output matrices and feed-forward networks; training combines prediction loss with a KL sparsity regularizer.Source: §1 · p. 2

2. Background and Related Work

Transformer-based sequential recommenders are strong at modeling historical preferences but do not readily adapt to immediate natural-language requests. Prior request-aware and parameter-efficient approaches have tradeoffs that motivate lightweight, instance-level control of frozen recommenders.

Background and Related Work

  • Sequential recommendation predicts next interactions from item histories, with Transformer models such as SASRec and BERT4Rec serving as strong baselines.These models capture temporal dynamics beyond static user profiles (Pan et al., 2024); Transformer-based approaches became predominant (Fang et al., 2020), and the representative models are reported as strong across scenarios (Zivic et al., 2024).Source: §2 · p. 2
  • Prior request-aware methods use request augmentation, language-to-item representations, or LLM-based recommendation, but trade shallow representations or domain adaptation for inference cost and latency.Examples include user-generated tags or requests, language-to-item bridging or sequence modeling (Hou et al., 2024; Li et al., 2023), and LLM semantic enhancement, constrained generation, reranking, or prompting (Liu et al., 2024; Liao et al., 2026; Zhang et al., 2025; Xu et al., 2025).Source: §2 · p. 2
  • Parameter-efficient adaptation freezes backbones, but prompts offer coarse control, adapters add latency, and latent token insertion adds parameters.Masking instead learns task-dependent masks over weights or activations for reversible, fine-grained control without retraining (Zhao et al., 2020; Ansell et al., 2022; Litschko et al., 2022; Svirsky et al., 2024), though request conditioning remains underexplored.Source: §2 · p. 2
  • CRAMER targets instance-level, request-conditioned posterior control of a frozen sequential recommender at inference time, beyond prior task- or domain-level adaptation.Retraining or fully fine-tuning complex Transformer backbones is computationally prohibitive for real-time adaptation; frozen-backbone lightweight modules are common in recommendation, NLP, and CV (Su et al., 2025; Son et al., 2025; Qin et al., 2024).Source: §2 · p. 2

3. Methodology

CRAMER adapts a frozen sequential recommender to natural-language requests by converting request semantics into sparse binary masks over backbone parameters. Its variationally motivated objective and discrete gating procedure enable request-conditioned control without retraining the backbone.

3.1. Task Definition

  • CRAMER incorporates each user’s immediate natural-language intent alongside their interaction history to predict the next item, targeting control of a frozen sequential recommender.The framework maps requests to control signals that edit backbone parameters rather than relying on request-agnostic sequence modeling.Source: §3.1 · p. 3

3.2. Variational Motivation for Model Control

  • CRAMER uses a variationally inspired surrogate—not exact variational inference—with a Bernoulli sparsity prior and KL regularizer to motivate compact request-conditioned control.The practical controller uses exact hard k-hot masks sampled with Gumbel–Top-k and a straight-through estimator; the KL regularizes the Bernoulli relaxation and is normalized by gate dimension.Source: §3.2 · pp. 3–4

3.3. Request-to-Mask Adaptation

  • A pretrained-language-model encoder mean-pools request tokens into a semantic embedding that conditions the frozen recommender on immediate intent.The request encoder is designed to handle variable-length requests, including negations, constraints, and attribute-specific preferences.Source: §3.3 · p. 4
  • CRAMER targets FFN weights, attention output projections WO, or both, and projects request embeddings to row–column gate logits for these controllable matrices.Structured row–column gating reduces trainable parameters relative to assigning a mask to every parameter while providing fine-grained control.Source: §3.3 · pp. 4–5
  • Gumbel–Top-k sampling retains exactly k = ⌈(1−ρ)d⌉ active entries, which are decomposed into row and column gates to mask the selected backbone matrices.Applying these masks edits the frozen backbone with minimal overhead while retaining fine-grained control.Source: §3.3 · p. 5

3.4. Learnable Components and Discrete Optimization

  • Training freezes the recommender and learns the projection layer plus either none, the last layer, or all request-encoder parameters, using straight-through gradients for hard k-hot gates.The forward pass uses hard Gumbel–Top-k gates; the backward pass uses a temperature-controlled soft surrogate to optimize the trainable parameters.Source: §3.4 · pp. 5–6

4. Experiments and Evaluation

Across four datasets and two frozen sequential-recommendation backbones, CRAMER outperforms request-aware baselines, with significant improvements in most tested settings. Sensitivity, efficiency, and interpretability studies show request-conditioned masking’s performance depends on design choices, adds little inference overhead, and can shift recommendations in line with user intent.

4.1. Experimental Setup

  • Evaluation uses ReDial, KuaiSAR, Beauty, and CDs&Vinyl with frozen SASRec and BERT4Rec backbones; it compares four request-aware baselines and reports HR@k, NDCG@k, and MRR at k=10 and 20.Metrics are calculated using one ground-truth positive and 100 randomly sampled unobserved items as negatives.Source: §4.1 · p. 6, §4.2 · p. 6

4.2. Overall Performance

  • 41 settings (85.42%) remained significantly better than the strongest baseline after BH correction, and CRAMER achieved the best results across datasets, metrics, and both backbones.The comparison covers four request-aware baselines and uses Table 1’s overall results.Source: §4.2 · pp. 6–7, Table 1 · p. 7

4.3. Sensitivity Analysis of Hyperparameters

  • A moderate drop ratio generally preserves performance, while extreme values hurt; RoBERTa and ModernBERT outperform MiniLM and BERT in the reported PLM comparisons.Around ρ=0.10 usually balances request influence and backbone capacity; smaller datasets tend to favor larger ρ, while large, information-dense datasets favor smaller ρ. RoBERTa performs best on ReDial and Beauty, whereas ModernBERT performs best on CDs&Vinyl and KuaiSAR.Source: Figure 2 · p. 8, §4.3 · p. 8, Figure 3 · p. 8

4.4. Efficiency and Overhead

  • 0.018s runtime overhead over vanilla SASRec places CRAMER among the most efficient request-aware methods, with inference cost comparable to LLM-ESR and lower than Query-SeqRec and BLaIR.CRAMER requires one forward pass with lightweight projection and masking after training; REARANK’s runtime is over two orders of magnitude higher.Source: Table 2 · p. 8, §4.4 · p. 8

4.5. Mask Interpretability

  • Across six romance-related request types, CRAMER shifts top-10 recommendations in the intended direction for clear, ambiguous, and rare-term positive or opposite requests.Clear requests produce substantial increases or decreases in romance-related recommendations; ambiguous requests produce smaller shifts, and rare-term requests remain actionable despite increased variance.Source: Table 3 · p. 9, §4.5 · p. 9

5. Conclusion

CRAMER adapts sequential recommenders to natural-language requests by using request-conditioned masks to control frozen Transformer backbones without retraining. Across four benchmark datasets and two backbones, it outperforms strong request-aware baselines across multiple metrics with minimal runtime and memory overhead.

5. Conclusion

  • CRAMER encodes each natural-language request into a semantic embedding and projects it into structured row–column masks that modulate frozen Transformer backbones without retraining.The masks provide fine-grained and efficient control of the backbone.Source: §5 · p. 9
  • CRAMER formalizes request-aware sequential recommendation and uses a variationally motivated training objective with KL regularization and STE for stable, efficient optimization.Source: §5 · p. 9
  • Across four benchmark datasets and two backbones, CRAMER consistently outperforms strong request-aware baselines across multiple metrics with minimal runtime and memory overhead.The conclusion presents these results as evidence for controllable, efficient, and scalable integration of immediate user intent into sequential recommendation.Source: §5 · p. 9

Impact Statement

CRAMER enables deployed recommenders to respond to immediate natural-language requests without retraining, potentially reducing deployment overhead and energy use. Its increased controllability also raises concerns about inappropriate steering and conflicts with users’ long-term interests.

Impact Statement

  • CRAMER adapts deployed recommender behavior to users’ immediate intents without retraining, potentially improving interaction quality while reducing computational overhead and energy consumption.It avoids costly backbone fine-tuning and large language model inference at deployment time.Source: Impact Statement · p. 9
  • Request-conditioned adjustments may steer recommendations inappropriately or conflict with long-term interests, motivating system-level safeguards and human-centered design.The method is designed for moderate adjustments rather than complete overrides of learned preferences.Source: Impact Statement · p. 9

A. Derivation of the Training Objective

The derivation uses a request-conditioned variational lower bound to motivate a tractable surrogate objective for editing a frozen recommender. In practice, it combines supervised prediction with an analytic sparsity penalty while retaining hard k-hot masks, so it is not a strict ELBO.

A. Derivation of the Training Objective

  • A request-agnostic factorized Bernoulli prior matches the realized keep ratio, while a request-conditioned variational distribution over binary masks yields a lower bound via Jensen’s inequality.The prior mean matches the hard budget of k = ⌈(1 −ρ)d⌉ active gates, subject to the ceiling operation.Source: A. Derivation of the Training Objective · p. 13
  • Training uses hard k-hot Gumbel–Top-k masks in the forward path and a softmax straight-through estimator for gradients, alongside a normalized analytic Bernoulli KL with prior mean k/d.At inference, deterministic Top-k on logits produces the exactly k-hot mask without Gumbel noise.Source: A. Derivation of the Training Objective · p. 14
  • The per-user surrogate combines supervised next-item prediction under the edited backbone with an analytic Bernoulli KL sparsity penalty, then averages the objective across users.The predictive term replaces negative log likelihood; the KL is computed from sigmoid logits and normalized by the mask dimension.Source: A. Derivation of the Training Objective · p. 14
  • The predictive term can be viewed as a single-sample Monte Carlo estimate drawn by Gumbel–Top-k, paired with an analytic KL regularizer computed on the logits.The supervised next-item loss stands in for negative log likelihood.Source: A. Derivation of the Training Objective · p. 14
  • The objective is not a strict ELBO because hard k-hot sampling conflicts with the independent-Bernoulli KL; sparse-gating work uses such relaxations or STE, and the authors report stable optimization, smooth activations, and reliable request effects across datasets and backbones.The cited practices include Bengio et al. (2013), Maddison et al. (2016), Jang et al. (2016), and Louizos et al. (2017).Source: A. Derivation of the Training Objective · p. 14

B. Details of Experiments · B.1. Data Preprocessing

The experiments preprocess four datasets into RecBole-style files using chronological histories, dataset-specific positive-interaction filters, and, for three datasets, random downsampling. Table 4 reports processed-dataset statistics, including item, interaction, and request-length counts.

B.1. Data Preprocessing

  • The preprocessing covers ReDial, KuaiSAR, Beauty, and CDs&Vinyl, producing RecBole-style atomic files for training and evaluation.Only items and interactions passing the preprocessing filters are retained in the .inter and .item files.Source: §B.1 · p. 15
  • Requests are built from information in the three interactions preceding each timestamp, with the current interaction as an optional fallback when history is absent.The concatenated prior-interaction information can include title, content, category, and search keyword.Source: §B.1 · p. 15
  • Positive-instance filtering keeps positive interactions for ReDial and KuaiSAR, but only ratings of at least 4.0 for Beauty and CDs&Vinyl.Source: §B.1 · p. 15
  • Random downsampling is applied to KuaiSAR, Beauty, and CDs&Vinyl because of computing-budget limits.Source: §B.1 · p. 15
  • Table 4 summarizes the processed datasets by item count, interaction count, and average request-character count.Each interaction contains a request; the supplied passage does not provide the table's numeric values.Source: Table 4 · p. 15, §B.1 · p. 15

B.2. Training of Backbones · B.3. Optimal Settings

The SASRec and BERT4Rec backbones used RecBole default settings. CRAMER’s hyperparameters were tuned for each backbone–dataset configuration using validation NDCG@10, with the selected settings summarized in Table 5.

B.2. Training of Backbones

  • Both SASRec and BERT4Rec backbones were trained with RecBole default settings for each dataset.The paper refers to RecBole v1.2.1 for the specific settings.Source: §B.2 · p. 15

B.3. Optimal Settings

  • Optimal hyperparameter settings were selected per backbone–dataset configuration from predefined search ranges using validation NDCG@10.Table 5 summarizes the selected settings.Source: §B.3 · p. 15, Table 5 · p. 16
  • The mask drop ratio ρ was searched over {0.05, 0.10, 0.15, 0.20, 0.25}, and KL weight λKL over {0.1, 0.2, 0.3, 0.4, 0.5}.Source: §B.3 · p. 15
  • The request encoder PLM was selected from {BERT, RoBERTa, MiniLM, ModernBERT}, with fine-tuning set to none, last, or all.These regimes mean fully frozen, last-layer fine-tuning, or end-to-end tuning, respectively.Source: §B.3 · p. 15
  • The masked backbone scope θM was chosen from {FFNs, WO, FFNs+WO}; Figure 4 evaluates these scopes using NDCG@10 across five evaluations averaged per setting.Source: §B.3 · p. 15, Figure 4 · p. 16
  • During training, gates were either sampled once and shared across the batch (shared=1) or sampled independently per instance (shared=0).Source: §B.3 · p. 15
  • Gumbel–Top-k sampling used a temperature schedule from 0.7 to 0.3, chosen from linear, exponential, or cosine annealing.Source: §B.3 · p. 15

B.4. Detailed Experiments · B.5. Efficiency and Overhead (BERT4Rec) · B.6. Controlled Scaling Study within BERT4Rec

CRAMER’s detailed experiments identify settings that perform best under different data and request conditions. In BERT4Rec scaling tests, it improves recommendation performance across tested depths while relative overhead decreases as the backbone grows.

B.4. Detailed Experiments

  • Across dataset sizes, masking FFNs and WO jointly is most often optimal on larger datasets, while either scope alone can be advantageous on smaller datasets.Overall performance remains relatively stable across masking scopes; the authors suggest constrained control may reduce overfitting with limited data.Source: §B.4 · p. 16
  • Last-layer request-encoder tuning tends to give stable gains over a frozen PLM, while end-to-end tuning benefits large, information-rich settings but may hurt low-data performance.The authors recommend last-layer tuning as a comparatively robust default; Figure 5 evaluates NDCG@10.Source: §B.4 · p. 16, Figure 5 · p. 17
  • CRAMER’s KL-weight sensitivity is moderate: λ_KL around 0.3–0.4 tends to work better on ReDial, 0.1–0.2 on KuaiSAR and CDs&Vinyl, and 0.2–0.3 on Beauty.The study recommends smaller weights for larger datasets and moderate weights for low-data regimes; Figure 6 reports NDCG@10.Source: §B.4 · p. 17, Figure 6 · p. 17
  • Independent per-instance mask sampling consistently yields higher NDCG@10 than sharing masks across a mini-batch, so the paper recommends non-shared sampling despite its slightly greater runtime overhead.The authors attribute the result to better alignment between each request and its control signal; Figure 7 evaluates the sampling regimes.Source: §B.4 · p. 17, Figure 7 · p. 18
  • Cosine temperature annealing most often performs best, linear decay is a reasonable alternative, and exponential decay tends to underperform after faster early cooling.The compared schedules anneal τ from 0.7 to 0.3; Figure 8 reports NDCG@10.Source: §B.4 · p. 18, Figure 8 · p. 18

B.5. Efficiency and Overhead (BERT4Rec)

  • Table 6 compares average per-request runtime and GPU-memory costs for the vanilla BERT4Rec backbone and the additional overhead of request-aware methods under identical settings.The supplied passage gives the table’s scope but no values for individual methods.Source: Table 6 · p. 18

B.6. Controlled Scaling Study within BERT4Rec

  • CRAMER improves recommendation performance across 4-, 6-, 8-, and 10-layer BERT4Rec backbones, while relative runtime and memory overhead decrease as depth increases.The KuaiSAR study attributes its linearly growing control dimensionality to row–column gates: α_l + β_l gates per matrix rather than α_lβ_l entrywise controls.Source: Table 7 · p. 19, §B.6 · p. 19

B.7. Further Discussion on PLM … C.3. Gradient Flow Through Discrete Gates

These appendices examine CRAMER’s sensitivity to request-encoder capacity and per-user behavior, detail how request-aware baselines are combined with backbones, and clarify the framework’s objective, reversible masking, and gradient flow.

B.7. Further Discussion on PLM

  • Across four datasets, CRAMER’s SASRec NDCG@10 strictly ranks request encoders Tiny < Mini < Medium < Base, consistent with capacity and suggesting sensitivity reflects encoder semantics rather than framework instability.The authors characterize this as robust in practice: even BERT-base achieves strong performance, and more capable encoders can provide richer request representations for finer control.Source: Table 8 · p. 19, §B.7 · p. 19

B.8. Intuitive User Case Study

  • CRAMER ranks ground-truth items highest for all five selected users under both SASRec and BERT4Rec, while request-aware baselines show fluctuations across users.The study uses CDs&Vinyl users and compares each ground-truth item’s rank under different backbones and request-aware methods.Source: §B.8 · pp. 19–20, Table 10 · p. 20

B.9. Details of Combination of Baselines and Backbones

  • Query-SeqRec concatenates request and history representations for candidate scoring, while BLaIR (Hou et al., 2024) fuses semantic similarity with the backbone’s collaborative score using a tunable weight.BLaIR’s fusion weight γ is tuned for each backbone and dataset.Source: §B.9 · p. 20
  • LLM-ESR (Liu et al., 2024) combines semantic and collaborative item and user representations, whereas REARANK (Zhang et al., 2025) sends backbone-ranked candidates and the request to an LLM reranker.Source: §B.9 · pp. 20–21

C.1. Clarification on the Variational Motivation

  • The variational perspective motivates structured sparsity and KL regularization; Equation (6) is a surrogate balancing predictive accuracy and control sparsity, not an exact ELBO objective.Source: §C.1 · p. 21

C.2. Non-Destructive and Reversible Masking

  • CRAMER applies request-conditioned masks multiplicatively at inference without changing backbone parameters, so removing the request restores original behavior without retraining or parameter updates.Source: §C.2 · p. 21

C.3. Gradient Flow Through Discrete Gates

  • CRAMER trains its discrete gating vector with a straight-through estimator and continuous backward relaxation, while restricting gradients to the request-to-mask module rather than the frozen backbone.Source: §C.3 · p. 21

Related papers

Loading 2608.25370v1…