PaperlayerView exact revision on arXiv

Source-linked AI summary

GenCAR: Generative Counterfactual Alignment with Risk-Controlled Selection for Out-of-Distribution Recommendation

Qianqian Wang, Yunshan Li, Jiawen Zeng, Wenwu Gong, Lili Yang

arXiv:2609.02162v1cs.IRcs.LG

TL;DR

OOD recommendation must recover useful candidates under changing environments without controlling only ranking quality or candidate construction. GenCAR formulates this as α-VCR, using preference-grounded counterfactual supervision and calibrated selection; it controls proxy-label risk theoretically and improves OOD candidate recovery empirically.

  • Problem

    Existing OOD methods improve ranking or construct counterfactual candidates without jointly controlling proxy-label FDR for the served set.

  • Method

    GenCAR fixes stable preference while intervening on the environment, grounds offline LLM proposals through preference constraints, and applies calibrated set selection for α-VCR serving.

  • Results

    GenCAR establishes finite-sample proxy-label FDR control under exchangeability and positive regression dependence, with a BY guarantee under arbitrary dependence, and improves OOD candidate recovery.

  • Takeaways & Limitations

    GenCAR connects environment-intervened candidate construction with FDR-calibrated selection while remaining LLM-free during online serving.

  • Takeaways & Limitations

    The FDR guarantee applies to the pooled serving batch and does not provide a separate FDR guarantee for each user.

Abstract

from arXiv · show

Serving useful recommendations under distribution shift is crucial for balancing utility and risk in out-of-distribution (OOD) recommendation. However, most existing OOD methods improve ranking or construct counterfactual candidates without controlling the proxy-label false discovery rate (FDR) of the served set. In this work, we formulate OOD serving as the $α$-Valid Counterfactual Recommendation ($α$-VCR) problem to retain candidate support learned from counterfactual supervision while controlling proxy-label FDR, and propose GenCAR, which couples preference-grounded counterfactual supervision with calibrated set selection. In particular, GenCAR fixes the stable-preference representation while intervening on the environmental factor, grounds offline large language model proposals through preference anchors and trust-radius filtering, and uses conformal $p$-values for Benjamini--Hochberg selection. We theoretically bound conditional counterfactual approximation error and prove finite-sample, distribution-free control of proxy-label FDR under exchangeability and positive regression dependence, with a Benjamini--Yekutieli guarantee under arbitrary dependence. Extensive experiments audit realized proxy false discovery proportions and demonstrate that GenCAR consistently enhances OOD candidate recovery across diverse benchmarks.

1. Introduction

OOD recommendation must balance recovering useful post-shift candidates against admitting misaligned items. GenCAR addresses this by combining preference-grounded counterfactual supervision with calibrated selection that controls proxy-label FDR.

  • Motivation: Distribution shift can make rankings learned from logged interactions unreliable, while enlarging served sets may admit low-quality candidates.The shift may arise from changes in exposure, item popularity, or temporal context.
  • Limitations of prior work: Existing counterfactual-generation methods provide supervision but do not specify how stable preference should constrain intervention-based generation.Generated candidates may retain environment-specific patterns and fail after the shift.
  • Research gap: Existing methods do not jointly construct environmentally intervened candidates and control their proxy-label FDR during serving.This creates the central α-VCR problem: retaining counterfactual candidates while controlling proxy-label FDR.
  • GenCAR: GenCAR formulates OOD serving as α-VCR and connects preference-grounded counterfactual supervision with calibrated set selection.The framework transforms offline LLM supervision into α-valid served sets while remaining LLM-free during online serving.
  • Theory: GenCAR provides conditional counterfactual approximation bounds, finite-sample proxy-label FDR control, BH-family maximality, and a proxy-transfer error bound.The FDR guarantee is established under exchangeability and positive regression dependence, with a BY extension for arbitrary dependence.
  • Empirical results: Across MovieLens-100K, Coat, and Amazon-Book, realized proxy FDP remains below α and Recall@10 improves by 11.0%, 19.1%, and 43.5% over CausalVAE backbones.The experiments also report consistent improvement in OOD candidate recovery across the evaluated candidate pools.

2. Background

The paper models OOD recommendation as an environmental shift that should preserve stable preference while changing environment-dependent variation. It formalizes candidate serving as pooled proxy-risk control over a ranked candidate family.

  • OOD setting: OOD serving differs between training and test because exposure policy, item popularity, or temporal context can change.The training and serving pair distributions are therefore modeled as P_train ≠ P_test.
  • Causal model: The SCM separates stable preference z_c, environmental factor z_e, and idiosyncratic noise η in the interaction-generation process.This factorization distinguishes preference-related information from environment-dependent variation.
  • Environmental intervention: The environmental-shift assumption keeps p(z_c), p(η), and the interaction mechanism invariant while allowing p_e(z_e) to change.The intervention do(z_e := z̃_e) changes the environment while preserving the user’s stable preference.
  • Serving challenge: Generative candidate recovery can find post-shift items, but ranking scores and fixed or uncalibrated selection thresholds do not control retained misalignment.The formal objective is pooled risk control over the candidate family Q(B).
  • Risk and support: The selection rule Γ returns a served set from the pooled candidate family and is evaluated by proxy-label risk and retained candidate support.The retained support is measured by E[|S|], while proxy-label FDR measures expected misaligned-item proportion.
  • α-VCR formulation: α-VCR seeks a valid selection rule whose pooled proxy-label FDR is at most α while retaining as much candidate support as possible.An empty served set has zero FDP and represents abstention; finite calibration data are required because the target proxy-null distribution is unknown.

3. GenCAR

GenCAR is a two-stage framework that constructs preference-grounded counterfactual supervision offline, calibrates proxy-misalignment evidence, and applies risk-controlled selection during serving. It fixes stable preference while intervening on environment, filters LLM proposals, and uses conformal evidence with BH or BY to form pooled served sets.

  • Framework: GenCAR constructs preference-grounded counterfactual supervision offline, then calibrates proxy-misalignment evidence and selects a variable-size served set.The accepted counterfactuals train a ranker, while a disjoint calibration split supports conformal selection.
  • Preference-grounded counterfactual construction: The backbone separates stable preference from environmental variation, using only the stable-preference representation in the ranking score.This keeps the preference coordinate fixed while making environmental intervention operational.
  • Preference-grounded counterfactual construction: GenCAR grounds offline LLM proposals with preference anchors and trust-radius filtering before using accepted proposals as item-level supervision.Invalid catalog identifiers are removed, and accepted items are used for counterfactual fine-tuning; LLM proposals are not served directly.
  • Theoretical guarantees: GenCAR proves a conditional counterfactual approximation bound after trust-radius filtering, separating approximation error from the target intervention.The bound concerns discrepancies among target, anchor-projected, and LLM-realized distributions.
  • Proxy-label risk calibration: GenCAR trains an alignment predictor on offline proxy labels and uses a disjoint proxy-null calibration split to construct conformal evidence for serving candidates.The encoder remains fixed while the predictor is trained, and calibration scores are required to be exchangeable with target proxy-null scores.
  • Risk-controlled serving: BH selects the largest admissible set in its nested step-up family and controls pooled proxy-label FDR under stated exchangeability and dependence conditions; BY covers arbitrary dependence.The FDR certificate applies to the pooled served set, whose membership decisions are projected onto user-level lists rather than guaranteed separately per user.

4. Theoretical Analysis

GenCAR’s analysis bounds counterfactual construction error and establishes calibrated selection guarantees for proxy-label risk. Under stated exchangeability and dependence conditions, its conformal BH/BY procedures control pooled proxy-label FDR while selecting admissible served sets.

  • Overview: GenCAR’s analysis covers counterfactual approximation, nested BH selection, conformal validity, proxy-label FDR control, arbitrary-dependence control, and proxy-to-causal risk transfer.These results address both construction fidelity and risk-controlled serving.
  • Counterfactual approximation: Trust-radius filtering yields a conditional approximation bound that separates anchor-projection and LLM-realization errors and accounts for retained probability mass.The bound compares the realized counterfactual distribution with the target intervention on the accepted region.
  • Selection structure: The BH candidate sets are nested, and BH returns the largest member satisfying its step-up condition at a fixed risk level.Increasing the risk level produces an ordered control over served-set size.
  • Proxy-null validity: Under conditional exchangeability, proxy-null conformal p-values are super-uniform, remaining conservative when scores contain ties.This validity result supplies the evidence required for calibrated FDR selection.
  • Proxy-label risk control: Under exchangeability and PRDS, BH controls pooled proxy-label FDR at level α for the served candidate-pair set.The guarantee is finite-sample and distribution-free when conditioned on learned artifacts and the candidate family.
  • Dependence and causal transfer: Under arbitrary dependence, the BY correction α/H_N provides the same proxy-label certificate, while a separate corollary transfers it toward causal risk through Δ_proxy.GenCAR uses BH under PRDS and BY under arbitrary dependence.

5. Experiments

Experiments evaluate GenCAR’s calibration, OOD candidate recovery, mechanisms, and deployment profile across temporal, exposure, and popularity shifts. Results report controlled proxy FDP, improved Recall@10, and an LLM-free online path.

  • Experimental Setup: GenCAR is evaluated on temporal, exposure, and popularity shifts using ML-100K, Coat, and Amazon-Book, with additional datasets for calibration and partial offline coverage.Recall@10 is the primary ranking metric, with NDCG@10/20 and IPS-weighted evaluation also reported; ranking results average three seeds.
  • Risk Control and Calibration: GenCAR keeps realized proxy FDP below α across all five diagnostic candidate pools while retaining variable amounts of candidate support.At α=0.30, realized proxy FDP values are 0.188, 0.195, 0.089, 0.054, and 0.048 across Coat, ML-100K, ML-1M, Amazon-Beauty, and Amazon-Book.
  • Risk Control and Calibration: Larger positive calibration support is associated with higher BH retention, while popularity-stratified calibration addresses heterogeneous score regions on the same Amazon-Book pool.Fig. 3 compares BH pass rate with positive calibration support and global versus popularity-stratified calibration at α=0.10.
  • OOD Candidate Recovery: GenCAR achieves the strongest LLM-free top-10 recovery across all three shifts and ranks first in 10 of 12 LLM-free dataset–metric comparisons.Recall@10 gains over corresponding CausalVAE references are 11.0%, 19.1%, and 43.5%, with positive gains in all nine dataset–seed comparisons.
  • OOD Candidate Recovery: GenCAR exceeds TallRec on ML-100K and Coat and approaches it on Amazon-Book while retaining embedding-based online scoring.Recall@10 is 0.0950 versus 0.0675 on ML-100K, 0.0761 versus 0.0637 on Coat, and 0.0066 versus 0.0070 on Amazon-Book.
  • Mechanism and Sensitivity Analysis: A 12.3% gain over matched random augmentation isolates counterfactual content, while LLM novelty and recovery gains vary independently.DeepSeek-Chat produces 88.6% novel candidates with an 8.0% Recall@10 gain; V4-Flash and V4-Pro reach 100% novelty with gains of 18.7% and 11.5%.
  • Mechanism and Sensitivity Analysis: Trust-radius filtering has stable operating regions, and preference-factorized generation can expand a shortlist with a relevant held-out item.The largest ML-100K gain occurs at δ=0.5, Coat is stable around δ=0.7, and one of six Coat User 240 candidates matches a held-out positive.
  • Deployment Profile: Offline counterfactual generation preserves an LLM-free, sub-millisecond online serving path, including when offline generation covers only some users.The standard pooled selector remains the certified output in the partial-coverage setting.

6. Related Work

Related work spans OOD recommendation, counterfactual candidate construction, generative recommendation, and risk-controlled set selection. GenCAR positions itself at their intersection by explicitly controlling pooled proxy-label FDR for intervention-generated candidates.

  • OOD and Counterfactual Recommendation: OOD recommendation methods address changing interaction distributions through causal representations, robust objectives, and generative approaches.These methods target ranking utility under distribution shift but generally learn from training-environment interactions.
  • Risk-Controlled Selection: Conformal recommendation controls FDR for selected sets from fixed candidate families paired with observed relevance, alignment, or unwanted-content labels.Existing guarantees therefore differ from GenCAR’s intervention-generated candidate setting.
  • GenCAR’s Position: GenCAR connects OOD candidate construction with predictive inference by making pooled proxy-label FDR the explicit serving constraint.Its positioning compares OOD scope, candidate construction, certified target, and online LLM usage across method families.

7. Conclusion

The paper presents GenCAR as a preference-grounded, risk-controlled framework for OOD recommendation with theoretical guarantees and empirical gains. Its guarantees remain scoped to proxy labels and their transfer assumptions.

  • Conclusion: GenCAR combines environment-intervened counterfactual construction with FDR-calibrated selection through the α-VCR formulation.The framework targets preference-grounded candidate support while controlling proxy-label risk.
  • Conclusion: The theory bounds counterfactual approximation error and establishes finite-sample proxy-label FDR control under exchangeability and PRDS, with transfer discrepancy characterized separately.The conclusion also notes a BY-style extension only as described in the theoretical results.
  • Conclusion: Empirically, realized proxy FDP stays below α across five evaluated pools and top-10 OOD candidate recovery improves over corresponding CausalVAE backbones.Offline LLM generation and proxy labeling preserve a sub-millisecond, LLM-free online path.
  • Conclusion: GenCAR places counterfactual OOD serving on an explicit statistical footing, with guarantees scoped to proxy labels and their transfer assumptions.This scope defines the boundary of the paper’s risk claims.

CRediT authorship contribution statement

The authors contributed across conceptualization, methodology, software, validation, formal analysis, investigation, writing, supervision, and resources.

  • CRediT authorship contribution statement: Qianqian Wang led conceptualization, methodology, software, validation, formal analysis, investigation, and the original draft.Yunshan Li, Wenwu Gong, and Lili Yang contributed to review and editing; Gong also contributed methodology and conceptualization, while Yang provided supervision and resources.

A.1. Proof of Proposition 1

The proof establishes Proposition 1 by applying the total-variation triangle inequality, taking a supremum, and combining the resulting bounds. A separate argument then proves the proposition’s second claim.

  • The proof begins with the triangle inequality for total variation.
  • Taking the supremum over B and combining the two bounds proves Proposition 1’s first claim.
  • The proof separately establishes Proposition 1’s second claim.

A.2. Proof of Lemma 1

The lemma proof constructs a nested family by adding candidates in increasing p-value order. The largest index satisfying the BH inequality identifies the selected family member.

  • Each S_k+1 contains all pairs in S_k plus one pair with the next smallest p-value.
  • The construction gives S_k ⊆ S_k+1 for every k ∈ {0, …, N−1}.
  • The index k̂_α is the largest index satisfying the BH inequality, so Ŝ_kα is the largest nested-family member meeting the step-up condition.

A.3. Proof of Lemma 2

The proof establishes super-uniformity of the weak-rank p-value and derives BH-based guarantees under conditional PRDS, with a Benjamini–Yekutieli adjustment for arbitrary dependence. It then applies these results to selected sets and the proxy discrepancy statement.

  • Proof of Lemma 2: For a proxy-null candidate, the proof conditions on learned artifacts and the candidate family and ranks its test score among calibration scores.
  • Proof of Lemma 2: Exchangeability bounds the number of indices whose upper rank is no greater than r for every r ∈ {0, …, n_0 + 1}.
  • Proof of Lemma 2: The implemented weak-rank p-value is super-uniform, and ties preserve the bound by increasing the upper rank.
  • Proof of Theorem 1: Conditional PRDS and super-uniform proxy-null p-values allow the Benjamini–Hochberg result to establish Theorem 1 after averaging over conditioning variables.
  • Proof of Corollary 1: Under arbitrary dependence, the Benjamini–Yekutieli procedure replaces α with α/H_N, yielding Corollary 1 after conditional application and averaging.
  • Proof of Corollary 2: Applying Theorem 1 to S_α(B) proves Corollary 2, while the empty selected set gives Δ_proxy(S_α(B)) = 0.
Loading 2609.02162v1…