Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation

Siliang Liu; Mohammad Ghasemi; Sapan Patel; Amin Banitalebi-Dehkordi

Source-linked AI summary

Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation

Siliang Liu, Mohammad Ghasemi, Sapan Patel, Amin Banitalebi-Dehkordi

arXiv:2609.05363v1Sep 4, 2026cs.LG

TL;DR

Trade-up recommendation requires meaningful upgrades at a scale where direct LLM inference is impractical. The paper distills retrieval-augmented LLM labels and rationales into an embedding-only student, then adapts it with product-type-specific few-shot training. The resulting system reaches AUC 0.941 after PT-TTT without serving-time LLM inference.

Problem
Direct LLM inference is operationally impractical for scoring hundreds of millions of product pairs in large e-commerce catalogs.
Method
A two-level framework distills retrieval-augmented LLM labels and rationales into a compact embedding-pair classifier, then fits lightweight product-type adapters over the frozen student.
Results
PT-TTT improves AUC from 0.924 to 0.941 while preserving inference without serving-time LLM calls.
Takeaways & Limitations
Rationale supervision primarily shapes the global student, while product-type-specific expert supervision provides the subsequent adaptation gain for catalog-scale scoring.
Takeaways & Limitations
Evaluation covers 29 product types and does not establish generalization to unseen categories.

Abstract

from arXiv · show

Trade-up recommendation identifies higher-quality alternatives that preserve a customer's purchase intent while offering upgraded benefits. Large language models (LLMs) can reason about such distinctions, but applying them directly to hundreds of millions of product pairs is operationally impractical. We introduce a two-level framework that distills LLM reasoning into an efficient non-generative student and adapts its decision boundary to product-type-specific trade-up criteria. At Level 1, a retrieval-augmented few-shot LLM teacher generates structured relation labels and natural-language rationales. These rationales supervise a compact embedding-pair classifier through alignment and contrastive objectives; at inference, the student uses only two precomputed 768-dimensional product embeddings, with no LLM calls or text generation. On a fixed human-annotated benchmark of 8,352 pairs, a 15.5M-parameter four-class reasoning-distilled student achieves AUC 0.924 (95% CI [0.918, 0.929]), compared with 0.912 for the four-class label-only student. At Level 2, product-type test-time training (PT-TTT) uses few-shot demonstrations to optimize lightweight category-specific adapters over the frozen student. PT-TTT improves AUC from 0.924 to 0.941 and average precision from 0.920 to 0.940. On a 100K-pair proxy catalog, the distilled student on a single eight-GPU machine is approximately 5,000x faster and 10,000x lower in estimated cost than direct LLM inference.

1. Introduction

Trade-up recommendation seeks directional, meaningful upgrades at catalog scale, but direct LLM scoring is operationally impractical. The paper therefore proposes a two-level framework combining reasoning distillation with product-type-specific adaptation and efficient inference.

Trade-up recommendation identifies higher-quality alternatives that preserve shopping intent while adding benefits beyond superficial product variation.
Hundreds of millions of product pairs make direct LLM inference costly and slow for large e-commerce catalogs.
The framework distills retrieval-augmented LLM labels and rationales into a compact non-generative student, then adapts it by product type.
Catalog-scale scoring uses the adapted student without serving-time LLM inference.

2. Related Work

Prior work treats product relations, rationale distillation, and test-time adaptation largely as separate problems. This paper couples them by globally shaping a discriminative pair representation with rationales and locally adapting its decision boundary using product-type supervision.

Product relations and LLM supervision: Trade-up is modeled as a directional relation requiring preserved shopping intent and an additional customer-relevant benefit.
Product relations and LLM supervision: The LLM operates offline as a black-box supervisor that produces fine-grained relation labels and textual rationales rather than serving online recommendations.
Rationale-guided representation distillation: Rationale-guided representation distillation transfers rationale geometry to an embedding-only pair classifier through alignment and contrastive matching, without decoding text.
Test-time product type-based adaptation: PT-TTT fits a low-rank adapter per product type using labeled demonstrations, then amortizes that adapter across all pairs in the category.
The framework combines global rationale-based representation learning with category-local decision-boundary specialization under catalog-scale constraints.

3. Task Setup and Data Preparation

The task predicts whether a candidate is a directional trade-up for a base product, using expert-labeled four-class relations and additional silver pairs within product types. The held-out golden benchmark supports evaluation of this catalog-scale relation-classification setup.

Task setup: A valid trade-up preserves shopping intent while adding benefits such as stronger branding, improved formulation, certifications, or better materials.
Expert-annotated data and golden benchmark: The expert corpus contains 17,200 pairs across 29 product types with four ordered relation classes.
Expert-annotated data and golden benchmark: 8,848 pairs form the teacher-demonstration and PT-TTT support pool, while 8,352 pair-disjoint examples constitute the held-out golden benchmark.
Silver supervision: Silver construction yields 1,019,241 semantically related candidate pairs across the same 29 product types.
Silver supervision: The black-box teacher receives product descriptions, type-specific criteria, and five retrieved expert demonstrations to generate four-class labels and concise rationales.

4. System Overview

The system first trains an embedding-only global pair classifier with task, rationale-alignment, and contrastive objectives, then adapts lightweight product-type modules before scoring. Both stages preserve efficient inference by avoiding generative LLM calls.

System overview: The two-level pipeline performs offline reasoning distillation into a frozen student, followed by product-type test-time training before scoring.
Embedding-pair student: The student consumes two precomputed 768-dimensional product embeddings and requires no product text, rationale, or LLM output at inference.
Embedding-pair student: Separate base and candidate branches plus an ordered base-to-candidate branch provide role-specific and asymmetric transformations.
Task supervision: The four-class objective preserves the teacher taxonomy, while binary supervision collapses non-trade-up classes into one negative class.
Reasoning distillation: Rationale alignment and contrastive distillation transfer semantic targets and pairwise relational structure during training, then their projection components are discarded at inference.
Product-type adaptation: PT-TTT fits lightweight LoRA adapters over the frozen student using labeled product-type support sets and reuses rationale alignment optionally during adaptation.
Product-type adaptation: Each adapter is optimized once per product type and amortized over all query pairs in that category.

5. Experiments

Experiments show that reasoning distillation produces a strong compact student, while product-type test-time adaptation further improves discrimination on the held-out golden benchmark. The resulting scorer also offers substantial speed and cost advantages over direct LLM inference.

Setup: 1,019,241 silver pairs train the students, while final evaluation uses a held-out golden benchmark of 8,352 human-annotated pairs.The shallow reasoning model has 15.5M parameters, compared with 65.9M for the deep model.
Main results: AUC 0.924 is achieved by the shallow four-class reasoning-distilled student, versus 0.912 for the same-architecture label-only student and 0.887 for the larger deep label-only model.The benchmark reports 95% CI [0.918, 0.929] for the reasoning-distilled student.
Main results: Rationale supervision leaves binary AUC unchanged at 0.911 but raises four-class AUC from 0.912 to 0.924, with paired ΔAUC = +0.013.The result indicates that rationale benefits depend on preserving fine-grained relation structure.
PT-TTT: At K=32, reasoning-guided PT-TTT increases AUC from 0.924 to 0.941 and AP from 0.920 to 0.940.Performance largely plateaus between K=16 and K=32, and label-only adaptation reaches nearly the same AUC, 0.940.
Scalability: Approximately 100K proxy-catalog pairs run about 5,000x faster and at 10,000x lower estimated cost with the distilled student than with direct LLM inference.The benchmark uses a single eight-GPU machine.

6. Discussion, Limitations, and Future Works

Discussion attributes the Level-1 gain primarily to rationale-guided global distillation and the Level-2 gain primarily to product-type-specific supervised adaptation. The evidence remains bounded by category coverage, support-data requirements, adaptation assumptions, and benchmark scope.

Discussion: Rationale supervision provides little benefit with binary labels but a clear gain with the four-class taxonomy.The component ablation also reports incremental silver-validation gains from rationale alignment and contrastive relational supervision.
Discussion: Pooled adaptation reaches AUC 0.929, while product-type-specific adaptation reaches 0.940 using the same support examples.Macro within-product-type AUC likewise rises from 0.910 to 0.925, indicating improved within-type discrimination rather than only score rescaling.
Discussion: Reasoning-guided and label-only PT-TTT are nearly identical at K=32, with AUCs of 0.941 and 0.940.Together, the results suggest that rationale guidance shapes the global student, whereas local supervised adaptation supplies most of the Level-2 gain.
Limitations: Evaluation covers 29 product types and does not establish generalization to unseen categories.PT-TTT also requires expert-labeled support examples and a separate optimization step for each product type.
Limitations: The benchmark is pair-disjoint but not explicitly product- or brand-disjoint, and its class prevalence may differ substantially from production candidate streams.Accordingly, stricter entity-disjoint evaluation remains future work, and benchmark precision and F1 should not be read directly as production positive predictive value.

7. Conclusion

The two-level framework distills black-box LLM supervision into an efficient embedding-only student, then specializes it with product-type test-time training. Rationale-guided distillation reaches AUC 0.924, while PT-TTT raises AUC to 0.941 without serving-time LLM inference.

AUC 0.924 is achieved by rationale-guided global distillation on the held-out human benchmark.The student transfers black-box LLM supervision into an efficient embedding-pair model.
PT-TTT improves the global student to AUC 0.941 without serving-time LLM inference.The second level specializes the student through product-type-specific adaptation.
Rationale supervision supports global representation learning, while product-type expert supervision supplies the subsequent adaptation gain.The framework assigns complementary roles to its two levels.
The separation enables LLM-derived semantic supervision for large-scale catalog scoring without generative inference at serving time.

A.1.1. Teacher Retrieval and Rationale Encoding

The teacher is a black-box frontier LLM that receives product descriptions, product-type criteria, and retrieved demonstrations, then emits a four-class relation label and rationale. Rationale embeddings are computed offline and reused for student training and PT-TTT support sampling.

The black-box teacher exposes only a generated relation label and rationale, not logits or hidden states.
Five nearest demonstrations are retrieved within the corresponding product type using Euclidean distance, irrespective of relation labels.Retrieved demonstrations are distinct from the queried pair.
The teacher receives retrieved examples, annotations, product descriptions, and product-type criteria, then emits one four-class label and a rationale of at most two sentences.
Teacher rationales are encoded offline into 768-dimensional vectors and stored with silver training data.The same human-annotated pool later supplies PT-TTT support examples.

A.1.2. Dataset Splits and Evaluation Statistics

The evaluation uses stratified silver data, a pair-disjoint human golden benchmark, and validation-selected operating points. Results compare lightweight and deep students, distillation objectives, product-type performance, calibration, and PT-TTT support-set effects.

Dataset and evaluation: 1,019,241 silver pairs are split by product type and relation class, while 8,352 pair-disjoint human pairs form the held-out golden benchmark.The golden benchmark is reserved for final evaluation and analysis.
Dataset and evaluation: Golden-set confidence intervals use 2,000 bootstrap resamples, with paired resampling for model comparisons.
Dataset and evaluation: AUC and AP are threshold-independent, whereas F1, precision, and recall use each model’s best-F1 threshold selected on silver validation.
Model and distillation: The student consumes precomputed 768-dimensional base and candidate embeddings through separate and ordered branches, with no text, rationale, or LLM output at inference.The branches provide role-specific and asymmetric transformations before classification.
Model and distillation: The shallow reasoning-guided model has 15.5M parameters, compared with 65.9M for the deep reasoning-guided model.The shallow and deep configurations differ in the number of attention layers.
Model and distillation: The contrastive objective combines InfoNCE with relational KL to transfer pairwise similarity structure beyond pointwise alignment.
Results: Figure 3 shows the shallow 15.5M reasoning-distilled student exceeding the 4× larger deep label-only student on golden-set AUC and AP, with non-overlapping 95% CIs.
Results: The student recovers recall and achieves higher F1 than the conservative teacher at a modest precision cost, with zero LLM calls at inference.