Source-linked AI summary
The Differential Reasoning Router: Operationalizing Cost-Aware LLM Annotation in E-commerce
Cheng Lyu, Jingyue Zhang, Vinny DeGenova, Mengwei Li, Yuanli Pei
TL;DR
Cold-start e-commerce annotation must operate with scarce labels, uncertain reasoning value, and costly human review. DRR jointly routes cases among direct inference, reasoning, and human escalation using differential success estimates. In production, it reaches accuracy parity with the strongest confidence-based router while saving more than 60% of reasoning-token cost.
Problem
Cold-start rule-based annotation lacks stable, representative human labels for choosing between direct and reasoning models while catalog-scale human review is costly.
Method
DRR jointly routes among direct inference, reasoning, and human review using separate sample- and rule-level success probabilities plus budgeted abstention.
Results
More than 60% reasoning-token cost savings are achieved versus MaxConf while DRR reaches the accuracy range of methods that route examples to humans.
Takeaways & Limitations
Human escalations provide targeted ground truth for prompt iteration, fine-tuning, calibration, and rule refinement during the transition toward automated routing.
Takeaways & Limitations
The study evaluates one production e-commerce rule-based annotation workflow, so behavior may differ across domains, modalities, rule structures, and review processes.
Abstract
from arXiv · showhide
Large Language Models (LLMs) are increasingly used to annotate structured product data in e-commerce, but early deployment often begins as a cold-start problem: only limited pre-launch labels are available, the value of expensive reasoning is unknown, and human review is needed before the system can be trusted at scale. This challenge is especially common in rule-based annotation workflows, where each item must satisfy multiple business rules and both model errors and ambiguous rule boundaries affect final decisions. We introduce the Differential Reasoning Router (DRR), a cost-aware framework for cold-start LLM annotation that jointly optimizes model selection and human escalation. Rather than treating a reasoning model as a default fallback, DRR estimates separate success probabilities for a direct model and a reasoning model at both the sample and business-rule levels, enabling adaptive routing: easy cases are handled directly, reasoning is reserved for cases where it is expected to improve the decision, and likely double-failure or rule-disagreement cases are escalated to human annotators. The resulting labels provide targeted ground truth for prompt engineering, supervised fine-tuning, calibration, and rule refinement, enabling a gradual shift from human-heavy cold-start annotation toward high-confidence automated routing. In a production e-commerce workflow, DRR reaches accuracy parity with the strongest confidence-based router while achieving more than 60\% reasoning-token cost savings.
1 Introduction
Cold-start e-commerce annotation lacks reliable labels for choosing between direct and reasoning models, while human review is costly at catalog scale. DRR addresses this by jointly routing among models and human reviewers under limited ground truth.
- Motivation: Limited, costly, and rapidly outdated human labels complicate immediate deployment for new product attributes and changing business definitions.Random sampling may miss challenging cases, while early annotations can become outdated as rule definitions evolve.
- Motivation: Conventional routing does not directly estimate when expensive reasoning improves a decision or when both automated models are likely to fail.Scalar confidence can identify uncertainty but does not distinguish reasoning-helpful cases from double failures.
- Approach: DRR predicts separate direct- and reasoning-model success probabilities at both sample and business-rule levels.These differential estimates support routing easy cases directly, reasoning-helpful cases to the reasoning model, and likely double failures or rule disagreements to humans.
- Contribution: Reviewed labels support prompt iteration, supervised fine-tuning, calibration, and rule refinement as deployment progresses.This creates a gradual transition from human-heavy cold-start operation toward higher-confidence automated routing.
- Approach: The framework formalizes cold-start annotation as joint routing and label acquisition across direct inference, reasoning, and human review.Its policy is budget-constrained and designed to detect cases requiring abstention.
2 Related Work
Prior work studies LLM annotation, adaptive computation, model routing, and learning to defer, but cold-start rule-based annotation requires coordinating model choice, reasoning cost, human review, and label acquisition.
- LLMs as Data Annotators: LLM-generated labels can underperform human-generated labels, motivating strategic use of scarce human annotations during task calibration.Related work also proposes statistical criteria for deciding when LLMs can credibly replace human annotators.
- Adaptive Computation and Model Routing: Adaptive routing methods reduce unnecessary test-time computation by routing inputs to cheaper models, shorter reasoning, or selected sampling budgets.These approaches focus on allocating computation according to query complexity or target quality.
- Learning to Defer and Selective Prediction: Learning-to-defer methods study abstention to a fixed expert or predictive model, without coordinating direct and reasoning paths during cold-start deployment.They also do not treat deferred examples as ground truth for later prompt, rule, and policy refinement.
- Limits of Reasoning Models: Reasoning effort can yield diminishing or non-monotonic returns as problem complexity increases.This limits the assumption that additional thinking-token compute is a universally beneficial fallback.
3 Methodology
DRR formulates multimodal rule-based annotation as cost-constrained three-way routing among direct inference, reasoning, and human review. It learns model and rule reliability, estimates reasoning value, enforces budgets, and uses escalation labels for continuous improvement.
- Problem Formulation: Each multimodal product query is evaluated against k business rules, and validity requires satisfying every rule.DRR chooses among a low-cost Direct model, a higher-cost Reasoning model, and human review.
- Problem Formulation: The policy minimizes expected error subject to an average reasoning budget Btarget and human-referral budget Htarget.The incremental reasoning cost is ∆Cq = Cr(q)−Cd(q) ≥ 0.
- Router Architecture: Cached multimodal image, text, and interaction features feed a supervised routing model compatible with production latency.The encoder is not fine-tuned, and expensive features are computed once and cached.
- Router Architecture: System-level heads predict full-rule success, rule-level heads predict agreement on individual rules, and an ambiguity head predicts likely double failure.These outputs support both model selection and escalation when additional reasoning is unlikely to help.
- Budget Optimization: The learned multiplier λ acts as a reasoning-budget price, increasing selectivity when expected reasoning cost exceeds the budget.Alternating primal and dual updates adjust model parameters and the multiplier during training.
- Training Objective: Differential supervision learns whether each model matches human ground truth rather than whether the Direct model should imitate the Reasoning model.This captures cases where reasoning has negative value because the Reasoning model fails while the Direct model is correct.
- Training Objective: The Marginal Value of Reasoning is the predicted risk reduction from using the Reasoning model instead of the Direct model.Positive MVOR favors reasoning, while negative MVOR favors direct inference; the routing loss combines these estimates with incremental cost.
- Inference Policy: At inference, human review is triggered when predicted double-failure probability exceeds τamb or maximum model confidence falls below τconf.Remaining cases undergo cost-adjusted automated model selection, with thresholds tuned under a human-referral budget.
4 Experiments
The experiments evaluate DRR on a production e-commerce image-eligibility task under cold-start conditions, comparing cost-aware routing, reasoning, and human escalation. Results show that human abstention and ambiguity supervision are central to achieving competitive accuracy while reducing reasoning-token use.
- 4.1 Setup: The production task evaluates whether a product image can serve as the primary image under k=11 conjunctive business rules.A single rule failure invalidates the image, and the rules include both objective checks and subjective visual judgments.
- 4.1 Setup: The cold-start corpus contains 9,358 labeled examples split into 6,550 training, 1,403 validation, and 1,405 test examples.The labels came from mandatory pre-launch review rather than a separate production annotation campaign for DRR.
- 4.1 Setup: DRR is compared with Direct-only, Reasoning-only, Random, Oracle, MaxConf, and MVOR-only using shared model outputs and validation-selected review thresholds.Thresholded methods are evaluated under a ≤20% human-referral budget, while Table 1 reports realized test-set review rates.
- 4.2 Main Results: 79.1% accuracy is the ceiling for automated selection without human review, whereas methods that route difficult examples to humans achieve a higher accuracy range.Some samples are unsolvable by both automated models, so selecting between their outputs cannot resolve those cases.
- 4.2 Main Results: More than 60% reasoning-token cost savings versus MaxConf accompany DRR’s accuracy parity with methods that route examples to humans.MVOR-only shows that cost-aware model selection alone remains near single-model baseline accuracy; adding an ambiguity gate reaches the higher accuracy range.
- 4.3 Routing Zone Analysis: DRR’s ambiguity zone has the highest unsolvable-instance rate, while subjective rules concentrate joint failures and objective checks remain more stable.These diagnostics connect human escalation with borderline rule interpretation and targeted rule clarification or oversight.
- 4.5 Pareto Analysis: Figure 2 treats the human-referral threshold as a deployment control, with steep accuracy gains at low review budgets and an operating point marked for Table 1.Moving right allocates more reviewer capacity; moving left preserves human capacity but requires more automated decisions.
5 Conclusions
DRR turns cold-start LLM annotation into an explicit operating policy that allocates direct inference, reasoning, and human review according to expected reliability and value.
- DRR identifies traffic suitable for automation, cases deserving reasoning tokens, and examples that should become human-labeled ground truth.The framework supports controlled launch and subsequent improvement of prompts, calibration, training data, and rule definitions.
Limitations
The evaluation has boundaries in task coverage, model vintage, label-budget analysis, and the assumption that human review is ground truth.
- DRR is evaluated on one production e-commerce rule-based annotation workflow, so behavior may differ across domains, modalities, rules, and review processes.
- Model evolution requires regenerating candidate predictions and recalibrating or retraining DRR for each new model pair.Although model-agnostic by design, the experiments reflect the deployment-time model landscape.
- DRR does not evaluate label efficiency across training-label budgets, so its comparison concerns deployment with a mandatory pre-production labeled corpus.It is not evidence that DRR is preferable when all labels must be newly acquired.
- Treating human review as ground truth may import disagreement or inconsistency, especially for subjective rules, into accuracy and training signals.
A.1 Lead Image Eligibility Rules
The studied task determines whether a product image is eligible as the customer-facing primary image by applying a conjunction of business rules.
- Lead-image eligibility is a rule-based decision over a product image and listing metadata, not a single visual-classification judgment.
- An image is valid only when it satisfies business rules covering visibility, presentation state, cleanliness, set consistency, and prohibited visual elements.
- The 11 annotation rules combine objective visual checks with contextual judgments about focus, front visibility, and listed set composition.This mixture creates cases suited to direct inference, additional reasoning, or human review.
A.2 Qualitative Examples of Subjective Rules
Subjective-rule examples show that visual evidence can be present while the business-rule boundary remains underspecified, motivating ambiguity routing and rule-level diagnosis.
- Borderline cases include competing room context, angled products, small edited artifacts, and difficult-to-infer set quantities.
- Additional reasoning does not always resolve underspecified visual judgments, while rule-level predictions identify the business rule behind a system-level failure.
- Reviewed ambiguity cases provide training signals for router updates and evidence for refining the underlying rule specification.
A.3 Implementation Details
DRR keeps the underlying multimodal models frozen and concentrates adaptation in a lightweight routing layer. Cached embeddings, shared router components, and budget-aware validation make online inference and training inexpensive.
- Architecture: DRR is implemented as a lightweight supervised routing layer on top of frozen multimodal features.No additional fine-tuning of the underlying direct or reasoning LLMs is required.
- Architecture: Offline-cached image and text embeddings let online inference use a compact neural router without repeated feature extraction.
- Architecture: A shared small trunk supports system-level, rule-level, and ambiguity heads while preserving rule-level diagnostics.
- Training: Training uses regularized optimization with early stopping on validation performance.
- Training: The Lagrangian multiplier is updated during training to enforce the reasoning-budget constraint, while escalation thresholds are validation-selected under the target review budget.
A.4 Optimality of Threshold Policy
After human escalation is gated, DRR selects between direct and reasoning models using a cost-adjusted marginal-value threshold. Its multiplier adapts reasoning pressure to budget usage, while theory establishes stabilization properties rather than global optimality for the neural router.
- Policy setup: Conditional on queries passing human escalation, the remaining action set is binary: direct model Md or reasoning model Mr.
- Policy setup: The Marginal Value of Reasoning is the estimated risk reduction from routing a query to Mr, with incremental cost ∆Cq = Cr(q) − Cd(q) ≥ 0.
- Threshold policy: For fixed λ, per-query decisions decompose independently, routing to Mr when its expected value exceeds the cost-adjusted threshold and otherwise choosing Md.Ties are broken conservatively in favor of Md.
- Threshold policy: The multiplier λ acts as a learned shadow price: increasing it makes reasoning more expensive in the decision rule and sends fewer queries to Mr.
- Relaxed policy: For fixed temperature T > 0, routing probability to Mr increases with MVOR and decreases with incremental cost and λ.
- Dual update: When expected routing cost exceeds budget, λ increases; when under budget, λ decreases, preserving positivity through the exponential update.
- Guarantees and limits: The budget feedback controls the threshold MVOR(q) > λ∆Cq, but the non-convex neural router does not support a claim of global convergence or exact constraint satisfaction from theory alone.The final operating point is selected on validation data with separately tuned human-escalation thresholds.