Source-linked AI summary
Positional task conditioning for scalable defect detection across product families in large product catalogs
Soham Satyadharma, Gabriel Roccabruna, Suleiman A. Khan
TL;DR
Product catalogs contain costly product-family inconsistencies, and long contexts make simultaneous multi-error detection difficult. The paper decomposes detection into focused sub-tasks and distills the capability into one smaller model using Positional Task Conditioning. PTC improves detection from 52.14% to 87.62% F1 and approaches frontier performance at substantially lower cost, while remaining unvalidated beyond one domain and four tasks.
Problem
Product-family inconsistencies degrade customer experience, while long-context multi-error classification and scalable manual auditing remain difficult.
Method
The paper decomposes detection into focused attribute-level sub-tasks and distills them into one student model using task tokens at structural prompt boundaries.
Results
52.14% to 87.62% F1: divide-and-conquer prompting improves frontier-model detection, while PTC outperforms rationale-based distillation across five models and comes within 1.79% F1 of the frontier at upto 98% lower inference cost.
Takeaways & Limitations
PTC enables one smaller model to handle the decomposed sub-tasks while retaining performance close to the frontier model.
Takeaways & Limitations
Evaluation covers a single domain, and PTC’s scalability beyond four sub-tasks remains unexplored.
Abstract
from arXiv · showhide
Product families in large product catalogs suffer from inconsistencies such as duplicates and unit mismatches that degrade customer experience. Detecting these requires reasoning over multiple error types across lengthy product listings, where LLM classification quality degrades due to long-context limitations. We address this by decomposing detection into focused sub-tasks that reduce context and isolate error types, improving F1 from 52\% to 87\%. For scalable deployment, we introduce Positional Task Conditioning (PTC), which distills this capability into a single smaller model by reinforcing task identity at structural prompt boundaries. PTC outperforms rationale-based distillation across five models and two architecture families, achieving within 1.79\% F1 of the frontier at upto 98\% lower cost. Our system is deployed across multiple countries processing 10+ million product families.
1 Introduction
Product-family inconsistencies harm customer experience, while existing approaches do not adequately address intra-family detection at scale. The paper decomposes long-context, multi-error detection into focused tasks and proposes PTC for efficient single-model deployment.
- Product-family inconsistencies such as duplicates, unit conflicts, and attribute overstuffing degrade customer experience and increase operational costs.
- Intra-family inconsistency detection remains underexplored, while rule-based methods lack generalization and manual auditing cannot scale.
- Positional Task Conditioning distills the decomposed capability into one smaller model by reinforcing task identity at structural prompt boundaries.PTC addresses the deployment complexity of distilling separate models for each task.
- 52.14% to 87.62% F1: divide-and-conquer prompting improves detection over monolithic prompting by isolating error types and reducing context.The approach also improves precision by 54% and recall by 13% over monolithic prompting.
2 Related work
Related work spans multi-label classification, multi-task learning, knowledge distillation, and product-catalog applications. PTC combines these directions by conditioning one LLM at multiple prompt positions rather than using separate heads or a single prefix.
- Multi-label classification and LLMs: Multi-label LLM classification can violate output constraints and miss conditional dependencies, motivating decomposition into modular three-class sub-tasks.
- Multi-task learning: PTC conditions one LLM across sub-tasks with task identifiers at structural prompt boundaries, avoiding architectural changes and task-specific heads.
- Distillation in LLMs: Knowledge distillation transfers frontier-model capabilities to smaller students, addressing deployment latency and cost constraints.
- Positional task conditioning: Unlike T5-style single-prefix conditioning, PTC reinforces task identity at multiple semantically meaningful prompt positions.
- LLMs in product catalogs: Prior product-catalog LLM work covers recommendation, search, matching, categorization, and attribute extraction, but not product-family quality detection.
3 Method
The method formalizes product-family inconsistency detection as multi-label classification, decomposes it into focused attribute-level tasks, and distills teacher outputs into one task-conditioned student model. PTC inserts task tokens at prompt boundaries to preserve task identity during generation.
- 3.1 Problem formulation: The task maps a product family, its varying attributes, and error-specific instructions to error-type and attribute pairs.Families contain products represented by attribute-value pairs, and varying attributes identify where family members differ.
- 3.2 Divide and conquer (D&C) approach: D&C replaces one multi-label problem with independent focused classifiers that process only one attribute’s values and one task’s instructions.The decomposition uses a labeling map to translate sub-task labels back into error-type and attribute pairs.
- 3.2 Divide and conquer (D&C) approach: Each sub-task prompt contains a role introduction, task-specific instructions, family attribute values, and a structured output schema.
- 3.3 Positional task conditioning: RBD trains a student on teacher-generated reasoning chains across tasks, but lacks explicit task awareness when prompts share similar structures.
- 3.3 Positional task conditioning: PTC inserts dedicated task tokens before the introduction, before input data, and after the output schema to reinforce task identity during distillation.The approach extends rationale-based distillation while retaining a single student model and teacher-generated training data.
- 3.3 Positional task conditioning: The method hypothesizes that initial tokens prime the task, middle tokens re-anchor it at the instruction-to-data transition, and final tokens mark generation.These positional effects are evaluated across seven configurations.
- 4 Experimental setup: Synthetic evaluation data uses controlled error injections and an injection planner that respects dependencies between injected error types.
4 Experimental setup
The experiments evaluate divide-and-conquer defect detection, task conditioning, model generalization, token placement, and dataset construction. They compare prompting strategies using human-labeled and synthetic evaluation data across multiple models.
- Tasks: Four independent tasks classify overstuffing, unit systems, duplicates, and theme errors into egregious, non-egregious, or no-error labels.Each task pairs a semantically related egregious and non-egregious error variant.
- RQ2: PTC inserts task identifiers at structural prompt boundaries, while RBD uses reasoning chains without explicit task conditioning.The comparison also includes a non-finetuned D&C prompting baseline.
- RQ3: Generalization is evaluated across five models spanning Qwen3 and Mistral architecture families with shared training data and hyperparameters for RBD and PTC.The models range from Qwen3-8B to Qwen3-32B, Mistral-Nemo 12B, and Mistral-Small 24B.
- RQ4: Task-token placement is ablated across all 7 non-empty subsets of three prompt positions on the human-labeled evaluation set.The positions are the prompt start, before input, and prompt end.
- Datasets and metrics: Evaluation uses precision, recall, and F1 on 73,723 training family-attribute pairs and human-labeled and synthetic test sets.The human-labeled test set contains 2,296 pairs, while the synthetic set contains 11,130 pairs; labels are stratified by error type for training.
- RQ1: 87.62% F1 versus 52.14% for monolithic prompting measures the divide-and-conquer comparison on Claude Sonnet 4.5.Table 1 compares all-error-type prompting with focused sub-task prompting.
5 Results
The results show that decomposing defect detection improves accuracy substantially, while PTC transfers this capability across models and positional configurations at lower deployment cost. PTC’s strongest gains appear on challenging tasks, and qualitative evidence suggests positional task conditioning helps models act on detected duplicate evidence.
- PTC achieves the highest F1 across five Qwen and Mistral architectures, improving over RBD by 1.53%–6.30%.Qwen3 8B reaches 84.23% F1, matching the 14B model’s 84.62%.
- On synthetic data, PTC reaches 96.62% F1 on Qwen3 8B versus 94.59% for RBD, with higher scores reflecting cleaner text-only decision boundaries.The synthetic pipeline omits ambiguous cases such as visually distinct products with near-identical names.
- PTC’s largest real-data gains occur on difficult tasks: duplicates reach 77.84% F1 versus a 20.01% baseline, while unit-system gains are 3%.Theme and overstuffing improve by 5% and 4% over RBD, respectively.
- The SIE positional configuration reaches 84.2% F1, outperforming RBD at 77.9% by placing task tokens at the prompt’s start, input boundary, and end.All evaluated PTC configurations outperform RBD.
- PTC deployment costs approximately $1.17 versus $63 for Claude Sonnet 4.5, a 98% reduction, while D&C teacher costs remain comparable to monolithic prompting.The deployment measurement uses Mistral Nemo 12B on 8× H100 GPUs; prices reflect public pricing at writing.
6 Conclusion
The paper decomposes product-family defect detection into focused sub-tasks and distills the resulting capability with Positional Task Conditioning (PTC). PTC approaches frontier performance while reducing inference cost.
- F1 improves from 52.14% to 87.62% when frontier LLM detection is decomposed into focused sub-tasks.
- PTC outperforms Rationale-Based Distillation across five models and reaches within 1.79% F1 of the frontier model at up to 98% lower inference cost.
7 Limitations
The evaluation has several scope and evidence limitations: it covers one domain, four tasks, and proprietary data, while leaving task-count scalability and sample efficiency untested.
- Evaluation on a single domain leaves PTC unvalidated on other multi-label classification tasks and alternative domains.
- PTC consolidates four sub-tasks, but whether task conditioning scales to larger numbers of tasks remains unexplored.
- Training-data-size ablations were not conducted, leaving PTC’s sample efficiency relative to RBD untested.
- The proprietary dataset limits direct external access, despite detailed descriptions of the tasks and dataset composition.
B Error categorization
The paper categorizes product-family inconsistencies into four tasks with egregious and non-egregious variants, reflecting how consistency affects customer harm. Detection difficulty varies substantially, with duplicates especially challenging for baseline models.
- Four tasks distinguish egregious errors that impair customer decisions from non-egregious issues that gradually degrade browsing quality.
- Consistency mitigates severity: uniform overstuffing or theme errors remain interpretable, whereas inconsistent patterns actively confuse customers.
- 77.84% F1 on Qwen3 8B and 75.00% on Mistral Small 24B show PTC’s large duplicate-detection gains over baselines.
- PTC’s advantage over RBD is largest for duplicates and theme detection, while gains are more modest for overstuffing.
C.2 Synthetic data
On synthetic data, PTC achieves the highest F1 on nearly every task-model combination. Synthetic scores are higher overall because the data provide cleaner decision boundaries.
- Synthetic data produce substantially higher absolute scores, reflecting cleaner decision boundaries than real data.
- PTC achieves the highest F1 on nearly every synthetic task-model combination.
- 94.04% F1 on Qwen3 8B exceeds RBD’s 85.04% on duplicates, a reported 11% improvement.
- 93.78% F1 on Mistral Nemo 12B exceeds RBD’s 88.93% on duplicates, a reported 5% improvement.
D Qualitative examples
On duplicate examples, RBD identifies offending pairs but rationalizes them away, whereas PTC flags them correctly by maintaining the duplicate-detection criterion. PTC’s gains over RBD are statistically significant on four of five models.
- Qualitative comparison: RBD identifies the offending pairs but fails during execution by rationalizing them away rather than applying the duplicate criterion.The error reflects a failure to act on noticed evidence, not a failure to perceive it.
- Mechanism: PTC’s task conditioning keeps the duplicate-detection objective active as the model transitions from instructions to product data.The proposed mechanism places a task token immediately before the input, re-anchoring the objective at the instruction-to-data boundary.
- Qualitative comparison: PTC flags Dark Gray/Dark Grey and Nimbus Cloud/Crimson/Nimbuscloud/Crimson as duplicates, while RBD treats the spelling and spacing differences as intentional variants.The examples show that PTC detects duplicates that RBD labels as correct variation families.
- Statistical comparison: PTC’s gains over RBD are statistically significant on four of five models, while the Mistral Small 24B improvement is directionally consistent but not significant.The paired subsampled bootstrap test reports p < 0.05 for four models and p = 0.0686 for Mistral Small 24B.