PaperlayerView exact revision on arXiv

Source-linked AI summary

IndustryLLM: Failure-Driven LLM Training for Industrial Procurement

Liang Ding, Zhiang Xu, Yuyang Sheng, Bin Chen, Songlin Bai, Run Zhu, Dingjun Wu, Hui Xu, Yandi Wang, Fulin Shi, Leilei Gan, Linlin Yu, Qihuang Zhong, Keqin Peng, Yalong Li, Chengfu Huo

arXiv:2609.31871v1cs.AIcs.CL

TL;DR

Industrial procurement must connect informal buyer requests to sparse supplier attributes without treating category relevance as proof of product eligibility. IndustryLLM adapts a model through failure-driven training and pairs request structuring with an evidence-gated eligibility interface. The paper reports offline query-structuring gains and production improvements in GMV, satisfied inquiries, and latency.

  • Problem

    Industrial buyers describe operating conditions and requirements that supplier records often fail to capture in consistent, sufficiently detailed CPV attributes.

    Source: §1 · p. 1

  • Method

    IndustryLLM adapts Qwen3.5-35B-A3B-Base through failure-driven CPT and SFT, with an evidence-gated interface evaluating product constraints under three-valued logic.

    Source: §1 · p. 2, §2 · p. 4

  • Results

    +2.97 percentage points in exact match for CPT+SFT over Base+SFT on offline procurement-query structuring; production trials reported +4.25% GMV and +8.3% satisfied inquiries.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The proposed division of labor assigns language-model adaptation to parsing buyer intent and an auditable evidence-gated interface to determining physical product eligibility.

    Source: §11 · p. 30

    Core data transformations were ablated only at 2B–4B proxy scales under single random seeds, and production trials evaluate bundled system interventions rather than isolated model effects.

    Source: §10 · pp. 29–30

Abstract

from arXiv · show

Industrial procurement requires language models to bridge informal buyer jargon, sparse marketplace attributes, and authoritative engineering standards under strict safety tolerances. We present IndustryLLM, an open-weight industrial language model trained from Qwen3.5-35B-A3B-Base (35B total parameters with ~3B activated per token, with the vision encoder frozen). Rather than relying on generic text scaling, we introduce a failure-driven adaptation recipe spanning continued pre-training (CPT) and supervised fine-tuning (SFT). CPT leverages a curated ~100B-token corpus integrating 5B tokens of national standards (e.g., GB/T) and technical archives, 10B tokens of de-identified real-world industrial transaction and inquiry records, and 60B tokens of general replay. To overcome register mismatch and factual brittleness, we systematically reconstruct an estimated 20B-token domain subset via multi-register rewriting across 10 genres and 8 writing styles, confidence-routed minimal factual editing, and error-targeted QA synthesis (resolving colloquial typos like '42-luo-mu' -> 42CrMo, expanding ambiguous codes like '16674' -> GB/T 16674, and clarifying conflicting dimensional specs). For downstream deployment, we formalize an evidence-gated constraint-evaluation interface enforcing three-valued logic where unverified product evidence remains unknown rather than satisfied. Offline evaluations demonstrate consistent gains on procurement-query structuring (+2.97 percentage points in exact match, 95% CI [2.11, 3.86] in No-Think mode), while randomized online A/B experiments in production yield substantial improvements (+4.25% GMV, +8.3% satisfied inquiries) alongside a latency reduction from 6-7 s to 1.5 s. Model weights and configs are released at https://huggingface.co/alibaba-multimodal-industrial-ai/IndustryLLM.

1 Introduction

IndustryLLM adapts a language model to parse industrial procurement requests, while a separate evidence-gated interface determines product eligibility; the introduction reports offline and production gains.

1 Introduction

  • Industrial buyers describe operating conditions and safety requirements, while supplier records often contain sparse CPV attributes with inconsistent names, units, and coverage.A catalog match may omit decision-critical details such as chemical concentration, temperature, seal construction, or flange interface.Source: §1 · p. 1
  • IndustryLLM adapts Qwen3.5-35B-A3B-Base through CPT and SFT, using diagnosed failures to guide data reconstruction, synthesis allocation, and supervision.The vision encoder and vision-text alignment modules remain frozen during adaptation.Source: §1 · p. 2
  • The approximately 100B-token CPT corpus combines standards and technical archives, marketplace records and inquiries, filtered industrial web text, and general-domain replay.Its sources include approximately 5B tokens of standards and archives, 10B of marketplace records and inquiries, 25B of industrial web text, and 60B of general replay.Source: §1 · pp. 2–3
  • Failure-driven reconstruction targets register mismatch, factual fragility, and model weaknesses through multi-register rewriting, confidence-routed minimal editing, and targeted QA synthesis.Rewriting covers 10 genres and 8 writing styles; editing uses verifiable diffs and abstains under uncertainty.Source: §1 · p. 3
  • SFT selects teacher-generated targets from four candidates per prompt, with Think retaining reasoning traces and No-Think keeping only the answer for latency-critical deployment.Task-specific filters enforce response-quality and chain-of-thought length constraints.Source: §1 · p. 3
  • The downstream interface separates request structuring from product verification: hard constraints require applicable, sufficiently fresh evidence, while unknown evidence remains unsatisfied.Decision-critical unknowns can trigger clarification, retrieval, or abstention.Source: §1 · p. 3
  • +2.97 percentage points in exact match for CPT+SFT over Base+SFT on offline procurement-query structuring, with randomized production trials reporting +4.25% GMV and +8.3% satisfied inquiries.The deployed inquiry stack reduced latency from 6–7 s to 1.5 s.Source: §1 · pp. 3–4
  • The paper releases an open-weight post-SFT checkpoint with 35B total parameters and approximately 3B activated per token, supporting Think and No-Think modes.The vision encoder remains frozen.Source: §1 · p. 3

2 Problem Formulation: From Demand to Eligible Products

Industrial procurement must distinguish category relevance from verified product eligibility: informal demands are converted into typed constraints, then candidate products are evaluated against authoritative evidence. The interface preserves unresolved requirements and treats missing or invalid evidence as unknown rather than satisfied.

2 Problem Formulation: From Demand to Eligible Products

  • The formulation separates informal buyer demand from product eligibility, requiring typed engineering constraints and evidence-based evaluation rather than category relevance alone.Source: §2 · p. 4

2.1 Demand Interpretation and Intent State

  • The demand interpreter maps a query and its context into an intent state under versioned category schemas and policy, resolving category and risk tier only when uniquely supported.Source: §2.1 · p. 4
  • Resolved constraints retain canonical properties, operators, normalized values, units, polarity, origin, and evidence references; missing decision-critical specifications remain unresolved.Source: §2.1 · p. 4

2.2 Evidence-Gated Eligibility Interface

  • The downstream eligibility interface applies category- and risk-specific policy to determine critical fields, admissible evidence, freshness, conversions, tolerances, and source precedence.Source: §2.2 · p. 5
  • Unverified, stale, missing, or conflicting evidence leaves a hard requirement unknown; unknown never satisfies it, and critical unresolved fields trigger clarification or abstention.Source: §2.2 · p. 5
  • Products enter the eligible set only after the intent is ready and every hard constraint is satisfied by valid, nonconflicting evidence in the product snapshot.Source: §2.2 · p. 5
  • Soft preferences rank products only after hard-constraint filtering; critical unknowns or unsupported derived requirements instead prompt abstention or targeted clarification.Source: §2.2 · p. 5

TAKEAWAY PROCUREMENT FORMULATION

  • The procurement formulation calls for auditable CPV constraints with provenance, clarification of missing duty context, and unknown status for unverified claims.Source: TAKEAWAY PROCUREMENT FORMULATION · p. 5

3 Related Work

Prior work addresses product understanding, domain adaptation, and synthetic supervision, but this paper combines failure-targeted data reconstruction with evidence-bounded procurement qualification. Its contribution separates demand interpretation from verification of current product evidence.

3.1 Product Understanding and Industrial Procurement

  • Industrial procurement requires hard constraints to disqualify items when unsatisfied or unverified, unlike graded-relevance or vector-retrieval approaches.Source: §3.1 · p. 6
  • The paper shifts from closed-book industrial question answering toward auditable query structuring and evidence-bounded supply verification.Source: §3.1 · pp. 6–7

3.2 Domain Adaptation and Data Reconstruction

  • The reconstruction framework targets register mismatch, latent factual bugs, and accessibility deficits through rewriting, minimal edits, and weakness-targeted QA pairs.Source: §3.2 · p. 7

3.3 Teacher-Generated Supervision and Evaluation Practice

  • Teacher-generated candidate trajectories are task-filtered for logical consistency and reasoning-length limits, an engineering choice evaluated under prompt-matched conditions rather than a new optimization objective.Source: §3.3 · p. 7
  • The paper reports proxy ablations, developmental diagnostics, offline metrics, and randomized online trials as separate evidence tiers rather than pooling them.Source: §3.3 · p. 7

3.4 Positioning the Contribution

  • The framework distinguishes parametric language understanding from non-parametric transaction evidence, withholding eligibility when evidence is incomplete.Source: §3.4 · p. 7

4 Approach Overview

IndustryLLM adapts a fixed base model through continued pre-training and supervised fine-tuning, then uses the resulting model for demand interpretation while delegating product qualification to evidence verification. Evaluations contrast supervision targets and checkpoints across reasoning-enabled and direct-response modes.

4 Approach Overview

  • The pipeline follows Base → CPT → SFT, using a 100B-token retained corpus for CPT and task-filtered teacher targets for SFT before downstream demand interpretation.Source: §4 · pp. 7–8
  • The evaluation contrasts original and filtered teacher targets, Base+SFT versus CPT+SFT, and Think versus No-Think inference regimes.Source: §4 · p. 8

4.1 Model Configuration

  • Parameter-scale comparisons describe relationships between scores and parameter counts, not runtime latency, hardware throughput, or operational cost.Source: §4.1 · p. 9
  • The shared Base+SFT and CPT+SFT setup is designed to isolate the foundational checkpoint, but no immutable experiment manifest prevents unreserved causal attribution to the full pipeline.Source: §4.1 · p. 9
  • IndustryLLM retains the base architecture, tokenizer, and vocabulary, with frozen vision modules; training changes data and supervision rather than model configuration.Source: §4.1 · p. 8, Table 2 · p. 9

5 Continued Pre-Training via Failure-Driven Reconstruction

IndustryLLM treats recurring model and data failures—not raw corpus volume or provenance labels—as the basis for industrial CPT data design. Proxy contrasts support multi-register rewriting, minimal factual edits, and QA synthesis, while the reported benchmark and production boundary remains explicit.

5.2 Failure-Driven Reconstruction of Industrial Data

  • Three diagnosed failures—register entrenchment, factual fragility, and knowledge latency—motivate reconstruction across more than 40 million documents totaling ≈20B tokens.The transformations are multi-register rewriting, confidence-routed minimal editing, and model-weakness-targeted QA synthesis.Source: §5 · p. 9, §5.2 · p. 11

5.1 Data Curation

  • The ≈100B-token retained source pool contains ≈40B domain tokens and ≈60B general replay, with the ≈20B reconstructed subset included within the domain subtotal.The curriculum shifts from a 4:6 domain-to-general ratio in main training to 2:8 during terminal annealing.Source: §5.1 · p. 10, Table 3 · p. 10

5.2.1 Genre-and-Style Diversification

  • +6.7 on IndustryBench was the highest domain gain for multi-register Setting C, while MMLU-Pro reached 63.77 and preserved general performance.Single-register adaptation gained +5.1 on IndustryBench but MMLU-Pro dropped from 63.51 to 62.56.Source: §5.2.1 · p. 13, Table 6 · p. 13

5.2.2 Model-Assisted Minimal-Edit Transformation

  • +13.9 percentage points: minimally edited data raised forced-choice preference to 66.1% versus 52.2% for unedited data after annealing.The authors call this suggestive evidence of data efficiency, not absolute factual acquisition, because option order was not randomized.Source: §5.2.2 · p. 14, Table 7 · p. 14

5.2.3 Model-Weakness-Targeted QA Synthesis

  • B > C > A: weakness-targeted QA beat uniform QA and plain text, but its margin over uniform sampling was modest.Against plain text, B−A was +0.0885 in log-likelihood with a 54.6% win rate; against uniform QA, B−C was +0.0113 with a 51.6% win rate.Source: §5.2.3 · p. 15, Table 8 · p. 15

TAKEAWAY FAILURE-DRIVEN DATA RECONSTRUCTION

  • At proxy scale, multi-register rewriting led dual-domain gains, minimal editing showed factual steering, and QA format helped activate passive knowledge, with only incremental benefit from weakness targeting.The marginal advantage of targeted over random QA warrants further scaling validation.Source: §5.2.1 · p. 13, §5.2.2 · p. 14, TAKEAWAY FAILURE-DRIVEN DATA… · p. 15

5.3 Reported CPT Configuration and Development Contrasts

  • Muon ended with a 0.030 lower final loss than AdamW (≈2.2% relative gain) and 12.6% lower mean global gradient norms in the reported short-horizon comparison.The full-scale 35B CPT run used Muon and WSD, but without a parallel full-scale AdamW run the paper does not claim unconditional production-scale optimizer superiority.Source: §5.3 · p. 16, Table 9 · p. 16

5.4 Benchmark Decontamination

  • IndustryBench is a developmental diagnostic, not a strictly blind testbed, because semantic overlap may remain after string-level filtering and its error diagnostics informed data-engineering iterations.Exact item matches were purged, but paraphrased or semantic overlap cannot be fully eliminated by the filtering method.Source: §5.4 · p. 16

6 Supervised Fine-Tuning

IndustryLLM uses teacher-reconstructed SFT targets to train operational checkpoints for reasoning-enabled and direct-response use. The comparison holds prompts and task distributions invariant, but does not isolate the rejection filter's causal contribution or match training compute.

6 Supervised Fine-Tuning

  • SFT aligns the model to technical instructions, multi-step engineering reasoning, and structured procurement outputs using teacher-reconstructed rather than raw heterogeneous targets.The prompt library and task distribution derive from Step-3.5-Flash-SFT; the choice over Dolci-Instruct-SFT followed internal comparisons without logged granular scores or thresholds.Source: §6 · p. 17

6.1 Teacher-Target Construction and Gating

  • Four teacher-generated candidates per prompt pass task-specific filters for correctness, explanation quality, and reasoning density, with trajectories over 8,192 tokens discarded.The highest-scoring candidate replaces the original target 1-to-1 under the standard autoregressive cross-entropy objective.Source: §6.1 · p. 17

6.2 Dual Operational Regimes: Think versus No-Think

  • Think trains on verified reasoning traces and answers, whereas No-Think trains only on final structured responses for latency-critical use.The No-Think variant is engineered for service-level agreements below 2 seconds.Source: §6.2 · pp. 17–18

6.3 Experimental Controls and Attribution Boundaries

  • The target comparison holds the base model, prompts, task mixture, epochs, and checkpoint selection fixed, but sequence lengths and aggregate compute are unmatched.Without isolated single-candidate or unfiltered Best-of-4 baselines, it measures the composite pipeline rather than the rejection filter alone; SFT also does not train one-pass emission of the full formal procurement state tuple.Source: §6.3 · p. 18, Figure 5 · p. 18

7 Core Capabilities and Benchmark Diagnostics

The section evaluates general-capability retention and industrial knowledge mastery, reporting mixed SFT-target effects and strong IndustryBench diagnostic scores for IndustryLLM.

7 Core Capabilities and Benchmark Diagnostics

  • The evaluation ledger separates proxy CPT contrasts, general-capability retention, IndustryBench domain diagnostics, query structuring, and production outcomes.Table 10 defines the experimental contrasts, endpoints, and epistemic boundaries across these tiers.Source: §7 · p. 18, Table 10 · p. 19

7.1 RQ1: General-Domain Capability Retention under SFT Target Reconstruction

  • Teacher-target reconstruction raises HumanEval-Plus by +8.6 points, GPQA-Diamond by +7.1, and LiveCodeBench by +2.2, but lowers MBPP-Plus by −1.8 and IF-Eval by −0.5.Across five multidisciplinary knowledge benchmarks, teacher targets retain slight advantages (+0.1 to +0.9 points); AIME26 remains at 80.0.Source: §7.1 · p. 20, Table 11 · p. 20
  • The single-run SFT ablation leaves supervised token exposure and optimization compute unmatched and does not trace retention across the full Base → CPT → SFT trajectory.The interpretation attributes divergent task results to possible task-specific inductive biases, not a causal isolation of those mechanisms.Source: §7.1 · p. 20

7.2 RQ2: Domain Knowledge Mastery and Parameter-Scale Diagnostics

  • IndustryBench comprises 2,049 expert-curated zero-shot procurement questions scored on a 0–3 scale, and its error profiles guided intermediate training iterations.The authors therefore treat it as an internal developmental diagnostic rather than an independent double-blind evaluation.Source: §7.2 · p. 21
  • IndustryLLM reaches 2.120 raw (70.7 on a 0–100 scale) and 2.030 Final SV, the highest safety-adjusted score among open-weight baselines.The comparison is developmental, not a protocol-aligned ranking: reference models used historical configurations, while IndustryLLM used Think mode.Source: Table 12 · p. 21, §7.2 · pp. 21–22
  • The raw IndustryBench score progresses from 1.64 at base to 1.80 after CPT and 2.120 after SFT, with the final increase combining teacher-target alignment and Think-mode activation.The checkpoint trajectory is a set of progress milestones, not an orthogonal parameter ablation.Source: §7.2 · p. 22

8 Procurement Tasks and Production Deployment

The section separates upstream query-structuring evaluation from bundled production-stack outcomes. IndustryLLM improves offline structuring and online procurement measures, while the authors qualify what those results establish.

8.1 Evidence Architecture and Causal Scope

  • Production trials assess the deployed stack rather than isolated model weights, while the observational search study lacks a simultaneous control arm.The randomized deployments use persistent user-level assignment; commercial baseline volumes and absolute denominators are withheld.Source: §8.1 · p. 22, §8.3 · p. 23, §8.3.1 · p. 24

8.2 RQ3: Procurement-Query Structuring

  • Under matched downstream SFT in No-Think mode, CPT+SFT improves exact match by +2.97 pp, semantic match by +5.18 pp, and multiple-choice accuracy by +3.05 pp.The exact-match difference has a paired-bootstrap 95% confidence interval of [2.11, 3.86] pp; these metrics measure query structuring, not product eligibility.Source: Table 13 · p. 23, §8.2 · p. 23

8.3.1 Search and Attribute Normalization

  • The observational search deployment reports +6.5% demand-satisfaction rate and +7.11% transaction-conversion rate, but its uncontrolled pre–post changes are associations, not unconfounded causal effects.Product-search PV_L2O rises from 0.148% to 0.156% (+0.008 pp; +5.41% relative).Source: §8.3.1 · pp. 23–24
  • The randomized attribute-normalization treatment increases product CTR per visitor by +2.4%, total A2A inquiries by +4.32%, and satisfied A2A inquiries by +8.3%.The trial estimates the bundled impact of numeric scaling, synonym alignment, and unit conversion rather than isolated sub-operations.Source: §8.3.1 · p. 24

8.3.2 Conversational Inquiry Intent Understanding

  • The inquiry pipeline extracts intent and attributes, checks completeness, maps missing decision-critical fields, and synthesizes structured RFQ and guidance outputs.In the fastener case, IndustryLLM preserves a conflicting length as unknown and asks for clarification instead of dispatching a premature catalog match.Source: §8.3.2 · pp. 24–25, Table 15 · p. 25
  • IndustryLLM trails slightly on JSON formatting robustness (−0.06) and conversational tone (−0.58), despite leading on technical-fidelity-related dimensions.The generator and evaluator share model lineages, so the authors treat these results as descriptive comparisons rather than absolute human-preference benchmarks.Source: §8.3.2 · pp. 25–26
  • In the blinded evaluation of 1,000 procurement dialogs, IndustryLLM scores 60.1 against 53.7 for Qwen3.5-Plus, leading on security compliance and several intent-understanding dimensions.Figure 6 reports leads of +3.22 for information security compliance, +1.39 for parameter coverage and faithfulness, and +1.00 for demand-understanding completeness.Source: §8.3.2 · p. 25, Figure 6 · p. 26

TAKEAWAY PRODUCTION-SCALE SYSTEM VALIDATION

  • Randomized online trials report a 4× latency reduction (6–7 s to 1.5 s), +4.25% GMV, +6.5% inquiries, and +22.7 pp in attribute standardization for the deployed inquiry stack.The outcomes are statistically significant (p ≤0.01), but the stack bundles model weights, prompts, inference optimizations, and routing.Source: Table 16 · p. 26, TAKEAWAY PRODUCTION-SCALE SYSTEM… · p. 27

9 Discussion

The discussion interprets adaptation results while drawing boundaries between parametric language understanding, transaction evidence, and system-level claims. It also identifies safety, governance, and reproducibility constraints.

9.1 Methodological Insights from Data Reconstruction and SFT Targets

  • Proxy experiments suggest reconstruction methods should be evaluated separately: multi-register rewriting has the strongest point estimates, while minimal editing shows a +13.9 pp targeted-probe advantage and weakness-targeted QA gains are marginal.These 2B–4B, single-seed experiments are design heuristics, not an additive causal decomposition of the final 35B model.Source: §9.1 · p. 27
  • Teacher targets improve complex deductive benchmarks but regress on concise completion and strict constraint following; CPT+SFT also exceeds Base+SFT across all three procurement-query endpoints.For exact match, the paired-bootstrap 95% confidence interval is [2.11, 3.86] pp; the authors conclude domain pre-training remains important for specialized industrial semantics.Source: §9.1 · p. 27

9.2 Where Model Knowledge Ends and Transaction Evidence Begins

  • CPT internalizes relatively invariant terminology and engineering regularities, whereas current inventory, prices, qualifications, revisions, and certificates require fresh, auditable transaction evidence.The stated boundary is that models structure technical intent while current transaction evidence determines physical eligibility.Source: §9.2 · pp. 27–28

9.3 Parameter Capacity, MoE Efficiency, and System Cost

  • IndustryLLM has 35B total parameters with about 3B activated per token and surpasses larger reference architectures on the reported IndustryBench point estimates.Active parameter count is descriptive capacity, not measured system efficiency; serving latency gains reflect the bundled stack and deployment optimizations.Source: §9.3 · p. 28

9.4 Safety and Physical Engineering Constraints

  • Safety-violation adjustment lowers the post-SFT Think score from 2.120 raw to 2.030 (∆= −0.090), motivating safeguards that keep missing or conflicting evidence unknown.High-consequence specifications additionally call for external standards retrieval, deterministic checks, and expert-in-the-loop abstention; these are downstream system constraints, not checkpoint guarantees.Source: §9.4 · p. 28

9.5 Data Governance, IP Boundaries, and Reproducibility

  • Open checkpoint availability does not make upstream corpus licensing uniform or permit exact bit-level retraining of the industrial corpus.The authors distinguish checkpoint licensing from upstream data terms and describe operational documentation as an auditable blueprint, not full lineage reconstruction.Source: §9.5 · p. 29

9.6 Open Model Release

  • The post-SFT weights and inference configurations are openly released with version histories, model cards, chat templates, and serving guidelines.Full training-lineage reconstruction specifications are documented in Appendix E.Source: §9.6 · p. 29

10 Limitations

The limitations section bounds the evidence across proxy-scale experiments, product-eligibility claims, production attribution, and market and architecture scope.

General-Capability Retention and Cross-Scale Inductive Leaps

  • Data-reconstruction ablations ran at 2B–4B proxy scales under single seeds, and unmatched training compute plus an incomplete checkpoint progression limit causal attribution to the final 35B model.Teacher targets also trade improved multi-step reasoning for moderate regressions on MBPP-Plus and IF-Eval.Source: §10 · p. 29

Parametric Demand Structuring versus Non-Parametric Product Eligibility

  • Offline evaluations establish query normalization, scenario inference, and ambiguity flagging, but parametric language modeling alone cannot certify product compliance or eligibility.The proposed evidence gate is not an end-to-end multi-source verification evaluation of the raw checkpoint.Source: §10 · pp. 29–30

Production Attribution and Observational Confounding

  • Online outcomes measure bundled interventions rather than isolated checkpoint effects; the uncontrolled search study, withheld denominators, and absent immutable deployment-to-checkpoint link further constrain attribution.The bundle includes model weights, prompt schemas, inference optimizations, and backend routing.Source: §10 · p. 30

Market Scope, Multimodal Constraints, and Data Governance

  • Evidence is limited to Chinese B2B industrial commerce, with cross-lingual transfer and foreign regulatory and catalog compatibility unverified; vision capabilities were not tested because the encoder remained frozen.The authors also note potential Qwen-family evaluation and data-generation blind spots and unresolved upstream licensing and lineage issues.Source: §10 · p. 30

11 Conclusion

IndustryLLM separates language-model interpretation of buyer intent from evidence-gated product eligibility, which depends on current, verifiable product records. The paper reports query-structuring gains and online deployment improvements while releasing the model weights and inference configurations.

11 Conclusion

  • IndustryLLM assigns domain adaptation to parsing buyer intent and an evidence-gated interface to determining physical eligibility under three-valued logic.Missing parameters trigger clarification, unverified supplier claims remain unknown, and ranking uses only hard-constraint-verified supply.Source: §11 · pp. 30–31
  • Offline query structuring improved by +2.97 pp exact match (95% CI [2.11, 3.86]), while randomized online deployment reported economic gains and a 4× latency reduction.The online result is reported alongside the offline result; the conclusion does not provide individual online metric values here.Source: §11 · p. 30
  • The released IndustryLLM model has 35B total parameters with approximately 3B activated per token.The paper releases post-SFT weights and inference configurations.Source: §11 · pp. 30–31

† Author Contributions

The listed project leader is Liang Ding; the core contributors are Zhiang Xu, Yuyang Sheng, Bin Chen, Songlin Bai, and Liang Ding.

† Author Contributions

A Minimal-Edit Transformation: Proxy-Scale Ablation

The proxy ablations examine minimal factual edits and weakness-targeted QA synthesis at 2B scale. Both report favorable comparisons, with explicit limits on causal attribution, uncertainty, and transfer to full-scale training.

A Minimal-Edit Transformation: Proxy-Scale Ablation

  • The paired-document design shares membership, order, upsampling, hyperparameters, and seed, but does not verify equality of all non-target tokens.The pipeline permits short-document regeneration; the filter favors omission over unsupported edits, and independent authority review is needed to rule out false edits.Source: A Minimal-Edit Transformation:… · p. 31
  • The minimal-edit probe measures forced-choice preference for exposed edited facts, not open-ended recall or generalization.Its 2B base model performed poorly in open-ended specialist generation, motivating the two-alternative forced-choice evaluation.Source: A Minimal-Edit Transformation:… · pp. 31–32
  • Group B was 13.9 points higher after annealing and 12.8 points higher at the end of main training; validation loss was 1.5329 for B and 1.5341 for A.Category gaps ranged from 9.4 to 18.4 points; option order was unrecorded, and paired intervals and significance tests were unavailable.Source: A Minimal-Edit Transformation:… · p. 32
  • The minimal-edit result cannot isolate retained edits from possible option-order bias or establish the absence of a general-capability trade-off.The probe also excludes missed errors and out-of-document generalization, and its arm ordering should not be projected onto the 35B run.Source: A Minimal-Edit Transformation:… · p. 32

B Proxy-Scale Ablation of Model-Weakness-Targeted QA Synthesis

  • The key QA contrast is B versus C: each adds 25,000 targeted or uniformly sampled pairs, while A receives body text to match total tokens.All groups share the starting checkpoint and training setup; comparisons against A therefore also reflect QA-formatted text.Source: B Proxy-Scale Ablation of… · p. 33
  • The QA questions are generated from training-corpus documents and linked to source documents and answer points, so the experiment tests access to existing material, not acquisition of new documents.The production pipeline generates and deduplicates more than one million questions, unlike the approximately 113,000 cleaned questions used for the proxy runs.Source: B Proxy-Scale Ablation of… · p. 33
  • Both QA evaluation channels ordered the groups B > C > A; B exceeded A by +0.0885 in reference likelihood and won 54.6% of judged comparisons.B exceeded C by +0.0113 in reference likelihood and won 51.6% of judged comparisons, a modest margin without paired intervals or repeated judging.Source: B Proxy-Scale Ablation of… · p. 34, Table B2 · p. 34
  • The QA effect is small, lacks paired uncertainty, and should not be transferred to full-scale training.The experiment injects QA only during annealing at 2B scale and uses one teacher-generated reference answer per question.Source: B Proxy-Scale Ablation of… · p. 34

C Protocol for Paired Procurement Cases

The appendices specify how paired procurement cases should be compared and distinguish illustrative examples from statistical evidence. They also document corpus-lineage limits and analyze two qualitative data transformations.

C.1 Comparator and selection protocol

  • The primary model comparison is Base+SFT versus CPT+SFT with downstream SFT held fixed.Comparisons should share mode, prompt, context, product records, evidence, decoding, and tool access.Source: §C.1 · p. 35

C.2 Paired case card

  • Paired procurement cases are explanatory, not statistical evidence, and require a frozen, outcome-independent case selection with wins, ties, and failures retained.Cases without candidate-product records and supporting evidence can demonstrate query structuring only, not product eligibility.Source: §C.2 · p. 35, §C.1 · p. 35, Table C1 · p. 35
  • A paired case card records common inputs, verbatim outputs, provenance, an expert reference state, and blinded adjudication.Full-set evaluation should include structuring and constraint metrics, with eligibility metrics only when catalog evidence exists.Source: §C.2 · p. 35, Table C2 · p. 36

D Data Composition and Lineage

D.1 Qualitative transformation records and provenance

  • The displayed transformation examples illustrate procedures but do not establish retained training-set membership or post-training improvement.The appendix distinguishes the Chinese source wording from accompanying English analyses, which are not model outputs.Source: §D.1 · p. 37

D.1.1 GS-01: From web QA to an encyclopedic article

  • The GS-01 rewrite reorganizes fragmented broadband replies and adds knowledge, including a distinction between propagation speed and information-carrying capacity.Its error analysis flags imprecise angle and concept relations, and warns that added thresholds and dates need applicable, dated sources.Source: §D.1.1 · p. 39

D.1.2 ME-01: Corrosion mechanism and applicability

  • The ME-01 revision replaces a potential-sign account of corrosion with passive-film protection tied to service conditions.It distinguishes average mass-loss assessment from localized corrosion measurements and calls for specified test conditions.Source: §D.1.2 · pp. 39–40

D.1.3 QA-01: Correct verdict, incorrect rationale

This section examines QA-01, showing how a correct pass/fail verdict can mask faulty reasoning and outlining reference prompting, scoring, weakness selection, and source-grounded answer synthesis. It also documents gaps in case provenance, SFT reporting, and reproducibility materials.

D.1.3 QA-01: Correct verdict, incorrect rationale

  • Both responses reject the indicator, but only the synthesized answer applies the 10% limit: |−16%| = 16% > 10%; the model answer substitutes ±5% and invents damage and performance-loss claims.This establishes failure of the named indicator, not the fiber’s overall qualification.Source: §D.1.3 · p. 40

D.1.4 Proposed reference prompts

  • Genre-and-style rewriting selects zero to four suitable genres, assigns one style to each, then rewrites separately while preserving supported technical details and avoiding invented specifications or broadened claims.The templates describe reference implementations, not verbatim prompts from reported runs.Source: §D.1.4 · p. 41, User message · pp. 41–43

Writing style guidance: Technical practitioner

  • Confidence-routed editing first flags serious factual or logical errors, then verifies each issue and makes the smallest supported correction or abstains; confidence estimates error likelihood, not correction reliability.Issues at or above a configured threshold use the direct route; others use web verification, and either route can decline to edit.Source: Writing style guidance: Technical… · p. 44, User message · p. 45

User message

  • Edits use exact-text SEARCH/REPLACE blocks that must match uniquely; the runner rejects ambiguous or conflicting patches, preserves non-target text, and checks whole-document consistency after assembly.These are application checks, not additional prompt stages.Source: User message · p. 46
  • Weakness-targeted QA separates source-based question generation, checkpoint-response scoring, and source-anchored answer synthesis; the runner, not the judge, controls eight rollouts and trimmed-mean calculation.Questions should have identifiable answer points, and hypothetical scenario values must be distinguished from source facts.Source: User message · pp. 46–47, System message · p. 47
  • The QA workflow scores eight checkpoint responses separately, trims one score from each end, and selects questions below a preset threshold for source-grounded answer synthesis.Missing or invalid scores block aggregation, and the synthesized answer receives the question and evidence—not the erroneous responses or judge scores.Source: User message · pp. 47–50

System message

  • The judge evaluates correctness, completeness, and evidential support rather than length or reference wording, and a correct conclusion does not excuse an incorrect governing rule.Material errors in thresholds, measurement bases, calculations, or consequential causal diagnoses cap the score at 2.Source: System message · p. 48, User message · p. 48

User message

  • QA-01 caps a correct verdict at 2 when its rationale uses a wrong threshold or unsupported causal diagnosis; the synthesized answer applies |−16%| > 10% only to the named indicator.The illustrative case says the supplied limit is 10%, not 5%, and irreversible damage is unsupported; these checks were not evaluated through model execution.Source: User message · pp. 48, 50

D.1.5 Prompt and record provenance

  • The QA-01 case lacks its rollout scores and selection record, while the supplied examples are not linked to retained training records or established historical prompt and run metadata.Reproducing selection requires rollout responses, raw scores, excluded-response identifiers, the trimmed mean, and the threshold; the documentation table lists requirements, not recovered records.Source: User message · p. 50, §D.1.5 · p. 51

D.2 SFT data card

  • The SFT target comparison matches prompts and task mixture but not target-token exposure, and missing counts, target-length distributions, and generation and filtering details limit reproduction without changing the example-matched RQ1 interpretation.The available documentation also lacks selection metadata, drop-or-resample behavior, and a complete teacher-and-filter manifest.Source: §D.2 · p. 51

E.1 Training and evaluation records

  • The report lacks records needed to reproduce training and evaluation, including checkpoint and code revisions, phase-level accounting, configuration details, and benchmark prompts and outputs.The report documents available configurations and evidence but does not provide a complete reproducible training recipe.Source: §E.1 · p. 52

E.2 Public artifacts and remaining records

  • IndustryLLM weights and configuration files are publicly available, but the release does not include the full corpus or records needed to reproduce training, evaluation, or deployment.Table E1 distinguishes the released model artifact from research materials documented only in aggregate or absent from the public package.Source: §E.2 · p. 52, Table E1 · p. 52
Loading 2609.31871v1…