PaperlayerView exact revision on arXiv

Source-linked AI summary

How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs

Yapei Chang, Kyle Lo, Mohit Iyyer, Luca Soldaini

arXiv:2602.08808v1cs.LG

TL;DR

Measuring and improving procedural validity at scale on real-world tasks remains challenging. How2Everything mines web procedures, builds evaluation and training resources, and scores critical failures; RL with its reward improves performance by more than 10 points across three models without systematic regressions on standard benchmarks.

  • Problem

    Measuring and improving procedural validity across diverse procedures remains challenging, despite its potential relevance to downstream tasks involving reasoning and planning.

    Source: §1 · p. 2

  • Method

    How2Everything mines and refines web procedures into How2Bench and How2Train, and uses How2Score with an open 8B judge to detect critical failures.

    Source: §1 · p. 2

  • Results

    Across three models, RL improves How2Everything performance by >10 points with no regressions—and sometimes improvements—on a standard post-training evaluation suite.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The framework could benefit user-facing assistants in troubleshooting, education, and everyday planning.

    Source: Impact Statement · p. 13

    How2Score is an evaluation proxy, not a guarantee of real-world correctness or a substitute for expert review or execution-based verification in safety-critical settings.

    Source: Impact Statement · p. 13

Abstract

from arXiv · show

Generating step-by-step "how-to" procedures is a key LLM capability: how-to advice is commonly requested in chatbots, and step-by-step planning is critical for reasoning over complex tasks. Yet, measuring and improving procedural validity at scale on real-world tasks remains challenging and understudied. To address this, we introduce How2Everything, a scalable framework to evaluate and improve goal-conditioned procedure generation. Our framework includes How2Mine, which mines 351K procedures from 980K web pages across 14 topics and readily scales to larger corpora. From this pool we build How2Bench, a 7K-example evaluation set balanced across topics. To reliably score model outputs, we develop How2Score, an evaluation protocol that uses an LLM judge to detect whether a generation contains any critical failure that would prevent achieving the goal. For low-cost, reproducible evaluation, we distill a frontier model into an open 8B model, achieving 80.5% agreement with human annotators. How2Bench reveals clear scaling trends across model sizes and training stages, providing signal early in pretraining. Finally, RL using How2Score as a reward improves performance on How2Bench by >10 points across three models without systematic regressions on standard benchmarks, with gains robust to superficial source-document memorization or format compliance. Taken together, How2Everything shows how pretraining web data can support a closed loop of capability evaluation and improvement at scale.

1 Introduction

How-to guidance is a common LLM use, and procedural content supports reasoning and planning. How2Everything mines web procedures to evaluate and improve step-by-step generation at scale.

Motivation and challenge

  • Open-world procedural validity is difficult to assess because real-world goals span diverse domains, success depends on entire action sequences, and task-specific executors or automatic oracles are unavailable.Prior approaches often focus on narrow domains, specific sources, subtasks, or metrics that trade accuracy for cost.Source: §1 · p. 2

Contribution 1

  • How2Mine processes 980K web documents into 351K procedures, sampling across 14 topics and filtering and standardizing the results.The framework uses mined procedures to create How2Bench and How2Train, turning web documents into reference anchors when execution-based verification is infeasible.Source: §1 · pp. 2–3

Contribution 2

  • How2Score flags omissions, extraneous actions, or deviations that prevent goal completion; the distilled open 8B How2Judge agrees with annotators 80.5%.Frontier LLM judges achieve 76.5–83.0% agreement with human majority across five models.Source: §1 · p. 2

Contribution 3

  • Across 1B–32B training runs, How2Bench shows scaling with model size and training stage while ranking models from ≈10^21 FLOPs to GPT 5.The benchmark reserves 7K of the mined procedures for evaluation.Source: §1 · pp. 2–3

Contribution 4

  • Across three models, RL on How2Train with How2Score rewards improves How2Everything performance by >10 points without regressions on a standard post-training suite.In some cases, performance on that suite also improves.Source: §1 · p. 3

Contribution 5

  • Improvements are not driven by format compliance or memorization: aggressive data contamination adds only +3 points for a 7B model, and stronger base models yield better post-RL checkpoints.The analysis finds limited memorization risk despite evaluation examples coming from web documents used for pretraining.Source: §1 · p. 3

2 Problem Setting and Related Work

The paper studies descriptive, goal-conditioned real-world procedures: textual step sequences whose accuracy matters to users and downstream systems. Existing datasets and evaluation methods leave broad coverage and scalable measurement of end-to-end validity unresolved.

Problem Setting

  • A procedure is a goal-conditioned action sequence; this work focuses on descriptive representations of real-world procedures, whose accurate steps support users and downstream automation.Unlike executable procedures, descriptive procedures are judged by their textual representation rather than execution in an environment.Source: §2 · p. 3

Datasets for goal-conditioned procedures

Evaluation challenges for end-to-end procedural validity

3 How2Mine: Extracting Realistic Step-by-Step Procedures from the Web

How2Mine mines structured, goal-conditioned procedures from topic-stratified web tutorials through extraction, filtering, and validation. Its scalable pipeline produces procedures that can support evaluation and training datasets.

3.1 Sampling Web Pages for Procedure Mining

  • The mining pool uses tutorial-labeled documents from the DCLM web corpus, stratified across 14 topics, and spans 189K unique domains.The pipeline focuses on tutorials but can be extended to academic writing and knowledge articles.Source: §3.1 · p. 4

3 How2Mine: Extracting Realistic Step-by-Step Procedures from the Web

  • From 980K web documents, How2Mine extracts 351,162 structured procedure instances and scales to larger corpora.Each instance contains a goal, resources list, and reference steps.Source: §3 · p. 4, Figure 2 · p. 5

3.2 From Web Documents to Structured Procedures

  • The pipeline extracts goals and ordered steps, then filters out procedures with fewer than 5 or more than 15 steps, high within-step n-gram overlap, or unsuitable content.The LLM filter excludes procedures dependent on named entities, mathematical calculations, UI interaction, open-ended creative generation, non-sequential tasks, or unreasonable content.Source: §3.2 · p. 5
  • Post-processing makes each goal more specific and deterministic, records resources referenced in the procedure, and a final LLM sanity check removes remaining invalid examples.These edits narrow the space of valid solutions.Source: §3.2 · p. 5
  • Each output record contains a topic, goal, resources, and reference steps; 500 instances per topic form How2Bench, while the rest form How2Train.How2Bench contains 7,000 instances total.Source: §3.2 · p. 5

4 How2Score: Measuring Procedural Validity by Detecting Critical Failures

How2Score evaluates procedures by detecting critical failures that would prevent goal completion, using binary judgments to make open-world evaluation practical. How2Judge, an open 8B judge distilled from a frontier model, supports low-cost, reproducible evaluation.

4 How2Score: Measuring Procedural Validity by Detecting Critical Failures

  • How2Score addresses the scalability–reliability trade-off by asking an LLM judge whether a generated procedure has any critical failure preventing goal completion.Reference-overlap metrics are cheap but misaligned with procedural validity, while human evaluation is reliable but does not scale.Source: §4 · pp. 5–6

4.1 Defining Critical Failures in an Open-World Setting

  • A critical failure is an omission, extraneous action, contradiction, severe vagueness, or other deviation severe enough to prevent goal completion or make instructions unusable.The reference anchors evaluation, but valid alternatives and superficial differences are not failures; the definition is a practical proxy in a subjective, non-executable setting.Source: §4.1 · p. 6

4.2 Evaluation Protocol

  • The judge identifies and describes failures with references to relevant steps, then assigns has_failure if any critical failure is found and no_failure otherwise; the success rate is the fraction labeled no_failure.The protocol asks the judge to enumerate all identified failures, while the binary aggregation avoids requiring agreement on a first-failure location.Source: §4.2 · p. 6

4.3 Validation via Human Annotations

  • 83.0% overall agreement with human-majority labels was achieved by GPT-5; binary aggregation yielded Krippendorff’s α = 0.593, versus α = 0.307 for first-failure localization.GPT-5 agreement was 83.7% on has_failure cases and 82.4% on no_failure cases.Source: Figure 3 · p. 7, §4.3 · p. 7

4.4 Distilling a Cost-Effective Judge

  • 80.5% agreement with human-majority labels was achieved by How2Judge, which also reached 90.5% agreement with GPT-5.Its human agreement was 79.6% on has_failure cases and 81.4% on no_failure cases.Source: §4.4 · pp. 7–8

5 How2Bench: Evaluating Performance on Step-by-Step Procedure Generation

How2Bench samples 7,000 procedures—500 per topic—and uses How2Score with How2Judge to evaluate models’ step-by-step procedure generation.

5.2 Evaluation Results and Analysis

  • How2Bench results scale with model size and training stage, and show a noticeable performance gap between open and closed models.The results are reported for selected models and include average generated tokens; the average reference length is 97.44 tokens.Source: Figure 4 · p. 8, §5.2 · p. 8

5.1 Inference Setup

  • At inference, models receive the goal, resource list, and required step count, then must produce exactly that many concise, single-action sentence steps.Conditioning on resources and step count is an evaluation control for comparability, although it may not reflect real-world usage.Source: §5.1 · p. 8

5.2 Evaluation Results and Analysis

  • Although absolute How2Score values vary across GPT, Gemini, Claude, and How2Judge evaluators, the relative ranking of closed models remains unchanged.This cross-judge check tests for self-preference bias.Source: §5.2 · pp. 8–9, Figure 5 · p. 9
  • Across five Olmo runs, performance scales smoothly with model size and training stage, with non-trivial 1B-model performance emerging about 5% into pretraining.That emergence is reported at around 1021 training FLOPs, with performance continuing to improve afterward.Source: §5.2 · p. 9
  • Formatting errors drop early and plateau while How2Score continues improving, indicating later gains are unlikely to come primarily from surface-format corrections.The tracked proxies are step-count mismatch, duplicate-step frequency, and n-gram repetition; the pattern holds over five Olmo runs.Source: §5.2 · p. 9
  • Across five Olmo runs, Spearman correlation between checkpoint rankings by How2Everything and reference-step perplexity ranges from 0.233 to 0.967.The reported range indicates How2Score is not simply measuring conditional likelihood of the reference procedure.Source: §5.2 · p. 9
  • Reference step count is the dominant instance-level predictor across models: procedures requiring more steps are consistently harder, making step count a monotonic difficulty knob.The analysis controls for topic.Source: §5.2 · p. 10
  • Qualitative analysis finds common failures in critical omissions, missing or wrong parameters, unsafe or invalid actions, and contradictions across steps.Refusals occur occasionally, primarily in frontier models.Source: §5.2 · p. 10

6 Improving Step-by-Step Procedure Generation with RL

How2Score-based RL improves goal-conditioned procedure generation, with gains persisting under external judges and no systematic degradation on standard out-of-domain benchmarks. Length calibration controls verbosity gaming, while an additional SFT stage offers limited gains on instruction-tuned checkpoints.

6.2 Results and Analysis

  • RL improves How2Bench performance while out-of-domain changes are mixed but generally modest, with no evidence of systematic degradation.The evaluation spans 12 standard out-of-domain benchmarks covering knowledge, chat, math, code, and logical reasoning.Source: Table 2 · p. 10, §6.2 · p. 11

6.1 Training Setup

  • Training uses 100K How2Train examples balanced across 14 topics and low in semantic similarity to How2Bench, with GRPO over 1,000 steps and three rewards.The rewards are How2Score from How2Judge, a step-format verifier, and a reference-calibrated length reward; models include Qwen 3 4B/8B Instruct and OLMo 3 7B Think (Qwen Team, 2025a; Olmo et al., 2025; Shao et al., 2024).Source: §6.1 · p. 11

6.2 Results and Analysis

  • Reference-calibrated length rewards keep generation length near the reference (∣gen∣/∣ref∣≈1.0), preventing length gaming seen without the reward (1.34×–1.53×).Without length control, models inflate outputs and achieve apparent How2Bench gains consistent with length gaming.Source: §6.2 · p. 11
  • Gains from RL persist when the same generations are evaluated by external judges GPT 5 and Gemini 2.5 Pro.How2Judge supplied How2Score during training; external re-evaluation tests whether gains are specific to that judge.Source: §6.2 · p. 11
  • SFT yields small gains on base checkpoints but does not improve instruction-tuned checkpoints, possibly because reference-text likelihood mismatches How2Score’s critical-failure objective.The proposed objective-mismatch explanation is tentative (Stiennon et al., 2022; Xie et al., 2025).Source: §6.2 · p. 11

7 Robustness to Format and Memorization Confounds

Targeted analyses find that neither implicit format compliance nor source-document memorization explains How2Score’s observed gains. RL gains depend on pretraining maturity and topic coverage, while repeated document exposure produces only modest, non-monotonic How2Score improvements.

7 Robustness to Format and Memorization Confounds

  • Neither implicit task-format compliance nor source-document memorization accounts for the observed How2Score gains.The section tests both explanations directly and finds evidence against each.Source: §7 · p. 11

7.1 Confound 1: Implicit Task Format Compliance

  • The format-compliance tests vary pretraining maturity while fixing post-training, and vary topic coverage while fixing the base model.The diagnostics ask whether weaker checkpoints recover similar gains and whether narrow-topic RL transfers broadly.Source: §7.1 · p. 11
  • 20.33 at 100% versus 3.56 at 10% pretraining FLOPs, while SFT gains range from 3.39 to 5.36 across checkpoints.RL gains grow substantially at later Olmo 3 7B checkpoints and account for most improvement there; the SFT range is similar across checkpoints.Source: §7.1 · pp. 11–12, Figure 6 · p. 12
  • +10.10 overall from all-topic RL, +9.41 from science-only RL, and +5.55 from dining-only RL on Qwen 3 8B.Science-only training generalizes strongly across topics, while dining-only training transfers more weakly; broad topic coverage yields the best overall performance.Source: §7.1 · p. 12

7.2 Confound 2: Memorization of Source Documents

  • The memorization test midtrained final Olmo 3 7B and 32B checkpoints for 10B tokens at five document-exposure frequencies, from 0 to 10 occurrences.The evaluation set contained 13,500 procedure examples, evenly balanced across the five exposure groups.Source: §7.2 · p. 12
  • Perplexity fell from 10.4 to 1.4 for Olmo 3 7B and from 8.0 to 1.2 for 32B, but How2Score gains were modest and non-monotonic.How2Score peaked at +3.3 for 7B and +6.1 for 32B, so repeatedly seeing source documents does not explain the gains.Source: §7.2 · pp. 12–13, Table 3 · p. 13

Impact Statement

How2Everything aims to improve large-scale evaluation and end-to-end procedural validity, with potential benefits for user-facing assistants. The paper also notes risks from online biases and safety-sensitive instructions and describes safeguards and limits.

Potential positive impact

  • The framework supports scalable evaluation of procedural instruction quality and offers a reward signal to improve end-to-end validity, potentially benefiting assistants in troubleshooting, education, and everyday planning.The authors describe these as practical and broad positive impacts.Source: Impact Statement · p. 13

Risks and negative societal impact

  • Web-derived procedures may reproduce online societal biases, while safety-sensitive instructions could enable harmful behavior if misused.The passage gives health, legal, and chemical instructions as examples of safety-sensitive areas.Source: Impact Statement · p. 13

Mitigations

  • How2Score is only an evaluation proxy, not a guarantee of correctness or a substitute for expert review or execution-based verification in safety-critical settings.Before release, the authors plan further safety and privacy filtering and documentation of intended use and known limitations; they will release the benchmark split, prompts, and distilled judge weights for reproducible evaluation without proprietary judges.Source: Impact Statement · p. 13

A Motivational analysis of query type distribution … B.2 Extracting procedures from documents of other formats

The section establishes the real-world relevance of how-to requests and summarizes data-pipeline details, including why tutorial pages are the primary source format despite extractable procedures appearing in other formats.

A Motivational analysis of query type distribution

  • How-to advice accounts for approximately 8.5% of ChatGPT conversations, ranking fourth among 23 categories, and corresponds to tens to hundreds of millions of daily interactions.It follows Specific Info (18.3%), Edit or Critique Provided Text (10.6%), and Tutoring or Teaching (10.2%), according to Chatterji et al. (2025).Source: A Motivational analysis of query type… · p. 18
  • WildChat-4.8M (Zhao et al., 2024) and LMSYS-Chat-1M show query-type skews relative to ChatGPT, so they contextualize alternative user behaviors rather than anchor usage estimates.The same query-type classifier is applied to both public corpora; the section attributes their distribution differences to collection from unrestricted public LLM endpoints rather than a deployed product.Source: A Motivational analysis of query type… · p. 18, Table 4 · p. 18
  • Table 4 summarizes query-type distributions across ChatGPT, LMSYS, and WildChat; the section uses ChatGPT as its real-world usage anchor.The open datasets are used primarily to contextualize results under alternative user-behavior regimes.Source: A Motivational analysis of query type… · p. 18, Table 4 · p. 18

B Details on data pipeline

  • The data-pipeline section points to subsection H.1 for the prompts used in the pipeline.It introduces further details related to the data.Source: B Details on data pipeline · p. 19

B.1 Top frequent URL domains

  • Table 5 summarizes the ten most frequent URL domains within each topic in the final pool of 351K procedure instances.Domain counts represent instances in each topic whose source URL belongs to that domain.Source: Table 5 · p. 19, §B.1 · p. 19

B.2 Extracting procedures from documents of other formats

  • “Tutorial & How-to Guide” yields the highest procedure-extraction, heuristic-filtering, and LLM-filtering rates, although valid, extractable procedures also occur in other formats.These stage yields are measured against each format’s original input documents, before postprocessing and final validation; the authors focus on tutorials for efficient large-scale mining.Source: Table 6 · p. 22, §B.2 · p. 22

B.3 Implementation details on the heuristics filter … D.1 Inference setup details

The appendices detail How2Mine’s filtering criteria and provide examples across the topics, then describe How2Score’s annotation and distillation procedures and the controls used for model inference. Inference is standardized to limit verbosity bias and improve comparability across outputs.

B.3 Implementation details on the heuristics filter

  • The heuristic filter first requires 5–15 steps by default, then rejects procedures exceeding repetition-rate thresholds for 2-, 3-, or 4-grams.The step-count limits are configurable; the repetition thresholds are ≥0.40 for bigrams, ≥0.35 for trigrams, and ≥0.30 for fourgrams.Source: §B.3 · p. 22

B.4 Rationale behind LLM filter criteria

  • How2Mine’s LLM filter removes entity-dependent, math, UI, creative, non-sequential, and unreasonable procedures because they are unsuitable for reliable critical-failure evaluation.Unreasonable reference procedures are excluded because they could add noise to both evaluation and training.Source: §B.4 · pp. 22–23

B.5 Example procedures for all 14 topics

  • Table 7 provides a goal, resource list, and reference steps for one example from each of How2Everything’s 14 topics.The examples span topic areas including Electronics & Hardware, Food & Dining, Fashion & Beauty, Home & Hobbies, Science, Math & Technology, Sports & Fitness, and Travel & Tourism.Source: Table 7 · p. 23, §B.5 · p. 23

Electronics & Hardware / Food & Dining / Fashion & Beauty / Home & Hobbies

Home & Hobbies / Science, Math & Technology / Sports & Fitness / Travel & Tourism

C.1 Human annotation setup

  • Human annotator training defines critical failures and walks through five failure examples and six acceptable variances, with attention checks and a 90-second minimum before submission.Annotators acknowledge the goal and click through generated steps to confirm they read them.Source: §C.1 · p. 26, Figure 7 · p. 26

C.2 Distillation

  • To distill GPT 5 into a Qwen 3 8B judge, the authors collect 72,920 annotations across generator models and retain examples with consistent judgments across two GPT 5 runs.Qwen 3 8B is finetuned for three epochs with learning rate 5e−6 and batch size 64.Source: §C.2 · pp. 26, 29

D.1 Inference setup details

  • To control length bias, inference uses standardized 3-shot prompts and concise single-action steps; non-thinking endpoints use greedy decoding, while reasoning-enabled APIs use temperature 0.6.Generators receive the goal, resource list, and reference step count and are asked to produce exactly that number of steps; this conditioning is an evaluation control, not typical real-world usage. Explicit length control brings generated lengths closer to the reference distribution.Source: §D.1 · pp. 29–30, Figure 12 · p. 29

D.2 Full intermediate checkpoint evaluation results on How2Bench … D.4 Conditional perplexity vs. How2Score

The appendix reports selected OLMo midtrain and posttrain How2Bench scores, defines formatting proxies for checkpoint outputs, and compares checkpoint ordering under conditional perplexity and How2Score.

D.2 Full intermediate checkpoint evaluation results on How2Bench

  • 43.16 How2Score for OLMo-3-1125 32B Posttrain compares with 38.31 for Midtrain.Table 8 presents full checkpoint results, including pretraining trajectories.Source: Table 8 · p. 30, §D.2 · p. 31
  • 30.23 How2Score for OLMo-3-1025 7B Posttrain compares with 24.91 for Midtrain.Table 8 presents full checkpoint results, including pretraining trajectories.Source: Table 8 · p. 30, §D.2 · p. 31

D.3 Formatting proxy metrics over intermediate checkpoints

  • Step-count mismatch measures the fraction of examples whose predicted and reference procedures have different numbers of steps.Duplicate steps measure the fraction of examples containing an exact repeated step string.Source: §D.3 · p. 31
  • Dup n-gram rate is averaged across examples, with Table 9 reporting the unweighted mean across n = 1, 2, 3, and 4.The metric is computed from whitespace-tokenized concatenated predicted steps.Source: §D.3 · p. 31, Table 9 · p. 31

D.4 Conditional perplexity vs. How2Score

  • Conditional perplexity is teacher-forced on reference steps, conditioned on each example’s goal and resources prompt.The metric is compared with checkpoint ordering under How2Score.Source: §D.4 · pp. 32–33
  • Table 16 reports per-checkpoint How2Score scores, conditional reference-step perplexities, and induced within-run ranks.For each OLMo trajectory, Spearman rank correlation is calculated across 9 checkpoints: 8 stage-1 pretraining checkpoints and 1 stage-2 midtrained checkpoint.Source: §D.4 · p. 33

D.5 Instance-level correlates of How2Score no_failure … E.2 Details on training setup

Topic-controlled logistic regressions show that How2Score no_failure varies with reference step count, residual verbosity, and topic. The appendix also describes embedding-based train–evaluation deduplication and SFT/RL training procedures.

D.5 Instance-level correlates of How2Score no_failure

  • The analysis fits topic-fixed-effects logistic regressions across seven models, using reference steps, resource count, and generation-to-reference token ratio as instance-level covariates.The seven models comprise two open 7–8B models, two open 32B models, and three closed frontier models.Source: §D.5 · p. 33
  • Across models, each additional required step is associated with lower no_failure odds (ORsteps ∈[0.74, 0.85]), even after controlling for topic, resources, and residual verbosity.Because the inference setup requests exactly as many generated steps as the reference contains, this association combines task complexity with the increased error surface of longer outputs.Source: Table 10 · p. 33, §D.5 · p. 33
  • Residual verbosity is positively associated with no_failure: ORratio ≈1.01–1.02 per +1 percentage point, corresponding to roughly 10–20% higher odds for a +10pp increase.This association holds controlling for topic, step count, and resources, indicating residual verbosity bias despite step-count and concision constraints; the authors therefore report average generated tokens alongside How2Bench results.Source: Table 10 · p. 33, §D.5 · p. 34
  • Education & Jobs and Food & Dining tend to have higher conditional no_failure odds than Art & Design, while Electronics & Hardware and Industrial tend to have lower odds.These directions are broadly consistent across models after controlling for step and resource counts, though absolute magnitudes vary with model strength.Source: Figure 13 · p. 34, §D.5 · p. 34
  • The step-count association appears less extreme for strongest models such as GPT 5 and Claude Opus 4.5, where high no_failure rates leave less residual variance for predictors.Source: §D.5 · p. 34

E.1 Training data deduplication

  • Evaluation examples are deduplicated against training data by embedding goals and numbered reference steps, filtering nearest-neighbor cosine similarity above τ = 0.65, then sampling a topic-balanced clean set.The process uses Qwen/Qwen3-Embedding-0.6B (Qwen Team (2025b)) and samples up to a fixed number per topic with a fixed random seed.Source: §E.1 · pp. 34–35

E.2 Details on training setup

  • Training uses 100K pipeline-generated examples balanced across 14 topics; SFT and RL then apply separate fine-tuning setups to selected Qwen 3 and OLMo 3 checkpoints.SFT runs for one epoch at learning rate 5e−6 and batch size 64. RL uses GRPO for 1000 optimizer steps at learning rate 5e−7, with How2Score, a step-format verifier, and a reference-calibrated length reward.Source: §E.2 · p. 35

E.3 SFT results … G Qualitative examples of common failure patterns

The appendices report limited gains from SFT, define auxiliary rewards used during RL, and describe robustness, benchmark, topic-transfer, and checkpoint diagnostics. The supplied passages provide detailed methods and table scopes, but few underlying result values.

E.3 SFT results

  • SFT yields at best small gains from base checkpoints and does not improve, or can reduce, performance on instruction-tuned checkpoints.The authors suggest this may reflect a mismatch between imitating one reference procedure and rewarding any procedure that avoids critical failures.Source: Table 11 · p. 35, §E.3 · p. 35

E.4 Auxiliary format and length rewards used during RL

  • The RL reward adds binary How2Score, a step-format verifier, and a reference-calibrated length reward to the scalar GRPO reward.The verifier requires consecutively numbered steps starting at 1 and, when specified, the expected step count; the length reward gives full credit within τ = 0.2 of the reference-length ratio and decays exponentially outside that band with α = 5. Table 12 compares runs with and without length reward, gives an average reference length of 97.44 tokens, and notes that the no-length ablation was not run for OLMo-3-7B-Think.Source: §E.4 · pp. 35–36, Table 12 · p. 36

E.5 Judge robustness for RL gains

  • Table 13 reports scores before and after RL, absolute and percentage gains, and is titled as showing gains persisting under external judges.The supplied passages do not provide the table’s score values.Source: Table 13 · p. 36, §E.5 · p. 36

E.6 Details on out-of-domain benchmarks

  • The out-of-domain benchmark appendix identifies each benchmark’s target capability and says evaluations used the OLMES framework.Table 14 is described as listing the benchmarks and their primary capabilities.Source: §E.6 · p. 36, Table 14 · p. 37

E.7 Topic-restricted RL transfer across topics

  • Topic-restricted RL uses PCA projections of goal-text embeddings to select contrasting topic subsets, with Table 15 reporting overall and per-topic scores and deltas for Qwen3-8B with thinking.The table reports results after RL at step 1000; the supplied passages do not include its values.Source: §E.7 · p. 36, Table 15 · p. 37, Figure 14 · p. 38

F Analyses and diagnostics

  • The checkpoint diagnostic reports How2Score, conditional reference-step perplexity, and within-run ranks for both measures.The table specifies that lower perplexity is better, and the supplied header identifies the columns without providing checkpoint values.Source: Table 16 · p. 39, F Analyses and diagnostics · p. 40

G.1 Qualitative analysis details · H Prompt templates

This section illustrates how How2Judge identifies critical failures in generated procedures and documents the prompts used for data processing, inference, and judging, including criteria that distinguish goal-preventing omissions or vagueness from acceptable variations against a reference procedure.

G.1 Qualitative analysis details

  • Gemini 2.5 Pro omits the legally required 30-day co-owner waiting period, while Claude 4.5 Opus and GPT 5 preserve the notice-to-transfer sequence despite varying phrasing and order.The omission can lead to transfer before the co-owners’ purchase rights have been respected.Source: §G.1 · pp. 40–41
  • In the PCR example, GPT 5 refuses actionable step-by-step instructions and Gemini 2.5 Pro leaves the thermocycling program vague, both critical failures for executable procedures.The judge also flags omitted reaction setup instructions in the refusal example.Source: §G.1 · pp. 40, 42
  • All three frontier models produce coherent, actionable Bible-stamping procedures in one example, including helpful additions without omitting requirements.The examples thus include successful outputs alongside failures concentrated in recognizable patterns such as refusals and critical omissions.Source: §G.1 · pp. 40, 43

H.1 Prompts for the data pipeline

  • The data-pipeline prompts screen for named-entity-focused, pure-math, UI-interaction, creative, non-sequential, and unreasonable procedures, then rewrite goals and steps toward a deterministic, necessary, aligned sequence.The rewrite instructions call for one major action per step, while avoiding both excessive micro-steps and unnecessary details.Source: §H.1 · p. 44

H.2 Prompts for inference

H.3 Prompts for the LLM judge for How2Score

  • How2Score’s judge compares candidate steps L2 with a reliable, non-exclusive reference procedure L1 and flags failures that prevent goal achievement or make instructions unusable.Its criteria include contradictions, internal inconsistency, and incoherence that makes the procedure hard to follow.Source: §H.3 · pp. 49–50
  • Critical failures include omitted essential actions, severe vagueness, unnecessary or counterproductive additions, and redundant repetition.The prompt notes that these categories are not exhaustive and a single failure may span multiple categories.Source: §H.3 · p. 50
  • Judge only against the provided goal and L1, treating differences as failures only when severe enough to prevent success or make L2 incoherent or unusable.Stylistic or verbosity differences, harmless changes in ordering, neutral extra steps, and reasonable implicit equivalents are acceptable; missing essential detail that makes instructions unusable is severe vagueness.Source: §H.3 · p. 50
  • Examples classify altered soak time or cooking method, contradictions, misplaced steps, and omitted reinforcement as critical when they materially undermine the procedure.The prompt also illustrates severe vagueness with an L2 that omits many details needed to make the instructions usable.Source: §H.3 · p. 51
  • Examples treat practical extra steps, implied actions, valid posture substitutions, and adequate simplification as acceptable when essential actions remain clear.The camera example contrasts useful specified settings with vague terms such as “appropriate value,” which would make the instructions unusable.Source: §H.3 · pp. 51–52

Related papers

Loading 2602.08808v1…