PaperlayerView exact revision on arXiv

Source-linked AI summary

How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs

Yapei Chang, Kyle Lo, Mohit Iyyer, Luca Soldaini

arXiv:2602.08808v1cs.LG

TL;DR

Evaluating and improving LLMs’ procedural validity across diverse real-world tasks remains challenging, despite the importance of how-to generation for practical guidance and reasoning. How2Everything addresses this with web-scale procedure mining, benchmark construction, and critical-failure evaluation, while reinforcement learning improves performance by >10 points across three models without regressions on a standard evaluation suite.

  • Problem

    Evaluating and improving LLMs’ how-to generation across diverse procedures remains challenging, although this capability supports practical guidance and reasoning-intensive tasks.

  • Method

    How2Everything mines realistic procedures from broad web-document collections, creates evaluation and training resources, and uses How2Score with an open 8B judge to detect critical failures.

  • Results

    >10 points: reinforcement learning on mined procedures consistently improves How2Everything performance across three models without regressions on a standard post-training evaluation suite.

  • Takeaways & Limitations

    The framework enables more reliable evaluation of procedural instruction quality at scale and provides a practical reward signal for improving end-to-end procedural validity.

  • Takeaways & Limitations

    How2Score is an evaluation proxy rather than a guarantee of real-world correctness and does not substitute for expert review or execution-based verification in safety-critical settings.

Abstract

from arXiv · show

Generating step-by-step "how-to" procedures is a key LLM capability: how-to advice is commonly requested in chatbots, and step-by-step planning is critical for reasoning over complex tasks. Yet, measuring and improving procedural validity at scale on real-world tasks remains challenging and understudied. To address this, we introduce How2Everything, a scalable framework to evaluate and improve goal-conditioned procedure generation. Our framework includes How2Mine, which mines 351K procedures from 980K web pages across 14 topics and readily scales to larger corpora. From this pool we build How2Bench, a 7K-example evaluation set balanced across topics. To reliably score model outputs, we develop How2Score, an evaluation protocol that uses an LLM judge to detect whether a generation contains any critical failure that would prevent achieving the goal. For low-cost, reproducible evaluation, we distill a frontier model into an open 8B model, achieving 80.5% agreement with human annotators. How2Bench reveals clear scaling trends across model sizes and training stages, providing signal early in pretraining. Finally, RL using How2Score as a reward improves performance on How2Bench by >10 points across three models without systematic regressions on standard benchmarks, with gains robust to superficial source-document memorization or format compliance. Taken together, How2Everything shows how pretraining web data can support a closed loop of capability evaluation and improvement at scale.

1 Introduction

How2Everything addresses the difficulty of evaluating and improving end-to-end procedural validity across diverse real-world tasks. It mines web procedures, introduces scalable validity scoring, and uses the resulting resources to benchmark and improve models.

  • Motivation: 8.5% of ChatGPT conversations fall under How-To Advice, while procedural content exposure has improved reasoning- and planning-related downstream tasks.This motivates evaluating procedural generation as a broadly useful LLM capability.
  • Challenges: End-to-end procedural validity is difficult to evaluate because open-world procedures lack task-specific executors or automatic oracles, span diverse domains, and require valid entire action sequences.Existing work is often narrow in domain or focuses on subtasks such as step ordering.
  • Framework: How2Everything combines web-scale procedure mining, How2Bench and How2Train datasets, How2Score critical-failure evaluation, and the reproducible 8B How2Judge.How2Score checks whether omissions, extraneous actions, or deviations would prevent achieving the goal.
  • Evaluation: 80.5% agreement with human annotators makes How2Judge a low-cost, efficient, and reproducible evaluator for procedural generation.Frontier LLM judges showed 76.5–83.0% agreement with human majority across five models before distillation.
  • Benchmark: How2Bench shows clear scaling trends across model sizes and training stages, including models ranging from 1B to 32B parameters.The benchmark is intended to rank models across compute budgets and provide signal early in training.
  • Improvement: RL with How2Score improves How2Everything performance by >10 points across three models without systematic regressions on a standard post-training suite.The reported improvements sometimes also improve standard evaluations, suggesting procedure generation is useful across tasks.
  • Robustness: Performance gains remain robust to format compliance and are only modestly affected by aggressive source-document contamination.A 7B model improved by +3 points after contamination.
  • Broader contribution: The framework demonstrates how pretraining web data can support a closed loop of capability evaluation and improvement at scale.Mining and standardizing web data turns difficult-to-measure behavior into a practical development loop.

2 Problem Setting and Related Work

The paper studies descriptive, goal-conditioned procedures: textual action sequences whose validity must be assessed without execution. Prior datasets and evaluations are limited in coverage or reliability, motivating validity-oriented evaluation.

  • Problem setting: A procedure is a goal-conditioned sequence of actions, distinguished from executable procedures whose correctness is determined through execution.How2Everything focuses on descriptive procedures for real-world tasks.
  • Problem setting: Accurate textual procedure representation is a user-facing capability and a prerequisite for systems supporting or automating parts of real-world processes.The paper does not require models to execute the procedures themselves.
  • Related work: Existing descriptive-procedure datasets are constrained by narrow topical domains or limited instructional sources such as WikiHow and Instructables.Cooking is a common restricted domain.
  • Evaluation: Prior evaluations include step-pair prediction, reordering, question answering, constraint satisfaction, perplexity, and string overlap rather than pure end-to-end validity.These formulations often target subtasks or surface similarity.
  • Evaluation: String-overlap metrics are insufficient proxies for procedural validity, while human evaluation provides stronger signal at greater cost.This creates a reliability–scalability tradeoff that the paper addresses with a validity-oriented protocol.
  • Evaluation protocol: In How2Score, a model generates N steps from a goal and resources, and How2Judge checks whether any critical failure appears.Table 1 illustrates representative failures by highlighting relevant generated steps.

3 How2Mine: Extracting Realistic Step-by-Step Procedures from the Web

How2Mine extracts standardized, goal-conditioned procedures from topic-stratified web documents through filtering, refinement, and validation. The resulting pool supports a balanced benchmark and a larger training corpus.

  • Pipeline outputs: 980,000 web documents produced 351,162 structured procedure instances, and the pipeline scales to larger corpora without manual curation.Each instance contains a topic, goal, resources, and reference steps.
  • Sampling: How2Mine samples tutorial documents across 14 topics from the DCLM web corpus and covers 189K unique domains.Topic stratification is used to ensure equal topical coverage.
  • Scope: The demonstrated extraction focuses on one document format, although How2Mine can be extended to academic writing and knowledge articles.This is a scope boundary of the proof-of-concept pipeline.
  • Filtering: The pipeline first extracts goals and ordered steps, then removes trivial, overly complex, repetitive, domain-dependent, non-sequential, or nonsensical candidates.Heuristic and LLM-based filters enforce procedure quality and scope.
  • Refinement: Post-processing makes goals specific and deterministic and records referenced resources to narrow the space of valid solutions.A final LLM sanity check removes remaining invalid procedures.
  • Dataset construction: How2Bench samples 500 instances per topic for 7,000 total examples, while the remaining mined instances form How2Train.The benchmark is balanced across the 14 topics.

4 How2Score: Measuring Procedural Validity by Detecting Critical Failures

How2Score evaluates procedural validity by asking whether a generated procedure contains any critical failure that prevents achieving its goal. Binary aggregation improves agreement and supports scalable, reproducible judging through the distilled How2Judge model.

  • Defining Critical Failures: How2Score treats an omission, extraneous action, contradiction, severe vagueness, or similarly severe deviation as a critical failure when it prevents goal achievement or makes instructions unusable.The reference anchors evaluation without penalizing alternative valid procedures or superficial differences.
  • Evaluation Protocol: The protocol supplies the goal, resources, reference procedure, and generated procedure to an LLM judge, which identifies and describes critical failures.Failures are linked to relevant reference or generated steps before binary aggregation.
  • Evaluation Protocol: Binary aggregation labels a procedure has_failure when at least one critical failure is identified; otherwise it receives no_failure, and scores are success rates over no_failure examples.This formulation yields higher inter-annotator agreement than requiring agreement on the first failure location.
  • Human Validation: Krippendorff’s α = 0.593 for binary aggregation, compared with α = 0.307 when annotators must agree on the first failure location.The final human-annotation study labeled 200 examples with three annotators.
  • Judge Validation: GPT 5 achieves 83.0% overall agreement with human-majority labels, while How2Judge reaches 80.5% agreement with human annotators after distillation into an open Qwen 3 8B model.How2Judge is intended to make large-scale evaluation lower-cost and reproducible.

5 How2Bench: Evaluating Performance on Step-by-Step Procedure Generation

How2Bench evaluates step-by-step procedure generation across diverse models and training stages. It shows scaling with model size and training stage, early signal during pretraining, and robustness beyond formatting, reference likelihood, or judge-family preference.

  • Benchmark Construction: How2Bench contains 7,000 examples sampled evenly across 14 topics, with 500 procedures per topic.The benchmark is scored with How2Score using How2Judge.
  • Evaluation Results: Performance scales with model size and training stage, with a noticeable gap between open and closed models.Figure 4 reports How2Score alongside average generated-token counts; the average reference length is 97.44 tokens.
  • Judge Robustness: Across GPT, Gemini, and Claude judges, absolute scores vary but the relative ranking of closed models remains unchanged.The same generations are rescored with How2Judge, GPT 5, Gemini 2.5 Pro, and Claude 4.5 Opus.
  • Scaling Behavior: How2Bench exhibits smooth scaling across model size and training stage, with non-trivial performance emerging by about 5% into pretraining for a 1B model.Performance continues improving after this early-emergence point.
  • Validity Analysis: Formatting errors plateau early while How2Score continues improving, indicating later gains reflect procedural validity beyond surface formatting.The tracked proxies include step-count mismatch, duplicate-step frequency, and n-gram repetition.
  • Validity Analysis: How2Score is not simply reference-step likelihood: checkpoint-rank Spearman correlations with conditional perplexity range from 0.233 to 0.967 across five Olmo runs.The broad range indicates that benchmark ordering can differ substantially from perplexity ordering.
  • Difficulty Analysis: Reference step count is the dominant predictor of How2Score across models, making it a monotonic difficulty knob for controlled benchmark slices.Procedures requiring more steps are consistently harder.

6 Improving Step-by-Step Procedure Generation with RL

How2Score and How2Judge can serve not only as evaluation tools but also as an RL reward for improving procedure generation. RL produces substantial How2Bench gains while preserving performance without systematic out-of-domain degradation.

  • RL Method and Outcome: How2Score-based RL improves How2Everything performance by >10 points across three models.How2Judge supplies the reward signal using procedures from How2Train.
  • Controlling Length Gaming: Reference-calibrated length rewards keep generated-to-reference length near 1.0 and prevent apparent gains from length gaming.Without this reward, models inflate length to 1.34×–1.53× the reference.
  • External-Judge Robustness: RL gains persist when the same trained generations are reevaluated by GPT 5 and Gemini 2.5 Pro.This tests whether improvements are specific to the How2Judge reward used during training.
  • Out-of-Domain Effects: RL improves How2Bench performance without systematic degradation across standard out-of-domain evaluations.The suite covers knowledge, chat, math, code, and logical reasoning; changes are mixed but generally modest.
  • SFT Comparison: Additional SFT yields small gains on base checkpoints but does not improve instruction-tuned checkpoints.The authors suggest this may reflect mismatch between single-reference likelihood training and minimizing critical failures.

7 Robustness to Format and Memorization Confounds

The authors test whether How2Bench gains reflect implicit format learning or source-document memorization. The diagnostics indicate that improvements depend more on pretraining maturity and broad data coverage than on superficial compliance or repeated document exposure.

  • Robustness to Format and Memorization Confounds: The confound analysis finds evidence that neither implicit task format compliance nor source-document memorization explains the observed How2Score gains.These are the two confounding factors considered for a web-mined, LLM-judged benchmark.
  • Confound 1: Implicit Task Format Compliance: The pretraining-maturity diagnostic holds the post-training recipe fixed while varying the Olmo 3 7B pretraining checkpoint.The complementary data-coverage diagnostic holds the base model fixed while restricting RL topics.
  • Confound 1: Implicit Task Format Compliance: SFT gains remain similar across checkpoints at 3.39 to 5.36, whereas RL gains rise from 3.56 at 10% to 20.33 at 100% of pretraining FLOPs.The pattern suggests How2Bench is not primarily format-driven.
  • Confound 1: Implicit Task Format Compliance: Training on all topics yields the best overall performance, with a +10.10 gain, compared with RL using contrasting narrow-topic coverage.The comparison uses Science, Math & Technology and Food & Dining as diversity contrasts.
  • Confound 2: Memorization of Source Documents: The memorization test varies exposure to fixed source documents during midtraining and evaluates procedures extracted from those documents.Exposure frequencies are 0, 1, 3, 6, or 10 occurrences, with 0 as the unseen-document baseline.
  • Confound 2: Memorization of Source Documents: Increasing document exposure sharply lowers perplexity but improves How2Score only modestly and non-monotonically.The evaluation set contains 13,500 examples balanced across five occurrence groups.
  • Confound 2: Memorization of Source Documents: The memorization findings indicate that higher fit to source documents does not translate into comparable procedural-validity gains.This separates document memorization from the benchmark improvements measured by How2Score.

Impact Statement

How2Everything provides scalable measurement and training data for goal-conditioned, step-by-step procedures, with potential benefits for user-facing assistants. The authors also identify safety risks and state that How2Score is not a guarantee of real-world correctness.

  • Impact Statement: How2Everything mines and evaluates goal-conditioned step-by-step procedures from large-scale web corpora.The framework is intended to provide scalable measurement and data for improving procedural instruction quality.
  • Impact Statement: The framework could benefit assistants in troubleshooting, education, and everyday planning.
  • Impact Statement: Web-derived procedures may reflect societal biases, while safety-sensitive instructions could enable harmful behavior if misused.
  • Impact Statement: How2Score is an evaluation proxy, not a substitute for expert review or execution-based verification in safety-critical settings.The authors also plan additional safety and privacy filtering before release.

A Motivational analysis of query type distribution

How-to advice forms a substantial share of chatbot usage, motivating broad procedural coverage. The analysis anchors discussion to ChatGPT because publicly collected chat corpora show systematic query-type skews, while the mining pipeline supports multiple document formats.

  • Query type distribution: 8.5% of ChatGPT conversations are classified as How-To Advice, ranking fourth among 23 fine-grained categories.Specific Info, Edit or Critique Provided Text, and Tutoring or Teaching rank above it.
  • Query type distribution: Tens to hundreds of millions of how-to interactions may occur daily at current ChatGPT scale.
  • Query type distribution: WildChat and LMSYS-Chat show systematic query-type skews relative to ChatGPT because they come from unrestricted public LLM endpoints.The authors therefore anchor real-world usage discussion to the ChatGPT distribution.
  • Document formats: How2Mine supports procedure extraction beyond tutorial-style pages, although “Tutorial & How-to Guide” has the highest yield at every reported stage.The comparison covers extraction, heuristics filtering, and LLM filtering before postprocessing and final validation.

B.3 Implementation details on the heuristics filter

The heuristics filter removes procedures with unsuitable lengths or repetitive wording before later LLM-based filtering. Additional criteria exclude examples whose correctness or validity cannot be reliably judged in the intended setting.

  • Heuristics filter: Procedures are required to contain between min_steps and max_steps steps, with defaults of 5–15.The bounds are configurable through command-line flags.
  • Heuristics filter: Procedures are rejected when repeated n-gram rates reach 0.40 for bigrams, 0.35 for trigrams, or 0.30 for fourgrams.This primarily removes degenerate extractions containing near-identical phrases or steps.
  • LLM filter criteria: The LLM filter removes named-entity-focused procedures because correctness is often time- and access-dependent without entity context.
  • LLM filter criteria: The LLM filter also excludes pure math, UI interaction, open-ended creative generation, non-sequential processes, and unreasonable procedures.These cases are difficult to assess through critical-failure detection or lack clear sequential validity.

D.2 Full intermediate checkpoint evaluation results on How2Bench

The checkpoint evaluation reports How2Score alongside formatting proxies and conditional reference-step perplexity across model trajectories. The listed OLMo checkpoints improve from midtrain to posttrain, with larger gains for the 32B model.

  • Checkpoint results: How2Score rises from 24.91 for OLMo-3-1025 7B Midtrain to 30.23 for Posttrain.Average generation tokens increase from 96.67 to 101.60 across these checkpoints.
  • Checkpoint results: How2Score rises from 38.31 for OLMo-3-1125 32B Midtrain to 43.16 for Posttrain.Average generation tokens change from 95.19 to 100.77 across these checkpoints.
  • Formatting proxies: Formatting analysis reports step-count mismatch, duplicate steps, and duplicate n-gram rate as proxy metrics on model generations.The metrics complement How2Everything scores across checkpoint trajectories.
  • Checkpoint comparison: Conditional reference-step perplexity is compared with How2Score to assess checkpoint ordering within each OLMo trajectory.The comparison uses Spearman rank correlation across nine checkpoints.

D.5 Instance-level correlates of How2Score no_failure

How2Score labels are modeled using topic-controlled logistic regression over reference complexity, resource count, and generation verbosity. Longer procedures are harder to complete without critical failures, while topic and verbosity effects also vary systematically.

  • Reference step count is the dominant predictor: each additional required step decreases the odds of no_failure across models.ORsteps ranges from 0.74 to 0.85 after controlling for topic, resources, and residual verbosity.
  • The step-count effect combines intrinsic task complexity with the larger error surface created by requiring longer outputs.Inference requests exactly n generated steps, where n = |S⋆|.
  • Residual verbosity is positively associated with no_failure, with ORratio ≈1.01–1.02 per additional percentage point of generation-to-reference length.A +10pp increase corresponds to roughly a 10–20% increase in the odds of no_failure, holding other covariates fixed.
  • Estimated step-count effects are less extreme for frontier models, consistent with ceiling effects when overall no_failure rates are high.Higher baseline performance leaves less residual variance for simple predictors to explain.
  • Topic effects remain large after controlling for step count and resources: Education & Jobs and Food & Dining have higher conditional odds, while Electronics & Hardware and Industrial have lower odds than Art & Design.Figure 13 displays topic offsets as log-odds relative to Art & Design while also controlling for the generation/reference length ratio.

E.3 SFT results

The section reports SFT and RL training details alongside evaluation controls and benchmark references. SFT provides limited or negative gains, whereas the broader RL setup combines How2Score with formatting and length rewards.

  • SFT results: SFT yields at best small gains from base checkpoints, but does not improve and can decrease performance on instruction-tuned checkpoints.The authors attribute this pattern to objective mismatch between reference imitation and rewarding any valid procedure without critical failures.
  • Auxiliary rewards: The step-format verifier returns 1 when numbering starts at 1, remains consecutive, and matches the expected step count when one is provided; otherwise it returns 0.
  • Auxiliary rewards: The length reward gives full credit within a tolerance band around a generation/reference token ratio of 1.0 and decays exponentially outside it.The reported tolerance is τ = 0.2, and α = 5 controls the decay.
  • Evaluation coverage: Table 12 evaluates whether length control prevents verbosity hacking, while Tables 13–15 examine judge robustness, out-of-domain benchmarks, and topic-specific RL.

G.1 Qualitative analysis details

The qualitative analysis illustrates how How2Score identifies refusals, missing operational details, and subtle omissions while distinguishing coherent, actionable procedures from critical failures. The appendix also documents prompts used throughout extraction and judging.

  • Crime & Law: A plausible legal procedure can still fail by omitting a required waiting period that protects co-owners’ purchase rights.Claude 4.5 Opus and GPT 5 preserve the notice, delivery proof, waiting period, and transfer structure despite wording differences.
  • Science, Math & Technology: How2Score treats refusals and missing operational conditions as critical failures when they make technical procedures non-executable.Examples include refusing step-by-step experimental instructions and omitting thermocycling parameters in a PCR protocol.
  • Art & Design: Frontier models can produce coherent, actionable procedures with helpful additions without omitting requirements.The Art & Design example shows that critical failures concentrate in recognizable patterns rather than occurring universally.
  • Procedure extraction: The extraction prompt requires at least three sequential, imperative, atomic, and concrete executable steps, plus a goal stating the intended outcome and essential constraints.Extracted steps may differ from their original document presentation.
  • Critical-failure judging: The judge’s operational definition of critical failure includes omissions, extraneous actions, contradictions, severe vagueness, and other deviations severe enough to prevent goal achievement or make instructions unusable.The reference anchors evaluation but does not exclude alternative valid procedures or superficial differences.
Loading 2602.08808v1…