The Hidden Signal of Verifier Strictness: Controlling and Improving Step-Wise Verification via Selective Latent Steering

Motivation

• Verifier positivity bias can cause flawed solutions to be incorrectly accepted, while specialized verifier training is costly and may quickly become outdated.These limitations motivate methods applicable directly to general instruction-tuned models.Source: §1 · pp. 1–2

Core finding

• The verifier’s accept-or-reject tendency is encoded near the paragraph-boundary delimiter token, enabling hidden-state steering to modulate strictness without fine-tuning.Uniform contrastive steering can improve error detection or correctness certification while preserving semantic coherence.Source: §1 · pp. 2–3

VerifySteer

• VerifySteer routes samples using a correctness probe and selectively intervenes on misaligned paragraph-boundary states to address uniform steering’s strictness trade-off.Increasing strictness improves error detection but can reduce correctness certification, whereas decreasing strictness has the opposite effect.Source: §1 · p. 3

Results

• VerifySteer outperforms prompt optimization, activation steering, and self-consistency baselines across ProcessBench and Hard2Verify while using less inference compute.On ProcessBench with Qwen3-1.7B it surpasses self-consistency by 3.7 F1 using 4× less compute; on Hard2Verify with Qwen3-8B-thinking it gains 2.2 F1 using 7× less compute.Source: §1 · p. 3

Evaluation scope

• The evaluation spans models from 1.7B to 20B parameters, including dense and mixture-of-experts architectures and thinking and non-thinking modes.The method is also reported as compatible with verification fine-tuning.Source: §1 · p. 3

LLM Verification

• Generative verifiers provide an evaluation paradigm spanning step-level and sequence-level verification, but prompted approaches exhibit positivity bias in self-verification and intra-family verification.The related literature frames calibration as a central limitation of general instruction-tuned verifiers.Source: §2 · p. 3

Representation Engineering

• Representation engineering treats hidden representations as objects of analysis and intervention, with activation steering used to control behaviors such as refusal and hallucination.These studies motivate testing whether verification behavior is similarly encoded in activation space.Source: §2 · pp. 3–4

Step-Wise Verification

• Given a problem and step-by-step solution, the verifier predicts −1 when all steps are correct or the index of the earliest incorrect step otherwise.Its verification chain of thought evaluates individual solution steps separated by double-newline delimiters.Source: §3 · p. 4

Evaluation Metrics

• F1 is the harmonic mean of TPR and TNR, where TPR measures correct acceptance of fully correct solutions and TNR measures correct identification of errors.These metrics distinguish correctness certification from error detection.Source: §3 · p. 4

Benchmarks and Datasets

• ProcessBench and Hard2Verify evaluate first-error identification on mathematical reasoning with human-annotated error locations.ProcessBench spans GSM8K, MATH, OlympiadBench, and Omni-MATH, while Hard2Verify uses challenging IMO and Putnam problems.Source: §3 · p. 4

Categorization of Verification Mistakes

• Under-critical mistakes accept an erroneous step or miss its location, whereas over-critical mistakes reject correct reasoning; insufficient and excessive strictness correspond to these two failure modes.True rejection and true acceptance are the desired counterparts for incorrect and fully correct solutions, respectively.Source: §3 · p. 4

4.1 Dissecting Verification Failures

• Baseline verification produces superficial reasoning that repeats erroneous steps or promotes stylistic issues into errors, motivating a latent decision signal before paragraph generation.This pattern supports locating the accept-or-reject tendency at the delimiter preceding each verification paragraph.Source: §4.1 · p. 5

4.2 Verification Strictness is Adjustable by Hidden-State Steering

• The method builds dstrict from mean hidden states of false-acceptance and true-rejection segments, and dlenient from true-acceptance and false-rejection segments.These contrastive directions represent movement toward stricter or more lenient verification behavior.Source: §4.2 · p. 6
• At test time, the intervention perturbs every generated paragraph-boundary token and renormalizes the hidden state while scaling the perturbation by α.The framework uses contrastive activation addition as a standard steering method rather than introducing a new steering primitive.Source: §4.2 · pp. 5–6
• 33.3% fewer under-critical mistakes with dstrict and 56.5% fewer over-critical mistakes with dlenient at the most effective setting.The corresponding TNR and TPR improve as mistake counts decrease.Source: §4.2 · p. 6, Figure 2 · p. 7
• The effect changes smoothly with steering strength and concentrates around layer 22, indicating that verification decision-making is localized rather than uniform across layers.Experiments sweep layers across the 36-layer Qwen3-8B transformer and use steering strengths from 1.0 to 3.0.Source: §4.2 · p. 6

Adaptive Control

• Uniform strictness steering improves error detection but can reduce correctness certification, motivating adaptive control rather than a fixed steering policy.The strictness-only variant generally improves TNR while preserving TPR, whereas the bidirectional variant usually improves both metrics; uniform CAA has similar TNR but lower TPR and F1.Source: §5.1 · p. 7, §5.2 · p. 9
• VerifySteer combines sample-level routing with delimiter-level gating to select strictness directions and intervene only on misaligned delimiter states.A correctness probe routes likely-correct samples toward leniency or stricter steering, while the cosine-similarity gate avoids unnecessary intervention.Source: §5.1 · pp. 7–8

Main Results

• VerifySteer outperforms base and optimized prompts across all evaluated models and benchmarks, while closing the gap to larger models.With steering, Qwen3-8B reaches 72.3 overall F1 versus 71.6 for Qwen3-14B on ProcessBench, and Qwen3-8B-thinking reaches 33.4 versus 30.4 for Qwen3-14B-thinking on Hard2Verify.Source: §5.2 · p. 9
• VerifySteer surpasses Cons@4 by 3.7 F1 on Qwen3-1.7B, matches it on Qwen3-8B, and exceeds Cons@8 by 2.2 F1 on Hard2Verify with 7× fewer FLOPs.Cons@4 requires around 4× the inference FLOPs of VerifySteer, while self-consistency does not effectively improve TNR.Source: §5.2 · p. 9

Compatibility

• VerifySteer stacks with verification fine-tuning, raising F1 from 41.5 for FARE-20B to 51.1 after steering.On ProcessBench, steered Qwen3-1.7B reaches 29.7 overall F1 versus 25.1 unsteered and exceeds several larger, older-generation fine-tuned PRMs.Source: §5.2 · p. 9

Ablations

• Sample-level adaptivity balances TNR and TPR, whereas removing delimiter-level adaptivity lowers TNR by making steering less precisely targeted.The full method preserves both high TNR and high TPR; the ablations isolate the contribution of each adaptive component.Source: §5.2 · pp. 9–10, Figure 3 · p. 10

• VerifySteer matches or surpasses self-consistency while requiring 4–7× less inference compute.It outperforms prompting and self-consistency baselines across the reported ProcessBench and Hard2Verify experiments.Source: §6 · p. 10
• Verifier strictness is encoded in a verification-specific latent signal that can be modulated without fine-tuning.The signal appears near paragraph boundaries, and steering it changes the verifier’s tendency to accept or reject steps.Source: §6 · p. 10
• Uniform steering improves either error detection or correctness certification but creates an inherent trade-off.VerifySteer addresses this trade-off through sample-level routing and selective paragraph-level intervention.Source: §6 · p. 10
• The results suggest that verifier behavior in general instruction-tuned models contains controllable latent behaviors suitable for lightweight adaptive control.Source: §6 · p. 10

• The evaluation uses three prompt templates for step-wise verification, including FARE-specific, basic, and optimized prompts.The FARE prompt is used for FARE-20B to match its fine-tuning setup.Source: A Prompt Templates · pp. 16–17

Basic Prompt

• The basic templates ask the verifier to review each solution paragraph or step and identify the first incorrect index, returning -1 when no error is found.They require an explanation before the final verdict and use explicit output formatting.Source: A Prompt Templates · pp. 16–17

Optimized Prompt for ProcessBench

• The optimized ProcessBench prompt adds detailed analysis, recalculation, reflection, and a request to be “as critical as possible” to increase strictness.It is adapted from Zhang et al. (2025c) and reports F1 gains across all ProcessBench models and subsets.Source: A Prompt Templates · p. 16

Optimized Prompt for Hard2Verify

• The optimized Hard2Verify prompt uses a separate “strict, reliable math grader” instruction because the ProcessBench prompt does not transfer well to Qwen3-8B-thinking.It provides more structured mathematical-correctness and logical-consistency checks and is adapted from Pandit et al. (2025).Source: A Prompt Templates · p. 16

• The implementation localizes a candidate verification paragraph using the ground-truth or predicted step index and then checks for the expected verdict cue.The keyword list is provided in Table 4.Source: §B.1 · p. 18
• A paragraph is labeled Acceptance when it contains at least one required acceptance cue and none of the excluded cues.The same rule is applied analogously to identify Rejection paragraphs.Source: Table 4 · p. 18

• VerifySteer uses ActPRM-derived validation sets for steering-vector extraction and correctness-probe training, aligned with ProcessBench subsets by difficulty and problem type.Source: §B.2 · p. 18
• The correctness probe is a three-layer ReLU MLP with dropout 0.1, trained using cross-entropy and Adam with learning rate 10−5.Its input layer and pooling strategy vary by model based on validation AUC and model configuration.Source: §B.2 · p. 18
• A linear classifier’s validation AUC measures how well delimiter-token hidden states distinguish true rejection from false acceptance paragraphs.The steering layer is selected using this separability criterion, following prior work.Source: §B.2 · p. 19, Figure 4 · p. 19
• VerifySteer-Uni uses only dstrict with a routing threshold, whereas VerifySteer-Bi additionally applies dlenient with lower and upper routing thresholds.ρ is the delimiter-level cosine-similarity threshold introduced in Equation (5).Source: §B.2 · p. 19
• For Qwen3-8B on ProcessBench, αstrict is tuned per subset: 2.0 for GSM8K and MATH, and 1.0 for OlympiadBench and Omni-MATH.Source: Table 5 · p. 19

C Additional Results

• In the under-critical algebra case, dstrict led the verifier to recompute the expansion and identify the discrepancy between a4 −152a2 + 196 = 0 and a4 −197a2 + 196 = 0.Source: C Additional Results · pp. 19–20
• In the over-critical decimal case, dlenient made the verifier recognize that the approximate value was consistent with the exact value and accept the step.Source: C Additional Results · p. 20
• Steering corrected both under-critical and over-critical verification failures across four ProcessBench case studies.The top two cases concern false acceptance of erroneous steps, while the bottom two concern false rejection of correct steps.Source: C Additional Results · p. 19, Table 6 · p. 20