xVerify: Efficient Answer Verifier for Reasoning Model Evaluations

1 Introduction

• Reasoning models produce redundant information, intermediate results, and self-reflections, while existing evaluation methods and reward models often prove ineffective on such outputs.Rule-based systems struggle with varied formats and lengthy reasoning, while judge models are not explicitly trained for objective correct-or-incorrect decisions.Source: §1 · pp. 1–2
• xVerify-0.5B-I surpasses existing evaluation methods, including 32B models, on all metrics; larger variants exceed 95% F1 and accuracy on test and generalization sets.With xVerify as reward, Qwen2.5-7B improves 18.4% over direct generation, exceeding the gains with Math Verify.Source: §1 · p. 2
• xVerify processes full responses to identify final answers and checks answer equivalence across mathematical expressions, symbols, and natural language.It is designed for objective questions and tolerates formatting errors, including malformed LaTeX.Source: §1 · p. 2
• VAR contains answers from 19 LLMs across 24 benchmarks, with labels verified through multiple rounds of GPT-4o and human review.The dataset is designed for training and evaluating judge models on reasoning tasks.Source: §1 · p. 2

2 Related Work

• Rule-based methods can miss equivalent answers expressed in different surface forms and struggle to extract final answers from lengthy reasoning with intermediate results.They may rely on regular expressions or limited transformations and can fail when reasoning models revise initial predictions.Source: §2 · pp. 2–3
• LLM-based judges adapt across tasks but are less effective at binary correctness decisions because they are generally designed to score outputs, not make objective judgments.Thresholding scores is described as unreliable, and critic models and process reward models perform poorly at detecting errors in long reasoning traces.Source: §2 · p. 3
• Automatic evaluation of objective tasks remains underdeveloped, and human annotation is costly and difficult to scale; the paper proposes xVerify as a targeted judge model.The proposal targets objective evaluation of LLM responses.Source: §2 · p. 3

• The task maps questions, LLM responses, and reference answers to an evaluation function that returns whether each response is correct.The response set is generated by applying an LLM W to each question.Source: §3 · p. 3
• The extraction function selects the candidate answer with the highest relevance or suitability to the question from a response that may contain reasoning and multiple candidates.Source: §3 · p. 3
• The equivalence function integrates mathematical, natural-language, and symbolic checks to determine whether a predicted answer matches the reference.For mathematical expressions, normalization transforms expressions into a canonical form for comparison.Source: §3 · p. 3
• Natural-language answers are compared using semantic alignment and a threshold, while symbolic answers are normalized into a standardized form for character-level comparison.Source: §3 · pp. 3–4

4 Methodology

• The pipeline collects LLM responses, constructs and reviews VAR with data augmentation, then trains xVerify models to evaluate long, multi-step answers.Source: §4 · p. 4

4.1 VAR Dataset

• VAR was constructed as a dedicated dataset for training and evaluating xVerify because existing judge-model datasets did not suit lengthy reasoning-model responses.Source: §4.1 · pp. 4–5

4.1.1 LLM Response Generation

• Responses were collected from 19 LLMs across 24 evaluation datasets, including challenging reasoning benchmarks and long responses.The collection yielded 191,600 Q&A samples.Source: §4.1.1 · p. 5

4.1.2 Dataset Partitioning

• The test set shares the training distribution, while the generalization set uses seven held-out evaluation datasets and includes four LLMs unseen in training or testing.The training and test sets cover 17 evaluation sets; the generalization set covers all four question types.Source: §4.1.2 · pp. 5–6

4.1.3 Data Annotations

• GPT-4o annotated all samples in two rounds; humans reviewed challenging math and disagreements in training, and all test and generalization samples.Source: §4.1.3 · p. 6

4.1.4 Data Augmentation

• Augmentations vary multiple-choice labels and distractors, generate mathematically equivalent expressions, and transform final-answer statements to diversify formats.After augmentation, training, test, and generalization sets contained 43,204, 6,122, and 6,468 samples, respectively.Source: §4.1.4 · p. 6

4.2 Model Training

• The authors trained 14 xVerify models spanning 0.5B to 32B parameters and five model families using VAR training data.Training used LLaMA-Factory and QLoRA, with one epoch and a learning rate of 1e-4.Source: §4.2 · p. 6

5 Experiments

• The judge evaluation used VAR test and generalization sets with accuracy and F1 as main metrics, while RL experiments compared xVerify-7B-I rewards with Math-Verify.The generalization set assesses out-of-distribution robustness; the RL comparison was a proof of concept focused on reward-signal quality and training dynamics rather than maximizing final scores.Source: §5 · pp. 6–7, §5.2 · p. 8

5.1 Evaluation with xVerify as Judge Model

• 97.50% F1 and 97.41% accuracy were achieved by xVerify-7B-I on the VAR test set, while xVerify-0.5B-I reached 96.69% F1 and 96.85% accuracy.The smallest model matched GPT-4o and surpassed CompassJudger-1-32B; all xVerify models above 0.5B exceeded 95% on challenging math questions.Source: Table 1 · p. 7, §5.1 · p. 7
• On the VAR generalization set, xVerify’s overall F1 and accuracy drop by less than 1.5%; xVerify-0.5B-I retains 95.53% accuracy, outperforming all rule-based frameworks and most judge models except GPT-4o.Larger xVerify models reduce the performance gap further: xVerify-14B-Ia reaches 96.65% accuracy with over 90% on math questions.Source: §5.1 · p. 7

5.2 Reinforcement Learning with xVerify as Reward Model

• 18.4%: RL with xVerify improved Qwen2.5-7B’s seven-benchmark average over direct generation; its final average also exceeded Math-Verify’s for both policies.Final averages were 73.0% versus 72.2% for Qwen2.5-7B and 61.2% versus 60.4% for Llama3.1-8B; xVerify started higher and converged in fewer steps.Source: Table 3 · p. 8, §5.2 · p. 8

6 Conclusion

• The smallest xVerify-0.5B-I outperformed all methods except GPT-4o, while larger models achieved the best overall performance on test and generalization sets.xVerify was trained at multiple scales using VAR, which collects long-form answers from multiple LLMs and benchmarks with multi-round GPT-4o and human verification.Source: §6 · p. 8
• xVerify was effective as an RL reward model, enhancing policy performance compared with direct generation.Source: §6 · p. 9

Limitations

• xVerify currently targets objective questions with definitive answers and depends on reference answers, limiting its use for subjective questions and as a general-purpose reward model.The authors identify reference-free scenarios, including subjective questions, as requiring further work.Source: Limitations · p. 9

A Datasets and Models

• 24 datasets span multiple-choice, short-answer, math, and classification tasks, with Chinese and English examples for each question type and a 2:1 training-to-test split.The target split sizes were 2,000 training and 1,000 test instances per dataset where available.Source: A Datasets and Models · p. 12, Table 4 · p. 13
• 19 LLMs of varied sizes and types, particularly reasoning models, generate responses used to train xVerify.Source: A Datasets and Models · p. 12

B.1.1 Training Set

• VAR contains 43,204 training samples, 6,122 test samples, and 6,468 generalization samples.Source: §B.1.1 · p. 12, §B.1.2 · p. 12, §B.1.3 · p. 12

B.2 Details of Human Annotation

• Eight annotators handled VAR review, with two master’s-degree reviewers resolving complex cases and annotation disagreements.The process lasted three weeks across 15 working days.Source: §B.2 · p. 13
• Annotation compares outputs with the question and reference answer, applying exact or semantic matching for short answers and mathematical equivalence for numerical answers.For multiple choice, correct answer content takes precedence over an inconsistent option label; ambiguous or irrelevant matched answers are marked incorrect.Source: §B.2 · pp. 13, 14, 17

B.3 Examples from the VAR Dataset

• Representative VAR examples illustrate the structure of dataset instances.Source: §B.3 · p. 17

C.1 Training Hyperparameters

• xVerify uses QLoRA with consistent hyperparameter settings across base models.Table 18 presents the key training hyperparameters.Source: §C.1 · p. 17, Table 18 · p. 17

C.2 Original Model Details

• The study trains xVerify from 14 original models spanning different parameter scales and types.Table 19 lists the original models and corresponding xVerify models.Source: §C.2 · p. 18

D.1 RL Training Hyperparameters

• RL training uses the veRL framework, with the same hyperparameters for baseline methods.The experiments run on multiple GPU servers, and Table 20 lists the RL hyperparameters.Source: §D.1 · p. 18

D.2 Details of RL Training and Generalization Sets

• RL policy training uses 20,400 samples from the generalization set to avoid data contamination, with a separate 1,750-sample generalization set.Source: §D.2 · p. 18

E.1 Prompts for Generating LLM Responses

• Response-generation prompts populate task type, task description, examples, and question fields, using either zero-shot or five-shot examples.The prompt templates and example fields are illustrated in Figures 5–10.Source: §E.1 · p. 21, Figure 5 · p. 21, Figure 6 · p. 21

E.2 Prompts for GPT-4o Annotation

• GPT-4o dataset annotations use two prompt templates employing chain-of-thought to support annotation accuracy.Source: §E.2 · p. 21, Figure 11 · p. 23, Figure 12 · p. 23

E.3 Prompts for Data Augmentation

• Two GPT-4o prompts guide augmentation of mathematical question samples by generating alternative references and diverse final-answer expressions.Source: §E.3 · p. 21, Figure 13 · p. 24, Figure 14 · p. 25

E.4 Prompts for Judge Model

• Judge prompts draw on developers’ official templates, with alternatives used where those templates do not fit the evaluation tasks.The appendix identifies templates for GPT-4o, JudgeLM, PandaLM, Auto-J, and Prometheus; CompassJudger-1 uses the xVerify prompt.Source: §E.4 · p. 21, Figure 15 · p. 26, Figure 16 · p. 26

E.5 Prompts for xVerify

• The xVerify training and evaluation prompt contains question, output, and answer fields for the query, LLM response, and reference answer.Source: §E.5 · p. 29

F.3 The Consistency between xVerify and Human Evaluation

• Figure 21 shows that xVerify consistently achieves high alignment with human evaluation across multiple datasets and scenarios.The evaluation uses the datasets and models employed in the reinforcement-learning experiments.Source: §F.3 · p. 29, Figure 21 · p. 30

F.1 Evaluation Accuracy Results of All xVerify Models and Judge Models

• Every xVerify model exceeds 96.5% in both F1 and accuracy on the test set and 95.52% accuracy on the generalization set, outperforming other judge models.Generalization performance drops by no more than 1.6% from test performance; gains with increasing model size eventually plateau.Source: §F.1 · p. 29, Table 23 · p. 31, Table 24 · p. 31

F.2 Computational Efficiency and Operational Cost of xVerify and Judge Models

• All xVerify models average under 100 seconds and are the fastest for each question category; other judge models average over 100 seconds.Apart from deployment overhead, xVerify evaluation costs substantially less than GPT-4o, and local deployment offers higher invocation efficiency.Source: §F.2 · p. 29, Table 25 · p. 32, Table 26 · p. 32

F.4 Learning Curves of Qwen2.5-7B and Llama3.1-8B in Reinforcement Learning

• With xVerify as the reward model, Qwen2.5-7B and Llama3.1-8B show higher initial accuracy and faster convergence during reinforcement learning.The paper attributes these learning-curve results to more accurate and reliable feedback that improves training efficiency.Source: §F.4 · pp. 29, 31, Figure 22 · p. 33, Figure 23 · p. 33

F.5 Full Fine-Tuning vs. QLoRA for xVerify Models

• In a small-scale comparison, QLoRA outperforms full fine-tuning overall and maintains a better balance between test performance and generalization.Full fine-tuning’s larger decline on the generalization set suggests it may be more prone to overfitting; QLoRA also offers computational savings.Source: §F.5 · p. 32, Table 27 · p. 33

F.6 Comparison of xVerify models with their base models

• VAR-based fine-tuning improves the four xVerify variants by over 10 percentage points in both F1 and accuracy versus their base models.xVerify-0.5B-I and xVerify-1B-I each improve by 20%–50% on these metrics.Source: §F.6 · p. 33, Table 28 · p. 34

F.7 Evaluation without the original question

• Both evaluated xVerify models show virtually no performance drop across test and generalization sets when the original question is omitted.The results suggest that answer-reference equivalence is the primary basis of judgment, while the question is auxiliary context.Source: Table 29 · p. 34, §F.7 · p. 35