A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR

Thi Kim Trang Vo; Nam Tien Le; Thi Kim Nguyet Vo; Minh Khang Tran; Duy Phuong Tran

Source-linked AI summary

A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR

Thi Kim Trang Vo, Nam Tien Le, Thi Kim Nguyet Vo, Minh Khang Tran, Duy Phuong Tran

arXiv:2609.05221v1Sep 4, 2026cs.CLcs.AIcs.LG

TL;DR

LLM explanations in educational and scientific QA can be plausible but insufficiently grounded or verifiable. The paper combines gold-anchored QLoRA, task-aware symbolic verification, and verifier-guided RLVR, finding that RLVR substantially strengthens reasoning depth while symbolic verification complements the neural policy for system-level reliability.

Problem
Educational and scientific QA needs explanations that are not only correct but also supported by valid evidence, appropriate formulas, and consistent units.
Method
The framework combines gold-anchored QLoRA, task-aware logic and physics verification, verifier-guided candidate revision and RLVR, and separate P1/P2/P3 evaluation.
Results
RLVR raises P3 from 50.68% to 72.20%, while hybrid P1 remains close to calibrated SFT at 55.94% versus 56.62%.
Takeaways & Limitations
RLVR mainly reshapes explicit reasoning, while symbolic verification selectively corrects high-confidence numerical failures.
Takeaways & Limitations
Physics remains limited by formula selection, numerical execution, and scientific-notation normalization.

Abstract

from arXiv · show

Large language models (LLMs) show strong reasoning ability, but their explanations can remain inconsistent, weakly grounded, or difficult to verify. We propose a verifier-guided explainable reasoning framework for transparent educational question answering that combines gold-anchored QLoRA, task-aware symbolic routing, and group-relative RLVR. Qwen2.5-3B-Instruct is first adapted with field-weighted QLoRA supervision anchored to authoritative answers. A lightweight router then assigns logic problems to a FOL/Z3 verifier and physics problems to a formula- and unit aware symbolic solver. Verifier feedback is further used to support candidate evaluation, self-revision, and reward construction during RLVR. Candidate responses are evaluated along three complementary dimensions: P1 for answer correctness, P2 for evidence or unit consistency, and P3 for reasoning depth and explainability. At inference, gold-free self-consistency aggregates multiple candidate responses before an optional question-only physics verifier performs conservative system-level correction. On 438 held-out examples, RLVR increases P3 from 50.68% to 72.20%, while hybrid P1 remains approximately stable at 55.94%. Self-consistency improves model only P1 from 48.86% to 50.23%, with symbolic verification providing the remaining hybrid gain. These results indicate that RLVR primarily strengthens explicit reasoning structure, while symbolic verification complements the neural policy by improving answer reliability at the system level.

I. INTRODUCTION

The framework targets educational QA explanations that may be plausible yet unsupported, combining authoritative-answer adaptation, task-specific symbolic verification, and verifier-guided RLVR. It evaluates correctness, supporting consistency, and reasoning depth while combining neural generation with external verification.

LLM explanations can remain untrustworthy when correct answers rely on invalid inferences, inappropriate formulas, unsupported evidence, or inconsistent units.
Answer-only rewards cannot distinguish accidentally correct predictions from well-supported solutions.
The framework adapts Qwen2.5-3B-Instruct with gold-anchored, field-weighted QLoRA and routes logic and physics tasks to external symbolic experts.
P1 measures final-answer correctness, P2 evidence or unit consistency, and P3 reasoning depth and explainability.
The seven-stage pipeline combines canonicalization, QLoRA calibration, task-aware routing, symbolic verification, self-revision, RLVR, and gold-free self-consistency with optional physics verification.
The MoE is task-level routing that dispatches problems to external logic or physics verifiers rather than modifying the Transformer backbone.

B. Gold-Anchored QLoRA Fine-Tuning

Gold-anchored fine-tuning preserves authoritative answers while field-weighted supervision emphasizes structured answer, unit, and premise information. Task-aware routing then sends problems to external logic or physics verification experts.

Authoritative answers remain locked while teacher-generated supervision may be repaired.
Aligned targets contain answers, units, premises, explanations, and compact reasoning traces to prevent noisy teacher outputs from replacing authoritative labels.
4-bit NF4 QLoRA adapts Qwen2.5-3B-Instruct with r = 32, α = 64, and dropout 0.05.
Field-weighted completion-only SFT assigns larger weights to answer, unit, and premise fields, followed by answer-focused calibration.
The task-aware router operates at verification level and dispatches each problem to an external task-specific verifier.
The logic expert converts premises into FOL representations and checks candidate entailment using Z3.
The physics expert checks formula applicability, numerical consistency, and units while the LLM remains the primary reasoning model.

D. Verifier-Guided Self-Revision

Verifier feedback identifies specific defects in candidate answers and supports both revision and quality-based comparison. This couples answer correction with evaluation of whether the reasoning is adequately supported.

Verifier feedback flags unsupported premises, incorrect formulas, unit inconsistencies, and answer–explanation conflicts.
Candidates are revised using task-specific verifier feedback before further evaluation.
Verifier-derived quality signals favor candidates whose answers and reasoning are jointly better supported.

E. P1/P2/P3 Group-Relative RLVR

Group-relative RLVR trains responses using verifiable, multi-dimensional rewards and compares candidates against their group. The objective combines answer quality, consistency, reasoning depth, formatting, and reference-policy regularization.

RLVR samples K = 3 responses per training question and applies group-relative optimization from the calibrated SFT checkpoint.
The final reward combines exact answer correctness, dense answer credit, P2 consistency, P3 reasoning depth, and formatting quality.
The framework routes tasks through symbolic verification and uses verifier signals alongside P1/P2/P3 rewards.
A frozen calibrated SFT checkpoint provides the reference policy for constraining policy drift during RLVR.
The clipped group-relative objective limits overly large policy updates while reference regularization constrains departure from the calibrated SFT policy.
Relative advantages favor candidates whose verifiable rewards exceed the group mean, while negligible-variance groups are skipped.

F. Gold-Free Self-Consistency and Final Output

At inference, the system aggregates one greedy and four sampled responses by task-aware answer equivalence. It then applies an optional conservative physics verifier before producing an answer with supporting and verification information.

Five responses are generated: one greedy response and four sampled responses.
Responses are grouped by task-aware answer equivalence, and the largest cluster is selected.
Ties favor the cluster containing the greedy response, then use a gold-free structural quality score.
A question-only physics verifier may conservatively override the selected answer, otherwise it abstains.
The final output can include the answer, units, supporting premises, symbolic or FOL information, reasoning trace, and verification confidence.

III. EXPERIMENTS

The experiments use a group-safe split of the EXACT logic and physics data, with leakage controls and no retrieval. The implementation combines staged QLoRA training with gold-free five-generation evaluation and selective physics verification.

Experimental data and setup: 2,162 examples are divided into 1,724 training and 438 held-out validation samples using a group-safe 80:20 split.
Experimental data and setup: Shared logic premises, duplicate normalized physics questions, and identical model inputs remain within the same partition to reduce leakage.
Experimental data and setup: Retrieval is disabled, and validation labels are accessed only after generation.
Implementation: The backbone uses Qwen2.5-3B-Instruct with 4-bit NF4 QLoRA across SFT, calibration, and RLVR stages.
Implementation: Evaluation generates one greedy and four sampled responses, with physics verification restricted to supported rules having confidence ≥0.98.

B. Quantitative Results

RLVR substantially increases explicit reasoning depth while leaving overall answer correctness approximately stable and slightly reducing supporting consistency. Task-level and task-aware analyses show that P2 reflects different diagnostics across logic and physics, while deeper reasoning does not guarantee validity.

P3 rises from 50.68% to 72.20%, a 21.52-point gain, making reasoning depth the dominant observed post-training change.P3 is a reasoning-depth proxy, not a proof of logical validity.
P1 changes from 56.62% to 55.94% (−0.68 points), so RLVR adds no held-out answer-accuracy gain.The small Macro-P1 change from 76.82% to 76.14% indicates the shift is not caused by collapse of one task family.
P2 decreases from 77.70% to 75.33% (−2.37 points), showing that richer reasoning does not automatically preserve premise or unit consistency.Overall P2 combines logic premise-selection F1 with physics unit consistency.
Task-Level Analysis: Physics P2 remains high at 89.23% for RLVR, whereas logic P2 is lower because it measures premise selection rather than unit consistency.The seven-example Logic Uncertain subset declines from 45.31% to 38.16% and should not be over-interpreted.
Task-Level Analysis: P1 changes only modestly across tasks: Logic Multiple Choice remains 91.78%, Logic Uncertain remains 100%, Logic Yes/No decreases by 2.38 points, and Physics changes from 40.51% to 40.15%.The global P1 decrease is distributed across limited task-level changes rather than broad loss of answer capability.
Task-Level Analysis: Every populated task family improves in P3: Logic Multiple Choice by 32.19 points, Logic Yes/No by 32.14, Logic Uncertain by 17.86, and Physics by 15.51.Physics P3 rises while P1 stays near 40%, showing that deeper traces do not suffice to solve numerical or formula-selection errors.

C. Ablation Study

Inference ablations show that self-consistency produces small P1 gains, while deterministic physics verification supplies the dominant final-answer correction. Training dynamics likewise concentrate useful optimization in nonuniform reward groups and show the strongest sustained change in P3.

Five-generation self-consistency increases overall P1 by 0.68 points for SFT and 1.37 points for RLVR.The ablation measures final-answer correction at inference and does not infer intermediate P2/P3 traces.
Physics verification improves P1 by 8.76 points for SFT and 9.12 points for RLVR relative to self-consistency.The verifier intervenes on 43 of 274 physics queries, correcting 26 SFT and 27 RLVR errors while introducing two regressions for each policy.
The physics verifier produces the dominant final correction, preventing its gain from being misattributed to RLVR itself.The policy changes generated reasoning, while the deterministic checker repairs a smaller subset of answers.
Of 240 rollout groups, 57 (23.75%) have zero reward variance and are skipped, leaving 183 informative groups and 30 optimizer updates.Zero-variance groups provide no relative ranking signal under the group-relative objective.
Rolling means rise from approximately 0.53 to 0.74 for P3, compared with 0.93 to 0.95 for P1 during RLVR training.The approximately 0.94 value refers to sampled training P1, not held-out P1 in Tables II–IV.

D. Comparison with Prior Reasoning Frameworks

The framework is compared methodologically rather than as a direct leaderboard, using P1, P2, and P3 to distinguish correctness, consistency, and reasoning depth. Results show a trade-off: RLVR strongly increases explicit reasoning while leaving answer accuracy near baseline, whereas symbolic verification supplies selective correctness gains.

Table VI is a methodological comparison because datasets, models, inference budgets, and metrics differ across approaches.
Reporting P1, P2, and P3 together reveals whether gains reflect answer correctness, supporting consistency, or explicit reasoning depth.
RLVR changes P1 by −0.68 points and P2 by −2.37 points but improves P3 by +21.52 points relative to calibrated SFT.
Self-consistency reduces single-generation variance, while deterministic physics verification selectively corrects supported high-confidence numerical cases.
Figure 2 displays task-level P1, P2, and P3, while Figure 3 separates inference-stage P1 correction from RLVR training dynamics.
SFT progresses 50.5→51.1→56.6%, while RLVR progresses 48.9→50.2→55.9% from greedy decoding through self-consistency to hybrid verification.

G. Inference Results

Inference combines five-generation gold-free self-consistency with optional symbolic verification, using the LLM for general reasoning and symbolic intervention for supported cases. Qualitative examples make premise use, formula selection, numerical substitution, units, and verifier status inspectable while showing why P1, P2, and P3 should remain separate.

The inference pipeline groups one greedy and four sampled responses by answer equivalence before optionally checking the selected physics response.
Logic examples expose premise use, whereas physics examples expose relations, numerical substitution, units, and verifier status.
P1 can be correct despite incomplete premise selection, P2 can remain high despite a numerical error, and P3 can rise with more explicit steps.
The intended system uses the LLM for general reasoning and reserves symbolic intervention for supported, verifiable cases.

IV. CONCLUSION AND FUTURE WORK

The paper concludes that verifier-guided QLoRA, symbolic routing, self-revision, and RLVR improve transparent educational QA by separating answer correctness, consistency, and reasoning depth. RLVR mainly strengthens explicit reasoning, while symbolic verification selectively corrects high-confidence numerical failures, but current evaluation and physics execution remain bounded.

The framework combines gold-anchored QLoRA, task-aware symbolic routing, self-revision, and group-relative RLVR for transparent educational QA.
On 438 held-out examples, RLVR raises P3 from 50.68% to 72.20%, while hybrid P1 remains close to calibrated SFT at 55.94% versus 56.62%.
RLVR mainly reshapes explicit reasoning, while symbolic verification selectively corrects high-confidence numerical failures.
The current RLVR objective improves reasoning depth more strongly than answer accuracy, and longer outputs can introduce premise, unit, or formatting inconsistencies.
Physics remains limited by formula selection, numerical execution, and scientific-notation normalization.
The evaluation protocol should test stability across random seeds and validation splits because the Logic Uncertain subset is small and physics dominates the held-out distribution.
Future work targets adaptive candidate counts, lower verifier-routing overhead, reward-weight sensitivity, calibration, uncertainty, and multi-candidate checking costs.