Route-Verify-Vote: Procedure-Conditioned Self-Consistency for Mixed-Domain Reasoning

1 Introduction

• SCoRE 2026 tests whether models can transfer familiar operations and their ordering to three unseen mixed domains while selecting the exact subset of correct options.Training contains four atomic domains and only one mixed domain.Source: §1 · p. 1
• Preliminary pruning, fine-tuning, and reinforcement-learning experiments did not yield consistent official-test improvements, so the study kept the model frozen and adapted inference procedure and compute budget.Source: §1 · p. 1
• Generic chain-of-thought prompting does not enforce the representation, operator order, or complete exact-set decision this setting requires.Source: §1 · p. 2
• Runs vary on 179–248 of 1,000 questions despite nearly constant parse coverage, motivating aggregation over complete answer sets rather than reliance on one trace.Source: §1 · p. 2
• Procedure-conditioned self-consistency gives every sample a shared fixed natural-language procedure and uses agreement among parseable complete answer sets as a compute-allocation signal.The procedure specifies a state representation, operators, inference order, consistency check, and output format.Source: §1 · p. 2
• 79.4% accuracy; the final system ranked second among participating systems.Source: §1 · p. 2

2 Task and Evaluation

• An answer is correct only when every selected and omitted option matches the gold set, making the metric exact set match.Source: §2 · p. 2
• Three mixed-domain labels in the released data occur in the test set but not in training.Source: Table 1 · p. 2
• 848 of 3,600 training questions have at least two correct options, so multiple-correct-option cases are substantial rather than exceptional.Source: §2 · p. 2
• The task requires domain-suited constraint representations, ordered composition of mixed-domain reasoning, and avoiding both premature stopping and over-selection.Source: §2 · p. 2

Prompted reasoning

• Prior prompting methods elicit intermediate steps or decompose problems, while this system selects a domain-tailored state representation and verification procedure with explicit operation order.Source: §3 · p. 3

Reasoning-path aggregation

• Self-consistency aggregates sampled reasoning paths at the answer level; this work applies that principle to complete exact-set outputs generated under a shared route-specific prompt.The resulting vote distribution also serves as a compute-allocation signal.Source: §3 · p. 3
• RVV combines answer-set margin with procedure conditioning in a frozen-model inference loop.Source: §3 · p. 3

Compositional generalization

• Prior work reports declining accuracy as test compounds diverge from training compounds, while SCoRE places three mixed-domain labels only in the test set.Source: §3 · p. 3

4 Procedure-Conditioned Self-Consistency Framework

• RVV routes each domain through a shared route–verify–vote control flow, while route templates specify distinct constraint states and core operators.The eight routes share a full-constraint recheck, option-wise judgments, and structured output.Source: §4 · p. 3, Table 2 · p. 4

4.1 Route: Select a Procedure

• Each fixed route prompt specifies a state representation, constraint operators, their inference order, a consistency check, and a structured output format.Routing selects the prompt deterministically from the provided domain label; the prompts specify procedure through natural-language instructions.Source: §4.1 · p. 3

4.2 Construct a Route-Specific Constraint State

• Domain-specific states include taxonomic and attribute evidence, social relation graphs, spatial coordinates or seats, and temporal variables with equations or modular offsets.Mixed routes compose these primitives in an explicit order; Space+Social resolves relational descriptions before substituting entities into a coordinate-based constraint system.Source: §4.2 · p. 4

4.3 Verify: Predict a Complete Answer Set

• Within each generation, Verify judges all four options separately against the derived state, rechecks constraints, and serializes true options into a complete answer set.The parser extracts the final answer list.Source: §4.3 · p. 4

4.4 Vote: Aggregate Answer Sets

• Vote aggregates parseable, non-empty answer sets and continues sampling while the top-two vote-count margin is below τ; the final system also pairs models on three selected routes.Adaptive sampling uses τ = 12 at 32-sample increments; pairing merges the REAP and Qwen3-30B-A3B-Thinking-2507 answer-set pools after adaptive REAP sampling sets the per-question budget.Source: §4.4 · p. 5

5.1 Experimental Setup

• 79.4% accuracy is reported for the final system; adaptive REAP-only RVV reaches 77.3%, and model pairing adds 2.1 points.Table 3 summarizes official-test exact-set accuracy at each inference stage.Source: Table 3 · p. 5, §5.1 · p. 5

5.2 Comparison of Domain-Specific Configurations

• Test-size-weighted accuracy rose from 35.9% to 56.0%; the largest domain difference is 44.0 points on Space+Social.Domain-specific scores were non-monotonic: Space+Social fell to 35.0% before reaching 80.5%, while Time+Social fell from 45.0% to 34.5%.Source: §5.2 · p. 5, Table 4 · p. 6

5.3 Voting, Adaptive Allocation, and Final Result

• The pooled 16-run vote reaches 74.6%, versus a 70.67±0.81% mean and 68.9–72.3% range across scored runs.The vote distribution is reused to allocate subsequent computation; adaptive RVV adds REAP samples in blocks of 32 while m < 12, up to 512, and reaches 77.3%.Source: §5.3 · p. 6

5.4 Agreement Structure

• Across 16 outputs, 457 questions are unanimous; pairwise answer distance ranges from 179 to 248 questions, with a mean of 214.3.Ninety questions have margin at most two. Adding 16 samples to the 201 questions with margin at most six changes 33 answers, including 29 in the margin-0–2 bins.Source: §5.4 · p. 6
• Each run yields parseable non-empty sets for 986–991 questions, so parse failure alone cannot explain the variation; six questions remain unresolved after pooling.Source: §5.4 · p. 6

• The system selects a domain-specific procedure before sampling, then allocates additional samples based on answer-set disagreement.Source: §6 · p. 6
• Uniform voting adds only 0.3 points from K = 128 to K = 512, favoring targeted allocation here.Source: §6 · p. 6
• Adaptive RVV reaches 77.3%; route-conditioned model pairing adds 2.1 points, yielding 79.4% accuracy and second place.Given domain labels and fixed prompts, complete-set disagreement offers a useful signal for allocating test-time computation.Source: §6 · p. 6

Limitations

• Routing assumes provided domain labels; leaderboard feedback informed prompts and pairing, so the official score evaluates the final system rather than isolated components.Source: Limitations · p. 6
• The stopping threshold of 12 depends on the sampling schedule and does not maintain a constant confidence level across budgets.A normalized stopping criterion may be more efficient.Source: Limitations · p. 6

Appendix

• The appendix covers implementation, early train–test comparisons, answer-set cardinality, and additional analyses of models, prompts, decoding, scaling, and sampling variability and allocation.Source: Appendix · p. 7

• The final configuration uses temperature 0.85, top-p 1.0, min-p 0.05, disabled top-k, a 40,000-token context, and at most 36,000 generated tokens.Source: A Implementation Details · p. 7
• Self-hosted REAP experiments used approximately 280 H800 GPU-hours; compute for externally accessed models is excluded because provider-side GPU time is unavailable.The environment used Python 3.12.3, PyTorch 2.10.0 with CUDA 12.8, Transformers 5.6.0, and vLLM 0.19.1 on NVIDIA H800 PCIe GPUs.Source: A Implementation Details · p. 7
• The system starts with 32 REAP samples and adds blocks of 32 below margin 12, up to 512; Nature, Space, and Space+Social then receive equal Qwen3-30B-A3B-Thinking-2507 samples before the pools are merged.Other routes remain REAP-only.Source: A Implementation Details · p. 7

A.1 Early Train–Test Comparisons

• Training-subset accuracy was consistently higher, making those evaluations poor guides to official-test difficulty; the compared prompts and sampling budgets differed.The authors characterize the comparisons as preliminary.Source: §A.1 · p. 7, Table 5 · p. 7
• Preliminary dense 9B pruning showed no accuracy or speed advantage, while fine-tuning and reinforcement learning showed no consistent gains.Incomplete records limit these observations to a qualitative summary.Source: §A.1 · p. 7

A.2 Answer-Set Cardinality

• The 3,600-question training set includes two-option and higher-cardinality targets, making exact-set verification relevant beyond a small exceptional subset.Source: §A.2 · p. 7, Table 6 · p. 7

Model comparisons

• On the same 50-question protocol, the selected REAP checkpoint matched Qwen3-30B-A3B-Thinking-2507 while running faster; the newer REAP variant scored lower on accuracy, parse rate, and completion stability.The model comparison used P2, one sample, and fixed decoding settings.Source: §B.1 · p. 7, Table 7 · p. 8

Prompt comparisons

• With N = 50, P2 exceeded P5 by four points despite nearly identical mean output lengths, and P3 beat P1 by four percentage points with shorter outputs.Because prompts differ in several instructions, the comparison does not isolate a single prompting mechanism.Source: §B.1 · p. 8, Table 8 · p. 8
• Successive prompt variants did not improve accuracy monotonically, and the table reports representative comparisons from separate official domain evaluations.Variants can differ in multiple instructions.Source: §B.1 · p. 8, Table 9 · p. 8

Domain-specific configurations

• All selected domain-specific configurations scored above their initial counterparts, with the largest accuracy difference on Space+Social.Source: Figure 2 · p. 8, §B.1 · p. 8

Sampling-parameter sensitivity

• Temperature 0.9 and min-p 0.05 achieved 32/100 on the official Space subset; truncation analysis reached 100% relative vote accuracy at 16k generated tokens.The authors selected moderate sampling parameters and a higher output-token limit; top-k filtering was disabled.Source: §B.1 · pp. 8–9, Figure 3 · p. 9

B.2 Test-Set Scaling

• 77.3% accuracy from margin-triggered adaptive sampling rose to 79.4% with route-conditioned model pairing in the final system.Uniform voting improves monotonically with sample budget, but gains from each doubling decrease.Source: §B.2 · p. 9, Figure 4 · p. 9
• Figure 5 compares uniform scaling, margin-12 adaptive allocation, and route-conditioned model pairing on 40% of questions using sampled solutions as a budget proxy.The proxy does not measure tokens or GPU time; marginal gains from doubling uniform sampling decrease.Source: Figure 5 · p. 10

B.3 Run Variability and Allocation

• Under a common procedure and decoding setup, accuracy still varies; agreement with the full 16-run modal answer offers a clearer diagnostic than output length or parse coverage.Figure 7 examines how often additional samples change the modal answer by initial vote margin.Source: §B.3 · p. 10, Figure 6 · p. 11, Figure 7 · p. 11
• Among the 201 questions selected at margin ≤6, 33 modal answers changed after 16 additional samples; low-margin questions brought both more corrections and greater risk of damage.Source: Figure 7 · p. 11