PaperlayerView exact revision on arXiv

Source-linked AI summary

Reactivating Test-Time Scaling for Plane Geometry Problem Solving

Xiaoqiang Kang, Shengen Wu, Maizhen Ning, Xiaobo Jin, Kaizhu Huang, Yutao Yue, Xiaowei Huang, Qiufeng Wang

arXiv:2608.30156v1cs.CL

TL;DR

Plane geometry problem solving remains difficult because test-time scaling transfers poorly under rigid symbolic programs and insufficient explicit visual grounding. The paper introduces Multi-Trace Synthesis, Perception-Augmented training, and Consensus-Guided Multi-Trace Ensemble inference. Across three benchmarks, the method consistently improves geometry reasoning, while CG-MTE preserves high-budget self-consistency gains with up to 8× lower sampling cost.

  • Problem

    Test-time scaling transfers poorly to plane geometry because rigid symbolic programs limit reasoning diversity and visual errors can corrupt symbolic deductions.

  • Method

    The paper synthesizes heterogeneous reasoning traces, trains models to parse diagrams into semantic clauses before deduction, and adaptively allocates ensemble samples using cross-trace consensus.

  • Results

    Across PGPS9K, Geometry3K, and GeoQA, the method consistently improves geometry reasoning, while CG-MTE reduces sampling cost by up to 8× while preserving most high-budget self-consistency gains.

  • Takeaways & Limitations

    Diverse traces and explicit perception grounding support effective test-time scaling for plane geometry, with adaptive sampling improving the accuracy–cost trade-off.

  • Takeaways & Limitations

    The framework depends on structured solution programs and semantic clauses, is evaluated only on 2D plane geometry, and cannot verify the semantic faithfulness of all natural-language rationales.

Abstract

from arXiv · show

Plane geometry problem (PGP) solving has become a critical benchmark for multimodal reasoning because it requires accurate visual perception and precise multi-step symbolic deduction. Although test-time scaling (TTS) has demonstrated remarkable success in general mathematical reasoning, it fails to scale effectively under the symbolic-program paradigm for plane geometry. We identify two key obstacles: limited reasoning diversity induced by rigid symbolic programs and insufficient explicit visual grounding before symbolic deduction. To address these issues, we propose Multi-Trace Synthesis (MTS), which converts each symbolic program into heterogeneous reasoning traces, including executable Python scripts and CoT-augmented variants. We further propose Perception-Augmented (PA) training, which parses diagrams into structured semantic clauses before deduction, and Consensus-Guided Multi-Trace Ensemble (CG-MTE) for efficient self-adaptive inference. Experiments on three geometry benchmarks show that our method consistently improves PGP-solving across model scales and achieves strong performance against both general-purpose MLLMs and specialized geometry solvers. Under test-time scaling, CG-MTE achieves comparable accuracy to high-budget self-consistency while reducing sampling cost by up to 8x. Code and data are publicly available at https://github.com/Jason8Kang/ReTTS-PGPS.

1 Introduction

Plane geometry is a demanding multimodal reasoning benchmark where test-time scaling is weakened by limited symbolic-program diversity and visual-to-symbolic errors. The paper addresses these obstacles with heterogeneous reasoning traces, perception-augmented training, and adaptive ensemble inference, improving accuracy–compute trade-offs.

  • PGP solving requires accurate visual recognition of entities and relations together with precise multi-step deduction.
  • Self-consistency provides limited benefits for PGP because sampled trajectories often fail to reach a majority consensus.
  • Rigid symbolic programs constrain reasoning diversity, while perception errors such as reading 101° as 104° propagate into incorrect deductions.
  • MTS expands each symbolic program into semantically aligned Python scripts and CoT-augmented traces to generate diverse reasoning pathways.
  • PA training parses diagrams into structured semantic clauses before reasoning, strengthening visual-to-symbolic alignment for subsequent deduction.
  • CG-MTE checks cross-trace consensus at shallow decoding depths and increases sampling only when disagreement persists.
  • 76.0% accuracy is achieved by CG-MTE with an average sampling number of 4.89, compared with 76.7% for 40-sample self-consistency with beam search.

2 Related Work

Prior PGP research spans symbolic, neural, multimodal, and program-aided solvers, while data synthesis increasingly targets annotation quality and reasoning-trace diversity. This work distinguishes itself by combining diverse reasoning traces with adaptive test-time scaling.

  • Early neural-symbolic solvers parse diagrams and text into formal languages for interpretable and verifiable symbolic deduction.
  • Neural and MLLM-based methods improve geometry reasoning through visual-language pre-training and logical reasoning fine-tuning.
  • Program-aided reasoning uses executable programs, typically Python, to support symbolic computation in problem solving.
  • Unlike solver-centric approaches, this work focuses on diverse reasoning traces and adaptive test-time scaling for PGP solving.
  • PGP data synthesis addresses limited high-quality annotations, but prior approaches face limits in reasoning-trace diversity or natural-language alignment.

3 Methodology

The methodology represents geometry problems with semantic clauses and symbolic solution programs, then expands programs into verified executable and rationale-enriched traces. Perception-augmented training separates diagram parsing from reasoning, while multi-trace ensembles use consensus-guided depth expansion to balance accuracy and sampling cost.

  • 3.1 Problem Formulation: Each geometry instance comprises a diagram, question, semantic clauses, solution program, and numeric answer.The model predicts semantic clauses from the diagram and question before generating a reasoning trajectory leading to the answer.
  • 3.1 Problem Formulation: Solution programs encode rigorous geometric deductions as ordered steps applying predefined operators to variables and constants.For example, Gougu applies the Pythagorean theorem to known side lengths to calculate a hypotenuse.
  • 3.2 Multi-Trace Synthesis: MTS expands instantiated programs into executable Python scripts and CoT-augmented program and PAL variants, creating heterogeneous reasoning traces.PAL scripts are produced through rule-based translation and execution-based verification; CoT variants add natural-language rationales.
  • 3.2 Multi-Trace Synthesis: PAL scripts are verified by sandbox execution and must return an answer matching ground truth within relative tolerance ϵ = 0.001.CoT-PAL undergoes the same verification loop, while CoT-Program is parsed into an executable program during inference.
  • 3.3 Perception-Augmented Training: PA training first parses diagrams into semantic clauses and then generates executable reasoning traces conditioned on those clauses.The joint objective combines perception and reasoning losses, with structured visual evidence supplying the basis for deduction.
  • 3.4 Multi-Trace Ensemble: CG-MTE progressively pools top-d answers across trace types and expands depth only when no unique consensus emerges.Unlike Standard MTE, which always samples V D candidates, CG-MTE can terminate early when consensus is reached at shallow depths; it remains an empirical stopping rule rather than a correctness certificate.

4 Experiments

The experiments evaluate MTS-based PA training across three geometry benchmarks, model scales, and comparison systems. Results show consistent gains and strong benchmark performance.

  • Evaluation setup: The evaluation covers PGPS9K, Geometry3K, and GeoQA using Qwen-VL backbones and comparisons with general-purpose and specialized geometry solvers.MTS-All contains approximately 32.1K, 33.7K, and 13.9K training instances for the three datasets, respectively.
  • Main findings: PA training on MTS-All yields PGPS9K absolute gains of 7.4%, 11.6%, and 12.6% for the 2B, 3B, and 8B backbones.The improvement is also reported on Geometry3K and GeoQA.
  • Main findings: Qwen3-VL-8B achieves 71.2%, 74.5%, and 67.2% accuracy on PGPS9K, Geometry3K, and GeoQA, respectively.It surpasses Qwen2.5-VL-72B by 17.9%, 24.0%, and 11.7% on those benchmarks.
  • Main findings: Qwen3-VL-8B achieves the best results among compared specialized geometry solvers on PGPS9K and Geometry3K.It surpasses LANS by 4.5% and 2.4%, respectively.

5 Analysis

Ablations attribute performance gains to explicit perception grounding and heterogeneous reasoning traces, while scaling analyses show that CG-MTE improves accuracy–compute efficiency through adaptive sampling.

  • Ablation studies: Removing PA training causes absolute accuracy drops of 5.3%, 6.5%, and 5.8% for the 2B, 3B, and 8B backbones.The ablation supports explicit semantic parsing before symbolic deduction.
  • Ablation studies: Removing synthesized traces causes approximately 4% degradation under data-size-matched control, while merging all traces achieves the best performance.The results suggest complementary reasoning strategies improve generalization.
  • Ablation studies: CoT-PAL reaches 47.2%, 55.4%, and 68.3% accuracy on the 2B, 3B, and 8B backbones, respectively, outperforming other single-trace types.Its human-verified reasoning correctness is 98.5%, surpassing CoT-Program by 5.5%.
  • Ablation studies: Removing any individual trace type decreases performance, with CoT-PAL removal producing the largest degradation.CoT-augmented traces do not fully subsume their non-CoT counterparts.
  • Test-time scaling: PGPS9K-All models gain consistently from larger self-consistency budgets, whereas PGPS9K-only models show negligible improvement or degradation.For Qwen3-VL-8B, beam search increases accuracy from 71.2% at top-1 to 76.7% at top-40.
  • Adaptive inference: CG-MTE reaches its performance plateau earlier than Standard MTE and beam-search self-consistency, improving the accuracy–compute trade-off.Its efficiency is attributed to early cross-trace agreement and adaptive expansion when disagreement persists.

6 Conclusion

The paper targets limited reasoning diversity and perception-induced symbolic errors in plane geometry test-time scaling with MTS and PA training. CG-MTE preserves most high-budget self-consistency gains while reducing sampling cost by up to 8×.

  • Conclusion: The paper identifies limited reasoning diversity and perception-induced symbolic errors as obstacles to effective test-time scaling for plane geometry.These obstacles arise under the symbolic-program paradigm.
  • Conclusion: MTS expands symbolic programs into heterogeneous reasoning traces, while PA training grounds deduction in structured semantic clauses parsed from diagrams.The method is evaluated on PGPS9K, Geometry3K, and GeoQA.
  • Conclusion: CG-MTE preserves most gains of high-budget self-consistency while reducing sampling cost by up to 8×.It is presented as a self-adaptive inference strategy.

Limitations

The framework's limitations concern annotation dependence, restricted evaluation scope, and imperfect verification of natural-language rationales. Generalization beyond 2D plane geometry remains unverified.

  • The method depends on structured annotations: formal solution programs for MTS and semantic clauses for PA training.This limits direct applicability to datasets without such annotations.
  • Evaluation covers PGPS9K, Geometry3K, and GeoQA, but remains confined to 2D plane geometry.
  • Generalization to 3D geometry, physics, or other multimodal reasoning tasks remains unverified.
  • Execution verifies that CoT-PAL scripts run and return expected answers, but not that every rationale is semantically faithful.CoT-augmented traces may therefore contain rationales misaligned with their formal reasoning steps.

Ethics Statement

The study uses public geometry benchmarks and reports no private or sensitive personal data. Data releases follow source-benchmark licensing terms, and internal human evaluation was voluntary.

  • The work uses publicly accessible benchmarks, including PGPS9K, Geometry3K, and GeoQA, without private user data or sensitive personal information.
  • Derived MTS data and related artifacts comply with the licenses and redistribution terms of the source benchmarks.
  • Human evaluation was conducted by geometry-trained undergraduate research-group members, with voluntary participation.

A Method Details

The method-details section describes the geometry language, conversion and verification of multi-trace training data, and prompt-based rationale generation. The resulting corpora combine executable and natural-language reasoning formats across three benchmarks.

  • Language specification: The domain language uses 34 geometric theorems and axioms, with operands drawn from problem variables, process variables, and semantic-clause measurements.The operator set covers operations involving triangles, quadrilaterals, polygons, and circles.
  • Program conversion: Four of 8,021 solution programs fail PAL verification because they violate geometric constraints and are excluded from PAL-based branches.Their original Program and CoT-Program traces are retained.
  • Program conversion: Figure 6 shows filtered training samples excluded for geometric contradictions such as negative segment lengths or degenerate intersection angles.
  • Program conversion: The conversion yields 8,017 verified PAL scripts and CoT-PAL traces, while CoT-Program traces cover all 8,021 instances.Together, the synthesized formats form the 24K-trace PGPS9K-MTS mixture, combined with the original dataset as PGPS9K-All.
  • Cross-benchmark corpora: MTS is applied to Geometry3K and GeoQA, producing GEOMETRY3K-MTS and GEOQA-MTS; dataset and corpus statistics are summarized in Table 6.The table distinguishes synthesized MTS instances from totals including original training data.
  • Trace generation: MLLM-based rewriting enriches executable scripts with natural-language rationales through prompt templates for transformation, repair, and program rewriting.CoT-PAL repair uses traceback feedback in an iterative self-correction loop, while program rewriting aligns explanations with symbolic operators.

B.1 Training Details

Training uses Hugging Face TRL with DeepSpeed ZeRO-2 on eight A100 GPUs and fine-tunes three vision-language backbones. The reported setup uses fixed epoch, schedule, context-length, and decoding configurations.

  • Infrastructure: The pipeline uses Hugging Face SFTTrainer with DeepSpeed ZeRO-2 on 8 NVIDIA A100 80GB GPUs.
  • Models: Three backbones are fine-tuned: Qwen2-VL-2B-Instruct, Qwen2.5-VL-3B-Instruct, and Qwen3-VL-8B-Instruct.
  • Optimization: Training runs for 10 epochs with cosine learning-rate scheduling, 10% warmup, and a maximum sequence length of 1,024 tokens.
  • Decoding: Main results use greedy decoding with do_sample=false and num_beams=1 unless otherwise stated.Test-time scaling experiments use deterministic settings described in the same implementation details.

C.1 Additional Results on InternVL3.5 Backbones

InternVL3.5 experiments show that PA training consistently improves symbolic-program prediction across 2B and 8B backbones, while self-consistency further benefits test-time scaling. Qualitative and training analyses attribute these gains to stronger visual grounding and more learnable Python-based traces.

  • InternVL3.5 Results: +8.1 and +8.3 points improve PA training over direct symbolic-program prediction on InternVL3.5-2B and 8B, respectively.Removing either PA training or PGPS9K-MTS causes clear degradation across model sizes.
  • Test-Time Scaling: 44.8% to 49.9% and 48.5% to 54.9% are the InternVL3.5-2B and 8B gains under SC@40, respectively.Self-consistency remains effective as the inference budget increases.
  • Qualitative Analysis: PA training predicts semantic clauses before deduction, providing a more reliable visual-to-symbolic interface.The analysis links this interface to fewer invalid theorem applications and hallucinated geometric steps.
  • Trace Training Dynamics: Python-based PAL and CoT-PAL traces converge faster and reach lower training loss than program-based traces for Qwen3-VL-8B.The paper suggests their similarity to pre-training code-generation tasks makes them easier to learn, with CoT-PAL providing a more learnable target.
  • Cross-Benchmark Scaling: 71.2% to 77.2%, 74.5% to 80.7%, and 67.2% to 76.1% are the self-consistency gains on PGPS9K, Geometry3K, and GeoQA.The reported improvements span all three geometry benchmarks, with the largest absolute gain on GeoQA.

D.3 Comparative Analysis of Decoding Strategies

Beam search benefits from a changing mixture of reasoning traces and broader answer exploration, whereas temperature sampling remains concentrated. CG-MTE exploits shallow cross-trace consensus to reduce computation, but correlated errors prevent consensus from guaranteeing correctness.

  • Reasoning-Trace Distributions: At k = 4, executable PAL and CoT-PAL traces comprise 77% of beam-search predictions.At k = 40, CoT-Program and CoT-PAL together exceed 85%, while temperature sampling retains a nearly static program-heavy distribution.
  • Answer Exploration: Beam search generates broader answer diversity than temperature sampling, which tends to concentrate top-40 predictions on one numerical answer.This broader exploration is presented as part of the explanation for beam search's stronger majority-vote performance.
  • Adaptive Termination: More than 80% of problems reach CG-MTE consensus at d = 1 across model scales, accounting for an 8× reduction in sampling overhead.CG-MTE checks shallow cross-trace consensus and expands sampling only when disagreement persists.
  • Error Coupling: 34.3%–39.8% pairwise same-wrong agreement reveals non-negligible error correlation across trace formats.At least three traces agree on the same wrong answer in 8.7–14.9% of instances, so consensus is an early-stopping signal rather than a correctness guarantee.
Loading 2608.30156v1…