PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoReform: Reflective Formalization Evolution for Multimodal Geometry Problem Solving

Jialu Wang, Ruichen Zhang, Xiaoou Liu, Hua Wei, Tianlong Chen

arXiv:2610.12391v1cs.AI

TL;DR

MLLMs often struggle to identify and use geometric relations in diagrams, and explicit formalization can both correct and introduce errors. GeoReform evolves the formalization policy using failed reasoning rollouts. On Geometry3K, accuracy rises from 42.0% to 56.0%, while experiments across geometry reasoning benchmarks report consistent gains over the initial policy.

  • Problem

    MLLMs can struggle to extract and model geometric entities and relations, while adding explicit structure can correct some answers and turn others into errors.

    Source: §1 · p. 1

  • Method

    GeoReform executes the reasoning pipeline, diagnoses representation defects from failed rollouts, and evolves the policy for selecting, grounding, grouping, and presenting geometric information.

    Source: §1 · p. 2

  • Results

    GeoReform consistently improves over the initial policy across multiple geometry reasoning benchmarks; Geometry3K accuracy rises from 42.0% to 56.0%.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Effective formalization requires organizing geometric facts into representations that better support downstream reasoning, not merely extracting facts.

    Source: §1 · p. 2, §5 · p. 8

    The experiments mainly focus on plane geometry benchmarks, and GeoReform does not eliminate errors from diagram understanding or structure generation.

    Source: Limitations · pp. 8–9

Abstract

from arXiv · show

Multimodal large language models (MLLMs) often struggle to identify and use geometric relations in diagrams. Recent methods address this challenge by converting geometric entities, relations, and constraints into explicit textual representations for the model to reason over. However, effective formalization is highly non-trivial: on Geometry3K, structure injection fixes 28 errors but introduces 13 new ones among 200 examples. Redundant relations can distract the model, while ambiguous references to diagram elements can lead it to apply constraints incorrectly. This suggests that the key challenge is not merely extracting more geometric facts, but organizing them into representations that support downstream reasoning. To fully exploit the power of formalization, we further propose GeoReform, a reflective formalization evolution framework that treats formalization as an optimizable policy rather than a fixed parser output. GeoReform executes the full reasoning pipeline, collects failed rollouts, diagnoses defects in the current representation, and mutates the policy to better select, ground, group, and present geometric entities, relations, constraints, and targets. On Geometry3K, GeoReform improves Qwen3VL-2B accuracy from 42.0\% to 56.0\%. Extensive experiments and analyses across geometry reasoning benchmarks demonstrate that effective formalization is crucial for improving multimodal geometry reasoning.

1 Introduction

GeoReform treats formalization as an optimizable policy because explicit geometric structure can help reasoning but can also interfere when poorly selected or presented. On Geometry3K, evolving that policy raises accuracy from 42.0% to 56.0%.

1 Introduction

  • On 200 Geometry3K problems, structure injection corrected 28 previously incorrect answers but turned 13 previously correct answers into errors.The comparison uses Qwen3VL-8B and illustrates that adding explicit entities, relations, and constraints can both help and interfere.Source: §1 · p. 1
  • Redundant relations can distract reasoning, while unclear diagram grounding can lead the model to apply constraints to the wrong angles.Even valid relations can interfere when poorly selected or presented, so formalization must decide what to include and how to express it.Source: §1 · pp. 1–2
  • GeoReform executes the full reasoning pipeline, diagnoses representation defects from failed rollouts, and mutates the policy governing geometric information selection, grounding, grouping, and presentation.The framework aims for representations complete enough to support solutions while remaining concise enough to avoid distraction.Source: §1 · p. 2
  • 42.0% to 56.0%: evolving the formalization policy raises Geometry3K accuracy.The paper presents this as evidence that policy evolution mitigates failures and improves geometry problem solving.Source: §1 · p. 2

2 Related Work

Prior geometry problem-solving work spans formal representations, neural intermediate representations, symbolic deduction, and multimodal methods. GeoReform focuses on how geometric information is organized for downstream solving and evolves that intermediate representation using reasoning feedback.

2 Related Work

  • Early geometry systems formalize textual and visual inputs for symbolic reasoning with theorem knowledge, including Inter-GPS (Lu et al., 2021; Zhang et al., 2023a; Peng et al., 2023).Neural approaches also use geometric primitives, solution programs, and mathematical expressions (Zhang et al., 2022; Chen et al., 2021, 2022).Source: §2 · p. 2
  • AlphaGeometry combines neural search with symbolic deduction for geometry theorem proving.G-LLaVA and GeoX use geometry instruction data and formalized vision-language pretraining (Xia et al., 2025).Source: §2 · p. 2
  • AutoGPS, GOLD, and DFE-GPS incorporate structured diagram information into problem solving, where grounding, relation selection, or target specification errors can affect reasoning (Ping et al., 2025; Zhang and Moshfeghi, 2024; Zhang et al., 2024c).GeoReform studies how formalized information should be selected, grounded, and organized for problem solving.Source: §2 · pp. 2–3
  • Unlike prompt-evolution work that optimizes task instructions or reasoning prompts, GeoReform evolves the intermediate representation while keeping the downstream reasoning prompt fixed.Reasoning feedback is used to improve which geometric entities, relations, constraints, and targets are extracted and how they are organized and presented.Source: §2 · p. 3

3 Method

GeoReform makes formalization an optimizable interface between diagram understanding and downstream reasoning. It iteratively uses failed rollouts to revise the policy that organizes geometric information, while keeping the test set held out during evolution.

3.1 Method Overview

  • GeoReform treats formalization as an optimizable interface, refining how geometric information is selected, grounded, grouped, and presented for downstream reasoning.At inference, the evolved policy is fixed, and the reasoning model conditions on both the original problem and the structured representation.Source: §3.1 · p. 3

3.2 Formalization Policy Design

  • A formalization policy maps a diagram and problem statement to typed literals describing geometric entities, relations, numerical constraints, or target queries.The initial policy prohibits explanations, derivations, and extra text to keep the representation short and reduce noise.Source: §3.2 · pp. 3–4

3.3.1 Genetic Optimization Loop

  • The genetic loop selects a parent policy, collects failed cases from a training minibatch, generates a child by reflection, and retains candidates that pass minibatch and evaluation checks.The final test set is held out and is not used during evolution.Source: §3.3.1 · p. 4

3.3.2 Reflective Formalization Mutation

  • Reflective mutation compares formalizations with reasoning trajectories to diagnose policy defects, then revises only the policy while leaving the generator, reasoning model, and prompt fixed.Rollout traces help distinguish distracting verbosity from omitted necessary constraints; updates can clarify targets, group constraints, or restrain unsupported assumptions.Source: §3.3.2 · pp. 4–5

4 Experiment

Experiments evaluate formalization policies across three geometry benchmarks, reasoning models, and explicit-structure settings. GeoReform improves over the initial policy and competing formalizations, while analyses associate its gains with organizing relevant geometric information for downstream reasoning.

4.1 Experiment Setup

  • The evaluation uses Geometry3K, GeoQA, and PGPS9K, with Completion and Choice metrics and shared downstream prompts for formalization baselines.GeoQA uses GPT-5-extracted structured information because it lacks official formal geometric annotations; reflective evolution runs for at most 20 iterations.Source: §4.1 · pp. 5–6

4.2 Experiment Results

  • On Qwen3VL-2B, GeoReform raises choice accuracy from 42.0% to 56.0% and completion accuracy from 37.0% to 53.5%, improving over the initial policy.Source: §4.2 · pp. 6–7
  • Gains generally peak on 1B–2B models, reaching 17.5% on InternVL3.5-2B completion, while the largest models still improve by 4.5%–8.5%.Gains are not strictly monotonic with model size.Source: §4.2 · p. 7
  • On InternVL3.5, GeoReform performs best across evaluated settings, exceeding the best competing method by 10.0%–15.5% in choice and 7.5%–21.0% in completion.Table 2 reports Geometry3K accuracy for InternVL3.5 policies; some competing formalizations perform below the original MLLM baseline.Source: Table 2 · p. 6, §4.2 · p. 7

4.2.1 Ablation Study

  • Explicit structure improves six of eight GeoQA settings by up to 7.5%, while ground-truth PGPS9K annotations raise Qwen3VL choice accuracy by 5.5%–21.5%.Two GeoQA settings show small drops; the benefit depends on how well added structure complements the problem.Source: Table 3 · p. 6, §4.2.1 · p. 7

4.3 Case Analysis

  • GeoReform links local marks to global shape relations, reaching 100.0% accuracy on Rhombus versus InterGPS’s 76.9%, and 82.5% on Parallelogram versus 70.0%.These problem types depend strongly on global shape relations, not only local marks.Source: Figure 4 · p. 7, §4.3 · p. 8

5 Conclusion

The conclusion presents GeoReform as a framework for improving multimodal geometry problem solving by evolving formalization policies. Its central claim is that useful formalization depends on organizing geometric facts for downstream reasoning, not merely extracting them.

  • GeoReform optimizes how geometric entities, relations, constraints, and targets are selected, grounded, grouped, and presented using feedback from failed rollouts.Source: §5 · p. 8
  • The paper concludes that effective formalization organizes geometric facts into representations that better support downstream reasoning.Source: §5 · p. 8
  • Experiments and analyses across multiple geometry reasoning benchmarks show consistent improvement over the initial policy and strong performance across reasoning models.Source: §5 · p. 8

Limitations

The experiments mainly cover plane geometry, leaving broader mathematical reasoning settings underrepresented. Diagram-understanding and structure-generation errors remain possible, and evaluation lacks full significance analysis across repeated runs.

Limitations

  • Experiments mainly cover plane-geometry benchmarks, which do not fully represent solid geometry, algebraic diagrams, physics problems, or proof-oriented geometry tasks.The authors identify these broader settings as areas for future examination.Source: Limitations · pp. 8–9
  • GeoReform does not eliminate errors when models miss visual marks, misidentify entities, or incorrectly ground angles or segments.The authors suggest stronger visual parsers, symbolic consistency checks, or theorem-based verification as possible improvements.Source: Limitations · p. 9
  • Evaluation mainly reports accuracy without full statistical significance analysis across repeated runs, partly because repeated large-scale multimodal inference is computationally expensive.Source: Limitations · p. 9

A.1 Evolution Trace

The final policy lineage reaches 87.50% validation accuracy with P20, while the trace indicates that validation-based selection favors policies that generalize beyond sampled failures.

A.1 Evolution Trace

  • 87.50% validation accuracy is achieved by final policy P20, which adds named figure information while keeping property expansion restrained.Table 4 traces the final best path and omits rejected or neutral candidates.Source: Table 4 · p. 12, §A.1 · p. 12
  • P12 has lower training accuracy but higher validation accuracy than P3, suggesting that validation selection helps identify policies that generalize beyond sampled failures.Path denotes lineage depth rather than search order.Source: §A.1 · p. 12
  • The lineage shifts from P3's general formalization improvements to P12's explicit grounding of entities, angles, lines, and targets.Source: §A.1 · p. 12

A.2 Prompts and Policy Details

The appendix documents the initial representation interface, reflection and reasoning prompts, and final policy used at test time. It distinguishes the evolving formalization prompts and policies from the fixed downstream reasoning prompt.

A.2 Prompts and Policy Details

  • The initial policy defines the representation interface used to begin formalization-policy evolution.Source: §A.2 · p. 12
  • The appendix reports the prompts and policies that define the formalization and downstream-reasoning stages.Source: §A.2 · p. 12

A.2.4 Final Evolved Formalization Policy

Loading 2610.12391v1…