Source-linked AI summary
From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning
Weichen Dai, Rafael Medeiros Cabral, Ziyi Shou, Yan Cao, Xin Shen, Dongcai Lu, Yi Zhou
TL;DR
Plane geometry requires reliable integration of diagram perception and logical deduction, while existing systems face limitations in interpretability, computation, or benchmark difficulty. The paper augments a pure LLM with a Geometric Vision Parser, Symbolic Solver, and novel ZhongkaoGeo benchmark, achieving state-of-the-art performance comparable to or surpassing leading LMMs on challenging geometry problems.
Problem
Plane geometry remains difficult because systems must combine visual understanding with rigorous deduction, while existing benchmarks can be saturated and contaminated.
Method
The framework converts diagrams into symbolic representations, filters theorem-based deductions, and supplies the resulting facts to an LLM for solution synthesis.
Results
The method achieves state-of-the-art performance across three ZhongkaoGeo datasets and can match or surpass advanced proprietary LMMs.
Takeaways & Limitations
Specialized geometric perception and deduction modules can make pure-LLM geometry solving more transparent and human-like while retaining strong performance.
Takeaways & Limitations
The framework must account for schematic diagrams whose visual information can be redundant or misleading, and related formal systems remain incomplete for inequality and optimization problems.
Abstract
from arXiv · showhide
Plane geometry remains a significant challenge in AI, requiring the integration of visual perception and mathematical reasoning. While Large Multimodal Models (LMMs) naturally handle visuo-linguistic inputs, they are often computationally intensive and opaque. We demonstrate that a pure Large Language Model (LLM), when equipped with specialized modules, can rival state-of-the-art LMMs on complex geometry problems. Our framework integrates a Geometric Vision Parser, which translates diagrams into symbolic form, with a Symbolic Solver that performs formal deductions, thereby mitigating hallucinations and promoting interpretable reasoning. To enable rigorous evaluation, we curate a benchmark of challenging problems from the 2025 Chinese Zhongkao examinations, ensuring data novelty and testing deeper deductive skills. Experiments demonstrate that our approach achieves performance comparable to Gemini 2.5 Pro while delivering clearer, human-like solutions.
Introduction
The paper frames plane geometry as a demanding integration of visual perception and logical deduction, then proposes specialized modules that let an LLM process diagrams and produce more interpretable solutions. It also introduces a novel benchmark intended to test complex, contamination-resistant reasoning.
- Plane geometry requires parsing diagrams and text, performing logical deductions, and sometimes constructing auxiliary lines.
- Formal systems offer precision but can lose information during translation, produce non-human-readable outputs, and fail on inequality or optimization problems.
- LMMs process visuo-linguistic inputs naturally but remain computationally expensive, opaque, and vulnerable to geometric distribution shifts.
- The framework equips an LLM with a Geometric Vision Parser that converts diagram images into structured symbolic representations.
- A Symbolic Solver performs targeted deduction on angular relationships, guiding the model away from hallucinations and brittle coordinate-based reasoning.
- The 2025 Chinese Zhongkao benchmark targets complex multi-step deductions and auxiliary constructions while reducing the risk of training-data contamination.
Related Work
Prior geometry-solving research spans symbolic systems, end-to-end neural models, and neuro-symbolic hybrids, with recent work increasingly using LLMs as backend reasoners. These approaches trade off formal rigor, learned visual understanding, search efficiency, and specialized reasoning support.
- Symbolic systems prioritize logical soundness and interpretability by translating natural-language problems and diagrams into formal representations.
- End-to-end neural methods encode textual and visual problem information into learned representations for direct solution prediction.
- Neuro-symbolic methods combine neural theorem prediction with symbolic reasoning to reduce the cost of traditional search.
- Recent LLM-based systems use large language models as backend solvers while adding specialized representations, datasets, search algorithms, or reinforcement learning.
Method
The method converts diagram and text information into structured geometric facts, aligns visual evidence with textual constraints, and applies filtered theorem deductions before asking an LLM to synthesize the final solution. Its design addresses diagram ambiguity, angle-related hallucinations, and combinatorial theorem-search growth.
- Geometric Problem Formalization: The parser extracts geometric primitives, symbols, annotations, and higher-order relationships from diagram images.
- Geometric Problem Formalization: Symbols are assigned to compatible primitives using Euclidean distance subject to a geometric feasibility constraint.
- Geometric Problem Formalization: Textual alignment filters schematic or misleading visual information and adds properties implied by textual descriptions, such as rectangle constraints.
- Geometric Theorem Reasoning: The Theorem Reasoning Module expands literal relations, deduces new facts from a theorem library, and filters conclusions relevant to the original problem.
- Formal Definitions of Reasoning Steps: Constraint-based theorem matching prunes invalid arguments, reducing branching from O(N^k) to O(N_valid) where N_valid ≪ N.
- Solution Synthesis: The final prompt combines module outputs so the LLM can generate human-readable solutions with deductive rigor and explanatory clarity.
ZhongKao Geometry Benchmark
The benchmark addresses saturation, limited complexity, and contamination risks in existing plane-geometry datasets through a four-principle, three-tier suite sourced from recent Zhongkao examinations.
- The benchmark is designed around quality, difficulty, diversity, and contamination prevention.
- Existing GeoQA benchmarks offer limited difficulty, with average reasoning steps of 1.96 and 2.23 and maxima of 4 and 8, respectively.
- ZhongkaoGeo-L1, L2, and L3 increase difficulty while reducing contamination risk through examination sources from 2023 and earlier, 2024 to early 2025, and official 2025 exams.
- The final benchmark contains 89 L1 problems, 83 L2 problems, and 105 L3 problems after removing unclear, mismatched, or ambiguous items.
- Unlike traditional benchmarks, many problems contain multiple related sub-questions requiring multi-step reasoning.
Experiments
Experiments evaluate the method across three increasingly difficult datasets using strict accuracy for L1/L2 and rubric-based scoring rate for L3, with strong results in both reasoning and diagram parsing.
- The method is evaluated on ZhongkaoGeo-L1, L2, and L3 using Deepseek-R1 and comparisons with Qwen, DeepSeek-R1, GPT-o1, and Gemini 2.5-Pro.
- Performance on ZhongkaoGeo-L1 & L2: 92.13% L1 accuracy and 74.30% L2 accuracy are achieved in standard configuration, rising to 93.26% and 78.31% with Major@3.
- Performance on ZhongkaoGeo-L3: 88.4% mean scoring rate exceeds Gemini 2.5-Pro’s 87.2% on ZhongkaoGeo-L3.
- Performance on ZhongkaoGeo-L3: The method’s L3 performance remains stable across regional exams, with variance comparable to Gemini 2.5-Pro and consistent tracking or surpassing of its performance.
- Analysis of Geometric Parsing Performance: Holistic parsing scores reach 81 versus Qwen-VL-2.5-72B’s 23 for correctly identifying all geometric relations.
- Analysis of Geometric Parsing Performance: Geometric Theorem Reasoning guides readable solutions instead of brute-force coordinate methods.
Conclusion
The paper argues that a pure LLM augmented with geometric perception and symbolic deduction can match or surpass advanced LMMs while supporting transparent, human-like reasoning.
- A pure LLM with a Geometric Vision Parser and Symbolic Solver achieves and can surpass the performance of Gemini 2.5 Pro.
- The contamination-free ZhongkaoGeo benchmark provides a rigorous standard for evaluating deductive capabilities.
- The framework promotes human-like deductive reasoning over opaque, brute-force methods to advance transparent and interpretable mathematical problem-solving.
A. Prompt Design
The prompt design instructs the model to reconstruct diagrams from textual and symbolic information before solving each sub-question with condition-aware, structured explanations.
- Diagram reconstruction: The prompt first reconstructs the diagram from the problem text, using spatial relationships and pixel coordinates only as reconstruction references when necessary.
- Diagram reconstruction: Problem text takes precedence when diagram information conflicts with the description.
- Sub-question solving: For each sub-question, the model identifies shared and specific conditions and checks whether earlier conclusions remain applicable.
- Diagram information: The diagram parser describes concyclicity, collinearity, and angle relationships in a structured format.
- Output format: The normal prompt requests problem analysis, step-by-step solution steps, and a final answer.
B. Supplementary Tables, Figures, and Case Study
The supplementary materials present higher-order predicate examples, suspected Gemini data contamination, and contrasting case studies of reasoning with and without the proposed method.
- The supplementary materials include examples of higher-order predicates, suspected data contamination in Gemini, and a case study of the proposed method.
- Gemini-2.5-Pro appears to correct erroneous diagram-parsing information despite not being provided the original image.The highlighted corrections are presented as suspected data-contamination cases on the Mathvista test dataset.
- Without the proposed method, DeepSeek R1 receives parsed visual information but uses complex coordinate geometry, makes errors, and ultimately guesses incorrectly.
- With the proposed method, the model is guided toward a simple, correct solution based on angle relationships rather than a less-interpretable coordinate approach.