PaperlayerView exact revision on arXiv

Source-linked AI summary

Thinking with Drafting: Optical Decompression via Logical Reconstruction

Jingxuan Wei, Honghao He, Caijun Jia, Siyuan Li, Zheng Sun, Yuhang Xu, Yuanyuan Lin, Linzhuang Sun, Yuchen Wu, Bihui Yu, Xiangxiang Zhang, Cheng Tan

arXiv:2602.11731v2cs.CL

TL;DR

Multimodal systems can perceive visual symbols accurately but often fail to preserve the logical topology needed for precise reasoning. This paper reconceptualizes visual reasoning as optical decompression and introduces Thinking with Drafting, which uses a minimalist DSL to reconstruct and verify structure. On VisAlg, an 8B model with this scaffold outperforms leading proprietary frontiers, while the DSL remains scoped to bar-model visual algebra.

  • Problem

    The paper addresses the gap between high-fidelity visual transcription and reconstruction of rigorous logical topology in multimodal reasoning.

  • Method

    Thinking with Drafting reconstructs visual inputs into structured entities, relations, and constraints using a minimalist DSL with topological abstraction and deterministic rendering.

  • Results

    An 8B model equipped with TwD outperforms leading proprietary frontiers on VisAlg visual algebra problems.

  • Takeaways & Limitations

    TwD supports uniformly strong performance across visual algebra schemas, especially structure-intensive proportional-distribution and difference-analysis problems.

  • Takeaways & Limitations

    The DSL is intentionally scoped to bar-model visual algebra and linear topological relations, leaving broader scientific diagrams for future extension.

Abstract

from arXiv · show

Existing multimodal large language models have achieved high-fidelity visual perception and exploratory visual generation. However, a precision paradox persists in complex reasoning tasks: optical perception systems transcribe symbols without capturing logical topology, while pixel-based generative models produce visual artifacts lacking mathematical exactness. To bridge this gap, we propose that reasoning over visual inputs be reconceptualized as optical decompression-the process of reconstructing latent logical structures from compressed visual tokens. Guided by the axiom that Parsing is Reasoning, we introduce Thinking with Drafting (TwD), which utilizes a minimalist Domain-Specific Language (DSL) as a grounding intermediate representation. Unlike standard approaches that hallucinate answers directly, TwD forces the model to draft its mental model into executable code, rendering deterministic visual proofs for self-verification. To validate this, we present VisAlg, a visual algebra benchmark. Experiments demonstrate that TwD serve as a superior cognitive scaffold. Our work establishes a closed-loop system where visual generation acts not as a creative output but as a logical verifier, offering a generalizable path for visual reasoning.

1 Introduction

Modern multimodal systems can transcribe complex visual artifacts with high fidelity, yet transcription and pixel-based generation remain insufficient for preserving logical topology and mathematical exactness. Thinking with Drafting reframes visual reasoning as optical decompression, using a minimalist DSL to reconstruct and verify latent structure.

  • Motivation: MLLMs and OCR systems increasingly preserve dense text, layouts, tables, and formulas in high-fidelity visual representations.This progress enables machines to read complex visual documents accurately, but does not by itself establish their logical relations.
  • Motivation: OCR extracts symbols and numbers reliably, but their roles as totals, differences, or constraints remain context-dependent and unstructured.The resulting precision paradox separates perceptual fidelity from logical topology.
  • Optical Decompression: Optical decompression treats visual reasoning as reconstructing latent logical structures from compressed perceptual tokens.The proposed perspective shifts understanding toward explicit representations of entities, relations, and constraints.
  • Thinking with Drafting: Thinking with Drafting materializes this view through a minimalist geometric DSL tested on Singapore bar-model visual algebra.The DSL mediates between natural-language ambiguity, general-purpose code noise, and geometric axioms.
  • Thinking with Drafting: The DSL can compile into GeoGebra scripts or SVG code, allowing drafts to function as deterministic visual verifiers rather than mere visualizations.This supports mathematical validation and visual rendering within the proposed reasoning process.

2 Related Work

Prior work improves visual transcription and explores textual, executable, and pixel-space reasoning, but often leaves logical relations implicit or struggles to ground language constraints geometrically. Thinking with Drafting addresses this gap by converting perception and constraints into a verifiable intermediate DSL and visual reasoning loop.

  • Optical Perception: OCR and VLM approaches recover high-fidelity text while preserving layouts, tables, formulas, and other contextual structures.These systems demonstrate that pixels can efficiently encode textual information through contextual optical compression.
  • Optical Perception: Optical perception alone can capture visual syntax while leaving entities and quantitative relations implicit and ungrounded in mathematical problem solving.This motivates shifting from transcription accuracy toward logical reconstruction.
  • Optical Perception: Thinking with Drafting transforms raw perception into a verifiable intermediate representation of grounded logical structures.The approach explicitly bridges optical perception and reasoning rather than treating them as separate stages.
  • Reasoning Paradigms: Textual Chain-of-Thought and Program-of-Thought decompose reasoning, while vision-centric methods attempt to solve tasks directly in pixel space.These paradigms represent complementary strategies for reasoning after or within visual input.
  • Thinking with Drafting: Thinking with Drafting parses visual text into a rule-constrained canvas, decompressing implicit logical relations into explicit visual structures.This optical decompression loop targets semantic grounding between natural-language constraints and geometric artifacts.
  • Thinking with Drafting: The framework's verifier scores Logic Graphic DSL samples for syntactic validity, visual completeness, and logical consistency before retaining training data.The filtering process is intended to preserve topological and geometric correctness.

3 Method

Thinking with Drafting reconstructs visual logic in a structured DSL rather than relying on ambiguous text or imprecise pixel generation. The model drafts, renders, inspects, and refines this representation so the final answer derives from an explicit logical structure.

  • 3.1 Preliminaries: TwD maps visual input and queries into a structured DSL representation before deriving the final answer.The DSL space S represents geometric and logical constraints, replacing direct end-to-end answer generation.
  • 3.1 Preliminaries: Unlike text-only CoT, the DSL enforces explicit geometric rules that regularize reasoning.Natural language is described as ambiguous and lacking strict geometric constraints, whereas defined DSL entities must satisfy geometric rules.
  • 3.1 Preliminaries: Unlike image-based intermediate reasoning, TwD uses symbolic code whose rendered output is mathematically exact.Pixel-space generation can produce perceptually plausible but mathematically inaccurate diagrams; programmatic drafting supports reliable verification.
  • 3.2 The Logic Graphic DSL: The DSL exposes logical topology through entity, relational, and aggregation primitives.Entities encode quantities as horizontal segments, vertical lines encode equality through alignment, and braces encode part-whole, summation, or comparison relations.
  • 3.3 Topological Abstraction and Rendering: Topological abstraction maps the continuous canvas to a virtual discrete grid, letting the model reason about ordering and grouping instead of pixel offsets.The virtual grid uses logical rows and relative x-offsets to provide layout invariance.
  • 3.4 Thinking with Drafting: TwD forms an iterative generation-verification loop in which a DSL draft is rendered as a visual proof and then used as cognitive context for refinement.The model generates preliminary and refined explanations and DSL programs, with calculations guided by the explicit topology in the draft.

4 Dataset

VisAlg is a visual algebra benchmark built from bar-model problems paired with executable logical parses. Its construction combines drafting, refinement, expert-calibrated filtering, and structural evaluation across varied schemas, difficulty levels, and reasoning depths.

  • 4 Dataset: VisAlg pairs images of natural-language algebra problems with executable bar-model DSL representations of their ground-truth logical parses.The benchmark is designed to evaluate recovery of logical topology through optical decompression.
  • 4.1 Dataset Construction: 15,000 bar-model word problems were converted into synchronized textual analyses and DSL programs using Gemini-2.5-Pro.The source problems cover common visual algebra patterns, and each draft contains both schema analysis and DSL code.
  • 4.1 Dataset Construction: Drafts undergo syntax, analysis, and style checks before corrected instances are stored.The checks verify executability, consistency of instantiated objects and relations, and canonical bar-model layout conventions.
  • 4.1 Dataset Construction: 96% agreement between an expert-calibrated judge and expert evaluations supports filtering the full dataset to 11,372 product-ready instances.The judge retains only full-score instances after calibration on 1,000 examples.
  • 4.2 Dataset Analysis: The benchmark covers five canonical schemas, with proportional distribution and rate & percentage as dominant groups.The five schemas are proportional distribution, rate & percentage, change & revert, sum & split, and difference analysis.
  • 4.2 Dataset Analysis: The test set contains 942 instances and mirrors training distributions in schema and difficulty to emphasize structural generalization.Training contains 10,430 instances, with additional curated splits for fine-tuning, preference optimization, and evaluation.
  • 4.3 Evaluation: Evaluation combines code similarity, image similarity, LLM-judge dimensions, and human assessment of DSL quality.chrF is primary for code similarity, SSIM for image similarity, and the subjective score averages five verifier dimensions.

5 Experiment

Experiments evaluate VisAlg across multiple structural metrics, visual algebra schemas, human alignment, and set-theoretic reasoning. TwD achieves strong overall performance, structural fidelity, schema consistency, and verifier–expert agreement.

  • 5.2 Main Results: TwD achieves the highest overall VisAlg score of 82.63, exceeding Gemini-3-Pro at 79.96 and Gemini-2.5-Pro at 74.12.The evaluation spans code similarity, image similarity, and verifier-based evaluation.
  • 5.2 Main Results: TwD’s gains primarily arise from improved structural fidelity, with strong code and diagram alignment, information coverage, and semantic compliance.The remaining gap to top proprietary models is mainly numerical consistency, while structural legality and semantic completeness are largely preserved.
  • 5.3 Results by Visual Algebra Schema: Across five visual algebra schemas, TwD remains uniformly strong, with its largest gains on proportional distribution and difference analysis.These schemas require multi-segment decomposition and boundary-aligned comparison.
  • 5.4 Alignment with Human Expert: Verifier-based VisAlg scores correlate strongly with human expert ratings at r = 0.9575, while TwD remains top-ranked under both evaluations.This supports the verifier as a proxy for human judgment and indicates that the gains reflect structural correctness rather than metric artifacts.
  • 5.5 Generalize to Complex Logical Topology: TwD decomposes set-theoretic problems into sequential geometric operations and explicitly renders atomic intersections, whereas frontier models can violate boolean containment and alignment.The extended tasks involve high-order intersections and nested boolean boundaries.

6 Conclusion

The paper frames visual reasoning as reconstructing latent logical structures into verifiable artifacts and presents TwD as a structural drafting scaffold. A compact 8B model equipped with TwD outperforms leading proprietary frontiers on visual algebra problems.

  • 6 Conclusion: TwD addresses the precision paradox by enforcing structural invariants through a minimalist graphic DSL.The framework evaluates logical reconstruction through the VisAlg benchmark.
  • 6 Conclusion: A compact 8B model equipped with TwD outperforms leading proprietary frontier models on visual algebra problems.The conclusion characterizes explicit structural drafting as a foundation for trustworthy multimodal intelligence.

Limitations

The DSL’s structural representation is scoped to bar-model visual algebra and primarily supports linear topological relations. Extending it to broader scientific diagrams remains future work.

  • Limitations: The DSL is intentionally designed for bar-model visual algebra, emphasizing linear topological relations for structural supervision.Broader classes of scientific diagrams are outside the current representation scope.

A Additional Details for Dataset Construction

VisAlg is constructed through synchronized drafting, checklist-driven refinement, strict scoring, and human screening. Product-ready instances must satisfy numerical, informational, geometric, semantic, and format requirements.

  • Data Draft Generation: The initial drafting stage elicits structured problem analysis, diagram planning under bar-model constraints, and an executable DSL program.The synchronized draft establishes the logical and visual foundation for later refinement and verification.
  • Checklist-Driven Refinement: Checklist-driven refinement verifies each initial draft and applies minimal targeted corrections when the draft is not product-ready.The process uses the initial analysis and DSL generated in the preceding stage.
  • LLM-Based Verification: An LLM-based verifier performs strict criteria-based scoring, retaining only full-score instances as product-ready samples.The scoring stage follows checklist-driven revision.
  • Human Expert Screening: Human experts screen all refined instances under the same zero-tolerance philosophy as a final dataset-inclusion gate.Human evaluation supplements automated LLM-based verification.
  • Product-Ready Criteria: Accepted instances must satisfy numerical validity without arbitrary scaling or distortion.Bar-segment lengths must correspond to valid quantities in the correct solution process.
  • Product-Ready Criteria: The rendered diagram must contain enough visible information to solve the problem without consulting the original text.This criterion tests whether the artifact is self-sufficient.
  • Product-Ready Criteria: Brackets and alignment markers must coincide precisely with valid segment boundaries, while diagrams must preserve the stated object relationships.These requirements enforce geometric alignment and semantic fidelity.
  • Product-Ready Criteria: All constructions must adhere strictly to the prescribed DSL conventions for reduction, transfer, multiplicative relations, and alignment.Format compliance is one of the required acceptance criteria.

B.1 Evaluation Target and Boundary Conditions

The evaluation targets whether DSL outputs faithfully encode problem quantities and relationships so readers can reconstruct and solve the intended problem. It uses structured, checklist-based review with independent expert ratings and arbitration for substantial disagreements.

  • B.1 Evaluation Target and Boundary Conditions: Reviewers assess only whether the DSL expresses key quantities and relationships in a norm-compliant, non-leaking, structurally self-consistent form.The assessment excludes writing quality, fluency, and accompanying explanations, and reviewers may not add or modify information.
  • B.1 Evaluation Target and Boundary Conditions: The generation pipeline first analyzes the problem and produces an initial DSL draft, then verifies and conditionally refines that draft using a checklist.The prompts are identified as Step 1 data draft generation and checklist-driven refinement during VisAlg dataset construction.
  • B.1 Evaluation Target and Boundary Conditions: Three domain experts rate samples independently, with at least two reviewers per sample and third-reviewer arbitration for disagreements spanning two or more score levels.The protocol is designed to maximize objectivity and reproducibility.
  • B.1 Evaluation Target and Boundary Conditions: Each rating must include minimally sufficient evidence directly verifiable from the DSL, such as boundary misalignment, answer leakage, or unsatisfied arithmetic constraints.Examples include HB endpoints failing to meet segment boundaries and VLs failing to align with critical cross-row boundaries.

B.3 Evaluation Dimensions

DSL quality is evaluated across five dimensions covering geometry, information, arithmetic, semantics, and answer leakage. The resulting five-level score distinguishes fully compliant diagrams from outputs with fatal structural or leakage violations.

  • B.3 Evaluation Dimensions: DSL quality is characterized by structural alignment, information coverage, numerical consistency, semantic conformity, and answer leakage.These dimensions jointly determine whether the representation is usable and verifiable.
  • B.3 Evaluation Dimensions: Structural alignment requires horizontal-bracket endpoints and vertical-line coordinates to coincide with horizontal segment boundaries.This tests geometric legality and representational precision.
  • B.3 Evaluation Dimensions: Information coverage requires explicit textual representation of all key givens and the queried unknown; numeric segment lengths alone do not count.The criterion measures whether the intended problem structure can be recovered from diagram content.
  • B.3 Evaluation Dimensions: Semantic conformity requires task-specific encodings, including repeated equal-length units for multiplicative relations and paired −t/+t segments for transfers.It also constrains reduction, vertical links, and horizontal decomposition conventions.
  • B.3 Evaluation Dimensions: Answer leakage is a hard constraint: a final numeric answer appearing in quoted labels makes the output unacceptable.Leakage is treated as the most destructive violation because diagrams should encode structure rather than disclose solutions.
  • B.3 Evaluation Dimensions: The overall score ranges from 5 for complete, aligned, consistent, non-leaking outputs to 1 for fatal violations such as leakage or fundamentally invalid relations.Intermediate scores represent varying degrees of imperfections, ambiguity, and derivation risk.
  • B.3 Evaluation Dimensions: TwD case studies convert textual constraints into spatially aligned DSL structures across alignment-, decomposition-, and state-transition-centric schemas.Examples include repeated units for proportionality, complementary segments for sum-and-split, fractional unit grounding, and paired transfer segments.

D Additional Error Analysis

Baseline diagrams can remain arithmetically compatible while losing structural invariants needed for verification. The error analysis identifies semantic erasure, label–structure mismatch, and incompatible global boundaries as distinct degeneration patterns.

  • D Additional Error Analysis: Baseline outputs may remain arithmetically compatible while losing the structural invariants required for verification.This motivates analyzing errors beyond whether the final arithmetic is numerically plausible.
  • D.1 Semantic Erasure: Multiplicative Topology Collapsed: Semantic erasure collapses a ×3 multiplicative constraint into an additive difference layout, removing repeated-unit evidence even when arithmetic is correct.The multiplier is no longer visually provable because the repeated-unit structure has disappeared.
  • D.2 Label Injection: Numbers without Geometric Support: Label injection writes a computed difference as text without allocating a corresponding geometric sub-segment.This creates claims without geometric evidence and can cause downstream reasoning to treat labels as quantities.
  • D.3 Alignment Conflict: Incompatible Global Boundaries: Alignment conflict mixes incompatible boundary cues, so dashed completion and vertical guides encode different endpoints for the same relation.The resulting global inconsistency prevents the “less by 35” relation from being stably represented.

E Potential Risks

TwD’s structured drafting can make visual algebra constraints explicit across several schema types, but formalized representations introduce educational risks. The paper highlights automation bias and cognitive offloading as boundaries on how such diagrams should be used.

  • E Potential Risks: A structured DSL may amplify automation bias because a well-formed representation can be mistaken for a correct solution.The paper distinguishes structural validity from semantic correctness in educational use.
  • E Potential Risks: Automated drafting may cause cognitive offloading that weakens learners’ independent ability to translate textual constraints into spatial representations.The stated risk is skill atrophy in diagrammatic reasoning through reduced engagement with structural invariants.
  • C Case Studies on Visual Algebra Schemas: TwD represents proportional distribution through repeated equal-length units, boundary alignment, and an explicit unknown on the composed total.The multiplicative relation is enforced topologically rather than merely calculated.
  • C Case Studies on Visual Algebra Schemas: For change and revert, paired decrease/increase segments encode the transfer before the post-transfer multiplicative constraint is imposed on the aligned after-state.This supports reverse deduction while keeping computed answers from being leaked in labels.
  • C Case Studies on Visual Algebra Schemas: Rate and percentage schemas ground fractional change by fixing the base as unit “1” and attaching the increment as an aligned subsegment.This separates the increment from the whole and reduces unit confusion.
  • C Case Studies on Visual Algebra Schemas: Sum-and-split schemas isolate known parts and mark the complementary unknown, supporting completion by subtraction or addition.The figures use explicit whole–part structure to identify the target remainder or final total.
  • C Case Studies on Visual Algebra Schemas: Difference analysis encodes comparative relations as offset segments anchored across entities, turning “more than” and “fewer than” into aligned subtraction layouts.Vertical dashed lines act as spatial anchors for the relative positions of reference and derived entities.
Loading 2602.11731v2…