PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Where Outcomes Change:Credit-Addressable Reasoning for Multimodal Geometry

Jiani Guo, Junjie Wang, Jie Wu, Pengxiang Zhao, Dongdong Zhang, Shaohan Huang, Yujiu Yang, Furu Wei

arXiv:2608.30457v1cs.LGcs.CL

TL;DR

Multimodal geometry reasoning must preserve precise visual relations through multi-step deduction, but free-form traces obscure decisive choices and trajectory-level learning spreads one terminal signal across the response. The paper introduces credit-addressable reasoning through Code-CoT and CE-GRPO, achieving 76.04 average accuracy across nine geometry benchmarks and larger relative gains on longer reasoning chains.

  • Problem

    Free-form multimodal geometry traces hide answer-determining visual decisions, while trajectory-level reinforcement learning assigns one outcome signal to the whole response.

  • Method

    Code-CoT retains the diagram and organizes geometric reasoning as line-addressable executable typed events, while CE-GRPO compares complete shared-prefix continuations and localizes credit from outcome differences.

  • Results

    76.04 average accuracy across nine geometry benchmarks exceeds Qwen3-VL-8B by 8.09 points and trajectory-level GRPO by 3.43 points.

  • Takeaways & Limitations

    The results support aligning reasoning representations with credit assignment for long, dependency-heavy multimodal reasoning.

Abstract

from arXiv · show

Multimodal geometry reasoning requires VLMs to extract precise visual relations and preserve them through multi-step deduction. Existing free-form traces obscure the decisions that determine the answer, and trajectory-level reinforcement learning distributes a single terminal signal across the entire response. We introduce credit-addressable reasoning, in which the semantic units exposed during inference also define where learning compares alternatives and assigns credit. We instantiate this principle with Code-CoT, which retains the diagram, represents visual relations as line-addressable executable code, and organizes reasoning into typed events, and CE-GRPO, which selects event boundaries using structural priors and type-normalized entropy, samples complete continuations from shared prefixes, and converts outcome differences into localized advantages. Across nine geometry benchmarks, CE-GRPO achieves an average accuracy of 76.04, outperforming Qwen3-VL-8B and trajectory-level GRPO by $8.09$ and 3.43 points, respectively. Its relative advantage increases with the number of intermediate events, demonstrating the value of representation--optimization co-design for long, dependency-heavy multimodal reasoning.

1 Introduction

Geometry exposes a representation gap because decisive visual choices remain implicit in free-form traces, alongside a credit gap from trajectory-level rewards. The paper addresses both by making reasoning events explicit and assigning localized credit, with gains across nine geometry benchmarks.

  • Motivation: Geometry reasoning is brittle because errors in object binding, angle interpretation, or auxiliary construction can invalidate later deductions.Free-form traces leave these decisive visual choices implicit, while trajectory-level optimization supplies one outcome signal for the whole response.
  • Core idea: Credit-addressable reasoning uses the same semantic units for inference-time decisions, alternative comparison, and credit assignment.This directly targets the missing connection between exposed reasoning structure and optimization.
  • Method: Code-CoT retains the diagram, represents geometric operations as line-addressable executable code, and organizes reasoning into reference, auxiliary, coordinate, and think events.Each event is both a checkable geometric operation and an addressable unit of credit.
  • Method: CE-GRPO selects candidate event boundaries and compares complete continuations from shared prefixes to localize terminal credit without process annotations or an auxiliary value model.Structural event boundaries provide branch points, while reward differences reveal which candidates produce useful credit signals.
  • Results: 76.04 average accuracy across nine geometry benchmarks exceeds Qwen3-VL-8B by 8.09 points and trajectory-level GRPO by 3.43 points.The method improves over the backbone on all nine benchmarks and retains a 3.91-point gain over trajectory-level GRPO on validly terminated responses.

2 Related Work

Prior work either exposes reasoning structure without making it an optimization unit or localizes updates using learned, fixed, or token-level signals. The paper instead combines executable multimodal representations with event-boundary branching and outcome-based local credit.

  • Executable representations: Program-aided and geometry systems use executable programs, tools, formal relations, or code to represent multimodal reasoning.These approaches establish structured intermediate representations but do not necessarily make them optimization units.
  • Fine-grained optimization: Fine-grained reinforcement-learning methods assign credit to intermediate steps, high-entropy tokens, visual perturbations, or auxiliary rewards.Their update locations or signals are derived from learned values, fixed boundaries, token statistics, or additional rewards.
  • Positioning: Code-CoT integrates the original diagram with line-addressable executable events, while CE-GRPO branches at those event boundaries and derives local credit from terminal outcome differences.The approach uses type-normalized entropy for computation allocation without process labels, value models, or task-specific rewards.

3 Motivation: Code as a Reasoning Space

A controlled MathVerse study tests code as a shared reasoning space and finds that code works best alongside the diagram. These findings motivate persistent, line-addressable code that the model explicitly learns and organizes into reasoning events.

  • Controlled study: Across three open and three proprietary models, combined diagram and externally generated code outperforms either modality alone.For Qwen2.5-VL-7B, combining modalities reduces the Text-Dominant–Vision-Only gap from 13.3 to 3.0 points.
  • Controlled study: Code is primarily compensatory because its gain generally decreases as diagram-only accuracy increases.The controlled study evaluates diagram-only, external-code-only, combined, and self-generated-code-only conditions.
  • Design implications: The design requires code to remain alongside the diagram, be produced by the reasoning model, and be explicitly learned rather than merely prompted.Code-CoT therefore treats code as a persistent, line-addressable reasoning space rather than an external tool or fixed intermediate artifact.
  • Design implications: Code-CoT expresses visual retrieval, geometric construction, coordinate setup, and deduction as explicit events.These events provide the semantic units used to structure the reasoning process.

4 Method

Code-CoT makes multimodal geometry reasoning explicit through retained diagrams, executable perception code, and typed events. CE-GRPO then selects event boundaries and localizes credit by comparing complete continuations from shared prefixes.

  • Code-CoT: Code-CoT generates line-addressable executable perception code, a solution plan, typed events, and a final answer from each image–question pair.The event types are think, reference, auxiliary, and coordinate.
  • Code-CoT: The model retains the original diagram while using executable code as a persistent reasoning space for retrieving visual facts and extending the geometry.At inference, code, reasoning events, and the answer are generated in one response without an external solver or test-time branching.
  • Policy optimization: The reward combines structural validity, answer correctness, action validity, repetition penalties, and answer-leakage penalties without a learned reward model.The action-validity coefficient is λ = 0.3, and standard GRPO otherwise applies one advantage across every token in a response.
  • Candidate-event selection: CE-GRPO combines structural priors with type-normalized event entropy to select semantically meaningful candidate events without an auxiliary segmenter or step detector.Criticality is revealed only when sibling continuations from the same prefix receive different terminal rewards.
  • Shared-prefix branching: Shared-prefix branching fixes the image, question, and pre-event prefix, then samples complete continuations whose rewards produce state-conditioned advantages.The shared prefix is excluded from the policy loss, so updates affect only the regenerated event and its downstream consequences.

5 Experiments

The experiments evaluate CE-GRPO across nine geometry benchmarks and compare it with native, prompting, supervised, general post-training, and two-stage baselines. Ablations show that structural selection and entropy together improve accuracy and response validity.

  • Experimental setup: The evaluation spans nine benchmarks covering visual grounding, plane geometry, auxiliary construction, and process-level multimodal reasoning.The benchmark categories include MathVerse, VisOnlyQA-Syn, VisOnlyQA-Real, MathVista-GPS, Geometry3K, PGPS9K, GeoQA, GeoLaux-mini, and MM-Math.
  • Experimental setup: The baselines include native Qwen3-VL-8B-Instruct, prompting and SFT variants, DPO, PPO, DAPO, trajectory-level GRPO, and other fine-grained methods.All in-house post-training methods use a shared checkpoint, with LoRA for DPO and GPO and full-parameter updates for the remaining methods.
  • Main results: 76.04 average accuracy is the best overall result, exceeding the backbone, Code-CoT SFT, trajectory-level GRPO, and the strongest fine-grained baseline by 8.09, 6.49, 3.43, and 6.73 points, respectively.CE-GRPO also surpasses the backbone on all nine benchmarks.
  • Main results: CE-GRPO gains 15.16 points on GeoLaux-mini and 9.44 on MM-Math over trajectory-level GRPO, while gains are smaller or mixed on visual-grounding and standard plane-geometry tasks.The largest improvements occur where intermediate constructions and decisions affect multiple later steps.
  • Ablation study: 76.04 is achieved by combining structure and entropy, while structure alone reaches 74.26 and reduces the unclosed-response rate from 12.31% to 7.07%.The combined selector reaches the lowest unclosed rate of 4.73% and is best or tied on seven of nine benchmarks.

6 Discussion

The discussion links Code-CoT’s improved diagram-to-code fidelity and CE-GRPO’s event selection to stronger solution quality, especially for longer reasoning trajectories. Results across benchmarks and controlled analyses support localized credit assignment as the main advantage.

  • Representation fidelity: 80.43% macro recall and 99.0% render success show that CE-GRPO improves executable diagram representations beyond Code-CoT SFT.On 100 MathVerse-TD problems, macro recall rises from 55.21% with the baseline to 70.16% after SFT and 80.43% after CE-GRPO.
  • Solution quality: 3.91 points is CE-GRPO’s mean gain over trajectory-level GRPO on validly terminated outputs across nine benchmarks.The largest gains are +12.53 on GeoLaux-mini, +8.42 on MM-Math, and +5.07 on PGPS9K.
  • Modality dependence: 15.10 points on MathVerse vision-only inputs reduces the TD–VO gap from 30.07 to 14.09.The concentration of gains on vision-dependent variants is consistent with better preservation and reuse of diagram-derived relations.
  • Scaling with reasoning length: 3.77 points per additional intermediate event quantifies the widening CE-GRPO margin over trajectory-level GRPO.The relationship remains significant under leave-one-benchmark-out tests, with r = 0.866 and exact p = 0.0016.

7 Conclusion

The conclusion presents credit-addressable reasoning as a representation–optimization principle for multimodal geometry. Code-CoT exposes persistent, typed, executable events, while CE-GRPO uses shared-prefix continuations to localize outcome-based credit.

  • 7 Conclusion: Credit-addressable reasoning makes inference-time semantic units the locations where optimization compares alternatives and assigns credit.The principle is intended for long, dependency-heavy reasoning with delayed outcome supervision.
  • 7 Conclusion: Code-CoT retains the original diagram and represents geometric relations as line-addressable executable code organized into typed events.The protocol includes perception code, a plan, reasoning and action events, and a final answer.
  • 7 Conclusion: CE-GRPO branches complete continuations from shared event prefixes and converts terminal outcome variation into event-conditioned advantages.Candidate events are regenerated together with their downstream consequences rather than evaluated only as isolated tokens.
  • 7 Conclusion: Inference generates the perception code, reasoning events, and final answer in one response without an external solver or test-time branching.The perception block remains unchanged throughout the trace and supports later grounded actions.
  • 7 Conclusion: The protocol uses reference, auxiliary, and coordinate actions to ground visual facts and construct or transform working geometry.Reference actions expose perception lines, while auxiliary and coordinate actions extend or reframe the working figure.

A.2 Training Data Construction

The training-data pipeline constructs validated structured traces and shared-prefix reinforcement-learning examples for Code-CoT and CE-GRPO. It combines curated supervision, programmatic rewards, deterministic event parsing, and semantically guided candidate selection.

  • A.2 Training Data Construction: 18,302 retained SFT examples cover eight geometry datasets after removing question- and image-level overlaps with evaluation sets.The data span synthetic diagrams, formal geometry, auxiliary constructions, and multimodal mathematical reasoning.
  • A.2 Training Data Construction: Code-CoT traces are synthesized from line-numbered executable Matplotlib diagrams, questions, interleaved reasoning and action events, and final answers.The reference answer is used only for silent consistency checking during synthesis.
  • A.2 Training Data Construction: Trace filtering requires valid structure, executable code, grounded references, valid typed actions, and a correct final answer.This ensures SFT supervision contains both a valid protocol format and an executable reasoning trajectory.
  • A.2 Training Data Construction: Programmatic reward combines answer correctness, action-validity rate, and penalties for duplicated actions, repetitive generation, and answer leakage.The reward can reach 1.3 when the answer is correct, all actions are valid, and no behavioral penalty is triggered.
  • A.2 Training Data Construction: Candidate selection parses complete typed events, standardizes entropy within event type, and combines structural and entropy-based candidates.The earliest complete think event and the highest-entropy think or action events provide semantically meaningful branch points.
  • A.2 Training Data Construction: 3,270 unique shared prefixes produce 11,450 counterfactual-prefix rows mixed 1:1 with 11,450 ordinary RL problems.Shared-prefix groups regenerate the selected event and its suffix; zero reward variance yields no policy update.

A.5 Training and Inference Procedure

Training initializes Code-CoT from Qwen3-VL-8B-Instruct with supervised learning, then applies reinforcement learning using ordinary and shared-prefix examples under the same policy instruction. The training procedure differs in prompt construction rather than the underlying policy objective.

  • A.5 Training and Inference Procedure: Code-CoT is initialized from Qwen3-VL-8B-Instruct and trained on 18,302 traces for three epochs.The visual encoder is frozen while the multimodal aligner and language model are updated.
  • A.5 Training and Inference Procedure: SFT trains the model to generate the complete Code-CoT response, including perception code, the plan, typed reasoning events, and the final answer.The maximum sequence length is 16,384, with batch size 64 and learning rate 1 × 10^-5.
  • A.5 Training and Inference Procedure: Ordinary and shared-prefix reinforcement-learning examples use separate prompt forms but the same policy instruction.They differ by whether an assistant prefix is supplied.

B.1 Benchmarks and Evaluation Protocol

Evaluation covers nine geometry benchmarks and follows their official fixed-test-set protocols. Results use end-to-end runs, with the overall score computed as the unweighted mean across benchmarks.

  • Benchmarks: Nine geometry benchmarks span visual grounding, plane geometry, auxiliary construction, and process-level multimodal reasoning.Question- and image-level overlaps with training data are removed.
  • Evaluation Protocol: The evaluation reports accuracy on each benchmark’s fixed test set under its official evaluation protocol.The protocol follows corresponding official implementations wherever available.
  • Evaluation Protocol: The overall score is the unweighted mean across the nine benchmarks.
  • Evaluation Protocol: For Code-CoT models, judges evaluate the extracted final <answer> block, while other baselines are judged on their complete responses.Disagreements are reassessed until both judges agree.

B.2 Baseline Implementations

The comparison includes diverse training and inference baselines adapted to a common multimodal geometry setting. In-house post-training methods share the Code-CoT SFT initialization and response protocol, while external checkpoints retain their released configurations.

  • Baseline Implementations: CE-GRPO is compared with training-free, supervised, general post-training, critical-event, counterfactual, external-checkpoint, and two-stage baselines.In-house post-training baselines initialize from the same Code-CoT SFT checkpoint and retain its response protocol.
  • Baseline Implementations: LoRA updates are used by DPO and GPO, whereas the remaining post-training baselines use full-parameter updates.
  • Baseline Implementations: External models are evaluated with their officially released prompt templates, answer formats, and decoding configurations without Code-CoT adaptation.
  • Baseline Implementations: Two-stage systems pair Qwen3-VL-8B with first-stage models that convert diagrams into released intermediate representations before downstream problem solving.These systems require an additional model.
  • Qualitative Comparison: Figure 5 contrasts trajectory-level GRPO with CE-GRPO on an incorrect isosceles-triangle solution, showing branching at the structural decision preceding the error.Valid midpoint or Stewart’s-theorem continuations receive positive rewards, while unsupported assumptions are suppressed.

C Additional Results and Analysis

Additional analyses show how CE-GRPO localizes credit around structural errors and selects outcome-sensitive events. Structural selection improves critical-event identification, while type-normalized entropy improves reward variation and generation efficiency.

  • C.1 Qualitative Examples: Trajectory-level GRPO assigns one advantage to the complete response, so unsupported structural assumptions cannot be separated from dependent deductions.
  • C.1 Qualitative Examples: CE-GRPO samples complete continuations from a shared prefix, allowing valid midpoint or Stewart’s-theorem branches to recover AD = 3.
  • C.1 Qualitative Examples: Branches assuming AD ⊥BC or treating AD as an angle bisector produce incorrect or contradictory outcomes and are suppressed by localized policy updates.
  • C.2 Offline Selector Validation: 30.2% relative improvement: structural selection raises Crit.@2 from 0.222 to 0.289 on the offline selector evaluation.Crit.@2 measures traces containing at least one critical event among two selections.
  • C.2 Offline Selector Validation: Type-normalized entropy achieves |∆R| = 0.409 and reduces Tok./critical from 85.0k to 75.0k while retaining Crit.@2 = 0.287.These results support using structure to choose branch points and entropy to prioritize informative candidates.
Loading 2608.30457v1…