VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning

1 INTRODUCTION

• Final-render feedback does not expose which proposed edits survived execution, were overwritten, or interfered with shared scene resources.Additional iterations and logging alone do not reveal how observed values should become alternative programs.Source: §1 · p. 1
• VETR retains aligned executable records and compiles observed writes and state changes into alternative programs checked against runtime endpoints and preservation conditions.Repair construction uses no additional language-model calls; the ordinary visual verifier assesses goal relevance.Source: §1 · pp. 1–2
• VETR is the strongest complete system on both benchmarks and both backbones, with paired comparisons confirming gains are not completion-rate artifacts.The evaluation covers BlenderGym and BlenderBench; reported ablations also attribute gains to execution refinement and role asymmetry and execution history.Source: §1 · p. 2
• The framework treats executable programs as world representations for editing geometry, materials, lighting, and spatial relationships.Its propose–execute–observe–verify cycle uses execution-trace evidence for bounded program updates under preservation constraints.Source: §1 · p. 2

2.1 EXECUTABLE WORLD REPRESENTATION

• VETR executes each submitted complete program from the base scene and retains its program, render, captured state, and bounded write trace as an aligned record.The observer covers supported assignments, while imported-library writes and unresolved aliases do not provide owner-qualified events.Source: §2.1 · p. 3

2.2 EXECUTION REFINER

• The refiner derives recovery candidates from displaced observed writes and state changes, then compiles supported values with compatible resource bindings into constrained alternatives.Recovery uses current and initial traces as separate hypotheses; light projection instead modifies the previous delivered program.Source: §2.2 · p. 4
• Fresh execution checks candidate endpoints and preservation of captured properties and structure, but properties outside the captured numerical cells are not covered by that contract.If no alternative passes, VETR retains a successful raw result or otherwise reuses the exact preceding record; runtime acceptance does not guarantee lower image error.Source: §2.2 · pp. 4–5

2.3 ROLE-ASYMMETRIC EVIDENCE ORGANIZATION

• The refiner privately consumes writes, captured endpoints, and bindings, while the generator and visual verifier receive distinct bounded contexts.The generator retains actions and delivered outcomes; the verifier receives the current delivered comparison and goal-directed critique.Source: §2.3 · p. 5

3.1 EXPERIMENT SETUP

• The evaluation covers 280 BlenderGym editing tasks and 27 BlenderBench tasks, compares One-Shot, BlenderAlchemy, VIGA, and VETR, and reports N-CLIP and PL, where lower is better.The GPT and Qwen runs use Blender 3.6.0; VIGA and VETR allow ten generator interactions.Source: §3.1 · p. 5

3.2 BLENDERGYM

• Against One-Shot, VETR lowers N-CLIP/PL by 52.7%/58.8% on GPT and 28.1%/49.6% on Qwen; on Qwen, it also improves over VIGA by 40.0%/56.3%.Source: §3.2 · p. 6
• On complementary image metrics, GPT VETR improves significantly over One-Shot on all five measures; Qwen VETR improves significantly over VIGA on all five at N = 4.For Qwen, only SSIM, LPIPS, and CLIP meet the criterion against One-Shot; no extended metric establishes an advantage of N = 4 over N = 1.Source: §3.2 · p. 6, Table 2 · p. 7

3.2 BLENDERGYM; 3.3 BLENDERBENCH

• VETR leads on both benchmarks, beating baselines across BlenderGym’s aggregate metrics and nearly all families, and winning every BlenderBench level–metric cell on both backbones.The BlenderGym result is reported in Table 1; the BlenderBench result is reported by level in Table 3.Source: §3.2 · p. 6, §3.3 · p. 6, Table 1 · p. 7

3.3 BLENDERBENCH

• Against VIGA on BlenderBench, VETR reduces N-CLIP/PL by 55.0%/67.7% on GPT and 27.6%/48.6% on Qwen.Source: §3.3 · p. 6, Table 3 · p. 7
• In three selected final-output comparisons, VETR has lower PL than VIGA on light, geometry, and shape tasks; the examples do not identify which repair produced the difference.Source: §3.3 · p. 7, Figure 3 · p. 8

3.4 ABLATION STUDY

• Disabling refinement worsens 11 of 12 level–metric cells; on 27 GPT tasks, refinement lowers mean PL by 41.8% and mean N-CLIP by 32.2%.The paired PL reduction is statistically significant; its bootstrap interval excludes zero and the sign test gives p = 4.9 × 10^-5.Source: §3.4 · pp. 8–9, Table 4 · p. 9
• On GPT BlenderGym tasks, limiting history raises N-CLIP/PL by 48.5%/79.0%; removing role asymmetry raises them by 12.7%/23.7%, and removing the verifier by 6.6%/28.2%.Each context variant is paired with the default on the same 280 tasks.Source: §3.4 · p. 9, Table 4 · p. 9

4 DISCUSSION AND CONCLUSION

• VETR treats executions as evidence for revising scene programs, validating realization and preservation before visual verification assesses goal relevance.The discussion describes aligned records, observed writes and state transitions, and updates chosen from supported values and compatible bindings.Source: §4 · p. 9
• The refiner uses a fixed set of Blender-derived write types; extension to another domain requires additional write grammars and explicit preservation boundaries.The authors identify learned preservation models as a possible future direction.Source: §4 · p. 9

A RELATED WORK

• Iterative editors such as VIGA and BlenderAlchemy use rendered feedback, while VETR adds trace-derived alternatives (Yin et al., 2026).The related-work discussion says existing editing systems lack mechanisms to extract alternatives from execution traces.Source: A RELATED WORK · p. 14
• Unlike SemFix and bidirectional evaluation, which target internal values, VETR derives candidates from witnessed imperative executions against an image objective without additional model inference.Source: A RELATED WORK · p. 14
• VETR’s runtime checks instantiate local state reasoning as finite checks over captured properties.Source: A RELATED WORK · p. 14

B.1 COMPILING TRACE EVIDENCE INTO REALIZED EDITS

• Recovery uses displaced writes from current or initial traces as candidate values, applying them to the current complete program rather than rewinding it to the initial program.Displaced values are alternatives, not inferred intent or proof that a later assignment was erroneous.Source: §B.1 · p. 15
• Compilation combines supported values with compatible resource bindings, splitting shared ownership only when requirements conflict.The effective light-value map is non-unique, so bounds, feasible-solution ordering, and allowed copying or rebinding must come from the compiler.Source: §B.1 · p. 15
• Projection uses execution-derived endpoints and compiles into the previous delivered program, whereas recovery compiles into the raw program.Candidates run afresh from the base scene; their preservation checks use the designated reference, raw for recovery and previous delivered for projection.Source: §B.1 · p. 15
• Runtime acceptance checks successful execution, readable rendering, required endpoints, preservation, and applicable scope or binding conditions, but covers only captured properties.The finite checks do not guarantee uncaptured properties or visual quality; the ordinary verifier assesses goal relevance afterward.Source: §B.1 · p. 16
• An effective-endpoint match can make a raw light effect match by construction, and the reported full-effect shortcut may skip a compiled candidate without proving rerendering redundant.The example makes this a search-policy restriction; endpoint-match predicates need implementation confirmation.Source: §B.1 · p. 16
• The controller allows at most six additional execution attempts per submission, retains successful raw output if no alternative passes, and otherwise reuses the preceding delivered record after failure.Repair construction makes no additional language-model call, and raw fallback is not certification under a repair contract.Source: §B.1 · p. 16
• Exact reproduction depends on implementation-specific selectors and policies, including supported properties, owner matching, write grouping, projection scope, and light-inverse choices.Ordered rebinding observations and shortcut predicates also need to be tied to the evaluated code.Source: §B.1 · pp. 16, 18
• The refiner examines submitted executions, including improving raw renders, because distinct programs can render alike after execution erases intermediate differences.Planning actions do not invoke the refiner; observed effects supply bounded hypotheses, not proof that a request survived or that each repair improves the image.Source: §B.1 · pp. 14, 18

C.1 DIAGNOSTIC STUDY: CONSISTENT REALIZATION OF PROPOSALS

• 33.9% mean N-CLIP reduction with rebuilding, with a 95% interval of [19.8, 45.8] and 154 wins, 120 losses, and two ties.The effect was concentrated in geometry, with a 53.4% reduction.Source: §C.1 · p. 18
• Replaying one geometry20 program nine times on a persistent scene accumulated nine modifiers and reduced mesh volume to 1/256 of its first-execution value; rebuilding kept one modifier and constant volume.Explicitly replacing prior modifiers on the persistent scene also prevented accumulation.Source: §C.1 · p. 18

C.2 REFINEMENT COST AND CANDIDATE BREADTH

• N = 4 uses about 3.6× the model calls and 4.0× the renders of N = 1; the core contrasts use N = 1.The raw-program and repair stage has a worst-case bound of (1 + R)TN execution attempts, with R = 6, plus the separate initial-program observation.Source: §C.2 · pp. 18–19

D COMPLETE REFINEMENT PROCEDURE

• The procedure refines each render-producing candidate before visual selection, and keeps a cached render paired with its own program rather than a failed submission.Runs without a successful generated render, or that time out or remain incomplete, are invalid under the stated answer rule.Source: D COMPLETE REFINEMENT PROCEDURE · p. 19

E COMPLETE SYSTEM RESULTS

• Against matched One-Shot references, N = 1 reduces N-CLIP/PL by 40.3%/56.3% on the 212 tasks completed by both systems; N = 4 reduces them by 52.7%/58.8% on all 214 One-Shot tasks.Both variants improve each extended metric on their corresponding pairs.Source: E COMPLETE SYSTEM RESULTS · p. 20
• Against VIGA, N = 1 reduces N-CLIP by 36.3% (CI [24.3, 46.4], 161 wins and 114 losses, p = .0054); at N = 4, mean reductions are 40.6% in N-CLIP and 24.6% in PL.Both widths improve both primary metrics relative to BlenderAlchemy.Source: E COMPLETE SYSTEM RESULTS · p. 20
• Both VETR widths improve primary metrics over One-Shot, VIGA, and BlenderAlchemy across the reported GPT and Qwen comparisons.On Qwen3.8-27B, all four VETR-versus-VIGA comparisons meet both criteria.Source: E COMPLETE SYSTEM RESULTS · p. 20
• On Qwen3.8-27B, N=1 reduces N-CLIP/PL versus VIGA by 31.9%/50.8% on 261 paired tasks, and N=4 by 40.0%/56.3% on 264.Both widths also improve both metrics versus BlenderAlchemy; against One-Shot, N=4 reduces N-CLIP/PL by 28.1%/49.6% and N=1 by 22.1%/43.6%.Source: E COMPLETE SYSTEM RESULTS · p. 20

F EDITING STRATEGIES

• GPT-5.6-luna retains a median 10% of start-program lines under One-Shot, versus 96% in VETR’s first round on the same 100 geometry and material tasks.The One-Shot runner fails on 66 tasks, while the VETR line completes 100/100; VETR’s prompt asks for one to three edits over compatible bindings.Source: F EDITING STRATEGIES · p. 20

G COMPLETE CONTEXT ABLATIONS

• In the illustrated One-Shot attempt, GPT rewrites the program using a socket absent in Blender 3.6 and produces no render; inside VETR it preserves most of the program and edits it.The VETR example’s first-round program retains 99% of start lines, and all nine rounds render.Source: Figure 4 · p. 21, G COMPLETE CONTEXT ABLATIONS · p. 21
• Restricting the generator to its three most recent messages raises N-CLIP/PL errors by 48.5%/79.0% relative to default, with both metrics meeting the paired criterion.Equivalently, the default has 32.7%/44.2% lower N-CLIP/PL than the ablated arm.Source: Table 6 · p. 21, G COMPLETE CONTEXT ABLATIONS · p. 22
• Sharing one memory between generator and verifier raises both errors; the default wins on 164 and 169 of 280 tasks, with both metrics meeting the paired criterion.Table 6 reports the paired context-ablation statistics for this cohort.Source: Table 6 · p. 21, G COMPLETE CONTEXT ABLATIONS · p. 22
• Removing the visual verifier raises both mean errors, and the default wins on more tasks (159 and 161; p = .016 and .0098).The ablation cohort has valid answers on all 280 tasks, so comparisons are paired across the full benchmark.Source: G COMPLETE CONTEXT ABLATIONS · pp. 20, 22, Table 6 · p. 21

H SCENE-ORIGIN DETAILS AND ADDITIONAL TRAJECTORIES

• For geometry22, persistent execution elongates the object out of frame, while rebuild retains a target-like object; the scored answers are 23.52 for persistent and 1.44 for rebuild.Rebuild’s round 9 has no render, so its scored answer is round 8; on geometry20, persistent renders become progressively flatter while rebuild finishes closer to the goal (1.29 vs. 7.19).Source: Figure 5 · p. 23
• On the persistent-relative scale, rebuild reduces overall and geometry N-CLIP by 33.9% and 53.4%, respectively.The two arms retain generator history and differ in the scene supplied to complete-program execution.Source: H SCENE-ORIGIN DETAILS AND ADDITIONAL… · pp. 22–23

I CANDIDATE BREADTH

• At N=4, GPT’s paired mean N-CLIP and PL decreases are 7.7% and 9.4%, but their confidence intervals include zero and p values are .278 and .436.N=4 costs 3.6× the model calls, 4.0× the renders, and 3.6× the agent time of N=1; the tournament selects a nonzero candidate in 497 of 2,453 interactions.Source: I CANDIDATE BREADTH · p. 23, Table 9 · p. 24

J DETAILED PAIRED COMPARISONS

• The paired comparisons match valid task identities within each backbone; the GPT quality means for each One-Shot intersection are reported separately.The complete system comparisons and paired effects are reported in Tables 11 and 10, respectively.Source: J DETAILED PAIRED COMPARISONS · p. 24
• Against refinement disabled, VETR lowers mean PL by 41.8% and mean N-CLIP by 32.2% on GPT-5.6-luna; both comparisons meet the reported paired criterion.Both arms complete all 27 tasks.Source: J DETAILED PAIRED COMPARISONS · p. 24
• Against VIGA, VETR lowers mean N-CLIP by 55.0% and mean PL by 67.7% on GPT-5.6-luna, with both bootstrap intervals excluding zero.Both systems complete all 27 tasks.Source: J DETAILED PAIRED COMPARISONS · p. 24

• Qwen runs cover all 280 BlenderGym tasks using four H100 GPUs, Blender 3.6.0, Cycles CUDA rendering, and the same CLIP and RGB-MSE evaluators.One-Shot, VIGA, and VETR use enabled thinking; VIGA and VETR have a ten-round budget.Source: K QWEN3.8-27B EVALUATION: SETTINGS AND… · p. 24
• Table 11 covers all ten Qwen method pairs; timeout and no-render outcomes count as failures.The Qwen BlenderAlchemy run has one reported failure, after an upstream planning error produced no candidate render.Source: K QWEN3.8-27B EVALUATION: SETTINGS AND… · pp. 24–25, Table 11 · p. 25
• Qwen BlenderAlchemy uses depth 2, breadth 2, one try, disabled visual imagination, and disabled thinking for parseable planning responses.Source: K QWEN3.8-27B EVALUATION: SETTINGS AND… · p. 25
• Paired comparisons intersect valid task identities, bootstrap tasks 5,000 times, and use an exact two-sided sign test after removing ties.Tests are reported without multiple-comparison correction.Source: K QWEN3.8-27B EVALUATION: SETTINGS AND… · p. 26

L EXTENDED IMAGE METRICS

• Extended metrics are computed per image before task averaging, with explicit procedures for PSNR, SSIM, LPIPS, edge F1, and CLIP.The extended scorer uses stored final-answer images; the One-Shot extended metrics cover 214 valid GPT tasks, while GPT N=1 and N=4 comparisons use 212 and 214 jointly valid tasks, respectively.Source: L EXTENDED IMAGE METRICS · p. 26, Table 14 · p. 27

M ONE-SHOT EDITING STRATEGY: STATISTICS BEHIND SECTION F

• Of 92 GPT One-Shot programs retaining less than half the start program, 65 fail, versus 1 of 8 retaining at least half.VETR produces first-round programs on all 100 geometry and material tasks, with median retention 0.96 and 98 rendered outcomes; its full run logs 41 no-render outcomes in 814 interactions.Source: M ONE-SHOT EDITING STRATEGY:… · p. 26, Table 15 · p. 27

N EXECUTION SETTINGS

• The execution settings specify Blender 3.6.0, successful-render answer selection, and separate source cohorts for the listed experiments.Timed-out or incomplete runs and tasks without a successful render are invalid; primary comparisons intersect valid task identities.Source: N EXECUTION SETTINGS · p. 28, Table 16 · p. 28

O WITHIN-RUN PROGRESS AND GUIDANCE CONTROLS

• Among 256 tasks with both first-proposal and final-run results, 258 of 280 first proposals render and 278 of 280 final runs are valid.First proposals precede feedback from an executed program, and no metric-based historical selection is applied.Source: O WITHIN-RUN PROGRESS AND GUIDANCE… · p. 28, Table 17 · p. 29
• Neither replacing role prompts with VIGA prompts nor removing the blend-shape recipe and example produces a significant paired difference.The role-prompt control has 277 valid tasks; the no-recipe control has 84 valid tasks and is restricted to 85 blend-shape tasks.Source: O WITHIN-RUN PROGRESS AND GUIDANCE… · pp. 28–29, Table 18 · p. 29
• Persistent appending adds one modifier per execution and halves geometry20’s current volume each time, whereas rebuild and replacement keep both constant.All three policies use the same archived program; the normalized volume is based on each policy’s identical first execution.Source: Figure 6 · p. 29

P IDENTICAL-PROGRAM REPLAY OF SCENE OPERATIONS

• The replay executes the byte-identical geometry20 program nine times with the base scene and script fixed, comparing reload, persistent-scene, and compatible-replacement policies.No language-model requests or rendering calls are made.Source: P IDENTICAL-PROGRAM REPLAY OF SCENE… · p. 29
• At step 9, persistent execution yields nine modifiers and volume 0.015625 Blender units cubed, while rebuild and replacement retain one modifier and volume 4.The evaluated meshes remain at eight vertices and six faces under all policies, indicating repeated transformation rather than growing mesh topology.Source: P IDENTICAL-PROGRAM REPLAY OF SCENE… · pp. 29–30

Q ARTIFACTS AND TRACEABILITY

• The artifact map links experiment cohorts to task-level records, scripts, paired analyses, and the sources used to build Figure 5.The scene-origin records include invalid tasks and scored-render hashes.Source: Q ARTIFACTS AND TRACEABILITY · p. 30
• A complete proposal runs against the supplied native scene and assets; the runner does not execute the editable start program as a preparation step.Implementation records describe candidate selection, verification, history handling, and the placement of refinement before delivered-outcome selection.Source: Q ARTIFACTS AND TRACEABILITY · pp. 30–31