Source-linked AI summary
Figures as Programs: Recursive Generation of Editable Scientific Figures
Yepeng Liu, Dasen Dai, Chengzhi Liu, Yiren Song, Hai Ci, Yu Zhang, Qi Zhang, Mike Zheng Shou, Xin Eric Wang, Yuheng Bu
TL;DR
Scientific methodology figures are labor-intensive to create, and raster outputs are difficult to generate and edit precisely for complex diagrams. FigTree constructs editable SVG programs recursively from paper content, using localized fragments, hierarchical assembly, and render-critic repair. It produces high-quality figures and supports more effective iterative editing than raster-based approaches.
Problem
Complex scientific figures remain labor-intensive, while raster generation and precise editing are unreliable for diagrams with many interdependent components.
Method
FigTree grounds a paper’s content and style, recursively generates localized SVG fragments in a hierarchy, assembles them, and repairs defects by inspecting code and rendering together.
Results
FigTree produces high-quality figures, with especially strong hallucination control, and more efficient editing and higher quality than raster-based methods under the same editing rounds.
Takeaways & Limitations
Recursive SVG construction provides a structured representation that supports localized repair and more reliable iterative editing of scientific figures.
Abstract
from arXiv · showhide
Scientific methodology figures are essential for communicating complex methods clearly, yet creating them remains labor-intensive and typically requires multiple rounds of refinement. Recent image-generation models can synthesize visually appealing raster figures, but producing a human-satisfactory result in a single generation step remains difficult. Moreover, precise edits to raster figures are challenging for both humans and models. We formulate scientific figure generation as recursive SVG program construction and propose \textsc{FigTree}, a \textit{multi-agent} system that automatically transforms a scientific paper into a structured vector figure. \textsc{FigTree} grounds figure content in the source paper, decomposes a figure into a hierarchy of local regions, generates each region as a short SVG program, and assembles the resulting fragments. A render-critic refinement loop jointly inspects the rendered figure and its underlying program, enabling visual defects to be traced to specific statements and accurately repaired. We conduct extensive evaluations of \textsc{FigTree} on figure quality and editability, showing that \textsc{FigTree} produces high-quality figures, while also enabling more effective editing than existing raster-based methods.
1 Introduction
Scientific methodology figures are important but labor-intensive, while raster generation and editing remain unreliable for complex diagrams. FigTree addresses these challenges by constructing editable SVG figures recursively and refining them through structured evaluation.
- Challenges: Complex scientific figures are difficult to generate satisfactorily in one raster-generation step because they contain interdependent components and may include hallucinations, illegible text, or low resolution.Iterative refinement guided by humans or VLM critics is often necessary.
- Challenges: Raster figures are difficult to refine precisely because critics must specify spatial modifications and generators must avoid disturbing unrelated content.Manual editing is also constrained because raster images expose no explicit structure for selecting elements.
- Approach: FigTree formulates scientific figure generation as recursive SVG program construction, producing editable and resolution-independent vector graphics.The approach shifts generation from image synthesis to code generation.
- Approach: FigTree recursively decomposes figures into local regions, generates short SVG subprograms, and assembles them into complete figures.The hierarchy turns one global generation problem into simpler localized ones and supports parent-owned connections between regions.
- Evaluation: FigTree evaluates figure quality with a fine-grained rubric and editability through iterative editing over multiple rounds.These evaluations complement reference-based comparison and assess refinement effectiveness.
- Results: FigTree produces high-quality figures, controls hallucinations particularly well, and achieves more efficient editing and higher quality than raster-based methods under equal editing rounds.The reported outcome covers both figure quality and editability.
2 Related Work
Scientific figures remain a largely manual part of research workflows, and raster-based methods struggle with reliability and precise modification. Prior editable approaches either vectorize raster intermediates or face long, tightly coupled graphics programs, motivating direct structured generation.
- Motivation: Scientific figures remain outside much of automated research workflows and require substantial human effort.The paper treats figure generation as a distinct problem involving grounding, spatial organization, verification, and iterative editing.
- Raster Generation: Raster figure generators often struggle with one-shot synthesis for diagrams containing many components and dense spatial relationships.Agentic planning and refinement improve quality but leave the output as a flattened raster image.
- Editability: Flattened raster outputs are cumbersome for humans to modify, making editability a critical property for automated scientific figure generation.The limitation concerns the representation available for selecting and revising components.
- Editable Generation: Post-hoc vectorization improves final editability but does not solve the upstream raster stage’s difficulty in controlling or precisely revising individual components.The vectorization stage reproduces the raster intermediate rather than reconstructing the underlying design intent.
- Editable Generation: Direct LLM generation of complete methodology figures remains difficult because graphics programs are long and elements are tightly coupled by spatial constraints.This motivates structured decomposition rather than monolithic code generation.
3 Methodology
FigTree represents a methodology figure as an SVG program organized by a layout tree, then constructs it recursively through localized generation, fragment merging, and dual-space repair. Paper grounding supplies the content and style constraints that coordinate the process.
- Overview: FigTree treats figure generation as long-horizon SVG program construction and decomposes the diagram into short regional fragments assembled level by level.This converts one long-horizon task into shorter localized subproblems.
- Overview: The SVG program g is generated from paper text P, deterministically rendered as R(g), and edited locally because each visual element maps to an addressable program statement.The representation provides editability and supports generation-time repair.
- Representation: The figure hierarchy is represented as a layout tree whose nodes encode sub-figure descriptions, local canvases, boundary ports, depth, and global style.The style specification includes font, palette, stroke widths, grid pitch, and arrow-marker geometry.
- Paper Grounding: A grounder extracts a content plan and global style from the paper, ensuring subfigures share a consistent visual language and contain paper-supported content.The content plan specifies visualized components, data flow, and notation.
- Recursive Figure Construction: During top-down construction, each generator draws a leaf when the node is atomic or reaches Dmax; otherwise it creates children and delegates recursively.The recursion is organized around Decompose, Draw, and Merge operations.
- Recursive Figure Construction: Children return fragments, occupied bounding boxes, exposed ports, and unresolved defects, allowing parents to place fragments and connect siblings without sibling knowledge.Parent-owned cross-boundary connections provide a single owner for each such connection.
- Recursive Figure Construction: Leaf fragments draw only internal connections, while external connections are represented through named ports for later assembly.The ports are the sole interface through which sibling fragments connect to a node.
- Render-Critic Refinement: A critic reviews both SVG code and its rendering, then instructs the generator to edit only the offending element and rerender until defects disappear or retries are exhausted.Reusable condition–repair rules provide verification memory for recurring visual and structural defects.
4 Experiments
The experiments evaluate FigTree against raster and editable-vector baselines on figure quality, diagnostic dimensions, component contributions, and iterative editing. FigTree achieves stronger quality and more reliable refinement, with especially large gains in faithfulness, conciseness, and hallucination control.
- Experiment Settings: The evaluation covers reference-based quality, fine-grained diagnostics, mechanism ablations, and multi-round editing on PaperBananaBench.PaperBananaBench contains 292 methodology-figure cases, while comparisons keep method text and captions consistent across systems.
- Experiment Settings: FigTree is compared with proprietary raster generators, agentic raster frameworks, AutoFigure-Edit, and Direct-SVG.Direct-SVG uses the same SVG-generation backbone but omits recursive decomposition and critic-guided repair.
- Main Results: 9.59 points: FigTree improves the reference-based overall score over Crafter, the strongest baseline, with largest gains in faithfulness (+8.56) and conciseness (+8.90).Readability improves by +2.74 and aesthetics by +1.37; FigTree achieves the strongest performance across all four dimensions.
- Main Results: FigTree achieves the highest fine-grained overall score, ranking first or tied for first on hallucination control, semantic coverage, visual polish, and academic style.It remains within 0.4 points of the best method on each remaining dimension.
- Ablation Study: 8.38: full FigTree scores highest in the mechanism study; removing verification memory lowers the score to 4.43 (−3.95), followed by paper grounding to 5.91 (−2.47) and recursive decomposition to 6.27 (−2.11).The ablations associate verification memory with detecting and repairing recurring errors, paper grounding with preserving paper-specific content, and recursive decomposition with preserving components and relationships.
- Iterative-editing Evaluation: Over eight editing rounds, FigTree steadily improves and outperforms the raster branch, indicating more reliable and controllable refinement from vector-based representation.Both branches start from the same defective figure and initial edit request, and quality is measured as the mean of eight diagnostic scores after each round.
5 Conclusion
FigTree formulates editable scientific figure generation as recursive SVG program construction, using hierarchical decomposition and parent-owned connections. Experiments show high-quality figures and more reliable iterative editing than raster-based approaches.
- FigTree generates editable scientific figures as recursively constructed SVG programs.The system grounds content in the source paper, decomposes diagrams into localized subprograms, and assembles them hierarchically.
- A render–critic loop traces visual defects to specific SVG elements and repairs them locally.
- Experiments show that FigTree produces high-quality methodology figures and supports more reliable iterative editing than raster-based approaches.
A.1 Cost–Quality Trade-off
FigTree offers a favorable generation cost–quality trade-off. It attains the highest visual quality among compared systems while costing less per image than GPT-Image-2 and Crafter.
- FigTree attains the highest visual quality among the compared systems at lower cost per image than GPT-Image-2 and Crafter.Figure 3 measures average cost per generated image in USD and visual quality score.
- Strong visual quality and low average generation cost suit workflows involving iterative prompting and repeated edits.
A.2 Additional Qualitative Examples
FigTree generates methodology figures across diverse scientific structures rather than relying on a fixed template. Its outputs retain structured SVG components that remain independently editable.
- FigTree supports cyclic workflows, neural architectures, multi-stage pipelines, hierarchical taxonomies, equation-augmented optimization, and parallel-branch models.
- Its recursive decomposition, layout, and connection structure adapt to each method’s logical organization.The examples include sequential, cyclic, nested, parallel, and hierarchical organizations.
- Rendered previews preserve underlying SVGs whose labels, shapes, groups, and connectors remain independently editable.
B Evaluation Prompt Templates
The appendix documents reference-based and diagnostic evaluation prompt templates for judging scientific figures. The templates define criteria for content grounding, structure, rendering, publication readiness, and structured evaluator outputs.
- B.1 PaperBananaBench Evaluation Prompts: The benchmark protocol supplies method text, captions when required, human references, and model diagrams for four independent pairwise judgments.Each judgment returns Model, Human, Both are good, or Both are bad, with percentages computed by the official implementation.
- B.1 PaperBananaBench Evaluation Prompts: Faithfulness evaluates factual and logical alignment with the source text while permitting simplification that preserves core method logic.
- B.1 PaperBananaBench Evaluation Prompts: Conciseness measures visual signal-to-noise ratio and penalizes textual overload, literal copying, and cluttering with raw equations.
- B.1 PaperBananaBench Evaluation Prompts: Readability assesses visual flow, legibility, and interference using severe violations as pass/fail triggers.
- B.1 PaperBananaBench Evaluation Prompts: Aesthetics evaluates visual polish through hierarchy, white space, typography, color harmony, and vetoes for common low-quality artifacts.
- B.2 Fine-grained Diagnostic Evaluation Prompts: The diagnostic evaluator compares generated figures with references and source excerpts, scoring eight dimensions from 1–10 with justifications and diagnostic tags.Its system prompt requires concrete visual evidence and use of the full score range.
- B.2 Fine-grained Diagnostic Evaluation Prompts: The diagnostic prompt is split into blocks for overall context, content grounding, diagram structure, rendering quality, and publication readiness.The final block includes the required JSON output schema.
- B.2 Fine-grained Diagnostic Evaluation Prompts: Content grounding checks semantic coverage against the source text, while structure and rendering blocks assess topology, flow, text/glyph quality, and visual polish.Publication readiness additionally checks academic style, compactness, and caption compatibility.
C Methodology Prompts
The methodology prompts specify how FigTree grounds a paper, recursively generates and reviews SVG fragments, and merges or repairs them into a complete figure. The appendix documents the prompts and task packages for the grounder, generator, critic, and worker agents.
- C Methodology Prompts: The appendix provides the complete prompts for paper grounding, recursive figure construction, and render-critic refinement.The prompts follow the methodology described in Section 3, with bracketed fields replaced for each example.
- C.1 Paper Grounding Prompts: The grounder extracts a content plan and global style specification from the full paper text.The content plan covers contributions, components, stages, flows, inputs, outputs, and mathematical notation; the style specification covers colors, typography, strokes, grid alignment, arrows, padding, and spacing.
- C.2.1 Generator Agent Prompts: At each recursive node, the generator chooses Decompose or Draw, while the generator prompts enforce academic aesthetics, arrow geometry, prohibited styles, and decomposition constraints.The task package includes the node description, paper excerpt, local canvas, required boundary ports, style, depth, and optional repair instructions.
- C.2.2 Critic Agent Prompts: The critic performs dual-space review by inspecting both the rendered PNG and SVG source, then returns element-anchored issue reports rather than a regenerated figure.This review recurs after leaf and internal-node renders.
- C.2.3 Worker Agent Prompts: The worker merges child fragments on parent canvases and draws cross-boundary connections at internal nodes.The merge operation combines child SVG files in the parent coordinate frame and is used only for internal nodes.
- C.2.3 Worker Agent Prompts: For repair rounds, the worker applies the critic’s anchored issue reports directly to the current SVG instead of regenerating the figure.The local repair prompt specifies in-place correction of the SVG.