PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoFigure-Edit: Generating Editable Scientific Illustration

Zhen Lin, Qiujie Xie, Minjun Zhu, Shichen Li, Qiyao Sun, Enhao Gu, Yiran Ding, Ke Sun, Fang Guo, Panzhong Lu, Zhiyuan Ning, Yixuan Weng, Yue Zhang

arXiv:2603.06674v1cs.CVcs.AI

TL;DR

Automated scientific-illustration systems remain limited in editability, stylistic controllability, and efficiency despite the importance of high-quality figures. AUTOFIGURE-EDIT generates fully editable SVG illustrations from long-form text using reference-guided style adaptation and native SVG editing. Quantitative evaluations and a deployment-based user study found that it consistently outperformed prior methods and frequently produced outputs users judged ready for academic publication.

  • Problem

    Existing systems struggle to transform long-form scientific text into illustrations that are accurate, visually compelling, editable, and stylistically controllable.

  • Method

    AUTOFIGURE-EDIT combines long-context understanding, reference-guided style adaptation, an explicit structural scaffold, and native SVG editing to generate editable illustrations.

  • Results

    AUTOFIGURE-EDIT consistently outperformed prior methods, and users frequently judged its outputs ready for academic publication.

  • Takeaways & Limitations

    The system supports efficient creation and refinement of scientific illustrations while allowing semantic components and layouts to be edited directly.

  • Takeaways & Limitations

    The pipeline relies on closed-source foundation models, which may incur usage costs, raise privacy concerns, and limit reproducibility.

Abstract

from arXiv · show

High-quality scientific illustrations are essential for communicating complex scientific and technical concepts, yet existing automated systems remain limited in editability, stylistic controllability, and efficiency. We present AutoFigure-Edit, an end-to-end system that generates fully editable scientific illustrations from long-form scientific text while enabling flexible style adaptation through user-provided reference images. By combining long-context understanding, reference-guided styling, and native SVG editing, it enables efficient creation and refinement of high-quality scientific illustrations. To facilitate further progress in this field, we release the video at https://youtu.be/10IH8SyJjAQ, full codebase at https://github.com/ResearAI/AutoFigure-Edit and provide a website for easy access and interactive use at https://deepscientist.cc/.

1 Introduction

Generating accurate, visually compelling scientific illustrations from long-form text remains difficult, while prior systems also limit editability and stylistic control. AUTOFIGURE-EDIT addresses these gaps by producing editable SVG illustrations with reference-guided style control and interactive refinement.

  • Motivation: Creating high-quality scientific illustrations requires substantial time, domain expertise, design skills, and careful processing of long-form scientific text.The task must balance structural fidelity, visual quality, clarity, accuracy, and aesthetic appeal.
  • Limitations of Prior Systems: Prior AUTOFIGURE outputs were fixed and non-editable, relied on ambiguous prompt engineering for style, and coupled layout planning with raster rendering.These limitations made fine-grained layout edits difficult and text rendering unstable.
  • AUTOFIGURE-EDIT: AUTOFIGURE-EDIT transforms long-form scientific text and a reference style image into a fully editable SVG illustration.The system uses an explicit structural scaffold to separate layout planning from final rendering, allowing layout edits without rerunning the full refinement loop.
  • AUTOFIGURE-EDIT: Reference-guided style control adapts visual appearance through a user-provided exemplar while preserving semantic structure.The system supports changes to color palettes, typography, icon aesthetics, spacing density, and visual hierarchy.
  • AUTOFIGURE-EDIT: AUTOFIGURE-EDIT produces structurally organized, component-level editable SVGs with real-time refinement through an integrated interactive canvas.This provides direct editing of semantic elements such as modules, connectors, and annotations.
  • Evaluation: Quantitative experiments and user studies demonstrate AUTOFIGURE-EDIT's effectiveness and practical value for generating high-quality, editable scientific illustrations.The supplied comparison caption identifies scientific illustration generation and integrated GUI editing capabilities as evaluation dimensions.

2 Related Work

Scientific-figure generation has progressed from static text-to-figure systems toward programmatic and structurally guided approaches, but editability and robustness remain challenging. AUTOFIGURE-EDIT combines long-form context understanding with structural reconstruction and visual editing to improve this balance.

  • Text-to-Figure Generation: Existing text-to-figure systems generate high-quality illustrations from long descriptions but typically produce static outputs requiring full regeneration for minor adjustments.This limits iterative refinement and post-generation customizability.
  • Programmatic Synthesis: Diffusion models are effective for general visual synthesis but offer limited structural transparency for the strict compositional constraints of scientific figures.This makes direct control over scientific figure composition difficult.
  • Programmatic Synthesis: Text-to-Code-to-Image pipelines improve controllability through intermediate representations such as TikZ or SVG, but syntax brittleness and weak visual editing interfaces hinder refinement.Small syntax errors can cause rendering failures, while programmatic-only workflows increase iterative editing effort.
  • AUTOFIGURE-EDIT: AUTOFIGURE-EDIT combines long-form context understanding with robust structural reconstruction to balance stylistic flexibility and editability.Its transformation stages disentangle layout planning, object identity, and visual rendering while preserving semantic and stylistic coherence.

3 AutoFigure-Edit

AUTOFIGURE-EDIT transforms long-form scientific text and reference style images into structured, fully editable SVG illustrations. Its staged pipeline separates structural layout from visual appearance, enabling style adaptation and component-level refinement.

  • Framework Overview: AUTOFIGURE-EDIT transforms long-form scientific text into structured, fully editable scientific illustrations while supporting reference-image style adaptation.The system is designed to preserve textual semantics, conform to reference-image style, and support downstream editing.
  • Framework Overview: The pipeline first generates a style-conditioned raster draft from scientific text and a reference image, establishing semantic–stylistic alignment.This stage translates textual descriptions into explicit visual entities while incorporating high-level stylistic cues.
  • Framework Overview: Instance segmentation converts the raster draft into indexed structural layouts with uniform tones and unique identifiers, providing a coordinate-aware scaffold for vector generation.Masks and bounding boxes preserve spatial configuration and instance identity while suppressing texture and color.
  • Framework Overview: Injecting extracted appearance assets into the refined template produces an SVG whose layout, object identity, and visual appearance remain independently manipulable.Users can modify geometry, adjust style, or update individual components without disrupting the overall composition.
  • Framework Overview: The system generates and refines an SVG template using structural masks, raster previews, and SVG code, correcting positional and stylistic discrepancies in 0–2 iterations.Identifier mappings and placeholder groups remain preserved for subsequent asset injection.
  • Applications: Representative applications demonstrate generation from long-form descriptions, style adaptation, and interactive editing through an embedded visual editor.The system is presented as a productivity tool intended to lower the barrier to high-quality scientific illustration.

4 Evaluation

AUTOFIGURE-EDIT is evaluated on FigureBench and through a deployment-based user study, showing strong performance across visual quality, communication, content fidelity, and practical usability. Reference conditioning improves content fidelity and overall preference, while user ratings indicate semantic correctness, completeness, and style consistency.

  • Experimental Setup: The evaluation combines FigureBench experiments with a deployment-based user study involving 217 participants.FigureBench includes long-form method sections paired with publication-quality illustrations; the user study collects real-world usability feedback.
  • Quantitative Analysis: AUTOFIGURE-EDIT consistently outperforms prior approaches across Visual Design, Communication Effectiveness, and Content Fidelity.The comparison reports a strong balance between visual quality and scientific fidelity.
  • Quantitative Analysis: Reference conditioning raises Content Fidelity to Accuracy 8.83, Completeness 8.26, and Appropriateness 8.37, while Win-Rate increases from 76.0% to 83.0%.Visual Design slightly decreases, with Aesthetic falling from 8.32 to 7.37, indicating a trade-off between stylistic expressiveness and semantic fidelity.
  • User Study: User-study mean scores reach 4.04 for Scientific Semantic Correctness, 4.11 for Information Completeness, 3.95 for Visual Presentation Quality, and 4.09 for Style Consistency.Ratings are concentrated at the highest level, with Score 5 assigned in 48%, 51%, and 50% of evaluations for semantic correctness, completeness, and style consistency, respectively.
  • User Study: 126 of 262 evaluation samples were judged directly suitable for academic-paper inclusion without additional modification.Direct usability requires conceptual correctness, satisfactory layout, and stylistic quality.
  • User Study: PNG-to-SVG reconstruction achieves a 3.60 average Conversion Correctness score, with 36% of evaluations receiving Score 5.Minor geometric deviations may occur, but editable SVG output allows correction with minimal manual effort.

5 Conclusion

AUTOFIGURE-EDIT generates fully editable scientific illustrations from long-form text while supporting reference-guided style adaptation and native SVG editing. Quantitative evaluations and a deployment-based user study found that it consistently outperformed prior methods and was frequently judged ready for academic publication.

  • Conclusion: AUTOFIGURE-EDIT generates fully editable scientific illustrations from long-form text with reference-guided style adaptation and native SVG editing.These capabilities are presented as the system’s central contribution.
  • Conclusion: Quantitative evaluations and a deployment-based user study showed consistent improvement over prior methods and frequent judgments of publication readiness.The conclusion reports both comparative performance and user-assessed practical value.

Limitations

AUTOFIGURE-EDIT remains constrained by dependence on closed-source foundation models, error propagation from raster-derived vector structures, and limited editor scope and evaluation coverage.

  • Foundation-model dependence: Closed-source foundation models constrain reproducibility and may introduce usage costs and data-privacy concerns.The pipeline uses vision and vision-language models for style-conditioned raster synthesis and SVG template refinement.
  • Error propagation: Upstream segmentation errors can cascade from intermediate raster drafts into vector structures, requiring manual editor adjustments.One example is incorrectly merging split visual components.
  • Scope and evaluation constraints: The embedded editor supports localized component-level refinements but is not intended to replace comprehensive graphic design software.Its scope is therefore narrower than full graphic-design workflows.
  • Scope and evaluation constraints: Current evaluation emphasizes usability and workflow efficiency, while broader specialized-domain validation and rigorous expert correctness checks remain future work.The user study was primarily designed as a usability evaluation rather than an expert-only correctness assessment.

Ethics and Broader Impact Statement

AUTOFIGURE-EDIT carries risks of generating scientifically plausible but factually incorrect or misleading schematics. The authors address this through explicit limitations, transparency, and attribution requirements.

  • The primary ethical risk is misuse of AUTOFIGURE-EDIT to support false claims with scientifically plausible but factually incorrect or misleading schematics.
  • The authors state that AUTOFIGURE-EDIT is an assistive tool with limitations and not a substitute for expert verification.
  • The public code repository will prominently disclose that outputs may not be perfectly reliable.
  • An open-source license will require publications using generated figures to discuss AI's role and caption the figures as AI-generated.

A Quantitative Evaluation

The evaluation compares AUTOFIGURE-EDIT with end-to-end text-to-image and text-to-code baselines, and uses blind pairwise comparisons to measure figure preference.

  • The study compares AUTOFIGURE-EDIT with end-to-end text-to-image methods and text-to-code methods that generate renderable HTML or SVG.
  • Win-Rate measures how often a method is preferred in blind pairwise comparisons against reference illustrations.

B User Study

The user study separately evaluates the quality of the originally generated figure and the fidelity of its converted SVG through a standardized website-based feedback process.

  • The user study has separate Figure Evaluation and SVG Conversion Evaluation components.
  • All scalar metrics use a 5-point Likert scale, while the website presents users with an automated rating dialog.
  • Scoring guidelines combine a metric definition, guiding question, and rubric to standardize judgments and reduce subjective-interpretation variance.

Part I: Figure Evaluation (PNG)

Figure Evaluation assesses scientific correctness, completeness, visual quality, style consistency, and direct usability of the generated PNG figure.

  • Scientific Semantic Correctness (1–5) measures whether the figure accurately represents concepts, processes, and relationships from the method text.
  • Information Completeness (1–5) measures whether all key components and steps from the method text appear in the figure.
  • Visual Presentation Quality (1–5) evaluates clarity, readability, alignment, professionalism, and suitability for academic publication.
  • Style Consistency (1–5) measures how well the generated figure matches the provided reference image's visual style.
  • Usability (Binary: 0/1) determines whether the figure can be directly used in an academic paper without major modifications.

Part II: SVG Conversion Evaluation

The SVG conversion evaluation measures whether structural and semantic elements remain correctly preserved after conversion. Qualitative results and reference-versus-generated comparisons are also presented.

  • Conversion Correctness: Conversion Correctness measures whether structural and semantic elements are preserved during PNG-to-SVG conversion.The assessment uses a 1–5 scale.
  • Conversion Correctness: The evaluation checks component placement and correspondence between original objects and SVG elements.
  • Qualitative Comparisons: Figure 5 presents qualitative results of AUTOFIGURE-EDIT, while Figure 6 compares reference images with generated images.
  • Conversion Correctness: It also checks that conversion introduces no missing or duplicated elements.
Loading 2603.06674v1…