PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoFigure: Generating and Refining Publication-Ready Scientific Illustrations

Minjun Zhu, Zhen Lin, Yixuan Weng, Panzhong Lu, Qiujie Xie, Yifan Wei, Sifan Liu, Qiyao Sun, Yue Zhang

arXiv:2602.03828v2cs.AIcs.CLcs.CVcs.DL

TL;DR

Creating high-quality scientific illustrations from long-form texts is difficult and time-consuming, while prior automated datasets largely focus on captions or short snippets. AutoFigure introduces FigureBench and a reasoned-rendering framework that parses, plans, refines, and renders illustrations; evaluations show it produces structurally accurate, visually appealing results, with up to 66.7% judged publication-ready.

  • Problem

    Manual scientific illustration requires understanding long-form text, distilling critical information, and balancing structural fidelity with visual quality, while prior datasets mainly target captions, short snippets, or metadata.

  • Method

    AutoFigure uses reasoned rendering to convert long-form scientific text into a structured layout and style plan, then performs aesthetic rendering and text refinement through an erase-and-correct strategy.

  • Results

    AutoFigure consistently surpasses baseline methods, and up to 66.7% of generated results were judged by human experts to meet publication standards.

  • Takeaways & Limitations

    FigureBench and AutoFigure provide a benchmark and automated tool for generating scientifically rigorous, aesthetically appealing illustrations from long-form scientific texts.

  • Takeaways & Limitations

    Fine-grained text-rendering accuracy remains a bottleneck, with rare character-level errors under small fonts, dense layouts, or complex backgrounds.

Abstract

from arXiv · show

High-quality scientific illustrations are crucial for effectively communicating complex scientific and technical concepts, yet their manual creation remains a well-recognized bottleneck in both academia and industry. We present FigureBench, the first large-scale benchmark for generating scientific illustrations from long-form scientific texts. It contains 3,300 high-quality scientific text-figure pairs, covering diverse text-to-illustration tasks from scientific papers, surveys, blogs, and textbooks. Moreover, we propose AutoFigure, the first agentic framework that automatically generates high-quality scientific illustrations based on long-form scientific text. Specifically, before rendering the final result, AutoFigure engages in extensive thinking, recombination, and validation to produce a layout that is both structurally sound and aesthetically refined, outputting a scientific illustration that achieves both structural completeness and aesthetic appeal. Leveraging the high-quality data from FigureBench, we conduct extensive experiments to test the performance of AutoFigure against various baseline methods. The results demonstrate that AutoFigure consistently surpasses all baseline methods, producing publication-ready scientific illustrations. The code, dataset and huggingface space are released in https://github.com/ResearAI/AutoFigure.

1 INTRODUCTION

Scientific illustration generation from long-form text is difficult because it requires logical distillation, structural fidelity, and aesthetic rendering. AutoFigure addresses this gap with an agentic two-stage framework and FigureBench-based evaluation, reporting publication-oriented quality.

  • Motivation: Manual scientific illustration requires domain knowledge and design skills and can take human researchers several days.Effective illustrations must distill critical information while balancing structural fidelity and image quality.
  • Motivation: Existing benchmarks mainly reconstruct figures from captions, short snippets, or metadata rather than designing illustrations from entire long-form documents.The target task involves distilling methodologies from documents averaging more than 10k tokens and autonomously planning visual structure.
  • Approach: AutoFigure separates semantic parsing and layout planning from aesthetic rendering and text refinement within the Reasoned Rendering paradigm.Its second stage uses an erase-and-correct strategy to address blurry text rendering.
  • Benchmark and evaluation: FigureBench contains 3,300 high-quality long-form text–figure pairs, with 300 reserved for testing and the remainder for development.The benchmark supports comprehensive evaluation of automatic scientific illustration generation.
  • Results: 66.7% of generated results were judged by human experts to meet publication standards.The paper also reports extensive automated, human, and ablation evaluations of structural accuracy, text fidelity, aesthetics, and layout.

2 RELATED WORK

Related work has advanced automated scientific visuals and text-to-image generation, but existing systems remain limited in directly generating scientifically grounded illustrations from complex long-form text. The paper frames illustration generation as a missing capability for increasingly autonomous scientific workflows.

  • Automated scientific visuals generation: PosterAgent and PPTAgent primarily rearrange and summarize existing figures and textual content from source documents.These systems focus on posters and slides rather than generating new visual content directly from the original scientific text.
  • Text-to-image generation: Current text-to-image models struggle with scientific long-form texts containing specialized terminology, complex structures, and intricate conceptual relationships.Text-based conditioning offers flexibility, but scientific documents impose challenges beyond ordinary text-to-image generation.
  • Automated scientific discovery: AI Scientists can increasingly manage research workflows and produce textual artifacts, but their inability to generate illustrations limits visual articulation of their findings.Automating scientific illustration is presented as an essential next capability for autonomous discovery systems.

3 FIGUREBENCH: A BENCHMARK FOR AUTOMATED SCIENTIFIC ILLUSTRATION GENERATION

FigureBench is designed as a diverse benchmark for mapping long-form scientific text to publication-quality illustrations. It combines varied document sources, curated conceptual figures, challenging long-context statistics, and VLM-based evaluation of visual and scientific fidelity.

  • Benchmark scope: FigureBench targets long-context scientific illustration design across research papers, surveys, technical blogs, and textbooks.Its document diversity is intended to establish a challenging testbed for automatic illustration generation.
  • Data curation: The paper’s test-set curation begins with 400 Research-14K papers, uses GPT-5 to select methodology illustrations, and retains conceptual figures whose key visual elements are explicitly described.An additional 100 samples are manually curated from surveys, technical blogs, and textbooks.
  • Data curation: The Test Set is reserved for evaluation, while the Development Set supports training, development, and future trainable methods despite AutoFigure being inference-only.The datasets therefore have explicitly separated evaluation and development roles.
  • Dataset analysis: Text Tokens average 352 for Textbooks versus 12,732 for Papers, while average Text Density reaches 41.2% and average Colors 6.2.These statistics characterize substantial variation in long-context length and visual complexity.
  • Evaluation: The VLM-as-a-judge protocol combines referenced scoring across eight sub-metrics with blind pairwise comparison across seven criteria.Referenced scoring covers Visual Design, Communication Effectiveness, and Content Fidelity, with Overall computed as the average of sub-dimension scores.

4 AUTOFIGURE

AUTOFIGURE separates scientific-illustration generation into symbolic layout planning, iterative refinement, and style-guided rendering with text correction. Its pipeline converts long-form text into a structured blueprint, improves layout quality through critique, and renders a faithful final illustration.

  • Concept Extraction and Symbolic Construction: AUTOFIGURE uses an LLM to distill long-form text into a symbolic blueprint containing entities, relations, geometry, topology, and style information.The blueprint is represented in SVG/HTML, with a directed graph encoding visualized nodes and edges.
  • Critique-and-Refine: The system evaluates an initial layout and iteratively refines it through an AI designer–critic loop targeting alignment, balance, and overlap avoidance.Critic feedback is used to reinterpret the distilled methodology and generate candidate layouts for comparison.
  • Critique-and-Refine: The refinement loop retains superior candidates until a preset iteration limit or score convergence, producing a logically consistent, structurally coherent, and aesthetically balanced layout.The resulting layout and style description condition the subsequent rendering stage.
  • Aesthetic Rendering: A multimodal generative model converts the optimized layout and style description into an illustration using an exhaustive prompt and a structural graph that specifies element positions and interconnections.This stage aims to preserve layout structure while realizing the optimized aesthetic style.
  • Text Refinement: An erase-and-correct process improves textual legibility by removing rendered text, extracting OCR strings and boxes, and verifying them against ground-truth labels from the symbolic layout.The supplied passage describes the erasing, OCR, and verification stages; the final compositing step is truncated.

5 EXPERIMENTS

Experiments evaluate AutoFigure through automated benchmarks, expert judgments, and ablations, showing strong performance across document types and evidence that rendering, refinement, model choice, and intermediate format matter.

  • Automated evaluations: AUTOFIGURE achieves the highest Overall score across Blog (7.60), Survey (6.99), Textbook (8.00), and Paper (7.03) categories.
  • Automated evaluations: AUTOFIGURE’s Win-Rate reaches 75.0% for Blog, 78.1% for Survey, 97.5% for Textbook, and 53.0% for Paper.
  • Human evaluation: Expert evaluation reports an 83.3% win rate against other models, while 66.7% of experts would adopt AUTOFIGURE figures for camera-ready papers.
  • Ablation studies: Rendering raises GPT-5’s Overall score from 6.38 for symbolic layouts to 7.480 after rendering, improving visual design without compromising structural integrity or content fidelity.
  • Ablation studies: The refinement loop increases overall performance from 6.28 with zero iterations to 7.14 after five iterations.
  • Ablation studies: SVG (8.98) and HTML (8.85) outperform PPT (6.12) as intermediate formats, while AUTOFIGURE’s case study preserves aligned stages, hierarchy, spacing, and semantic cues.

6 CONCLUSION

The paper introduces FigureBench and AutoFigure to address challenges in generating high-quality scientific illustrations from long-form texts. Evaluations support AutoFigure’s ability to produce scientifically rigorous, aesthetically appealing illustrations suitable for academic publishing.

  • FigureBench contains 3,300 high-quality long-form scientific text–figure pairs spanning diverse scientific text types.
  • AUTOFIGURE is an agentic Reasoned Rendering framework that generates accurate and visually appealing illustrations through an iterative process.
  • Automatic evaluations and human expert assessments demonstrate that AUTOFIGURE produces scientifically rigorous, aesthetically appealing illustrations meeting academic publishing standards.
  • The work aims to automate a bottleneck in scientific communication and enable more efficient and accessible creation of publication-ready illustrations.

ETHICS STATEMENT

The paper frames AutoFigure as an assistive technology requiring transparency and expert verification because it can generate scientifically plausible but misleading schematics. Its broader evaluation and case studies support publication-ready output while exposing text-rendering and scientific-fidelity boundaries.

  • ETHICS STATEMENT: AutoFigure may generate scientifically plausible but factually incorrect schematics, so the authors require transparency, attribution, and expert verification.The system is explicitly described as an assistive tool whose outputs may not be perfectly reliable.
  • ETHICS STATEMENT: The human study recruited 10 computer-science first authors who evaluated figures from their own publications through scoring, ranking, and publication-readiness tasks.The scoring covered Accuracy, Clarity, and Aesthetics across six AI systems; holistic ranking also included the original human figure.
  • ETHICS STATEMENT: AUTOFIGURE mitigated baseline failures involving illegible text, topic hallucination, and structural collapse across blog, taxonomy, and textbook cases.The examples show incorrect topic generation, flattened categorical relationships, and text-rendering artifacts in baseline outputs.
  • ETHICS STATEMENT: Fine-grained text rendering remains a bottleneck, with rare character-level errors under small fonts, dense layouts, or complex backgrounds.The paper gives “ravity” instead of “gravity” as a representative error despite erase-and-correct post-processing.
  • ETHICS STATEMENT: When source text is underspecified, concretization can produce visually plausible but imperfect structures by compressing distinctions or imposing hierarchy on parallel concepts.The authors identify a tension between aesthetic presentation and scientific rigor.

G PERFORMANCE EVALUATION ON OPEN-SOURCE MODELS

The open-source evaluation tests whether AutoFigure remains effective with noncommercial reasoning backbones. Qwen3-VL performs especially strongly, and extended pairwise comparisons confirm AutoFigure’s dominance without a widespread low-quality baseline.

  • G PERFORMANCE EVALUATION ON OPEN-SOURCE MODELS: AutoFigure was evaluated with Qwen3-VL-235B-A22B-Instruct, GLM-4.5V, and ERNIE-4.5-VL to test noncommercial deployment.The evaluation targets high-quality generation without proprietary commercial APIs.
  • G PERFORMANCE EVALUATION ON OPEN-SOURCE MODELS: 7.08 Overall Score for Qwen3-VL-235B, exceeding GLM-4.5V (5.99), Gemini-2.5-Pro (6.99), Claude-4.1-Opus (6.80), and Grok-4 (6.76).The passage reports Qwen3-VL as outperforming the listed open-source and commercial comparators.
  • G PERFORMANCE EVALUATION ON OPEN-SOURCE MODELS: The results associate output quality with the backbone’s visual reasoning and instruction-following capabilities, supporting capable open-source backbones as a cost-effective deployment route.The paper specifically identifies Qwen3-VL’s success as evidence for this conclusion.
  • G PERFORMANCE EVALUATION ON OPEN-SOURCE MODELS: On the Paper subset, AutoFigure achieved a 72.5% win rate, while “Both Bad” occurred once and “Both Good” occurred three times.The extended comparison used absolute-quality options to test for race-to-the-bottom effects.

I ABLATION STUDY: CONTRIBUTION OF THE TEXT REFINEMENT MODULE

The ablation isolates Stage 2 Text Refinement by comparing the full AutoFigure pipeline with a version lacking erase-and-correct processing. Refinement yields its clearest gains in visual presentation quality.

  • I ABLATION STUDY: CONTRIBUTION OF THE TEXT REFINEMENT MODULE: The ablation compares full AutoFigure against a variant without Stage 2 Text Refinement to quantify erase-and-correct processing.The study complements ablations of the reasoning backbone and intermediate format.
  • I ABLATION STUDY: CONTRIBUTION OF THE TEXT REFINEMENT MODULE: 7.18 vs. 7.14 Overall score improvement accompanies gains of +0.10 in Aesthetic Quality, +0.10 in Professional Polish, and +0.08 in Visual Expressiveness.The granular gains are larger in visual-design dimensions than in the aggregate score.
  • I ABLATION STUDY: CONTRIBUTION OF THE TEXT REFINEMENT MODULE: Erase-and-Correct removes generative artifacts such as blurred text and elevates outputs from usable drafts to professional, publication-ready illustrations.The passage identifies the refinement stage as pivotal for this visual improvement.

J EFFICIENCY AND COST ANALYSIS

The efficiency analysis compares commercial API and local open-source deployment for generating illustrations from papers averaging more than 10k tokens. Local H100 deployment is faster and avoids recurring API costs while remaining accessible to research labs.

  • J EFFICIENCY AND COST ANALYSIS: The study measures inference latency and economic cost for papers averaging more than 10k tokens under commercial and local deployment settings.The commercial setting uses Gemini-2.5-Pro, while the local setting uses Qwen-3-VL on NVIDIA H100 hardware.
  • J EFFICIENCY AND COST ANALYSIS: 17.5 minutes and $0.20 per illustration with Gemini-2.5-Pro contrast with approximately 9.3 minutes locally on H100 GPUs.The local configuration provides a nearly 2× speedup, primarily by removing network latency and increasing inference throughput.
  • J EFFICIENCY AND COST ANALYSIS: Local deployment can run on 2× NVIDIA H100 GPUs or two NVIDIA DGX Spark servers valued at approximately $3,000 each.The authors present this setup as enabling local data privacy and high throughput without recurring API costs.

K HUMAN SANITY CHECK ON AUTOMATED DATASET STATISTICS

Human auditing found that automated FigureBench statistics were broadly consistent with manual estimates, while style variation preserved evaluation quality. The generation workflow combines iterative layout refinement, aesthetic rendering, OCR correction, background erasure, and final vector text composition.

  • Efficiency and cost: Local deployment reduced end-to-end generation time from approximately 17.5 minutes to ∼9.3 minutes, with reported marginal cost falling from ∼$0.20 to ∼$0.00 excluding hardware amortization and electricity.The reported speedup was approximately 2×, attributed primarily to eliminating network latency and increasing inference throughput.
  • Human sanity check: 21 human-audited text-figure pairs produced statistics broadly comparable to full-dataset automated measurements, supporting the automated dataset analysis as a sanity-checked estimate.Human and automated values were in the same order of magnitude, with relative deviations within a reasonable range.
  • Human sanity check: Components averaged 5.62 manually versus 5.3 automatically, while Colors averaged 7.29 versus 6.2, indicating substantial visual complexity in FigureBench.Human and automated estimates both identified the dataset as a non-trivial challenge for generation models.
  • Human sanity check: Text density was estimated at 54.29% by humans versus 41.2% automatically, partly because annotators assessed text-block bounding boxes while the model measured pixel-level density.The discrepancy reflects different operational definitions of text density rather than a direct failure of the automated measurement.
  • Style controllability: Overall style-control scores remained between 7.18 and 7.27 across three prompts, while blind-comparison win rates were similarly stable.The experiment fixed structural layout and textual content while varying only the Stage 2 style description.
  • Generation workflow: The workflow parses source documents, generates and critiques an initial vector layout, iteratively refines structure, and renders a polished illustration before OCR-based text verification.The pipeline additionally erases potentially blurred text and overlays corrected vector-quality text in the final composition.

N EXTENDED BASELINE EXPERIMENTAL RESULTS

Extended comparisons on the Paper category show that AutoFigure substantially outperforms TikZ-based code-generation methods and AutoPresent. The results support separating structural reasoning from rendering rather than directly serializing complex scientific layouts into code or arranging existing assets.

  • Extended baseline comparison: AutoFigure achieved Overall 7.03 and Win-Rate 53.0% in the extended Paper-category comparison, leading the added baselines.The comparison included TikZero, TikZero+, and AutoPresent.
  • Extended baseline comparison: TikZero and TikZero+ scored Overall < 1.5, reflecting severe difficulty for direct LaTeX TiKZ generation on high-dimensional scientific structures.The reported limitation extends beyond syntax errors to the cognitive burden of low-level coordinate calculation.
  • Interpretation: AutoFigure’s “Reasoning-then-Rendering” strategy bypasses the bottleneck of forcing an LLM to linearly serialize complex scientific structures into low-level code.The framework decouples macro-level logical construction from rendering.
  • Extended baseline comparison: AutoPresent reached Overall 2.55 but remained substantially behind AutoFigure because it arranges existing assets rather than designing explanatory schematics from scientific text.The paper attributes the gap to the absence of specialized reasoning modules for translating abstract scientific text into visual logic.

O QUALITATIVE ANALYSIS ON CHALLENGES IN THE "PAPER" CATEGORY

The Paper category is harder than Survey or Textbook categories because research illustrations combine dense hierarchical information with bespoke visual designs. This creates a recurring trade-off between preserving structural completeness and maintaining readable aesthetics.

  • Observed challenge: Paper-category win rates were lower than Survey or Textbook win rates, primarily because papers require hierarchical reasoning and novel, bespoke design patterns.The InstructGPT case study was used to investigate this performance gap.
  • Hierarchical information density: Research-paper diagrams may simultaneously encode macro workflows, micro-level procedural steps, and fine-grained entities such as roles or loss terms.Semantic parsing must identify critical visual nodes, condense other information, and arrange nested relationships in two dimensions.
  • Bespoke visual design: Unlike surveys or textbooks with established schemas, paper illustrations often represent unique pipelines without stable visual precedents for pattern matching.AutoFigure must therefore design a custom topology from scratch for each bespoke methodology.
  • Completeness–clarity trade-off: Preserving every detected node can produce clutter, whereas merging sub-steps can maintain a clean layout but incur penalties for incomplete information.This trade-off between structural completeness and aesthetic clarity helps explain lower Paper-category win rates.
  • Human-LLM validation: The VLM evaluator correlated positively with human judgments, with Pearson r = 0.659 (p < 0.001), Spearman ρ = 0.593, Kendall’s Tau = 0.497, and Mean Ranking Error 0.98.These results support the evaluation proxy used to compare generated illustrations across the study.
Loading 2602.03828v2…