PaperlayerView exact revision on arXiv

Source-linked AI summary

PaperBanana: Automating Academic Illustration for AI Scientists

Dawei Zhu, Rui Meng, Yale Song, Xiyu Wei, Sujian Li, Tomas Pfister, Jinsung Yoon

arXiv:2601.23265v2cs.CLcs.CV

TL;DR

Autonomous AI scientists still struggle to produce publication-ready illustrations, limiting visual communication of research. PaperBanana addresses this gap with a reference-driven agentic workflow, and experiments show consistent gains over leading baselines across diagram-quality dimensions while extending to statistical plots.

  • Problem

    Autonomous AI scientists can automate parts of research but still struggle to generate illustrations that meet publication standards for visual scientific communication.

  • Method

    PaperBanana orchestrates specialized agents and state-of-the-art VLMs and image-generation models to retrieve references, plan content and style, render illustrations, and iteratively self-critique them.

  • Results

    PaperBanana consistently outperforms leading baselines in faithfulness, conciseness, readability, and aesthetics, with an overall score gain of +17.0% on PaperBananaBench, and extends to statistical plots.

  • Takeaways & Limitations

    PaperBanana supports automated generation of publication-ready methodology diagrams and statistical plots for autonomous scientific communication.

  • Takeaways & Limitations

    PaperBanana’s raster outputs are difficult to edit compared with vector graphics, even when generated at 4K resolution.

Abstract

from arXiv · show

Despite rapid advances in autonomous AI scientists powered by language models, generating publication-ready illustrations remains a labor-intensive bottleneck in the research workflow. To lift this burden, we introduce PaperBanana, an agentic framework for automated generation of publication-ready academic illustrations. Powered by state-of-the-art VLMs and image generation models, PaperBanana orchestrates specialized agents to retrieve references, plan content and style, render images, and iteratively refine via self-critique. To rigorously evaluate our framework, we introduce PaperBananaBench, comprising 292 test cases for methodology diagrams curated from NeurIPS 2025 publications, covering diverse research domains and illustration styles. Comprehensive experiments demonstrate that PaperBanana consistently outperforms leading baselines in faithfulness, conciseness, readability, and aesthetics. We further show that our method effectively extends to the generation of high-quality statistical plots. Collectively, PaperBanana paves the way for the automated generation of publication-ready illustrations.

1. Introduction

PaperBanana addresses the difficulty autonomous AI scientists face when producing publication-ready academic illustrations. It combines specialized agents and reference-driven refinement with a benchmark and shows strong performance across diagram-quality dimensions while extending to statistical plots.

  • Motivation: Current autonomous AI scientists can automate research tasks but struggle to visually communicate discoveries through publication-standard diagrams and plots.The challenge is especially pronounced for illustrations that must satisfy rigorous academic-manuscript standards.
  • Motivation: Code-based diagram methods support structured content but may lack expressiveness for specialized icons and custom shapes common in modern AI publications.The introduction contrasts programmatic approaches using TikZ, Python-PPTX, or SVG with the visual demands of contemporary academic diagrams.
  • PaperBanana: PaperBanana orchestrates specialized VLM and image-generation agents to retrieve references, plan content and style, render illustrations, and refine them through self-critique.The reference-driven workflow is designed to capture both logical composition and stylistic norms for publication-ready outputs.
  • Evaluation: 292 test cases and 292 reference cases from NeurIPS 2025 publications form PaperBananaBench, spanning research topics and illustration styles.Generation quality is evaluated against human illustrations using faithfulness, conciseness, readability, and aesthetics.
  • Results: PaperBanana outperforms leading baselines across faithfulness (+2.8%), conciseness (+37.2%), readability (+12.9%), aesthetics (+6.6%), and overall score (+17.0%).The framework also extends to statistical plot generation.
  • Contributions: The paper contributes an automated agentic illustration framework, a benchmark for methodology diagrams, and experiments indicating promise for automating academic illustration generation.The stated contribution scope covers publication-ready illustrations rather than methodology diagrams alone.

2. Task Formulation

The paper formulates academic illustration generation as mapping source context and communicative intent to an image. Reference examples can optionally augment this mapping, while methodology-diagram inputs use technical descriptions and figure captions.

  • Core formulation: Automated academic illustration generation maps source context and communicative intent to a visual representation.The generated image should faithfully visualize the source context while fulfilling the specified intent.
  • Core formulation: Source context S contains essential information, while communicative intent C specifies the desired illustration’s scope and focus.The target image is denoted by I.
  • Reference augmentation: The input may include N reference examples, each defined as a tuple (S_n, C_n, I_n) linking context and intent to a reference illustration.These examples act as ground-truth demonstrations for guiding the mapping function.
  • Reference augmentation: When no reference examples are used, E defaults to ∅, corresponding to zero-shot generation.Reference augmentation is therefore optional rather than required by the formulation.
  • Methodology-diagram setting: For methodology diagrams, S is a textual method description and C is a figure caption specifying the illustration’s scope and focus.An example intent is an overview of the framework.

3. Methodology

PaperBanana uses a reference-driven team of specialized agents to transform scientific content into publication-quality diagrams and plots. The workflow retrieves examples, plans content and style, renders descriptions, and iteratively refines outputs through Visualizer–Critic collaboration.

  • PaperBanana orchestrates five specialized agents—Retriever, Planner, Stylist, Visualizer, and Critic—to transform raw scientific content into publication-quality diagrams and plots.
  • The Retriever selects reference examples by matching research domain and diagram type, prioritizing visual structure over topic similarity.The selected references provide a foundation for structural logic and visual style.
  • The Planner uses source context, communicative intent, and retrieved examples to produce a detailed textual description of the target illustration.This description is formed through in-context learning from the retrieved demonstrations.
  • The Stylist synthesizes an aesthetic guideline from the reference collection and applies it to optimize each illustration description.The guideline covers color, shapes, lines, layout, typography, and icons.
  • The Visualizer converts descriptions into images while the Critic checks factual alignment and visual issues, then supplies targeted revisions.Their closed loop runs for T=3 rounds, with each revised description fed back for regeneration.
  • For statistical plots, the framework adapts the Visualizer and Critic to generate executable Python Matplotlib code and applies the same T=3-round refinement process.

4. Benchmark Construction

PaperBananaBench addresses the lack of dedicated benchmarks for automated diagram generation with a curated NeurIPS 2025 dataset and referenced VLM-based evaluation. Its protocol compares generated illustrations with human references across content and presentation dimensions.

  • PaperBananaBench targets the lack of rigorous evaluation for automated diagram generation and is curated from NeurIPS 2025 methodology diagrams.Dataset statistics are reported in Figure 3.
  • Data Curation: 2,000 papers were randomly sampled from 5,275 NeurIPS 2025 publications, and MinerU extracted methodology text, diagrams, and captions.
  • Data Curation: Filtering removed papers without methodology diagrams and restricted diagram aspect ratios to 1.5–2.5, yielding 610 candidates.The aspect-ratio restriction reflects typical landscape layouts and current image-generation support.
  • Data Curation: 292 test samples were curated for PaperBananaBench, with average source-context and caption lengths of 3,020.1 and 70.4 words.
  • Data Curation: The diagrams are categorized into Agent & Reasoning, Vision & Perception, Generative & Learning, and Science & Applications.Gemini-3-Pro assigns hybrid samples to their predominant category.
  • Data Curation: Human curation verifies methodology descriptions, captions, categorizations, and visual quality before retaining the final samples.Annotators filter overly simplistic, cluttered, or abstract designs.
  • Evaluation: VLM-as-a-Judge compares generated diagrams with human references across faithfulness, conciseness, readability, and aesthetics.These dimensions cover content and presentation perspectives.
  • Evaluation: Each referenced comparison labels the model as winning, losing, or tying, mapped to scores of 100, 0, or 50.

5. Experiments

Experiments compare PaperBanana with vanilla, few-shot, and Paper2Any baselines, evaluate its agent components, and assess extensions to statistical plots. PaperBanana shows broad gains over the Vanilla Nano-Banana-Pro baseline, while ablations reveal trade-offs among retrieval, styling, and critique.

  • Baseline Methods and Models: PaperBanana is compared with vanilla prompting, few-shot prompting using 10 examples, and Paper2Any, an agentic framework for high-level paper-idea diagrams.The experiments use Gemini-3-Pro as the default VLM backbone and Nano-Banana-Pro and GPT-Image-1.5 as image-generation models.
  • Main Results: PaperBanana consistently outperforms leading baselines on PaperBananaBench, according to the reported main-results comparison.The benchmark evaluates generated diagrams across faithfulness, conciseness, readability, and aesthetics.
  • Main Results: +2.8% Faithfulness, +37.2% Conciseness, +12.9% Readability, and +6.6% Aesthetics yield a +17.0% Overall gain over Vanilla Nano-Banana-Pro.These are the reported improvements for methodology diagram generation.
  • Main Results: Human judges reported a 72.7% / 20.7% / 6.6% win/tie/loss rate for PaperBanana versus vanilla Nano-Banana-Pro on 50 cases.The comparison used three human judges in a blind evaluation.
  • Ablation Study: The Stylist improves Conciseness by +17.5% and Aesthetics by +4.7% but reduces Faithfulness by -8.5%, while the Critic substantially recovers Faithfulness.Additional iterations further improve all metrics, balancing aesthetics and technical accuracy.
  • PaperBanana for Statistical Plots Generation: PaperBanana improves over vanilla Gemini-3-Pro for statistical plots by +1.4% Faithfulness, +5.0% Conciseness, +3.1% Readability, +4.0% Aesthetics, and +4.1% Overall.For statistical plots, the framework adopts executable code for visualization because numerical precision makes standard image generation unsuitable.

6. Discussion

PaperBanana is also evaluated for refining human-drawn diagrams and generating statistical plots, revealing both aesthetic gains and modality-specific trade-offs.

  • Coding vs Image Generation for Visualizing Statistical Plots: Image generation excels in statistical-plot Readability and Aesthetics but underperforms in Faithfulness and Conciseness.Manual inspection finds numerical hallucinations or element repetition especially in dense or complex plots.
  • Coding vs Image Generation for Visualizing Statistical Plots: Hybridly using image generation for sparse visualizations and code for dense plots may offer the best balance.Image models faithfully render sparse plots but struggle with dense or complex data.

7. Related Work

Related work includes code-based diagram generation, image-generation approaches, emerging benchmarks, and language-model methods for statistical visualization.

  • Diagram Generation: Code-based systems using TikZ or Python-PPT synthesize structured diagrams programmatically but face expressiveness limitations for intricate visual designs.These limitations concern visual elements prevalent in modern AI publications.
  • Image-Generation Approaches: Recent image-generation models provide a promising alternative by synthesizing high-fidelity, visually sophisticated figures.AutoFigure and AutoFigure-Edit transform scientific content into symbolic representations before rendering images with GPT-Image.
  • Evaluation Benchmarks: SridBench is the closest related benchmark for evaluating automated diagram generation from method sections and captions across scientific domains.The cited passage notes that results will be reported once SridBench is publicly available.
  • Statistical Visualization: Statistical visualization research progressed from LSTM-based JSON-to-Vega-Lite conversion to few-shot and zero-shot coding approaches using large language models.The passage identifies Dibia and Demiralp’s LSTM-based system as an early effort.

8. Conclusion

PaperBanana automates publication-ready academic illustrations through specialized agents and introduces a benchmark for rigorous evaluation. Experiments report improvements across core quality dimensions and extend the framework to diagrams and statistical plots.

  • Conclusion: PaperBanana orchestrates Retriever, Planner, Stylist, Visualizer, and Critic agents to transform scientific content into methodology diagrams and statistical plots.The framework is designed for publication-ready academic illustrations.
  • Conclusion: PaperBananaBench supports evaluation, while experiments report higher faithfulness, conciseness, readability, and aesthetics than existing baselines.The conclusion describes the benchmark as curated from top-tier AI conferences.

9. Limitations and Future Directions

The paper identifies limitations in editability, stylistic diversity, fine-grained faithfulness, evaluation, and single-output preference coverage, while proposing broader reference-driven extensions.

  • Towards Editable Academic Illustrations: Raster outputs are difficult to edit and do not fundamentally solve post-generation modification, even at 4K resolution.Proposed directions include image editing, element reconstruction, and GUI agents for vector-design software.
  • Towards Editable Academic Illustrations: Direct generation of fully editable vector graphics would require agents with exceptional perception, planning, and interaction capabilities.A GUI Agent operating professional vector design software is proposed as an advanced direction.
  • The Trade-off between Style Standardization and Diversity: A unified style guide ensures compliance with academic standards but inevitably reduces stylistic diversity.Future work could use dynamic style adaptation while maintaining professional rigor.
  • The Challenge of Fine-Grained Faithfulness: Fine-grained faithfulness errors include misaligned connectivity and incorrect arrow directions that current critics often fail to detect.The paper links closing this gap primarily to improved fine-grained visual perception in foundation VLMs.
  • Advancing Evaluation Paradigms: Reference-based VLM-as-a-Judge evaluation struggles with subtle structural correctness and fully aligning textual prompts with human aesthetic preferences.The paper suggests structure-based or rubric-based metrics and customized reward models as future directions.
  • Test-Time Scaling for Diverse Preferences: Generating a single output may not satisfy diverse aesthetic preferences because generative models are stochastic and aesthetic judgments are subjective.The proposed extension generates multiple candidates and selects among them using humans or preference models.
  • Extension to Broader Domains: The framework’s reference-driven paradigm is presented as a route toward broader domains by separating structural planning from aesthetic rendering without expensive domain-specific fine-tuning.Retrieval teaches what to generate, while automatic style summarization teaches how to generate.

Impact Statement

PaperBanana aims to democratize access to high-quality academic visual communication and reduce the manual effort required for diagram creation.

  • PaperBanana is designed to automate academic illustration generation.
  • The framework targets researchers who may lack professional design resources.
  • Reducing manual diagram-creation effort is intended to accelerate the scientific workflow.

A. Dedicated Case Studies

Case studies show PaperBanana improving diagram conciseness and visual style relative to vanilla image generation, while statistical-plot generation exposes a faithfulness trade-off and diagrams still fail on connectivity.

  • Diagram generation: PaperBanana produces more concise and aesthetically pleasing diagrams than vanilla Nano-Banana-Pro while maintaining faithfulness to the source context.
  • Diagram enhancement: Style guidelines improve human-drawn diagrams’ color schemes, typography, and graphical elements.
  • Statistical plots: Image generation produces more visually appealing statistical plots but introduces numerical hallucination and element-repetition errors.
  • Failure analysis: PaperBanana’s primary failure mode is connectivity errors, including redundant connections and mismatched source-target nodes.

B. Human Evaluation Setup

The study uses human evaluations to validate automated scoring and compare PaperBanana with a strong baseline across four shared quality dimensions.

  • Evaluation protocol: Both human evaluations use faithfulness, conciseness, readability, and aesthetics with the same detailed rubrics as the VLM judge.
  • Evaluation protocol: VLM-judge validation samples 50 cases, split evenly between the Vanilla baseline and PaperBanana, for assessment by two experienced researchers.
  • Blind comparison: The blind A/B test compares PaperBanana with Vanilla Nano-Banana-Pro using 50 cases and three experienced researchers.

C. Implementation Details

Implementation details cover diagram categorization, generation settings, candidate selection, and construction of a balanced statistical-plot test set from ChartMimic.

  • Diagram categorization: Diagrams are categorized into four classes based on visual topology and content.
  • Generation settings: Experiments use generation temperature 1 and match generated-image aspect ratios to human-drawn counterparts.
  • Candidate selection: Paper figures marked “[Generated by]” are produced entirely by PaperBanana, with multiple candidates generated and the best manually selected.
  • Statistical-plot test set: The statistical-plot test set consolidates ChartMimic into seven common categories and samples 480 plots with an increased proportion of difficult cases.
  • Statistical-plot test set: The seven plot categories include bar, line, tree and pie, scatter, heatmap, radar, and miscellaneous charts.

E. Textual Description of our Methodology Diagram

The diagram presents PaperBanana as a left-to-right workflow that moves from source inputs through planning and styling to visualization and iterative refinement. Its visual language separates planning from refinement while using reference-driven aesthetics, differentiated connectors, and structured shapes.

  • Textual Description of our Methodology Diagram: PaperBanana’s workflow begins with Source Context (S) and Communicative Intent (C), which enter the Linear Planning Phase.The inputs merge into the main flow line entering the first phase.
  • Textual Description of our Methodology Diagram: The Retriever Agent uses the inputs and Reference Set (R) to produce Relevant Examples (E), which inform the Planner Agent.The Planner also receives a direct flow from the main inputs.
  • Textual Description of our Methodology Diagram: The Planner produces an Initial Description (P), while the Stylist Agent combines it with Aesthetic Guidelines (G) to output an Optimized Description (P*).The guidelines are synthesized from the reference collection across color, shapes, lines, layout, typography, and icons.
  • Textual Description of our Methodology Diagram: The Visualizer Agent renders the Optimized Description, and the Critic Agent returns a Refined Description through a T=3 Rounds iterative loop.The critic receives the Generated Image (I_t), while the visualizer accepts both the optimized and refined descriptions.
  • Textual Description of our Methodology Diagram: The final output is a polished scientific illustration labeled Final Illustration (I_T), with blue planning accents, orange refinement accents, and dashed gray secondary inputs.The diagram uses a clean white background and divides the workflow into Linear Planning Phase and Iterative Refinement Loop regions.
  • Style Guide for Methodology Diagrams: The style guide favors softened geometry, light pastel grouping, curved feedback connectors, and distinct typography and icon conventions for academic diagrams.Rounded rectangles represent generic process nodes, cylinders are reserved for databases or memory, and dashed lines indicate auxiliary or logical flows.
Loading 2601.23265v2…