PaperlayerView exact revision on arXiv

Source-linked AI summary

Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers

Wei Pang, Kevin Qinghong Lin, Xiangru Jian, Xi He, Philip Torr

arXiv:2505.21497v2cs.CVcs.AIcs.CLcs.MA

TL;DR

Academic poster generation must compress long, interleaved scientific papers into a single coherent page, motivating a benchmark and a visual-in-the-loop multi-agent pipeline. The proposed framework structures paper assets, plans layouts, and iteratively refines panels; evaluations show open-source variants outperform GPT-4o-based multi-agent systems on nearly all metrics while using 87% fewer tokens.

  • Problem

    Poster generation requires compressing long-context papers containing interleaved text and visuals into one visually coherent page while preserving core content.

  • Method

    PosterAgent uses a Parser asset library, a Planner for text-visual alignment and layout, and a Painter-Commenter loop for panel rendering and visual refinement.

  • Results

    Open-source PosterAgent variants outperform existing GPT-4o-based multi-agent approaches on nearly all metrics while consuming 87% fewer tokens.

  • Takeaways & Limitations

    Paper2Poster establishes systematic evaluation of poster generation, including whether posters communicate core paper knowledge through PaperQuiz.

  • Takeaways & Limitations

    Sequential panel refinement is an efficiency bottleneck, making poster creation take approximately 4.5 minutes per document and limiting large-scale or interactive workflows.

Abstract

from arXiv · show

Academic poster generation is a crucial yet challenging task in scientific communication, requiring the compression of long-context interleaved documents into a single, visually coherent page. To address this challenge, we introduce the first benchmark and metric suite for poster generation, which pairs recent conference papers with author-designed posters and evaluates outputs on (i)Visual Quality-semantic alignment with human posters, (ii)Textual Coherence-language fluency, (iii)Holistic Assessment-six fine-grained aesthetic and informational criteria scored by a VLM-as-judge, and notably (iv)PaperQuiz-the poster's ability to convey core paper content as measured by VLMs answering generated quizzes. Building on this benchmark, we propose PosterAgent, a top-down, visual-in-the-loop multi-agent pipeline: the (a)Parser distills the paper into a structured asset library; the (b)Planner aligns text-visual pairs into a binary-tree layout that preserves reading order and spatial balance; and the (c)Painter-Commenter loop refines each panel by executing rendering code and using VLM feedback to eliminate overflow and ensure alignment. In our comprehensive evaluation, we find that GPT-4o outputs-though visually appealing at first glance-often exhibit noisy text and poor PaperQuiz scores, and we find that reader engagement is the primary aesthetic bottleneck, as human-designed posters rely largely on visual semantics to convey meaning. Our fully open-source variants (e.g. based on the Qwen-2.5 series) outperform existing 4o-driven multi-agent systems across nearly all metrics, while using 87% fewer tokens. It transforms a 22-page paper into a finalized yet editable .pptx poster - all for just $0.005. These findings chart clear directions for the next generation of fully automated poster-generation models. The code and datasets are available at https://github.com/Paper2Poster/Paper2Poster.

1 Introduction

Scientific poster generation requires compressing long, interleaved papers into one visually coherent page, while Paper2Poster and PosterAgent address evaluation and creation respectively.

  • Poster creation must condense long multimodal papers into a single page while interleaving text and graphics for rapid comprehension.
  • Paper2Poster introduces a benchmark and metric suite for systematic comparison of agent-generated and author-designed posters.
  • PaperQuiz evaluates whether posters convey core paper content by having VLMs answer questions derived from the source paper.
  • PosterAgent globally organizes papers into structured visual posters and then performs panel-level refinement with visual feedback.
  • GPT-4o outputs can appear visually appealing but exhibit noisy text and poor PaperQuiz performance, whereas open-source variants surpass GPT-4o-based multi-agent systems on nearly all metrics using 87% fewer tokens.

2 Related Work

Poster generation remains an underexplored multimodal design problem distinct from text-to-slide and pixel-level image-generation approaches.

  • Diffusion-based text-rich image generation can synthesize detailed visuals, but embedded pixel-level text often remains blurry and difficult to control.
  • Unlike slide automation, poster generation requires compressing an entire paper into one coherent visual page rather than distributing content across multiple slides.

3 Paper2Poster Benchmark

Paper2Poster frames poster generation as multimodal context compression and evaluates both visual presentation and how effectively posters communicate paper knowledge.

  • Task Definition: The task generates a single-page poster from interleaved paper text, figures, and tables while preserving core content and spatial coherence.
  • Data Curation: Paper2Poster contains 100 paper-poster pairs spanning 280 topics, with papers averaging 20370.3 tokens and 22.59 figures versus posters averaging 1416.2 tokens and 8.7 figures.
  • Evaluation Metrics: The benchmark evaluates visual quality, textual coherence, VLM-as-judge scores, and PaperQuiz content conveyance.
  • Evaluation Metrics: Visual quality compares generated and author-designed posters using CLIP-based visual similarity, while textual coherence uses Llama-2-7b-hf perplexity.
  • Evaluation Metrics: PaperQuiz generates 100 questions per paper and has six VLMs answer them from poster images, covering verbatim and interpretive comprehension.
  • Evaluation Metrics: The length-adjusted PaperQuiz score uses poster length L relative to the median human-poster length W, rewarding concise posters without unlimited brevity benefits.

4 PosterAgent

PosterAgent uses top-down organization followed by local visual refinement to construct editable academic posters from papers.

  • Overview: PosterAgent first restructures the full document into concise sections, then refines individual panels for fine-grained control.
  • 4.1 Parser: global organization: The Parser converts the PDF into a structured asset library containing section-level text synopses and extracted figures or tables.
  • Planner: The Planner selects content iteratively, matches visual assets to relevant sections, and constructs layouts using hierarchical constraints and estimated content length.
  • Painter–Commenter: The Painter turns section-figure pairs into concise bullet points and executable presentation code, rendering each panel for inspection.
  • Painter–Commenter: The Commenter uses zoomed panel images and in-context examples to flag overflow or whitespace and guide Painter revisions.

5 Experiments

Experiments compare oracle, end-to-end, multi-agent, and PosterAgent systems across visual, textual, holistic, comprehension, and efficiency measures. PosterAgent generally improves information quality and PaperQuiz performance, while engagement remains a central weakness and GPT-4o-based commenting improves visual refinement.

  • Baselines and Settings: The evaluation compares oracle posters, GPT-4o end-to-end systems, multi-agent workflows, and PosterAgent variants using different language and vision backbones.PosterAgent-4o uses GPT-4o internally, whereas PosterAgent-Qwen uses Qwen-2.5-7B and Qwen-2.5-VL-7B.
  • Visual Quality & Text Coherence: 4o-Image achieves the highest visual similarity but the worst perplexity, indicating visually appealing posters can contain noisy or incoherent text.The original paper has the strongest textual coherence, while the author-designed poster also has relatively high perplexity.
  • Visual Quality & Text Coherence: PosterAgent achieves the highest figure relevance and ranks second in visual similarity, closely following the human-designed poster.The result is attributed primarily to visual-semantic-aware asset-library construction and asset matching.
  • VLM as Judge Metrics: PosterAgent-4o achieves an average VLM-as-Judge score of 3.72, comparable to human-designed posters.PosterAgent variants using GPT-4o as visual commenter outperform those using Qwen2.5-VL-7B, particularly for preventing text overflow.
  • VLM as Judge Metrics: Engagement is the primary bottleneck: all variants score below 3, while most variants achieve good Information scores.All PosterAgent variants surpass baselines in information quality, including versions using Qwen-2.5-7B.
  • PaperQuiz: PosterQuiz rewards concise context compression: GT Poster performs best with brevity penalties, PosterAgent variants consistently achieve the best scores, and stronger closed-source readers score higher.Verbatim questions are generally harder than broader interpretive questions, and reader-model scores align with human evaluation on the tested subset.
  • Qualitative Analysis: PosterAgent produces structurally coherent and readable posters with the highest scores while using fewer words than 4o-HTML and OWL, though human-designed versions remain stronger.Figure 8 contrasts pixel-rendering text impairment, text-dense blog-like layouts, and missing panels from competing methods.
  • Efficiency: PosterAgent uses 101.1K tokens with 4o and 47.6K with Qwen, reducing cost by 60%–87% compared to OWL-4o.The reported costs are $0.55 for 4o and $0.0045 for Qwen per poster; parallel panel generation reduces runtime by 40.7%.

6 Conclusions

Paper2Poster introduces a benchmark for academic poster generation and PosterAgent for producing posters from long-context, layout-sensitive papers. The framework combines structured parsing, hierarchical planning, and visual feedback to improve quality and efficiency while narrowing the gap with human-designed posters.

  • 6 Conclusions: Paper2Poster is a benchmark for evaluating poster generation from academic papers and exposes challenges in long-context, layout-sensitive tasks.The benchmark is paired with author-designed posters and supports systematic evaluation of generated outputs.
  • 6 Conclusions: PosterAgent combines structured parsing, hierarchical planning, and visual feedback to enhance poster-generation quality.The framework is presented as a practical approach for transforming paper content into coherent poster layouts.
  • 6 Conclusions: PosterAgent narrows the performance gap with human-designed posters and establishes an efficiency standard for practical, scalable scientific communication.The conclusion characterizes the approach as practical and scalable within the evaluated poster-generation setting.

A Limitations and Future Work

The paper identifies sequential panel refinement as its main efficiency bottleneck and proposes panel-level parallelism, while also suggesting external knowledge and human–AI collaboration as future directions.

  • Limitations: Sequential panel refinements make poster creation take approximately 4.5 minutes per document, limiting large-scale or interactive workflows.Each panel’s generate–revise cycle is independent but processed serially to preserve modularity and output quality.
  • Future Work: Panel-level parallelism could reduce runtime and improve scalability for batch generation and real-time editing.The paper identifies parallelizing independent panel refinements as a clear future-work direction.
  • Future Work: Future work could incorporate external knowledge, including community feedback and institutional or conference assets.Examples include OpenReview comments, social-media reactions, institutional icons, and conference logos.
  • Future Work: The authors also propose human–AI collaboration in which an agent drafts a poster, solicits user feedback, and iteratively refines the output.This workflow is intended to meet user requirements through iterative interaction.

C Ablation Study

The ablation study isolates binary-tree layout planning, visual commenting, and in-context examples, showing progressively better layouts as these components are combined.

  • Setup: The study evaluates binary-tree layout, a visual commenter, and in-context examples as three key PosterAgent design choices.All variants use PosterAgent-4o while keeping other components unchanged where possible.
  • Direct: Direct layout generation often produces structurally compromised or blog-style posters with missing text boxes, blank areas, and incorrect figure aspect ratios.These defects are reported across Figures 17a–21a, including missing text boxes in Figures 17a–20a and aspect-ratio failure in Figure 20a.
  • Tree: Removing the commenter from the tree layout causes severe text overflow and overlaps across all test cases.Overflow spills outside designated textboxes or section panels and overlaps text or visual elements.
  • Tree + Commenter: Adding the commenter improves results, but without in-context examples it leaves residual overflow and substantial unused white space.The commenter fails to flag some white-space problems in Figures 19c and 20c without in-context guidance.
  • Tree + Commenter + IC: The full Tree + Commenter + IC system achieves the best results among the ablation variants.The complete configuration combines the tree layout, commenter, and in-context examples.
  • Efficiency: PosterAgent-Qwen-2.5-7B uses 47.6K tokens and 192.0 seconds, compared with PPTAgent’s 255.7K tokens and 230.7 seconds, while maintaining output quality.The comparison illustrates a balance between efficiency and poster quality.
  • Efficiency: Parallelized content generation reduces runtime from 176.69s to 54.16s, a 69.3% reduction, bringing overall runtime to 166.80s.The parallelized version improves total runtime by 40.7% over the sequential implementation.
  • Backbone Analysis: PosterAgent variants show stable performance across backbone combinations, with PaperQuiz scores spanning 114.09 to 118.25.The approximately four-point spread indicates limited sensitivity to the specific LLM–VLM pairing.

E.4 Additional Backbone Evaluations

Additional backbone evaluations show that PosterAgent’s gains over 4o-Image persist across models, while the hybrid code-render-feedback design supports editable, from-scratch generation.

  • Results: PosterAgent variants substantially outperform the 4o-Image baseline across nearly all metrics, with VLM-as-Judge scores of 3.56–3.72 versus 2.33.This corresponds to a +1.23–1.39 absolute improvement, or approximately 53–60% relative improvement.
  • Results: Visual similarity remains high and stable at 0.75–0.78 across backbones, with PosterAgent-Qwen-72B reaching 0.78.PosterAgent-o4-mini achieves the highest raw and augmented PaperQuiz scores among the additional backbones.
  • Results: Information scores cluster at 3.83–3.87 versus 1.77 for the baseline, while aesthetic scores reach 3.26–3.58 versus 2.90.These results indicate improvements in both informativeness and visual quality across backbones.
  • Results: The additional evaluations support strong generalization across reasoning-oriented closed-source models and larger open-source alternatives.Performance remains strong despite changes in backbone choice.
  • Design Rationale: PosterAgent generates layout code, renders it to PPTX, and uses visual feedback to iteratively refine layout and content.This hybrid design combines precise layout control with rendered-poster inspection.
  • Design Rationale: The approach provides precise control, editable outputs, true from-scratch generation, efficient global changes, and feedback-driven refinement.These properties address the combined demands of content accuracy and visual layout quality.

E.6 VLM-as-Judge Robustness Analysis

Repeated VLM-as-Judge evaluations are highly stable, supporting single-run model comparisons while recommending periodic multi-run audits.

  • Stability: Five independent evaluations across the 100-sample dataset found standard deviations below 0.024 across all metrics.The overall average had especially low variance, with std = 0.005.
  • Stability: Most measures have a coefficient of variation below 1%, and the confidence intervals are extremely narrow.The results indicate high consistency across evaluation runs.
  • Implication: The observed stability supports using a single evaluation run for practical model comparison.The paper recommends periodic multi-run audits to verify continued metric stability.

F.1 Visual Quality Metrics

The evaluation suite measures poster quality through visual similarity, figure relevance, textual coherence, and VLM-based holistic scoring. It combines image-level semantic alignment with criteria covering graphics, layout, engagement, and readability.

  • Visual Similarity: Visual Similarity compares generated and ground-truth poster images using cosine similarity between CLIP image embeddings.The measure targets semantic alignment and content fidelity between individual poster images rather than distributional statistics.
  • Figure Relevance: Figure Relevance evaluates whether generated figures are contextually appropriate for their corresponding paper sections.It compares CLIP image embeddings of figure crops with CLIP text embeddings of section text.
  • Textual Coherence: Textual Coherence is measured with full-sequence perplexity under the Llama-2-7b-hf language model.Lower perplexity indicates more predictable and coherent poster text.
  • Holistic Assessment: Layout Balance assesses arrangement, alignment, spacing, and reading path, while Engagement assesses design elements that capture and sustain attention.The scoring guidance emphasizes readability, visual flow, typography, color harmony, and effective use of white space.

G Human Evaluation Protocol

Human evaluators score posters using six specified criteria and accompanying text-based instructions. The protocol requires reading the poster and applying the criteria consistently.

  • Evaluation Procedure: Each human evaluator receives a poster and six text files describing the evaluation criteria.The evaluator reads the poster before assigning criterion-level scores.
  • Evaluation Procedure: Evaluators provide scores according to the six criteria specified in the accompanying instructions.The protocol explicitly directs evaluators to read the poster and score it using those criteria.
  • Evaluation Procedure: The human-evaluation criteria are the same as those outlined for PaperQuiz in Appendix F.4.This links the human protocol to the PaperQuiz criterion definitions.

H Error Analysis

Across the evaluated poster-generation pipelines, errors cluster around text integrity, visual and layout quality, missing visuals, and overflow. The examples show failures ranging from garbled text and truncation to absent figures and content spilling beyond panel boundaries.

  • Four recurring failure modes are identified: text integrity issues, visual or layout flaws, missing visuals, and overflow issues.These failure classes are observed across 4o-Image, 4o-HTML, OWL-4o, PPTAgent, and PosterAgent.
  • Text Integrity Issues: 4o-Image often produces garbled or unreadable text, while PPTAgent may leave placeholders intact or partly overwritten.The contrast reflects different text-rendering failures in pixel-level synthesis and template-based generation.
  • Visual / Layout Flaws: 4o-Image can truncate content or hallucinate figures, while PPTAgent may shrink figures excessively or leave substantial blank regions.These failures arise in outputs lacking robust visual feedback or despite predefined templates.
  • Missing Visuals: OWL-4o can fail to complete figure retrieval and insertion, leaving posters purely textual, while 4o-HTML is text-only by design.Both pipelines therefore produce posters without visual figures.
  • Overflow Issues: PPTAgent and PosterAgent sometimes produce text beyond panel boundaries because PPTX lacks strict parent–child containment.Overflow is relatively more pronounced in the Qwen variant, whose weaker visual grounding makes feedback less reliable.
Loading 2505.21497v2…