PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoPresent: Designing Structured Visuals from Scratch

Jiaxin Ge, Zora Zhiruo Wang, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten Sap, Alane Suhr, Daniel Fried, Graham Neubig, Trevor Darrell

arXiv:2501.00912v2cs.CVcs.CL

TL;DR

The paper addresses automated generation of structured presentation slides from natural-language instructions, a task requiring both content creation and visual planning. It introduces SlidesBench, program-generation methods, and AutoPresent, finding that AutoPresent with SlidesLib approaches GPT-4o and that iterative refinement improves slide quality.

  • Problem

    Automated slide generation lacks established tools for evaluating whether generated presentations are well-structured and high-quality.

  • Method

    The paper builds SlidesBench, generates editable slides through executable programs, trains AutoPresent on instruction–code pairs, and evaluates iterative self-refinement.

  • Results

    AutoPresent with SlidesLib achieves performance comparable to GPT-4o, while iterative refinement improves performance across all three challenges.

  • Takeaways & Limitations

    Programmatic generation supports user-interactable slide formats, and self-refinement improves content layout and detailed color and sizing controls.

  • Takeaways & Limitations

    The work focuses on single-slide generation and produces full slide code in a single pass without iterative design workflows.

Abstract

from arXiv · show

Designing structured visuals such as presentation slides is essential for communicative needs, necessitating both content creation and visual planning skills. In this work, we tackle the challenge of automated slide generation, where models produce slide presentations from natural language (NL) instructions. We first introduce the SlidesBench benchmark, the first benchmark for slide generation with 7k training and 585 testing examples derived from 310 slide decks across 10 domains. SlidesBench supports evaluations that are (i)reference-based to measure similarity to a target slide, and (ii)reference-free to measure the design quality of generated slides alone. We benchmark end-to-end image generation and program generation methods with a variety of models, and find that programmatic methods produce higher-quality slides in user-interactable formats. Built on the success of program generation, we create AutoPresent, an 8B Llama-based model trained on 7k pairs of instructions paired with code for slide generation, and achieve results comparable to the closed-source model GPT-4o. We further explore iterative design refinement where the model is tasked to self-refine its own output, and we found that this process improves the slide's quality. We hope that our work will provide a basis for future work on generating structured visuals.

1. Introduction

The paper frames natural-language-to-slide generation as a difficult structured-visual task requiring both content creation and visual planning. It introduces SlidesBench, programmatic generation methods, and AutoPresent to address evaluation and generation quality.

  • Natural-language slide generation requires creating content while planning layouts that communicate insights clearly.
  • SlidesBench provides 7k training and 585 testing examples from 310 slide decks across 10 domains, with reference-based and reference-free evaluation.
  • Program generation produces editable PPTX slides, but small models often fail to generate executable code and GPT-4o remains below human design quality.
  • AutoPresent is an open-source 8B model fine-tuned on SlidesBench that approaches GPT-4o, while SlidesLib simplifies program generation and improves model performance.

2. SLIDESBENCH

SlidesBench evaluates natural-language slide generation across three instruction settings with different levels of design freedom. Its dataset combines curated slide decks, generated annotations, and reference slides.

  • 2. SLIDESBENCH: SlidesBench pairs each natural-language slide instruction with a PPTX reference and tests three user-input scenarios of varying difficulty.
  • 2.1. Slides Data Collection: The benchmark draws from slide decks across 10 domains, selecting visually structured decks with extractable media for training and testing.
  • 2.2. Three Task Setups: Each example includes detailed instructions with images, detailed instructions only, and high-level instructions, evaluated with reference-based and reference-free metrics.
  • 2.3. Example Annotation: Annotations combine human-written examples with model-generated instructions, followed by manual refinement for the test set.
  • 2.3. Example Annotation: The instruction sets average 115.6, 118.3, and 26.6 words across the three settings, with an average of 1.1, 0.0, and 0.0 provided images.

3. Evaluation Metrics

The evaluation combines reference-based instruction-following measures, reference-free design-quality measures, and executability for code-generated slides. This separates similarity to a target from standalone visual quality while accounting for invalid programs.

  • 3.1. Reference-Based Metrics: Reference-based metrics measure element matching, content similarity, color similarity, and positional similarity between generated and reference slides.
  • 3.1. Reference-Based Metrics: Reference-based similarity can be lowered by inaccurate element matching, which may pair semantically different textboxes and thereby distort text, color, or position scores.
  • 3.2. Reference-Free Metrics: Reference-free metrics assess text, images, layout, and other design principles because a good slide may differ substantially from its reference.
  • 3.2. Reference-Free Metrics: Reference-free scores agree strongly with human ratings, yielding intraclass correlation coefficients of 73.8%–85.3% across four metrics.
  • 3.3. Executability: Code-generation methods report scores on executable slides and assign zeros to non-executing slides in overall scores.

4. Method

The method generates slides by converting natural-language instructions into executable programs, optionally using SlidesLib to shorten and simplify code. AutoPresent is trained on instruction–program pairs, and iterative refinement lets models revise rendered slides.

  • 4.1. Slides via NL-to-Code Generation: The core pipeline generates a Python program from a natural-language instruction and executes it to produce an editable PPTX slide.
  • 4.1. Slides via NL-to-Code Generation: SlidesLib reduces average program length from 170 lines to 13 lines by providing high-level interfaces for common slide and image operations.
  • 4.2. AutoPresent: AutoPresent is an open-source 8B model trained from instruction–program pairs constructed from SlidesBench training slides.
  • 4.2. AutoPresent: The training data contains four 7k-example variants combining three instruction types with basic Python or SlidesLib programs.
  • 4.3. Iterative Refinement: Iterative refinement gives GPT-4o the instruction, first-pass program, and rendered slide so it can self-refine the generated design.

5. Experiments and Results

The experiments compare program generation, end-to-end image generation, and iterative refinement across instruction settings. Programmatic approaches, especially AutoPresent, produce stronger structured slides, while refinement improves performance across all scenarios.

  • Experimental Setup: The benchmark compares code-generation models, end-to-end image-generation systems, and human-created reference slides across multiple instruction settings.Code-generation evaluation samples three responses and selects the first successfully executing program; image-generation methods are evaluated without execution failures.
  • Quantitative Results: 49.9–55.0 points separate smaller open-source models from GPT-4o when images are provided, shrinking to 22.2–34.6 points without supplied visuals.The reported gaps compare LLAMA 3.1 and LLAVA with GPT-4o across detailed-instruction settings with and without images.
  • Quantitative Results: SLIDESLIB mitigates performance degradation as instructions become less specified and improves outcomes across detailed-without-images and high-level settings.Without helper functions, LLAVA exceeds LLAMA by 5.1–7.5 points; library-augmented LLAMA exceeds LLAVA by 12.1–26.2 points.
  • Qualitative Case Study: End-to-end image generation produces more creative designs but often invalid text, while weaker code-generation models produce overlapping or off-canvas elements.AutoPresent instead produces appropriate layouts, follows instructions more closely, and avoids undesirable overlaps.
  • Perceptual Evaluation: AUTOPRESENT and GPT-4o are significantly preferred over LLAMA, but all three remain below human-designed slides in perceptual evaluation.GPT-4o and AutoPresent do not differ significantly with images; without images, AutoPresent is slightly worse than GPT-4o.
  • Result after Iterative Refinement: Refinement improves performance across all three challenges, with the first iteration usually providing the largest improvement.Further iterations often increase scores, and examples show improvements in content layout and controls over coloring and sizing.

6. Related Work

Prior work applies language and vision agents to tasks such as web navigation, software engineering, and web development. Program generation has produced structured visuals, but complex editable presentation slides remain insufficiently addressed.

  • LLM- and VLM-based agents have been widely adopted for web navigation, software engineering, and web development.
  • Program-generation methods support structured outputs such as TikZ figures, SVGs, posters, and user interfaces.
  • These methods often require detailed inputs or target simple figure types, leaving complex editable slide generation from scratch unresolved.

7. Conclusion and Limitations

The work introduces SLIDESBENCH and evaluates AUTOPRESENT with SLIDESLIB for automated structured-visual generation. It demonstrates self-refinement effectiveness while identifying scope boundaries and future directions.

  • Contributions: SLIDESBENCH is introduced as the first benchmark for automatic slide generation, with reference-based and reference-free evaluation metrics.The benchmark covers 10 domains, including art, business, and technology, and its domain statistics are shown in Figure 6.
  • Contributions: AUTOPRESENT with SLIDESLIB achieves comparable performance with GPT-4O, while iterative self-refinement shows certain effectiveness.The conclusion presents these as the main empirical findings of the work.
  • Limitations and future work: The study focuses on single-slide generation and produces full slide code in a single pass without iterative design workflows.The authors identify full-deck generation, gradual interactive generation, animations, attention capture, and information clarity as future directions.

B.1. SLIDESLIB Implementation

SLIDESLIB provides documented basic and image-oriented functions, with example programs demonstrating how multiple functions can generate slides. Common generated actions are adding text, images, and titles.

  • Function documentation: Figure 7 documents SLIDESLIB's basic functions, while Figure 8 documents its image-oriented functions.The implementation section distinguishes these two functional groups.
  • Examples: Figure 9 presents two example programs that use multiple SLIDESLIB functions to produce slides.The examples illustrate composing library operations into slide-generation programs.
  • Action analysis: 36.3% of reported actions are add text, 20.3% are add image, and 13.5% are add title.These are the average action frequencies across GPT-4o, AUTOPRESENT, and Llama-3.1 in all three scenarios.

C. Training Details for AUTOPRESENT

The appendix summarizes AUTOPRESENT's training configuration and describes an auto-refinement procedure that lets the model correct previously generated slide code using rendered output.

  • Training configuration: Table 7 summarizes AUTOPRESENT's training details, including its LoRA and Trainer parameters.The supplied passage points to the table rather than listing the parameter values.
  • Auto-refinement: Auto-refinement provides the instruction, in-context examples, previous generated code, and a rendered slide snapshot to the model for code correction.The model is asked to correct the code after seeing both the prior program and its executed visual result.
  • Auto-refinement: The refinement procedure operates by revising code after execution produces a slide snapshot.This connects the generated program to a subsequent visual correction step.

E. Detailed Results

The detailed-results materials define the reference-based and reference-free evaluation reporting and document SLIDESLIB operations for constructing slides, adding images, and controlling appearance.

  • Evaluation reporting: Evaluation metrics are reported both as averages over all slides and on successfully rendered slides weighted by execution success.The supplied passages distinguish unweighted reporting from reporting conditioned on successful rendering.
  • Basic slide functions: add_title inserts a title with configurable text, font size, font color, and optional background color.The function returns the slide object with the title added.
  • Basic slide functions: add_text places a configurable text box at specified coordinates, with options for font size, boldness, colors, and automatic sizing.The documented coordinate format is [left, top, width, height] in inches.
  • Image and appearance functions: add_image inserts an image from a provided path at specified coordinates and dimensions, while set_background_color changes the slide background using an RGB tuple.The image and background functions return modified slide objects.
  • Evaluation reporting: Tables 8, 10, and 12 report metrics down-weighted by execution success, while Tables 9, 11, and 13 report metrics without that down-weighting.The passages identify both reference-based and reference-free metrics in the corresponding table groups.
  • Image-oriented functions: SLIDESLIB also supports Google search screenshots, image search, and diffusion-model image generation for slide assets.The documented functions save the resulting screenshot or image to a specified path.

F. Perceptual Analysis

The perceptual analysis evaluates generated slides by asking users to score their preference for real-presentation use on a 1–5 scale. The section also documents prompts and code examples for generating and refining slides.

  • Perceptual evaluation: Human evaluators score each slide from 1–5 based on their preference for using it in a real presentation.A score of 5 is best and 1 is worst.
  • Generation examples: The documented generation examples use Python APIs to create slides, add text and images, and save the resulting presentation file.Examples include a slide with a search-result screenshot and an interior-design slide with a generated background.
  • Auto-refinement: The auto-refinement prompt supplies the model with APIs, instructions, a previous slide, and previous code so it can rewrite the code to improve the design.The task explicitly asks the model to modify existing code based on observed drawbacks in the slide image.
  • Perceptual evaluation: The evaluation asks participants to read each slide’s content carefully before ranking it.
Loading 2501.00912v2…