Source-linked AI summary
PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation
Xin-Sheng Chen, Jiayu Zhu, Pei-lin Li, Hanzheng Wang, Shuojin Yang, Meng-Hao Guo
TL;DR
Slide-generation evaluation remains difficult because existing approaches are often coarse, holistic, and insufficiently sensitive to instance-specific content and requirements. PresentBench addresses this gap with a real-world benchmark of curated instances, background materials, and atomic checklist rubrics, and its experiments show stronger human alignment while identifying NotebookLM as a leading system.
Problem
Existing slide-generation evaluations often use coarse holistic judgments that inadequately verify instance-specific content, requirements, and multimodal quality.
Method
PresentBench evaluates automated slide generation with 238 curated instances, authentic background materials, task-specific instructions, and more than 50 atomic checklist items per instance.
Results
PresentBench provides more accurate evaluations, substantially stronger agreement with human preferences, and clearer performance differences, with NotebookLM consistently outperforming other systems.
Takeaways & Limitations
Fine-grained, instance-specific rubrics make slide-generation evaluation more diagnostic and better aligned with human preferences than existing methods.
Abstract
from arXiv · showhide
Slides serve as a critical medium for conveying information in presentation-oriented scenarios such as academia, education, and business. Despite their importance, creating high-quality slide decks remains time-consuming and cognitively demanding. Recent advances in generative models, such as Nano Banana Pro, have made automated slide generation increasingly feasible. However, existing evaluations of slide generation are often coarse-grained and rely on holistic judgments, making it difficult to accurately assess model capabilities or track meaningful advances in the field. In practice, the lack of fine-grained, verifiable evaluation criteria poses a critical bottleneck for both research and real-world deployment. In this paper, we propose PresentBench, a fine-grained, rubric-based benchmark for evaluating automated real-world slide generation. It contains 238 evaluation instances, each supplemented with background materials required for slide creation. Moreover, we manually design an average of 54.1 checklist items per instance, each formulated as a binary question, to enable fine-grained, instance-specific evaluation of the generated slide decks. Extensive experiments show that PresentBench provides more reliable evaluation results than existing methods, and exhibits significantly stronger alignment with human preferences. Furthermore, our benchmark reveals that NotebookLM significantly outperforms other slide generation methods, highlighting substantial recent progress in this domain.
1 Introduction
PresentBench addresses limitations in slide-generation evaluation by combining realistic, instance-specific tasks with fine-grained rubrics. Experiments indicate that it provides more human-aligned assessments and distinguishes system performance, including NotebookLM’s strong results.
- Motivation: Slide generation is difficult to evaluate because it jointly requires accurate content synthesis, coherent structure, visual design, factual correctness, and quantitative fidelity.These multimodal and tightly coupled requirements challenge static metrics and coarse holistic judging.
- Motivation: Existing benchmarks often evaluate isolated subtasks, omit concrete background materials, or apply the same coarse questions across instances.These settings can mismatch real-world end-to-end authoring and overlook instance-specific requirements.
- PresentBench: 238 expert-curated evaluation instances span five application domains and pair slide-generation tasks with authentic background materials and explicit constraints.Instructions cover objectives, structure, content fidelity, visual layout, quantitative accuracy, and audience definition.
- PresentBench: More than 50 atomic checklist items per instance assess presentation fundamentals, visual design, completeness, correctness, and fidelity.The checklist enables instance-specific, verifiable, fine-grained assessment.
- Results: PresentBench yields more accurate evaluations, stronger agreement with human preferences, and clearer performance differences than existing methods.NotebookLM consistently outperforms other slide-generation systems across scenarios.
2 Related Work
Prior slide-generation evaluation relies heavily on holistic LLM judging and mixes reference-based and reference-free approaches. Related work introduces finer-grained evaluation ideas elsewhere, but slide generation still lacks a similarly systematic diagnostic standard.
- Existing slide-generation evaluation: PPTEval uses multimodal LLMs to score content quality, visual design, and coherence, while SlidesBench combines reference-based and reference-free assessment.Both illustrate the field’s reliance on broad evaluation frameworks for slide quality.
- Fine-grained evaluation beyond slides: GenExam, MMMG, and OneIG-Bench improve fine-grained diagnosis for text-to-image or knowledge-image generation through explicit criteria, graphs, or decomposed capability dimensions.These methods provide item-level or multidimensional analysis outside slide generation.
- Open gap: Slide generation still lacks a systematic evaluation standard with comparably strong fine-grained diagnostic capability.This gap limits in-depth analysis and reliable comparison of slide-generation systems.
3 PresentBench
PresentBench evaluates end-to-end slide generation using diverse real-world materials, constrained instance-specific instructions, and fine-grained checklist-based scoring. Its workflow links benchmark construction, slide generation, and automated item-level assessment to produce verifiable overall scores.
- Dataset Construction: 238 evaluation instances span five thematic categories and use diverse real-world slide-related materials from authoritative public sources.The categories are Academia, Education, Economics, Talk, and Advertising; materials include research papers, textbooks, economic reports, speeches, talks, and brochures.
- Dataset Construction: Experts manually inspect and filter sources for correctness, relevance, suitability, difficulty, accessibility, language consistency, and usage terms.The curation excludes overly trivial instances and primarily uses English materials with a small Chinese subset.
- Instance-Specific Instructions: Each instance uses highly constrained instructions requiring material-grounded content, specified structure, quantitative fidelity, presentation quality, visual organization, and audience-appropriate tone.Instructions can require a slide-count range, ordered mandatory sections, traceable quantitative details, annotated visualizations, and concise slide-style content.
- Evaluation Workflow: The evaluation workflow has a slide agent generate a deck, after which a judge MLLM applies structured checklist items and aggregates per-item decisions into an overall score.The benchmark workflow also includes expert-designed data collection, instructions, and checklists before automated evaluation.
- Checklist-Based Evaluation: The checklist-based framework decomposes assessment into atomic criteria across material-independent and material-dependent tiers.The material-independent tier covers presentation fundamentals and visual design and layout, while the material-dependent tier includes instruction-derived content completeness and correctness.
4 Experiments
Experiments show that PresentBench distinguishes current slide-generation systems, aligns more closely with human preferences than holistic baselines, and exposes persistent weaknesses in grounding, design, and open-source performance.
- Main Results: 62.5 overall, NotebookLM outperforms Manus 1.6 at 57.8 across 238 real-world slide-deck generation instances.Commercial systems cluster in the mid-range at 48–55, while the results still show substantial room for improvement.
- Main Results: PPTAgent scores 50.2, below NotebookLM (62.5) and Manus (57.8), indicating a persistent performance gap between open-source and closed-source systems.The authors associate the gap with end-to-end planning and grounding, layout and design engines, and richer rendering components.
- Main Results: NotebookLM reaches 62.8 on Visual Design and Layout, while Manus scores 53.7 and most other systems score in the 40s or below.Fundamentals scores are generally higher, indicating that reasonable structure does not consistently translate into polished, presentation-ready layouts.
- Main Results: Content Completeness exceeds Content Correctness for NotebookLM (67.8 vs. 56.0) and PPTAgent (60.2 vs. 37.9), while Content Fidelity remains low for NotebookLM (45.1) and Manus (45.4).These gaps indicate persistent difficulty achieving broad coverage together with factual and numerical accuracy.
- Human Alignment Results: PresentBench achieves a Spearman correlation of 0.532 with human preferences, exceeding PPTEval (0.303) and an MLLM-as-a-Judge Ranking baseline (0.258).Its instance-specific, source-grounded binary checklist items provide finer-grained and more diagnostic evaluation than global judgments.
- Ablation Study: Fine-grained material-grounding criteria do not always improve human-alignment scores in a ranking task completed within about 3 minutes.The authors attribute this pattern to users relying on readily perceived structure, layout, and coherence rather than verifying factual correctness against source materials.
5 Limitations
PresentBench has scope and reliability limitations despite its progress in evaluating automated slide generation. It focuses on static, general-purpose content and still depends on multimodal LLM verifiers.
- PresentBench evaluates static slide content but does not model animations, transition pacing, or other temporal presentation factors.Future extensions could cover animation quality, presentation rhythm, and cross-slide narrative flow.
- General-purpose tasks provide limited coverage of highly specialized domains such as medical or legal reporting.
- Multimodal LLM verifiers may limit the reliability of PresentBench’s resulting scores.The paper identifies improving automated verification accuracy and robustness as an important direction.
6 Conclusion
PresentBench is a material-grounded benchmark that evaluates slide decks through fine-grained, verifiable checklists. It aligns more closely with human preferences than existing evaluations while exposing continuing challenges in layout, design, and faithful grounding.
- PresentBench contains 238 expert-curated instances with long-context materials and an average of 54.1 verifiable binary checklist items.The checklist spans five quality dimensions and uses source-grounded item-level verification with principled aggregation.
- PresentBench yields more reliable and diagnostic evaluation signals than holistic judging while posing a greater challenge for slide generation systems.
- Experiments across diverse slide generation systems show that PresentBench aligns more closely with human preferences than existing evaluations.Material-grounded slide authoring remains challenging, particularly in layout, design, and faithful grounding.
- PresentBench is intended as a rigorous testbed and catalyst for developing slide generation systems that are well-designed and grounded in background materials.
A.1 Detailed Composition of the Dataset
The dataset composition is documented in Table 7, which organizes PresentBench by category, source, language, and sample count.
- Table 7 summarizes PresentBench’s dataset composition by category, source, language, and sample count.
A.2 Instruction Example
The instruction example specifies a technical, source-faithful slide deck for an introductory computer science chapter. It combines comprehensive content coverage, quantitative fidelity, and explicit visual and presentation requirements.
- Technical Fidelity Requirements: Slides must explain underlying principles, avoid excessive information load, and maintain accurate text, spelling, grammar, typography, and technical fidelity.The instruction also requires at least 7 slides containing quantitative content and textbook-consistent formulas, data, calculations, figures, and tables.
- Deck Structure: The deck is framed as an academic, instructional introduction to Chapter 1: Data Storage in Computer Science: An Overview.Its agenda covers digital representation, bits and logic, storage hardware, data representation, binary arithmetic, and data integrity.
- Technical Coverage: The technical outline covers bits and Boolean logic, memory and mass storage, text and multimedia representation, binary systems, integer storage, floating-point storage, compression, and communication errors.
- Content Constraints: The example instruction requires definitions, formal models, worked examples, summaries, and corresponding slides for key diagrams, tables, and visual examples.It prohibits omitting technical material and requires content to remain faithful to the background materials.
- Visual & Design: Visual requirements call for appropriate charts, diagrams, annotations, relevant high-quality images, legibility, balance, clean layout, and consistent style.
- Presentation Tone and Audience: The presentation should use an academic, clear, instructional tone and precise technical language for undergraduate students encountering the material for the first time.Conversational language, rhetorical questions, emojis, jokes, and storytelling are disallowed.
A.3 Checklist Example
The checklist examples evaluate generated lecture slides through binary, material-independent criteria spanning structure, clarity, language, visual design, layout, and content safety. They also include material-dependent checks for factual fidelity and technical correctness.
- Evaluation protocol: Binary checklist items require evaluators to answer “yes” or “no,” explain failures, and reference specific slides when possible.Partially satisfying a requirement must receive “no”; “yes” is reserved for complete compliance without exceptions.
- Content and structure: Material-independent checks assess slide count, central theme, logical flow, relevance, titles, conciseness, lecture suitability, and avoidance of placeholder or non-slide content.The examples also check harmful or biased content, spelling, grammar, and language consistency.
- Diagnostic reporting: Checklist prompts request concrete failure descriptions, such as identifying slides with broken flow, irrelevant details, misleading titles, or problematic visual elements.This makes the evaluation diagnostic rather than a bare pass/fail judgment.
- Visual design and layout: Visual-design checks cover consistency, text–visual balance, decorative and content relevance, layout reasonableness, and unobstructed text.Blank slides, title-only slides, and unjustified empty space are treated as potentially unreasonable layouts.
- Material-dependent checks: Material-dependent checklists verify whether slide content is traceable to the source materials without omission, fabrication, or contradiction.They can encode detailed technical checks, including two’s-complement negation, overflow, floating-point fields, normalization, and truncation error.
A.4 Case Study
Figures 6–9 illustrate representative PresentBench evaluation reports. The reports explain passed checklist items and identify missing content, content errors, inconsistencies with background materials, and design inconsistencies.
- Case study: Figures 6–9 present representative cases of PresentBench’s fine-grained, rubric-based evaluation results.They serve as examples of how the benchmark reports slide-generation quality.
- Case study: Figure 6 shows an evaluation report providing a detailed explanation for a passed checklist item.
- Case study: Figure 7 shows an evaluation report identifying missing content.
- Case study: Figure 8 shows specific content errors and inconsistencies between generated slides and background materials.
- Case study: Figure 9 shows an evaluation report identifying design inconsistencies.