PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation

1 Introduction

• Because slide generation jointly requires accurate content synthesis, coherent structure, visual design, and factual and quantitative fidelity, evaluation needs instance-specific fine-grained rubrics.PresentBench's stricter rubric poses a greater challenge to systems and better reveals their failure modes than instance-agnostic frameworks such as PPTEval [12].Source: Figure 1 · p. 1, §1 · pp. 2–3
• Existing benchmarks often target isolated tasks, lack concrete background materials, reuse instance-agnostic questions, or rely on coarse rubrics that obscure specific failure modes.These limitations make it harder to assess fidelity to user inputs and produce reliable, interpretable diagnoses.Source: §1 · p. 2
• PresentBench contains 238 expert-curated instances across five domains, each paired with background materials and more than 50 task-specific checklist items.The checklists cover presentation fundamentals, visual design and layout, content completeness, correctness, and fidelity to background materials.Source: §1 · pp. 2–3
• PresentBench aligns significantly more closely with human preferences than existing frameworks, while NotebookLM consistently outperforms other slide-generation systems across scenarios.The benchmark also reveals remaining limitations in current approaches.Source: §1 · p. 3

2.1 Slide Generation Benchmarks

• Slide-generation benchmarks use holistic LLM judges, including PPTEval [12] and SlidesBench [11], while preference-trained judges [18] [19] still face cognitive burden that can limit accuracy.PPTEval evaluates content quality, visual design, and coherence; SlidesBench combines reference-based and reference-free assessment. Preference-trained judges aim to align with human or strong-judge preferences and support flexible criteria.Source: §2.1 · p. 3

2.2 Rubric-Based Evaluation

• Rubric-based evaluation uses explicit scoring or diagnostic dimensions in other generation tasks [20] [21] [22], but slide generation lacks a comparably systematic fine-grained standard.GenExam [20] verifies semantic correctness and visual plausibility item by item; MMMG [21] uses graph representations for semantic fidelity, and OneIG-Bench [22] decomposes image-generation capabilities across multiple dimensions.Source: §2.2 · pp. 3–4

3.1 Data Source Curation

• PresentBench contains 238 instances across five domains, curated from authoritative real-world materials and expert-screened for correctness, relevance, challenge, accessibility, and usage concerns.Sources include academic papers, university textbooks, institutional and corporate reports, speeches, educational talks, and brochures; source selection prioritizes cases originally pairing materials with slides.Source: §3.1 · p. 4

3.2 Instructions

• Instance-specific instructions require source-grounded decks with defined slide counts, ordered sections and required content, accurate traceable quantities, and visual and layout standards tailored to the audience.They also specify concise presentation style and exclude non-slide artifacts.Source: §3.2 · pp. 4–5

3.3 Fine-Grained Evaluation Checklist

• A checklist decomposes slide assessment into atomic criteria across material-independent and material-dependent tiers, jointly evaluating intrinsic quality and fidelity to the supplied materials.Each criterion receives an independent binary verdict with localized evidence, such as slide indices and affected elements.Source: §3.3 · p. 5, §3.3.3 · p. 7

3.3.1 Material-Independent Checklist

• Material-independent checks assess presentation fundamentals and visual design, including clarity, logical flow, consistency, text-visual balance, legibility, and freedom from clutter or overlap.They also consider concision, safety, layout appropriateness, image quality, and suitable use of charts or diagrams.Source: §3.3.1 · pp. 5–6

3.3.2 Material-Dependent Checklist

• Material-dependent checks assess completeness of required ordered content, correctness against sources, and page-level fidelity; missing required content and unsupported, contradictory, or new details count as errors.Correctness is analogous to recall, while fidelity is analogous to precision.Source: §3.3.2 · pp. 6–7

3.3.3 Scoring and Aggregation Protocol

• The scoring protocol verifies each item independently, normalizes scores within each of five dimensions, and averages those dimension scores for each evaluation instance.Binary item verdicts are accompanied by localized evidence.Source: §3.3.3 · p. 7

3.4 Overview of PresentBench

• Instances average 22.2 × 10^3 input tokens and 34.0 material pages, while each has 54.1 binary checklist items on average across five dimensions.PresentBench is predominantly English with a small Chinese subset; the long inputs require systems to distill materials into coherent slide decks.Source: §3.4 · p. 7

4.1 Experimental Setup

• The evaluation compares commercial slide-generation products and PPTAgent using the same background materials and instructions, with optional operations set to framework defaults.PPTAgent is evaluated in v2.0.0 freeform mode; the tested systems include NotebookLM, Manus 1.6, Gamma, Doubao, Tiangong, Zhipu, and Qwen.Source: §4.1 · p. 7

4.2 Main Results

• Across 238 instances, NotebookLM scores 62.5 overall, ahead of Manus 1.6 at 57.8, while other strong commercial systems cluster at 48–55.Even the best system falls short on grounded, end-to-end authoring: inputs average 22.2k tokens, including 19.9k from background materials (about 34 pages), and systems may omit required points, introduce unsupported claims, misstate facts or numbers, or lose cross-slide consistency.Source: §4.2 · pp. 7–8, Table 3 · p. 8
• PPTAgent scores 50.2 versus NotebookLM’s 62.5 and Manus 1.6’s 57.8 despite a leading image-generation model, indicating a persistent open-source capability gap.The paper attributes strong results partly to end-to-end pipelines for long-context planning and grounding, layout and design, and rendering—not only to backbone models.Source: §4.2 · p. 8
• NotebookLM leads Visual Design and Layout at 62.8, while Manus ranks second at 53.7; low layout scores contrast with Fundamentals scores often around 70–80.The results suggest that systems can form reasonable structures yet struggle to render polished layouts; strong content generation does not necessarily imply strong design capability.Source: §4.2 · p. 8, Table 4 · p. 9
• NotebookLM’s Content Completeness exceeds its Content Correctness (67.8 vs. 56.0), as does PPTAgent’s (60.2 vs. 37.9); Content Fidelity is 45.1 for NotebookLM and 45.4 for Manus.The three dimensions indicate persistent difficulty achieving both broad coverage and factual accuracy, including ungrounded details and hallucinations.Source: §4.2 · p. 8, Table 4 · p. 9

4.3 Human Alignment Results

• 0.532 is PresentBench’s Spearman correlation with human preferences, above PPTEval [12] at 0.303 and an MLLM-as-a-Judge Ranking baseline at 0.258; human agreement is 0.664.The comparison uses human rankings of five systems on 24 samples across five domains; the higher correlation supports stronger human alignment for PresentBench’s fine-grained checklist approach.Source: §4.3 · pp. 8–9, Table 6 · p. 9

4.4 Comparison with Other Evaluation Frameworks

• PPTEval [12] consistently gives higher scores than PresentBench, while generic global judgments can miss subtle errors and provide less diagnostic value than source-grounded binary checklist items.Together with the human-alignment findings, the comparison suggests PresentBench offers a more challenging and diagnostic measure of material-based slide generation.Source: §4.4 · pp. 9–10

4.5 Ablation Study

• Complementary dimensions generally improve human alignment, but fine-grained correctness and fidelity do not always add gains, as quick rankings favor readily visible cues.In the study, participants ranked five decks in about three minutes and were unlikely to verify claims against background materials; these ablations therefore do not show that grounding criteria are unnecessary, since they reward source grounding and penalize factual errors and hallucinations.Source: §4.5 · p. 10

5 Limitations

• The benchmark evaluates static slide content, omitting animations, transition pacing, presentation rhythm, and cross-slide narrative flow.Future work could extend it to dynamic presentation elements, potentially enabled by advances in video understanding.Source: §5 · p. 10
• PresentBench tasks mainly reflect general-purpose use cases, with limited coverage of specialized domains such as medical or legal reporting.Source: §5 · p. 10
• Checklist-based evaluation reduces subjectivity but still relies on multimodal LLM verifiers, whose capabilities may limit score reliability.Future work should improve automated verification accuracy and robustness to provide more dependable measurements of real-world, material-grounded slide quality.Source: §5 · p. 10

6 Conclusion

• PresentBench includes 238 expert-curated instances with long-context materials and an average of 54.1 verifiable binary checklist items spanning five quality dimensions.It evaluates source-grounded, item-level verification with principled aggregation.Source: §6 · p. 10
• PresentBench yields more reliable and diagnostic signals than holistic judging and aligns more closely with human preferences than existing evaluations.Experiments covered diverse slide generation systems.Source: §6 · p. 10
• Material-grounded slide authoring remains challenging, particularly in layout, design, and faithful grounding, and PresentBench is intended as a testbed and catalyst for better-designed, grounded systems.The systems should be grounded in their background materials.Source: §6 · p. 10

A.1 Detailed Composition of the Dataset

• Table 7 summarizes dataset composition across categories, sources, languages, and sample counts.Source: §A.1 · p. 13, Table 7 · p. 13

A.2 Instruction Example

• The instruction is manually created from expert-collected materials and requires explanations of the principles behind slide claims, not just results.Source: §A.2 · pp. 13–14
• The example outlines a data-storage chapter spanning bits and logic, memory and mass storage, text and multimedia, and binary arithmetic.Source: §A.2 · p. 14
• Its remaining topics cover integer and floating-point storage, compression methods, and communication-error detection and correction.Source: §A.2 · pp. 14–15
• The brief requires key definitions, formal models, worked examples, summaries, and corresponding slides for textbook visuals, while forbidding omissions or unsupported additions.Source: §A.2 · p. 15
• It demands factual accuracy and substantive technical depth alongside concise, relevant wording and avoidance of unnecessary markup.Source: §A.2 · p. 15
• Every slide must identify its textbook chapter and finest-grained section, and textbook figures or data must be referenced on-slide with references placed bottom-left.Source: §A.2 · p. 15
• Visual guidance calls for annotated charts and diagrams, relevant high-quality images, legible uncluttered slides, and balanced text and visuals.Source: §A.2 · p. 15
• The deck must include at least 7 slides with quantitative content, match textbook data and formulas precisely, and use an academic instructional tone for undergraduate beginners.Source: §A.2 · p. 15

Evaluation rule

• A requirement receives “yes” only when fully satisfied; partial compliance is “no,” with an explanation and, where possible, specific slide references.Source: §A.3 · p. 16

Content and Structure

• Content-and-structure checks include a 21–35-slide count, a focused theme, logical progression, relevant content and titles, no placeholder slides, and lecture suitability.Source: §A.3 · pp. 16–17

Visual Design, Layout, and Graphical Information Representation

• Visual-design checks assess consistency, text–visual balance, moderation and relevance of decorative elements, and reasonable layouts, including avoiding unjustified blank or title-only slides.Source: §A.3 · p. 18

Graphical Information Representation

• Graphical-information checks assess consistent presentation, proportional visual encodings, and clear annotations; the checklist marks “no” when required graphs, charts, tables, or diagrams are absent.Source: §A.3 · p. 19

A.3 Checklist Example

• The checklist spans foundational computing concepts, memory and storage, data representation, numeric formats, compression, and error integrity.Its topics include Boolean operations, hexadecimal notation, text/image/audio encoding, integer and floating-point storage, compression methods, parity, and Hamming distance.Source: §A.3 · pp. 20–21
• Checklist questions test precise definitions and mechanisms for Boolean logic, memory terminology, and mass-storage technologies.Examples check XOR versus OR, memory-prefix ambiguity and volatility, disk geometry and zoned-bit recording, and the single-spiral structure of CDs/DVDs.Source: §A.3 · p. 21
• Further checks target exact rules for integer and floating-point representation, compression algorithms, and error correction.They include the two’s-complement negation procedure, an 8-bit floating-point model, LZW dictionary growth, JPEG versus GIF mechanisms, parity limits, and Hamming-distance thresholds.Source: §A.3 · p. 22

• Figure 6 shows a detailed explanation for a checklist item that the generated slides passed.Source: Figure 6 · p. 24
• Figure 7 shows the report identifying missing content in the generated slides.Source: Figure 7 · p. 25
• Figure 8 shows the report identifying content errors and inconsistencies between generated slides and background materials.Source: Figure 8 · p. 26
• Figure 9 shows the report identifying design inconsistencies in the generated slides.Source: Figure 9 · p. 27