PaperlayerView exact revision on arXiv

Source-linked AI summary

Human-Agent Collaborative Paper-to-Page Crafting

Qianli Ma, Siyu Wang, Yilin Chen, Yinhao Tang, Yixiang Yang, Chang Guo, Bingjie Gao, Zhening Xing, Yanan Sun, Zhipeng Zhang

arXiv:2510.19600v2cs.SEcs.AIcs.CL

TL;DR

Manual project-page creation is repetitive, and prior automation has focused on static media rather than interactive webpages. AutoPage uses a verified, collaborative multi-agent pipeline, while PageBench supports evaluation of the task. Experiments report high-quality pages and generation in under 15 minutes for less than $0.1 using Gemini-2.5-Flash.

  • Problem

    Project pages can make dense papers more accessible, but manual creation is repetitive, and existing static-media automation does not address webpages’ responsive, interactive structure.

    Source: §1 · pp. 1–2

  • Method

    AutoPage uses a coarse-to-fine multi-agent pipeline with phase-level checkers and optional human checkpoints to turn papers into interactive webpages.

    Source: §1 · p. 2

  • Results

    AutoPage generates factually accurate, visually appealing, high-quality pages, with a page taking under 15 minutes and costing less than $0.1 using Gemini-2.5-Flash.

    Source: §1 · p. 2

  • Takeaways & Limitations

    PageBench provides a first benchmark for paper-to-page generation, and experiments show AutoPage produces coherent, dynamic, accessible webpages that lower scientific communication costs.

    Source: §1 · p. 2, §6 · p. 9

    PageBench currently covers project pages linked to ICML, ICLR, and NeurIPS from 2023–2025, and the prototype relies on commercial API-based models that may incur costs and limit reproducibility.

    Source: Limitations and Future Work · p. 10

Abstract

from arXiv · show

In the quest for scientific progress, communicating research is as vital as the discovery itself. Yet, researchers are often sidetracked by the manual, repetitive chore of building project webpages to make their dense papers accessible. While automation has tackled static slides and posters, the dynamic, interactive nature of webpages has remained an unaddressed challenge. To bridge this gap, we reframe the problem, arguing that the solution lies not in a single command, but in a collaborative, hierarchical process. We introduce $\textbf{AutoPage}$, a novel multi-agent system that embodies this philosophy. AutoPage deconstructs paper-to-page creation into a coarse-to-fine pipeline from narrative planning to multimodal content generation and interactive rendering. To combat AI hallucination, dedicated "Checker" agents verify each step against the source paper, while optional human checkpoints ensure the final product aligns perfectly with the author's vision, transforming the system from a mere tool into a powerful collaborative assistant. To rigorously validate our approach, we also construct $\textbf{PageBench}$, the first benchmark for this new task. Experiments show AutoPage not only generates high-quality, visually appealing pages but does so with remarkable efficiency in under 15 minutes for less than \$0.1. Code and dataset will be released at $\href{https://mqleet.github.io/AutoPage_ProjectPage/}{Webpage}$.

1 Introduction

The introduction frames academic project-page creation as a manual way to make dense papers accessible, while prior static-media automation does not address webpages’ responsive, interactive structure. It presents AutoPage’s collaborative, coarse-to-fine pipeline and PageBench as a benchmark for evaluating the task.

1 Introduction

  • Project pages can make dense research more accessible, but adapting templates manually is repetitive, time-consuming, and can produce inconsistent quality.The introduction asks whether high-quality pages can be generated directly from academic papers.Source: §1 · p. 1
  • Prior automation targets static slides, posters, and videos, whereas webpages need responsive flow, non-linear navigation, and interactive DOM structures.The cited works include slides (Zheng et al., 2025), posters (Sun et al., 2025; Pang et al., 2025), and videos (Zhu et al., 2025; Liu et al., 2025).Source: §1 · pp. 1–2
  • AutoPage replaces a monolithic conversion with hierarchical generation that establishes narrative structure before refining multimodal details, with human feedback supporting authorial control.The framework comprises narrative planning, multimodal content generation, and interactive page rendering, with verification at each phase and optional human checkpoints.Source: §1 · p. 2
  • PageBench is introduced as the first benchmark for paper-to-page generation, with over 1,500 paper–project-page pairs and evaluation of accuracy, coherence, and visual design.The contributions also describe holistic evaluation of factual consistency, aesthetics, and authorial alignment.Source: §1 · p. 2
  • AutoPage generates factually accurate, visually appealing, high-quality project pages; a page takes under 15 minutes and costs less than $0.1 using Gemini-2.5-Flash.The evaluation also reports a substantial performance uplift, without specifying its comparison here.Source: §1 · p. 2

2 Related Works

Prior agent systems support research communication through static media, while automated generation of interactive project webpages remains unexplored. This section situates AutoPage against work on LLM agents and presentation generation.

LLM Agents

  • Agent systems have supported literature surveys, scholarly writing (Ma et al., 2026), and reproducibility (Seo et al., 2025), with research dissemination emerging as a further application.The section describes agents as planning, retrieving knowledge, self-reflecting, or collaborating to handle multi-step tasks.Source: §2 · pp. 2–3

Automated Presentation Generation

  • Poster and slide systems use agents for layout planning, rendering code, and visual self-correction, but project webpages remain comparatively unexplored.Examples include Paper2Poster (Pang et al., 2025), P2P (Sun et al., 2025), PPTAgent (Zheng et al., 2025), and SlideSpawn (Kumar and Chowdary, 2024).Source: §2 · p. 3
  • Static formats use fixed, bounded canvases, whereas webpages require continuous responsive layouts and web-native interaction, making static-layout agents ill-suited to the task.The section identifies navigation bars and collapsible modules as examples of features supporting non-linear exploration.Source: §2 · p. 3

3 AutoPage

AutoPage transforms papers into interactive webpages through a verified, coarse-to-fine workflow with optional author input. It builds a web-oriented narrative, populates it with aligned text and visuals, and renders it using user-selected styles.

3 AutoPage

  • AutoPage structures paper-to-page generation as narrative planning, multimodal content generation, and interactive rendering, with phase-level verification and optional human checkpoints.The system can also operate autonomously without human intervention.Source: §3 · p. 3, Figure 2 · p. 4, §3.3 · p. 5

3.1 Narrative Planning and Structuring

  • The Paper Content Parser converts PDFs into structured text and visual assets, while the Page Content Planner creates and verifies a web-oriented narrative blueprint rather than mapping paper sections one-to-one.The asset library links section headings to paragraph summaries and figures or tables to captions and image files.Source: §3.1 · p. 3

3.3 Interactive Page Rendering

  • For rendering, users can select template styles through descriptive tags, after which content is compiled into HTML, CSS, and JavaScript and checked for layout and visual integrity.Authors can optionally issue language commands to fine-tune page styling; templates may also be specified or selected automatically.Source: Figure 2 · p. 4, §3.3 · pp. 4–5

3.2 Multimodal Content Generation

  • The Page Content Generator writes prose first, then selects and renders figures or tables to support that narrative; a checker verifies text–visual consistency before optional author refinement.Authors can iteratively revise content until it aligns with their intent, producing author-approved content modules.Source: §3.2 · pp. 3–4

4 PageBench

PageBench is a curated benchmark of project pages from three AI conferences, with a representative test set and distinct template library. Its evaluation covers content and visual quality, and Table 1 reports that AutoPage improves both over different base models.

4.1 Dataset Curation

  • PageBench contains over 1,500 project pages from NeurIPS, ICML, and ICLR (2023–2025), with a 100-page test set and an 87-page template library.The test set samples page archetypes through clustering; the template library is formed by deduplicating the test set.Source: §4.1 · p. 5

4.2 Evaluation Metrics

  • Evaluation metrics assess two dimensions: content quality and visual quality.The suite is designed to measure accurate information conveyance and visually coherent, pleasing presentation.Source: §4.2 · p. 5

4.2.1 Content Quality

  • Content quality measures readability, semantic fidelity to source paragraphs, and factual preservation under compression.Compression-Aware Information Accuracy combines answer correctness with text compression ratio.Source: §4.2.1 · p. 5

4.2.2 Visual Quality

  • AutoPage improves both content and visual quality over different base models in the main PageBench evaluation.Table 1 reports the main evaluation results across the full suite.Source: Table 1 · p. 6
  • Visual quality uses a VLM-as-Judge framework to assess visual content accuracy, layout and cohesion, and aesthetic appeal.The assessment uses a five-point scale and considers formula rendering, image relevance, layout balance, and design harmony.Source: §4.2.2 · pp. 5–6

5 Experiments

AutoPage improves webpage generation across backbone models, earns the highest human preference score, and shows gains from its planning, content-generation, and checking components. Experiments compare automated systems using templates from different papers and include qualitative examples and component ablations.

5.1 Experiment Setup

  • The evaluation benchmarks AutoPage against advanced LLMs and AutoGen, runs without human intervention, and cross-pairs each paper with a template from another paper’s project website.The authors describe these automated results as a conservative lower bound on the system’s full potential.Source: §5.1 · pp. 6–7

5.3 User Study

  • 7.66 was AutoPage’s highest average user-study score, ahead of Gemini2.5-Flash (6.87) and Grok4-fast (6.07).Figure 3 presents user preference scores with 95% confidence intervals.Source: Figure 3 · p. 7, §5.3 · p. 8

5.2 Main Results

  • 0.49-point Visual Content Accuracy gain for Qwen (2.52 to 3.01), versus 0.31 for Gemini-2.5-Flash, narrows their gap from 0.30 to 0.12.AutoPage also improves other evaluated metrics across end-to-end backbones.Source: §5.2 · p. 7

5.4 Case Study

  • AutoPage’s examples show formula rendering, image galleries, and themed tables, while its content planning adds relevant visualizations to dense text.End-to-end baselines often misrender formulas and stack image galleries vertically.Source: §5.4 · p. 8

5.5 Ablation Study

  • Removing the Page Content Planner lowers all metrics, especially Layout & Cohesion (2.15 to 1.80) and Compression-Aware Information Accuracy (1.744 to 1.523).The reported analysis links global structural planning to webpage organization.Source: §5.5 · p. 9
  • Removing the Text Content Generator raises Readability and Semantic Fidelity slightly through verbatim copying, but Compression Rate drops from 9.168 to 4.326 and Compression-Aware Information Accuracy falls to 1.449.The copied text is described as highly redundant.Source: §5.5 · p. 9
  • Removing both verifiers produces the largest drop; Visual Content Accuracy falls from 3.13 to 2.75 and Aesthetic Score from 2.69 to 1.90.Disabling only the HTML checker lowers Layout and Cohesion from 2.15 to 1.65; removing the full content checker lowers Semantic Fidelity from 0.739 to 0.695.Source: Table 3 · p. 9, §5.5 · p. 9

6 Conclusion

AutoPage turns academic papers into interactive project webpages, and PageBench provides a benchmark for evaluating them across fidelity, compression, and aesthetics.

6 Conclusion

  • AutoPage transforms academic papers into interactive project webpages, with PageBench enabling principled evaluation across fidelity, compression, and aesthetics.The authors frame the work as a foundation for future systems that expand research accessibility and impact.Source: §6 · p. 9

Limitations and Future Work

The current benchmark and prototype have limits in coverage, model access, and evaluation scope.

Limitations and Future Work

  • PageBench covers project pages from ICML, ICLR, and NeurIPS for 2023–2025, leaving other venues and broader domain coverage for future work.The authors specifically identify ACL, CVPR, and KDD as possible additions.Source: Limitations and Future Work · p. 10
  • The prototype relies on commercial API-based models, which may add monetary costs and limit reproducibility.The authors suggest open-source or locally deployable models as a future direction.Source: Limitations and Future Work · p. 10
  • The evaluation covers content fidelity and visual aesthetics, but additional user-centric studies could complement it on usability and long-term impact.Source: Limitations and Future Work · p. 10

Broader Impact and Ethics Statement

The work aims to reduce scientific-communication costs while preserving author agency, and tests whether AutoPage’s gains depend on templates.

Broader Impact and Ethics Statement

  • The dataset uses publicly released project webpages from ICML, ICLR, and NeurIPS papers, collected from open sources for academic research.The authors intend to release the dataset and benchmark tools openly to support transparency, reproducibility, and further study.Source: Broader Impact and Ethics Statement · p. 10
  • The authors’ stated goal is to lower scientific-communication costs through automation while preserving authors’ agency over final outputs.Source: Broader Impact and Ethics Statement · p. 10

A.1 Template-Free Generation

  • AutoPage outperforms end-to-end LLMs across metrics in both template-based and template-free settings, with gains persisting without templates.The template-free result is presented as evidence that gains stem from the structured agent pipeline rather than template guidance.Source: Table 4 · p. 13

A.2 Generation in Different Evaluation Scope

The supplementary analyses test generalization beyond the original benchmark and describe the readability, semantic-fidelity, and compression-aware accuracy measures used for evaluation.

A.2 Generation in Different Evaluation Scope

  • On non-paper reports, Semantic Fidelity rises from 0.668 to 0.728 and Compression-Aware Information Accuracy from 1.327 to 1.511; on non-ML papers, the latter rises from 1.339 to 1.611 and Aesthetic Score from 2.53 to 2.66.AutoPage consistently outperforms the Gemini-2.5-Flash baseline across metrics in both settings.Source: §A.2 · p. 13, Table 5 · p. 14

B.1 Readability

  • For technical webpages, the authors use LLM-as-a-judge readability assessment because perplexity can diverge from human readability judgments on technical text.The cited rationale is that LLM-based assessments better capture discourse-level clarity, coherence, and organization.Source: §B.1 · p. 13

B.2 Semantic Fidelity

  • Semantic Fidelity aligns each generated section with its most similar source paragraph using Sentence-BERT embeddings, then computes cosine similarity for each pair.The webpage score averages the aligned-pair similarities.Source: §B.2 · p. 14
  • On STS-B, Sentence-BERT sentence-level cosine similarity correlates with human judgments at r=0.881, versus r=0.649 for token-level BERTScore F1.The authors use this result to support the metric for settings involving paraphrasing and structural reorganization.Source: Figure 5 · p. 14, §B.2 · p. 14

B.3 Compression-Aware Information Accuracy

  • Compression-Aware Information Accuracy combines QA accuracy from 100 source-derived questions with the token compression ratio using accuracy × ln(compression ratio).The logarithm rewards concision while dampening the effect of extreme compression.Source: §B.3 · p. 15

B.4 VLM-as-Judge for Visual Quality Evaluation

The paper evaluates generated-page visual quality with a VLM-as-Judge focused on correctness and relevance. A strict system prompt supports consistent, scalable review.

B.4 VLM-as-Judge for Visual Quality Evaluation

  • A Vision Language Model serves as an automated judger for the visual quality of generated pages.The paper terms this approach “VLM-as-Judge.”Source: §B.4 · p. 15
  • The evaluation focuses on visual-element correctness and relevance rather than subjective aesthetic appeal.Source: §B.4 · p. 15
  • A carefully designed system prompt directs the VLM to act as an extremely strict visual-elements reviewer.The paper presents this setup as enabling consistent and scalable evaluation.Source: §B.4 · p. 15

C Detailed Analysis of Compression-Aware QA Performance

AutoPage variants improve their base models’ QA scores and balance higher compression with comparable raw accuracy. The benchmark assets include a diverse test set and a curated library of stylistically distinct templates.

C Detailed Analysis of Compression-Aware QA Performance

  • AutoPage-GPT-4o-mini raises its base model’s score from 1.786 to 1.941, while AutoPage-Gemini-2.5-flash raises its base model’s score from 1.276 to 1.591.Source: C Detailed Analysis of… · p. 15
  • 1.941 is AutoPage-GPT-4o-mini’s highest overall Compression-Aware ACC, alongside higher compression rates and comparable raw accuracy to end-to-end models.Source: C Detailed Analysis of… · pp. 15–16

D Details for the Test Set and Template Library Construction

  • Clustering selected approximately 100 pages as a test set, and deduplication produced an 87-page library of stylistically distinct templates.Deduplication combined SimHash filtering with Zhang-Shasha tree edit distance on standardized DOM structures.Source: D Details for the Test Set and… · p. 16

E User Study Protocol and Materials

The user study combines forced-choice comparisons with absolute ratings, and automatic metrics show positive correlations with human judgments. AutoPage pages take 6–15 minutes to generate, while additional comparisons highlight its handling of complex page content.

E User Study Protocol and Materials

  • Forced-choice judgments were adopted to reduce scale-use bias and order effects; participants assigned unique scores to each page in a group.A supplementary five-point Likert study provided absolute, dimension-level ratings alongside the relative comparisons.Source: E User Study Protocol and Materials · pp. 16–17
  • AutoPage scored highest across five Likert dimensions, and those absolute ratings aligned with forced-choice outcomes.The 20 expert evaluators compared groups of eight pages using forced-choice scores; the supplementary Likert study rated pages on five dimensions.Source: E User Study Protocol and Materials · pp. 16–17, Table 7 · p. 17

F Correlation with human judgments.

  • Readability’s Kendall’s τ correlation was 0.69, compared with 0.40 for Visual Content Accuracy, 0.36 for Layout and Cohesion, and 0.22 for Aesthetic Score.All reported dimensions had positive correlations with human judgments.Source: F Correlation with human judgments · pp. 17–18

G Discussion

H Additional Comparision Resuls

  • Across template-free and template-based examples, AutoPage preserves formulas, images, tables, and section content where baseline pages show rendering, planning, or resource-resolution failures.Source: H Additional Comparision Resuls · p. 18

I Effectiveness of Human-in-the-loop Feeback

This section examines human-in-the-loop review and revision of project pages, including source-grounded content and figure checks, constrained recommendations, HTML and layout requirements, and evaluation rubrics. It reports that one round of visual feedback improves structural visual quality, with smaller gains in accuracy and aesthetics, while noting that deeper refinements may require multiple iterations.

I Effectiveness of Human-in-the-loop Feeback

Layout Specification

  • The layout specification calls for full-width, single-column sections and balanced columns, with images centered in single-column blocks and spaced apart.It suggests grouping excess images into columns and setting column flex growth according to image aspect ratios to align column heights.Source: Layout Specification · p. 28

3. Number of tables

  • The page-generation rules cap tables at three, while the review prompt instructs reviewers to retain only the two most important tables when there are more than two.The review prompt says not to restrict the number of figures.Source: Layout Specification · p. 29, §3 · p. 29

1. Figure Placement and Usage

  • Reviewers check that figures appear in appropriate sections, relate closely to nearby text and captions, and are explained in the section text.When figures convey similar ideas, retain the more important one; if figure-text correlation is weak, suggest removing or moving the figure.Source: §1 · p. 29, §2 · p. 29

Instructions

  • Reviewers compare generated page content with the paper and figures, report weaknesses, strengths, and suggestions, then revise without changing section names or text.Revision should fix weaknesses, preserve strengths, apply suggestions, and maintain accuracy, clarity, and structure without adding or removing sections or duplicating figures.Source: §3 · p. 30, Instructions · p. 31

Specific Requirements

  • Image reviewers assess width against an approximately 70–90% target of the text block, flag widths above it or below 65%, and recommend containers only for viewport overflow.For overflow, the requirements suggest a container matching the screen width, generally 1280 pixels; otherwise, recommend proportional resizing.Source: Specific Requirements · p. 28, Instructions · pp. 31–32

2. Vertical spacing

  • Image checks cover balanced vertical spacing and internal text near body-text size; suggestions use pixel margins or proportional image resizing, while skipping table text.The review is limited to width, spacing, and internal text, and formulas should not be placed in separate paragraphs.Source: §2 · p. 32, §3 · p. 32, §4 · p. 32

4. Strictness Rules

  • Existing HTML formatting and content must be preserved, while images and tables retain their paths and aspect ratios unless suggestions call for adjustments.The rules also require responsive, single-column, full-width presentation and professional visual design.Source: §4 · p. 33
  • Image sizing and placement must follow suggestions, with spacing, alignment, symmetry, and column-height balance maintained.Adjusted images require a comment recording display width and height; images and tables should be centered where applicable.Source: §4 · pp. 33–34
  • Aesthetics are judged conservatively on color, style consistency, visual hierarchy, text-image coordination, and overall impression.The rubric reserves its highest score for publication-level aesthetics and a unified, harmonious visual presentation.Source: §4 · pp. 34–35
  • A separate conservative five-point rubric assesses formula display, image relevance and clarity, and whether labels, captions, and colors aid comprehension.Its criteria distinguish missing or incorrect formulas and images from fully displayed formulas and clear, relevant images.Source: §4 · pp. 35–37
  • The layout judge evaluates page balance and typesetting, focusing on whitespace, image size, and whether element arrangements disrupt reading or information acquisition.Its rubric treats chaotic layouts and disruptive whitespace or oversized images as problems, and reserves its highest level for publication-quality presentation.Source: §4 · pp. 36–37

Related papers

Loading 2510.19600v2…