PaperlayerView exact revision on arXiv

Source-linked AI summary

Paper Reconstruction Evaluation: Evaluating Presentation and Hallucination in AI-written Papers

Atsuyuki Miyai, Mashiro Toyooka, Zaiying Zhao, Kenta Watanabe, Toshihiko Yamasaki, Kiyoharu Aizawa

arXiv:2604.01128v1cs.CLcs.AIcs.LG

TL;DR

As AI tools automate research writing, rigorous evaluation of their capabilities and risks is needed. PaperRecon evaluates agents by reconstructing papers from compressed source information; Claude Code achieves higher presentation quality, while Codex produces fewer hallucinations (around 3 versus more than 10 per paper on average).

  • Problem

    Rigorous evaluation of coding agents’ research-writing capabilities and risks is needed as AI-written submissions emerge and AI Scientist tools grow.

    Source: §1 · p. 2

  • Method

    PaperRecon gives agents a compressed research overview and minimal materials to reconstruct an existing paper, isolating the writing component of AI Scientist systems.

    Source: §1 · p. 2

  • Results

    Claude Code achieves higher presentation quality but averages more than 10 hallucinations per paper, while Codex produces around 3; writing capability improves with model advances.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The presentation–hallucination trade-off underscores the importance of robust evaluation methods and safeguards against unreliable AI-generated content.

    Source: Ethics Statement · p. 10

    Section-wise evaluation may not fully reflect overall paper quality because human scientific writing is diverse and not fully captured by current LLMs.

    Source: §6 · p. 10

Abstract

from arXiv · show

This paper introduces the first systematic evaluation framework for quantifying the quality and risks of papers written by modern coding agents. While AI-driven paper writing has become a growing concern, rigorous evaluation of the quality and potential risks of AI-written papers remains limited, and a unified understanding of their reliability is still lacking. We introduce Paper Reconstruction Evaluation (PaperRecon), an evaluation framework in which an overview (overview.md) is created from an existing paper, after which an agent generates a full paper based on the overview and minimal additional resources, and the result is subsequently compared against the original paper. PaperRecon disentangles the evaluation of the AI-written papers into two orthogonal dimensions, Presentation and Hallucination, where Presentation is evaluated using a rubric and Hallucination is assessed via agentic evaluation grounded in the original paper source. For evaluation, we introduce PaperWrite-Bench, a benchmark of 51 papers from top-tier venues across diverse domains published after 2025. Our experiments reveal a clear trade-off: while both ClaudeCode and Codex improve with model advances, ClaudeCode achieves higher presentation quality at the cost of more than 10 hallucinations per paper on average, whereas Codex produces fewer hallucinations but lower presentation quality. This work takes a first step toward establishing evaluation frameworks for AI-driven paper writing and improving the understanding of its risks within the research community.

1 Introduction

PaperRecon evaluates AI paper writing by having agents reconstruct an existing paper from a compressed overview and minimal resources, then comparing the result with the original. The framework addresses the need for rigorous evaluation of AI-written papers and their risks.

1 Introduction

  • Prior AI-review approaches can reward severe fabrications, while earlier hallucination work focused on citation errors or individual cases rather than systematic evaluation.AI-review approaches are discussed in Liu et al. (2024), Yamada et al. (2025), Weng et al. (2025), and Zhu et al. (2025); prior hallucination work includes citation-error studies by Walters & Wilder (2023), Ansari (2026), and Sakai et al. (2026), and individual cases by Yamada et al. (2025) and Miyai et al. (2026).Source: §1 · p. 2
  • PaperRecon compresses an existing paper into a structured overview and asks agents to reconstruct it using that overview and minimal additional resources.The reconstruction task isolates the writing component of existing AI Scientist systems.Source: §1 · pp. 2–3
  • PaperRecon evaluates writing along two orthogonal dimensions: Presentation, scored by rubric, and Hallucination, assessed agentically against the original paper source.This enables assessment of how faithfully key elements are preserved and detection of factual inconsistencies.Source: §1 · pp. 2–3
  • PaperWrite-Bench comprises 51 papers published after 2025 from diverse top-tier venues, including NeurIPS, ICLR, CVPR, ICCV, ACL, and ACMMM.The benchmark is intended to support systematic evaluation across diverse research domains.Source: §1 · pp. 2–3
  • Claude Code achieves higher presentation quality than Codex, while Codex averages around 3 hallucinations per paper versus Claude Code’s more than 10; writing capability improves with model advances.The authors present these findings as a trade-off between presentation quality and hallucination frequency, with writing capability also improving as models advance.Source: §1 · pp. 2–3

2 Related Work

Prior work highlights hallucination risks in AI research writing but lacks a systematic protocol for evaluating scientific papers’ substantive content. PaperRecon addresses this gap by comparing generated papers with original papers on presentation quality and hallucination.

AI-driven Research Automation

  • Miyai et al. (2026) report hallucination risks across research stages and particular susceptibility to inconsistencies with experimental results and fabricated content during writing.Their findings motivate measuring the writing capabilities of modern agents with PaperRecon.Source: §2 · p. 3

Evaluation of AI-written Articles

  • Scientific-paper evaluation remains limited because writing requires justified significance, research context, sound protocols, reproducibility, verifiability, and evidence-aligned claims.These demands make scientific writing and its evaluation substantially more challenging than Wikipedia or essay writing.Source: §2 · p. 3

Hallucination and Evaluation of AI-written Papers

  • Prior paper evaluations rely mainly on venue-review judgments, while AI reviewers can miss hallucinations, and hallucination studies often cover only citation errors or individual cases.Existing work includes review-based evaluations (Lu et al., 2024; Yamada et al., 2025; Weng et al., 2025; Zhu et al., 2025), AI-reviewer limitations (Jiang et al., 2025), and surface-level or individual hallucination analyses (Walters & Wilder, 2023; Ansari, 2026; Sakai et al., 2026; Miyai et al., 2026).Source: §2 · p. 3
  • PaperRecon evaluates generated papers against originals on two complementary axes: rubric-based presentation quality and agentic hallucination assessment.Figure 2 depicts the evaluation pipeline; prior work on generating papers from source materials instead evaluates novelty, methodological validity, and empirical performance.Source: §2 · pp. 3–4

3 Paper Reconstruction Evaluation

PaperRecon reconstructs papers from an overview and selected original-paper resources, then evaluates generated papers against the ground truth for presentation, hallucinations, and citation use. Its protocol combines section matching, rubric-based scoring, claim verification, and citation metrics.

3.1 Problem Definition

  • Agents reconstruct papers using an overview, original figures and tables, references with abstracts, and code when available; reusing references isolates writing ability from reference collection.The generated paper is compared with the original from multiple perspectives.Source: §3.1 · p. 4

3.2 Evaluation Protocol

  • Evaluation covers presentation, hallucinations, and citation use, with section classification and matching performed before scoring matched sections.The protocol compares generated papers with ground-truth papers.Source: §3.2 · p. 4

3.2.1 Section Classification and Matching

  • Sections are mapped to seven common categories using keyword rules first and LLM classification when needed, then merged by category for shared downstream evaluation.The categories are Abstract, Introduction, Method, Benchmark Construction, Experiment, Related Work, and Conclusion.Source: §3.2.1 · pp. 4–5

3.2.2 Rubric Evaluation

  • Each ground-truth paper gets reviewed, section-specific rubrics, and GPT-5.4 scores each non-conclusion section’s elements on a 1–5 scale.Rubric elements specify expected, verifiable content and relative importance; authors review and refine LLM-generated rubrics. Preliminary experiments found LLM-as-a-judge less discriminative than rubric evaluation.Source: §3.2.2 · p. 5
  • Figure and table items also enter rubric scores: figures are judged for contextual appropriateness, while matched tables are scored for numerical accuracy, structure, and content consistency.Unreferenced generated figures and ground-truth tables missing from the generated paper receive scores of 1; each section’s final rubric score averages its text, figure, and table items.Source: §3.2.2 · p. 5

3.2.3 Hallucination Analysis

  • GPT-5.4 extracts verifiable claims outside the Conclusion, labels them supported, neutral, or contradictory, and grades contradictions as major or minor.A coding agent re-examines flagged contradictions against ground-truth resources, potentially revising false positives to supported or neutral.Source: §3.2.3 · pp. 5–6

3.2.4 Citation-Level Evaluation

  • Citation-level evaluation compares ground-truth and generated citation-key sets using precision, recall, and F1, and identifies hallucinated, missing, and extra citations.Hallucinated keys are cited in the generated paper but absent from its bibliography; missing keys occur in the ground truth but not the generated paper.Source: §3.2.4 · p. 6

3.2.5 Overall Metrics

  • Aggregate reporting includes mean rubric score across sections and elements, major contradictory-claim counts after verification, and citation precision, recall, F1, and invalid-citation counts.Rubric scores use a 1–5 scale and include figure and table items.Source: §3.2.5 · p. 6

4 PaperWrite-Bench

PaperWrite-Bench is a manually curated benchmark of 51 recent papers spanning multiple venues, domains, and contribution types. Its construction provides agents with verified paper overviews, extracted resources, and fixed section structures for reconstruction.

4.1 Benchmark Overview

  • PaperWrite-Bench contains 51 manually curated papers from top-tier venues across computer vision, NLP, machine learning, and multimedia, including method, benchmark, and combined contributions.The set includes 32 method papers, 12 benchmark papers, and 7 combining both contribution types.Source: §4.1 · p. 6
  • Unlike Exp-Bench and PaperBench (Starace et al., 2025), which largely draw on papers around 2024, PaperWrite-Bench uses more recent sources to reflect modern agents.Source: §4.1 · p. 6

4.2 Benchmark Construction Pipeline

  • Each paper receives a GPT-5-generated research_overview.md, manually checked for reconstruction sufficiency and averaging 463 words.Source: §4.2 · p. 7
  • The pipeline extracts figures, tables, references, and available code, supplies structured summaries and bibliography abstracts, and removes abstract or introduction sections from code READMEs when present.Table and figure summaries provide file paths and brief descriptions; bibliography entries are augmented with Semantic Scholar abstracts.Source: §4.2 · p. 7
  • Agents write into templates preserving each original paper’s section headings, enabling more consistent comparisons while requiring only minimal human preprocessing.Source: §4.2 · p. 7

5 Experiments

Claude Code achieved higher presentation scores but produced more hallucinations than Codex, while expert judgments correlated with rubric scores and manual checks confirmed most detected hallucinations. Longer research overviews improved presentation and reduced hallucinations; NLP papers scored highest across conference types.

5.1 Experimental Setup

  • Five configurations covered Claude Code, Codex, and Claude Agent Teams; the simple writing pipeline included a LaTeX compilation feedback loop and page-limit adjustment.Claude Code and Codex were tested with two backbone models each, while Claude Agent Teams used Sonnet 4.6.Source: §5.1 · p. 7

5.2 Main Results

  • 3.86 was Claude Code with Sonnet 4.6's best presentation score; Claude Code still exceeded 10 hallucinations per paper, versus around 3 for GPT-5.4 Codex.Claude Code consistently scored higher than Codex in presentation across sections.Source: §5.1 · p. 8, §5.2 · p. 8
  • Claude achieved higher Citation F1 scores, while Codex produced substantially fewer hallucinated citations.The citation evaluation showed a trade-off between citation coverage and factual reliability.Source: §5.1 · p. 8, §5.2 · p. 8
  • Writing quality consistently increased from Sonnet 4 to Sonnet 4.6 and from GPT-5 to GPT-5.4.The evaluation framework captured gains for Claude Sonnet 4 (Anthropic, 2025a) to Claude Sonnet 4.6 (Anthropic, 2026), and GPT-5 (OpenAI, 2025) to GPT-5.4 (OpenAI, 2026).Source: §5.2 · p. 8

5.3 Human Validation

  • τb = 0.578 (p < 0.001) across 72 generated-paper pairs showed strong, significant correlation between rubric rankings and expert human judgments.Three top-tier conference reviewers judged 24 paper pairs; disagreements often reflected subjective preferences such as conciseness over detailed explanations.Source: §5.3 · p. 9
  • 96% of 97 claims labeled major contradictory in GPT-5, GPT-5.4, and Sonnet-4.6 papers were verified as genuine contradictions or fabrications.Authors manually checked these instances; the validation measured precision because exhaustively finding all hallucinations was prohibitively labor-intensive.Source: §5.3 · p. 9

5.4 Analysis

  • 1,492-word long overviews, versus 463-word default overviews on average, led to higher presentation scores and fewer hallucinations.Long overviews included more detailed methodology and experimental design.Source: §5.4 · p. 9
  • NLP conferences achieved the highest performance; the authors attributed this to their focus on findings-based research and fewer complex mathematical formulations or methods.The authors concluded that writing progress should be evaluated separately by research field.Source: §5.4 · p. 9

6 Conclusion, Limitations and Future Work … A.2 Example of Rubrics

PaperRecon and PaperWrite-Bench provide a controlled framework for assessing AI-generated scientific papers’ presentation quality and hallucinations. The work also identifies limitations in its resource assumptions and section-wise evaluation, and stresses safeguards against misleading or fabricated research.

6 Conclusion, Limitations and Future Work

  • PaperRecon is presented as the first systematic framework for evaluating AI-generated scientific papers, and PaperWrite-Bench supports a comprehensive assessment of writing agents’ capabilities and risks.The authors also discuss limitations and future directions.Source: §6 · p. 9
  • The evaluation supplies agents with figures, tables, and references to isolate core writing ability, leaving evaluation with fewer resources or external systems for future work.This design reduces dependencies on retrieval and reference collection.Source: §6 · p. 10
  • Because human scientific writing is diverse and not fully captured by current LLMs, section-wise evaluation may not reflect overall paper quality; more robust methods remain a future need.The authors characterize evaluation of scientific papers as inherently challenging.Source: §6 · p. 10

Ethics Statement

  • AI systems may generate plausible but misleading scientific content, and misuse to produce fabricated or low-quality papers that evade standard review motivates robust evaluations and safeguards.The authors point to the presentation-quality and hallucination trade-off as underscoring this need.Source: Ethics Statement · p. 10

Authorship Statement

  • The authorship statement credits Miyai with project leadership, Toyooka with core-code and framework implementation, Zhao with figure evaluation, and Watanabe with document review and the Citation Score.Yamasaki provided managerial suggestions, while Aizawa gave guidance and resource support.Source: Authorship Statement · p. 10

A.1 Statistics of PaperWrite-Bench Sources

  • PaperWrite-Bench’s included conferences are mostly from ML and CV, followed by multimedia and NLP.The appendix presents a distribution of papers by research area.Source: §A.1 · p. 14
  • The example rubric calls for EgoLife’s abstract to state its egocentric-assistant goal, its introduction to motivate personalized long-term assistance, and its related work to situate the dataset historically.The examples use EgoLife (Yang et al., 2025), a CVPR25 benchmark-construction paper.Source: §A.1 · p. 14
  • The further rubric examples require describing EgoLife’s week-long, six-volunteer multimodal collection, EgoButler’s EgoGPT and EgoRAG subsystems, and the unified experiment comparison.These are representative high-importance rubric points.Source: §A.1 · p. 14

B Detailed Prompts · C Generated Papers · Abstract

Together, these sections provide example research overviews, templates and prompts for generating and revising papers, and materials for evaluating presentation and hallucinations. They also include examples of generated-paper results and abstracts, spanning EgoLife and PINO.

B.1 Examples of research_overview.md

  • EgoLifeQA contains 3,000 multiple-choice questions across five task types, built on 266 hours of recordings from six participants over seven days; 2,003 questions require over two hours of context.The benchmark records look-back requirements as certificate lengths and assesses evidence retrieval from week-long recordings.Source: §B.1 · pp. 15–16
  • EgoButler pairs EgoGPT, a personalized omni-modal clip-understanding model, with EgoRAG, which retrieves evidence through hierarchical clip-, hour-, and day-level memory.The overview describes this combination as a system for personalized, long-horizon egocentric question answering.Source: §B.1 · pp. 15–16
  • On EgoLifeQA, personalization in EgoGPT (+D1) boosts average from 33.1 for non-personalized EgoGPT to 36.0, a reported +2.9-point gain under this comparison.Source: §B.1 · pp. 15, 18
  • EgoRAG markedly improves long-context QA accuracy for certificate lengths over 24 hours, increasing the reported accuracy from 25.0 to 35.4, a 10.4-point gain.The benchmark shows that 2,003 of 3,000 QAs need more than 2 hours of context.Source: §B.1 · pp. 15–16
  • Human visual-audio captions reach 45.5 average accuracy, compared with 27–28 for audio-only models and 31–34 for visual-only models; audio-plus-visual models reach 36.0.The overview identifies caption quality as important to EgoLifeQA performance.Source: §B.1 · p. 16
  • The overview positions EgoLife and EgoLifeQA as addressing the lack of week-long, multiperson egocentric data and benchmarks for long-context, life-oriented assistance.It highlights rich ego-exo multimodal capture and social dynamics as distinguishing scope.Source: §B.1 · pp. 16–17
  • With Day-1 personalization, EgoGPT scores 75.4 on EgoSchema, 33.4 on EgoPlan, and 61.4 on EgoThink, with respective EgoSchema and EgoPlan gains of 2.2 and 1.0.The overview also summarizes these scores as state-of-the-art results on the egocentric benchmarks.Source: §B.1 · pp. 15, 18
  • For 6–24h and >24h look-back, accuracy rose from 26.8 to 38.9 and from 25.0 to 35.4; average accuracy was 31.5 narration-only, 29.6 transcript-only, and 45.5 with human-verified visual-audio captions.For <2h look-back, accuracy shifted from 28.2 to 27.2, while the 2–6h result rose from 29.1 to 35.7.Source: §B.1 · pp. 18–19
  • The overview attributes gains to personalization and joint visual-audio understanding, while noting overfitting risks, audio-only limitations, and the importance of hierarchical retrieval for long-context QA.It also flags ASR-centric training, single-pass retrieval failures, and challenges in recognizing subtle social cues.Source: §B.1 · p. 18
  • EgoLife comprises a 266-hour week-scale egocentric dataset, a 3,000-question long-context QA benchmark, and EgoButler, which combines personalized EgoGPT with hierarchical EgoRAG retrieval.The dataset retains 266 hours from about 300 hours of recordings; the benchmark spans five categories and certificate lengths up to >24h.Source: §B.1 · p. 19

B.2 Prompts for Creating Overviews

  • The excerpt provides a concise Markdown research-overview prompt with a target length of around 1,500–2,500 characters, and labels a separate detailed prompt for long or method papers.The concise prompt limits each section to 2–4 sentences at most and specifies a section structure. The detailed prompt targets around 4,000–8,000 characters.Source: §B.2 · p. 19

B.3 Prompts for Creating Rubrics

  • The rubric prompt asks reviewers to extract 5–15 essential, distinct elements per section, assign importance, and ground each element in evidence from the source text.Its guidance tailors criteria to section type, including methods, experiments, related work, and abstracts.Source: §B.3 · p. 21

B.4 Writing Prompts

  • The writing prompt directs agents to use the overview, experimental data, table and figure descriptions, code, and LaTeX template to produce a complete, accurate paper.It also asks the agent to use the external references.bib file rather than embedding bibliography contents.Source: §B.4 · p. 22
  • A reflection prompt directs agents to inspect LaTeX style and compilation feedback and fix reported errors, especially compilation failures.Source: §B.4 · pp. 22–23
  • A separate page-limit prompt specifies the current and target page counts, prohibits moving or adding appendix content, and caps revision changes at 1,000 characters.Source: §B.4 · p. 23

B.5 Evaluation Prompts

  • Section-level rubric elements are scored from 1 to 5, and scores must be penalized when required visuals are missing or have low match scores.The rubric rates coverage from absent (1) to fully and accurately described (5).Source: §B.5 · p. 23
  • The first hallucination stage extracts all verifiable claims and labels them supported, neutral, or contradictory, assigning major or minor severity only to contradictions.Claims absent from the ground truth are not contradictory unless they conflict with it.Source: §B.5 · p. 24
  • A second agent re-examines flagged contradictions against the ground-truth paper and relevant source files to confirm them or identify false positives.For method or implementation claims, it checks the code when available; for other claims, it checks the paper and tables.Source: §B.5 · pp. 24–25

C Generated Papers

  • Claude Code (Sonnet4.6) received a rubric score of 3.95 and had 13 major contradictions in the included PINO paper.Source: C Generated Papers · p. 26

Abstract

  • Overlap was 0.000 vs. 0.119 for InterGen in the two-person case, penetration volume was 275.65 vs. 3112.72, and FID was 13.163 vs. 13.278.The abstract also reports that these benefits scale to groups of five or more without additional training.Source: Abstract · p. 26
  • PINO composes arbitrary-sized interactions training-free from a pretrained two-person diffusion model using pivot-anchored pairwise generations, masked diffusion, and optimized initial noise.Per-pair prompts assign roles, while physics-based penalties guide overlap avoidance, root position, movement region, facing orientation, and relative positioning.Source: Abstract · p. 26
  • For long sequences, PINO uses motion inpainting with boundary smoothness penalties to extend interactions while preserving temporal coherence.Source: Abstract · p. 26

1. Introduction · 2. Related Work · 3. Preliminaries

PINO extends a pretrained two-person diffusion model to larger groups through inference-time noise optimization, addressing the scalability and physical-plausibility challenges of multi-person motion generation. On InterHuman, it achieves near-zero overlap in five-person scenarios and competitive or superior FID scores without additional training.

2. Related Work

  • Prior multi-person methods adapt single-person models, train directly on multi-person data, or compose single-person priors, but adding characters or constraints often requires retraining or redesign.Source: §1 · p. 26

1. Introduction

  • PINO composes arbitrarily large groups through sequential pairwise generation using a pretrained two-person diffusion model, without retraining.Source: §1 · p. 26
  • For each added person, PINO anchors generation on a pivot character and optimizes initial noise using penalties for overlap, root positions, movement regions, facing, and relative positions.Source: §1 · p. 26
  • Per-pair prompts enable role assignment and customizable group behaviors, while motion inpainting with boundary smoothness supports coherent long-duration extensions.Source: §1 · pp. 26–27
  • PINO achieves near-zero overlap in five-person scenarios, dramatically reduces penetration volume, and attains competitive or superior FID scores against InterGen and FreeMotion without additional training.Source: §1 · p. 27

2. Related Work

  • Controllable-motion work includes spatial guidance, programmable latent-code optimization, and inpainting; ProgMoGen and DNO motivate PINO’s inference-time noise optimization.Source: §2 · p. 27
  • Prior scalable approaches include ControlNet-adapted InterControl and FreeMotion, dataset-trained transformers, and other interaction methods; unlike them, PINO supports per-pair prompts and uses noise optimization for physical plausibility.Source: §2 · p. 27

3. Preliminaries

  • PINO builds on InterGen, which generates two-person motion using weight-sharing transformer denoisers with mutual attention, and on DNO and ProgMoGen, which optimize initial noise without changing model weights.Source: §3 · p. 27

4. Method

PINO generates multi-person interactions sequentially, anchoring new pairwise motions to existing group motion and optimizing their noise with physics-based constraints. It also supports longer sequences and distinct prompts for each pair.

4.1. Overview

  • PINO builds groups sequentially: it generates an initial pair, adds each person through a pivot-conditioned pair, and can extend sequences for longer durations.The initial pair and each added pair use text prompts; noise is optimized to satisfy physics-based penalties and avoid overlap with previously generated people.Source: §4.1 · p. 28

4.2. Masked Pivot Conditioning

  • During pair generation, masked conditioning fixes the pivot’s motion through denoising while leaving the new person’s noise free to optimize.The initial noise combines the fixed pivot motion with random noise for the unmasked person features.Source: §4.2 · p. 28

4.3. Physics-Aware Noise Optimization

  • PINO’s physics-based loss penalizes overlap, root-position deviations, region violations, facing-direction mismatches, and deviations from target relative positions.The relative-position penalty guides spatial arrangement in addition to discouraging overlap.Source: §4.3 · pp. 28–29

4.4. Long-Duration Generation via Inpainting

  • For longer sequences, temporal inpainting extends motion through overlapping segments initialized from existing motion, applying overlap and boundary-smoothness penalties before concatenation.The overlap penalty is applied throughout the extension to maintain physical plausibility.Source: §4.4 · p. 29

4.5. Per-Pair Prompt Switching

  • Distinct prompts for each pivot-person pair let PINO specify different interactions within one group, unlike shared-prompt methods that tend to produce similar motions.The examples include handshakes, high-fives, bowing, and dancing in a five-person group.Source: §4.5 · p. 29

5. Experiments

Across two-person, multi-person, and long-duration motion generation, PINO’s training-free noise optimization reduces physical artifacts while generally preserving semantic quality. Ablations show that individual penalties target their corresponding constraints, with inference-time optimization adding runtime and retaining limitations from the underlying model and pairwise decomposition.

5.1. Experimental Setup

  • The setup evaluates PINO on InterHuman using a pretrained InterGen checkpoint without fine-tuning, with physical and semantic metrics across group sizes and tasks.InterHuman contains approximately 107M frames across 23,337 text-annotated interactions; evaluations include overlap, penetration volume, foot skate, acceleration, FID, diversity, and R-Precision.Source: §5.1 · p. 30

5.2. Two-Person Interaction Generation

  • PINO reduces two-person overlap and penetration while preserving or improving semantic metrics relative to InterGen.Overlap is 0.000 vs. 0.119, and PenVol is 275.65 cm3 vs. 3112.72 cm3. FID is 13.163 vs. 13.278, Diversity is 7.904 vs. 7.793, and R-Precision is 0.675 vs. 0.674; Foot Skate and Max Acceleration remain on par.Source: Table 2 · p. 30, §5.2 · p. 30

5.3. Multi-Person Interaction Generation

  • For multi-person generation, PINO eliminates overlap and reduces penetration versus InterGen with inpainting, while five-person results show low overlap and lower pairwise FID.In multi-person results, overlap is 0.000 vs. 0.766 and PenVol is 785.01 vs. 25215.78; Foot Skate improves from 0.099 to 0.091, while Max Acceleration is 0.031 vs. 0.030. For pair (1, 5), overlap is 0.069 vs. 0.977 for InterGen and 0.991 for FreeMotion, and FID is 16.911 vs. 19.501 and 25.671, respectively.Source: Table 3 · p. 30, §5.3 · p. 30, Table 4 · p. 31

5.4. Long-Duration Motion Extension

  • In long-duration motion extension, PINO improves physical metrics in both non-semantic and semantic evaluations against InterGen inpainting.Non-semantic overlap is 0.031 vs. 0.672, PenVol is 1375.24 vs. 15622.16, Foot Skate is 0.045 vs. 0.070, and Max Acceleration is 0.043 vs. 0.044. In semantic evaluation, FID is 18.930 vs. 20.390, overlap is 0.0 vs. 0.089, Foot Skate is 0.054 vs. 0.081, and Max Acceleration is 0.025 vs. 0.036; a boundary smoothness penalty prevents transition-frame discontinuities.Source: Table 5 · p. 31, §5.4 · p. 31, Table 6 · p. 31

5.5. Ablation Study

  • Adding penalties incrementally from the vanilla InterGen baseline (all metrics at their worst) reduces positional error to 0.0, overlap to 0.0, region violations from 0.958 to 0.043, and orientation error from 1.0 to 0.208.Metrics use thresholds of 20 cm for positional error, 10 cm for region violations, and 20 degrees for orientation error. Residual orientation error reflects inherent ambiguity in complex motions.Source: §5.5 · p. 31, Table 7 · p. 31

5.6. Runtime Analysis

  • PINO’s inference-time optimization increases generation time but avoids retraining, and its per-person optimization can be parallelized across GPUs.A two-person, 150-frame sequence takes approximately 90 seconds on one NVIDIA A100, versus under 5 seconds for standard InterGen; a five-person group takes roughly 6 minutes, while retraining on InterHuman requires hours.Source: §5.6 · p. 31

5.7. Discussion

  • PINO’s modular penalties support inference-time adaptation to differentiable spatio-temporal constraints and domain-specific motion controls.The discussion identifies interactive animation and robotics simulation as potential applications, and notes that experts can specify custom region, orientation, spacing, or collision-avoidance constraints.Source: §5.7 · pp. 31–32
  • PINO’s generated motion quality remains bounded by InterGen, and pivot-anchored pairwise generation may miss interactions requiring higher-order group dependencies.Underrepresented prompts may need more optimization iterations to balance semantic and physical objectives; the discussion proposes lightweight in-domain fine-tuning as future work and gives group lifts as an example poorly captured by bilateral relationships.Source: §5.7 · p. 32

6. Limitations … Abstract

PINO generates arbitrary-size human interactions without retraining and improves physical plausibility, but its geometry, inference cost, and pivot-based scaling impose limitations.

6. Limitations

  • The pelvis-distance overlap penalty omits full-body mesh geometry, so contorted poses may retain slight limb penetrations; fixed XZ-plane generation also excludes some interaction geometries.Source: §6 · p. 32
  • Noise optimization slows inference because gradients pass through the full denoising chain, and retaining its computation graph can exhaust GPU memory for long sequences or large groups.Source: §6 · p. 32
  • As groups grow, conditioning each added person on person 1 may limit motion diversity; alternative pivot-assignment graphs such as rings or trees remain future work.Source: §6 · p. 32

7. Conclusion

  • PINO combines pivot-anchored pairwise diffusion and physics-penalty noise optimization without retraining; pair-text switching assigns roles, while boundary-smoothed inpainting supports long sequences with improved physical plausibility and competitive semantics.Source: §7 · pp. 32–33

Abstract

  • PINO reduces two-person overlap from 0.119 to 0.000 and penetration volume from 3112.72 to 275.65; multi-person values fall from 0.766 to 0.000 and 25215.78 to 785.01.Source: Abstract · p. 35
  • For motion extension, FID drops from 20.390 to 18.930 and overlap from 0.089 to 0.000.Source: Abstract · p. 35

PINO: Person-Interaction Noise Optimization for Long-Duration and

1. Introduction · 2. Related Work · 3. Method

PINO composes customizable group interactions from a frozen two-person diffusion prior by sequentially generating pivot-centered pairs and optimizing diffusion noise at inference time. The reported gains are strongest in physical plausibility and scalability, while semantic metrics are preserved or improved.

1. Introduction

  • PINO addresses the difficulty of coherent, physically plausible arbitrary-sized interactions by sequentially composing pivot-centered pairs from a frozen pretrained two-person diffusion model, without retraining.The framework optimizes each pair’s initial noise and uses masked conditioning to keep the pivot motion fixed while adding a participant under a pair-specific prompt.Source: §1 · pp. 35–36
  • PINO eliminates two-person overlap and sharply reduces penetration volume without degrading FID; multi-person gains in physical plausibility and scalability widen with group size, while extension improves semantic fidelity and realism.The reported multi-person gains are especially pronounced as the number of people increases.Source: §1 · p. 36

2.1. Diffusion Models for Human Motion; 2.2. Interactive and Multi-Person Motion Generation

  • Diffusion priors support realistic, controllable motion, but most focus on single people; interactive and multi-person generators still target fixed participant counts or jointly model groups.PINO instead reuses a pretrained two-person model and makes group size an inference-time choice.Source: §2.1 · p. 36, §2.2 · p. 36

2.3. Inference-Time Control and Optimization

  • Unlike prior inference-time control methods, PINO decomposes groups into pivot-centered pairs and optimizes for physical consistency among generated participants while allowing prompt switching.Related approaches use spatial constraints, optimize latent variables or diffusion noise, or apply joint-level interaction control.Source: §2.3 · p. 36

3.1. Problem Setup

  • The setup targets groups of N people with pair-specific prompts and motion constraints, without changing pretrained weights, and also supports extension beyond the model’s native duration.The assumed backbone is a pretrained text-conditioned two-person diffusion model.Source: §3.1 · pp. 36–37

3.2. Pivot-Based Pairwise Composition

  • For each new participant, PINO fixes the shared pivot through masked inpainting at every reverse step, then sequentially generates a compatible pair from its own prompt.This preserves a stable temporal reference and avoids asking one prompt to specify every participant’s role.Source: §3.2 · p. 37

3.3. Person-Interaction Noise Optimization

  • PINO optimizes initial diffusion noise against motion-space penalties for overlap, root-position anchors, spatial regions, and orientation, controlling collisions, placement, and facing direction.Overlap penalties account for previously generated non-pivot participants as well as the pivot.Source: §3.3 · p. 37

3.4. Prompt Switching and Customizable Roles; 3.5. Long-Duration Motion Extension

  • Pair-specific prompts allow different roles around one pivot; for longer interactions, fixed observed frames anchor inpainting while optimized noise generates the remaining interval without long-horizon training.The known segment anchors continuity at the transition boundary while retaining collision-aware and control-oriented objectives.Source: §3.4 · p. 38, §3.5 · p. 38

4. Experiments

Across two-person generation, multi-person composition, and motion extension, inference-time PINO optimization improves physical plausibility while maintaining or improving reported semantic quality. The experiments use a frozen pretrained generator and evaluate both semantic and non-semantic metrics.

4.1. Experimental Setup

  • PINO-InterGen applies inference-time optimization to frozen InterGen, following the InterHuman protocol and comparing multi-person generation with inpainting-based InterGen and FreeMotion where available.Evaluation reports semantic metrics (FID, diversity, and R-precision when available) and physical-plausibility metrics (overlap, penetration volume, foot skating, and maximum acceleration).Source: §4.1 · p. 38

4.2. Two-Person Interaction Generation with Overlap Avoidance

  • Overlap falls from 0.119 to 0.000 and penetration volume from 3112.72 to 275.65, while FID improves from 13.278 to 13.163 and diversity rises from 7.793 to 7.904.R-precision is marginally higher; these results isolate inference-time noise optimization using the same pretrained motion prior.Source: §4.2 · p. 38, Table 2 · p. 38

4.3. Multi-Person Interaction Generation

  • Overlap falls from 0.766 to 0.000 and penetration volume from 25215.78 to 785.01 in multi-person composition, with slightly improved foot skating and nearly unchanged maximum acceleration (0.031 vs. 0.030).The comparison is against InterGen with inpainting-based multi-person composition; the acceleration result indicates the gain does not come from overly damped motions.Source: §4.3 · p. 38, Table 3 · p. 38
  • For pair (1, 5), FID drops from 19.501 to 16.911 and overlap from 0.977 to 0.069; PINO-InterGen has the best FID for every pair from two to five people.Overlap is dramatically lower at each step, with earlier pairs remaining near zero; sequential composition therefore accumulates severe geometric failures less quickly than the baseline.Source: §4.3 · pp. 38–39, Table 4 · p. 39

4.4. Long-Duration Motion Extension

  • 20.390 to 18.930: motion-extension FID improves, while overlap drops from 0.089 to 0.000, foot skating from 0.081 to 0.054, and maximum acceleration from 0.036 to 0.025.In the non-semantic extension benchmark, every reported metric improves, including overlap (0.672 to 0.031), penetration volume (15622.16 to 1375.24), and foot skating (0.070 to 0.045); the results support extension while preserving realism at the transition.Source: §4.4 · p. 39, Table 5 · p. 39, Table 6 · p. 39

5. Analysis · 6. Conclusion

PINO combines a pretrained two-person diffusion model with inference-time motion penalties to generate controllable, physically plausible group interactions. Across benchmarks, its strongest gains are in collision-related metrics, while semantic fidelity is preserved or improved in the reported settings.

5.1. Ablation on Motion Penalties

  • No single motion penalty satisfies every control requirement; PINO combines constraints as needed at inference time while keeping the pretrained motion prior fixed.The ablation attributes PINO’s physical gains to explicit geometric control rather than prompt decomposition alone.Source: §5.1 · p. 39

5.2. Qualitative Observations

  • PINO maintains clearer personal space and less entangled body configurations, while pivot-based prompt switching lets a new participant take a distinct role rather than collapse toward averaged group behavior.The qualitative results describe customizable pairwise interactions within a larger group while maintaining physically cleaner spacing.Source: §5.2 · p. 40, Figure 4 · p. 41

5.3. Discussion

  • Across benchmarks, PINO’s largest gains are in collision-related metrics, especially overlap and penetration volume, with wider margins on harder composition problems.The discussion reports that semantic fidelity is preserved in the base two-person setting and improves in motion extension.Source: §5.3 · p. 40
  • Motion smoothness metrics remain close to baseline rather than improving dramatically, indicating PINO mainly improves physical feasibility and controllability rather than motion style.Semantic fidelity is preserved in the base two-person setting and improves in motion extension.Source: §5.3 · pp. 40–41

6. Conclusion

  • PINO composes arbitrary-sized, long-duration interactions around a pivot using a pretrained two-person diffusion model and inference-time noise optimization, reducing overlap and penetration and enabling motion extension without retraining.The conclusion describes the approach as training-free and presents inference-time noise optimization as a practical interface for controllable group motion generation.Source: §6 · p. 41

5.1. Ablation on Motion Penalties

  • 0.043 is the best region-violation score with the region loss, while orientation error falls from 1.0 to 0.208 with the orientation loss.The ablation also reports that root-position loss eliminates positional error but does not address collisions and substantially increases region violations; overlap loss removes overlap entirely.Source: §5.1 · p. 39, Table 7 · p. 42

Related papers

Loading 2604.01128v1…