Source-linked AI summary
Story2Proposal: A Scaffold for Structured Scientific Paper Writing
Zhuoyang Qian, Wei Shi, Xu Lin, Li Ling, Meng Luo, Ziming Wang, Zhiwei Zhang, Tengyue Xu, Gaoge Liu, Zhentao Zhang, Shuo Zhang, Ziqi Wang, Zheng Feng, Yan Luo, Shu Xu, Yongjin Chen, Zhibo Feng, Zhuo Chen, Bruce Yuan, Biao Wu, Harry Wang, Kris Chen
TL;DR
Automated scientific manuscript generation must preserve coherence, evidence alignment, and visual consistency across long documents. Story2Proposal uses specialized agents and a persistent shared visual contract, with iterative evaluation updating that contract during generation. Across four model backbones, it achieved 6.145 versus 3.963 for DirectChat, while incomplete research stories and missed subtle reasoning weaknesses remain limitations.
Problem
Automated manuscript generation lacks reliable global coherence, claim-evidence alignment, citation grounding, and structural integrity across paper-length documents.
Method
Story2Proposal coordinates architect, writer, refiner, and renderer agents under a persistent shared visual contract that tracks structure, visual artifacts, and validation rules.
Results
6.145 versus 3.963 for DirectChat (+2.182) was the overall expert evaluation score across four model backbones.
Takeaways & Limitations
The framework improves structural consistency, visual integration, and manuscript coherence, including stable figure placement and narrative-artifact alignment.
Takeaways & Limitations
Evaluation agents may miss subtle scientific weaknesses, while incomplete research stories can remain logically consistent yet reflect gaps in the underlying narrative.
Abstract
from arXiv · showhide
Generating scientific manuscripts requires maintaining alignment between narrative reasoning, experimental evidence, and visual artifacts across the document lifecycle. Existing language-model generation pipelines rely on unconstrained text synthesis with validation applied only after generation, often producing structural drift, missing figures or tables, and cross-section inconsistencies. We introduce Story2Proposal, a contract-governed multi-agent framework that converts a research story into a structured manuscript through coordinated agents operating under a persistent shared visual contract. The system organizes architect, writer, refiner, and renderer agents around a contract state that tracks section structure and registered visual elements, while evaluation agents supply feedback in a generate evaluate adapt loop that updates the contract during generation. Experiments on tasks derived from the Jericho research corpus show that Story2Proposal achieved an expert evaluation score of 6.145 versus 3.963 for DirectChat (+2.182) across GPT, Claude, Gemini, and Qwen backbones. Compared with the structured generation baseline Fars, Story2Proposal obtained an average score of 5.705 versus 5.197, indicating improved structural consistency and visual alignment.
1. Introduction
Story2Proposal addresses structural and cross-section consistency problems in automated scientific manuscript generation with a contract-governed multi-agent framework. Its persistent visual contract and iterative evaluation improve manuscript structure, visual integration, and coherence across generation.
- Motivation: Automated academic writing struggles with global coherence, claim-evidence alignment, citation grounding, and structural integrity across paper-length documents.Direct prompting can produce section drift, claim-experiment misalignment, citation-argument disconnection, and inter-section contradictions.
- Story2Proposal: Story2Proposal coordinates architect, writer, refiner, and renderer agents through a persistent shared visual contract that tracks section obligations and visual artifacts.The contract governs manuscript structure, generated drafts, narrative refinement, and materialized figures, tables, and LaTeX structure.
- Evaluation: Story2Proposal improves structural consistency, visual integration, and overall manuscript coherence across language-model settings and Jericho-derived manuscript scenarios.Qualitative analysis reports more stable figure placement and alignment between narrative reasoning and supporting artifacts.
- Story2Proposal: Generate–evaluate–adapt feedback updates the contract state using reasoning validation, data-fidelity checking, and visual-coherence assessment.This mechanism provides continuous verification rather than relying only on post-generation evaluation.
2. Related Work
Related work uses decomposition, abstraction, and multi-agent collaboration to improve control and quality in complex generation. Story2Proposal focuses on maintaining global consistency across interdependent sections through structured research stories and provenance tracking.
- Modular Approaches to Complex Text Generation: Decomposed Prompting breaks complex tasks into independently solvable subtasks that are later composed to improve multi-step reasoning performance.Take a Step Back adds an abstraction layer by generating high-level principles before addressing specific instances.
- Modular Approaches to Complex Text Generation: Story2Proposal introduces structured research stories as a semantic intermediate layer with explicit provenance tracking between story fields and generated paper sections.This design targets global consistency across multiple interdependent sections in long-form documents.
- Multi-Agent Systems: Multi-agent systems address complex tasks through specialized agent collaboration, with prior frameworks composing agents or language models with distinct capabilities.Examples include automatic agent search in modular design spaces and orchestration of multimodal reasoning models.
3. Method
Story2Proposal frames manuscript generation as coordinated work by specialized agents operating under a persistent shared visual contract. The contract records structural and visual obligations, while evaluation feedback updates it to maintain document-wide consistency.
- Multi-agent architecture: Story2Proposal coordinates architect, writer, refiner, and renderer agents through an evolving contract state.Evaluation agents assess reasoning, data fidelity, and visual coherence during generation.
- Shared visual contract: The shared visual contract stores global artifact registrations, section-level obligations, and validation rules for labels, references, and narrative–visual alignment.Its registry covers figures, tables, descriptions, labels, and expected reference locations.
- Multi-agent generation pipeline: The architect converts a research story into an ordered manuscript blueprint, section arguments, and registered visual artifacts with canonical labels.Artifacts are mapped to section-level requirements in the contract.
- Multi-agent generation pipeline: Writers draft sections under contract constraints, matching visual markers and citation identifiers to the registry for consistent references.Drafts follow planned argument structures and include required visual references.
- Evaluation and adaptation: Evaluation feedback updates the contract state, allowing new validation rules, placement constraints, or explanatory requirements to be introduced during generation.The resulting process replaces a static prompt with coordinated generation and verification under a continuously updated contract.
- Multi-agent generation pipeline: The refiner aligns drafts globally by reducing redundancy, harmonizing terminology, and ensuring referenced visual elements are appropriately described.Detected inconsistencies can trigger further contract updates through evaluation feedback.
4. Experiments
The experiments evaluate Story2Proposal across language-model backbones and expert-reviewed manuscript-generation settings, using aggregate scores over eight manuscript-quality dimensions. Story2Proposal consistently outperforms DirectChat and Fars, with gains in structural consistency, formatting stability, and narrative–visual alignment.
- Implementation details: The implementation coordinates architect, writer, refiner, and renderer agents through a shared visual contract and evolving contract state.Agents operate sequentially within a generate–evaluate–adapt loop, while the renderer produces deterministic LaTeX and validates contract compliance before evaluation.
- Evaluation protocol: The evaluation combines two generation methods, four LLM backbones, and ten independent expert reviewers assessing complete manuscripts.Reviewers score structural integrity, writing clarity, methodological rigor, experimental substance, citation hygiene, reproducibility, formatting stability, and visual communication.
- DirectChat comparison: Story2Proposal achieved an average expert evaluation score of 6.145 versus 3.963 for DirectChat, an improvement of +2.182 across four model settings.Story2Proposal outperformed DirectChat with GPT, Claude, Gemini, and Qwen.
- DirectChat comparison: Expert reviewers reported fewer structural inconsistencies, clearer experimental descriptions, and more reliable formatting in Story2Proposal outputs.Improvements were particularly visible in structural integrity and formatting stability, including more consistent section organization and fewer layout errors.
- Fars comparison: Story2Proposal achieved a mean expert evaluation score of 5.705 versus 5.197 for Fars across four benchmark papers, an improvement of +0.508.Story2Proposal scored higher on Escrowed Batch Reveal, Symbolic Execution, Hazard-Signature Tombstones, and Poisoning LLM-Induced Rules.
5. Analysis
Story2Proposal’s persistent contract state and generate–evaluate–adapt loop improve structural and visual consistency during manuscript generation. Across model backbones and against Fars, it outperforms baselines, while its gains are smaller against already structured systems and remain dependent on evaluation quality and input completeness.
- Cross-Model Consistency: 6.145 versus 3.963 for DirectChat, a +2.182 improvement, across GPT, Claude, Gemini, and Qwen backbones.The improvement remained stable across multiple model backbones.
- Structural Robustness: The persistent shared visual contract specifies section structure, visual artifacts, and validation rules throughout manuscript construction.Agents operate within an explicit structural specification rather than relying on implicit formatting decisions.
- Feedback Mechanisms: The generate–evaluate–adapt loop continuously assesses reasoning, structural integrity, and visual consistency, then modifies the contract state to correct issues early.Evaluation occurs during generation rather than only after the manuscript is produced.
- Visual Communication: Registered visual artifacts with placement obligations help preserve alignment between figures, tables, and narrative text during iterative rewriting.Agents reference registered artifacts rather than regenerating figure markers.
- Comparison with Structured Baselines: 5.705 versus 5.197 for Fars, a +0.508 improvement, with Story2Proposal scoring higher across all four benchmark papers.The benchmark papers were Escrowed Batch Reveal, Symbolic Execution, Hazard-Signature Tombstones, and Poisoning LLM-Induced Rules.
- Failure Modes and Scope: The improvement over Fars is smaller than the improvement over DirectChat because structured baselines already mitigate many basic structural errors.The largest benefits arise when replacing unconstrained prompt-based generation systems.
- Failure Modes and Scope: If evaluation agents miss subtle reasoning weaknesses, manuscripts may remain structurally valid while containing less rigorous reasoning.Richer evaluation signals and more specialized critique modules are identified as potential improvements.
- Failure Modes and Scope: Incomplete research stories can yield logically consistent manuscripts that still reflect gaps in the underlying narrative because the framework does not generate missing scientific evidence.Addressing this boundary may require retrieval or evidence-verification mechanisms before generation.
6. Conclusion
Story2Proposal models scientific manuscript generation as structured construction by coordinating specialized agents under shared structural and visual obligations. It achieves consistent gains over DirectChat and Fars across evaluated backbones and benchmark papers.
- Story2Proposal transforms research stories into coherent manuscripts while enforcing shared structural and visual obligations.
- 6.145 versus 3.963 for DirectChat, a +2.182 improvement, across four model backbones.
- Story2Proposal averages 5.705 versus 5.197 for Fars, with higher scores on all four benchmark papers.