Source-linked AI summary
WiseSpec: Requirements-Driven Agents for Code Generation
Zhao Tian
TL;DR
Complex code-generation tasks remain difficult because requirements can be ambiguous, incomplete, or missing critical context, while existing approaches largely focus on agent capabilities. WiseSpec constructs structured requirements, assesses them through execution-based evaluation, and iteratively refines them; it consistently outperforms baselines, with an average improvement of 13.17% in %Resolved.
Problem
Ambiguous, incomplete, and context-poor task requirements remain an overlooked challenge in complex repository-level code generation.
Method
WiseSpec retrieves repository context, constructs structured requirements with a DSL, evaluates them through executable code and tests, and iteratively refines them.
Results
WiseSpec consistently outperforms all baselines across two LLMs and three benchmarks, achieving an average improvement of 13.17% in %Resolved.
Takeaways & Limitations
The results support requirements engineering as an effective direction for improving LLM-based repository-level code generation.
Abstract
from arXiv · showhide
Code generation aims to automatically generate source code from task requirements and has attracted significant attention with the rapid advancement of large language models (LLMs). Despite remarkable progress, LLMs often struggle to generate correct code for complex software engineering tasks because task descriptions are frequently incomplete, ambiguous, or lack critical contextual information. Existing approaches primarily improve the capabilities of coding agents through more sophisticated tools, skills, and workflows, while largely overlooking the quality of the task requirements themselves. To address this limitation, we draw inspiration from software requirements engineering and propose WiseSpec, a novel requirements-driven agent framework for repository-level code generation. WiseSpec automatically constructs structured and information-rich requirements, assesses their quality through execution-based evaluation, and iteratively refines them to better guide code generation. Experimental results show that WiseSpec consistently outperforms all baselines, achieving an average improvement of 13.17% in %Resolved.
1 Introduction
WiseSpec addresses a central weakness in repository-level code generation: task requirements are often ambiguous, incomplete, or context-poor, while existing methods mainly improve how coding agents solve tasks. It therefore constructs richer requirements before generation.
- Existing coding-agent approaches primarily enhance tools, skills, and workflows rather than the quality of task requirements.
- Task descriptions often omit, ambiguously state, or incompletely specify intended program behavior and critical context.
- WiseSpec collects context and constructs structured, information-rich requirements using a predefined domain-specific language.
- WiseSpec evaluates requirement quality through execution-based code evaluation and iteratively refines and aligns requirements.
2 Approach
WiseSpec generates structured requirements from repository context, evaluates their quality through executable code and tests, and iteratively refines deficient requirements using categorized feedback and quality scores.
- WiseSpec comprises Requirement Generation, Requirement Quality Assessment, and Requirement Refinement.
- Requirement Generation: Requirement Generation iteratively retrieves and analyzes code snippets along program dependencies, then organizes them through a DSL with nine attributes and seventeen sub-attributes.
- Requirement Quality Assessment: Requirement Quality Assessment generates executable code and tests from synthesized requirements, accepting requirements only when all generated tests pass.
- Requirement Refinement: Requirement deficiencies are categorized as Conflict, Omission, or Ambiguity and addressed with predefined alignment rules and iterative reassessment.
- Requirement Refinement: A greedy strategy retains the candidate requirement with the highest quality score at each refinement iteration.
3 Experiments and Results
The evaluation compares WiseSpec with three coding agents across three repository-level benchmarks and two LLMs, using syntactic and functional correctness metrics. WiseSpec performs best across all six settings and also improves results with Claude-Opus-4.8.
- Process: %Applied measures whether generated code can be applied to the codebase, while %Resolved measures whether it passes all gold tests.
- Results: Across all six experimental settings, WiseSpec achieves the best performance and improves %Resolved by 2%∼29% and %Applied by 11%∼63% over baselines.
- Results: WiseSpec raises %Resolved from 53% with Claude Code to 56% with Claude-Opus-4.8 on SWE-bench-Pro.
- Results: The improvements over all baselines are statistically significant for both %Resolved and %Applied, with 𝑝< 2.5 × 10−4.
4 Conclusion
WiseSpec identifies task-requirement quality as a fundamental bottleneck in repository-level code generation and addresses it with a requirements-driven agent. Its experiments show consistent gains over baselines across multiple metrics.
- WiseSpec automatically constructs structured requirements, assesses their quality through execution, and iteratively refines them to improve code correctness.
- WiseSpec consistently outperforms all baselines across multiple evaluation metrics, supporting the effectiveness of requirements engineering for LLM-based code generation.