PaperlayerView exact revision on arXiv

Source-linked AI summary

FullStack-Agent: Enhancing Agentic Full-Stack Web Coding via Development-Oriented Testing and Repository Back-Translation

Zimu Lu, Houxing Ren, Yunqiao Yang, Ke Wang, Zhuofan Zong, Mingjie Zhan, Hongsheng Li

arXiv:2602.03798v1cs.SEcs.CLcs.CV

TL;DR

Existing LLM coding agents often produce frontend-only websites, while production-grade full-stack generation requires reliable data flow, complex codebase handling, and obscure-bug localization. FullStack-Agent combines multi-agent development, repository-based self-improvement, and full-stack benchmarking. FullStack-Dev outperforms the prior state of the art across frontend, backend, and database tests, while FullStack-Learn improves a 30B model across all three test sets.

  • Problem

    Existing code agents often generate frontend-only websites without real backend data processing and storage, despite the greater complexity of production-level full-stack applications.

  • Method

    FullStack-Agent combines FullStack-Dev for multi-agent coding and debugging, FullStack-Learn for repository augmentation and back-translation, and FullStack-Bench for frontend, backend, and database evaluation.

  • Results

    FullStack-Dev outperforms the previous state-of-the-art method by 8.7%, 38.2%, and 15.9% on frontend, backend, and database tests, respectively.

  • Takeaways & Limitations

    The results demonstrate the effectiveness of the unified approach for full-stack code generation and of FullStack-Learn for self-improving the backbone model.

  • Takeaways & Limitations

    Agent-generated websites may create unclear responsibility for errors or vulnerabilities and may reduce demand for human programmers.

Abstract

from arXiv · show

Assisting non-expert users to develop complex interactive websites has become a popular task for LLM-powered code agents. However, existing code agents tend to only generate frontend web pages, masking the lack of real full-stack data processing and storage with fancy visual effects. Notably, constructing production-level full-stack web applications is far more challenging than only generating frontend web pages, demanding careful control of data flow, comprehensive understanding of constantly updating packages and dependencies, and accurate localization of obscure bugs in the codebase. To address these difficulties, we introduce FullStack-Agent, a unified agent system for full-stack agentic coding that consists of three parts: (1) FullStack-Dev, a multi-agent framework with strong planning, code editing, codebase navigation, and bug localization abilities. (2) FullStack-Learn, an innovative data-scaling and self-improving method that back-translates crawled and synthesized website repositories to improve the backbone LLM of FullStack-Dev. (3) FullStack-Bench, a comprehensive benchmark that systematically tests the frontend, backend and database functionalities of the generated website. Our FullStack-Dev outperforms the previous state-of-the-art method by 8.7%, 38.2%, and 15.9% on the frontend, backend, and database test cases respectively. Additionally, FullStack-Learn raises the performance of a 30B model by 9.7%, 9.5%, and 2.8% on the three sets of test cases through self-improvement, demonstrating the effectiveness of our approach. The code is released at https://github.com/mnluzimu/FullStack-Agent.

1. Introduction

FullStack-Agent addresses the difficulty of generating production-grade full-stack websites, where existing agents often produce frontend-only interfaces without real backend data flow. It combines coordinated development, model self-improvement, and comprehensive evaluation, achieving substantial gains across frontend, backend, and database tests.

  • Motivation: Existing coding agents often generate frontend-only websites that simulate interactivity without implementing real backend data flow or storage.This limitation can make submitted forms appear successful while failing to persist data.
  • Challenges: Production-grade full-stack generation requires codebase navigation, obscure-error localization, long-term reasoning, and reliable evaluation of backend and database functionality.The paper identifies complex frameworks, demanding workflows, and insufficient existing benchmarks as key challenges.
  • Approach: FullStack-Dev coordinates planning and frontend/backend coding agents equipped with code editing, execution, and debugging tools.Its specialized debugging tools are designed to locate and correct subtle frontend and backend errors.
  • Approach: FullStack-Learn improves backbone LLMs by generating training trajectories through repository augmentation and back-translation of website repositories.The method uses repository-derived trajectories for supervised fine-tuning.
  • Evaluation: FullStack-Bench evaluates generated websites across frontend, backend, and database functionalities rather than relying only on visible frontend reactions.The benchmark uses agent judges and multiple test cases, while gathering database logs during frontend and backend tests.
  • Results: 64.7%, 77.8%, and 77.9% accuracy on frontend, backend, and database tests respectively exceed the previous state of the art by 8.7%, 38.2%, and 15.9%.FullStack-Learn additionally improves a Qwen3-Coder-30B model by 9.7%, 9.5%, and 2.8% on the three test sets.

2. FullStack-Agent

FullStack-Agent integrates a multi-agent coding framework, a repository-based self-improvement pipeline, and a benchmark that tests full-stack functionality. Its components coordinate planning, implementation, debugging, trajectory generation, data scaling, and frontend/backend/database evaluation.

  • FullStack-Dev: FullStack-Dev uses a Planning Agent and sequential backend and frontend Coding Agents to implement full-stack websites in sandbox environments.The backend agent supplies API summaries that guide subsequent frontend development.
  • FullStack-Dev: The Planning Agent converts user instructions into structured frontend and backend plans covering layouts, components, data flow, entities, and API endpoints.Plans are output in JSON for downstream coding agents.
  • FullStack-Dev: FullStack-Dev equips coding agents with code manipulation, shell execution, and specialized frontend and backend debugging tools.The frontend tool monitors browser and terminal outputs, while the backend tool sends requests and returns responses plus console output.
  • FullStack-Learn: FullStack-Learn back-translates real repositories into agent trajectories and uses repository augmentation to scale training data for backbone-model improvement.Information gathering produces repository summaries and plans, while trajectory back-translation reproduces repositories in empty framework templates.
  • FullStack-Learn: Repository Augmentation creates five times the number of original repositories, increasing the scale of generated trajectories.The method modifies existing repositories rather than generating entirely new codebases from scratch.
  • FullStack-Bench: FullStack-Bench tests frontend, backend, and database functionality with comprehensive test cases to detect deficiencies that frontend-focused benchmarks miss.Its design targets false positives where frontend effects appear correct despite missing backend implementation.

3. Experiments

Experiments evaluate FullStack-Dev and FullStack-Learn on frontend, backend, database, and related reliability measures. The results show improvements from the unified agent framework, self-improvement training, debugging tools, and repository back-translation.

  • FullStack-Dev Results: FullStack-Dev with Qwen3-Coder-480B-A35B-Instruct achieves 64.7%, 77.8%, and 77.9% accuracy on frontend, backend, and database tests.These are the highest reported accuracies in the evaluation.
  • FullStack-Dev Results: 8.7%, 38.2%, and 15.9% are FullStack-Dev’s improvements over WebGen-Agent on frontend, backend, and database tests, respectively.WebGen-Agent reports 56.0%, 39.6%, and 62.0% on the same tests.
  • FullStack-Dev Results: FullStack-Dev produces higher backend than frontend accuracy, whereas most baselines have much lower backend accuracy and often use mock data despite explicit backend requests.The comparison indicates that FullStack-Dev’s generated websites mostly possess functional backends.
  • FullStack-Learn Results: 9.7%, 9.5%, and 2.8% are the gains from two rounds of FullStack-Learn training on frontend, backend, and database accuracy.The resulting accuracies are 46.9%, 48.2%, and 53.7%, respectively, using Qwen3-Coder-30B-A3B-Instruct.
  • Ablation Studies: Removing the multi-agent mechanism reduces accuracy across all metrics, while each debugging tool most affects its corresponding frontend or backend accuracy.Removing both debugging tools considerably degrades both frontend and backend performance.
  • Ablation Studies: Repository Back-Translation significantly improves accuracy and appearance, while directly generated trajectories fail to notably improve overall performance.The performance gap is attributed to learning from high-quality, real-world repositories.
  • Ablation Studies: FullStack-Bench’s frontend, backend, and database human-alignment accuracies all exceed 90% on 200 sampled instances per test type.Four student volunteers manually checked correctness using evaluation trajectories and database interaction logs.

4. Related Work

Prior website-development agents and benchmarks largely emphasize frontend generation and evaluation, while FullStack-Agent targets full-stack code generation and testing. Existing training methods also address different tasks or do not leverage existing codebases.

  • Website Development Agents and Pipelines: MRWeb generates only HTML and CSS, while Bolt.diy, WebGen-Agent, and TDDev generally produce simple codebases with little or no backend or database implementation.These systems may add backend components only when specially prompted.
  • Website Development Benchmarks: Existing website benchmarks mostly focus on frontend appearance and functionality, with some evaluating simple HTML generation from design images.Web-Bench uses fixed pipelines and does not evaluate agentic-system abilities comprehensively.
  • Training Methods to Improve Software Development: WebCode2M and WebSight use screenshot–HTML pairs, while other reinforcement-learning and supervised fine-tuning methods target related software-development abilities.These methods are described as distinct from generating full-stack websites from scratch.
  • Training Methods to Improve Software Development: Existing training approaches do not apply directly because this task generates full-stack websites from scratch rather than patches to existing codebases.WebGen-Agent uses visual-feedback reinforcement learning but does not leverage existing codebases.

5. Conclusion

FullStack-Agent combines multi-agent full-stack generation, repository-based self-improvement, and comprehensive frontend, backend, and database evaluation. Experiments report substantial gains for both FullStack-Dev and FullStack-Learn.

  • Conclusion: FullStack-Agent combines FullStack-Dev, FullStack-Learn, and FullStack-Bench into a unified full-stack agentic coding system.The components cover development, backbone-LLM improvement, and evaluation of frontend, backend, and database functionality.
  • Conclusion: FullStack-Dev outperforms the previous state-of-the-art method by 8.7%, 38.2%, and 15.9% on frontend, backend, and database tests.The reported accuracies are 64.7%, 77.8%, and 77.9%, respectively.

Impact Statement

The impact statement notes that agentic code generation may affect software-development employment and complicate responsibility for errors or vulnerabilities. It emphasizes responsible deployment.

  • Impact Statement: Agentic code generation may reduce demand for human programmers as it potentially improves software-development efficiency.The statement presents this as a possible societal implication rather than an established outcome.
  • Impact Statement: Responsibility for errors or vulnerabilities in agent-generated websites remains unclear and could introduce new challenges.The statement highlights this issue as a reason for responsible deployment.

A. Tool Details

The system uses specialized tools and a trajectory-transformation procedure to normalize repository-derived actions into coding-agent workflows. Experiments primarily use Next.js and NestJS, with Vue.js and Django added to assess template generalization.

  • The toolset supports code reading, directory listing, file search, editing, shell execution, backend debugging, and frontend debugging.
  • Next.js and NestJS are used as frontend and backend templates to stabilize comparisons and reduce data-generation and training costs.
  • Adding Vue.js and Django enables template selection based on user instructions and slightly increases frontend, backend, and database accuracies.
  • Back-Translation Trajectory Transforming Process: Back-translation trajectories are normalized by rewriting project paths, canonicalizing staged prompts, cleaning repository references, and pruning incompatible tool calls.
  • Back-Translation Trajectory Transforming Process: The transformation resets the project, replays remaining actions in chronological order, recomputes tool outputs, and injects corrected responses into the final trajectory.

D. Data Filtering Details

Generated trajectories are filtered using appearance, frontend-functionality, and backend-functionality scores derived from debugging outputs. A trajectory is retained only when all three aggregated scores are positive.

  • Appearance and frontend-functionality scores come from frontend debugging calls, while backend-functionality scores come from backend debugging calls.
  • Scores from multiple debugging calls are aggregated with γ = 0.9, causing earlier scores to receive lower weight.
  • For appearance and frontend functionality, scores above threshold 3 are positive; backend functionality uses threshold 0.
  • Trajectories are kept only when the aggregated appearance, frontend-functionality, and backend-functionality scores all exceed zero.

E. Baseline Implementation Details

The baseline study evaluates website-specific and general coding agents using open-source implementations to generate websites for FullStack-Bench. Because these agents typically produce frontend-only pages, the evaluation prompt explicitly requests full-stack support.

  • Baselines include WebGen-Agent, TDDev, Bolt.diy, OpenHands, and Qwen-Code.
  • The tested agents generally generate frontend pages without backend or database support by default.
  • Open-source implementations are used to generate websites for FullStack-Bench, with baseline URLs reported for reproducibility.
  • The authors also plan to release baseline implementation code.

F. Error Analysis

Error analysis identifies distinct frontend, backend, and database failure modes in 300 sampled cases. The largest concentrations involve unimplemented or unresponsive frontend functionality, missing backend database interaction or APIs, and empty databases.

  • The analysis manually inspects 300 errors sampled evenly from six tested code agents and identifies nine frontend, four backend, and four database error types.
  • Frontend errors: Other notable frontend errors include website start failure at 14.3%, data-fetching failure at 9.7%, and form-submission errors at 9.7%.
  • Backend errors: Backend errors include no database interaction at 34.3% and unimplemented APIs at 33.3%, followed by database setup errors at 19.7% and failed connections at 12.7%.
  • Database errors: Database errors are dominated by empty databases at 46.7%, followed by missing data fields at 26.0% and missing tables at 19.7%.
  • Additional error types: Data structures not clearly provided account for 7.7% of database errors, while syntax errors account for 2.7% of frontend errors.
  • Frontend errors: Frontend errors are led by unimplemented functionality at 29.7% and unresponsive components at 23.7%, together exceeding half of frontend errors.

G. Human Annotation Details

The benchmark’s human-annotation process provides interfaces and guidelines for validating frontend, backend, and database test results. Annotators require both evaluation trajectories and database interaction logs to support a sample’s final result.

  • Annotation Interfaces: Human annotators review separate interfaces for frontend, backend, and database test samples.The interfaces display GUI-agent or backend testing trajectories, database interaction logs, schemas, and model responses.
  • Validation Criteria: A sample is correct only when its evaluation trajectory and database interaction logs fully support the final result.Human alignment accuracy is computed as the percentage of samples judged correct under this criterion.
  • Agent Workflow: The framework presents prompts for choosing templates, generating development plans, and coordinating backend and frontend coding agents.The coding agents receive starting, validation, and summary prompts for dynamic coding and debugging.
  • FullStack-Learn: FullStack-Learn uses information gathering, trajectory back-translation, and augmentation prompts to generate training data from repositories.The prompts cover repository exploration, new-project implementation from an old codebase, augmentation planning, implementation, and verification.

J. FullStack-Bench Prompts

The FullStack-Bench prompts coordinate template selection, development planning, agent coding, repository inspection, API and database testing, and visual evaluation. They operationalize checks for frontend behavior, backend interfaces, database state, and website appearance.

  • Frontend Testing: Frontend testing prompts combine GUI-agent testing with database-interaction validation.The validation prompt is appended to the GUI-agent trajectory to assess database interactions accompanying frontend actions.
  • Template Selection: The benchmark prompts select an appropriate website template and identify projects that are pure frontend applications without backend needs.The selection prompt receives the user instruction and available template descriptions, then sets the pure-frontend flag when applicable.
  • Agent Prompts: The system and coding-agent prompts support frontend and backend development, validation, debugging, and testing-summary generation.The GUI-agent summary handles natural completion and premature termination, reporting trajectories, triggering actions, appearance, functionality, and appearance scores.
  • Repository Back-Translation: Repository back-translation prompts gather codebase information and guide backend and frontend implementation of a new project from an old codebase.Additional prompts plan augmentations and verify their implementation using concrete code evidence.
  • Backend Testing: Backend testing prompts gather backend configuration and test API endpoints and database interactions.Information gathering covers the backend port, endpoint methods and paths, request and response schemas, status codes, and database configuration.
  • Database and Appearance Evaluation: Database prompts evaluate whether a runtime-dumped schema contains the data required by a test case, while appearance prompts grade visual design from screenshots on a 1–5 scale.Appearance evaluation considers rendering, colors, fonts, and component display against the instruction.
Loading 2602.03798v1…