HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale

1 INTRODUCTION

• Most existing software agents focus on a specific SE task, while real-world challenges call for versatility across tasks, languages, and development scenarios.Source: §1 · p. 1
• HyperAgent is a generalist multi-agent system designed to resolve a broad range of SE tasks by drawing on typical software-engineering workflows.Source: §1 · p. 1
• The developer workflow moves through planning, feature localization, editing, and execution, repeating earlier phases when testing reveals issues.The precise process can vary by task and developer; some tasks need only one phase, while others take multiple iterations.Source: §1 · p. 2
• The framework’s stated advantages are generalizability across tasks, efficiency through role-specific models, and scalability for numerous subtasks.Source: §1 · p. 3
• HyperAgent works off-the-shelf across diverse software engineering tasks and programming languages, often exceeding specialized systems.Its evaluation includes GitHub issue resolution, repository-level code generation, and fault localization and program repair.Source: §1 · p. 3

2.1 DEEP LEARNING FOR AUTOMATED PROGRAMMING

• Code-focused language models underpin tools for programming tasks, including competitive coding benchmarks such as HumanEval, MBPP, APPS, and CRUXEval.Source: §2.1 · p. 3

2.2 BENCHMARKS FOR SOFTWARE ENGINEERING

• Software-engineering benchmarks add dimensions such as libraries, test coverage, edit scope, and contamination robustness, while many still use short self-contained problems.SWE-bench models GitHub issue resolution, while Defects4J and BugsInPy address fault localization and repair.Source: §2.2 · p. 3

2.3 AUTONOMOUS CODING AGENTS

• Autonomous coding agents improve code-generation workflows and extend toward requirements-based software generation and GitHub issue resolution.Examples include MetaGPT, AgileCoder, and ChatDev for complex software generation, and SWE-Agent, AutoCodeRover, and agentless approaches for GitHub issues.Source: §2.3 · p. 4

3 HYPERAGENT: A GENERALIST SOFTWARE AGENT FRAMEWORK

• HyperAgent places advanced reasoning in the Planner and delegates computationally intensive but conceptually simpler work to specialized child agents.Source: §3 · p. 4

3.1 CENTRALIZED MULTI-AGENT SYSTEM

• The Planner plans and coordinates iteratively, the Navigator retrieves codebase information, the Editor modifies code, and the Executor validates solutions and reproduces issues.Source: §3.1 · pp. 4–5

3.2 AGENT COMMUNICATION AND SCALABILITY

• An asynchronous distributed Message Queue lets child agents process subtasks in parallel, while summaries return execution details to the Planner.Tasks use messages with Context and Request fields; failed tasks can be requeued, and agents can be scaled independently.Source: §3.2 · p. 5

3.3 TOOL DESIGN

• Role-specific tools support code search and navigation, patch editing, and stateful command execution, with concise outputs designed for LLM interpretation.The Navigator uses Zoekt-based trigram search and IDE-like features; the Editor’s repair tool handles syntax and indentation issues.Source: §3.3 · pp. 5–6

3.4 ADAPTING INTO SPECIFIC SE TASKS WITH MINIMAL CONFIGURATION

• Task templates provide task-specific inputs and general instructions; Prediction tasks omit the Editor, while Patch tasks require code editing.The framework is designed to adapt with minimal configuration, and templates can be populated with task data.Source: §3.4 · p. 6

4 IMPLEMENTATION DETAILS

• The study tests four HyperAgent configurations using different combinations of language models for the Planner, Navigator, Editor, and Executor.Source: §4 · p. 6
• The Planner and Editor are assigned more capable models for reasoning and coding, while the Navigator and Executor can use faster, less powerful models.Source: §4 · p. 6
• The architecture allows individual agent models to be updated without overhauling the system and balances capability against computational cost.Source: §4 · p. 6

5.1.1 SETUP

• SWE-bench evaluates issue–pull-request pairs through unit tests; this study uses Lite (300 instances) and Verified (500 instances) and tracks resolution, time, and token cost.The table reports resolved-instance percentages, average time in seconds, and average cost in US dollars.Source: §5.1.1 · p. 6, Table 1 · p. 7

5.1.2 RESULTS

• On SWE-bench Lite, HYPERAGENT-Full-1 reaches 26.00%, ahead of Agentless + GPT-4o (24.30%) and SWE-Agent + Claude 3.5 Sonnet (23.00%).On Verified, it scores 33.00%, close to SWE-Agent + Claude 3.5 Sonnet (33.60%) and Agentless + GPT-4o (33.20%).Source: Table 1 · p. 7, §5.1.2 · p. 7
• HYPERAGENT-Lite-1 and Lite-2 average 106 and 108 seconds, respectively, versus 720 seconds for AutoCodeRover + GPT-4o.Lite-1 costs $0.45 and scores 25.33% on Lite, compared with SWE-Agent + GPT-4o at $2.55.Source: §5.1.2 · p. 7

5.2.1 SETUP

• RepoExec contains 355 Python code-generation samples with 96.25% test coverage; HYPERAGENT is evaluated without supplied gold contexts to test its repository navigation.The comparison includes RAG baselines and full-context CodeLlama and StarCoder models; Pass@1 and Pass@5 require all tests to pass.Source: §5.2.1 · p. 7

5.2.2 RESULTS

• On RepoExec, HYPERAGENT-Lite-3 achieves Pass@5 of 53.33%, while CodeLlama-34b-Python with full context has the highest Pass@1 at 42.93%.HYPERAGENT-Lite-3 automatically retrieves relevant context; the passage reports RAG-based models underperforming both HYPERAGENT and full-context models.Source: §5.2.2 · p. 7, Table 2 · p. 8

5.3.1 SETUP

• Defects4J evaluation covers 353 active version 1.0 bugs and additional version 2.0 repair bugs, using Acc@k for localization and plausible and correct patch counts for repair.Correctness is assessed by comparing the generated fix's abstract syntax tree with the developer's original fix.Source: §5.3.1 · p. 8

5.3.2 RESULTS

• HYPERAGENT records Acc@1 of 59.70% on Defects4J fault localization, exceeding AutoFL's 51.00% by 8.7 percentage points.The comparison table covers Acc@1 across fault-localization methods.Source: Table 3 · p. 8, §5.3.2 · p. 8
• HYPERAGENT makes 82 correct fixes (20.8%) on Defects4J v1.2 and 110 (25.0%) on v2, outperforming the listed competing repair tools.Across project-level results, it achieves 249 plausible fixes and 192 correct fixes, corresponding to 29.8% and 22.9%.Source: §5.3.2 · pp. 8–9, Table 4 · p. 9

6.1 ABLATION STUDIES ON AGENT ROLES

• Removing any agent role raises cost and lowers resolution, with Navigator removal causing the largest performance drop.When a medium-long-context LLM replaces the Editor or Navigator as Planner, the resolving-rate drop is more severe.Source: §6.1 · p. 9, Table 5 · p. 9

6.2 ANALYSIS OF TOOL DESIGN

• Proximity-based search raises go-to-definition performance from 9% without search to 15% with search.For open-file navigation, keyword search increases resolving rate by 3%; search context and result reranking raise performance from 3% without the tool to 14%.Source: §6.2 · pp. 9–10, Table 6 · p. 10

6.3 AGENT BEHAVIOR

• Planner activity is highest early; Editor requests often follow Navigator requests, while Executor use rises later for tests, with an early peak for issue reproduction.Editor requests peak at iterations 4 and 8.Source: §6.3 · p. 10, Figure 4 · p. 10

6.4 ERROR ANALYSIS

• HYPERAGENT has a lower Edit failed loop error ratio than SWE-Agent through automatic code repair, but trajectories can exit early through hallucination or time out.The paper attributes hallucination risk to communication losses that can leave the Planner without execution results or context location.Source: §6.4 · pp. 10–11, Figure 3 · p. 10

• HYPERAGENT follows software engineering workflows across GitHub issue resolution, repository-level code generation, and fault localization and program repair.The conclusion describes stages for analysis, planning, feature localization, code editing, and execution or verification.Source: §7 · p. 11
• Future work includes integration with development environments and version control, expansion into specialized domains, improved explainability, and continual knowledge-base updates.The authors identify security-focused code review and performance optimization as possible specialized domains.Source: §7 · p. 11

• Identify the cause of a GitHub issue, collect relevant information, and provide a solution.The issue itself is supplied as a placeholder.Source: Github Issue Resolution · p. 18

• Given a failed test and its error message and call stack, identify the responsible method without editing the code.The requested output is the method name in package.ClassName.methodName format.Source: Fault Localization · p. 18

A.2.1 AGENT CONFIGURATION

• HYPERAGENT flexibly uses LLMs of different strengths according to each agent’s needs.Table 7 is titled “HYPERAGENT Configurations.”Source: §A.2.1 · p. 18, Table 7 · p. 18
• GPT-4 and Claude-3 Sonnet are designated stronger closed-source models, while Claude-3 Haiku is weaker.Source: §A.2.1 · p. 18
• For open-source models, Llama-3-70B is stronger and Llama-3-8B is its weaker counterpart.Source: §A.2.1 · p. 18
• The authors describe HYPERAGENT as the first system to evaluate SWE-Bench with open-source models like Llama-3, offering a cost-efficient alternative with competitive performance across SE tasks.Source: §A.2.1 · p. 18

A.3.1 NAVIGATION TOOLS

• Zoekt search parses code with an AST, ranks symbols, and can fall back to line-by-line matching; results summarize files, identifiers, and snippets for LLMs.Navigation also includes locating definitions, finding references, listing symbols, viewing directory trees, and opening files.Source: §A.3.1 · p. 18, interactive shell · p. 19, code search · p. 19

interactive shell

• The interactive shell maintains execution state across command sequences and can open testing and setup documentation.Source: interactive shell · p. 19

A.3.2 GO TO DEFINITION

• The go-to-definition tool uses LSPToolKit to retrieve symbol definitions across languages, with optional line refinement, documentation, snippets, and partial output for long definitions.LSPToolKit abstracts backend protocols for languages including Python, JavaScript, and Java.Source: §A.3.2 · p. 21

A.3.3 GET ALL REFERENCES

• The get-all-references tool uses LSPToolKit to find a symbol’s uses within a file and across the project, supporting codebase navigation and usage analysis.Inputs include a symbol name and relative file path, with an optional line number.Source: §A.3.3 · p. 22

A.3.4 GET ALL SYMBOLS

• The get-all-symbols tool uses ctags for fast, lightweight extraction across languages, returning symbol names, line ranges, and definitions, with optional keyword filtering.Keyword filtering prioritizes exact over partial matches, and the tool avoids requiring an LSP server.Source: §A.3.4 · p. 24

A.3.5 GET TREE STRUCTURE

• The tree-structure tool displays a project’s directory hierarchy to a user-specified depth, enabling initial exploration from a relative path.For an incorrect path, it suggests trying the parent directory.Source: §A.3.5 · p. 24

A.3.6 OPEN FILE

• Open file supports keyword, line-range, and semantic queries, returning bounded file content or matching segments and suggesting paths when a file is not found.Output limits help keep inspection focused and avoid overwhelming the agent’s context.Source: §A.3.6 · pp. 25–26

A.3.7 EDITOR TOOLS

• The repair editor replaces a specified code range with an agent-provided patch, checking paths, ranges, and syntax and optionally using an integrated LLM for review.It can format Python and check syntax with autopep8 and flake8, then applies valid patches or returns errors and before-after snippets.Source: §A.3.7 · p. 27
• The editor supports precise modifications in large codebases, while the interactive shell executes bash in a Docker container and captures output.The container is automatically stopped and cleaned up after completion or error.Source: §A.3.7 · p. 28, §A.3.8 · pp. 28–29

A.4.1 GITHUB ISSUE RESOLUTION

• SWE-bench evaluation uses Lite (300 instances) and Verified (500 instances), measuring resolved-instance percentage, average time cost, and average token cost.Lite filters instances with heuristics, while Verified instances were manually validated by professional annotators.Source: §A.4.1 · pp. 29–30

A.4.2 REPOSITORY-LEVEL CODE GENERATION DETAILS

• RepoExec evaluation omits supplied gold contexts to test independent repository navigation, using pass@1 and pass@5 across 355 samples with 96.25% test coverage.Comparisons include RAG and sparse-RAG baselines, plus CodeLlama and StarCoder with full context as upper bounds.Source: §A.4.2 · p. 30

A.4.3 FAULT LOCALIZATION

• Fault localization covers all 353 active Defects4J v1.0 bugs and uses acc@k with an ordinal tiebreaker to assess whether buggy locations appear among top suggestions.Baselines include deep-learning methods DeepFL, AutoFL, and Grace, and traditional static-analysis methods DStar and Ochiai.Source: §A.4.3 · p. 30, Baselines · p. 30, Metrics · pp. 30–31

A.5.1 DATASET

• For repair evaluation, HYPERAGENT Lite-1 is compared with RepairAgent, SelfAPR, and ITER, counting plausible patches and AST-matched correct patches.Defects4J provides gold-standard fixes and test cases; passing all tests establishes plausibility, while correctness requires an exact AST match to the developer fix.Source: §A.5.1 · p. 31

A.6 RESULT DETAILS ON PROGRAM REPAIR

• 249 plausible fixes and 192 correct fixes give HYPERAGENT the best Defects4J repair performance reported in Table 10.The table reports plausible and correct fix counts for HYPERAGENT, but only correct-fix counts for RepairAgent, ITER, and SelfAPR.Source: Table 10 · p. 31

A.7 PROMPTS

• The Planner prompt directs sequential delegation among Navigator, Editor, and Executor, emphasizing root-cause localization, targeted patching, and verification before task completion.It assigns distinct roles: Navigator gathers codebase information, Editor modifies code, and Executor reproduces issues and validates fixes.Source: §A.7 · p. 32

A.7.1 PROMPT TEMPLATE FOR NAVIGATOR

• The Navigator prompt limits the agent to gathering repository information for the Planner, using predefined tools strategically and returning relevant findings without guessing.It encourages trying alternate tools or parameters when initial searches or file views are insufficient.Source: §A.7.1 · p. 33

A.7.2 PROMPT TEMPLATE FOR EDITOR

• The editor prompt directs careful codebase edits: inspect the latest file, use predefined functions, preserve syntax and context, and return a Thought followed by an Action block.It also describes editor functions for patching, exploring folders, opening files, and finding symbol definitions or references.Source: §A.7.2 · pp. 34–35

A.7.3 PROMPT TEMPLATE FOR EXECUTOR

• The executor prompt limits the agent to requested bash commands, one action per code block, and reporting execution results rather than editing or solving the issue.It permits creating and running test files only when the planner requests test execution.Source: §A.7.3 · p. 36

A.8 HYPERAGENT TRAJECTORIES ON SWE-BENCH-LITE

• The Astropy issue requests header_rows support for ascii.rst output, which currently raises a TypeError when that argument is passed.The issue’s example reports that RST.__init__() received an unexpected keyword argument.Source: §A.8 · pp. 36–37

HYPERAGENT Planner

• The planner proposes extending Astropy’s RST writer to accept and format additional header rows in its output.The suggested changes add a header_rows parameter and insert formatted rows before writing the table.Source: HYPERAGENT Planner · p. 39

HYPERAGENT Executor

• Verification encounters a missing cols attribute in the RST class, and the executor’s subsequent attempt still reports that the attribute is absent.The supplied passages do not show a successful verification result.Source: HYPERAGENT Executor · p. 42
• The Executor identifies that the RST writer lacks header-row support and first proposes the CSV writer as a workaround, which does not address the requested RST output.The example output contrasts tables without and with header_rows using CSV.Source: HYPERAGENT Executor · p. 43

HYPERAGENT Planner

• The Planner’s proposed RST implementation accepts header_rows, formats those rows before column names, and adds separators around the formatted table.The patch also forwards keyword arguments to the superclass and returns an empty list for empty input.Source: HYPERAGENT Planner · p. 43, Patch · pp. 48–49
• The RST writer successfully formats tables with and without additional header rows, placing the requested name and unit rows before the regular column headers and data.The reported test output confirms the added rows appear in RST-style output.Source: HYPERAGENT Planner · pp. 46–47

Patch

• The patch updates astropy/io/ascii/rst.py to add header_rows handling and table-formatting helpers, while retaining a stated limitation on reading continuation-line and column-span tables.The helpers determine column widths, build separators, and format rows.Source: Patch · pp. 48–49