PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence

Yuyuan Feng, Zhishang Xiang, Chaobin Yang, Qichao Ma, Zerui Chen, Yujing Zhang, Ke Huang, Chuanjie Wu, Zhaoxu Liu, Yili Wang, Xin He, Jiapu Wang, Zijin Hong, Hao Chen, Yuanchen Bei, Kun Wang, Shengyuan Chen, Ningyu Zhang, Enyan Dai, Linhao Luo, Qingyi Pan, Qi Wang, Wenqi Fan, Guangjing Wang, Na Zou, Yangqiu Song, Xin Wang, Zechao Li, Xia Hu, Qing Li, Xiao Huang, Zhihong Zhang, Jinsong Su, Qinggang Zhang, Yi Chang

arXiv:2608.21156v1cs.IRcs.AIcs.ET

TL;DR

Heterogeneous, interdependent, long-horizon tasks require system-level organization beyond expanding a single agent’s capabilities or context. This survey organizes Graph Engineering around work organization, agent coordination, and runtime state management. Across application domains, work organization and agent team engineering are common, but persistent system evolution remains rare.

  • Problem

    Heterogeneous, interdependent, long-horizon tasks require system-level organization beyond augmenting an individual agent’s capabilities or context.

    Source: §1 · p. 5

  • Method

    The paper reviews Graph Engineering through three complementary graph views: work organization, agent coordination, and runtime state management.

    Source: §1 · p. 5, §10 · p. 35

  • Results

    Across application domains, work organization and agent team engineering are common, while persistent system evolution remains rare.

    Source: §9.7 · p. 34

  • Takeaways & Limitations

    System-level intelligence depends more on explicitly organizing relationships among work, agents, and state than on simply adding models or agents.

    Source: §10 · p. 35

    Explicit graph structures do not guarantee consistent interpretations of task completion, evidence, valid states, or authorized actions across agents.

    Source: §6.1 · p. 25

Abstract

from arXiv · show

LLMs have evolved from language generators to autonomous agents capable of complex, long-horizon tasks. This evolution has produced paradigms including Prompt Engineering to elicit model capabilities, Context Engineering to manage information access, Harness Engineering to organize external tools and resources, and Loop Engineering to support continual reflection and self-improvement. Yet as tasks grow more complex, individual intelligence faces a fundamental limit: many tasks require heterogeneous expertise, interdependent subtasks, parallel execution, independent verification, and persistent state, exceeding any single agent's organizational capacity. Augmenting one agent's capabilities or context cannot resolve this architectural mismatch; intelligence must instead be distributed across specialized agents and organized at the system level. We call this System Intelligence: an agent system's ability to organize and coordinate multiple intelligent components into a coherent, adaptive whole pursuing a shared objective. Achieving it requires more than adding agents; it demands explicit structures to organize work, coordinate heterogeneous agents, and maintain evolving execution states. We introduce Graph Engineering, an emerging paradigm for next-generation agent systems. Unlike prior paradigms that mainly optimize individual interactions or agent-level behavior, Graph Engineering constructs explicit, dynamic, evolving graph structures representing tasks, agents, and system states. These abstractions provide a unified foundation for organizing complex objectives, orchestrating heterogeneous agents, modeling system dynamics, and enabling scalable agent evolution. We systematically review the principles, methodologies, and applications of Graph Engineering for LLM agents. Related papers, open-source data, and projects are collected at https://github.com/DEEP-JLU/Awesome-Graph-Engineering.

1 Introduction

The introduction proposes Graph Engineering as a system-level paradigm for task organization, agent coordination, and runtime state management. It argues that complex objectives require coordinated specialized agents, not merely expanded individual capabilities or more agents.

1 Introduction

  • Harness and Loop Engineering produce Individual Intelligence by combining external resources with persistent cycles of planning, action, observation, verification, and adaptation.Prompt and Context Engineering elicit Model Intelligence through structured instructions and organized task-relevant information. The survey distinguishes individual agents’ foundation models, harnesses, loops, and local states from agent systems’ teams, shared resources, environments, coordination mechanisms, and system-level states.Source: §1 · pp. 4–5
  • System Intelligence requires explicit work organization, responsibility allocation, coordination, and state management because heterogeneous, interdependent tasks exceed a single agent’s organizational capacity.Single-agent loops create contextual competition and predominantly sequential control, preventing independent work isolation, synchronization, and partial recovery. Expanding capabilities or context, or simply increasing agent numbers, does not resolve these organizational limitations.Source: §1 · p. 5
  • Graph Engineering uses operational graph structures to organize tasks, coordinate heterogeneous agents, and manage evolving runtime state as a foundation for System Intelligence.Task Organization represents decomposition, dependencies, concurrency, and verification; Agent Coordination structures assignment, communication, synchronization, and integration. Runtime State Management tracks progress, reconciles concurrent updates, preserves provenance, isolates failures, and supports recovery and adaptation.Source: §1 · p. 5
  • Future research directions include ontology engineering, dynamic and self-evolving graph systems, and graph-native agent operating systems.Source: §1 · p. 6
  • The survey organizes benchmarks, datasets, and executable environments across Model, Individual, and System Intelligence, alongside principles and challenges for evaluating agent systems.Evaluation concerns include structural fidelity, operational correctness, system evolution, and governance.Source: §1 · p. 6
  • The survey reviews engineering stacks and applications, highlighting gaps in interoperability, cross-run structural evolution, state provenance, and persistent System Evolution.Software stacks support model development, persistent agent runtimes, and multi-component orchestration. Applications span software engineering, scientific discovery, healthcare, enterprise workflows, digital agents, and social and economic simulation, where work organization, coordination, and runtime state management are increasingly common.Source: §1 · p. 6

2 Preliminaries

An Individual Agent combines a Foundation Model, an Agent Harness, and an iterative Agent Loop. An Agent System organizes multiple agents through shared resources, an external environment, coordination mechanisms, and system-wide runtime state.

2.1 Individual Agent

  • An Individual Agent combines Foundation Model cognition with an Agent Harness that provides access to resources and actions.The model supports language understanding, reasoning, planning, and generation. The Harness supplies perception and context construction, memory and knowledge access, tool invocation, reusable skills, and runtime governance.Source: §2.1 · p. 6
  • The Agent Loop repeatedly performs perception, reasoning, action, feedback processing, and state updates to continuously employ the agent’s capabilities.The agent is autonomous: it perceives its environment, makes decisions, executes actions, and adapts its behavior according to feedback.Source: §2.1 · p. 6

2.2 Agent System

  • An Agent System organizes agents with potentially different roles, capabilities, and tasks around shared resources and an external environment.Each agent retains its own model, Harness, Loop, and local state; shared resources include tools, model services, memory, knowledge bases, verifiers, and human support accessed through its Harness. The environment supplies observations and feedback and evolves in response to agent actions.Source: §2.2 · pp. 6–7
  • Coordination mechanisms govern collaboration, while system state captures the entire system’s operational condition rather than an individual agent’s local state.Coordination covers task assignment, information exchange, result integration, conflict resolution, and failure handling. System state includes task progress, shared results, agent availability, resource status, environmental changes, and failure records.Source: §2.2 · p. 7

3 From Model Intelligence to Individual Intelligence

Foundation training establishes model capabilities, prompt and context engineering condition their use, and harnesses plus feedback-driven loops enable sustained individual agency. Single-agent organization still limits parallel scheduling, independent verification, and traceable failure recovery.

3.1 Foundation Models: Establishing Model Intelligence

  • Pre-training establishes general capabilities, while post-training develops instruction following, alignment, reasoning, and agentic behavior through demonstrations, preferences, rewards, and interaction feedback.Pre-training balances model size, data, and compute through data curation, dense or mixture-of-experts architectures, and coordinated training strategies. Post-training combines supervised fine-tuning, preference alignment, and reinforcement learning, increasingly incorporating verifiable rewards, tool use, and multi-turn environmental interaction.Source: §3.1 · p. 8, §3.1.1 · pp. 8–9, §3.1.2 · p. 9

3.2 Prompt and Context Engineering: Eliciting and Conditioning Model Intelligence

  • At inference time, Prompt Engineering specifies tasks, organizes reasoning, and optimizes instructions; Context Engineering acquires, processes, and maintains information without changing model parameters.Prompt optimization can use execution trajectories and reflection; context processing uses ranking, compression, pruning, and restructuring, while management preserves useful working information over time. Lost in the Middle shows that longer context alone does not guarantee effective information use.Source: §3.2 · p. 9, §3.2.1 · pp. 9–10, §3.2.2 · p. 10

3.3 Harness Engineering: Orchestrating Agent Capabilities

  • Harness Engineering supplies persistent tools, interfaces, and execution environments across model calls, with runtime orchestration governing resource organization, verification, maintenance, and improvement.Tool integration expands from function invocation and API access to code, files, shells, browsers, and richer computer interaction. Whereas Context Engineering determines information presented to a model call, the harness maintains resources and executable capabilities across calls.Source: §3.3 · p. 10, §3.3.1 · p. 10, §3.3.4 · p. 11
  • Memory management actively maintains reusable experience across calls, while skill composition turns successful procedures and external knowledge into persistent, evolving capability libraries.Memory mechanisms increasingly control storage, retrieval, updating, consolidation, and deletion, while supporting relational organization and execution-state recovery. Skill engineering extends beyond procedure reuse to construction, composition, feedback-driven optimization, provenance, governance, lifecycle maintenance, and reduced runtime costs.Source: §3.3.2 · pp. 10–11, §3.3.3 · p. 11

3.4 Loop Engineering: Enabling Iterative Agent Execution

  • Loop Engineering uses bounded, stateful feedback control to continue, revise, recover, escalate, or terminate execution according to progress and evidence.Goals, acceptance criteria, unresolved requirements, verification gates, resource limits, and stopping conditions make continuation conditional rather than automatic. ResearchLoop maintains durable contracts and evidence, while Proof-or-Stop [131] requires fresh, mechanically verifiable evidence for lifecycle transitions.Source: §3.4 · p. 12, §3.4.1 · p. 12

3.4.2 Interaction Paradigm

  • Loop interaction preserves continuity among controller, model, harness, and supervisors by exchanging task state, action intents, observations, validation results, and errors.Sovereign Agentic Loops [115] checks structured intents against state and policy before execution. AgentRx [19] localizes failures from trajectories, while Supervising Ralph Wiggum [407] adds metacognitive supervision when repeated refinement stagnates.Source: §3.4.2 · p. 12

3.4.3 Environment Feedback

  • Environment Feedback converts observed state changes into evidence that determines whether the loop accepts progress, retries, replans, restores state, or escalates.Intended effects, actual state transitions, and observed outcomes can differ, so observations must reflect current state and justify subsequent decisions. This concerns how environmental outcomes govern loop continuation, not the execution environment supplied by the harness.Source: §3.4.3 · pp. 12–13, §3.4 · p. 12

3.5 Limitations of Individual Intelligence

  • Individual intelligence remains limited by serialized task scheduling, insufficiently independent specialist roles and verification, and difficulty maintaining persistent state and recovering from failures.Serial execution wastes parallelism and can propagate faulty intermediate results; shared control can produce role confusion and confirmation bias. Context alone is not organized persistent state, making errors difficult to localize and affected steps difficult to repair traceably.Source: §3.5 · pp. 13–14

4 Graph Engineering: From Individual Intelligence to System Intelligence · 4.1 Overview of Graph Engineering

Graph Engineering uses graph structures to externalize relationships among tasks, components, and runtime states, supporting system-level organization, coordination, monitoring, recovery, and optimization.

4.1 Overview of Graph Engineering

  • Individual agents struggle with parallel, interdependent tasks, heterogeneous capabilities, and runtime state, motivating complementary components forming an adaptive whole around a shared goal [154] [159].Source: §4.1 · p. 14
  • System intelligence requires explicitly represented, constrained, and optimized relationships—not merely aggregated agents—to support schedulable tasks, capability-aware coordination, and persistent, recoverable runtime state.These requirements make governing relationships among tasks, components, and runtime states central to system intelligence.Source: §4.1 · p. 14
  • Graphs transform complex objectives into schedulable, executable operations through objective decomposition, dependency modeling, and workflow refinement [91] [158] [280].Source: §4.1 · p. 14
  • Graphs coordinate heterogeneous intelligent components by representing operational topologies and communication patterns that enable collaboration [69] [79] [205] [433].Source: §4.1 · p. 14
  • Graphs record events, dependencies, and state transitions, converting information scattered across contexts and logs into auditable, recoverable system states [35] [41] [467].Source: §4.1 · p. 14
  • Graph Engineering establishes graph structures as the foundation for organizing tasks, coordinating components, and managing runtime states at the system level.Execution feedback and state evidence support iterative structural improvement and the continual evolution of system intelligence.Source: §4.1 · p. 15

4.2 Task Organization: Structuring What to Do

Task Organization externalizes objectives and executable operations as graph structures that can be scheduled, optimized, and revised. Goal decomposition exposes subtask dependencies, while workflow optimization constructs executable operations and adapts them using runtime feedback.

4.2.1 Goal Decomposition

  • Goal decomposition represents objectives as explicit subgoal graphs, with edges encoding precedence, data, or logical dependencies for coordinated scheduling.These graphs define the objective space for workflow construction and execution adaptation. Together, subgoal and workflow graphs replace implicit reasoning and acting with schedulable, optimizable, revisable work structures.Source: §4.2.1 · p. 15, §4.2 · p. 15, §4.2.2 · p. 16
  • Schedulable task graphs enable dependency-constrained parallel execution, while dynamic decomposition refines tasks as intermediate results arrive.LLMCompiler [156] dispatches ready dataflow-DAG nodes in parallel, and Plan-over-Graph [464] generates parallelizable schedules under dependency constraints. TDAG [362] and Flow refine task decomposition during execution; evolving graphs can also drive agent generation, task assignment, and parallel collaboration.Source: §4.2.1 · p. 15

4.2.2 Workflow Optimization

  • Workflow optimization compiles semantic subgoals into executable graphs whose operator nodes and dependency edges support scheduling, coordination, and verification.Operators can include LLM calls, specialized agents, retrieval modules, tools, memory operations, aggregators, and verifiers.Source: §4.2.2 · p. 15, Figure 6 · p. 16
  • Workflow search optimizes operator behavior, graph connections, and executable control flow, reducing reliance on manually designed agent pipelines.GPTSwarm optimizes nodes and edges, while AFlow searches executable workflow code. A2Flow [473] learns operators from demonstrations, MermaidFlow [476] combines a structured intermediate representation with safety-constrained evolution, and VFlow [373] incorporates domain-specific verifiers.Source: §4.2.2 · p. 16
  • Runtime adaptation revises workflow graphs using intermediate feedback, addressing failures that static optimization cannot prevent in open-ended environments.DyFlow [363] generates and adjusts subsequent operator subgraphs, potentially revising remaining subgoals; EvoFlow [450] evolves diverse workflow candidates during inference. QualityFlow [127] selects acceptance, debugging, clarification, rollback, or continuation using quality signals, while FlowSteer [181] modifies workflow structure inside execution.Source: §4.2.2 · p. 16

4.3 Agent Coordination: Structuring Who Works

Agent coordination uses graph structures to model capabilities, organize team responsibilities, and manage runtime communication. Together, these functions determine who performs work, how responsibilities are arranged, and how interactions adapt as execution unfolds.

4.3.1 Agent Capability Modeling

  • Persistent, updateable capability graphs would make agents’ expertise, resources, permissions, and reliability queryable and reusable for task assignment and reconfiguration.Existing approaches often encode capabilities in scores or routing policies rather than reusable relations. SkillGraph [254] and MaAS [451] connect capabilities with graph organization, but their representations remain task- or orchestration-specific.Source: §4.3.1 · pp. 17–18

4.3.2 Agent Team Organization

  • Team organization graphs turn agent capabilities into an executable division of labor by specifying assignments, handoffs, delegation, supervision, verification, and reporting.A single system can combine specialist routing, parallel execution, output aggregation, and reviewer chains. Its graph must represent both stable responsibilities and task-dependent structural changes.Source: §4.3.2 · p. 18
  • Chains, specialist routing, and fan-out/fan-in structures address stage dependencies, heterogeneous expertise, and parallel candidate generation, respectively, with different coordination tradeoffs.Chains clarify execution order but largely fix paths beforehand; centralized routing can burden the coordinating agent. Parallel branching and aggregation increase reasoning diversity while adding communication, computation, and aggregation costs.Source: §4.3.2 · p. 18
  • Team structures can evolve through design-time optimization and runtime reconfiguration rather than remaining fixed as task requirements or agent performance change.Puppeteer [53] dynamically selects and sequences agents; AgentNet adjusts connections and routing without a central controller. SwarmAgentic jointly optimizes agent functions and collaboration patterns, while self-organizing-agent studies [60] suggest specialization and shallow hierarchies can emerge.Source: §4.3.2 · p. 18

4.3.3 Multi-agent Communication

  • Communication optimization selects useful information paths while balancing task performance, communication cost, and robustness, then adapts those paths using runtime feedback.Methods prune redundant connections or low-contribution agents and rebuild sparse edges around information needs. Feedback can incorporate changing capabilities and resources, while execution traces provide reusable, revisable communication-design rules.Source: §4.3.3 · p. 19
  • Communication graphs can include humans as active collaborators who clarify requirements, correct errors, review outputs, approve actions, or assume control during execution.Human participation matters when agent feedback cannot reliably address implicit preferences, specialized expertise, or high-risk actions. Assistance, feedback, approval, and escalation edges integrate humans into the ongoing feedback loop rather than only final evaluation.Source: §4.3.3 · p. 19
  • Communication graphs capture runtime information exchange and feedback, distinct from the relatively stable roles and responsibilities represented by team organization.Activated edges specify who exchanges what information and how it changes subsequent actions. Generation–evaluation–revision loops help detect and correct errors, but additional connections do not necessarily improve collaboration.Source: §4.3.3 · p. 19

4.4 Runtime State Management: Structuring How the System Operates

Runtime State Management makes distributed execution traceable through state recording, evidence-based fault localization, and recovery from validated states. These capabilities support monitoring, diagnosis, recovery, and continual system evolution.

4.4 Runtime State Management: Structuring How the System Operates

  • Runtime State Management complements task organization and agent coordination by maintaining consistent runtime views, identifying execution deviations, and restoring or redirecting execution from validated states.Without reliable execution records, agents may hold inconsistent views, failures become difficult to localize, and valid progress becomes difficult to recover.Source: §4.4 · pp. 19–20, Figure 8 · p. 20

4.4.1 State Recording

  • State Recording preserves transition evidence, provenance, and versions in an explicit, queryable execution record, with structured representation, governed updates, scoped visibility, and consistency management.Proposal-validation-commit boundaries distinguish proposed changes from authoritative state, while concurrent writers require isolation, causal ordering, and conflict resolution. Event-sourced designs preserve append-only committed histories for reconstruction, replay, and branching [252].Source: §4.4.1 · p. 20
  • Existing state-recording studies establish reliability requirements but do not yet provide a unified graph-native implementation.Source: §4.4.1 · p. 20

4.4.2 Fault Localization

  • Fault Localization uses runtime records and external evidence to detect faults, trace their effects, test possible causes, and determine which execution remains valid for recovery.Errors may propagate across tasks and agents before becoming visible; temporal or structural links do not establish causality. Preserving actors, transitions, dependencies, validation evidence, and evidence-backed diagnoses supports attribution and subsequent recovery.Source: §4.4.2 · p. 21

4.4.3 Failure Recovery

  • Failure Recovery selectively repairs execution from explicit recovery boundaries, preserving valid work while accounting for state dependencies and external effects.Recovery can retract invalid states, replay computation, compensate external effects, or branch into alternative execution; reconstructable internal states are distinguished from effects requiring compensation. Recording recovery boundaries, corrective actions, and resulting states closes the recording, diagnosis, and recovery loop.Source: §4.4.3 · p. 21

4.5 System Evolution

System evolution turns runtime experience into improvements across executions by refining task organization, agent coordination, and runtime state management. Validation and rollback close this loop by allowing reliable improvements to persist while failures inform subsequent decisions.

Evolution of Task Organization

  • Task organization evolves decomposition, dependencies, and workflows from execution feedback, but structural revisions require trustworthy signals.TDAG [362] dynamically decomposes tasks and generates specialized agents; DynTaskMAS maintains dependencies; EvoFlow [450] evolves workflow candidates; and QualityFlow [127] uses quality checks to proceed, clarify, or revert. FlowSteer [181] shows that manipulated planning signals can redirect replanning and dependency formation toward undesirable execution paths.Source: §4.5 · p. 22

Evolution of Agent Coordination

  • Agent coordination evolves team structures and communication patterns through collaborative experience as task requirements and component capabilities change.SwarmAgentic jointly optimizes agent functionality and collaboration structures, while AgentNet adjusts connectivity and task routing using local expertise and context. DyTopo [228] reconstructs communication pathways each reasoning round, while QueenBee Planner [335] distills execution traces and evaluations into reusable communication-design rules.Source: §4.5 · p. 22

Evolution of Runtime State Management

  • Runtime state management accumulates reusable experience while validating and revising persistent state to prevent erroneous knowledge from propagating.ReCreate [112] extracts domain patterns from successes and failures, SkillGraph [254] turns failures into reasoning heuristics, and Swarm Skills [470] extracts and refines successful coordination skills. MemTX [195] separates tentative writes from validated commits and repairs retracted beliefs; ActiveGraph [252] preserves event-sourced histories for deterministic replay and forking from prior states.Source: §4.5 · p. 22

System Evolution

  • System evolution connects execution, experience, and system-level updates so successful strategies accumulate and failures inform subsequent decisions.Validation and rollback mechanisms support reliable improvement beyond runtime adaptation.Source: §4.5 · pp. 22–23

5 Open Challenges and Research Opportunities

Graph Engineering must progress from task-specific structures toward reusable infrastructure that consistently interprets, reliably executes, and continually evolves interconnected graphs. Key research directions include graph-native capabilities, controlled structural evolution, shared runtime infrastructure, and privacy and ethical safeguards.

5.1 Graph-Native Capability Substrates

  • Unified capability graphs should connect models, tools, skills, memories, data sources, verifiers, and execution environments with task, agent, and runtime state graphs.Currently implicit capability relationships would become typed edges encoding dependency, compatibility, composition, substitution, authorization, cost, and reliability. Task decomposition would expose capability requirements, agent allocation would consider available capability subgraphs, and execution outcomes would update capability reliability and applicability.Source: §5.1 · p. 23

5.2 Self-Evolving Graph Systems

  • Self-evolving graph systems must transform execution evidence into persistent, reusable structural changes rather than merely adapting individual execution trajectories.This requires observation, structural credit assignment, modification, validation, and commit or rollback, including determining whether changes generalize beyond the current execution. Changes across task, agent, capability, and state graphs must respect shared constraints and remain governable through provenance, versioning, validation, replay, and rollback.Source: §5.2 · pp. 23–24

5.3 Graph-Native Agent Operating Systems

  • A graph-native agent operating system could unify fragmented infrastructure through typed, versioned task, agent, capability, and state graphs with shared runtime services.Ontology Engineering would define types, relations, and constraints, while runtime services would enforce semantics through scheduling, capability discovery, storage, permissions, provenance, structural transactions, checkpointing, replay, rollback, and observability. Linking execution traces to their generating structures would support historical or counterfactual evaluation of candidate changes and commitment of validated improvements as new graph versions.Source: §5.3 · p. 24

5.4 Privacy and Ethics

  • Distributed, persistent agent systems heighten privacy and accountability risks, requiring privacy-preserving state management, scoped permissions, provenance-aware logging, and strong human oversight.Sensitive information can spread across components and persist across tasks, enabling unauthorized access, cross-task leakage, and inference of private attributes from execution traces. Distributed decisions complicate accountability when biased evidence, faulty reasoning, or adversarial inputs are amplified through the system.Source: §5.4 · p. 24

6 Future Direction: Ontology Engineering for Next-Generation System Intelligence

Ontology Engineering provides shared, machine-interpretable semantics that explicit graph structures alone cannot ensure. Future directions include modular ontologies, grounded shared meanings, rigorous system-level evaluation, and governed runtime enforcement, rather than treating ontologies as a complete solution to System Intelligence.

6 Future Direction: Ontology Engineering for Next-Generation System Intelligence

  • Ontology Engineering complements Graph Engineering by defining shared entities, relations, and constraints needed to connect, validate, reuse, and evolve graph structures.Concept meanings can diverge across graph views, components, and execution stages in open, long-running environments. Shared semantics provide a foundation, but do not resolve every challenge of goal formation or system-level evaluation.Source: §6 · pp. 24–25, §6.1 · p. 25

6.1 Limitation of Graph Engineering-based System Intelligence

  • System Intelligence evaluation must distinguish organizational contributions from gains caused by stronger models, longer contexts, additional reasoning samples, or greater computational cost.Assessment should cover goal formation, semantic consistency, parallel efficiency, heterogeneous capability allocation, collective decisions, state consistency, recovery, transfer, and runtime overhead. Incomplete objectives, concurrent workloads, distributed information, failures, and environmental changes should be tested using interventions, structural ablations, and execution-trace analysis.Source: §6.1 · p. 25
  • System ontologies should combine a shared core with specialized modules and domain extensions, rather than require one monolithic model.Modules cover goals and values, agents and capabilities, observations and evidence, actions and states, and evaluation criteria. Beyond annotating graphs, ontologies define entities, relation meanings, required constraints, and derivable conclusions, addressing disagreements about completion, evidence, valid states, and authorized actions.Source: §6.1 · p. 25

6.2 Goal Formation and Value Alignment

  • Goal ontologies make provenance, priority, authorization scope, completion criteria, and constraints explicit, enabling conflict detection, unauthorized-change detection, and verification of completion evidence.They cannot decide which values a system should adopt. Their role is to make goals and normative constraints explicit and verifiable, externalizing semantic constraints for sharing across agents.Source: §6.2 · p. 25

6.3 Shared Semantics and World Grounding

  • Shared semantics require grounding in tool outputs, observations, timestamps, provenance, and validation results because semantic consistency alone does not guarantee factual correctness.Agents can help construct, enrich, align, and update ontologies while preserving structural constraints, provenance, and human oversight. Semantic concepts should also connect to executable capabilities and tool interfaces.Source: §6.3 · p. 26

6.4 Measuring System Intelligence

  • Ontology-based measurement standardizes success, failure, agent contribution, recovery, state consistency, and runtime cost to support comparison of system organization.Shared representations of configurations, events, evidence, interventions, and outcomes make traces more comparable and support structural ablation and causal analysis. Ontologies clarify whether metrics concern foundation models, individual agents, or system organization, but do not replace evaluation methods.Source: §6.4 · p. 26
  • Ontology governance must connect validated semantic changes and lifecycle management to runtime enforcement of permissions, evidence requirements, and valid state transitions.LLM-proposed concepts and relations require provenance checking, consistency validation, and impact analysis; evolving ontologies also need version control, compatibility checking, migration, and rollback. Ontologies define the conceptual model, graphs instantiate task-specific structures, and runtime mechanisms enforce their operational consequences.Source: §6.4 · p. 26

7 Benchmarks, Datasets, and Evaluation

Evaluation is organized around bounded model outputs, sustained individual-agent trajectories, and the organization of multi-component systems. Graph-engineered systems also require structural, operational, and governance checks, while fair attribution and persistent system evolution remain open challenges.

7 Benchmarks, Datasets, and Evaluation

  • Evaluation resources are organized by Model, Individual, and System Intelligence, distinguishing bounded interactions, sustained single-agent trajectories, and coordinated multi-component systems.Benchmarks define tasks, protocols, and scoring; datasets provide reusable instances, annotations, graphs, records, or traces. Environments expose observable, modifiable execution state, and these three resource types can overlap.Source: §7 · p. 26

7.1 Model Intelligence

  • Model Intelligence evaluates bounded model outputs, with persistent interaction and environmental state largely outside its scope.Benchmarks cover knowledge, reasoning, instruction following, code generation, multimodal understanding, and retrieval-augmented reasoning [33] [117] [487]. LiveBench refreshes questions to reduce contamination, while NPPC generates automatically verifiable NP-complete instances with scalable difficulty.Source: §7.1 · pp. 26–27

7.2 Individual Intelligence

  • Individual Intelligence evaluates sustained agent trajectories, including external capabilities, persistent memory, and adaptation, while task organization remains centered on one agent or local runtime.A2E captures standardized execution traces and assesses efficiency, tool use, planning, and error recovery [345]. Evo-Bench evaluates whether models can improve their own agent harnesses [133].Source: §7.2 · p. 28

7.3 System Intelligence

  • System Intelligence evaluates work organization, coordination, and runtime state, but durable, transferable improvements across repeated executions remain underexplored.MASEval treats topology, orchestration, framework, and runtime design as evaluation variables [72]. BenchAgent compares single-agent, fixed multi-agent, and evolving workflows under controlled protocols [88].Source: §7.3 · p. 28

7.4 Evaluation Principles and Open Challenges

  • Evaluation should measure effectiveness, efficiency, and robustness alongside structural fidelity, operational correctness, and evolution and governance.These additional dimensions assess whether structures are valid, graph operations execute correctly, and structural changes remain traceable and controllable.Source: §7.4 · p. 28
  • System-level gains require controls for model strength, context, tools, retries, and compute, while fragmented evaluation, weak structural credit assignment, and dynamic evaluation remain challenges.Future benchmarks should provide matched execution budgets, versioned graph artifacts, complete traces and state snapshots, controlled structural perturbations, and repeated evaluations across tasks and time.Source: §7.4 · p. 28

8 Open-Source Libraries and Engineering Ecosystem

The ecosystem spans model infrastructure, persistent agent runtimes, and multi-component organization, classified by primary engineering target rather than exclusive functionality. Its central gaps are fragmented interfaces and state, limited cross-run structural evolution, and insufficient causal tracing of system improvements.

8 Open-Source Libraries and Engineering Ecosystem

  • Libraries are classified by their primary engineering target—model capabilities, persistent agent runtimes, or multi-component organization—because functionality overlaps across intelligence levels.Coverage requires publicly available source and documentation plus reusable abstractions relevant to intelligent-system construction or execution. Source-available projects are retained with explicit licensing disclosures; generic utilities and domain-specific applications require directly relevant reusable abstractions.Source: §8 · p. 29

8.1 Model Intelligence

  • Model Intelligence libraries engineer model parameters and execution through model interfaces, distributed pretraining, post-training, reinforcement-learning pipelines, and inference infrastructure.Examples include Transformers and Megatron Core [323], LLaMA-Factory, verl, and slime, and vLLM and SGLang [163] [479]. Reinforcement-learning pipelines connect training, rollout generation, rewards, and increasingly agentic environment interaction, while inference engines also provide rollout infrastructure.Source: §8.1 · p. 29

8.2 Individual Intelligence

  • Individual Intelligence frameworks turn model–tool loops into persistent, observable execution using permissions, validation, sessions, state, human control, and explicit workflow transitions.LangChain, OpenAI Agents SDK, Claude Agent SDK, and Pydantic AI expose model–tool loops and runtime controls. LlamaIndex Workflows, Haystack, and Burr emphasize context construction, workflow transitions, event routing, and persistent execution.Source: §8.2 · p. 29
  • Agent runtimes also integrate long-lived memory, temporal knowledge, standardized capability access, and visual workflow composition without necessarily making system organization their primary abstraction.Letta maintains persistent state and memory, while Graphiti models changing contextual knowledge as a temporal graph with provenance [296]. MCP standardizes exposure of tools, resources, and prompts, while Langflow and Dify offer visual workflow composition.Source: §8.2 · p. 29

8.3 System Intelligence

  • System Intelligence frameworks explicitly organize tasks, agents, executors, and shared state through graph execution, message passing, and organizational abstractions.LangGraph, Microsoft Agent Framework, and Google ADK support conditional, concurrent, cyclic, or collaborative execution. CrewAI separates Crews from Flows, CAMEL combines task decomposition with hierarchical worker coordination [182], and Mastra combines agents, workflows, and persistent execution state.Source: §8.3 · pp. 29–30

8.4 Open Challenges in the Engineering Ecosystem

  • Incompatible execution semantics and representations fragment the intelligence stack; MCP improves capability interoperability but does not standardize executable system organization.Multi-agent systems rarely share representations of task dependencies, capabilities, authority, communication, and runtime state.Source: §8.4 · pp. 30–31
  • Current frameworks mostly vary execution within predefined structures; persistent topology evolution across runs remains uncommon, limiting the reuse of execution evidence as structural improvements.GPTSwarm optimizes node prompts and inter-agent edges. Conditional routing, parallel fan-out, worker assignment, and recovery can change execution paths without changing the persistent organization governing future executions.Source: §8.3 · p. 30, §8.4 · p. 31
  • Fragmented state and weak causal tracing motivate typed, versioned structures with validated structural transactions, provenance, replay, rollback, and controlled retention of improvements.State spans checkpoints, memories, workflow snapshots, histories, logs, and temporal knowledge stores. Graph-level tracing and counterfactual comparison should connect observations, decisions, structural mutations, failures, recovery, and later improvements.Source: §8.4 · p. 31

9 Applications of Graph Engineering

Across application domains, Graph Engineering increasingly organizes work, specialist teams, and persistent runtime state, but lasting evolution of these structures remains rare. Most systems still adapt execution within predefined organizations rather than permanently revising them from accumulated evidence.

9.1 Software Engineering and IT Operations

  • Software engineering increasingly manages concurrent branches, dependencies, testing, and merges, while Project ALICE coordinates IT specialists over telemetry and software dependency evidence.MetaGPT and ChatDev introduced staged specialist teams [119], SWE-agent emphasized repository interfaces [418], and OpenHands connected tools and delegation through persistent events. The remaining challenge is a versioned structure connecting planning, code dependencies, ownership, external side effects, testing, and recovery to explain why work organization succeeds.Source: §9.1 · p. 31

9.2 Scientific Discovery and Laboratory Automation

  • Scientific agents increasingly close hypothesis–experiment feedback loops, but reproducible Graph Engineering requires preserving evidence and dependencies while allowing evidence to reshape system organization.Co-Scientist coordinates generation, critique, ranking, and refinement [99], Robin incorporates laboratory results into subsequent hypotheses, and the Virtual Lab connects specialist teams to physical validation. Hypothesis refinement alone is not organizational evolution; stronger systems must preserve negative results, data lineage, experimental interventions, and causal dependencies.Source: §9.2 · p. 33

9.3 Healthcare and Clinical Decision Support

  • Clinical Graph Engineering must coordinate specialists and longitudinal state while preserving provenance, uncertainty, access boundaries, and human authorization; graph organization alone does not establish clinical correctness.DeepRare coordinates phenotype, genotype, retrieval, and analysis agents while accumulating traceable diagnostic evidence. AMIE separates conversational state from longitudinal management reasoning and evolving care plans, illustrating how earlier symptoms, treatments, and responses affect later actions.Source: §9.3 · p. 33

9.4 Enterprise Workflows and Digital Organizations

  • Enterprise Graph Engineering must distinguish planned workflows from committed external state while enforcing permissions, separation of duties, policies, transaction boundaries, and rollback obligations.WorkTeam and SOAN organize workflow-building agents and reusable hierarchical workflows [396]. Gemini Enterprise Agentic RAG checks information sufficiency and continues retrieval when required information remains missing.Source: §9.4 · p. 33

9.5 General-Purpose Digital Agents and Personal Automation

  • Persistent digital agents maintain cross-session state and reusable capabilities, but current cross-run adaptation does not yet constitute general structural self-evolution.OpenClaw separates identities, workspaces, authentication, sessions, and channels, while Hermes Agent combines delegation, scheduling, persistent memory, and reusable skills. The remaining challenge is deciding which experiences should change future work structures, capability assignments, or agent relations, with validation, versioning, and reversal.Source: §9.5 · pp. 33–34

9.6 Social and Economic Simulation

  • Social and economic simulations expose interaction topology and evolving shared state, but simulated emergence cannot establish real-world causality without observational calibration and uncertainty analysis.Agent interactions and local decisions change the environment faced by subsequent agents. Graph-engineered simulators can expose assumptions about models, personas, topology, memory, prompts, and environment rules while supporting topology interventions, replay, and controlled ablations.Source: §9.6 · p. 34

9.7 Cross-Domain Findings

  • Work organization and agent teams are widespread, runtime state management is increasingly explicit, and persistent system evolution remains rare.Applications expose checkpoints, shared task boards, event streams, longitudinal records, and evolving environments. Most adapt execution within predefined organizational structures rather than permanently revising those structures.Source: §9.7 · p. 34
  • System Intelligence requires engineered coordination, not merely additional agents or explicit graph structures.System value depends on work decomposition, responsibility assignment, shared state, and failure diagnosis and recovery. Full Graph Engineering additionally requires structural objectives, graph-level observability, controlled mutation, cross-structure consistency, and evidence that structural improvements persist and transfer.Source: §9.7 · p. 34

10 Conclusion

The survey introduces Graph Engineering as a structure-centered paradigm for System Intelligence, explicitly organizing relations among work, agents, and runtime state rather than simply adding more agents.

10 Conclusion

  • System Intelligence organizes complex objectives, coordinates heterogeneous components, and maintains coherent runtime state across the task lifecycle.Although agents now support sustained interaction, tool use, and iterative execution, a single loop struggles with parallel work, specialized expertise, independent verification, and persistent state.Source: §10 · p. 34
  • Graph Engineering uses graph abstractions to make system relations explicit, operational, and adaptable.The survey’s central lesson is that system-level intelligence depends more on organizing relations among work, actors, and state than on simply adding models or agents.Source: §10 · pp. 34–35
  • Three complementary graph views address work organization, agent coordination, and runtime state management.Beyond representing tasks, agents, and states, graphs support scheduling work, binding capabilities, tracing execution, localizing failures, and enabling controlled evolution.Source: §10 · p. 35
  • Graph Engineering remains an emerging field facing challenges in semantic alignment, graph governance, evaluation, privacy, and safe self-improvement.The survey aims to guide graph-native infrastructure toward more scalable, controllable, and evolvable agent systems.Source: §10 · p. 35

11 Appendix

Graph Engineering treats graphs as explicit structures for organizing tasks, agents, and runtime states, rather than mechanisms for enhancing particular agent capabilities. The appendix distinguishes this system-level scope from capability-focused, runtime-centered, and evolution-centered surveys.

11.1 Comparison with Related Surveys

  • Broad agent surveys and graph–agent surveys [20] [218] characterize capabilities rather than organize the full engineering progression toward system intelligence.Their coverage includes planning, reasoning, memory, tool use, environmental interaction, execution, and multi-agent coordination.Source: §11.1 · p. 62
  • Infrastructure surveys primarily examine agent runtimes,, externalized capabilities and code-centered harnesses [255] [484], or executable workflows [432] [443].They cover execution loops, tool interfaces, context management, persistent state, lifecycle control, evaluation, and workflow optimization, overlapping individual intelligence and parts of task organization.Source: §11.1 · p. 62
  • System-oriented surveys address orchestration, collaboration and autonomous evolution [284], and persistent component improvement [90], approaching Graph Engineering’s organizational concerns.This survey instead organizes Task Organization, Agent Coordination, and Runtime State Management as interconnected structures, complemented by System Evolution and Ontology Engineering.Source: §11.1 · p. 62

11.2 Distinction with Graph-based Approaches in Agents

  • Graph Engineering makes graphs the system’s organizational substrate, jointly representing task dependencies, agent capabilities and responsibilities, and runtime execution state.These structures are coupled: task changes affect capability requirements and allocation, agent changes affect communication and execution assumptions, and runtime evidence can revise both.Source: §11.2 · pp. 62–63
  • System Evolution preserves experience-driven structural improvements, while Ontology Engineering supplies shared machine-interpretable definitions of entities, relations, and constraints.Improvements can be validated, retained, reused, or rolled back across executions. Unlike the evolution-centered dynamic-graph perspective, Graph Engineering treats evolution as one dimension of organizing system intelligence.Source: §11.2 · p. 63
  • Graph Engineering expands the engineering object from model behavior and persistent agent execution to explicitly coordinated system-level relationships.Prompt and Context Engineering govern capability elicitation and available information; Harness Engineering supplies executable capabilities, and Loop Engineering organizes feedback-driven execution. Existing graph–agent approaches instead use graphs to support particular capabilities.Source: §11.2 · pp. 62–63

Related papers

Loading 2608.21156v1…