MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programming

1 Introduction

• Code- or JSON-based workflows entangle planning with implementation, so validity may only be assessed at runtime and robustness, reuse, and search are hindered.The introduction frames brittle workflow logic and coordination breakdowns as a core limitation of existing multi-agent systems.Source: §1 · pp. 1–2
• MermaidFlow represents agentic plans as declarative graphs with explicit nodes and information-flow edges, enabling static checks such as role consistency and type-safe connections.The representation separates symbolic planning from executable code.Source: §1 · p. 2
• Its evolutionary programming framework uses safety-constrained operations to preserve workflow correctness while exploring the structured graph space.Historical workflows accumulate as structured experience for reuse and adaptation across tasks.Source: §1 · p. 2
• The paper reports that MermaidFlow significantly outperforms existing code-based methods on standard agentic reasoning benchmarks in success rates, search efficiency, and interpretability.The introduction describes these as the paper’s empirical contributions.Source: §1 · p. 2

Agentic Workflows with LLMs

• Many agentic systems use imperative code or loosely structured prompts, which the paper contrasts with MermaidFlow’s typed, declarative workflow space.The related-work discussion also identifies fragile workflows, rather than model errors, as a primary source of multi-agent failures [26] [31].Source: §2 · p. 3

Workflow Representation

• Prompting methods are expressive but underspecified, while code-centric approaches offer precision with brittleness and poor editability; graph and template alternatives also have semantic or flexibility limits.The paper positions explicit role semantics and data flow as enabling pre-execution structural checks.Source: §2 · p. 3

Workflow Search and Optimization

• Existing search approaches include tree search, heuristics, learned optimization, and evolutionary methods, but the paper identifies limited validity guarantees as a shared gap.MermaidFlow is presented as addressing this gap with structured graphs and safety-aware mutation operators.Source: §2 · p. 3

3 A Novel Declarative Graph Representation for Agentic Workflows

• MermaidFlow’s Mermaid-based formalism uses a human-readable declarative graph with type-aware syntax and explicit control-flow semantics.The section presents the representation as supporting correctness by construction, symbolic manipulation, and modular workflow composition.Source: §3 · p. 3

3.1 Declarative Workflow Graphs with Mermaid

• Each workflow is a declarative computation graph with typed, annotated nodes and directed, role-labeled edges that encode dependencies and information flow.Nodes carry type-specific attributes, while edges can label relations such as input or problem.Source: §3.1 · p. 4
• Mermaid provides a lightweight syntax for symbolic manipulation and static validation, with node types and attributes guiding translation into Python calls.The formalism makes execution flow subject to type constraints.Source: §3.1 · p. 4

3.2 Agentic Workflow Search Space

• The search space S contains Mermaid-expressible workflows satisfying static constraints such as type compatibility, role-consistent edges, and connectivity.The paper states that these constraints are enforced by the parser and extended structural schema, making workflows valid and executable by construction.Source: §3.2 · p. 5
• Type-consistent local edits preserve global correctness without forward propagation or re-evaluation, while type-compatible subgraphs can be composed without revalidation.The space is intentionally bounded but expressive enough for motifs such as hierarchical refinement and dataflow composition.Source: §3.2 · p. 5

4 Constraint-Aware Evolutionary Workflow Optimization

• Correctness-preserving evolutionary operators explore declarative Mermaid workflows while keeping candidates within a statically verifiable space.Each candidate can be checked by the Mermaid compiler during search.Source: §4 · p. 5, §4.1 · p. 6

4.1 Constraint-Preserving EP Operators for Declarative Workflow Graphs

• Node substitution, addition, edge rewiring, and deletion change agents or paths subject to node-type-compatible input and output constraints.Deletion reconnects the surrounding nodes only when their types are compatible.Source: §4.1 · p. 6
• Subgraph mutation preserves input and output interfaces, while crossover swaps subgraphs rooted at a shared interface node without violating type and interface constraints.Operations are applied at the Mermaid-syntax level for compiler validation.Source: §4.1 · p. 6
• A Mermaid parser/compiler defines a subset of workflows satisfying structural, well-typed I/O, role-validity, and connectivity constraints; valid operators keep evolution within that subset.The section states that applying valid operators repeatedly to an initially valid graph preserves membership in the safe subspace.Source: §4.1 · p. 6

4.2 Evaluation and Selection in Workflow Populations

• The system stores scored workflow experiences in a history buffer, samples two distinct parents using temperature-scaled softmax, and generates candidates with correctness-preserving operators.The sampling parameters control score influence and the exploration–exploitation balance.Source: §4.2 · p. 7
• An LLM judge scores candidates for semantic fit, structure, and task relevance; the highest-scoring candidate is selected and added to the history buffer.Compiler verification makes candidates syntactically valid, type-safe, and structurally executable, supporting population evolution without rollout-based evaluation over the full population.Source: §4.2 · p. 7

5.1 Experiment Setup

• Evaluation covered GSM8K, MATH, HumanEval, and MBPP against 13 baselines, using solve rate for math and pass@1 for code.All methods used gpt-4o-mini; results were averaged over three test-set runs.Source: §5.1 · p. 7, §5.2 · p. 8, Table 1 · p. 8

5.2 Experimental Results

• 80.75% average score across tasks exceeded MaAS's 79.35% average, while MermaidFlow achieved the best performance on all four benchmarks.On MATH, it exceeded the second-best method, AFlow, by 2.61%.Source: Table 1 · p. 8, §5.2 · p. 8

5.3 Ablation Study

• On MATH, MermaidFlow showed more consistent training improvement and better test generalization than AFlow across the solve-rate learning curves.Figure 3 compares the methods on 119 training problems and 486 test problems.Source: §5.3 · p. 8, Figure 3 · p. 8
• >90% of MermaidFlow workflows produced valid Python code, versus AFlow's 50%, supporting more effective workflow-space exploration.MermaidFlow evolves workflows at the graph level, using structured node operations rather than textual edits to Python code.Source: §5.3 · pp. 8–9
• Higher-capacity optimization LLMs consistently improved HumanEval and GSM8K performance while GPT-4o-mini remained the execution LLM.Table 2 compares optimization LLMs on those two benchmarks.Source: §5.3 · p. 9, Table 2 · p. 9

5.4 Case Study: Workflow Evolution with MermaidFlow

• A HumanEval crossover combines Workflow_4's test node with Workflow_5's diverse ensemble to create Workflow_8, whose Mermaid graph is translated into executable Python.The generated Python code resembles Workflow_8, including its ensemble section and test function.Source: §5.4 · p. 9, Figure 4 · p. 9
• The case study illustrates efficient search for high-quality workflow populations and a stable, composable workflow lifecycle.Source: §5.4 · p. 9

• MermaidFlow combines a declarative, compiler-verifiable graph representation with safety-constrained evolutionary programming, but evaluation remains in controlled agentic reasoning settings.Integration with real-world multi-agent systems and user-in-the-loop workflows merits further exploration.Source: §6 · p. 10

A.1 Details of Mermaid

• Custom node types represent specialized reasoning, interfaces, programming, solution ensembling, testing, and code generation, with some types reserved for math or code problems.ScEnsemble nodes require multiple inputs; TestOp and CustomCodeGenerateOp are used only for code problems, while ProgrammerOp is used only for math problems.Source: §A.1 · p. 13
• Connections use arrows to pass labeled information between nodes, such as passing the problem input to a programmer node.Node types have specific connection rules and usage patterns to support correct data flow.Source: §A.1 · p. 13
• Problem-specific prompt templates guide structurally appropriate Mermaid generation, and a Mermaid checker verifies generated code.Math templates emphasize computational clarity and mathematical rigor; code templates emphasize code generation, testing, validation, formatting, and execution requirements.Source: §A.1 · p. 13
• A dedicated prompt.py module stores prompts too verbose for the Mermaid diagram, keeping the workflow clean and readable.Source: §A.1 · p. 13

A.2 Mermaid Checker Implementation

• Soft checks use regex patterns to extract Mermaid components, including node initialization, class definitions and assignments, and connection statements.Source: §A.2 · pp. 13–14
• The checker requires PROBLEM and RETURN interface nodes, valid node types, and paths connecting every node from PROBLEM to RETURN.It also verifies that PROBLEM and RETURN are classified as Interface types and that ScEnsembleOp nodes have at least two incoming connections.Source: §A.2 · p. 14
• Hard checks compile Mermaid code through the Mermaid CLI to verify syntax and catch compilation errors.Source: §A.2 · p. 14

A.3 Algorithms

• The pipeline initializes a workflow, repeatedly samples a parent, optimizes its Mermaid graph, generates Python code, evaluates it, and updates workflow history.Optimization retries graph generation; when a checker is available, it applies hard and soft checks, while the pipeline returns the accumulated history after the configured rounds.Source: §A.3 · pp. 14–15
• The graph-generation routine asks an LLM judge to select a candidate, while optimization accepts the first response if no checker is available.With a checker, optimization retries until both hard and soft checks pass or the configured attempts are exhausted.Source: §A.3 · p. 15

A.3.1 Prompt Templates

• Prompt templates guide workflow updates, Mermaid generation, Python-code generation, and candidate judging against coherence, innovation, complexity balance, prompt quality, and modification rationale.The judge also considers historical graph performance and avoids candidates with structural defects.Source: §A.3.1 · pp. 15–16
• The evolution prompt combines two parent graphs through crossover, mutation, insertion, or deletion, while requiring structurally correct graphs and proper node connections and data flow.Crossover typically separates components at interface or ensemble nodes; the prompt also allows retaining a parent architecture while improving prompts or adding an ensemble node.Source: §A.3.1 · p. 18
• A.3.1’s supplied content is the placeholder {mermiad_custom_prompt}.Source: §A.3.1 · p. 18

MERMAID_GUIDANCE

• Mermaid guidance requires PROBLEM and RETURN interface nodes, predefined operators, coherent data-flow connections, and multiple inputs to ScEnsemble nodes.For math and code workflows, the guide documents distinct operator types and connection patterns.Source: MERMAID_GUIDANCE · pp. 18–20

GENERATE_PYTHON_CODE

• Python generation follows the new graph’s nodes, edges, operators, and prompts; unspecified prompts use empty strings, and returned fields must never be None.The template provides old graph/code/prompt examples alongside the new graph, prompt, and operator descriptions.Source: GENERATE_PYTHON_CODE · pp. 21–22

LLM_AS_JUDGE

• The LLM judge scores candidates on coherence, innovation, complexity balance, prompt quality, and modification rationale, then selects the highest-total-score graph.It also cautions against structural defects such as circular dependencies, missing connections, and incorrect operator connections.Source: LLM_AS_JUDGE · pp. 22–23

A.4 Details of Baseline Methods

• CoT [14] decomposes reasoning into intermediate steps, while ComplexCoT [7] additionally models and controls reasoning complexity.Source: §A.4 · p. 24
• Self-Consistency [22] votes across five sampled reasoning trajectories, while LLM-Debate [6] selects answers by majority vote after up to two rounds among five role-specific agents.Source: §A.4 · p. 24
• DyLAN [18] iteratively shares and refines reasoning; MacNet uses a fully meshed agent graph; GPTSwarm coordinates parallel proposals and shared feedback.Source: §A.4 · p. 24
• MaAS [28], AutoAgents [3], and ADAS [12] use dynamic workflow assignment, autonomous task pipelines, and adaptive debate-and-selection, respectively.AFlow [30] was constrained to gpt-4o-mini with MAX_ITERATION set to 20 for comparison.Source: §A.4 · p. 24

A.5 Details of Tasks and Benchmarks

• Each dataset is partitioned into training and test sets at a 1:4 ratio.Source: §A.5 · p. 24
• The MATH benchmark uses 617 challenging level-5 problems from Combinatorics & Probability, Number Theory, Pre-algebra, and Pre-calculus.Source: §A.5 · p. 24
• Table 3 summarizes statistics for the datasets used in the experiments.Source: Table 3 · p. 24

B Case Study

• Mermaid graphs align prompt semantics and operator roles, support visual inspection and static verification, and enable Python compilation with consistent type and role guarantees.The paper presents these properties as supporting modular, interpretable, and safer agentic workflows.Source: B Case Study · p. 25

B.1 GSM8K

• For GSM8K, a Custom solution and five Programmer responses cover stepwise solving, edge cases, code execution, validation, and alternatives before ensemble selection and final refinement.The workflow returns the refined output along with usage cost.Source: §B.1 · pp. 25–27

B.2 MATH

• For MATH, five Custom operators generate varied solution forms and a Programmer solves step by step; an ensemble selects a response that a Custom operator refines.The variants include an alternative approach, a detailed outline, and a comprehensive solution.Source: §B.2 · pp. 28–30

B.3 HumanEval

• For HumanEval, multiple code-generation candidates are ensembled and tested; a failed test triggers an improved candidate that is tested before the workflow returns a solution.The workflow passes on the combined solution or uses the improved solution when it passes.Source: §B.3 · pp. 31–32
• For HumanEval, prompts request an optimized function based on prior attempts and outcomes, then select among generated solutions for likely correctness and edge-case robustness.The prompts require the function name to match the specified entry point and call for necessary imports and clear code.Source: §B.3 · p. 33

B.4 MBPP

• For MBPP, the workflow generates four candidate solutions, ensembles them, tests the result, and returns the solution with total cost.The candidates comprise two differently instructed solutions, an optimized solution, and a fix attempt.Source: §B.4 · pp. 34–35