PaperlayerView exact revision on arXiv

Source-linked AI summary

DEI: Diversity in Evolutionary Inference for Quality-Diversity Search

John Donaghy, Shikhar Rastogi

arXiv:2605.27130v1cs.LGcs.AI

TL;DR

Distributed LLM-based QD search lacks evidence separating model diversity from mere parallel computation. DEI assigns heterogeneous LLMs as mutation operators with asynchronous champion exchange, and finds stronger merged archives and generality than homogeneous or solo baselines at equal call budgets.

  • Problem

    Existing parallel LLM search replicates one model’s inductive biases, leaving potential behavioral-space gaps and limited evidence on heterogeneous model diversity.

  • Method

    DEI distributes heterogeneous LLM mutation operators across nodes and uses asynchronous champion sharing to create cross-model adversarial pressure in QD search.

  • Results

    +124% QD-Score and +28% coverage versus a solo node, while diverse ensembles outperform homogeneous ensembles across QD-Score, coverage, and held-out generality at equal budgets.

  • Takeaways & Limitations

    Model diversity, not merely parallelism, is supported as a key design principle for distributed LLM-based QD search.

  • Takeaways & Limitations

    Generality beyond domains with structured behavioral-characteristic spaces or inexpensive fitness evaluation remains unestablished.

Abstract

from arXiv · show

We present DEI: Diversity in Evolutionary Inference, a distributed Quality-Diversity (QD) search framework that assigns heterogeneous large language models (LLMs) as mutation operators across peer nodes communicating with non-blocking collective operations. Unlike homogeneous parallel search, which replicates a single model's inductive biases across all workers, DEI treats each LLM's distinct creative prior as a complementary source of behavioral novelty. Extending the Digital Red Queen framework with DEI, nodes share local optimal solutions at the end of each round to seed the next round's population. This creates cross-model adversarial pressure that drives robustness beyond intra-model self-play. Evaluated on the Core War domain, a competitive programming benchmark in which Redcode warrior programs battle inside a simulated machine, a four-node heterogeneous ensemble (GPT-5.4-mini, Claude Sonnet 4.6, GPT-5.2, and Claude Haiku 4.5) achieves 124 percent higher merged-archive QD-Score (45.90 vs. 20.46) and 28 percent higher coverage (80.6 percent vs. 63.0 percent of cells) than a single-node baseline at equal total LLM-call budget. The heterogeneous ensemble also outperforms an equally-budgeted homogeneous ensemble on QD-Score, coverage, and held-out solution generality across all four model families. These results provide the first empirical evidence that model diversity, not merely parallelism, is the key driver of gain in distributed LLM-based QD search.

1. Introduction

DEI reframes distributed LLM evolutionary search as parallel cognition: heterogeneous models contribute distinct inductive biases, while asynchronous sharing blends their archive niches without synchronization barriers.

  • Motivation: Evolutionary algorithms iteratively mutate candidate solutions and retain high-performing variants in a population.Mutation operators perturb existing solutions before fitness evaluation and population inclusion.
  • Motivation: Distinct LLM inductive biases can populate different regions of a shared Quality-Diversity archive, covering behavioral space beyond one model’s reach.The biases arise from differences in training data, architecture, and alignment procedures.
  • Motivation: Existing parallel LLM search mainly replicates one model’s generative prior, while Digital Red Queen adds adversarial pressure within a single-node setting.Under homogeneous parallelism, diversity comes primarily from stochastic sampling rather than fundamentally different generative priors.
  • DEI framework: Asynchronous champion sharing seeds peers with elites from niches they would rarely generate internally, enabling archives to blend across nodes in later rounds.The design addresses heterogeneous model latencies by allowing nodes to advance independently without synchronization barriers.
  • DEI framework: DEI assigns heterogeneous LLMs as mutation operators on distinct nodes and integrates fully asynchronous champion sharing into distributed QD search.The framework explicitly exploits heterogeneity rather than treating parallelism as additional computation alone.
  • Evaluation claim: Holding total compute fixed isolates diversity from extra computation, with heterogeneous nodes improving generality, archive coverage, and QD-Score over homogeneous nodes.The comparison is designed to show that the gain cannot be attributed to additional computation.

2. Background

The paper builds on Digital Red Queen and MAP-Elites to search Core War’s rich behavioral space, evaluating warriors with adversarial fitness and held-out generality.

  • Core War: Core War places Redcode warriors in a circular MARS memory, where programs battle by overwriting instruction pointers or halting processes.The domain supports strategies including imps, dwarfs, scanners, and fortress warriors.
  • Evaluation: Warrior fitness measures survival across battle timesteps against an opponent set, while held-out generality counts wins or ties against human-authored warriors.The generality definition uses a fixed held-out cohort and treats a win or tie as success.
  • Quality-Diversity search: MAP-Elites maintains one highest-fitness elite per discretized behavioral cell, seeking diverse high-performing solutions rather than a single optimum.Each iteration selects, mutates, evaluates, and archives a solution when it fills an empty cell or improves its occupant.
  • Quality-Diversity search: The two-dimensional behavioral space uses Time-Space Product and Memory Coverage to represent persistence–footprint trade-offs and address-space exploration.Time-Space Product multiplies code length by average lifespan, while Memory Coverage measures the fraction of core locations touched.
  • Digital Red Queen: Digital Red Queen replaces hand-crafted genetic operators inside MAP-Elites with LLM-generated or LLM-mutated Redcode warrior programs.Candidates are evaluated against a round-specific opponent pool containing seeds, retained champions, and peer-collected warriors.
  • Distributed execution: Fully asynchronous all-gather lets nodes publish champions whenever ready and peers consume them at their own pace despite differing model latencies.This avoids throttling the ensemble to its slowest participant while propagating champions across nodes.

3. Method

DEI combines local DRQ optimization with asynchronous champion sharing across heterogeneous nodes, while evaluation compares ensembles under equal total LLM-call budgets.

  • System architecture: Each node runs an asynchronous messaging layer alongside a local DRQ optimizer that uses its assigned LLM for warrior generation and mutation.
  • Per-node DRQ loop: With probability 0.1, the LLM generates a warrior from scratch; otherwise, it mutates an archive warrior sampled uniformly.
  • Per-node DRQ loop: The round opponent pool combines initial seeds, retained champions, and champions collected from peer nodes.
  • Champion exchange: At each round’s end, the champion is selected, broadcast, and drained from peer buffers for local use.Received champions enter the opponent pool and seed previously empty archive cells.
  • Experimental design: The experiment compares diverse, homogeneous, and solo conditions while keeping total LLM-call budgets fixed across conditions.The diverse ensemble uses GPT-5.4-mini, Claude Sonnet 4.6, GPT-5.2, and Claude Haiku 4.5; distributed coverage uses the merged archive.
  • Evaluation metrics: Evaluation measures held-out champion generality, archive coverage, QD-Score, and niche novelty.QD-Score sums fitness across occupied cells and reflects both archive breadth and per-cell quality.

4. Results

Diverse collaboration improves individual-node generality and niche novelty, and its merged archive surpasses solo performance on coverage and QD-Score under matched compute.

  • Individual node: Diverse collaboration achieves higher generality than homogeneous collaboration across all four model families under equal total LLM-call budgets.
  • Individual node: Niche novelty improves when moving from homogeneous to diverse ensembles, indicating broader collaborative archive contributions.
  • Merged archive: The merged diverse archive uses 4 nodes × 62 iterations = 248 LLM calls per round, matching the solo budget of 250 calls.
  • Held-out generality: Figure 2 compares held-out champion generality over rounds for Diverse Ensemble, Homogeneous Ensemble, and Solo across four model-specific panels.Diverse Ensemble has the highest generality for all four models.
  • Merged archive: Diverse Ensemble achieves +124% higher QD-Score and +28% more coverage than a solo node in the final-round merged archive.It reaches 80% coverage, while the homogeneous ensemble does not outperform solo on coverage.

5. Discussion

DEI’s central finding is that diverse ensembles outperform homogeneous ensembles and solo search across generality and merged-archive metrics at matched compute. The asynchronous design also accommodates heterogeneous node speeds, while the empirical scope remains limited to Core War.

  • Findings: Diverse ensembles outperform homogeneous ensembles and solo search across individual-node generality and merged-archive Coverage/QD-Score at equal compute.Four collaborative nodes running 62 iterations each contribute the same total LLM calls as one solo node running 250 iterations per round.
  • Systems implications: Asynchronous communication lets a slow node improve coverage without penalizing faster nodes.The architecture avoids synchronization barriers between heterogeneous inference systems.
  • Scope: The findings are empirically limited to Core War, so generality to less structured behavioral spaces or more expensive fitness evaluations remains unestablished.The authors call for further testing in other domains.
  • Future work: Future directions include adaptive topology, heterogeneous behavioral-characteristic axes, and multi-agent tasks beyond Core War.These proposals extend the framework’s treatment of complementary archives and domains.

6. Related Work

DEI builds on LLM-based evolutionary operators, Quality-Diversity archives, distributed island models, adversarial co-evolution, and diversity-aware learning. Its distinctive contribution is model-family diversity in distributed evolutionary search rather than parallelism or within-family scaling alone.

  • LLMs as evolutionary operators: LLM-based evolutionary methods use language models as context-aware mutation or optimization operators for generating program variants.The related work includes Evolution through Large Models, EvoPrompting, OPRO, and FunSearch.
  • LLMs as evolutionary operators: DEI differs from prior ensemble work by diversifying across model families rather than only model size within one family.The paper identifies this as the first study of heterogeneous model ensembles in distributed evolutionary search.
  • Quality-Diversity algorithms: Quality-Diversity methods maintain archives of high-performing solutions spanning discretized behavioral-characteristic spaces.MAP-Elites is presented as the canonical archive-plus-behavioral-characteristic framework.
  • Distributed and island-model evolutionary algorithms: DEI’s asynchronous gossip substrate extends the island-model blueprint with publish-subscribe migration, heterogeneous compute budgets, and LLM identity as the primary diversity source.Earlier Core War island-model work lacked LLM operators and Quality-Diversity objectives.
  • Adversarial co-evolution and Red Queen dynamics: Competitive co-evolution provides the broader Red Queen context for adversarial evolutionary search and escaping local optima.The cited lineage includes adversarial test cases and computational evolutionary arms races.
  • Code generation with LLMs: The system relies on diverse LLMs generating syntactically valid Redcode warriors, repurposing code-generation capabilities as Quality-Diversity operators.The related work connects this use to competition-level program synthesis.
  • Diversity-aware reinforcement learning and multi-agent reasoning: Diversity-aware reinforcement learning and multi-agent reasoning motivate explicit generative diversity as a route to gains in quality, novelty, exploration, and task performance.The cited examples include semantic-diversity objectives and multi-agent debate.

7. Conclusion

DEI uses heterogeneous LLM inductive biases and asynchronous communication to create cross-model adversarial pressure and niche novelty. At equal total call budget, its merged archive achieves substantially higher QD-Score and coverage than a solo node.

  • Contribution: DEI exploits distinct inductive biases of heterogeneous LLMs as the primary source of behavioral diversity in distributed Quality-Diversity search.The framework connects these models through an asynchronous distributed communication layer.
  • Contribution: Asynchronous communication enables cross-model adversarial pressure and niche novelty without synchronization barriers.The design supports collaboration among nodes with different inference speeds.
  • Empirical conclusion: +124% higher QD-Score and +28% more coverage are achieved by the merged diverse archive than by a solo node at equal total LLM-call budget.The conclusion frames model diversity, rather than parallelism alone, as a first-class design principle.

Broader Impact

The asynchronous gossip architecture enables participation from researchers who can run local open-weight models alongside cloud-hosted counterparts. The paper connects this broader access to possible acceleration of scientific discovery through LLM-driven program synthesis.

  • Access and participation: Researchers without dedicated GPU clusters can contribute to collaborative search by running local open-weight models alongside cloud-hosted counterparts.This broader participation is enabled by the asynchronous, gossip-based architecture.
  • Potential applications: The paper suggests that democratized participation in LLM-driven program synthesis could accelerate scientific discovery in combinatorial optimization and automated theorem proving.These domains are presented as examples of potential broader impact.

A. MARS Configuration Details

The experiments hold the MARS simulation configuration constant and use two prompt templates for generating and mutating Redcode warriors. Most calls mutate existing warriors, while a smaller fraction generates new warriors from scratch.

  • MARS configuration: All simulations use a fixed MARS configuration across experimental conditions.The configuration includes an 8,000-instruction core, 80,000 maximum cycles per battle, 20 rounds per pair, random placement with minimum separation, and unlimited processes.
  • Prompt templates: Two prompt templates govern LLM calls.The templates support both new-warrior generation and mutation of existing warriors.
  • Prompt templates: 10% of calls generate novel Redcode warriors from scratch using the rules and current archive state.
  • Prompt templates: 90% of calls mutate an existing warrior using its fitness score and behavioral-characteristic coordinates as context.
  • Prompt templates: The full prompt text is included in the code release and the DRQ repository.

C.1. Yggdrasil Overlay

DEI uses Yggdrasil-based networking through AXL, which exposes a local application interface and avoids direct remote sockets. GossipSub provides asynchronous peer communication for ensemble experiments.

  • Yggdrasil Overlay: Yggdrasil assigns each node a stable IPv6 address from its public key and supports NAT traversal without manual port forwarding.
  • AXL interface: AXL bridges the DRQ application to Yggdrasil through a local HTTP API, so DRQ never opens remote sockets directly.AXL is an open-source peer-to-peer network for decentralized agentic and AI/ML applications.
  • AXL interface: AXL consists of an HTTP API, inbound message multiplexer, userspace gVisor TCP/IP stack, and Yggdrasil core.The Yggdrasil core manages the node keypair, derives a 200::/7 IPv6 address, and peers over TLS/TCP.
  • AXL interface: DEI uses POST /send, GET /recv, and GET /topology to exchange payloads and obtain peer identity and topology information.The receive endpoint returns queued payloads or 204 when the queue is empty; topology reports the node address, public key, and peers.
  • Asynchronous communication: GossipSub runs over AXL and Yggdrasil, using a three-peer mesh with lazy message announcements and heartbeat-based mesh repair.The three-peer mesh provides O(log N) propagation diameter for N nodes, while IHAVE and IWANT messages manage message availability.
  • Disclosure: An LLM-based coding and writing agent assisted with prose, code debugging, and plotting-code preparation under the authors’ direction.The disclosure states that scientific claims, experimental choices, and final wording were reviewed by the authors.
Loading 2605.27130v1…