Breaking the Central Bias: Spatially Partitioned Experts for Coordinate-Based Neuroevolution

Romain Claret; Arthur Gygax; Michael O'Neill; Paul Cotofrei; Michael Palma Mendes; Pascal Felber

Source-linked AI summary

Breaking the Central Bias: Spatially Partitioned Experts for Coordinate-Based Neuroevolution

Romain Claret, Arthur Gygax, Michael O'Neill, Paul Cotofrei, Michael Palma Mendes, Pascal Felber

arXiv:2609.11518v1Sep 10, 2026cs.NEcs.CVcs.LG

TL;DR

ES-HyperNEAT can collapse onto centrally located MNIST pixels, raising whether the problem is evolutionary search or architecture. The paper partitions the image into non-overlapping spatial segments assigned to separately evolved specialists and evaluates their aggregation. With 13 experts, the Independent MoE reaches 43.07% mean accuracy versus 20.95% for the monolithic baseline, while the authors identify MNIST-only evaluation as the main scope limitation.

Problem
ES-HyperNEAT’s coordinate-based encoding concentrates on a small central subset of MNIST pixels, leaving unclear whether the ceiling reflects optimization failure or an architectural limitation.
Method
The paper partitions MNIST inputs into non-overlapping spatial segments and assigns each segment to a separately evolved specialist network.
Results
43.07% mean accuracy is achieved by the 13-expert Independent MoE versus 20.95% for monolithic ES-HyperNEAT, a 106% improvement.
Takeaways & Limitations
Partitioning restores broad receptive-field coverage and supports the conclusion that central bias is an architectural limitation rather than a failure of evolutionary search.
Takeaways & Limitations
The diagnosis rests on MNIST alone, whose centered digits confound coordinate-system bias with information-density bias; off-center datasets are needed for generalization.

Abstract

from arXiv · show

Evolvable-Substrate HyperNEAT (ES-HyperNEAT), a bio-inspired indirect encoding that determines neuron placement and connection weights from spatial coordinates, exhibits a failure mode on MNIST as a diagnostic benchmark. Because input pixels map to a coordinate space centered at the origin, evolved networks converge on a small central cluster of input pixels, a spatial-concentration bias; prior work observed only 21% mean accuracy in this regime. Is this bias an optimization artifact or an architectural ceiling? Inspired by Mixture-of-Experts (MoE) principles, we partition the input into non-overlapping spatial segments, each assigned to a separately evolved specialist network. With 13 such experts, this design reaches 43% mean accuracy, a 106% relative improvement over the baseline. The architectural gain does not depend on data-driven aggregation: equal-weighted averaging, which uses no validation data, already yields a 70% improvement; the gain comes from partitioning, not the weighting. Receptive-field analysis shows the mechanism: partitioning forces evolution to discover features across the entire image, expanding active pixel coverage from 4% to 79%. Absolute accuracy stays below gradient-trained baselines, but the relative gain points to central bias, not the evolutionary search. Two tools are designed to generalize beyond MNIST: a receptive-field diagnostic for silent input-coverage collapse, and a spatial-partitioning remedy that restores coverage.

1 Introduction

The paper examines whether ES-HyperNEAT’s central input concentration on MNIST reflects an optimization failure or an architectural limitation. It tests spatially partitioned specialist networks as a divide-and-conquer intervention.

Motivation: Spatial-concentration bias causes emergent architectures to ignore features outside a small image region, especially in coordinate-based encodings anchored at the substrate center.This motivates testing whether the limitation belongs to the architecture rather than the evolutionary search.
Motivation: ES-HyperNEAT’s baseline concentrates on a small central subset of the 784 pixels, producing 20.95% mean accuracy over 30 MNIST runs.The study treats MNIST as a diagnostic benchmark rather than a competitive one.
Approach: The intervention partitions the image into non-overlapping segments and assigns each segment to a dedicated specialist, using deterministic routing rather than learned MoE gating.The study compares Independent MoE specialists with a Shared MoE that processes segments sequentially through one network.
Results: 43.07% mean accuracy from Independent MoE versus 20.95% for monolithic ES-HyperNEAT indicates a 106% relative increase.The Shared MoE reaches 26.03% despite using the same partitions, isolating independent specialist evolution as the key architectural difference.

2 Background

This section explains how ES-HyperNEAT uses spatial coordinates to evolve connectivity and why central bias can emerge. It contrasts deterministic spatial partitioning with classical gated Mixture-of-Experts.

ES-HyperNEAT: ES-HyperNEAT evolves a CPPN that maps pairs of neuron coordinates to connection weights while quadtree decomposition places nodes where connectivity varies.The substrate begins with a uniform coarse grid, and a variance threshold controls further subdivision.
Central Bias: Central bias combines center-anchored coordinates, symmetric CPPN activations, and the absence of fitness pressure to shift responses off-center.Once central pixels provide a sufficient MNIST solution, evolution can settle without exploring peripheral inputs.
Limitations: MNIST cannot cleanly separate coordinate-system bias from information-density bias because centered digits align discriminative pixels with the substrate origin.Partitioning bypasses this confound by restricting each expert to its assigned input window.
Experimental Context: The study reuses a baseline Bayesian-optimized hyperparameter configuration to isolate architectural changes from new hyperparameter tuning.The referenced optimizer is Tree-structured Parzen Estimation.
Mixture-of-Experts: Classical MoE uses a trainable gate to route inputs among specialized experts, whereas this study fixes expert-to-input assignments spatially.The resulting system is a spatially partitioned ensemble, with prediction aggregation remaining an open design choice.

3 Related Work

The work combines prior ideas on modularity, Mixture-of-Experts, and cooperative coevolution, but applies specialization directly to spatial input regions in ES-HyperNEAT.

Modularity: Spatial partitioning is presented as an input-side structural prior within ES-HyperNEAT, analogous to modular connectivity constraints used elsewhere in the HyperNEAT family.The method combines MoE principles with cooperative coevolutionary decomposition.
Novelty: The paper applies modularity to the input itself, assigning separately evolved experts to spatial regions rather than specializing internal connectivity or behavioral outputs.This distinguishes the approach from earlier modular neuroevolution work focused on reusable modules or multi-modal behavior.
Cooperative Coevolution: The Independent MoE resembles cooperative coevolution because separate expert populations evolve for decomposed subproblems before their outputs are combined.Post-evolution aggregation supplies expert weighting, sidestepping the usual credit-assignment problem described for CCEAs.

4 Experimental Setup

Experiments evaluate Independent and Shared spatially partitioned architectures on MNIST across expert counts, aggregation rules, and controls designed to separate architectural effects from evaluation variance.

Task and Baseline: The task is 10-class MNIST classification against a monolithic model with 20.95% mean accuracy over 30 runs and a maximum accuracy of 29%.Prior baseline networks used only a small central subset of pixels.
Architectures: The Independent MoE substantially outperforms the monolithic baseline, whereas the Shared MoE improves only modestly.The experiment presents peak performance before analyzing aggregation and receptive fields.
Input Partitioning: Figure 1 partitions the flattened 784-pixel image into Ne equal, contiguous, non-overlapping segments assigned to corresponding experts.For Ne = 4, the segments map to four horizontal slices after reshaping the vector to 2D.
Aggregation Strategies: 43.07% mean accuracy and 49% maximum accuracy are achieved with Perf-Weighted aggregation, while naive averaging still exceeds the monolithic baseline.Perf-Weighted uses validation-set per-class F1 scores, whereas simple heuristics treat experts equally.
Experimental Controls: Controls include same-batch evaluation, same-batch-per-expert evaluation, and Shared-Mix input shuffling across seven conditions plus the monolithic baseline.Each condition is run for 30 independent replicates, while evolutionary hyperparameters remain fixed across conditions.
Expert Granularity: Performance peaks at Ne = 13 experts with Perf-Weighted aggregation after sweeping expert counts from 1 to 17.Ne = 1 replicates the monolithic baseline architecture.
Receptive-Field Analysis: Active receptive fields are extracted from champion genomes to compare input coverage quantitatively across generations and spatially across architectures.An active pixel has a non-zero connection to at least one hidden or output neuron.

5 Experimental Results

Spatially partitioned independent experts substantially outperform the monolithic and shared architectures, with performance peaking at 13 experts. The gain reflects broader receptive-field coverage and persists under aggregation methods that use no learned weighting.

5.1 Spatially Partitioned Experts Outperform the Monolithic Baseline: 43.07% mean accuracy at Ne = 13 yields a 106% relative increase over the 20.95% monolithic baseline, while the Independent MoE outperforms the Shared MoE and baseline across conditions.Architecture choice was significant, F(1, 3536) = 11915.88; the best Independent versus Shared comparison was t = 29.19, d = 7.54.
5.1 Spatially Partitioned Experts Outperform the Monolithic Baseline: 26.03% mean accuracy for the best Shared MoE is only a 24% increase over the monolithic model, showing that partitioning alone is insufficient without independent specialists.The Shared MoE processes disjoint segments through one nonspecialized network, creating a representational bottleneck.
5.2 Impact of Expert Granularity: The Independent MoE scales with partitioning whereas the Shared MoE does not, with their architecture-by-granularity interaction highly significant, F(16, 3536) = 114.43.Independent models trend upward and outperform the baseline for Ne > 5; the Shared MoE remains mostly at or below baseline.
5.2 Impact of Expert Granularity: Ne = 13 is the statistically superior granularity for the Default Independent MoE, with 43.07% mean accuracy and a significant expert-count effect, F(16, 493) = 230.78.The Ne = 13 configuration had SD = 2.45% and exceeded all other tested granularities in Tukey HSD comparisons.
5.3 How Aggregation Strategy Determines Success: 43.07% versus 32.93% at Ne = 13 shows that Perf-Weighted aggregation outperforms naive Avg, yet naive averaging still reaches 35.60%, a 70% improvement over baseline.Aggregation strategy significantly affects performance, but the partitioned architecture accounts for much of the gain without data-driven weighting.
5.4 Receptive Field Analysis: 78.6% peak receptive-field coverage at Ne = 11 contrasts with sparse central activation in the monolithic baseline, while the Ne = 13 accuracy peak uses 71.7% coverage.The ensemble activates over ten times as many pixels as the approximately 48-pixel monolithic baseline, indicating a trade-off between broad coverage and refined feature selection.

6 Discussion

The discussion identifies spatial partitioning with independently evolved specialists as the main source of improvement, while receptive-field coverage supports central bias as an architectural limitation. It also states the scope boundaries and proposes tests for separating partitioning, aggregation, and CPPN-related effects.

6 Discussion: 43.07% mean accuracy for the Independent MoE exceeds the 20.95% baseline, while validation-free averaging already clears the baseline and weighted aggregation amplifies the gain.The Independent MoE also outperforms the Shared MoE, indicating that partitioning alone is insufficient without independent specialization.
6 Discussion: Receptive-field evidence shows all baseline activity concentrated in a central band, whereas the 13-expert ensemble achieves near-complete image coverage.Across 30 baseline runs, the union covered only 169 of 784 pixels, or 21.6%.
6 Discussion: Forced spatial diversity, rather than ensemble size alone, drives the improvement because independently evolved experts discover complementary features in assigned partitions.The Shared MoE reaches only about 26% despite processing the same partitions, supporting the role of per-partition specialization.
6 Discussion: The authors limit the diagnosis to coordinate-based indirect encodings and note that MNIST alone cannot separate coordinate-system bias from information-density bias.They identify off-center datasets, plain HyperNEAT comparisons, and mechanism-isolating ablations as future tests.
6 Discussion: The proposed extensions include geometry-preserving partitions, jointly evolved boundaries, per-expert pruning, and specialist-specific hyperparameter optimization.These extensions are presented as ways to test geometry, utility, and scalability beyond the fixed 1D partitioning design.

7 Conclusion

The conclusion reports that spatially partitioned independent specialists substantially improve ES-HyperNEAT on MNIST by restoring receptive-field coverage. It frames receptive-field analysis and spatial decomposition as complementary tools for diagnosing and addressing coordinate bias in high-dimensional inputs.

7 Conclusion: 43.07% mean accuracy is a 106% improvement over the 20.95% monolithic ES-HyperNEAT baseline, with the Independent MoE outperforming the Shared MoE.Equal-weighted averaging alone yields a 70% relative improvement, while data-driven aggregation provides additional amplification.
7 Conclusion: Receptive-field coverage expands from 3.6% to 78.6%, indicating that forced spatial diversity, rather than ensemble size alone, drives the improvement.The conclusion presents receptive-field analysis as a diagnostic for silent input-coverage collapse.
7 Conclusion: The paper proposes spatial decomposition into evolutionarily discovered specialists and receptive-field analysis as tools for high-dimensional bio-inspired pattern-recognition systems.The conclusion emphasizes diagnosis and coverage restoration rather than absolute competitiveness with standard classifiers.