Evaluating the Transfer of Co-Evolved Communication from 2D to 3D Simulation

1 Introduction

• A signal intended to encourage agents to approach each other may instead function as stall assistance rather than as general navigation guidance.Prior work also reports that evolved communication can stabilize and grow more complex without an explicit communication reward.Source: §1 · p. 1
• Simulated controllers can exploit regularities absent from more realistic physical environments, and prior communication mechanisms have not always transferred with the same efficiency.Related studies examined signal communication in foraging and co-evolved communication in a 2D environment [6].Source: §1 · pp. 1–2
• The study tests whether communication evolved in a discrete 2D simulator without real-world physics retains its function in a 3D simulator with friction, collision, and inertia.It examines episode duration, the social signal’s functional role, and asymmetry between the co-evolved agents.Source: §1 · p. 2

2.1 Simulation Engines

• The 2D engine uses a discrete grid without physics, while the PyBullet 3D engine models friction, collision, and inertia and updates decisions every 15 physics steps at 240 Hz.Both simulators contain two agents, shared obstacles, and shared objectives; each robot has eight proximity sensors arranged around its body.Source: §2.1 · p. 2, Figure 1 · p. 3

2.2 Controller Architecture

• Each role uses a co-evolved GRU controller with recurrent memory and a direct residual input-to-output path; its output selects among five discrete actions.The residual path also carries the social input, which is amplified in the official checkpoint; 3D actions map to differential-drive wheel velocities.Source: §2.2 · pp. 2–4

2.3 Social signal and fitness incentive

• The social channel combines relative partner angle with exponential distance decay, while episode fitness combines navigation, stall, social, and terminal-reward components.The social rescue bonus requires approaching a signaling partner at the food and exceeding the agent’s own approach toward food; social bonuses are not conditioned on the receiver being stuck.Source: §2.3 · pp. 4–5

2.4 Stall criterion

• Stall activation uses the inherited 25-step displacement threshold, while evaluation releases an agent when 24 of the last 30 physics steps show movement above threshold.The tolerant release window was calibrated to avoid prolonged stalled states when agents were already moving on most ticks.Source: §2.4 · p. 5

2.5 Basal ganglia as an action selection mechanism

• A fixed, inherited action-selection layer can override the network for recruitment, stall assistance, or training-time anti-fixation behavior.During stalls, the signal/explore action has a base probability of 0.20 that increases after eight consecutive stalled decisions, up to 0.95; the anti-fixation schedule is zero for reported checkpoints.Source: §2.5 · pp. 5–6

2.6 Evolutionary procedure and final checkpoint

• The reported v15_full_final checkpoint continued evolution for 150 generations from a prior run, using a genetic algorithm with elite selection, crossover, and mutation.Training used the strict 30-consecutive-movement release criterion, whereas reported evaluations used the tolerant 24-of-30 criterion; only scratch-started runs count as independent samples in formal tests.Source: §2.6 · p. 6

2.7 Diagnostic and analysis tools

• Diagnostics compare weights and cue-present versus cue-absent decisions, with reversed agent processing order used to test for a physics-engine order effect.Signal-effect analyses measure net approach and stall resolution; the diagnostic battery also measures success rates and stalled time on independent episode samples.Source: §2.7 · p. 7

3.1 Performance baseline and social signal effect

• At 6,000 steps, agent A's success was 49.8% with signal versus 49.0% without (−0.8 percentage points), while B's was 71.2% versus 73.4% (+2.2); neither difference exceeded sampling noise.Agent B outperformed A in both conditions.Source: §3.1 · p. 7

3.2 Effect of the time budget on performance

• Success rose approximately linearly without plateau: Agent A increased from 15.7% to 52.7%, and Agent B from 21.0% to 72.7%, between 2,000 and 6,000 steps.The same checkpoint was evaluated without retraining; the 6,000-step budget was selected for subsequent analyses.Source: §3.2 · pp. 7–8, Figure 3 · p. 8

3.3 Effect of social signaling on stall resolution

• Signal-present decision points had higher stall-resolution rates; differences were significant at 3,000, 4,000, and 6,000 steps, but not at 2,000, where n=36 limited statistical power.The 2,000-step difference was 7.6 percentage points; the significant differences were 8.8, 7.6, and 10.7 percentage points at 3,000, 4,000, and 6,000 steps, respectively.Source: §3.3 · p. 8, Figure 4 · p. 9
• The signal’s association with stall resolution, alongside baseline results, was more consistent with local assistance than navigation toward the partner.Source: §3.1 · p. 7, §3.3 · p. 9

3.4 Weight asymmetry between agents

• Across eight independent runs, the B/A w_res ratio exceeded parity (mean 1.33, p=0.012); w_out showed the same nonsignificant trend (mean 1.46, p=0.098), with run-to-run variability.The runs comprised four from each simulator.Source: §3.4 · p. 9, Figure 5 · p. 10
• Reversing agent processing order changed neither agent’s relative performance in 3D nor in 2D, making processing order, body creation, and initial state unlikely explanations for the asymmetry.Source: §3.4 · p. 9

4.1 Summary of findings

• Success rose approximately linearly with episode budgets, without a plateau in the evaluated range; signaling mainly assisted stall resolution, and the two roles showed consistent weight asymmetry.The asymmetry may contribute to persistent agent performance differences, but its precise origin remains unresolved.Source: §4.1 · p. 10

4.2 Same mechanism, different consequences depending on the body

• A right-turn bias had no measurable consequence at near-ceiling 2D success but was associated with sustained one-way turns, prolonged stalling, and lower 3D success.The same communication mechanism also had a higher stall-resolution rate per signal in 2D than in 3D, consistent with different physical constraints affecting its practical effectiveness.Source: §4.2 · pp. 10–11

4.3 Interpretation of the observed weight bias

• Bias direction recurred in most independent runs, but magnitude varied and tended to increase in lineages with more generations of selection.A from-scratch control showed no pattern at 60 generations and a weak pattern at 150; the eight runs showed a trend toward role differentiation under indirect, weak fitness coupling.Source: §4.3 · p. 11

4.4 Limitations

• Three 2D-versus-3D diagnostics relied on one checkpoint per simulator, making those comparisons qualitative rather than replicated validation.The budget search stopped at 6,000 steps without a plateau, and eight runs limit estimates of bias magnitude and distribution.Source: §4.4 · p. 12

4.5 Future work

• Future work would analyze controller dynamics and revisit the stall-activation criterion, which may contribute to the higher 3D stall frequency.Latent-state and learned-world-model approaches are also proposed for investigating whether asymmetries reflect distinct internal representations or decision regimes.Source: §4.5 · p. 12

• The research received partial support from the Mexican National System of Researchers (SNII).Source: Funding Information · p. 12

• The research was not related to human or animal use.Source: Ethical Approval · p. 13