Source-linked AI summary
Where Does Randomness Matter in Neural Cellular Automata?
Fei Zuo, Jiaqi Shi, Yujing Liu
TL;DR
The paper asks whether stochastic updates help an NCA learn and whether they must remain during execution, and how training-state exposure shapes later behavior. Controlled Growing NCA comparisons and a linear-lattice analysis separate these questions. Results distinguish optimizer-dependent learning success, deterministic execution, and task-specific retention and repair.
Problem
NCA update randomness is often used during both training and rollout, leaving its role in learning a usable rule entangled with whether randomness is needed at execution.
Source: §1 · p. 1Method
The study crosses training and evaluation update modes in controlled Growing NCA experiments, compares training-state recipes, and derives an exact second-moment criterion for a scalar linear lattice.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
Update schedule, training-state exposure, and task evaluation are separate design choices: in this cohort, deterministic execution works for passing asynchronous models, while training states distinguish retention from repair.
Source: §1 · p. 2, §6 · p. 6, §8 · p. 8The evidence is limited to two synthetic targets, one Growing NCA family, five seeds per target and recipe, one primary learning rate, and a finite rollout horizon.
Source: §6 · p. 7
Abstract
from arXiv · showhide
Stochastic cell updates are often used throughout the life of a neural cellular automaton (NCA), from backpropagation through time to final rollout. This leaves two questions entangled: does update randomness help learn a useful rule, and must that randomness remain at execution? We separate training and evaluation update modes in controlled Growing NCA experiments, then vary the states shown during training. Under the standard constant-rate persist recipe, asynchronous training passes the short-horizon quality test in 10/10 runs, compared with 3/10 synchronous runs. All ten asynchronous models also retain the target for 4,096 steps under deterministic evaluation. For a scalar translation-invariant lattice, we derive an exact mean-square criterion: random masking can damp mean modes, but it also injects variance, and a mean-only test misclassifies four non-marginal settings. Finally, among 30 models that all pass the same reconstruction test, eight of ten grow-trained models become off-target at 4,096 steps, while all persist and regenerate models retain the target; damage recovery separates persist from regenerate. The results distinguish optimization reliability, execution mode, and task-specific behavior instead of treating them as one stability property.
1 INTRODUCTION
The introduction separates whether stochastic updates help learn a usable NCA rule from whether they must remain during execution, and asks how training-state exposure shapes later behavior. It previews controlled update-mode results, an exact linear-system criterion, and recipe-dependent retention and repair.
1 INTRODUCTION
- The motivating gap is that prior findings on update schedules and training states do not separate the schedule used to learn a rule from the schedule used to execute it.The introduction frames the second question as which behaviors follow from training states once a rule can reconstruct its target.Source: §1 · p. 1
- 10/10 asynchronous models pass the T = 96 quality test, versus 3/10 synchronous models, while all ten asynchronous models retain the target through 4,096 deterministic steps.This controlled comparison uses one constant learning rate and persist recipe; four exploratory lower-rate synchronous retrainings also pass, limiting the training advantage to the tested optimizer setting.Source: §1 · pp. 1–2, Figure 1 · p. 2
- An exact second-moment recursion for a scalar translation-invariant lattice shows that random masks change mean dynamics and inject variance, so a mean-only criterion can mislead.The linear theorem bounds inference for a fixed linear rule; it does not explain nonlinear training failures.Source: §1 · p. 2
- The study compares retention and damage recovery across grow, persist, and regenerate models after they all pass the same short-horizon reconstruction test.The recipes expose the rule to different training states, and the outcomes are treated as distinct tasks rather than one stability score.Source: §1 · p. 2
2 MODEL AND EXPERIMENTAL DESIGN
The experiments use a shared local Growing NCA rule and compare update schedules and training-state recipes under controlled model, optimizer, and target settings. They assess short-horizon reconstruction, long rollouts, perturbation growth, and damage recovery as distinct behaviors.
2 MODEL AND EXPERIMENTAL DESIGN
- The canonical Growing NCA applies a shared two-layer local network to fixed identity and Sobel-filter perceptions, predicting an additive state change for each cell.Each cell stores four RGBA and twelve hidden channels; the network has 128 hidden units, ReLU activation, and a zero-initialized output layer.Source: §2 · p. 3
- Twenty persist models compare synchronous and asynchronous updates, while thirty asynchronous models compare grow, persist, and regenerate recipes; the two groups share ten models.Base runs use five seeds per configuration, two targets on 72 × 72 grids, Adam, a constant 2 · 10^-3 learning rate, and 8,000 optimization steps.Source: §2 · p. 3
- Grow starts from the seed, persist samples from a pool of generated states, and regenerate adds circular damage to half the sampled batch.The recipe comparison controls architecture, objective, optimizer, budget, targets, and update probability while changing the training-state distribution.Source: §2 · p. 3
- Quality is RGBA MSE below 0.02 at T = 96; long-rollout outcomes distinguish dead, divergent, off-target, and retained states using separate thresholds.A finite-time perturbation test evolves perturbed and clean states for 64 steps, while damage recovery is measured over a separate fixed 80-step window.Source: §2 · p. 3
3 RANDOM UPDATES CHANGE LEARNING SUCCESS MORE THAN EXECUTION
The controlled comparison finds that asynchronous training more reliably passes the short-horizon quality test under the tested fixed-rate persist setup, but successful asynchronous models do not require stochastic evaluation. Lower-rate probes show the training advantage is optimizer-dependent.
3 RANDOM UPDATES CHANGE LEARNING SUCCESS MORE THAN EXECUTION
- 10/10 asynchronous models pass the T = 96 quality test, compared with 3/10 synchronous models (two-sided Fisher exact p = 0.003).This is a comparison under one learning rate and one training recipe, not evidence that synchronous training cannot succeed.Source: §3 · p. 3
- All nine recorded synchronous training traces cross a minibatch loss of 10^-3, although seven of ten final synchronous checkpoints fail the quality test.The incomplete training-minibatch traces show that low observed loss can occur during optimization, not that every failed run previously held a robust rule.Source: §3 · pp. 3–4
- All ten asynchronous models retain the target for 4,096 deterministic steps from the seed, showing stochastic evaluation is not required for these successful models.The preregistered asymmetry comparison did not establish a directional swap effect.Source: §3 · p. 4
- Four exploratory lower-rate synchronous retrainings all pass, so asynchronous updating improves the pass rate at the tested constant learning rate but is not necessary for synchronous success.The retrainings used learning rates 10^-3 and 5 · 10^-4 on two failed flower configurations.Source: §3 · p. 4
4 A RANDOM MASK DAMPS SOME MEAN MODES AND INJECTS VARIANCE
For a fixed scalar linear lattice rule, random cell updates affect both mean dynamics and modal power. The exact mean-square test captures variance injection that a mean-only test misses.
4 A RANDOM MASK DAMPS SOME MEAN MODES AND INJECTS VARIANCE
- For 0 < α ≤ 1, the expected state follows E[x_t+1] = (I + αA)E[x_t], while the exact second-moment dynamics track expected modal power.The analysis assumes a translation-invariant periodic scalar lattice and independent Bernoulli cell masks.Source: §4 · p. 4
- Neutral modes are excluded from the mean-square decay criterion.Source: §4 · p. 4
- Random masking adds a rank-one power-injection term that couples modal powers, so replacing the mask by its mean can overstate stability.The recursion also propagates each mode under the averaged rule.Source: §4 · p. 4
- The second-moment test matches all 25 non-marginal diffusion-lattice labels and resolves 4 settings that the mean-only test calls stable.The grid contains 27 configurations; the two exactly marginal cases are reported separately.Source: §4 · p. 5
5 TRAINING-STATE EXPOSURE DETERMINES RETENTION AND REPAIR
Models trained on different state distributions can pass the same short-horizon reconstruction test yet differ in long-horizon retention and damage recovery. Perturbation growth and frozen-state spectra provide complementary measurements, not substitutes for direct task outcomes.
5 TRAINING-STATE EXPOSURE DETERMINES RETENTION AND REPAIR
- 8/10 grow models were off-target at T = 4096, while all 10 persist and all 10 regenerate models remained in the retained class.All 30 models passed the same T = 96 quality threshold; Table 2 uses a separate long-horizon retention threshold.Source: §5 · p. 5, Table 2 · p. 6
- Persist models had negative mean λpert on both targets, grow models had positive means, and regenerate intervals included zero on both targets.Five models with positive λpert still retained the pattern, so perturbation growth is not itself a failure label.Source: §5 · p. 5
- Regenerate recovery averaged +85% on flower and +99% on heart; persist averaged −7% and +4%, while grow averaged −46% on each target.The grow-minus-regenerate interval on heart includes zero, so the target-specific contrast is not equally strong everywhere.Source: §5 · p. 5
- All 30 models had a largest returned frozen-state Jacobian eigenvalue above one, yet the measure did not separate their observed target-retention outcomes.Removing living masks left rollouts numerically bounded, while two regenerate models changed target class.Source: §5 · p. 5
6 DISCUSSION
The discussion recommends treating optimization reliability, execution schedule, and training-state exposure as separate design choices. Its evidence is bounded to a narrow Growing NCA study, and its measurements address distinct properties.
6 DISCUSSION
- In the tested fixed-rate persist setup, asynchronous updates improved final pass rate, but lower-rate synchronous probes make that advantage optimizer-dependent.Every passing asynchronous model remained on-target under deterministic evaluation; the design sequence also calls for training on states the deployed rule must handle.Source: §6 · p. 6
- The exact random-mask criterion applies only to a fixed scalar translation-invariant linear operator and does not predict nonlinear NCA absorbing-state failures.Independent masks change expected linear dynamics and add a variance term; direct evaluation of the trained rule remains necessary.Source: §6 · p. 6
- Figure 3 tracks individual finite-time perturbation growth and group intervals, alongside flower median MSE beyond the maximum training horizon.Heart outcomes are summarized in Table 2.Source: Figure 3 · p. 7
- The study covers two synthetic targets, one Growing NCA family, five seeds per target and recipe, one primary learning rate, and a finite rollout horizon.Broader claims require other targets, architectures, and optimizer schedules.Source: §6 · p. 7
7 RELATED WORK
Related work spans NCA training recipes and applications, dynamical analysis, and stochastic stability. The paper situates its controlled comparisons within these lines of research.
7 RELATED WORK
- Growing NCA established seed growth, pool-based persistence, and damage-based regeneration, while later work studied asynchronous timing and diverse applications.Synchronous training brittleness was also reported in multi-target self-replication (Sinapayen, 2023).Source: §7 · p. 7
- Prior dynamical work examines deterministic NCA attractors, perturbation responses, and transient dynamics (Kvalsund & Stovold, 2026; Masumori et al., 2026; Sato et al., 2026).This paper complements that perspective with controlled train/evaluation crossovers and common-quality recipe comparisons under the canonical Growing NCA setting.Source: §7 · p. 7
- Work on multiplicative noise, dropout, and stochastic depth informs the stochastic-update context, while classical stochastic stability tools support exact second-moment lattice recursions.Related foundations also include residual dynamics, signal propagation, transient growth, and optimization in recurrent and continuous models.Source: §7 · p. 7
8 CONCLUSION
The conclusion separates update schedule, state exposure, and task evaluation as design choices for Growing NCA. The paper also distinguishes controlled comparisons from exploratory controls and states the assumptions of its linear analysis.
8 CONCLUSION
- Stochastic updates play different roles in learning and execution; state exposure distinguishes later retention from repair after reconstruction.The conclusion frames update schedule, state exposure, and task evaluation as separate design choices for Growing NCA.Source: §8 · p. 8
A STUDY DESIGN AND INTERPRETATION
- The study fixes the persist recipe and optimizer for update-mode comparisons, and fixes network and base optimization settings while changing training states for recipe comparisons.All thirty recipe models pass a common short-horizon quality threshold; the recorded protocols are distinguished from exploratory controls.Source: A STUDY DESIGN AND INTERPRETATION · p. 11
B PROOFS
- The linear model uses a fixed A and independent Bernoulli cell masks, with mask draws independent across cells, time, and the current state.This model does not include the living masks of Eq. (1).Source: B PROOFS · p. 11
B.1 PROOF OF PROPOSITION 1
The appendix develops stability criteria for partially updated linear systems, establishes locality under spatial and temporal extension, and defines the study’s measurements and comparison protocol. Lattice simulations mostly agree with the exact mean-square criterion, while boundary cases explain the remaining discrepancy.
B.1 PROOF OF PROPOSITION 1
- The multiplier condition defines a disk centered at 1 − 1/α with radius 1/α; its rightmost point +1 is marginal, not strictly stable.As α decreases, the disks approach the half-plane Re µ < 1.Source: §B.1 · p. 11
B.2 PROOF OF THEOREM 1
- The exact second-moment condition gives mean-square asymptotic decay when S < 1 and rules it out when S ≥ 1, provided dα(ω) < 1 on non-neutral modes.Neutral modes can remain bounded without tending to zero, so they are excluded from the decay claim.Source: §B.2 · p. 12
B.3 LOCALITY UNDER SPATIAL AND TEMPORAL EXTENSION
- After T steps, a local rule of dependence radius r depends only on initial states and update masks within a radius-rT space-time dependency cone.For Eq. (1), the complete update has radius at most two because the post-update living mask reads neighboring candidate states; at T = 96 and L = 288, the boundary-fraction bound is 1.00.Source: §B.3 · p. 12
C MEASUREMENT DEFINITIONS
- The measurement definitions distinguish retained rollouts from dead, divergent, or off-target outcomes, using a long-horizon MSE threshold of 0.15 rather than the 0.02 reconstruction threshold.Finite-time perturbation growth samples unrenormalized perturbations over 64 steps and is not a maximum asymptotic Lyapunov exponent.Source: C MEASUREMENT DEFINITIONS · p. 13
D PRESPECIFIED COMPARISONS AND OBSERVATIONS
- The comparison table preserves original criteria alongside observed values; it reports the complete 27-configuration lattice comparison separately from the later non-marginal diagnostic.The recipe and spatial comparisons likewise retain per-target intervals and error ratios.Source: D PRESPECIFIED COMPARISONS AND… · p. 14
E LINEAR LATTICE VALIDATION
- The finite-time classification agrees with the strict theoretical criterion for 26/27 configurations; excluding the two marginal boundary cases yields 25/25 agreement and four mean-only misclassifications away from the boundary.At c = 0.25, α = 1, the checkerboard multiplier is exactly −1 and preserves amplitude despite a decay label; c = 0.50, α = 0.50 also lies on the mean-stability boundary.Source: E LINEAR LATTICE VALIDATION · p. 14
F SPATIAL EXTENSION
Spatial extension preserves low reconstruction error across the tested world sizes, while the adjacent synchronous-failure analyses show extinction as a common terminal outcome and successful lower-rate retrainings. A single replay also demonstrates that a temporarily usable checkpoint can occur in a configuration that ultimately fails.
F SPATIAL EXTENSION
- Every persist and regenerate group median remains at the 10^-4 error scale and at least 30 times below the quality threshold across one, four, and sixteen seeds.The largest per-model median tile error is 4.6 × 10^-3; median per-model error ratios are 1.45 for persist and 1.197 for regenerate.Source: F SPATIAL EXTENSION · pp. 14–15, Figure 4 · p. 16
G.1 FINAL STATES OF FAILED SYNCHRONOUS MODELS
- Six failed synchronous models are empty with zero non-DC power; the seventh, heart seed three, remains alive with deterministic MSE approximately 0.0290.The final-state probe identifies extinction as the most common terminal outcome.Source: §G.1 · p. 16
G.2 LOWER LEARNING RATES
- All four lower-rate synchronous retrainings of two failed flower configurations pass the quality test.The exploratory runs use the same training budget as the corresponding main runs.Source: §G.2 · p. 16
G.3 SINGLE-RUN CHECKPOINT REPLAY
- A replayed failed synchronous configuration had a usable checkpoint at step 1,053, with T = 96 MSE 0.002099, but its final checkpoint became empty and died by step 47.This single replay does not estimate how often such checkpoints occur or explain why later updates change the outcome.Source: §G.3 · p. 16
G.4 REMOVING THE LIVING MASKS
Living-mask removal changes few long-horizon outcomes, while frozen spectral measurements show that local expansion does not determine nonlinear fate. The returned eigenvalue window is incomplete, so its counts provide lower bounds rather than full-spectrum totals.
G.4 REMOVING THE LIVING MASKS
- Removing living masks at maturity changes two outcomes: both regenerate-heart models become off-target, while the other 28 outcomes remain unchanged.Neither mask-removal condition produces a dead or divergent run; six models in each intervention increase living-cell counts by more than 20%.Source: §G.4 · pp. 16–17, Table 5 · p. 18
H FROZEN SPECTRAL MEASUREMENTS
- Among 803 returned eigenvalues with modulus above one, 499 lie inside the strict D0.5 disk, 145 are real and exceed one, and 159 are complex with real part above one.Grow contributes 198 values with real part above one, versus 32 for regenerate.Source: H FROZEN SPECTRAL MEASUREMENTS · p. 17, Figure 6 · p. 18
- All 32 returned eigenvalues lie outside the unit disk for 23 models, but eight grow models become off-target while all fourteen persist and regenerate models retain the pattern.The 32-eigenvalue window makes the total count a lower bound on such multipliers in the full Jacobians.Source: H FROZEN SPECTRAL MEASUREMENTS · p. 17, Figure 6 · p. 18
J REPRODUCIBILITY DETAILS
The reproducibility details specify the study’s model counts, training settings, and evaluation controls. Recipe and factorial tables preserve model-level measurements for the reported comparisons.
J REPRODUCIBILITY DETAILS
- The study uses thirty asynchronous recipe models and ten synchronous persist models, with ten persist models reused in the update-mode comparison, for forty distinct base runs.Four additional runs test lower learning rates.Source: J REPRODUCIBILITY DETAILS · p. 18
- Training uses Adam at constant rate 2 · 10^-3, batch size eight, 8,000 steps, and rollout lengths sampled uniformly from 64 through 96.The network has sixteen state channels and 128 hidden units; pool recipes use 1,024 states, and regenerate damages half of each batch.Source: J REPRODUCIBILITY DETAILS · p. 18
- Recipe checkpoints are shared across trajectory, spectral, damage, and spatial measurements; mode swaps freeze weights, and living-mask interventions reuse original update tapes.The tables retain model-level measurements for the thirty recipe models and twenty factorial entries, which together describe forty distinct base models.Source: J REPRODUCIBILITY DETAILS · p. 18, Table 6 · p. 20, Table 7 · p. 21