Source-linked AI summary
Multiclass Boundary Refinement with Neural Cellular Automata for Remote Sensing Land Cover Segmentation
Pascoassis Souza Santos Meira, Gilson Antonio Giraldi, Rafael Duarte Coelho dos Santos
TL;DR
Remote-sensing segmentation can locate class boundaries poorly even when it identifies region interiors, and boundary position error is measured in a narrow band. The paper iteratively refines a frozen segmenter's multiclass probabilities with BRNCA; its best LandCover.ai configuration raises mean BIoU from 46.63 to 48.99.
Problem
Remote-sensing segmenters can identify regions more reliably than they place their boundaries, while global IoU is dominated by interiors rather than boundary-position error.
Source: §1 · p. 1Method
BRNCA iteratively applies a learned local rule to a frozen segmenter's class probabilities, keeping visible channels on the simplex and adding free hidden channels.
Source: §3.1 · p. 3, §6 · p. 11Results
Mean BIoU rose from 46.63 to 48.99 for the best LandCover.ai configuration, and woodland gained +8.5 pp BIoU.
Source: §6 · p. 11Takeaways & Limitations
The class ranking by gain almost follows boundary fractal-dimension ranking, while boundary gains accompany interior-IoU losses.
Source: §5 · p. 8, §6 · p. 11The evaluation uses one frozen DPT–DINOv2 segmenter on one benchmark, with the conditioning encoder and automaton steps held fixed.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Semantic segmentation of remote sensing imagery often places class boundaries a few pixels away from where they truly lie, and this position error is concentrated in the thin band measured by Boundary IoU (BIoU). We treat boundary refinement as the iterated application of a small learned local rule: the Boundary Refinement Neural Cellular Automaton (BRNCA), that starts from the class probabilities of a frozen, pre-trained segmenter and corrects them cell by cell. We generalise the binary rNCA to the multiclass case by keeping the visible channels on the probability simplex through an Aitchison (log-space) update and by adding free hidden channels. The best configuration improves the mean BIoU over the base segmenter by 2.36 points (from 46.63 to 48.99). The gain is concentrated on the irregular woodland class (+8.5 points BIoU); this suggests that position error is correlated with the fractal dimension of the class boundaries.
1 Introduction
Remote-sensing maps depend on correctly placed class boundaries, but modern segmenters are less reliable at locating boundaries than identifying regions. The paper introduces BRNCA to iteratively refine multiclass boundaries and examines boundary complexity and design choices.
1 Introduction
- Land-use and land-cover maps support territorial-change monitoring and environmental policy, but segmenters are less reliable at placing boundaries than deciding what a region is.The paper situates this gap across convolutional networks, Vision Transformers, and foundation-model encoders.Source: §1 · p. 1
- Boundary IoU (BIoU) isolates position errors in a narrow band around annotated boundaries, unlike global IoU, which mixes them with semantic errors and is dominated by region interiors.A position error displaces the boundary between correctly classified neighboring regions; a semantic error assigns a whole region the wrong class.Source: §1 · p. 1
- Natural land-cover contours are irregular, and finite scene windows motivate local, iterated refinement; the paper quantifies boundary fractal dimension to examine the first obstacle.The stated obstacles are irregular natural-class contours and a segmenter's limited view of a continuous scene.Source: §1 · p. 1
- BRNCA generalizes binary rNCA to multiclass boundary refinement, with simplex-valued class probabilities, free hidden channels, and a deliberately minimal design.The paper also proposes a controlled ablation of hidden-channel and update-MLP sizes and per-class BIoU/IIoU analysis related to boundary fractal dimension.Source: §1 · p. 2
2 Related Work
Prior work includes neural cellular automata for image growth and binary segmentation repair, alongside multiclass boundary-refinement methods. This paper's related-work context distinguishes its target—multiclass land-cover boundary position—from binary-mask topology repair.
2 Related Work
- NCAs use a shared local update rule, while rNCA applies an image-conditioned NCA to repair binary medical-image masks and targets topological errors.The cited NCA lineage includes image growth and later variants that remove the alive mask or stochastic asynchronous updates.Source: §2 · p. 2
- Unlike rNCA's binary topology target, BRNCA addresses boundary position in multiclass land-cover mapping.The figure depicts a BRNCA step: a frozen segmenter initializes the state, then an image-conditioned MLP computes increments in a synchronous loop.Source: §2 · p. 3, Figure 2 · p. 3
- Multiclass refiners include dense CRF variants, SegFix boundary-label replacement, and SegRefiner's discrete diffusion of coarse masks.The dense CRF combines base-prediction unary terms with pairwise penalties for nearby pixels that disagree in label and appearance.Source: §2 · p. 3
3 Boundary Refinement Neural Cellular Automaton (BRNCA)
BRNCA treats pixels as cells whose state combines base-segmenter class probabilities and hidden channels. A shared, image-conditioned local rule iteratively updates that state, using a simplex-preserving visible update; the minimal design is trained with the base segmenter frozen.
3.1 Setting and state
- Each pixel cell starts with the frozen segmenter's simplex-valued class probabilities and zero-initialized hidden channels that evolve as short-term memory.The refined segmentation is the visible probability state after T steps.Source: §3.1 · p. 3
3.2 Update rule
- A fixed identity-and-Sobel perception bank and an image-conditioned shallow MLP produce increments, with an Aitchison visible update and additive hidden-channel update.The multiplicative reweighting of current class evidence keeps visible probabilities on the simplex.Source: §3.2 · p. 4
3.3 Design choices: a minimal BRNCA
- The minimal BRNCA removes the alive mask, sample pool, and stochastic asynchronous updates; every cell updates synchronously.The dense initial state motivates omitting a growth-front mask, and the pool is omitted because training starts from the base prediction and zero hidden state, matching inference while avoiding extra memory transfers.Source: §3.3 · p. 4
3.4 Training and inference
- The frozen base segmenter is held fixed while BRNCA is trained by BPTT on final-state cross-entropy, with 32–64 training steps and 64 inference steps.The cross-entropy is averaged over pixels and computed from the final visible probabilities.Source: §3.4 · p. 5
4 Experiments
Experiments on LandCover.ai evaluate BRNCA against a frozen DPT–DINOv2 segmenter, testing core configurations and where boundary gains arise. The best configuration improves mean BIoU, with gains varying by class and performance declining when inference exceeds the trained step range.
4.1 LandCover.ai Dataset
- LandCover.ai contains aerial orthophotos of Poland with five classes, and its annotated boundary fractal dimensions range from D = 1.13 for water to D = 1.33 for background.Background's high D reflects its role as the complement of the other four classes, so it does not represent a natural land-cover class's boundary property.Source: §4.1 · p. 5
4.2 Setup
- The frozen DPT–DINOv2 base segmenter scores 83.69% mIoU and 46.63% BmIoU on LandCover.ai, with BIoU measured in a 5-pixel boundary band.BRNCA processes four 259 × 259 sub-windows of the segmenter's 518 × 518 input; reported results use the epoch with best BmIoU.Source: §4.2 · pp. 5–6
4.3 Core ablation
- 2.36 pp is the largest BmIoU gain: every core configuration improves both metrics, and m = 15, dh = 256 gives 48.99 BmIoU versus the 46.63 base.The best mIoU is 84.31 with m = 20, dh = 256; BmIoU gains range from 0.93 to 2.36 pp, while mIoU gains range from +0.17 to +0.62 pp.Source: Table 1 · p. 6, §4.3 · p. 6
- 48.86 is the mean BmIoU for dh = 256, versus 48.10 for dh = 64; averaged across settings, hidden-channel count has a weaker, non-monotonic effect.Even m = 10, dh = 64 captures 75% of the maximum gain, while absolute position error remains large.Source: §4.3 · p. 6
4.4 Where does the gain come from?
- Woodland gains 8.49 pp BIoU, while the mean gain without woodland falls from 2.36 to 0.83 pp; boundary gains vary sharply by class.The class-gain ordering is similar to the fractal-dimension ordering, but the authors report only a weak Pearson correlation (r = 0.38); road and woodland have nearly equal D but gains of 0.82 and 8.49 pp.Source: §4.4 · p. 7
- IIoU decreases for every class, but by an average of −0.23 pp against +2.36 pp mean BIoU gain; global mIoU rises by +0.54 pp.The authors note that BIoU and IIoU use different pixel populations and cannot simply be subtracted; they suggest the shared rule may perturb some interior cells. Woodland's massive patches concentrate more pixels outside the boundary band than road's thin, tubular regions.Source: §4.4 · p. 7
4.5 How the automaton evolves
- Between t = 8 and t = 32, the example building sharpens and road edges straighten, while uncertainty narrows toward contours; small isolated pixel groups appear from about t = 40.Although changes become subtle after t = 32, BmIoU gain continues growing until about t = 62.Source: §4.5 · p. 8
4.6 Behaviour beyond the maximum training step
- At inference steps beyond 64, BmIoU gain declines from 2.36 pp near t = 60–72 to about 0.60 pp at t = 128.The minimal design omits a sample pool, so the rule is not trained on states beyond the maximum training step and its behavior there is unconstrained.Source: §4.6 · p. 8
5 Discussion
BRNCA improves boundary scores modestly across configurations, with gains concentrated on woodland and some cost to interior accuracy. The discussion also highlights the step-count trade-off, study scope, and directions for further evaluation.
What the evidence supports
- Across all nine ablation configurations, BmIoU rises by 0.9 to 2.4 pp while mIoU changes by 0.2 to 0.6 pp, but BmIoU remains below 50%.The gains are concentrated on woodland, and the authors characterize the net effect as reallocating accuracy toward the boundary band while removing only a small part of position error.Source: §5 · p. 8
What the dynamics tells us about the design
- Figure 4 follows predicted labels and entropy-based uncertainty maps at nine steps, from t = 1 through t = 64.The panels are read column by column, with predictions on the left and uncertainty on the right.Source: Figure 4 · p. 9
- The best configuration’s BmIoU gain peaks at 2.36 pp around t = 60–72, then falls to about 0.60 pp at t = 128.The chosen inference setting, T = 64, falls within the peak plateau; running beyond the maximum training step departs from the training objective and loses performance.Source: Figure 5 · p. 10, §5 · p. 10
- The rule updates every cell without an alive mask, enabling contour shifts but also perturbing non-boundary regions and lowering IIoU.Removing the sample pool makes training computationally efficient, but leaves dynamics beyond T = 64 uncontrolled.Source: §5 · p. 10
Scope and limitations
- The study uses one frozen DPT–DINOv2 segmenter on LandCover.ai, fixes the encoder and step count, and evaluates only one BIoU band width.The effects of the removed alive mask, sample pool, and asynchronous updates are not directly ablated; comparisons with other refiners and computational costs are also unreported.Source: §5 · p. 10
Future work
- Future work proposes restoring the sample pool, comparing refiners across at least two base segmenters, and evaluating additional land-cover datasets.The proposed comparisons would report ∆IoU, ∆BIoU, and computational cost.Source: §5 · p. 10
6 Conclusion
BRNCA refines multiclass boundaries through a learned local rule applied to a frozen segmenter’s output. On LandCover.ai, all nine tested configurations improve BmIoU, with gains concentrated on woodland and declining when run beyond the training step count.
6 Conclusion
- BRNCA applies a small learned local rule to a frozen segmenter, keeping class probabilities on the simplex with an Aitchison update and adding free hidden channels.The design uses a fixed perception bank and synchronous updates, without a sample pool or alive mask.Source: §6 · p. 11
- All nine configurations improve BmIoU over the DPT–DINOv2 base by 0.9 to 2.4 pp, with the best rising from 46.63 to 48.99.Across configurations, mIoU changes by +0.2 to +0.6 pp.Source: §6 · p. 11
- Woodland gains +8.5 pp BIoU, and class gains broadly track boundary fractal dimension; the gain is accompanied by a small interior-IoU loss and decays beyond T = 64.The smallest configuration retains three quarters of the best BIoU gain.Source: §6 · p. 11