Source-linked AI summary
Stable Unsupervised Continual Chunking with Sheaf SyncMap
Xueyuan Li, Danilo Vasconcellos Vargas
TL;DR
Unsupervised continual chunking seeks groups of states that co-occur in temporal sequences while maintaining stable chunks. Sheaf SyncMap regularizes pairwise forces in Decentralized SyncMap, and achieves high NMI on independent graphs and after sequential graph shifts.
Problem
Continual chunking seeks to identify frequently co-occurring states in temporal sequences, while Decentralized SyncMap can lose formed chunks when limited observations cause substantial repulsion.
Source: §1 · pp. 1–2Method
Sheaf SyncMap adds radial sheaf regularization to pairwise forces, suppressing radial velocities to stabilize chunking dynamics.
Source: §1 · p. 2, §3.2 · p. 6Results
Sheaf SyncMap achieves the highest mean NMI of 0.9793 under dynamic memory, scores highest on 17 of 18 graphs, and recovers high NMI after each sequential graph transition.
Source: §5.1 · p. 9, §5.2 · p. 9Takeaways & Limitations
Across the evaluated CGCP graphs and sequential transitions, Sheaf SyncMap maintains high NMI while reducing frame-to-frame breathing without preventing adaptation to new graphs.
Source: §6 · p. 12Unconstrained relative motion in pairwise updates can still produce chunk expansion, contraction, and breathing.
Source: §2.1 · p. 4
Abstract
from arXiv · showhide
Unsupervised Continual chunking is a fundamental problem in machine learning and neuroscience, where the goal is to identify groups of states that frequently co-occur in temporal sequences. A key challenge is to form accurate chunks while maintaining their stability over time. In this work, we propose sheaf regularization to reduce local inconsistencies in Decentralized SyncMap, a self-organizing system, and thereby stabilize its chunking dynamics. We introduce a radial sheaf structure that penalizes distance-dependent radial motion between pairs of variables. Experimental results show that the proposed method achieves the highest normalized mutual information (NMI) among the evaluated SyncMap variants on 12 of 18 probabilistic Continual General Chunking Problem (CGCP) graphs with two-state memory and on 17 of 18 graphs with dynamic memory. In the sequential adaptation experiment, Sheaf SyncMap also achieves high NMI after shifts in the input distribution, indicating that it can adapt to new knowledge while avoiding the negative transfer commonly observed in modern machine learning systems such as neural networks.
1 Introduction
Self-organizing systems can adapt through local interactions, but preserving global structure from limited local observations remains a challenge. Sheaf regularization is proposed to stabilize Decentralized SyncMap’s chunking dynamics.
1 Introduction
- Self-organized biological structures can reorganize as environments change, motivating artificial systems that adapt while preserving previously formed global structures.The challenge is that individual components may have only limited local observations that do not explicitly encode those structures.Source: §1 · p. 1
- SyncMap recognizes co-occurring temporal patterns and encodes them as spatially chunked representations in an unsupervised manner.It can reorganize based on current observations.Source: §1 · p. 1
- Without regularization, chunking can fail when chunks outnumber variables per chunk or when observed variables collapse while other chunks expand and vanish.Sheaf regularization stabilizes chunking dynamics in both illustrated cases.Source: Figure 1 · p. 2
- Limited local observations can destabilize previously formed chunks in Decentralized SyncMap, causing them to spread and eventually vanish.Pairwise forces preserve more local structure than center-based updates, but do not prevent this instability.Source: §1 · p. 2
- Sheaf regularization stabilizes chunking by reducing local inconsistencies, using a radial structure that penalizes distance-dependent motion between variable pairs.The method also introduces directional history repulsion to distinguish interactions between positive and negative variables from same-group interactions.Source: §1 · p. 2
2 Related Work
This section introduces Decentralized SyncMap and cellular sheaves, defines the Continual General Chunking Problem, and describes NMI as the chunking-quality measure.
2.1 Decentralized SyncMap
- Decentralized SyncMap replaces positive and negative group centers with pairwise forces, preserving more local structure, but unconstrained motion can still destabilize chunks.Its history term accumulates repulsion for co-repelling pairs and clears memory when both endpoints become attractors.Source: §2.1 · pp. 3–4
2.2 Cellular Sheaf
- A cellular sheaf assigns vector spaces, called stalks, to graph vertices and edges, with restriction maps relating neighboring vertex data to edge data.Strain quantifies local inconsistency, and implicit sheaf-Laplacian diffusion updates vertex data to reduce it.Source: §2.2 · p. 4
2.3 Continual General Chunking Problem
- CGCP identifies frequently co-occurring states in temporal sequences while allowing the underlying data-generating structure to change over time.The experiments use 18 connected probabilistic graphs with 600 variables each and evaluate both two-state and 30-state memory.Source: §2.3 · p. 5
2.4 Mutual Information
- NMI measures similarity between predicted and ground-truth clusterings, is invariant to cluster-label permutations, and ranges from 0 to 1.Predicted clusterings come from DBSCAN, with its density scale selected by maximizing NMI against ground-truth labels, an oracle evaluation.Source: §2.4 · p. 5
3 Methodology
The method modifies Decentralized SyncMap with directional history repulsion and a radial sheaf regularizer to stabilize chunking dynamics. It also defines BRMS to measure frame-to-frame changes in chunk distances.
3.1 Directional history repulsion
- Directional history clearing preserves the reverse transition memory: after observing i→j, only H_ij is cleared while H_ji remains, and the history matrix is re-symmetrized before pairwise forces are computed.This incorporates sequential information while keeping the pairwise history-repulsion term symmetric.Source: §3.1 · p. 6
3.2 Radial Sheaf SyncMap
- The radial sheaf regularizer suppresses pairwise radial velocities to resist chunk expansion or collapse, while applying relative radial geometry to every variable pair.Its edge stalk represents relative radial velocity; the corrected velocities remain close to the proposed velocities while radial strain is reduced.Source: §3.2 · p. 6
- The objective balances closeness to the proposed velocity against radial inconsistency, with λ controlling the trade-off; conjugate gradient solves the resulting system for a unique corrected velocity.The radial sheaf Laplacian is positive semidefinite, making the system matrix positive definite.Source: §3.2 · pp. 7–8
3.3 Chunking Instability Score
- BRMS measures chunk expansion and contraction through changes in the intra-chunk mean pairwise distance across consecutive recorded frames.A BRMS of 0 indicates constant mean pairwise distances; lower values indicate less frame-to-frame breathing.Source: §3.3 · p. 8
4 Experiment
The experiment compares four SyncMap variants on 18 probabilistic CGCP graphs under two-state and dynamic memory. It uses shared trajectories across methods, fixed Sheaf SyncMap hyperparameters, and sequential graph initialization for adaptation experiments.
4 Experiment
- Each graph and memory setting uses five shared-seed trajectories of 80,000 states in a three-dimensional embedding, with mean ± sample standard deviation reported.Coordinates are recorded every 100 updates, yielding 799 frames after the initial singleton state is removed.Source: §4 · p. 8
- Sheaf SyncMap uses λ = 1, b = 10, and S = 200 across graphs and seeds without graph-specific tuning, while baseline hyperparameters follow.NMI is computed using DBSCAN with min_samples=2; BRMS uses ground-truth chunks over all recorded frames.Source: §4 · p. 8
- Adaptation experiments initialize each next graph from the previous graph’s final embedding coordinates and model state; comparisons are restricted to the SyncMap family.The stated scope is to examine how sheaf regularization stabilizes chunking dynamics.Source: §4 · p. 8
5 Results and Discussion
Sheaf SyncMap achieves the strongest overall NMI results on independent CGCP graphs and recovers after sequential graph changes while generally reducing chunking instability. Its radial regularization suppresses radial inconsistency more strongly than uniform velocity rescaling.
5 Results and Discussion
- The experiments compare NMI on independent CGCP graphs under two state-memory settings and evaluate sequential adaptation using both NMI and BRMS.NMI measures chunking alignment, while BRMS distinguishes stability from embeddings that are stable but poorly aligned with ground-truth chunks.Source: §5 · p. 9
5.2 Sequential Adaptation Experiments
- Sheaf SyncMap approaches NMI 1.0 on the first four sequential stages and remains near 0.95 on the final graph, while generally producing less frame-to-frame breathing than Decentralized SyncMap.It exceeds Decentralized SyncMap in the first three stages and is comparable in the final two; stages inherit the preceding graph’s embedding coordinates and model state.Source: §5.2 · p. 9, Figure 3 · p. 10, Figure 4 · p. 11
5.1 Independent CGCP Chunking Performance
- Sheaf SyncMap achieves mean NMI 0.9663 with 2-state memory and mean NMI 0.9793 versus 0.7608 for Decentralized SyncMap with dynamic memory, winning on 17 of 18 graphs.It does not win on probabilistic3_200, where Decentralized SyncMap scores 0.9382 and Sheaf SyncMap scores 0.8999.Source: §5.1 · p. 9, Table 1 · p. 10, Table 2 · p. 11
5.3 Sheaf Behavior Analysis
- The retained radial-inconsistency magnitude is approximately 10–100 times lower than retained overall velocity, indicating selective suppression beyond uniform velocity rescaling.The reported analysis measures retained overall velocity magnitude and radial-inconsistency magnitude after regularization.Source: §5.3 · pp. 9, 12, Figure 5 · p. 12
6 Conclusion
Sheaf SyncMap uses a radial sheaf structure to regularize pairwise forces and stabilize chunking. Experiments report higher NMI on independent CGCP graphs and high NMI with reduced breathing during adaptation; future work will test real-world temporal data and theoretical properties.
6 Conclusion
- Sheaf SyncMap incorporates a radial sheaf structure to regularize pairwise forces and stabilize chunking dynamics.Experiments report higher NMI than other SyncMap variants, particularly for graphs with many small chunks, and high NMI with reduced frame-to-frame breathing during adaptation.Source: §6 · p. 12
A.1 Ablation Study
The ablation compares Sheaf SyncMap with variants that remove either or both of its two modifications. In the 75 × 8 scenario, both modifications contribute to NMI, while the radial sheaf structure contributes more strongly to stability across scenarios.
A.1 Ablation Study
- The ablation compares the full Sheaf SyncMap against radial-sheaf-only, directional-history-only, and unmodified Decentralized SyncMap configurations.The study evaluates the contributions of the radial sheaf structure and directional history repulsion.Source: §A.1 · p. 14
- In the 75 × 8 scenario, NMI ranks Sheaf SyncMap > directional-history-only > radial-sheaf-only > baseline.The comparison indicates that both modifications contribute to performance, with directional history repulsion making the larger contribution.Source: Figure 6 · p. 14, §A.1 · p. 15
- Across all scenarios, the radial sheaf structure contributes more strongly to chunking stability.Figure 7 reports the local chunking instability score BRMS(t).Source: §A.1 · p. 15, Figure 7 · p. 15