Source-linked AI summary
Measuring Collective Semantic Change in Populations of Language Model Agents
Elena Kopteva
TL;DR
The paper asks how collective semantic change in language-model agent populations can be measured before conventional outcomes appear. It introduces Kopterix, a predefined passive longitudinal protocol that repeatedly samples and age-stratifies an agent-native feed. On Moltbook, lexical, geometric, and temporal controls reveal measurable change and persistence at bounded, qualified levels.
Problem
Existing endpoints such as task failure and team performance do not measure collective semantic change as a temporal process before terminal outcomes.
Method
Kopterix repeatedly observes bounded feed samples, divides each observation into surface, mid-stream, and residue layers, and analyzes lexical, embedding, geometric, and temporal quantities under predefined controls.
Results
The instrument resolves collective semantic motion, layer structure, and temporal persistence, including recurring mid-stream-to-residue separation and rapid similarity loss without strong weekly recurrence.
Takeaways & Limitations
Collective semantic change can be measured as a dynamical property while an agent population operates, without requiring a single conversation or a known mechanism.
Takeaways & Limitations
The embedding record stores only run and layer aggregates, preventing recomputation of embedding quantities on discursive subsets.
Abstract
from arXiv · showhide
Collective semantic change in populations of language model agents is a measurable dynamical phenomenon. We present a passive longitudinal instrument called Kopterix that observes the semantic state of an agent population as a sequence of bounded observations under a protocol defined before the observations begin. Each observation divides the sampled feed by post age into surface, mid-stream, and residue layers, which makes semantic differences across content age measurable alongside run-to-run change. We validate the instrument on Moltbook, an agent-native social platform, over a two-month window of scheduled observations, with the periodicity check extended across approximately four months. At the lexical level, rarefied entropy resolves an April-May difference in the evenness of the stored top 200 unigram distributions, and adjacent states are lexically closer than states paired after timestamp shuffling. At the geometric level, grand mean centering exposes the scale of a common embedding direction, and scheduled shuffle checks support a recurring excess in the mid-stream to residue separation relative to the shuffled reference. At the temporal level, detrended scalar quantities and centered layer centroids lose much of their similarity over several hours, and a weaker positive component declines across longer separations with no strong weekly recurrence. Several attractive apparent structures failed their controls, and each reading is limited to the level its controls support. The design applies wherever a population of agents produces a timestamped language environment that can be observed repeatedly and divided by content age.
1 Introduction
The paper addresses whether collective semantic organization in language-model agent populations can be measured as a temporal process before conventional outcomes or failures appear. It introduces and validates Kopterix, a controlled repeated-observation instrument that stratifies feed content by age and measures lexical, geometric, and temporal structure.
- Motivation: Collective semantic organization can change while agents continue functioning, yet ordinary monitoring does not directly measure that process.Prior studies primarily use task failure, team performance, or collective outcomes as endpoints, leaving earlier temporal change unresolved.
- Research questions: The paper asks whether collective semantic change can be measured early enough to support mission assurance in autonomous teams.
- Approach: Kopterix repeatedly observes an open agent-native population, stratifies each feed by post age, and interprets measurements through predefined controls and an inference ceiling.The design tests lexical, embedding, geometric, and temporal structure while keeping interpretation bounded by support conditions.
- Scope: The study distinguishes standing observation from retrospective archive analysis, with Moltbook serving as the validation setting rather than an operational agent-team deployment.
- Validation: The instrument resolves changes in lexical concentration, within-run homogeneity, run-to-run drift, cross-layer geometry, and temporal dependence during the two-month observation period.
2 Relation to existing work
Existing work studies Moltbook through interaction graphs, stored-corpus analyses, controlled collective behavior, and retrospective semantic development. This paper addresses a specific gap: no prior design repeatedly observes an open agent-native population while comparing content-age layers within each feed observation.
- Interaction graphs: Moltbook interaction-graph studies examine connectivity, central actors, communities, fragility, and coordination across time.
- Stored corpora: Stored-corpus research examines topics, toxicity, social structure, lexical properties, and semantic geometry using longitudinal archives.
- Controlled populations: Controlled experiments show that LLM agent groups can develop conventions, collective biases, and systematic failures to surface distributed information.
- Semantic development: Retrospective semantic analysis measures stabilization, lexical turnover, individual inertia, and influence persistence, while identifying shared temporal context as a rival explanation for apparent influence.
- Gap addressed: No prior design divided each feed observation by content age and compared the resulting semantic layers within the same run under a predefined observational protocol.The paper addresses this practical measurement gap with a standing instrument and explicit internal controls.
3 Instrument design
Kopterix treats each bounded feed sample as one observation, extracting fixed textual projections and deterministic lexical, embedding, layer, and temporal quantities. Its design supports controlled comparisons while explicitly limiting interpretation when records or measurements are insufficient.
- Observation unit: Each run is one bounded feed observation whose interpretation is based on the run as a whole.
- Observation protocol: The protocol samples surface, mid-stream, and residue layers covering 0-2, 3-8, and 12-24 hours before the run, with each layer capped at 250 posts.A complete run can therefore contain at most 750 fetched posts.
- Stored text: Each stored text consists of a title followed by the first 300 characters of the body, excluding full post bodies from measurement.
- Measurements: The instrument derives token counts, sentence embeddings, run means, layer centroids, and preceding-run state for deterministic comparisons.
- Support conditions: Embedding measurements are skipped when fewer than 10 usable stored texts remain, and sample support is recorded before quantities that depend on it are compared.
- Lexical and geometric quantities: Rarefied entropy measures lexical evenness within the stored top 200 unigram distribution, while embedding and layer quantities describe semantic position, homogeneity, drift, and cross-layer geometry.The embedding record stores aggregates, limiting later recomputation on discursive subsets without individual-text embeddings.
- Residual geometry: Grand-mean and layer-residual quantities separate run displacement from the common embedding center and describe layer displacement relative to the run mean.
- Design limitation: The available embedding record cannot recompute embedding quantities for discursive subsets because it stores only run and layer aggregates.
4 Observation record
The observation record covers two months with daily observations and irregular multi-hour gaps, while individual analyses use smaller subsets when fields or sample support are missing. Coverage is therefore broad but quantity-specific.
- Coverage: 231 observation rows and 215 state rows cover 2026-04-01 through 2026-06-01, with at least one observation each day.The median gap was 5.75 hours and the maximum gap was 23.64 hours.
- Record linkage: The record contains 121 April observations and 110 May observations, with 214 observation rows linked to state rows.Timestamp matching left 17 unmatched observation rows and 1 unmatched state row.
- Coverage summary: Table 2 summarizes observation and state row coverage within the observation window.
- Analysis-specific support: Individual analyses may use fewer rows when required fields are missing or fetched samples lack sufficient support.Usable row counts are reported separately for each analysis.
5 Lexical measurements
The lexical analysis first establishes that raw entropy is confounded by sample support, then uses rarefaction to compare stored top-200 unigram distributions at a common token budget. Rarefied entropy shows a robust April-May decrease that persists after sensitivity checks, while mint payload tokens do not explain it.
- Dependence on sample support: Raw Ht is strongly confounded by sample support: May gives r = 0.9353, while April's correlation is undefined because ntotal is constant.The two-month correlation is r = 0.9182, driven by May's variation in ntotal.
- Rarefied lexical entropy: Rarefaction recomputes Shannon entropy from the stored top 200 unigram distribution using 50 multinomial samples at the common budget B = 703.The calculation uses seed 42; B = 703 equals the smallest stored top-200 token total among valid rows.
- Interpretation boundary: Hrare measures evenness only within the truncated stored top 200 distribution, not entropy over the full vocabulary.A full-vocabulary metric would require complete unigram counts.
- April-May comparison: At B = 703, Hrare decreases from 6.297 bits in April to 6.137 bits in May, a −0.160-bit shift with p = 4.4 × 10−31 and q = 3.9 × 10−30.The shift is −1.374 times the two-month standard deviation, indicating a large separation relative to observed run-to-run variation.
- Sensitivity to sparse observations: Excluding six May observations with ntotal ≤300 changes the Hrare shift from −0.1600 to −0.1721 bits, whereas the raw Ht shift reverses from −0.0349 to 0.0501 bits.Across the full sample, exclusion changes mean Hrare by only −0.0038 bits, or 3.3% of the two-month standard deviation.
- Sensitivity to mint payload tokens: Subtracting identified mint payload tokens lowers Hrare by −0.001475 bits in April and −0.004949 bits in May, only 2.2% of the observed April-May shift.The payload tokens slightly increase entropy before subtraction and therefore do not account for May's increased lexical concentration.
6 Cross-layer geometry
The age-layer construction makes relative semantic motion and inter-layer geometry measurable within each run. Controls support a recurring mid-stream–residue separation, but not a stable ordering across all layers or periods.
- Measurement design: Each run compares surface, mid-stream, and residue centroids in a common 384-dimensional embedding space.The layers represent recent and older content separated by post age.
- Residual geometry: The median run-mean/equal-weight centroid gap is 1.39 × 10^-7, with 203 of 210 rows no larger than 10^-6.This supports an approximately zero-sum residual configuration in most observations.
- Residual geometry: Residual motion remains comparable to full centroid motion: mean residual/raw ratios range from 0.79 to 0.84 across layers.The mean displacement of the run mean is 0.0568 across 204 adjacent pairs.
- Residual geometry: The observed mean angle near 120° is a baseline imposed by centering three layer centroids around a common mean.Departures from this baseline depend on residual-norm differences and the gap between the run mean and equal-weight centroid mean.
- Shuffle control: The mid-residue distance exceeded its shuffled value in 10 of 13 checks, with a mean observed-minus-shuffled difference of 0.0166.The corresponding directional counts were 6 of 13 for surface–mid-stream and 7 of 13 for surface–residue.
- Cross-month stability: The April layer-ordering pattern did not recur in May, and current tests do not support stable ordering across periods.The thinned May checks also supported neither recurrence nor a clear departure from uniform ordering.
7 Temporal organization of the measured quantities
Temporal analyses test whether nearby lexical states are more similar and whether measured series contain intervals with different local behavior. True temporal neighbors are lexically closer than shuffled pairings, while changepoints identify candidate boundaries that require further semantic validation.
- Lexical temporal dependence: Timestamp shuffling tests whether consecutive lexical states are closer than rows made adjacent after temporal order is destroyed.The comparison uses stored top 200 unigram distributions and cosine distances.
- Lexical temporal dependence: Observed mean JS distance was 0.2513 versus 0.3184 shuffled, while cosine distance was 0.00815 versus 0.01611 shuffled.Across 214 consecutive pairs, the observed-to-shuffled ratios were 0.789 for JS and 0.506 for cosine distance.
- Lexical temporal dependence: April and May preserved the same lexical ordering, with observed-to-shuffled ratios of 0.765 and 0.812 for JS distance.The corresponding cosine-distance ratios were 0.475 and 0.537.
- Candidate temporal boundaries: PELT finds 12 changepoints in detrended semantic homogeneity and 5 in detrended run-to-run drift at the selected penalty.The strongest surviving boundaries are those that remain at the largest penalty.
- Candidate temporal boundaries: Within this segmentation, semantic homogeneity changes local statistical behavior more often than run-to-run drift.A candidate boundary is not itself evidence of a collective semantic-state transition.
8 Time scales of collective semantic change
Temporal analyses reveal a fast component of collective semantic change over several hours, followed by a weaker longer-lag component whose precise source and timescale remain only partly resolved. Centering controls show that common embedding direction and broad observation-window structure account for much of the apparent persistence.
- Aggregate scalar series: Detrended aggregate scalar series show autocorrelation decay estimates of about 4.5–17.1 h, resolving a fast temporal component.The estimates cover semantic homogeneity, run-to-run drift, and consecutive lexical cosine step; some confidence intervals extend beyond 24 h.
- Layer centroid similarity: Raw layer centroid similarities are high at lag 1—0.9565, 0.9570, and 0.9576—yet their long-lag decay times are not usefully constrained.Point estimates near one week have confidence intervals extending from roughly 15–21 h to 11,900–16,300 h.
- Centering controls: Grand mean centering reduces lag-1 centroid similarity to 0.36–0.39, indicating that much of the raw similarity reflects a common embedding direction.Month centering and coordinate-wise detrending reduce it further to 0.26–0.32, with decay estimates comparable to the 5.74 h sampling interval.
- Extended temporal structure: Within-month semantic-homogeneity fluctuations remain temporally dependent for several days, but no strong weekly recurrence is resolved.The controlled series loses resolved dependence by approximately four days, while the original negative lobe does not survive monthly centering.
- Long-lag persistence: Month centering and linear detrending reduce the long-lag floor by about 50%–64%, while a smaller positive remainder survives both controls.The surviving component remains positive across layers, but its source is unresolved; excluding cross-month pairs slightly raises rather than lowers the floor.
9 Discussion
The instrument treats collective semantic state as a population-level dynamical object, measured through repeated embedding, lexical, geometric, and temporal observations. Its readings support controlled claims about motion and persistence, while mechanisms remain unresolved without richer individual, identity, interaction, and external-context records.
- Population-level object: Repeated observations turn run means, semantic homogeneity, and post-age layer centroids into trajectories of population-level change.The three layers position different temporal parts of each sampled population relative to the same run state.
- Population-level object: Population observables describe collective state rather than individual behavior, so center movement and alignment changes can occur independently.The measured feed is a mixture of simultaneous activities, and layer centroids do not reconstruct communication paths.
- Interpretive limits: Temporal similarity cannot identify whether nearby observations reflect second-order memory or shared exogenous context.The measurements constrain compatible timescales but do not distinguish these mechanisms.
- Temporal dependence: A weaker positive similarity component remains at lags 12-16, approximately 0.067-0.092, while larger separations continue to decline and no strong weekly recurrence appears.A common external driver changing only over many days cannot alone explain the rapid several-hour loss of similarity, but comparable-timescale context remains compatible.
- Interpretive limits: The instrument cannot test mechanisms requiring individual embeddings, stable identifiers, direct post links, or independent external-event records.Those observables would allow internal distributions, persistent groups, transmission paths, and contextual residuals to be analyzed.
- Scope: The design generalizes beyond Moltbook to timestamped language environments that can be repeatedly observed and divided by content age.The required conditions are a defined schedule, timestamped activity, and an age-layer assignment rule.
10 Conclusion
Kopterix measures collective semantic change as a dynamical property through bounded, age-stratified observations. Moltbook validation finds lexical, geometric, and temporal structure, while controls reject several stronger interpretations and the design remains applicable to other timestamped agent environments.
- Instrument and scope: Kopterix measures population change through bounded observations whose feeds are stratified into surface, mid-stream, and residue layers.Age stratification places recent and older content within the same run, enabling within-run and run-to-run comparisons.
- Lexical results: Rarefaction reveals a clear April-May difference: the stored top 200 unigram distribution is less even in May than in April.Adjacent lexical states are closer than states paired after timestamp shuffling.
- Geometric results: In 13 usable shuffle checks, mid-stream and residue layers were farther apart than shuffled in 10 cases, with an average excess of 0.0166.Surface-mid-stream and surface-residue separations did not show the same consistency.
- Temporal results: Detrended quantities and centered layer centroids lose much similarity over several hours, while a weaker positive component declines at longer separations without strong weekly recurrence.Week-scale raw fit estimates have confidence intervals spanning hours to years and do not resolve a week-scale memory time.
- Controls and scope: The April modal layer ordering did not recur, global label permutations were not distinguished, and the approximately 120° residual pattern was construction-dependent.These failed checks enforce the instrument’s inference ceiling.
- Instrument and scope: The measurement design applies wherever agents repeatedly produce a timestamped language environment that can be divided by content age.Examples include shared messages, memory entries, task records, tool-mediated text, and operator instructions when timing and population scope are available.
11 Reproducibility and provenance
The paper documents its frozen deposit, extended analyses, computational provenance, and authorship responsibilities. Reproducibility depends on respecting differences between the frozen deposit and later extension materials and on handling timestamp-resolution assumptions explicitly.
- Frozen deposit: The frozen Zenodo deposit preserves archived data products, analysis code, software environment, reconstruction records, corrected bootstrap settings, superseded calculations, and known reproduction limits.The documentation identifies provenance materials and exact-reproduction constraints.
- Computational assumptions: Timestamp calculations assume nanosecond datetime resolution, whereas current pandas versions may use microsecond resolution and change time-based scales.Explicit conversion to nanoseconds is therefore part of reproducing those calculations.
- Extended analyses: The extended four-month autocorrelation analysis lies beyond the frozen deposit and is supplied separately with scripts, inputs, outputs, and checksums.The extension folder was added to the public GitHub mirror while the frozen v1.0.0 contents remained unmodified.
- Authorship and assistance: The author retained responsibility for instrument design, scientific questions, analysis selection and review, source and output verification, interpretation, and reported results.ChatGPT and Claude assisted with development, processing, checks, audits, and editing.
Appendix A Geometric origin of the residual angle baselines
The appendix derives two reference angle baselines for interpreting residual geometry: random directions concentrate near 90°, while three similarly sized vectors summing approximately to zero form pairwise angles near 120°.
- Reference baselines: Independent random directions in 384-dimensional embedding space concentrate near 90°.This supplies the random-direction geometric reference value.
- Reference baselines: Three similar-length vectors that approximately sum to zero have pairwise angles near 120°.This supplies the zero-sum geometric reference value used for residual-angle interpretation.
A.1 Independent random directions in high dimensions
Independent random directions in high-dimensional spaces have cosines concentrated near zero, corresponding to angles near 90°. This follows from rotational symmetry and the equal expected contribution of each coordinate.
- Rotational symmetry lets one vector align with the first coordinate axis, reducing their cosine to the other vector’s first component.
- Each squared coordinate contributes an equal expected share, so a unit vector in 384 dimensions has typical coordinate magnitude approximately 1/384.
- The cosine distribution is symmetric around zero, giving independent random directions an expected cosine of zero.
- 1/√384 ≈ 0.051 corresponds to an angular scale of approximately 0.051 radians, or 2.9°, around 90°.
- Rescaling rotationally symmetric Gaussian vectors to observed residual norms changes lengths but preserves directions and pairwise angles.
A.2 Three centered vectors of equal length
Three equal-length residual vectors that sum to zero form a symmetric 120° configuration in a two-dimensional plane, despite inhabiting a 384-dimensional space. Approximate centering and similar norms make 120° the natural reference, while deviations reflect imperfect balance.
- The zero-sum geometry follows by expanding the squared norm of the vector sum and substituting the common norm.
- Three equal-length vectors summing to zero have pairwise angles of 120° and lie in the two-dimensional plane spanned by any two vectors.
- The normalized dot product is −1/2, whose negative sign indicates an obtuse angle and whose magnitude fixes the angle at 120°.
- Measured residuals satisfy the ideal zero-sum condition when the run mean embedding equals the equal-weight centroid mean.
- Approximate centering and comparable residual norms make 120° the reference value; norm differences and mean gaps produce departures from it.