Lexical Consensus: Grounded Word Learning and Shared Meaning in Artificial Agents

Patricio M. Vera

Source-linked AI summary

Lexical Consensus: Grounded Word Learning and Shared Meaning in Artificial Agents

Patricio M. Vera

arXiv:2606.22207v1Jun 20, 2026cs.CLcs.AI

TL;DR

The paper asks whether artificial agents can acquire, stabilize, and use new lexical meanings from grounded experience. Using artificial labels over frozen visual embeddings, it finds that acquisition follows perceptual coherence, while naming and retrieval reveal distinct capacities.

Problem
Existing evaluations leave open whether artificial agents can acquire new lexical meanings from grounded experience, maintain them across unseen instances, and use them as names and retrieval cues.
Method
Lexical Consensus tests artificial word–concept mappings over frozen visual embeddings, comparing native, coherent overextended, and disjunctive concepts with bidirectional evaluation.
Results
Acquisition follows a perceptual-coherence gradient from native categories to chance-level far-disjunctive concepts; perceptual distance predicts accuracy (partial R2 = 0.245, p < 10−7), while naming and retrieval differ.
Takeaways & Limitations
Grounded lexical learning over frozen perception is measurable but bounded by perceptual coherence, and complete evaluation should assess both naming and retrieval.
Takeaways & Limitations
The experiments cover controlled visual word–concept episodes rather than open-ended language learning, excluding syntax, pragmatics, dialogue, compositional semantics, and embodied action.

Abstract

from arXiv · show

Artificial intelligence systems are commonly evaluated through task performance and behavioral imitation, but such evaluations leave open whether an artificial agent can acquire, stabilize, and use new lexical meanings from grounded experience. This paper introduces Lexical Consensus, an experimental framework for studying grounded word learning over a structured perceptual substrate. Using frozen DINOv2 visual embeddings, Carroll-style nonce words, and interpretable lexical learners plus linear baselines, we test whether agents can acquire artificial labels for visual concepts, generalize them bidirectionally, and stabilize them across controlled settings. The main result is a robust perceptual-coherence gradient: native categories are easiest to learn, coherent overextensions remain learnable, mid-range disjunctive concepts degrade, and far-disjunctive concepts approach chance. A pre-registered CIFAR-100 dissociation experiment confirms that this gradient is governed by perceptual distance rather than semantic relatedness: perceptual distance predicts acquisition accuracy (partial R^2 = 0.245, p < 1e-7), while semantic distance adds no significant explanatory power (partial R^2 = 0.002, p = 0.660). Bidirectional evaluation shows that naming and retrieval are distinct: exemplar-based mechanisms outperform centroid prototypes in label-to-image retrieval, exposing a memory-fidelity dimension separate from naming accuracy. Falsification controls, homogeneous candidate-pool evaluations, and null results on representational restructuring indicate that frozen perceptual geometry both enables lexical grounding and limits what can be acquired without representational adaptation.

1 Introduction · 2 Experimental Framework

Lexical Consensus operationalizes grounded lexical acquisition as the learning, bidirectional use, and stabilization of artificial word–concept mappings over frozen visual perception. Its framework tests how acquisition varies with perceptual coherence while separating naming, retrieval, consensus, and representational alignment.

1 Introduction: Lexical Consensus tests whether agents can acquire novel Carroll-style labels from visually grounded examples and use them for naming, retrieval, and multi-agent coordination.The framework treats a word as meaningful when it generalizes to unseen instances, supports both directions of grounding, and stabilizes across agents.
2 Experimental Framework: The experiments use opaque Carroll labels such as slithy, mimsy, and vorpal with frozen DINOv2 embeddings, making meanings emerge from associations with visual examples.Labels are not supplied with definitions, category names, or semantic descriptions.
2.5 Bidirectional Grounding: Bidirectional grounding distinguishes label-to-image retrieval from one-way naming, exposing separate dimensions of grounded word learning.A label can support naming while failing retrieval, or retrieve valid instances while remaining difficult to discriminate among competing labels.
2.1 Perceptual Substrate: The framework separates a frozen perceptual encoder, learnable lexical layer, optional consensus mechanism, and measurement layer for grounding, retrieval, convergence, and alignment.This separation makes the perceptual geometry the substrate over which lexical attachment and coordination are measured.
2.2 Artificial Lexicon: Taught lexical concepts are extensions assigned to artificial labels and may represent one native category, coherent overextensions, or disjunctive unions of multiple categories.Native category names are removed from the agent-facing protocol; the hidden mapping is used for evaluation and data construction.
2.3 Concept Types and Perceptual Coherence: Concept-carving evaluates native, near-disjunctive, mid-disjunctive, and far-disjunctive concepts ordered by their alignment with frozen perceptual geometry.Lower inter-member distance indicates nearby perceptual regions, whereas higher distance indicates concepts joining distant regions.
2.8 Information-Theoretic Measures: The measurement framework includes information-theoretic diagnostics and falsification controls, including normalized mutual information and out-of-vocabulary rejection.Normalized mutual information is set to 0 when H(L) = 0, while rejection tests whether unknown inputs are incorrectly forced into known labels.
2.12 Scope of the Framework: The framework is a constrained test rather than a full model of language acquisition, because agents use fixed perception and do not learn syntax or embodied interaction.Within this scope, perceptual-coherence gradients and representational-alignment null results define what can be acquired without representational adaptation.

3 Results

Results show that arbitrary words can be grounded in native visual categories, but acquisition is constrained by perceptual coherence in frozen DINOv2 geometry. Bidirectional evaluation further separates naming accuracy from retrieval fidelity, with exemplar memory improving retrieval while linear baselines often perform best under difficult pools.

Native-category acquisition: 0.9833 accuracy was achieved with 5 examples per label, rising to 1.0000 with 10 examples and remaining 1.0000 with 15 examples.The frozen embedding space also exceeded the predefined clustering threshold, with silhouette score 0.2826 versus 0.25.
Concept carving: C1 naming accuracy follows a monotonic perceptual-coherence gradient: native categories are easiest, coherent overextensions remain learnable, and far-disjunctive concepts approach chance.Mid-disjunctive concepts show partial degradation as taught concepts depart from native perceptual categories.
Concept carving: Centroid, exemplar, and linear learners follow the same C1 ordering across tiers, indicating that naming accuracy reflects compatibility with frozen perceptual geometry rather than arbitrary set learning.Bootstrap 95% confidence intervals do not overlap across tiers.
Perceptual-semantic dissociation: Perceptual distance predicted C1 accuracy (partial R2 = 0.245, p < 10^-7), whereas semantic distance added no significant explanatory power (partial R2 = 0.002, p = 0.660).Across 4,950 pairs, perceptual and semantic distances were weakly correlated: Pearson r = 0.155 and Spearman ρ = 0.089.
Bidirectional evaluation: Exemplar-based retrieval outperformed centroid prototypes even for native categories, with gap = +0.032, 95% CI [0.007, 0.056], paired Wilcoxon p = 0.017.The advantage was larger for near-disjunctive than native concepts, with ∆gap = +0.042, 95% CI [0.007, 0.077].
Bidirectional evaluation: C1 naming reflects concept–geometry compatibility, whereas C2 retrieval reflects memory fidelity and candidate-set discrimination; linear logistic and SVM baselines often excel under hard pools.The exemplar-over-centroid gap persists across pool constructions, including random pools and heavy OOV contamination.

4 Positioning with Respect to Prior Work

Lexical Consensus positions itself as a constrained, reproducible framework for studying grounded lexical acquisition and stabilization rather than unrestricted language emergence or complete symbol grounding. Its results show that acquisition depends on perceptual coherence, while consensus feedback coordinates decisions without substantially restructuring frozen perceptual geometry.

Acquisition-based evaluation: Lexical Consensus tests whether novel words acquired from limited grounded examples support both naming and retrieval, evaluating a learning process rather than only an output state.The framework is designed as a controlled empirical step beyond ordinary performance benchmarks.
Grounding and perceptual coherence: Labels are most reliably acquired for native perceptual regions, remain learnable for coherent overextensions, and degrade toward chance for arbitrary unions of distant regions.The experiments treat lexical acquisition as graded rather than all-or-nothing.
Grounding and perceptual coherence: The framework defines grounding relationally: a label becomes learnable when instruction stabilizes over a coherent region of experience and difficult when assigned across distant regions.This restricted account does not claim that arbitrary labels alone generate meaning.
Representation and consensus: Shared perceptual geometry is the dominant stabilizing force, whereas consensus feedback coordinates lexical decisions without substantially reshaping the underlying frozen representation.The no-feedback baseline already achieves high consensus accuracy, and alignment and divergence experiments find little representational reorganization.
Scope and contribution: The paper presents Lexical Consensus as a reproducible scaffold for falsifiable questions about grounded acquisition, bidirectionality, broken perception or binding, convergence, and representational restructuring.It explicitly limits its claims: acquisition and consensus are supported under controlled conditions, but unrestricted language acquisition and deeper representational change are not.

5 Limitations and Future Scaling

Lexical Consensus validates grounded lexical acquisition only within controlled visual episodes, not open-ended language learning or perception-changing language effects. Its limitations motivate scaling toward richer concepts, datasets, compositional labels, larger agent populations, and trainable perceptual components.

Scope: The framework measures word–concept mappings over visual embeddings, not syntax, pragmatics, dialogue, compositional semantics, or embodied action.The paper therefore avoids claiming full human-like language acquisition.
Perceptual substrate: A frozen DINOv2-small encoder isolates lexical acquisition over perception but prevents conclusions about perception learned from scratch or language-reorganized visual representations.The grounding claim applies to the lexical layer, because the encoder geometry is pretrained.
Dataset and domain breadth: The controlled visual domain does not establish that the perceptual-coherence gradient generalizes to richer datasets, finer-grained domains, or naturalistic category hierarchies.Future experiments should broaden dataset and domain coverage.
Concept structure: Current concept carving tests disjunctive category structures but not compositional labels, relational descriptions, or systematic recombinations of attributes and objects.This limits conclusions about compositional language.
Learner architecture and bidirectional grounding: Different learner mechanisms expose distinct capacities: naming depends mainly on concept–geometry compatibility, whereas retrieval depends on memory fidelity and candidate-set discrimination.The architecture comparisons include centroid, multi-centroid, exemplar k-NN, logistic regression, linear SVM, and random baselines.
Future scaling: Future scaling should add richer concept hierarchies, larger vocabularies and datasets, compositional labels, more agents, longer interactions, and trainable lexical or perceptual adapters.Consensus should also be tested on mid-disjunctive concepts, where feedback has more room to improve individual accuracy.

6 Conclusion

Lexical Consensus shows that grounded lexical acquisition over frozen perception follows a perceptual-coherence gradient, while naming, retrieval, consensus, and representational change reveal distinct capacities and limits. The framework provides a controlled, falsifiable scaffold for studying shared meaning without yet demonstrating complete artificial language acquisition.

Scope and future directions: Lexical Consensus is a reproducible experimental scaffold rather than a final benchmark for complete artificial language acquisition.The framework supports controlled lexical mappings, falsifiable controls, acquisition gradients, and future extensions toward richer environments, interaction, syntax, pragmatics, and compositionality.
Core findings: Lexical acquisition over frozen perception follows a perceptual-coherence gradient rather than arbitrary set learning.Native categories are easiest to acquire, coherent overextensions remain learnable, and increasingly disjunctive concepts degrade toward failure.
Core findings: Naming and retrieval expose different capacities: naming depends on concept–geometry compatibility, whereas retrieval reveals memory fidelity.C1 tests assigning labels to images; C2 tests retrieving valid perceptual instances from learned labels.
Core findings: Falsification controls show that grounding effects degrade or collapse under randomized labels, randomized embeddings, permuted bindings, and out-of-vocabulary examples.Repeated scrambles and homogeneous candidate pools reduce artifact explanations involving label frequency, candidate construction, or favorable splits.
Core findings: Agents trained on disjoint seed sets can converge toward shared lexical usage, and feedback improves agreement, but consensus does not substantially reorganize internal representations.Passive centroid alignment, mutable adapters, and regional divergence tests identify shared perceptual geometry as the dominant stabilizing force.

A Supplementary Experimental Inventory

The appendix inventories the study’s experimental sequence and clarifies the scope of empirical validation supporting the main results. It introduces no additional claims.

Purpose: Table 6 summarizes the experimental sequence used in the study and makes its empirical-validation scope explicit.The appendix is intended to document coverage rather than extend the paper’s claims.
Experimental inventory: The inventory covers experimental families that include sub-experiments, controls, homogeneous re-runs, and dissociation analyses.These components are summarized as part of the study’s experimental families.

B Concept-Carving Supplement

The concept-carving supplement tests whether artificial labels remain learnable as their extensions depart from native visual categories. Results show a perceptual-coherence gradient from reliable native-category acquisition to near-chance learning for far-disjunctive concepts.

Concept tiers: Concept-carving organizes taught concepts into four tiers—native, near-disjunctive, mid-disjunctive, and far-disjunctive—according to their relationship to frozen perceptual geometry.These tiers extend the native-category acquisition setting to concepts whose extensions depart from native visual categories.
Perceptual-coherence gradient: Native categories are acquired most reliably, coherent overextensions remain highly learnable, mid-disjunctive concepts degrade, and far-disjunctive concepts approach chance.This pattern is reported for representative C1 naming in the homogeneous 3-way, 5-shot setting.
Supplement purpose: The supplement clarifies how the concept tiers should be interpreted rather than duplicating the numerical C1 gradient table.Its purpose is to make the tier interpretation explicit for the concept-carving evaluation.

C C2 Retrieval Pool Conditions

C2 retrieval evaluates whether learned labels recover valid images from candidate pools under controlled pool conditions. Hard-pool results show that exemplar memory outperforms centroid prototypes, while logistic regression extracts additional discriminative structure from frozen embeddings.

C2 Retrieval Pool Conditions: Homogeneous candidate-pool conditions were used across all concept tiers because retrieval accuracy depends on candidate-set construction.These conditions are summarized in Table 8.
C2 Retrieval Pool Conditions: Exemplar memory improves retrieval over compressed centroid prototypes across all concept tiers in hard pools.Hard pools use distractors that are visually close to the target concept in frozen embedding space.
C2 Retrieval Pool Conditions: Logistic regression recovers additional discriminative structure from the frozen embedding space in hard-pool retrieval.Corresponding accuracies for each pool construction are reported in Figure 5.

D NMI Gate Diagnostic Audit

The NMI pre-episode gate was non-discriminative for balanced unions because its value is fixed by the two-from-ten construction. Consequently, concept tiers relied on inter-member distance and native separability rather than NMI-based selection.

Diagnostic computation: NMIgate(B, Y) = 0.3570 for balanced two-category unions drawn from ten native categories.The diagnostic used normalized mutual information between binary union membership and the ten-way native category label.
Diagnostic limitation: Because this value depends only on the balanced two-from-ten construction, it is constant across all two-category unions.The audit therefore found no variation in the diagnostic across candidate pairs.
Diagnostic limitation: The NMI gate provided no filtering power for pair selection and was excluded as evidence for concept selection.The audit specifically concluded that the gate could not distinguish candidate disjunctive concepts behaving like native categories.
Final selection procedure: Final concept tiers were organized using inter-member distance and native separability diagnostics.These diagnostics replaced NMI-based filtering in the paper’s concept-selection procedure.

E Supplementary Figures

Supplementary figures provide diagnostic and control views of the frozen DINOv2 space, lexical mapping, grounding dependencies, and stabilization. They also document small alignment gains and centroid structure in additional experiments.

Figure 6 projects the initial visual categories into the frozen DINOv2-small embedding space using PCA.
Figure 7 shows that single-agent mapping saturates after 10 examples per label in the initial native-category setting.
Figure 8 shows that C1 performance degrades or collapses when labels, embeddings, or image–embedding bindings are disrupted.
Figure 9 provides an entropy curve offering an additional view of lexical stabilization over rounds.
Figures 10 and 11 show small label-wise alignment gains in passive centroid alignment and PCA visualizations of regional-divergence cluster centroids.

F Implementation Parameters

The experiments use a frozen DINOv2-small perceptual substrate with normalized 384-dimensional CLS embeddings and lexical centroids formed from normalized assigned-example means. Evaluation parameters include controlled CIFAR-10 episodes, a 70% consensus threshold, and empirically defined information-theoretic measures.

Perceptual substrate: DINOv2-small produces normalized 384-dimensional CLS-token embeddings, and lexical centroids are normalized means of embeddings assigned to each artificial label.All experiments use the encoder as a frozen perceptual substrate.
Evaluation design: Native-category experiments use CIFAR-10 with controlled seed sets of 5, 10, and 15 examples per label.Concept-carving uses homogeneous 3-way, 5-shot episodes with 30 paired episodes; images are encoded once, cached, and not augmented.
Consensus and OOV control: τ = 0.70 sets consensus convergence, accepting a label when at least 70% of agents assign it to an item.The reported OOV control uses γ = 0.5289 as the optimal threshold, based on maximum association with learned lexical labels.
Information-theoretic measures: Information-theoretic quantities are computed from empirical label distributions, including conditional entropy from per-image agent assignments and normalized mutual information.When H(L) = 0, normalized mutual information is defined as 0 and distinguished from the pre-episode concept-gate diagnostic.

G Dissociation Experiment Regression Supplement

The CIFAR-100 dissociation supplement shows that perceptual distance predicts C1 naming accuracy, whereas semantic distance and superclass membership add no significant explanatory power after perceptual control.

Regression and robustness statistics for the dissociation experiment are reported in Table 9.
Perceptual distance remains a strong predictor of C1 naming accuracy after controlling for semantic distance.
Semantic distance contributes no significant explanatory power after controlling for perceptual distance.

H Reproducibility Statement

The appendix supports reproducibility by logging configurations, data assignments, learner and consensus events, and per-round metrics in structured artifacts. Independent reruns, paired evaluation splits, inspectable experiment graphs, and publicly available code and records make the controlled empirical basis auditable.

Reproducibility infrastructure: The implementation logs configurations, random seeds, image identifiers, concept assignments, learner types, splits, candidate pools, agent assignments, consensus events, and per-round metrics.Results are stored as structured JSON, CSV, and image artifacts; selected experiment graphs are additionally stored in Neo4j for temporal inspection.
Reproducibility infrastructure: Experimental families can be rerun independently, while paired concept-carving episodes use identical support and query splits for all learners.The paired design supports paired statistical testing and reduces variance from split differences.
Scope and access: Code, experiment ledgers, figure-generation scripts, and the internal pre-registration record are publicly available in the project repository.The appendix is intended to make the empirical basis inspectable without extending claims beyond the controlled setting reported in the main text.

I Pre-registration Record for the Dissociation Experiment

The dissociation experiment used decision rules fixed before Phase 3 data collection, and the observed results satisfied all four criteria for the pre-registered “perception governs” outcome. A documented VIF calculation error and a minor omitted sanity check were reported as deviations.

Preregistration: Decision rules were fixed before Phase 3 episode data collection and documented in a repository preregistration dated 2026-06-08.The authors use “pre-registered” internally because the rules were written before dissociation analysis, not externally registered.
Post-hoc correction: VIF = 9.69 arose from computing the variance inflation factor without the intercept column, creating a nominal Rule B conflict that was corrected after internal consistency checking.The intercept omission was identified as a known statsmodels artifact, and the preregistration specified no precedence between conflicting rules.
Specification deviation: The run report omitted the backward-compatibility sanity check comparing original exp 007 tier pairs with CIFAR-100 distance matrices.This was documented as a minor deviation from the registered specification’s Section 5.3.