PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards High Semantic Fidelity: Hyperdimensional Symbolic Messages in Multi-Agent Communication

Shoucheng Song, Chang Yao, Hao Wu, Youfang Lin, Sheng Han, Kai Lv

arXiv:2610.23092v1cs.MA

TL;DR

Existing numerical and neural messages do not jointly provide robust transmission and recoverable native semantics. Herm introduces structured hyperdimensional symbolic messages and receiver-side conversion mechanisms, and reports robustness to noise and improved readability across benchmarks. Its encoding relies on entity-wise observation partitioning, described as common practice and requiring no manual prior knowledge.

  • Problem

    Numerical messages can suffer semantic distortion under mild interference, while neural messages’ black-box encodings do not support recovery of agents’ native semantics.

    Source: §1 · p. 1

  • Method

    Herm combines entity-attribute-value HSM encoding with receiver-centric ego-binding, similarity-aware fusion, and iterative algebraic decoding.

    Source: §1 · p. 2, §4.2.1 · p. 5

  • Results

    Across Hallway, LBF, SMAC, and SMACv2, Herm is reported to provide robustness against noise and a readability advantage.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper concludes that HSM and Herm address both transmission fidelity and conversion fidelity in multi-agent communication.

    Source: §6 · p. 10

    The encoding uses entity-wise observation partitioning, which the paper describes as common practice and as introducing no manual prior knowledge.

    Source: §4.1.1 · p. 5

Abstract

from arXiv · show

Semantic fidelity is essential for multi-agent communication under non-ideal channels, comprising transmission fidelity and conversion fidelity. Yet existing numerical and neural message formats cannot simultaneously achieve both. To address this, we propose a novel message format-\textbf{H}yperdimensional \textbf{S}ymbolic \textbf{M}essages (HSM) and its corresponding communication framework \textbf{Herm}. For transmission fidelity, Herm suppresses semantic entanglement via an entity-attribute-value structured encoding pipeline. For conversion fidelity, Herm first leverages a receiver-centric ego-binding mechanism to align semantics. Then, a similarity-aware dual-branch fusion module is proposed to prevent critical semantic dilution. To realize explicit message-semantic conversion, we devise an iterative decoding scheme via algebraic inverse operations. Experiments exhibit that Herm outperforms baselines across various noisy scenarios while maintaining superior structural readability.

1 Introduction

Semantic fidelity requires messages to remain stable through noisy channels and to support accurate conversion back to agents’ native semantics. Herm combines structured Hyperdimensional Symbolic Messages (HSM) with receiver-side mechanisms to address both requirements.

1 Introduction

  • Semantic fidelity comprises transmission fidelity, meaning stable semantics under noise or attack, and conversion fidelity, meaning bidirectional recovery between messages and agents’ native semantics.The recovered semantics should be decision-valid rather than dependent on black-box interpretation.Source: §1 · p. 1
  • Numerical messages can be distorted by mild interference, while neural messages’ black-box encoding prevents agents and humans from recovering native semantics.The paper frames this as a gap in achieving transmission and conversion fidelity together.Source: §1 · p. 1
  • Directly applying HDC to MARL can entangle multi-entity information at practical low dimensions and misalign sender-centric messages with receivers’ local decision contexts.Its equal-weight bundling can also dilute decision-valid semantics.Source: §1 · p. 2
  • Herm pairs HSM with an entity-attribute-value encoding pipeline to suppress semantic entanglement and improve transmission fidelity.The framework also uses receiver-centric ego-binding and similarity-aware dual-branch fusion to align semantics and assess message relevance.Source: §1 · p. 2
  • Herm adds iterative decoding through algebraic inverse operations and explores integration with LLMs; experiments across four benchmarks report robustness to noise and a readability advantage.The benchmarks named are Hallway, LBF, SMAC, and SMACv2.Source: §1 · p. 2

2 Related Works

Related work covers methods for robust multi-agent communication and prior hyperdimensional computing in reinforcement learning. The paper distinguishes Herm by using hyperdimensional symbolic messages for both robustness and readability.

2 Related Works

  • Robust-communication methods detect or mitigate malicious messages, restore them, or train against channel adversaries, often adding communication stages and training or deployment costs.Other work studies robustness through graph information bottlenecks or multi-view certification.Source: §2 · pp. 2–3
  • Herm replaces neural messages with hyperdimensional symbolic messages, aiming to combine transmission robustness with conversion readability.Source: §2 · p. 3
  • Prior hyperdimensional reinforcement-learning work spans policy-based and value-based paradigms, robotics, cybersecurity, and contextual bandits, while multi-agent communication remained unexplored.The paper presents its work as the first integration of HDC with multi-agent communication.Source: §2 · p. 3

3 Preliminaries

The preliminaries describe communication-enabled Dec-POMDPs and the hyperdimensional computing operations used to compose message representations and map them to semantics.

3 Preliminaries

  • A Dec-POMDP models multi-agent collaboration using joint states, actions, observations, rewards, transitions, and a discount factor; communication adds a message space.Its objective is to derive a joint policy maximizing expected discounted return.Source: §3 · p. 3
  • HDC represents concepts with high-dimensional random vectors whose quasi-orthogonality supports robust encoding.A vector can represent an attribute, value, or relational structure.Source: §3 · p. 3
  • HDC Bind composes concepts into representations dissimilar to their inputs, while Bundle combines items and retains similarity to each; unbinding and similarity matching map vectors back to semantics.The paper uses HRR as its HDC framework and states compatibility with other frameworks in an appendix.Source: §3 · p. 3

4 Methods

Herm uses hyperdimensional symbolic messages (HSM) and sender- and receiver-side mechanisms to address transmission and conversion fidelity. Its methods structure observations and actions, align incoming messages to receiver context, preserve decision-relevant information during fusion, and decode messages into semantics.

4 Methods

  • Herm pairs training-free hyperdimensional computing with sender-side structured encoding and receiver-side ego-binding, similarity-aware fusion, and iterative decoding.The framework targets semantic entanglement, misalignment, and dilution.Source: §4 · p. 3

4.1 Structured Message Encoding for Transmission Fidelity

  • Entity-wise encoding structures observations and historical actions into HSM, using a modest hyperdimensional space to balance expressiveness, interference resistance, and overhead.The message combines instantaneous environmental perceptions with temporally smoothed actions.Source: §4.1 · p. 4

4.1.1 Perception Encoding via Observation Transformation.

  • Perception encoding represents observations as entity-wise identity, attribute, and value tuples, then binds and bundles them into HSM.Attributes are orthogonalized; continuous values are discretized and cyclically encoded to preserve their inherent correlations.Source: §4.1.1 · pp. 4–5

4.1.2 Behavior Encoding via Action-Sequence Transformation.

  • Action encoding discretizes actions and uses time vectors to represent their temporal dependencies across a sliding window.Because actions are self-related, this encoding omits identity binding and entity-wise bundling.Source: §4.1.2 · p. 5

4.2 Receiver-Centric Fusion for Conversion Fidelity

  • Standard HDC provides static sender-centric information without controlling how receivers interpret it, motivating receiver-centric alignment and preservation.Source: §4.2 · p. 5

4.2.1 Ego-Binding for Semantic Alignment

  • Ego-binding binds each incoming message with the receiver’s own encoding, creating a joint representation that captures second-order semantic correlations while preserving relative message similarity.The preserved similarity topology supports algebraic consistency with subsequent fusion.Source: §4.2.1 · p. 5

4.2.2 Similarity-Aware Message Fusion for Semantic Preservation

  • Dual-branch fusion separates observations from actions, prioritizing dissimilar observations for added perception and similar observations for interpreting teammate behavior.The method uses dissimilarity scores for perception fusion and similarity scores for action fusion.Source: §4.2.2 · pp. 5–6

4.3 Iterative Message Decoding via Similarity

  • Iterative decoding extracts entity attributes through identity and attribute unbinding, then matches the result against a value dictionary; actions are recovered by timestep unbinding and similarity matching.Decoded semantics can also be converted into natural language, providing an explicit bridge between agent messages and LLMs.Source: §4.3 · p. 6

5 Experiments

Experiments evaluate Herm on standard MARL benchmarks under noise-free and noisy conditions, examining collaboration, semantic decoding, communication ablations, parameter choices, and transmission fidelity. Herm outperforms baselines across four benchmarks without noise and maintains robustness across noise settings, while decoded semantics align closely with manually translated observations.

5 Experiments

  • Experiments use SMAC, SMACv2, Lbforaging, and Hallway, with a full communication topology across methods and an attribute dictionary that provides a semantic-conversion prior but no performance gain.Baselines include MAIC, T2MAC, CoDe, and TGCNET, as well as CroMAC and AME-enhanced methods for transmission-fidelity comparisons.Source: §5 · p. 6

5.1 Performance in Noise-Free Environments

  • Herm consistently outperforms all baselines across four benchmarks in noise-free settings.The benchmarks are Hallway, LBF, SMAC, and SMACv2.Source: §5.1 · p. 7, Figure 3 · p. 7

5.2 Performance in Noisy Environments

  • Herm maintains superior robustness across varying random noise in Hallway and LBF, and nearly achieves optimal robustness in adversarial settings, with slight fluctuations under PGD.The adversarial evaluation attributes PGD fluctuations to gradients propagating through the mapping network to original messages.Source: §5.2 · pp. 7–8, Figure 4 · p. 8, Table 1 · p. 8

5.3 Message-Semantics Transformation

  • In LBF, decoded observation semantics show near-perfect alignment with manually translated observation texts, and transformation accuracy improves as hyperdimensional space expands.Structural readability comes from HDC's algebraic properties; manual labels provide an optional human-readable interface.Source: §5.3 · p. 8, Figure 5 · p. 9

5.4 Ablation over Communication Process

  • Ablations show that both observation and action-sequence encoding matter, while removing similarity metrics causes catastrophic degradation and removing ego-binding reduces sample efficiency and performance.Without similarity metrics, redundant messages impede extraction of decision-valid semantics; without ego-binding, semantic misalignment burdens downstream correlation learning.Source: §5.4 · p. 9, Figure 6 · p. 9

5.5 Parameter Analysis

  • Action-sequence length w = 2 limits behavioral understanding; increasing w improves performance, but excessively large w can compromise encoding orthogonality, so the default is w = 4.For message dimension H, 128 suffices for simpler tasks, while 256 has advantages for complex tasks; H = 32 performs poorly in both noiseless and noisy settings.Source: §5.5 · p. 9, Figure 6 · p. 9

5.6 Transmission Fidelity Quantification

  • Under varied noise perturbations, HSM stably maintains similarity relationships between raw observations and transmitted encoded messages, supporting transmission fidelity.The study measures rank-order similarity with Spearman correlation coefficient ρ; values closer to 1 indicate better preservation.Source: §5.6 · p. 10, Table 2 · p. 10

6 Conclusions

The conclusion identifies two remaining design limitations: encoding is decoupled from policy optimization, and per-dimension message encoding lacks flexibility. The appendix describes communication-intensive SMAC tasks and reports configuration choices for Herm.

6 Conclusions

  • Herm’s encoding module is decoupled from policy optimization, and its per-dimension encoding lacks flexibility; joint training and adaptive attribute selection are proposed future directions.The proposed dynamic mechanism would let agents decide what to communicate.Source: §6 · p. 10

A.1 SMAC Configuration

  • SMAC agents must find allies and enemies, then form balanced clusters before combat because solo agents are vulnerable to elimination.Randomized starting positions and terrain obstacles make the communication-intensive tasks suitable for evaluating exploration and coordination.Source: §A.1 · p. 13, Figure 7 · p. 14

A.2 Experiment Parameter

  • The experiment parameter appendix identifies Table 3 as the source of default parameters.The appendix also describes retaining most Py-MARLzoo+ defaults and using P = 10 ablation trials for evaluation.Source: §A.2 · pp. 13–14, Table 3 · p. 14
  • Herm uses H = 256 and w = 3 or 4 for SMAC’s larger entity and attribute complexity.The appendix says hallway tasks use H = 64 and w = 1, while most LBF tasks use H = 64 and w = 1 and more complex 15×15 maps use H = 256.Source: §A.2 · pp. 13–14

A.3 Analysis of Model Size and Efficiency

The appendix examines Herm’s efficiency, robustness, and extensions to other HDC frameworks, numerical messages, and policy-based methods. It reports qualitative improvements in noisy settings and adaptability, while noting trade-offs in decision-making efficiency and conversion accuracy.

A.3 Analysis of Model Size and Efficiency

  • Herm has a minimal model parameter count because it removes neural networks from message encoding and fusion, although its programmatic HDC encoding slightly disadvantages decision-making efficiency.Reusing stored binding outputs of attribute-value pairs reduces encoding time to near baseline levels; Table 5 compares model size and interaction time.Source: §A.3 · p. 14, Table 5 · p. 15

A.4 Additional Comparison on Robustness

  • Herm achieves optimal robust performance in additional tests with many entities, although some MAGI results are missing because it is not open-sourced.Figure 8 presents the additional robustness comparison.Source: §A.4 · p. 15, Figure 8 · p. 15

A.5 Scalability of Hyperdimensional Computing Framework

  • The appendix uses HRR in its experiments and provides supplementary performance comparisons across HDC frameworks, including HRR, FHRR, and VTB.HRR’s commutative binding limits representation of non-commutative relations; FHRR uses frequency-domain operations, while VTB uses a learnable transformation.Source: §A.5 · pp. 15–16, Figure 9 · p. 15

A.6 Extension with Numerical Messages

  • HSM conversion improves MAIC’s noise robustness on hallway, with improvement positively correlated with message dimension H; complex 5z_vs_1ul requires higher dimension for digit conversion.MAIC-E has a slight performance decline without noise but greater stability under noisy conditions; Table 6 presents HSM-enhanced performance.Source: §A.6 · pp. 16–17, Table 6 · p. 16

A.7 Extension with Policy-Based Frameworks

  • Herm adapts to MADDPG and MAPPO in continuous-action MPE experiments and yields improvements for both policy-based methods.The results also support hyperdimensional encoding for continuous observation and action spaces by discretizing continuous values into discrete semantics.Source: §A.7 · p. 17, Figure 10 · p. 17

A.8 Ablation Study on Message Format and Fusion Process.

The ablations attribute the primary performance gains to HSM, while showing that the dual-branch fusion design contributes downstream. The appendix also formalizes HRR operations and their statistical properties.

A.8 Ablation Study on Message Format and Fusion Process.

  • Replacing HSM with neural encoding causes over 20% performance degradation on the 11 × 11 task in both noise-free and noisy settings.The tested variants share the QMIX backbone, message dimension, and network topology; the Neural-Herm variant preserves dual-branch fusion while replacing HSM.Source: §A.8 · p. 17, Table 7 · p. 18
  • Mean fusion and inverse-similarity fusion both reduce performance, with inverse similarity causing the larger degradation.Mean fusion incurs global semantic dilution, while inverse similarity misaligns attention; these comparisons support the dual-branch fusion design.Source: §A.8 · p. 17
  • The ablations conclude that performance gains primarily come from HSM, with fusion serving as an auxiliary downstream component tailored to HSM.The variants also include neural encoding with mean fusion and reversed similarity-based fusion.Source: §A.8 · p. 17

B.1 Preliminaries and Problem Setup

  • The appendix’s HRR proofs define similarity using cosine similarity, which reduces to the dot product for unit-norm vectors, and use the DFT to analyze frequency-domain operations.Parseval’s theorem preserves inner products up to a scaling factor in the frequency domain.Source: §B.1 · p. 18
  • For atomic hypervectors, the appendix assumes independent random components with zero mean and variance 1/D, yielding expected unit norm and quasi-orthogonality.The stated fourth-moment condition is E[v4_i] = 3/D2.Source: §B.1 · p. 18

B.2 Formal Definitions of HRR Operations; B.3.1 Bundling

  • In HRR, bundling aggregates vectors into a normalized composite while preserving similarity to its constituents.For D →∞ and n ≪D, the variance of the composite’s dot product with a constituent is approximately 1.Source: §B.2 · p. 19, §B.3.1 · p. 20

B.2 Formal Definitions of HRR Operations; B.3.2 Binding

  • HRR binding uses circular convolution to associate vectors, producing a composite quasi-orthogonal to its constituents.The stated dot-product expectation is zero and its variance is 2/D as D →∞.Source: §B.2 · p. 19, §B.3.2 · p. 20

B.2 Formal Definitions of HRR Operations

  • HRR unbinding uses involution as an approximate inverse because the exact inverse of circular convolution is numerically unstable.Unbinding is inherently noisy: the bound vector and its involution only approximate the identity impulse, with residual noise determined by deviation of |F(b)k|2 from unity.Source: §B.2 · p. 19

B.3.3 Unbinding

The appendix analyzes HRR unbinding as approximate recovery of a bound vector’s constituent. It reports that the signal-to-noise ratio grows linearly with dimension, supporting asymptotically accurate retrieval.

B.3.3 Unbinding

  • The analysis models Fourier-domain unbinding as F(a′)k = F(a)k|F(b)k|2 and treats similarity as the sample mean of Zk.For real-valued vectors, Hermitian symmetry makes paired frequency components correlated; the derivation accounts for this in the variance of the mean.Source: §B.3.3 · p. 21
  • The recovered vector has expected similarity E[a′ · a] = 1 under the stated independence assumptions for a and b.The frequency-component variance is reported as 3, with symmetry accounted for in the variance of the mean.Source: §B.3.3 · p. 21
  • The appendix defines SNR as squared expected signal divided by noise variance.For unbinding, the reported SNR scaling is O(D).Source: §B.3.3 · pp. 20–21
  • The unbinding signal-to-noise ratio scales as O(D), so increasing dimension suppresses multiplicative noise from the approximate inverse and supports reliable symbol retrieval.The analysis considers a′ = (a ⊗b) ⊛b and states asymptotic retrieval accuracy in high dimensions.Source: §B.3.3 · pp. 20–21

C Conversion Fidelity Analysis on Message Length

The analysis varies message length to assess semantic transmission and decoding accuracy. Larger dimensions reduce decoding errors and semantic discrepancy, with decoding near-perfect at H=512.

C Conversion Fidelity Analysis on Message Length

D Message Similarity Analysis under Noise

In SMAC, observation-encoding similarity tracks scenario-based semantic similarity, and that correspondence remains intact under different noise conditions.

D Message Similarity Analysis under Noise

  • The rightmost stalker’s observation encoding is most similar to the middle stalker’s, matching their shared race and similar relative positions; this correspondence persists under noise.Figure 14 distinguishes stalkers with triangles and zealots with squares; its b–d subfigures show encoding similarity matrices under different noise conditions.Source: D Message Similarity Analysis under… · p. 24, Figure 14 · p. 24

E Similarity-Preservation Verification for Ego-Binding

Ego-binding preserves agents’ relative message-similarity relations despite raising absolute similarity values. The accompanying sections describe environment-specific hyperdimensional encoding and an HSM-based collaboration pipeline connecting agents, symbolic semantics, natural language, and LLMs.

E Similarity-Preservation Verification for Ego-Binding

F.1 SMAC / SMACv2

  • SMAC and SMACv2 share fundamental observation and action structures, while the attribute design is for readability and agents encode raw observations dimension-wise.The encoding assumes attribute independence and stochastically constructs the dictionary.Source: §F.1 · p. 25, §F.1.1 · p. 25

F.1.1 Attribute Dictionary; F.1.2 Value Dictionary

  • The SMAC attribute dictionary lists executable actions including NOTHING, movement, ATTACK, and HEAL; value types include independent or associated, and discrete or continuous values.Continuous variables are interval-discretized and typically encoded with 10 bins.Source: Table 8 · p. 25, §F.1.1 · p. 25, §F.1.2 · p. 26

F.1.3 Encoding Process

  • SMAC encodes self, ally, and enemy features separately before bundling them; entity identity is bound to attributes, with visibility determining ally/enemy feature encoding.Self features include identity, HP, and type; visible entities additionally include position, while terminated agents return a zero vector.Source: §F.1.3 · p. 26

F.2 Lbforaging (LBF)

  • LBF reuses the described encoding procedure and specifies its own attribute and value dictionaries, with executable actions including CAPTURE.Its dictionary is presented in Table 9.Source: §F.2 · p. 26, Table 9 · p. 26

F.3 Hallway

  • Hallway likewise reuses the earlier encoding procedure and provides environment-specific attribute and value dictionaries; its executable actions are STAY, MOVE_LEFT, and MOVE_RIGHT.The dictionary is presented in Table 10.Source: §F.3 · p. 26, Table 10 · p. 26

G Collaboration with Large Language Models

  • HSM links message encoding, symbolic semantics, and natural language, using symbolic semantics as a bridge and avoiding reliance on foundation-model embedding spaces.The framework presents natural language as interpretable by human operators and LLMs.Source: G Collaboration with Large Language… · p. 27

G.1 Message Transformation; G.2 Decision Making Process

  • Decoded HSMs become entity-wise attribute-value pairings that can be aggregated into natural language; A-Agents exchange HSMs, while an L-Agent receives reports and returns encoded messages.The L-Agent is LLM-controlled, whereas the other agents operate under policies; the L-Agent selects executable actions guided by LLM reasoning.Source: §G.1 · p. 27, §G.2 · p. 27

G.3 Experiments

This section evaluates Herm for hybrid decision-making on SMAC, with one team member replaced by an LLM-controlled agent at test time, and reports results across three LLMs and varying hyperdimensional-space sizes. It also includes algorithm pseudocode and checklist responses, including disclosures about LLM use and items marked not applicable.

G.3 Experiments

  • LLMs were excluded from training because of real-time interaction overhead; testing replaced one of the N−1 algorithm-controlled agents with an LLM-controlled agent.The benchmark comprises N−1 A-Agents and one L-Agent.Source: §G.3 · p. 28
  • The illustrated LLM reasons from decoded, processed natural-language messages and makes corresponding decisions.Figure 18 presents an example of the reasoning process during decision-making.Source: §G.3 · p. 29, Figure 18 · p. 29
  • Herm maintained performance with three LLMs in hybrid decision-making, supporting message generalizability; performance varied by LLM and rose with hyperdimensional-space size, with gains slowing at high dimensions.The authors attribute LLM-to-LLM differences to intrinsic decision-making competence, and the slowing trend to other message sources correcting minor transformation errors.Source: Table 11 · p. 29, §G.3 · p. 29

H Algorithm Pseudocode

  • Herm’s overall training architecture uses QMIX, while its pseudocode separately describes training and single-agent decision-making.The decision-making algorithm initializes local Q-networks, observation and action projections, a trajectory encoder, and attribute, value, action, and time dictionaries.Source: H Algorithm Pseudocode · pp. 29, 31

NeurIPS Paper Checklist

  • The checklist answers Yes for experimental reproducibility, setting details, statistical-significance reporting, compute-resource information, and societal-impact discussion; safeguards for high-risk releases are marked N/A.The cited checklist answers are author responses, not independent verification of those disclosures.Source: NeurIPS Paper Checklist · pp. 33–36
  • The authors mark new-asset documentation as not applicable, stating that the paper does not involve new assets.The checklist also marks crowdsourcing, human-subject research, and related IRB reporting as not applicable because they do not involve such research.Source: NeurIPS Paper Checklist · p. 37
  • The checklist reports LLM use and refers readers to Appendix H for details.Source: NeurIPS Paper Checklist · p. 38
Loading 2610.23092v1…