PaperlayerView exact revision on arXiv

Source-linked AI summary

Decentralizing AI Memory: SHIMI, a Semantic Hierarchical Memory Index for Scalable Agent Reasoning

Tooraj Helmi

arXiv:2504.06135v1cs.AIcs.MA

TL;DR

Existing vector-based retrieval has limitations in abstraction, semantic precision, interpretability, and decentralized operation. SHIMI uses hierarchical semantic memory and decentralized synchronization; it outperforms flat vector approaches across multiple dimensions, including a reported reduction in synchronization data transferred.

  • Problem

    Flat vector retrieval has limited abstraction, can return lexically similar but semantically irrelevant items, offers limited transparency, and often depends on centralized infrastructure.

    Source: §2.1 · p. 3

  • Method

    SHIMI organizes knowledge in a semantic hierarchy for retrieval and synchronizes agent-local trees using a protocol based on Merkle-DAG summaries, Bloom filters, and CRDT-style merging.

    Source: §1 · p. 2

  • Results

    SHIMI outperforms flat vector approaches across retrieval accuracy, traversal efficiency, synchronization cost, and scalability; data transferred during sync events is reduced by over 90% versus naive replication.

    Source: §4.5 · p. 12, §6 · p. 14

  • Takeaways & Limitations

    SHIMI is positioned as a memory framework suited to agentic infrastructures with distributed memory, federated ownership, and continually updated knowledge.

    Source: §4.5 · p. 12

    Evaluation uses controlled simulations and synthetic or curated descriptions and queries, which may not capture real-world network conditions or deployment noise.

    Source: §5.5 · p. 14

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) and vector-based search have become foundational tools for memory in AI systems, yet they struggle with abstraction, scalability, and semantic precision - especially in decentralized environments. We present SHIMI (Semantic Hierarchical Memory Index), a unified architecture that models knowledge as a dynamically structured hierarchy of concepts, enabling agents to retrieve information based on meaning rather than surface similarity. SHIMI organizes memory into layered semantic nodes and supports top-down traversal from abstract intent to specific entities, offering more precise and explainable retrieval. Critically, SHIMI is natively designed for decentralized ecosystems, where agents maintain local memory trees and synchronize them asynchronously across networks. We introduce a lightweight sync protocol that leverages Merkle-DAG summaries, Bloom filters, and CRDT-style conflict resolution to enable partial synchronization with minimal overhead. Through benchmark experiments and use cases involving decentralized agent collaboration, we demonstrate SHIMI's advantages in retrieval accuracy, semantic fidelity, and scalability - positioning it as a core infrastructure layer for decentralized cognitive systems.

1 Introduction

Decentralized agents need structured, interpretable memory, while current retrieval is flat, centralized, and hard to justify. SHIMI combines hierarchical semantic retrieval with decentralized synchronization.

1 Introduction

  • Decentralized agents need structured, interpretable memory, but existing retrieval methods rely on flat embeddings, centralized indices, and results that are difficult to justify.These limitations matter especially for autonomous systems and interactions among multiple peers.Source: §1 · pp. 1–2
  • SHIMI organizes memory as a dynamic tree of semantic concepts and retrieves knowledge by descending from abstract goals to specific entities.This provides meaning-based matching and interpretable reasoning paths.Source: §1 · p. 2
  • Each agent maintains and updates a local semantic tree, while Merkle-DAG summaries, Bloom filters, and CRDT-style merging support partial, low-bandwidth synchronization.The protocol is designed to preserve convergence and consistency across asynchronous networks.Source: §1 · p. 2
  • In a simulated multi-agent task-assignment scenario, SHIMI improves retrieval accuracy and interpretability while significantly reducing synchronization cost.The evaluation involves multiple agents.Source: §1 · p. 2
  • The evaluation compares SHIMI with vector-based baselines across accuracy, traversal efficiency, and sync overhead, and covers agent matching, decentralized indexing, and synchronization use cases.Source: §1 · p. 2

2 Background and Related Work

The section reviews retrieval and memory approaches, their limitations in decentralized settings, and the technologies informing SHIMI’s design. SHIMI is positioned as a unified retrieval architecture for semantic abstraction, hierarchical memory, and decentralized synchronization.

2 Background and Related Work

  • The section surveys knowledge-retrieval approaches, their limitations in decentralized environments, and technologies that inform SHIMI’s design.Source: §2 · p. 2

2.1 Retrieval-Augmented Generation and Vector Search

  • Vector retrieval commonly uses dense embeddings and approximate nearest-neighbor search, but flat structure, semantic drift, black-box decisions, and centralized indices limit abstraction, precision, transparency, and decentralized use.RAG retrieves relevant passages as additional language-model context.Source: §2.1 · pp. 2–3

2.2 Semantic, Hierarchical, and Graph-Based Memory Structures

  • Cognitive-inspired hierarchies support abstraction, while ontologies require manual curation; graph systems support relational traversal but face scalability, schema rigidity, centralization, and synchronization gaps.Hierarchical embedding and memory-augmented models typically operate in closed environments with static or preloaded data.Source: §2.2 · p. 3

2.3 Memory in Decentralized Systems

  • Decentralized storage frameworks provide content-addressed persistence rather than meaning-aware retrieval; Merkle-DAGs, Bloom filters, and CRDTs supply primitives for synchronization.These primitives respectively support structural hashing and diffing, compact probabilistic summaries, and convergence without global coordination.Source: §2.3 · pp. 3–4

2.4 Positioning SHIMI

  • SHIMI is presented as unifying semantic abstraction, hierarchical memory, and decentralized synchronization, with agents retrieving independently while the network converges eventually.Source: §2.4 · p. 4

3 Approach

SHIMI represents memory as a dynamic semantic tree and retrieves entities by traversing from abstract roots toward relevant leaves. Its decentralized protocol synchronizes local trees incrementally, while the complexity analysis describes retrieval and synchronization costs under stated structural assumptions.

3.1 Semantic Memory Architecture

  • SHIMI models memory as a rooted semantic tree whose nodes can store entities or abstractions, and insertion matches an entity to a subtree before attaching it or creating an abstraction.The tree uses a maximum branching factor T, maximum abstraction depth L, similarity threshold δ, and compression ratio γ.Source: §3.1 · p. 4

Semantic Retrieval

  • Retrieval descends from roots and expands only nodes whose summaries meet the query-similarity threshold, returning entities associated with matching leaves.Leaf entities may be ranked by frequency, recency, or proximity to the query; retrieval can fall back to embeddings when no semantic path exists.Source: Semantic Retrieval · pp. 5–6

3.2 Decentralized Synchronization

  • Synchronization compares tree hashes, identifies divergent subtrees, filters missing or changed nodes with Bloom filters, and merges conflicts using a CRDT-style function.Only the minimal divergent subtree and related edits are exchanged; semantic summaries favor greater abstraction depth or observed usage, supporting eventual consistency across local asynchronous updates.Source: §3.2 · pp. 6–7

3.3 Complexity Analysis

  • In balanced trees, retrieval latency remains sublinear and bounded, while interpretability improves with shallower trees and well-separated concepts.Retrieval can also be cheaper than insertion because it skips merging and may terminate once relevant leaves are found.Source: §3.3 · p. 8
  • SHIMI’s Merkle root comparison is constant-time; Bloom filters reduce unnecessary transmission, and CRDT-style merges run in bounded time under deterministic node representations, avoiding full replication and supporting bandwidth-efficient convergence in asynchronous networks.Source: §3.3 · p. 9

4 Evaluation

SHIMI is evaluated against flat vector retrieval across semantic retrieval, traversal efficiency, synchronization cost, and scalability. The evaluation reports stronger retrieval and interpretability, lower synchronization bandwidth, and bounded traversal and latency as the system scales.

4.1 Retrieval Accuracy and Precision

  • On 20 semantically non-trivial queries, SHIMI improves raw accuracy and interpretability over the RAG-style embedding baseline.Table 1 presents the retrieval-accuracy comparison; the supplied passages do not provide its cell values.Source: Table 1 · p. 9, §4.1 · p. 9

4.2 Traversal Efficiency

  • SHIMI’s traversal cost grows sublinearly with tree depth, contrasting with RAG’s nearly linear cost.The figure tracks nodes visited per query as tree depth increases.Source: Figure 1 · p. 10, §4.2 · p. 10

4.3 Synchronization Setup and Cost

  • SHIMI partial synchronization reduces bandwidth by more than 90% across node counts compared with full-state replication.The protocol exchanges divergent subtrees identified using Merkle hashes and Bloom filters.Source: Table 2 · p. 10, §4.3 · p. 10
  • As conflict rates rise, SHIMI’s conflict-resolution time scales moderately and remains bounded and efficient in higher-contention scenarios.The figure plots resolution time against conflict rate during synchronization.Source: Figure 2 · p. 11, §4.3 · p. 11

4.4 Scalability

  • With trees grown to 2,000 entities, SHIMI’s query-latency curve remains relatively flat rather than degrading linearly.The evaluation uses entity growth to model expanding agent ecosystems.Source: Figure 3 · p. 11, §4.4 · p. 11

4.5 Discussion

  • SHIMI consistently outperforms flat vector retrieval across multiple performance dimensions, with semantic abstraction supporting disambiguation and interpretable paths.The discussion also characterizes its partial synchronization and hierarchical traversal as suitable for decentralized settings.Source: §4.5 · p. 12
  • Semantic-tree pruning reduces comparisons per query, which the discussion links to faster responses and lower computational overhead than RAG’s global candidate search.The discussion describes this selective retrieval as increasingly relevant as agents or memory entries grow.Source: §4.5 · p. 12

5 Applications

SHIMI is presented as a decentralized memory protocol for applications including agent markets, federated knowledge graphs, autonomous multi-agent systems, and blockchain task orchestration. The section also identifies structural, model-dependence, and evaluation limitations.

5.1 Decentralized Agent Markets

  • Layered concept matching can surface suitable agents despite lexical differences and provide traceable reasoning paths for marketplace auditing and reputation systems.The proposed use includes matching high-level tasks to advertised agent capabilities.Source: §5.1 · p. 13

5.2 Federated Knowledge Graphs

  • CRDT-style conflict resolution and semantic merging support asynchronous partial synchronization of independent ontologies without requiring global ontology agreement.The section says this allows eventual semantic consistency as each node evolves locally.Source: §5.2 · p. 13

5.3 Autonomous Multi-Agent Systems

  • Local memory growth and inter-agent sharing avoid centralized indexing, while partial synchronization supports onboarding or memory reallocation as agents join or leave a subnet.Source: §5.3 · p. 13

5.4 Blockchain-Based Task Orchestration

  • Blockchain task platforms can use SHIMI to index task descriptions, agent bids, and milestone evaluations with structured memory logic.The passage frames this as an alternative to embedding-heavy retrieval that cannot guarantee explainability or deterministic matching.Source: §5.4 · p. 13

5.5 Limitations and Threats to Validity

  • The tree-based design may struggle with concepts requiring polyhierarchical or graph-based representations, where entities belong to overlapping categories.Source: §5.5 · p. 14
  • Language-model-driven generalization, similarity, and merging lack formal guarantees and make some memory transformations difficult to audit or reproduce.Source: §5.5 · p. 14
  • The controlled simulation omits real-world latency variability, partial node failure, and adversarial input; synthetic or curated descriptions may miss deployment noise and inconsistency.Source: §5.5 · p. 14
  • Semantic-depth query design may overfit query types or descriptions, while simulated nodes may underrepresent large-scale heterogeneous networks; the authors say these constraints matter for robust deployment.Performance may differ on surface-level tasks.Source: §5.5 · p. 14

6 Conclusion and Future Work

The conclusion presents SHIMI as a semantic hierarchical memory index for decentralized AI, reporting simulation-based advantages over flat vector approaches. It highlights partial synchronization and explainable access, and outlines directions for broader deployment and more flexible semantic models.

6 Conclusion and Future Work

  • Simulation-based experiments showed SHIMI’s superiority over flat vector approaches across retrieval accuracy, traversal efficiency, synchronization cost, and scalability.Source: §6 · p. 14
  • Partial synchronization enables bandwidth-efficient updates without sacrificing consistency, while the tree structure supports fast and explainable memory access.Source: §6 · p. 14
  • Future work includes real-world deployment tests, non-tree graphs and cyclic ontologies, on-device LLMs for local semantic resolution, and formalized interoperable protocol procedures.The stated aims include reducing network latency and expanding edge applicability through on-device models.Source: §6 · p. 15
  • The authors envision these directions extending SHIMI from a proof-of-concept into infrastructure for distributed AI.Source: §6 · p. 15

Related papers

Loading 2504.06135v1…