PaperlayerView exact revision on arXiv

Source-linked AI summary

Trust Mechanisms for Offline Generative AI at the Edge: A Connectivity-Regime Framework and an Empirical Study on a Raspberry Pi 5

Georgios Feretzakis, Alexandros Papaspyridis

arXiv:2610.33732v1cs.CRcs.AIcs.LG

TL;DR

Many IoT deployments have intermittent or absent connectivity, while evidence about trust mechanisms and their costs on isolated edge devices remains limited. This paper proposes a connectivity-regime framework, trust-mechanism taxonomy, and signed knowledge packs, then evaluates them on an isolated Raspberry Pi 5. Its findings show uneven protection, including that a validly signed poisoned pack can pass checks and subvert answers.

  • Problem

    Trust mechanisms for RAG and their behavior and costs remain largely unmeasured on CPU-only edge devices under air-gapped conditions.

    Source: §1 · p. 2

  • Method

    The paper proposes a C0–C4 connectivity framework, a taxonomy of inference-time trust mechanisms, and signed knowledge packs, and evaluates them on a network-isolated Raspberry Pi 5.

    Source: §1 · p. 2

  • Results

    A validly signed poisoned pack passes every pack check and eliminates correct answers on targeted questions; signature-and-hash checks alone accept six of nine tested update attacks rejected by layered verification.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The evaluation indicates that pack authenticity and integrity checks do not establish content trust, so signed packs can still carry poisoning that subverts answers.

    Source: §1 · p. 2

    The network-isolation experiment used a namespace rather than physical separation, so it did not exercise the threat model of a physical air gap.

    Source: §6.3 · p. 15

Abstract

from arXiv · show

Generative AI and large language models (LLMs) are moving into the Internet of Things, yet most deployments assume cloud connectivity, and many IoT settings have intermittent, restricted, or no connectivity. We propose a connectivity-regime framework (C0-C4), a taxonomy of inference-time trust mechanisms, and signed, versioned knowledge packs as the update channel for disconnected regimes, and evaluate them on a Raspberry Pi 5 running fully local retrieval-augmented generation under network isolation, with three Qwen2.5 model sizes, three context sizes, and 300 SQuAD 2.0 questions. An empirically calibrated retrieval-score gate detects knowledge gaps almost perfectly (AUROC 0.97) but in-topic unanswerable questions poorly (0.68). A deterministic output validator converts malformed outputs into explicit abstentions with no observed loss of correct answers, while the NLI-based verifier we tested adds little; for the 7B model, 96% of substantive answers to unanswerable-labelled questions match normalized spans of the retrieved passages. A 3B model achieves the same gold-answer containment rate as a 7B model at less than half the latency and estimated energy, although the 7B model far more often exceeds a token-F1 threshold of 0.5; five retrieved passages lower correctness from 75% to 52% compared with three. Verifying a signed pack takes about 4.5 ms versus about 29 min for embedding its passages on the device; signature and hash checks alone accept six of nine tested update attacks that layered verification rejects. Yet a validly signed poisoned pack passes every pack check: on 30 previously correct questions, correct answers drop to zero and 14 return the injected answer. None of the mechanisms we evaluated establishes the truth of the evidence. We release code and data.

1 Introduction

Offline IoT deployments expose a gap in trust research: local RAG is feasible, but trust mechanisms and their costs remain largely unmeasured on isolated edge hardware. The paper combines a connectivity framework, trust-mechanism taxonomy, and Raspberry Pi 5 evaluation, finding uneven safeguards and a successful validly signed poisoning attack.

Motivation

  • Offline deployments matter because users may be unable to cross-check fluent but unsupported LLM content in settings with intermittent, restricted, or absent connectivity.Examples include maritime, remote-infrastructure, disaster-response, industrial, and defense settings.Source: §1 · p. 1

Research gap

  • Prior work measures edge RAG efficiency and retrieval quality, but joint trust-mechanism behavior and costs remain largely unmeasured on CPU-only edge hardware under air-gapped conditions.Local and offline RAG itself is not new.Source: §1 · p. 2

Contributions

  • The paper proposes a C0–C4 framework that treats disconnection as a spectrum and traces consequences for inference, updates, federation, threats, and governance evidence.It also presents a taxonomy of inference-time trust mechanisms and motivates signed, versioned knowledge packs as the update channel for disconnected regimes.Source: §1 · p. 2

Empirical evaluation

  • A validly signed poisoned pack passes every pack check and eliminates correct answers on targeted questions, while most injected answers also pass tested score gating and post-hoc checks.The evaluation also found that signature-and-hash checks alone accepted six of nine update attacks rejected by layered verification.Source: §1 · p. 2
  • On a network-isolated Raspberry Pi 5, the study evaluates five configurations, three model sizes, three context sizes, 1,500 generations over 300 questions, and 30 generations on a poisoned pack.A deterministic validator converts malformed outputs into explicit abstentions with no observed loss of correct answers; the 3B model matches the 7B model's gold-answer containment rate at less than half the latency and energy.Source: §1 · p. 2

Contributions

  • The paper contributes design guidelines for trustworthy generative edge systems and releases code and data.Source: §1 · p. 2

2 Background and Related Work

Related work establishes that local inference, retrieval augmentation, selective answering, federation, and security each have relevant prior art. The paper positions its contribution as combining connectivity regimes with empirical trust-mechanism and signed-knowledge activation measurements under network isolation.

2.1 Local LLM Inference at the Edge

  • Edge-model and offline-RAG studies measure speed, memory, and retrieval quality, but not grounding, abstention, or trust-mechanism costs.Prior work includes edge benchmarks, memory-budgeted RAG, Bluetooth-mesh offline RAG, and offline-first architectures.Source: §2.1 · p. 3

2.2 Retrieval Augmentation, Hallucination, and Abstention

  • RAG reduces hallucination without eliminating it, and prior selective-answering and unanswerability studies do not evaluate these mechanisms jointly on edge hardware under disconnection.UAEval4RAG reports that rejection often stems from retrieval failure rather than recognizing that a request should not be answered.Source: §2.2 · p. 3

2.3 Federation and Knowledge Distribution

  • Federated learning and federated RAG distribute parameters or knowledge, respectively, but the cited federation approaches presuppose recurring collaboration.Source: §2.3 · p. 3

2.4 Security and Governance of Local Generative AI

  • Local deployment removes the external inference provider from the data flow but retains risks including prompt injection, corpus poisoning, update attacks, and privacy threats.Air-gapped settings shift attention toward covert channels and removable media, while secure-update frameworks address rollback and freeze attacks.Source: §2.4 · p. 4

2.5 Positioning

  • To the authors’ knowledge, prior work does not combine a connectivity-regime framework with edge measurements of trust mechanisms, their latency and energy costs, and signed knowledge activation under network isolation.Source: §2.5 · p. 4

3 A Connectivity-Regime Framework

The framework treats connectivity as an architectural dimension that constrains feasible mechanisms and changes threat and governance priorities. It pairs inference-time safeguards with signed knowledge packs and local governance records for disconnected deployments.

3.1 Connectivity Regimes C0–C4

  • As connectivity decreases, gradient-based federation loses viability and update-channel, threat, and governance priorities shift toward knowledge artifacts, supply-chain risks, provenance, and local auditability.The framework spans C0 cloud-connected to C4 air-gapped deployment; its regimes are a proposal, not an established standard, and are not a strict connectivity scale.Source: §3.1 · pp. 4–5, Figure 1 · p. 5

3.2 Inference-Time Trust Mechanisms

  • Inference-time mechanisms address different failure modes and are complements, not substitutes; generator-independent signals provide assurance distinct from generator self-assessment.The retrieval-score gate can withhold generation for low query-to-passage similarity, while deterministic validation rejects malformed responses such as citation markers without answer text.Source: §3.2 · pp. 5–6

3.3 Knowledge Packs as the Update Channel for Disconnected Regimes

  • Signed, versioned knowledge packs offer attributable evidence and cryptographic activation checks, but distribute knowledge rather than skills and still require freshness and content-trust safeguards.A signature establishes publisher authenticity and byte integrity, not freshness or freedom from poisoned passages.Source: §3.3 · p. 6

3.4 Reference Architecture and Governance Evidence

  • The C3–C4 reference architecture logs confidence signals, sources, and routing decisions locally, supporting governance evidence without provider telemetry but not establishing legal compliance.Its records also do not capture parametric knowledge in model weights.Source: §3.4 · p. 7

4 Experimental Methodology

The study evaluates a fully local RAG pipeline on a Raspberry Pi 5 under network isolation, then tests layered inference-time trust mechanisms and signed-pack update checks. Its methods also define measurement procedures and probe a validly signed poisoned pack.

4 Experimental Methodology

  • The proof-of-concept ran on a Raspberry Pi 5 inside a loopback-only network namespace, approximating C4 but not an operational air gap’s physical-security threat model.The study is an illustration on one device and one dataset, not a benchmark.Source: §4 · p. 7
  • The pipeline used quantized Qwen2.5-Instruct models at 1.5B, 3B, and 7B with k = 3, plus 3B runs at k = 1 and k = 5.The corpus contained 950 passages from 28 SQuAD 2.0 development articles; seven held-out articles supplied out-of-corpus questions, and labels were not re-annotated against the full corpus.Source: §4 · pp. 7–8
  • Trust policies layered generator self-abstention, a validator for citation-only outputs, a top-1 cosine retrieval-score gate, and a post-hoc NLI entailment verifier.The gate threshold was calibrated on answerable queries to target at least 90% retention; this guarantee concerns retention, not correctness or selective risk.Source: §4 · pp. 8–9
  • Correctness used normalized gold-answer containment and token-F1 ≥0.5; risk was the fraction of answered queries that were incorrect.End-to-end latency combined retrieval and generation; energy was estimated from mean generation power and latency, without idle subtraction.Source: §4 · p. 9
  • Update tests compared signature-and-hash checks with layered verification enforcing the manifest-declared non-revoked key, increasing sequence number, expiry, and embedding compatibility.The tests included a valid newer pack, nine attack variants, and a poisoned pack; the poisoning test added 150 incorrect passages to a validly signed pack and reran 30 previously correct questions.Source: §4 · pp. 9–10

5 Results

Across the tested edge configurations, safeguards address different failure modes but leave in-topic unanswerability and validly signed poisoned content unresolved. The results also expose trade-offs among answer quality, latency, energy, and update verification.

5.1 RQ1: Retrieval-Score Gating

  • AUROC was 0.97 [0.94, 0.99] for separating answerable from out-of-corpus queries, but 0.68 [0.60, 0.76] for in-topic unanswerable queries.The out-of-corpus separation reflects one article split and is conditional on it.Source: §5.1 · p. 10, Figure 3 · p. 11, §5.4 · p. 13

5.2 RQ2: Layered Trust Mechanisms

  • The validator converted malformed outputs into abstentions with no observed correct-answer loss; adding the score gate cost 2–8 percentage points of correct answers.Even the strongest tested layered policy declined only 84% of in-topic unanswerable questions.Source: §5.2 · p. 11
  • The entailment verifier separated correct from incorrect answers weakly (AUROC 0.50–0.64); 96% of 7B substantive answers to in-topic unanswerables matched retrieved-passage spans.The verifier's weakness was confounded by rewriting and model size, and span matching did not distinguish these answers from correct ones.Source: §5.2 · pp. 11–12, Table 5 · p. 12

5.3 RQ3: Model Size and Context Size

  • For the 3B and 7B models, lenient answer correctness was 75% for each, while the 7B model reached 74% versus 31% on the strict criterion at 2.3 times the latency and energy.Equal point estimates on 100 questions did not establish equivalence.Source: §5.3 · p. 12
  • At k = 5, 3B correctness fell to 52% from 75% at k = 3 despite recall remaining 98%; median latency rose from 22 s to 36 s.The study says distraction from additional passages plausibly explains falling precision, but does not isolate that cause.Source: §5.3 · pp. 12–13
  • Prompt processing accounted for 89–95% of model time at k ≥ 3, while mean board power stayed between 7.6 and 8.0 W, so energy per query scaled with latency.Query embedding and retrieval took a median of 0.72 s.Source: §5.3 · p. 13

5.4 RQ4: Knowledge-Pack Activation

  • Signature-and-hash checks accepted six of nine tested update attacks, whereas layered verification rejected all nine; verification took a median 4.4 ms.The layered verifier's measured median on the valid pack was 4.55 ms, compared with 4.53 ms for the baseline.Source: §5.4 · p. 13, Table 6 · p. 14

5.5 RQ5: Validly Signed Poisoning

  • Correct answers fell from 30 of 30 to 0 of 30 on poisoned-pack targets; 14 received the injected answer, and none of the evaluated mechanisms established evidence truth.The poisoned passages were retrieved for all 30 targets; score gating and post-hoc checks offered little protection.Source: §5.5 · pp. 13–14

6 Discussion

The discussion translates the study into deployment guidance while identifying governance uses, validity boundaries, and unresolved trust problems. Its recommendations distinguish integrity checks from the separate work of establishing content trust.

6.1 Design Guidelines

  • At C3–C4, use signed knowledge packs with activation checks for version state and expiry, while establishing content trust through publisher-side curation and testing.The authors also recommend choosing the connectivity regime first because it determines admissible update and assurance mechanisms.Source: §6.1 · pp. 14–15
  • Tune model size and retrieved context to the quality, latency, and energy budget: the 3B model matched 7B gold-answer containment at less than half its latency and estimated energy, while five passages reduced correctness versus three.The optimum is configuration-specific, and maximizing retrieved passages is not recommended.Source: §6.1 · p. 14
  • A deterministic output validator converted malformed, citation-only outputs into explicit abstentions with negligible overhead and no observed loss of correct answers.The authors note this matters most for small models.Source: §6.1 · p. 14
  • Use score gating for knowledge gaps, but do not expect support checks to resolve in-topic unanswerability; the authors recommend answerability-aware verification or human review for high-stakes use.The gate detects out-of-corpus queries almost perfectly, whereas tested checks improved only modestly on in-topic unanswerable questions.Source: §6.1 · pp. 14–15

6.2 Implications for Governance Evidence

  • Per-query records of pack verification, retrieval, validator and gate decisions, verifier scores, and cited passages can reconstruct what knowledge was available and why the system answered or abstained.The authors connect these records to applicable AI Act record-keeping and human-oversight obligations and NIST AI RMF functions, while noting local deployment alone does not ensure GDPR data minimization.Source: §6.2 · p. 15

6.3 Threats to Validity

  • Validity is bounded by sequential runs with cache effects, estimated rather than externally measured energy, one device and model family, clustered questions from seven held-out articles, and a simulated rather than physical air gap.The study also used an on-device signing key, unlike the off-device publisher expected in a real C3–C4 deployment.Source: §6.3 · p. 15

6.4 Open Problems

  • Open problems include disconnected federation, human reliance, integrated benchmarks, multilingual evaluation, freshness under disconnection, and a RAG-specific cross-air-gap update profile.The paper notes no widely adopted benchmark jointly measures quality, abstention, latency, energy, and trust under disconnected conditions.Source: §6.4 · pp. 15–16
  • Retrieval-side calibration cannot detect in-topic unanswerability, lexical support can accept incorrect answers matching evidence spans, and the tested entailment verifier separated these cases only weakly; signed packs can also carry poisoned content through every check.The authors identify richer calibration, answerability-aware verification, and publisher-side curation, provenance, and adversarial testing as open needs.Source: §6.4 · pp. 15–16

7 Conclusion

The paper proposes a connectivity-regime framework, inference-time trust mechanisms, and signed knowledge packs, then evaluates them on a network-isolated Raspberry Pi 5. The study finds useful but bounded trust controls: several mechanisms address format or integrity, while none establishes whether evidence is true.

7 Conclusion

  • The paper combines a C0–C4 connectivity-regime framework, a taxonomy of inference-time trust mechanisms, and signed knowledge packs with a network-isolated Raspberry Pi 5 evaluation.The evaluation found score gating strong for knowledge gaps but poor for in-topic unanswerable questions, while deterministic validation turned malformed outputs into explicit abstentions.Source: §7 · p. 16
  • A 3B model matched the 7B model’s gold-answer containment rate at less than half the cost, but more retrieved context lowered answer quality and a validly signed poisoned pack eliminated correct answers on targeted questions.Layered verification stopped tested update attacks missed by signature and hash checks, but did not stop the poisoned pack.Source: §7 · p. 16
  • None of the evaluated mechanisms establishes evidence truth, and the tested support checks did not reliably capture answerability.The conclusion frames the remaining trust gaps as evidence truth and answerability.Source: §7 · p. 16

Statements and Declarations

The declarations report the authors’ use of writing and AI tools, funding and competing-interest statements, and research materials and contributions.

Statements and Declarations

  • The authors used generative AI assistants for candidate literature, bibliographic checks, and experiment and analysis scripts, but not to generate the framework, findings, or conclusions; authors verified references and reviewed code and results.Grammarly and Overleaf were used for language editing and manuscript preparation.Source: Statements and Declarations · p. 17
  • The paper reports no preparation funding and discloses the authors’ affiliation with securagen.ai P.C., while stating that the work was prepared independently of a specific product.The authors report no other relevant financial or non-financial interests.Source: Statements and Declarations · p. 17
  • Ethics approval, consent, and materials availability are reported as not applicable; the authors provide data and code availability information and describe their contributions.The data and code are linked to the same Zenodo DOI.Source: Statements and Declarations · p. 17

Related papers

Loading 2610.33732v1…