PaperlayerView exact revision on arXiv

Source-linked AI summary

SwarmReconGuard: Black-Box Detection of Distributed Collective Reconnaissance by Individually Benign-Looking Agent Populations

Vahid Tavakkoli, Kabeh Mohsenzadegan, Kyandoghere Kyamakya

arXiv:2610.09138v1cs.CRcs.AIcs.MA

TL;DR

Individually valid, low-rate requests can collectively acquire broad system knowledge while hiding coordination from a service-boundary defender. SwarmReconGuard formalizes this as DCR and benchmarks black-box detectors; known-policy detection can be excellent, but unseen-policy performance falls sharply.

  • Problem

    The paper asks whether a service can detect coordinated information acquisition when identities stay within ordinary request budgets and internal coordination is hidden.

    Source: §1 · p. 1

  • Method

    SwarmReconGuard formalizes DCR and evaluates boundary-only detection in a reproducible benchmark using semantic coverage and synthetic weakness discovery as information-acquisition proxies.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Gaussian LLR achieves 100% known-policy detection with zero observed false positives, but detection falls to 3% on unseen policies.

    Source: §8.1 · p. 13, §12 · p. 17

  • Takeaways & Limitations

    Strong separation of known attacks does not establish robust DCR defense, and sequential evidence accumulation improves unseen-policy recall without solving generalization.

    Source: §1 · p. 2, §8.1 · p. 13, §12 · p. 17

    One synthetic municipal-style API cannot establish performance across production services; cross-service calibration and evaluation should precede deployment claims.

    Source: §9 · p. 15

Abstract

from arXiv · show

Autonomous and agentic clients can distribute reconnaissance across many identities so that each request remains valid, low-rate, and benign-looking while the population collectively acquires broad system knowledge. We formalize this threat as Distributed Collective Reconnaissance (DCR) and present SwarmReconGuard, a reproducible black-box benchmark in which the defender observes only service-boundary telemetry. The Docker-isolated study evaluates 11 benign and attack behaviors across 10-10,000 virtual identities, comprising 440 test runs and 3,666,300 requests, with complete telemetry integrity. We compare semantic, Gaussian, conditional, graph, kernel, hybrid, and CUSUM-based detectors. Gaussian likelihood-ratio detection achieves 100$\%$ detection with 0$\%$ observed false positives on known attacks but only 3$\%$ on unseen policies. CUSUM yields 36.1$\%$ overall detection at 1.25$\%$ false positives, while hybrid CUSUM reaches 85.7$\%$ detection with 0$\%$ observed false positives at 10,000 identities. Results expose a major policy-generalization gap and motivate exposure-aware, scale-aware defenses.

1 Introduction

Distributed Collective Reconnaissance (DCR) describes individually valid, low-rate actions that collectively acquire harmful knowledge while coordination remains hidden. SwarmReconGuard formalizes and benchmarks boundary-only detection, with results showing a sharp gap between known- and unseen-policy performance.

1 Introduction

  • DCR is collective information acquisition through individually plausible actions, with hidden coordination and service availability rather than disruption as the objective.Unlike a traditional scan by one source, attribution can be distributed across identities, while the defender sees only boundary events.Source: §1 · pp. 1–2
  • SwarmReconGuard is a reproducible Docker-isolated, black-box framework that observes API-gateway telemetry but not prompts, coordination, swarm membership, or scenario labels.Its scripted virtual identities provide population-scale experimental control without claiming that all 10,000 entities are LLM agents.Source: §1 · p. 2
  • The paper formalizes DCR under per-identity validity, hidden coordination, and service-availability constraints, and states that identical benign and adversarial observable distributions defeat boundary-only intent recovery.The observability limit makes observable distributional differences necessary for behavioral detection.Source: §1 · p. 2
  • The benchmark separates training, calibration, and testing, evaluates ten online detector families and three ablations, and measures detection alongside false positives, delay, and exposure.Its generalization study compares known and unseen policies across population scales.Source: §1 · p. 2
  • A generalization study across known versus unseen attack policies and population scales finds near-perfect known-policy performance, but unseen-policy recall falls sharply; excellent known-attack separation does not establish robust DCR defense.Source: §1 · p. 2

2 Related Work and Research Gap

Prior work supplies useful intrusion, authorization, and multi-agent perspectives, but the paper identifies a distinct gap: detecting harmful collective information acquisition from boundary telemetry alone. It formalizes the observable setting and the limits of such detection.

2.1 Network Intrusion, Botnet, and DDoS Detection

  • Conventional IDS work centers on flow, host, or identity classification and availability-oriented anomalies, while DCR can preserve throughput and distribute semantic exploration across clients.Reproducible testbeds such as Edge-IIoTset and the SCADA dataset target different outcomes, including packet or flow attack class.Source: §2.1 · pp. 2–3

2.2 Zero Trust, Context, and Boundary Enforcement

  • Zero-trust contextual authorization is complementary because it can add differentiating information when behavioral observation alone is insufficient.SwarmReconGuard measures traffic inference after valid requests are admitted; it does not implement authorization mitigation.Source: §2.2 · p. 3

2.3 Distribution Shift, Graphs, and Explainability

  • Because network behavior may be nonstationary or non-Gaussian, the detector suite pairs interpretable and statistical baselines with temporal graph descriptors and a kernel distribution-shift baseline.The graph descriptors examine cross-identity organization without reconstructing a hidden swarm graph; related foundations include dynamic graph learning, log anomaly detection, and explainable IDS.Source: §2.3 · p. 3

2.4 Agentic and Multi-Agent Cybersecurity

  • Multi-agent risk studies often observe agent frameworks or internal signals, whereas SwarmReconGuard assumes runtime, inter-agent messages, and internal logs are inaccessible.Prior work spans white-box probes [37], controlled risk toolkits, long-horizon interaction [39], and synchronized system logs.Source: §2.4 · pp. 3–4

2.5 Research Gap

  • The research gap combines valid low-budget actions, hidden coordination or complementary exploration, cumulative semantic coverage, and a defender limited to boundary telemetry.The benchmark operationalizes this combination and explicitly measures policy generalization rather than only in-distribution accuracy.Source: §2.5 · p. 4

3.1 Service, Agents, and Observables

  • DCR is modeled as population-level cumulative information acquisition while each member remains locally plausible; semantic coverage and synthetic weakness discovery proxy information gain.Exact mutual information is unavailable in the synthetic benchmark, and the defender’s operational objective balances exposure, false alarms, and service cost.Source: §3.1 · p. 4

3.2 Observability Limit

  • If benign and adversarial distributions over complete observable streams are identical, a boundary-only detector cannot outperform chance.The paper points to observable differences, added context such as authorization, or reduced disclosure as ways to address this limit.Source: §3.2 · p. 5

3.3 Attacker and Defender Assumptions

  • The benchmark distributes valid HTTP GET requests among virtual identities, but excludes host shells, real secrets, external systems, and detector access; production stream segmentation is out of scope.Trial boundaries are supplied by the laboratory, and scenario labels and mixed-swarm membership are unavailable to the defender at inference time.Source: §3.3 · p. 5

4 SwarmReconGuard Architecture

SwarmReconGuard uses an isolated synthetic service and gateway to measure collective information acquisition from boundary telemetry. Its testbed provides three measurable information surfaces and scenarios designed to make benign behavior a meaningful control.

4 SwarmReconGuard Architecture

  • The Docker environment uses a synthetic municipal-style API, telemetry-recording gateway, event stream, online detectors, and evaluator, with protections against public targeting and real exploitation.The network is internal, the target exposes no shell-execution or exploitation path, and the runner rejects non-lab hosts.Source: §4 · p. 5, §4.1 · p. 6

4.1 Synthetic Information Channels

  • Three harmless information surfaces provide measurable ground truth: ticket existence/status structure, permit-prefix aggregate counts, and unnecessary synthetic backend metadata.Coverage measures information acquisition only as a proxy, not as a percentage of real secrets reconstructed.Source: §4.1 · p. 5

4.2 Traffic Regimes

  • The 11 scenarios comprise four benign controls and seven attack policies, including five unseen variants, a marginal-matched control, and a mixed swarm with a hidden 20% attacker subset.Benign exploration and bulk access create hard negatives, while legitimate diagnostic access removes an exclusive diagnostic cue.Source: §4.2 · p. 6

5 Detection Methods

SwarmReconGuard scores common event windows with semantic, distributional, graph, kernel, hybrid, and sequential evidence. Its methods include transparent baselines and independently calibrated alarms.

5 Detection Methods

  • Online detectors score identical non-overlapping windows, publishing windows of up to 64 events or one second and explicitly scoring the terminal tail.The first window contains 16 events where available, so events are not silently discarded.Source: §5 · p. 6

5.1 Transparent Semantic Heuristic

  • The transparent heuristic combines novelty, namespace span, gap uniformity, diagnostic ratio, endpoint-family entropy, and semantic coverage pressure using fixed auditable weights.It is an auditable baseline for the hypothesis that systematic exploration has semantic structure beyond request volume, not a claim of optimal weights.Source: §5.1 · p. 6

5.2 Gaussian and Conditional Distribution Models

  • Gaussian models provide benign out-of-distribution and known-policy likelihood-ratio scores, while conditional models regress transformed features on workload and scale context.Conditional residual models use request and identity counts at both window and cumulative levels.Source: §5.2 · p. 7

5.3 Collective Graph Descriptors

  • A temporal bipartite identity–resource-region graph yields interpretable descriptors of cross-identity novelty, balance, specialization, progression, and connectivity.The detector tests whether complementary exploration is statistically observable without access to a true attacker coordination graph.Source: §5.3 · p. 7

5.4 Kernel Distribution Shift

  • Random Fourier features approximate a Gaussian kernel to compare recent conditional-residual means with a benign reference mean, producing an approximate MMD distribution-shift score.This score is not an exact permutation-test p-value; the conceptual connection is kernel two-sample testing.Source: §5.4 · p. 8

5.5 Hybrid and Sequential Evidence

  • The hybrid averages standardized training-derived signals, while CUSUM accumulates sequential likelihood-ratio evidence with drift allowance κ = 0.5.Three ablations remove graph, likelihood-ratio, or heuristic evidence; the CUSUM applies classical sequential change accumulation to collective semantic evidence.Source: §5.5 · p. 8

5.6 Independent Calibration

  • Model parameters use 72 measured HTTP training streams, while alarm thresholds use 299 independent benign calibration streams at target α = 0.01.The calibration controls one alarm decision per complete stream under exchangeability and comparable horizons, not a universal 1% production FPR at every scale.Source: §5.6 · p. 8

6 Experimental Design

The study tests 11 scenarios across four population scales with independent training, calibration, and test phases, and validates telemetry integrity. It reports online operating-point metrics and complementary offline generalization analyses.

6.1 Study Matrix and Integrity

  • The test matrix spans 11 scenarios, four population sizes from 10 to 10,000, and 10 repetitions, yielding 440 runs.Each identity receives three requests; training and calibration are independent and add 3,066,990 measured reference HTTP requests.Source: §6.1 · p. 8
  • 440/440 runs passed telemetry integrity checks, covering 3,666,300 test requests with zero telemetry failures.A valid run required client, gateway, and detector counts to agree, with no telemetry failure or unrecovered transport error.Source: §6.1 · pp. 8–9

6.2 Metrics

  • Evaluation reports AUROC and average precision for threshold-independent ranking, plus calibrated detection, false-positive rate, precision, F1, and Wilson 95% intervals.Repeated continuous metrics use sample standard deviation and Student-t intervals; Hedges’ g is reported for selected effect-size contrasts.Source: §6.2 · p. 9
  • Exposure at alarm or end counts cumulative exposure at alarm for detected attacks and at run end for missed attacks.Averaging exposure only over detected attacks is selection-biased; event-prefix exposure is distinguished from wall-clock exposure at observer alarm time.Source: §6.2 · p. 9
  • Service measurements include latency, throughput, transport errors, matched-benign SLA deltas, detector processing time, and processing lag.Because the detector observes rather than blocks requests, these quantify benchmark overhead and traffic effects, not mitigation benefit.Source: §6.2 · p. 9

6.3 Offline Generalization Analysis

  • Offline Gaussian baselines use leave-repeat-out, leave-scale-out, and leave-attack-policy-out folds, with an inner benign repetition reserved for threshold calibration.Macro-fold AUROC/AP are primary because independently fitted folds may not share a score scale; these analyses do not establish the online 1% FPR target.Source: §6.3 · p. 9

7 Results

Online methods occupy different precision–recall–exposure trade-offs, and strong known-policy detection does not reliably transfer to unseen policies. Detection also varies by population scale; graph evidence weakens the current hybrid, while offline and service analyses qualify operational interpretation.

7.1 Overall Online Operating Points

  • 36.1% detection and F1 0.527 make CUSUM the strongest overall detector, at 1.25% FPR and 98.1% precision; Gaussian LLR detects 30.7% with zero observed benign alarms.The transparent heuristic achieves AUROC 0.828 with 32.1% detection at 0.625% FPR; hybrid CUSUM is more conservative overall but stronger at the largest scale.Source: §7.1 · p. 9

7.2 Known Versus Unseen Attack Policies

  • 100% known-policy detection with zero observed benign false positives falls to 3.0% detection on unseen policies for Gaussian LLR.The heuristic falls from 93.75% known-policy detection to 7.5% unseen; CUSUM detects 75.0% known and 20.5% unseen, while hybrid CUSUM detects 48.75% and 21.5%, respectively.Source: §7.2 · p. 10, Table 3 · p. 11, Figure 3 · p. 12

7.4 Ablation: Graph Evidence Is Not Yet Helpful

  • 28.9% detection and AUROC 0.842 without graph evidence outperform the pointwise hybrid’s 8.2% detection and AUROC 0.674 with graph evidence.Removing heuristic or LLR evidence collapses performance; the handcrafted graph representation is not sufficiently transferable under hard benign controls.Source: Table 4 · p. 11, §7.4 · p. 11

7.5 Offline Leave-Attack-Out Validation

  • 39.3% detection at 4.46% FPR accompanies hybrid leave-attack-out pooled AUROC 0.841 and macro AUROC 0.846.The offline folds have fewer calibration observations than the online 299-stream calibration, so their operating-point values are not directly comparable.Source: §7.5 · p. 11

7.6 Gaussian Assumption Diagnostics

  • p < 10^-8 for semantic coverage and Mardia p = 5.81 × 10^-92 for skewness reject Gaussian assumptions in the pooled benign feature diagnostics.Conditional mixtures and the kernel baseline address some heterogeneity, but the kernel threshold produces no attack alarms in this study.Source: §7.6 · p. 12

7.7 Service Behavior and Detector Overhead

  • At 10,000 identities, mean p95 latency is 3.06 s and throughput is near 261 requests/s; attack-versus-matched-benign p95 latency differs by about 1.3%.The service is capacity-limited in this laboratory configuration, so these results do not establish production-SLA performance.Source: §7.7 · p. 12
  • 61 µs/event is the detector’s average processing cost at 10,000 identities, versus roughly 178 µs/event at the smallest scale; mean p95 processing lag remains around 5–7 ms.All telemetry integrity checks pass, and no test run has a transport error after bounded retry recovery.Source: §7.7 · pp. 12–13

7.3 Population Scale

  • 85.7% detection with zero observed benign false positives at 10,000 identities marks hybrid CUSUM’s strongest scale result; it detects 31.4% at 1,000 and is ineffective at 10 and 100.CUSUM detects 52.9% at 100, 34.3% at 1,000, and 57.1% at 10,000; scale effects differ by method.Source: §7.3 · p. 11, Figure 4 · p. 13

8 Discussion

The discussion finds strong separation for fitted reconnaissance policies but a sharp drop on unseen policies, while sequential accumulation offers the most promising direction in these results. It argues that exposure and observability boundaries matter alongside detection, and cautions that scripted policies do not establish behavior unique to LLM agents.

8.1 What the Results Establish

  • Population-level semantic organization distinguishes the two fitted reconnaissance policies from four diverse benign controls without relying primarily on per-identity budget violations.This supports the hypothesis that request volume alone does not explain collective behavior.Source: §8.1 · p. 13
  • 100% known-policy detection at zero observed FPR falls to 3% unseen-policy detection for Gaussian LLR, exposing a substantial policy-generalization gap.An adaptive adversary can change how it achieves the same goal.Source: §8.1 · p. 13
  • Sequential accumulation is the most promising direction in these results: CUSUM has the best overall recall and F1, while hybrid CUSUM improves substantially at 10,000 identities.The discussion recommends calibrating sequential detectors across variable stream horizons and using explicit time-to-harm objectives.Source: §8.1 · p. 13
  • Removing graph evidence improves pointwise hybrid AUROC, identifying the current handcrafted graph descriptor set as a limitation.The graph component is a negative ablation result, not a validated component.Source: Figure 5 · p. 14

8.2 Exposure Matters More Than AUROC Alone

  • Exposure-at-alarm-or-end assigns final exposure to missed attacks, avoiding selection bias from reporting exposure only for detected attacks.The discussion proposes evaluating whether alarm time precedes a service-specific harm threshold or knowledge predicate.Source: §8.2 · p. 14

8.3 Implications for API Defense

  • Because boundary-only detection cannot distinguish identical benign and adversarial observable distributions, API defenses should also change available information through controls such as contextual authorization or response minimization.The study measures residual observable signal and the cost of relying on behavioral detection alone.Source: §8.3 · p. 14
  • Measured p95 latency and throughput versus population size show the synthetic service saturating at high scale; security conclusions therefore compare attacks with matched benign load, not production-SLA claims.Source: Figure 6 · p. 15

8.4 Agentic AI Relevance Without Overclaiming

  • The benchmark uses scripted policies for controlled population-scale experiments, so its results concern distributed behavioral policies rather than a property unique to LLM reasoning.Transfer to real constrained LLM agents remains future work.Source: §8.4 · p. 15

9 Threats to Validity and Limitations

The study's evidence is bounded by a single synthetic service, controlled scripted policies, lab-supplied trial boundaries, and synthetic exposure measures. Capacity saturation, the graph ablation, and observational-only detectors further constrain operational and prevention claims.

External validity

  • One synthetic municipal-style API cannot establish general performance across production services; cross-service calibration and evaluation should precede deployment claims.A second independently implemented service is identified as the highest-priority extension.Source: §9 · p. 15

Policy realism

  • Controlled scripted attack and benign policies do not reproduce all properties of real users, bots, or autonomous agents, and matched-swarm construction only approximates marginal matching.The study does not prove equality of complete individual distributions.Source: §9 · p. 15

Boundary segmentation and calibration

  • Production use requires continuous stream segmentation, decay, sessionization, or rolling-horizon alarms because the laboratory supplies trial boundaries.Calibration of maxima assumes comparable monitoring horizons; workload, scale, or seasonal shifts can violate the exchangeability assumption behind the 299-stream rank procedure.Source: §9 · pp. 15–16

Synthetic exposure

  • Semantic coverage counts unique synthetic resource keys, not an information-theoretic estimate of secrets learned by a real adversary.The vulnerability-discovery predicates are harmless benchmark markers.Source: §9 · p. 16

Service capacity

  • The target saturates at 1,000–10,000 identities on the experiment host, so matched-load comparisons do not establish strict SLA preservation.A separate capacity study is required for such claims.Source: §9 · p. 16

Graph method

  • Handcrafted graph evidence does not improve the current hybrid and is a negative ablation, not a validated component.Source: §9 · p. 16

No active mitigation

  • Because detectors only observe and do not alter responses, the study measures detection and exposure rather than prevention benefit or user-friction cost.Source: §9 · p. 16

10 Reproducibility, Safety, and Artifact Availability

The artifact provides reproducible configurations, event traces, detector bundles, and reports, with replay verification that does not reissue requests. The benchmark is isolated from public targets and excludes real exploitation or destructive actions.

Artifact availability

  • The repository provides Docker Compose profiles, source code, deterministic scenario seeds, calibration configuration, statistical reports, raw traces, and fitted detector bundles.The publication profile generates HTML, CSV, JSON/JSONL, compressed traces, integrity metadata, and calibration artifacts.Source: §10 · p. 16

Replay verification

  • Replay re-scores saved events and verifies peak scores, alarm decisions, request counts, and exposure without reissuing HTTP requests.Source: §10 · p. 16

Safety

  • The benchmark runs on an internal Docker network with no public target port, real exploit engine, credential theft, persistence, or destructive actions.The synthetic target contains only harmless proof conditions.Source: §10 · p. 16

11 Future Research

Future work prioritizes cross-service evaluation, detector methods conditioned on scale and load, and smaller-scale tests with constrained LLM or planner agents. It also proposes adaptive attacker–defender evaluation and linking detection to selective disclosure controls.

Cross-service and detector evaluation

  • Evaluate additional API domains with train-on-A/test-on-B transfer, and test scale/load-conditioned mixtures, robust density estimation, and calibrated conformal or sequential alternatives.Source: §11 · p. 16

Agent evaluation

  • Introduce constrained LLM or planner agents at smaller scale while retaining scripted policies for causal and population-scale controls.Source: §11 · p. 16

Adaptive evaluation and controls

  • Formulate an adaptive attacker–defender game in which the swarm maximizes information gain while penalizing detector score, and connect detection to selective disclosure controls.The proposed work would quantify the joint objective.Source: §11 · p. 16

Research target

  • A more meaningful research target is high unseen-policy recall at sub-1% FPR, low exposure-at-alarm-or-end, and negligible incremental SLA impact below service saturation.Source: §11 · p. 16

12 Conclusion

SwarmReconGuard makes collective semantic behavior measurable from boundary telemetry, but its strong detection of known policies does not transfer automatically to unseen attacks. The study supports cautious further research, not a claim of production-ready prevention.

12 Conclusion

  • 100% known-policy detection with zero observed false positives came from Gaussian LLR, but that success did not transfer automatically to unseen attack policies.The study reports 440 measured runs and 3.66 million requests with exact telemetry integrity.Source: §12 · p. 17
  • Sequential evidence accumulation improves unseen-policy recall, especially at the largest population scale, but no evaluated method solves generalization.The current graph descriptor fails its ablation test, and the service saturates at high load.Source: §12 · p. 17

Data and Code Availability

  • The implementation, scenario definitions, Docker configuration, and reproducibility tooling are publicly available for regenerating the experiment.The complete final report bundle is included with the manuscript submission package.Source: Data and Code Availability · p. 17
Loading 2610.09138v1…