Source-linked AI summary
Pattern-Derived Visual Swarm Games: Multi-Scale Drone-Vision States for Interception and Sustainability Audits
Faruk Alpay, Levent Sarioglu
TL;DR
The paper asks whether drone-vision datasets can provide reproducible virtual agent characteristics and strategic visual states beyond object detection. It converts annotation patterns into virtual swarm states, evaluates multi-scale formation and repeated Markov games, and keeps the operational boundary explicit: the artifact studies virtual states, not real drone control.
Problem
The paper examines whether drone-vision datasets can be transformed into reproducible virtual agent characteristics and studied as visual strategic states, rather than only used for object detection.
Method
The paper compresses drone-vision annotations into a Bloom-filter sketch, derives deterministic capability vectors, populates multi-scale formation games, and adds a repeated Markov-game layer with explicit system states.
Results
The benchmark connects computer vision, pattern recognition, and finite/repeated game analysis through virtual swarm states rendered as human-readable images and 32-by-32 rasters.
Takeaways & Limitations
The resulting artifact provides an auditable, reproducible framework for studying dataset-derived visual strategic states across multi-scale finite and repeated games.
Takeaways & Limitations
The model is a controlled benchmark rather than an aerodynamic or operational simulator, uses a nonphysical capability model and weak classifier, and does not claim tactical completeness.
Abstract
from arXiv · showhide
We convert drone-vision annotation streams into virtual swarm-game states without controlling physical drones. VisDrone and UAVSwarm metadata are compressed into a Bloom representation; deterministic probes produce bounded capability vectors, image-space formations, finite zero-sum payoffs, and human-readable visual overlays. The audit scales from $6\times 6$ to $32\times 32$ finite games and adds a repeated Markov layer with stock, fatigue, adaptation, exposure, stress, budget, data-growth, model-improvement, and entropy-budget state variables. Local screen tuning raises robust screen security from $0.526$ to $0.593$, and the $32\times 32$ tuned screen reaches value $0.616$. A field readout audit shows that fixed-pixel rasters do not improve monotonically: $128\times 128$ accuracy is $67.2\%$ and hotspot error is $0.136$. The diagnosed error is shrinking image-plane bandwidth. A finite empirical-risk encoder over scale-normalized Gaussian bandwidths selects a scale-normalized encoder with $λ=1.50$, reaching $77.6\%$ accuracy at $128\times 128$ and reducing joint loss by $0.185$. A server-side audit checks $16{,}777{,}216$ target-localization states, and a 32-round repeated-game audit over $16{,}777{,}216$ trajectories selects a budget-adaptive policy with value $0.461$.
1 Introduction
The paper turns drone-vision annotation patterns into reproducible virtual swarm characteristics and studies them as visual strategic states rather than physical drone controls. It combines deterministic capability generation, multi-scale formation games, tactical overlays, and field-resolution audits.
- Operational boundary: The benchmark is explicitly not a flight-control system, targeting system, deployment planner, or source of real-world action recommendations.Its scope is a reproducible visual abstraction of dataset-derived virtual swarm states.
- Dataset-to-state construction: Dataset-derived patterns become deterministic virtual drone capability vectors through a Bloom-filter pipeline.Seeded probes convert the sketch into bounded agent characteristics without measuring or controlling physical drones.
- Formation games: 6v6-to-32v32 formation games regenerate payoff matrices from dataset-derived vectors and evaluate adversarial interception strategies.Defenders use ring, screen, wedge, or dispersed formations, while attackers use line, column, pincer, or cloud formations.
- Visual readout: Tactical overlays render virtual agents, distances, confidence scores, and predicted leakage on aerial imagery for human or raster inspection.The same visual state supports both person-readable inspection and image-based classification.
- Field-resolution audit: Field-resolution audits test when larger rendered fields improve strategic-regime readout and when they dilute occupancy evidence.The study treats raster resolution as an empirical design choice rather than assuming monotonic improvement.
- Repeated system game: The repeated Markov layer adds stock, fatigue, adaptation, exposure, stress, budget, dataset-growth, model-improvement, and entropy-budget states.This extends the single-frame game to repeated interaction with evolving operational and information variables.
2 Data Sources
The study combines compact local drone-vision sources with larger server-side sources and documents their provenance and reproducibility boundaries. Raw archives remain outside the source package while manifests, download scripts, and reference-to-evidence mappings support expansion and auditing.
- Primary sources: VisDrone and UAVSwarm supply the main aerial annotations, backgrounds, swarm geometry, and tracking material for the local run and server-side expansion.Sheffield supplies additional UAV vision provenance, while Stanford is retained as a future trajectory/background reference.
- Reference sources: Anti-UAV and DUT-Anti-UAV serve as external detection/tracking references rather than payoff sources.The paper also maintains a separate Russian-language comparison track for UAV detection and small flying-object recognition.
- Reproducibility boundary: Large raw video and image archives remain outside the source bundle, which contains source manifests, server download scripts, and ancillary reproducibility files.The local smoke run uses two compact sources, while the server-side expansion downloads fuller material remotely.
- Evidence audit: A reference-to-evidence CSV maps mathematical and dataset citations to implementation hooks and audit verdicts.Examples connect Bloom hashing, concentration bounds, multiplicative weights, visual readouts, and source-ablation evidence to specific artifacts.
3 Dataset Representation
The dataset representation is a deterministic Bloom sketch of tokenized annotation and co-occurrence patterns, sampled through seeded probes into bounded capability channels. Fixed inputs reproduce the same virtual capability matrix, while changing the seed samples another draw from the same sketch.
- Sketch construction: The Bloom sketch is built from tokenized annotation records using salted hash families over a fixed bit array.Tokens represent object and co-occurrence information extracted from the dataset.
- Sketch parameters: The local run uses m = 220 bits, k = 4 hash functions, and sketch density ρ = 12.95%.These parameters define the compact representation used for the local dataset mixture.
- Dataset tokenization: VisDrone tokens encode category, grid cell, object size, occlusion, and capped same-image co-occurrences; UAVSwarm tokens encode boxes, image dimensions, track buckets, and same-frame co-occurrences.Dataset-family prefixes preserve source identity at token level while sharing one bit array.
- Capability draw: Each seeded probe selects a sketch bit, averages P = 256 probes per channel, clips the result, and yields speed, agility, sensing, endurance, and strength capabilities.The centering treats sketch density as a baseline, producing a bounded seed-indexed contrast statistic.
- Reproducibility: Capability matrices are deterministic for fixed annotation files, tokenizer, Bloom parameters, and seed; changing only the seed produces a different virtual swarm from the same sketch.This supports exact figure regeneration without embedding large raw datasets in the source bundle.
- Concentration: Probe concentration improves as O(P^-1/2), while dataset changes affect the draw through the occupied Bloom bits.The clipping map is 1-Lipschitz and therefore does not increase deviations after affine normalization.
4 Formation Game
The formation game converts deterministic capability draws into image-space defender and attacker formations, finite zero-sum payoffs, and auditable strategic, leakage, and visual metrics. Screen tuning improves robust security and leakage across scales, while scale-normalized raster bandwidth addresses resolution-dependent visual degradation.
- State and payoff construction: 6v6 to 32v32 formations map capability-derived agents to normalized image coordinates without producing physical control commands.Defenders use ring, screen, wedge, or dispersed formations; attackers use line, column, pincer, or cloud.
- Screen tuning: 0.593 robust screen security is achieved in the local 6v6 experiment, up from 0.526.At 32v32, tuning raises robust screen security from 0.537 to 0.616 and reduces line-attack leakage from 0.471 to 0.397.
- Auditable guarantees: A single rendered state can be audited through strategic residual, equalization leakage, and visual-regime generalization metrics.The linkage theorem treats these as distinct audits sharing the same seed-indexed rendered state.
- Visual-regime audit: Scale-normalized Gaussian bandwidth prevents the active-cell envelope from collapsing as raster resolution increases, and empirical selection cannot exceed the fixed-pixel baseline’s joint loss.The 128-by-128 fixed-pixel failure is attributed to increasingly sparse image-plane kernels rather than an intrinsically worse larger visual field.
- Screen tuning: 0.035 interception dispersion is reached by the tuned screen in the seed-17 line-attack counterfactual, versus 0.175 for the fixed ring.The tuned screen also reduces leakage from 0.556 to 0.431 and tightens the certified upper bound from 0.609 to 0.452.
5 Repeated System Game
The repeated system game extends frame-level security into a finite-horizon Markov model whose state tracks operational, social, resource, learning, and information variables. Local and server audits identify budget-adaptive policies, while the ethics stress audit evaluates additional non-operational risks.
- State and transitions: The normalized repeated-game state has 11 coordinates covering pressure, stocks, fatigue, adaptation, exposure, stress, insurance/oil pressure, budget, dataset growth, model improvement, and entropy budget.The state remains in [0, 1]^11, and the entropy budget represents information diversity preserved for learning.
- State and transitions: The repeated payoff combines system health, budget, dataset growth, model improvement, and entropy with penalties for fatigue, adaptation, exposure, stress, insurance/oil pressure, and low interceptor stock.The system-health term combines pressure, interceptor stock, model improvement, and entropy budget.
- Audit results: The local audit evaluates 6 candidate policies for 24 rounds, with budget adaptive achieving value 0.438 and 100.0% survival.Its final entropy budget is 0.802, dataset growth is 0.468, and model improvement is 0.799.
- Audit results: The server audit evaluates 16,777,216 trajectories for 32 rounds, selecting a budget-adaptive policy with value 0.461 and gain 0.064 over fixed screen.Unlike the single-frame result, the repeated policy is budget adaptive because it preserves entropy budget and model improvement while limiting fatigue, exposure, and insurance/oil pressure.
- Risk audit: The ethics stress audit scores false-alarm burden, overcollection risk, escalation pressure, and autonomy-risk pressure; the best local policy has risk-adjusted value 0.376.These dimensions are presented as a reproducibility check rather than a moral verdict or deployment authorization.
6 Visual Pattern Readout
The paper renders dataset-derived virtual swarm states as inspectable tactical overlays and rasterized fields, then audits whether visual readouts preserve strategic structure across resolutions. Scale-normalized encoding improves the 128-by-128 readout after fixed-pixel bandwidth loss reduces performance.
- Tactical overlays: Real VisDrone overlays render virtual agents, links, confidence scores, asset and hotspot markers, and certificate quantities from the same state.The seed-17 tuned screen reports leakage 0.431, payoff 0.585, hotspot score 0.970, and RI = 0.035.
- Tactical overlays: The tuned screen reduces seed-17 leakage from 0.556 to 0.431 and interception dispersion RI from 0.175 to 0.035 versus a fixed-ring defender.The comparison also reduces hotspot score from 1.202 to 0.970 using ∆x = 0.25 and h = 0.50.
- Tactical overlays: A denser 32v32 VisDrone state remains virtual and is rendered with subsampled labels and links for legibility.Its readout identifies defender-holds with leakage 0.325, payoff 0.689, and hotspot score 3.799.
- Resolution audit: The raster audit tests regime accuracy, hotspot error, active-cell ratio, and entropy from 32-by-32 through 128-by-128 fields.Under fixed-pixel encoding, accuracy peaks at 48-by-48 at 74.1%, while 128-by-128 reaches 67.2% with hotspot error 0.136 versus 0.078 at 32-by-32.
- Resolution audit: At 128-by-128, the selected scale-normalized encoder with λ = 1.50 reaches 77.6% accuracy, lowers hotspot error to 0.054, and reduces joint loss by 0.185.The encoder improves classification loss by 0.103 and hotspot error by 0.082; the server audit selects the same family.
- Summary: The experiment summary juxtaposes adversarial security gains against a held-out visual-regime benchmark.The figure’s center panel compares the mixed policy with a fixed-ring defender, while the right panel reports visual-regime performance.
7 Reproducible Environment
The reproducibility environment separates article sources from executable materials and supports local or isolated execution. Server scripts provide larger datasets and GPU audits without shipping raw data in the local bundle.
- Repository structure: The article source contains main.tex, refs.bib, numbers.tex, and selected figures, while reproducibility materials contain code, results, manifests, instructions, and formal checks.Large raw datasets are not shipped with the source bundle.
- Formal checks: The ancillary Lean and Coq checks verify the finite encoder-selection inequality and fixed-point 128-by-128 numeric gain identities.They do not formalize the entire computer-vision pipeline.
- Execution: The local command sequence is make experiments make paper make bundle.
- Execution: A Dockerfile supports isolated execution of the repository.The documented workflow builds and runs the visual-swarm-games container with the repository mounted at /work.
- Server expansion: Server scripts download full dataset and provenance materials and include a CUDA batched game-theory audit for large remote GPU sweeps.The larger Sheffield video archives can be enabled with a script flag.
8 Limitations
The benchmark intentionally favors compact, inspectable abstractions over physical or operational realism. Its capability model, classifier, dataset scale, and action set bound the scope of its conclusions.
- Scope: The payoff model is a controlled benchmark rather than an aerodynamic or operational simulator.It emits compact game states but does not model physical flight dynamics.
- Representation: Bloom-filter capability draws are deterministic and tied to annotation statistics, but they are not learned physical latent models.
- Evaluation: The visual classifier is deliberately weak, and stronger recognition models are deferred until larger datasets and source-aware leakage controls are available.
- Game design: The small formation action set keeps the matrix inspectable but does not claim tactical completeness.
9 Conclusion
The paper introduces a pattern-derived virtual swarm benchmark that converts drone-vision annotations into multi-scale game states and renders the same state for human inspection and regime classification. Its operational boundary remains explicit: the artifact studies virtual states rather than real drone control.
- Contribution: Real drone-vision annotations become a Bloom-filter sketch that generates synthetic capability vectors for formation games up to 32v32.
- Contribution: The same virtual state is rendered as a human-readable image and as a 32-by-32 raster for strategic-regime classification.
- Scope: The benchmark connects computer vision, pattern recognition, and finite/repeated game analysis while keeping the operational boundary explicit.It studies virtual states, not real drone control.