Source-linked AI summary
Electrosense: Open and Big Spectrum Data
Sreeraj Rajendran, Roberto Calvo-Palomino, Markus Fuchs, Bertold Van den Bergh, Héctor Cordobés, Domenico Giustiniano, Sofie Pollin, Vincent Lenders
TL;DR
Electrosense addresses limited knowledge of spectrum utilization by building a collaborative, large-scale monitoring network with low-cost sensing nodes. It stores and processes measurements in a big-data backend and exposes them through an open spectrum data service, with modulation classification achieving over 90% average accuracy across 0-20 dB SNR conditions.
Problem
Spectrum utilization is difficult to understand because fragmented, diverse activity produces large data volumes, while monitoring must support varied applications and stakeholders.
Method
Electrosense combines crowdsourced low-cost sensors, scalable backend processing, open spectrum data access, and tools for securely managing distributed measurements.
Results
Over 90% average modulation-classification accuracy was obtained across SNR conditions from 0 dB to 20 dB, independent of channel characteristics.
Takeaways & Limitations
Electrosense provides public access to spectrum data through storage, applications, and an open API for more efficient and transparent spectrum monitoring.
Abstract
from arXiv · showhide
While the radio spectrum allocation is well regulated, there is little knowledge about its actual utilization over time and space. This limitation hinders taking effective actions in various applications including cognitive radios, electrosmog monitoring, and law enforcement. We introduce Electrosense, an initiative that seeks a more efficient, safe and reliable monitoring of the electromagnetic space by improving the accessibility of spectrum data for the general public. A collaborative spectrum monitoring network is designed that monitors the spectrum at large scale with low-cost spectrum sensing nodes. The large set of data is stored and processed in a big data architecture and provided back to the community with an open spectrum data as a service model, that allows users to build diverse and novel applications with different requirements. We illustrate useful usage scenarios of the Electrosense data.
I. INTRODUCTION
Electrosense addresses fragmented, rapidly changing spectrum use by combining low-cost crowdsourced sensing with scalable data management and open access. The initiative targets diverse monitoring needs while accommodating security, privacy, and deployment challenges.
- Motivation: Spectrum use is increasingly fragmented, bursty, and diverse, making dense monitoring across time, frequency, and space difficult.The resulting data volume creates a central challenge for understanding spectrum usage patterns.
- Motivation: Spectrum monitoring must serve varied users, including consumers, operators, regulators, military applications, and emerging IoT deployments.Their needs include electrosmog analysis, WiFi optimization, coverage mapping, regulatory enforcement, signal positioning, and cooperative detection.
- System challenge: The initiative responds to barriers involving sensor variability and cost, large-scale data management, reliability, security, privacy, and diverse use cases.Earlier systems were either application-specific or too expensive for large-scale deployment.
- Electrosense initiative: Electrosense uses crowdsourcing and low-cost software-defined radio sensors to pursue worldwide spectrum sensing and public access to processed data.The platform supports inexpensive embedded hardware, optional 1.7 GHz-to-6 GHz extension, high-end SDR participation, and open-source setup modules.
- Platform model: Electrosense combines sensor control, backend data storage, algorithm deployment, and open API access to provide spectrum information as a service.Users can retrieve processed information through the API, while the framework includes secure registration, remote control, access permissions, and aggregation tools.
A. Service Model
Electrosense introduces an Open Spectrum Data as a Service model that lets multiple applications use shared spectrum data with different requirements. Its API supports bulk, streaming, aggregated, raw, and algorithm-output access while applying privacy restrictions.
- OSDaaS model: Open Spectrum Data as a Service allows several applications to use spectrum data simultaneously despite having different requirements.The model avoids dedicating sensing nodes to individual campaigns and supports reuse of historical data.
- API access: The open API provides bulk and streaming retrieval of spectrum data, sensor details, and backend algorithm outputs.Available outputs include modulation classification and anomaly detection results.
- API access: Users can request aggregated data at specified frequency and time resolutions or raw FFT data without modifications.Raw FFT access is restricted to the sensor owner.
- Data permissions: The IQ pipeline supports algorithms requiring raw in-phase and quadrature samples, while IQ data access is restricted and derived conclusions are shared more broadly.IQ data is directly available only to its owner through the API, whereas algorithmic conclusions are available to all users.
- Security and privacy: Security and privacy design considerations include secure sensor identification, restricted data access, and caution around storing decodable IQ data.These concerns are especially relevant for sensitive spectrum such as military bands.
III. SYSTEM ARCHITECTURE
Electrosense is designed as a volunteer-driven, scalable spectrum-monitoring system built from low-cost sensors, centralized control, and a backend for collecting and analyzing measurements. Its architecture supports heterogeneous sensing capabilities, fault tolerance, and both low-latency and batch processing.
- Design goals: Volunteer crowdsourcing and low-cost sensors are central design choices for keeping the entry barrier low and enabling large-scale deployment.The system is intended to grow continuously through participation by users deploying sensing nodes.
- Hardware: The sensor hardware combines a single-board computer with an RF front-end, while a custom converter extends scanning toward the 0-6 GHz range.The figure presents an RPi-2, RTL-SDR front-end, and first converter prototype.
- Design goals: The backend is designed to accommodate wide-band sweeping on low-end sensors and real-time scanning on more capable SDR platforms or spectrum analyzers.Its components are intended to scale horizontally and tolerate failures as the network grows.
- Processing requirements: Electrosense supports both low-latency stream processing and large-scale batch analysis for applications with different response requirements.Some applications require immediate responses, while others analyze complete datasets.
- Architecture: Electrosense’s architecture has three components: user-deployed sensors, centralized controller infrastructure, and a backend for data collection and algorithmic insights.The controller administers sensors and measurement campaigns, while the backend processes measurements from across the network.
B. Sensor
Electrosense sensors use low-cost embedded SDR hardware and two selectable preprocessing pipelines for spectrum retrieval. PSD reduces transmitted data through averaged FFT magnitudes, while IQ preserves raw samples for phase-sensitive applications.
- Sensor hardware: Electrosense sensing nodes are small, low-cost software-defined embedded devices with RF front-ends and antennas that can measure from 20 MHz to 6 GHz.An optional GPS supports time synchronization for collaborative scanning and detection algorithms.
- Sensor pipelines: Each sensor supports PSD and IQ pipelines for retrieving spectrum information.The sensor architecture figure identifies two software pipelines.
- PSD pipeline: The PSD pipeline converts RF signals to the frequency domain with FFT and sends averaged squared-magnitude FFTs at approximately 50-100 Kb/s.Averaging and compression reduce bandwidth and storage requirements.
- IQ pipeline: The IQ pipeline transmits compressed raw in-phase and quadrature samples and can produce up to 50 Mb/s.The data is stored temporarily in the backend and supports applications requiring phase and non-averaged information.
- Software: Open-source node software and a GNU Radio module allow both low-cost and high-end SDR users to participate in Electrosense.The GNU Radio module is named gr-electrosense.
C. Controller
The controller infrastructure coordinates sensor measurement campaigns and enables targeted spectrum scanning. It uses messaging and ingestion components to communicate with sensors and buffer measurements for backend processing.
- C. Controller: The command-and-control layer configures sensor frequency ranges, hopping strategies, and sampling rates for in-depth measurements.
- C. Controller: An MQTT broker cluster provides scalable, fault-tolerant messaging between sensors and the master controller.The brokers maintain secure TLS connections to sensors and the master controller.
- C. Controller: The master controller lets administrators start and stop measurement campaigns across sensors through MQTT commands.
- C. Controller: The ingestion layer decouples data collection from batch and speed processing through a distributed queue and asynchronous operation.It also buffers incoming measurements so maintenance of processing layers does not cause data loss.
1) Ingestion Layer:
The backend organizes spectrum processing around durable batch storage and continuously updated speed processing. This architecture supports scalable analysis of incoming and historical data.
- 1) Ingestion Layer:: The batch layer stores incoming raw data as an immutable master dataset for long-running analysis and recovery from algorithmic errors.The dataset is stored in distributed, replicated files.
- 1) Ingestion Layer:: The control layer configures sensors, which send spectrum information to the backend for processing and API distribution.
- 1) Ingestion Layer:: Batch processing uses distributed computation and has latency on the order of minutes or hours.
- 1) Ingestion Layer:: The speed layer uses Spark Streaming over recent data, with a default window length of 5 seconds, and persists results in Cassandra.
3) Speed Layer:
The serving layer hides the separate batch and speed stores by fusing their outputs for users. It exposes an open API, while applications can be built on the resulting data.
- 3) Speed Layer:: The serving layer combines results from batch and speed layers to answer user queries through a RESTful HTTP API.
- 3) Speed Layer:: When both layers contain results, the serving layer prefers batch outputs because they are more accurate.
- 3) Speed Layer:: Electrosense aims to let users implement their own applications through the open API.
A. Live and Historical Spectrum Visualization
Electrosense supports interactive viewing of historical and live spectrum data, including regional TV-band occupancy analysis. Its tools provide adjustable temporal and frequency detail for exploring measurements.
- A. Live and Historical Spectrum Visualization: The spectrum visualizer supports interactive sensor selection and variable frequency resolution for detailed viewing.
- A. Live and Historical Spectrum Visualization: Historical spectrum data is precomputed at multiple temporal and frequency aggregation levels for low-latency web access.
- A. Live and Historical Spectrum Visualization: Live spectrum updates are displayed with 5 second delays through a streaming display.
- A. Live and Historical Spectrum Visualization: TV-band occupancy can be analyzed across cities using 1 MHz frequency and 60 seconds time resolution.Figure 5 presents occupancy in Madrid and Trieste; the infrastructure can detect regional white spaces.
C. Spectrum Cop
Electrosense supports spectrum-related applications through anomaly detection, modulation classification, and calibrated RSSI measurements. These capabilities address analysis, identification, and localization needs in spectrum monitoring.
- Anomaly Detection: Automated anomaly detection is important for identifying unauthorized, deceptive, or disruptive transmissions in the spectrum.Examples include pirate FM stations, fake GSM towers, unauthorized radar interference, and excessive transmission power.
- Modulation Classification: Over 90% average modulation-classification accuracy was achieved across SNRs from 0 dB to 20 dB, independent of channel characteristics.The backend model uses time-domain IQ amplitude and phase information without expert-engineered features.
- Localization: Transmitter localization can generate automated transmitter maps and transmitter fingerprints.The approach requires verifying Electrosense RSSI accuracy through sensor calibration and system-gain measurement.
- Localization: Calibrated Electrosense sensors can provide accurate RSSI information for transmitter localization.Measurements compared a low-cost Electrosense sensor with a LoRa receiver using synchronized 435 MHz ISM-band transmissions.
V. CONCLUSION AND FUTURE WORK
Electrosense provides a crowdsourced framework for broad spectrum monitoring and public access to spectrum data. Future work focuses on expanding backend applications, improving analytics, detecting forged data, and deploying complex algorithms on low-cost sensors.
- Conclusion: Electrosense combines crowdsourced spectrum monitoring with public accessibility, free data storage, applications, and an open API.The initiative targets more efficient, safe, and reliable use of electromagnetic space.
- Future Work: The next development step is enabling usable backend applications, including spectrum prediction, anomaly detection, and spectrum geomaps.Planned backend capabilities include temporal and spatial interpolation and advanced spectrum-estimation techniques.
- Future Work: Future research will address forged data, sensor ranking, analytics for storage constraints, and large-scale deployment of complex algorithms on cheap sensors.The paper identifies malicious-user detection and algorithm disintegration as active research areas.