SafeStep: An Interactive Demonstration of Semantic Communication for Pedestrian Safety Monitoring

Christian McDowell; Andrea Panebianco; Jeremiah Yang; Sirin Chakraborty; Samuel Chamoun; Travis Ross; Yin Sun

Source-linked AI summary

SafeStep: An Interactive Demonstration of Semantic Communication for Pedestrian Safety Monitoring

Christian McDowell, Andrea Panebianco, Jeremiah Yang, Sirin Chakraborty, Samuel Chamoun, Travis Ross, Yin Sun

arXiv:2608.27688v1Aug 27, 2026cs.LG

TL;DR

SafeStep addresses the need to observe how noisy, limited, and stale communication affects safety-critical pedestrian information rather than only averaging conventional performance metrics. It provides an interactive browser platform that transmits pedestrian histories over an AWGN channel using selectable transceivers and conditions. Meta-VIB achieves substantial task-loss reductions while the platform supports concurrent access, though per-browser frame rate declines at high user counts.

Problem
Existing evaluations typically average throughput, delay, and inference accuracy, which does not directly reveal how channel noise and codelength affect downstream safety information.
Method
SafeStep processes four live traffic-camera feeds, transmits pedestrian semantic histories through selectable transceivers over an AWGN channel, and renders user-specific positions, trajectories, and risk labels.
Results
Meta-VIB achieves task-loss reductions of up to 92.1% relative to baselines across evaluated SNR, codelength, and AoI conditions.
Takeaways & Limitations
SafeStep makes AoI-related downstream degradation directly observable in live monitoring while maintaining 5 frames/s through 20 concurrent users.

Abstract

from arXiv · show

In this paper, we develop SafeStep, an interactive browser-based semantic communication platform for live pedestrian safety monitoring. SafeStep extracts pedestrian information from four live traffic-camera feeds, transmits it through a semantic communication transceiver over an Additive White Gaussian Noise (AWGN) channel, and renders user-specific positions, trajectories, and risk labels. The platform allows to independently select the transceiver, Signal-to-Noise Ratio (SNR), codelength, and Age of Information (AoI), and demonstrates the transceiver performance of the selected configuration through live pedestrian safety monitoring to each browser. SafeStep compares a recently proposed semantic communication design called Meta-VIB with five baseline transceivers. Meta-VIB uses a compact neural model with only $4.16$ million parameters to generalize across varying SNR, codelength, and AoI values without online retraining. Experimental results show that Meta-VIB achieves mean task-loss reductions of up to $92.1\%$. On one high-end GPU server, the integrated concurrent-access workload maintains the target $5$ frames/s through $20$ users. At $100$ users, each requesting a distinct configuration, SafeStep records no request failures and a mean application response time below $1$ s, but its mean per-browser frame rate falls to approximately $1$ frame/s. To our knowledge, SafeStep is the first real-time semantic communication platform to make AoI-induced downstream degradation directly observable in live monitoring applications.

I. INTRODUCTION

SafeStep demonstrates semantic communication for live pedestrian safety monitoring, making the effects of communication conditions observable through an interactive browser platform. It combines Meta-VIB and baseline transceivers with concurrent user access and reports task-loss and responsiveness results.

Meta-VIB generalizes across varying SNR, AoI, and codelength values without online retraining using a 4.16 million-parameter neural architecture.Its design uses FiLM layers and an information-concentration regularizer for dynamic codelength truncation.
SafeStep uses four live traffic-camera feeds to transmit safety-critical pedestrian information over an AWGN channel and display reconstructed positions and safety statuses.Users can compare original traffic scenes with reconstructed pedestrian information while varying transceivers, SNR, codelength, and AoI.
SafeStep supports interactive visualization of pedestrian positions and safety statuses across SNR values from −10 to 20 dB, codelengths from 2 to 16 symbols, and AoI values from 0 to 6 seconds.Each user can independently select the transceiver and communication conditions.
92.1% is the maximum reported Meta-VIB task-loss reduction relative to baselines across evaluated SNR, codelength, and AoI conditions.The largest improvements occur under low-SNR and high-AoI conditions.
5 frames/s is maintained through 20 concurrent users, while 100 users produce no request failures and below-1-second mean response time but approximately 1 frame/s per browser.Each browser requests a distinct configuration in the concurrent-access experiment.

II. RELATED WORK

Related work spans learned JSCC, task-oriented representation learning, freshness-aware communication, and hardware demonstrations. SafeStep differs by using AoI within Meta-VIB and by exposing communication-condition effects to concurrent browser users over a software-emulated AWGN channel.

DeepJSCC-based methods jointly learn source-channel mappings, while VIB-family methods learn compressed representations for downstream tasks.Adaptive DeepJSCC variants support changing SNR and bandwidth; DeepJSCC-ℓ++ supplies both to encoder and decoder during training and inference.
Prior freshness-aware studies compute age outside the encoder and receiver rather than using it as a neural-network input or training loss.Examples include peak Age of Task Information and Age of Misclassified Information.
Meta-VIB trains over multiple age offsets, distinguishing it from prior methods that evaluate or adjust freshness without conditioning neural networks on age.The cited prior studies use age for channel-use adjustment or evaluation.
SafeStep complements hardware prototypes by studying concurrent browser observation of SNR, codelength, and AoI effects on live pedestrian reconstructions over a software-emulated AWGN channel.Earlier prototypes demonstrated image reconstruction over FPGA, software-defined-radio, and edge-graphics-processing testbeds.

III. SAFESTEP SYSTEM DESIGN

SafeStep separates shared live-scene processing, on-demand transceiver serving, and browser rendering. This architecture lets many users explore the same scene while requesting different semantic communication configurations.

SafeStep consists of data source and preprocessing, on-demand transceiver serving, and user-interface rendering components.The three-part design avoids repeating the entire processing pipeline for every browser.
SafeStep’s system design is presented as a block diagram, linking shared source processing to transceiver serving and user rendering.The figure caption identifies the visual as the SafeStep system design block diagram.
A source update contains shared scene information and each tracked pedestrian’s recent history.The platform processes each newly processed set of camera frames as a source update.
The server reuses a reconstruction when multiple requests require the same computation, while each browser combines shared scene information with its requested pedestrian reconstruction.This supports shared processing once per source update.

A. Data Source and Preprocessing

SafeStep preprocesses four live camera views into tracked pedestrian histories and risk labels before semantic transmission. The encoder sends only pedestrian-history symbols through the emulated AWGN channel, while rendering data is shared separately.

SafeStep separates a 2×2 traffic-camera mosaic into four views and processes them once per source update.The views come from one public YouTube stream showing Toomer’s Corner in Auburn, Alabama.
Vehicle and pedestrian detections are produced before transceiver execution, with pedestrian detections supplying position observations and vehicle detections indicating traffic motion.Separate YOLO26l vehicle and YOLO11s pedestrian detectors support this preprocessing.
Detector-derived pedestrian centroids provide reference positions, but may contain missed detections or localization errors because live frames are not manually labeled.The centroid is the center of each pedestrian’s bounding box.
A risk zone is a fixed polygon labeled safe, cautious, or dangerous, and each pedestrian receives the label of the zone containing its centroid.Traffic-signal state and observed vehicle motion update the zone labels.
Every 0.2 s, SafeStep records normalized centroids and risk labels; five recent records form the semantic history sent as complex channel symbols.The decoder reconstructs positions, trajectories, and risk-label probabilities, while camera and detection data bypass the emulated channel as shared rendering information.

B. On-Demand Transceiver Serving

SafeStep serves browser-specific transceiver requests while sharing pretrained models and reusing reconstructions to avoid duplicated computation.

On-Demand Transceiver Serving: Users can request different transceivers, SNR, codelength, and AoI while SafeStep shares pretrained models and reuses reconstructions.This avoids repeating server-side work for identical requests or computations.
On-Demand Transceiver Serving: N(t) counts distinct requested configurations across M(t) connected browsers, not model copies or transceiver executions.An effective operation ignores settings unsupported by the selected model, so multiple configurations may share computation.
On-Demand Transceiver Serving: Six supported designs remain loaded in GPU memory, keeping model-parameter memory fixed as the number of browsers increases.The supported set includes Meta-VIB and five baselines.
On-Demand Transceiver Serving: SafeStep checks a cache indexed by source update and effective operation before running a transceiver, returning stored results when available.Otherwise, it applies the AWGN channel, decodes the representation, and stores the reconstruction until the next source update.

1) Supported Semantic Communication Designs:

SafeStep supports six semantic communication designs through a common pedestrian-safety interface, with Meta-VIB adapting to selected communication conditions while baselines use fixed inputs and representations.

Supported Semantic Communication Designs: Six transceiver designs share one interface: Meta-VIB plus JSCC, DeepJSCC, VIB, Hyper-VIB, and ATROC.Each receives the same pedestrian history and returns five estimated pedestrian centroids with risk-label distributions.
Supported Semantic Communication Designs: Meta-VIB summarizes pedestrian histories recurrently and uses a hypernetwork-generated FiLM modulation to adapt intermediate features.The history summary is computed once per history and reused across Meta-VIB requests during a source update.
Supported Semantic Communication Designs: Meta-VIB orders K = 16 complex channel symbols so shorter requested codelengths retain the representation’s first ηu(t) coordinates.The retained coordinates are normalized, transmitted over AWGN at the selected SNR, and placed into the decoder inputs.
Supported Semantic Communication Designs: Meta-VIB converts AoI into a discrete offset δu(t), which its decoder uses to predict targets farther ahead of the newest pedestrian-history observation.The five targets lie δu(t) + 1 through δu(t) + 5 samples ahead.
Supported Semantic Communication Designs: Baseline models do not receive SNR, codelength, or AoI inputs; the live platform uses Km = 2, so every baseline transmits both symbols.Separately trained Km = 8 baselines are evaluated in Section IV-A with test-time truncation.

C. User Interface and Rendering

SafeStep combines shared live scene delivery with browser-specific reconstruction and rendering, allowing each user to change communication settings independently and visualize AoI effects.

User Interface and Rendering: Each browser starts with Meta-VIB at 20 dB, codelength 16, and AoI 0 s, then independently changes its transceiver or communication conditions.The receiver view displays reconstructed pedestrian positions, trajectories, and risk labels over the shared scene.
User Interface and Rendering: Server-Sent Events broadcast shared scene updates, while HTTP requests return each browser’s decoded centroid sequence and risk-label probabilities.The browser draws the requested reconstruction over the shared scene instead of receiving a separate video stream.
User Interface and Rendering: Each browser stores 75 recent scene updates, covering about 15 s at 5 updates/s and supporting the selectable AoI range of 0 to 6 s.The server buffer and SSE queue keep delivery current while browser history supports age-based rendering.
User Interface and Rendering: The browser aligns every decoded trajectory’s first point with the pedestrian’s current detector-derived position while preserving subsequent relative movement.The detector-derived alignment position is side information and does not pass through the emulated channel.
User Interface and Rendering: For selected AoI, the browser displays scene update max{0, J − δu(t)} and moves the pedestrian marker backward along observed motion.These browser-side changes affect visualization; only Meta-VIB also receives δu(t) as a decoder input.

IV. EVALUATION

The evaluation traces SafeStep from receiver-side monitoring and transceiver behavior to complete-platform scalability as system conditions and user counts vary.

User interface: Fig. 3 shows how transceiver selection changes displayed pedestrian positions, trajectories, and risk labels.
Transceiver performance: Fig. 4 compares transceivers as SNR, codelength, and AoI change.
Server capacity: Fig. 5 shows how the complete platform responds as the number of users increases.

A. Performance Across Communication Conditions

SafeStep makes transceiver-dependent pedestrian reconstructions visible and evaluates mean task loss as SNR, codelength, and AoI vary. Meta-VIB performs best across most conditions, with especially clear gains under substantial noise and increasing AoI.

Different transceivers produce visibly different reconstructed pedestrian positions, trajectories, and risk labels in the receiver view.
Mean task loss generally decreases as SNR or codelength increases and rises as AoI increases.The task loss averages over cameras, pedestrians, and the five-step prediction horizon; lower values indicate better preservation of safety-relevant outputs.
Meta-VIB produces the lowest plotted mean task loss over most of the SNR range.Its clearest gains occur when channel noise is substantial.
Meta-VIB performs well across several codelengths using one model, whereas each baseline width requires a separately trained model.
Meta-VIB remains below the baselines throughout the AoI sweep as prediction targets move farther into the future.The fixed five-sample input and AoI-aware decoder are described as consistent with this behavior.

B. Concurrent-Access Stress Test

The stress test measures SafeStep under increasing concurrent browser demand, including scene delivery and pedestrian reconstruction requests. Reconstruction delivery meets the target through 20 users but slows substantially at higher loads.

Fig. 5 compares stress-test performance with the number of concurrent browser sessions.
Each user requests a different configuration, increasing both scene-delivery demand and configuration diversity as users are added.Scene-plus-reconstruction delivery adds HTTP requests for pedestrian reconstructions, unlike scene-only delivery.
Scene-plus-reconstruction delivery holds the target rate with low delay through 20 users.
At 40 users, delay rises, throughput stops growing, and frame rate drops to about 2.5 frames/s despite approximately 14% CPU and 29% GPU utilization.The evidence points to server-side request handling or scheduling rather than global CPU or GPU saturation.
At 100 users, scene-plus-reconstruction delivery remains operational but provides about 1 frame/s.

V. CONCLUSION

SafeStep demonstrates live semantic communication for pedestrian safety monitoring while exposing performance across communication conditions and concurrent users. Meta-VIB delivers strong task-loss reductions, but high concurrency lowers frame delivery despite sub-second response delay and no request failures.

SafeStep connects four traffic-camera feeds to user-selected semantic transceivers using a single GPU server.Browsers independently select the transceiver, SNR, codelength, and AoI and display pedestrian positions, trajectories, and risk labels.
Meta-VIB operates across the evaluated communication conditions without online retraining.
92.1% is the maximum reported Meta-VIB task-loss reduction.Its clearest gains occur under noisy channels.
5 frames/s is maintained through 20 users with distinct requested configurations.
At 100 users, SafeStep records no request failures and mean response delay below 1 s, while delivery falls to approximately 1 frame/s.
SafeStep is presented as the first real-time semantic communication platform to make AoI-induced downstream degradation directly observable in live monitoring applications.