Source-linked AI summary
Wireless Sensing Data Collection and Processing for Metaverse Avatar Construction
Jiacheng Wang, Hongyang Du, Xiaolong Yang, Dusit Niyato, Jiawen Kang, Shiwen Mao
TL;DR
Avatar construction needs data describing users in the physical world, while existing wireless-sensing datasets have limitations. The paper presents a Nexmon-based dataset and a phase-shift detection and correction method; correction brings ToF estimation error near the no-shift case.
Problem
Existing wireless-sensing datasets are mostly based on 802.11n devices with constrained bandwidth and limited antennas, while avatar construction needs substantial physical-world user data.
Source: §I · p. 1Method
The paper presents a Nexmon-based wireless-sensing dataset and detects CSI phase shifts with a sliding window before applying phase compensation.
Source: C. Data analysis and processing · p. 5, §VII · p. 7Results
Correction brought ToF estimation error to about 1.5 ns, near the approximately 1 ns no-shift case, versus 5 ns with phase shift.
Source: A. The impact of phase shift · p. 6Takeaways & Limitations
The Nexmon dataset is presented for wireless human sensing to assist avatar construction.
Source: §VII · p. 7
Abstract
from arXiv · showhide
Recent advances in emerging technologies such as artificial intelligence and extended reality have pushed the Metaverse, a virtual, shared space, into reality. In Metaverse, users can customize virtual avatars to experience a different life. While impressive, avatar construction requires a lot of data that manifest users in the physical world from various perspectives, and wireless sensing data is one of them. For example, machine learning (ML) and signal processing can help extract information about user behavior from sensing data, thereby facilitating avatar behavior construction in the Metaverse. This article presents a wireless sensing dataset to support the emerging research on Metaverse avatar construction. Rigorously, the existing data collection platforms and datasets are analyzed first. On this basis, we introduce the platform used in this paper, as well as the data collection method and scenario. We observe that the collected sensing data, i.e., channel state information (CSI), suffers from a phase shift problem, which negatively affects the extraction of user information such as behavior and heartbeat and further deteriorates the avatar construction. Therefore, we propose to detect and correct this phase shift by a sliding window and phase compensation, respectively, and then validate the proposed scheme with the collected data. Finally, several research directions related to the avatar construction are given from the perspective of datasets.
I. INTRODUCTION
Human-centric Metaverse services need physical-world user information, and wireless sensing can support tasks from behavior tracking to heartbeat detection. The paper presents an avatar-construction framework and an 802.11ac-based CSI dataset to address limits in existing data.
I. INTRODUCTION
- Wireless sensing can support user tracking, activity recognition, localization, behavior and gesture recognition, breathing, and heartbeat detection across Metaverse service scales.The introduction identifies non-intrusive wireless sensing as a better option than wearable sensors for tracking behavior in VR-based services.Source: §I · p. 1
- Most existing datasets use 802.11n devices with constrained bandwidth and few antennas, limiting support for diverse sensing research.The paper notes that phase information is especially important for high-accuracy tasks such as heartbeat analysis.Source: §I · p. 1
- The paper proposes applying wireless sensing to Metaverse services and presents a general framework for constructing avatars from wireless sensing data.The authors describe this as the first proposal to apply wireless sensing to Metaverse services.Source: §I · p. 2
- The authors conducted real-world experiments and made a publicly available dataset with larger bandwidth and more antennas for avatar-related wireless sensing research.The dataset is intended to help validate a range of wireless sensing algorithms.Source: §I · p. 2
II. WIRELESS SENSING ASSISTED AVATAR CONSTRUCTION
The framework uses wireless sensing data to extract user information and combine it with other sensed information for avatar construction and rendering.
II. WIRELESS SENSING ASSISTED AVATAR CONSTRUCTION
- Physical-world smart devices collect wireless signals, signal-processing and machine-learning modules extract user information, and the service provider combines it with image and sound data to construct and render avatars.The framework is illustrated for virtual conferences and exhibitions.Source: §II · p. 2
A. Virtual conference
Wireless sensing priorities depend on the Metaverse service: conferences emphasize fine-grained behavior, while exhibitions emphasize movement and location. Both contexts also call for vital-sign information.
A. Virtual conference
- Virtual conferences call for fine-grained user behavior and gestures, alongside breathing, heartbeat, and other subtle physiological characteristics.The text connects vital signs with analysis of users’ mentality and emotion for avatar construction.Source: A. Virtual conference · p. 2
B. Virtual exhibition
- Virtual exhibitions prioritize large-scale activity, location, movement direction, and speed over gestures, while also treating breathing and heartbeat as relevant to users’ physical state.The text says physical-state information can inform personalized notifications for avatars.Source: B. Virtual exhibition · p. 2
- The authors argue that ubiquitous wireless signals provide sufficient data sources and support sensing tasks at different scales, making sensing data indispensable for avatar construction.The conclusion is stated across the discussion of both conference and exhibition needs.Source: B. Virtual exhibition · pp. 2–3
III. TESTBEDS AND DATASETS
The reviewed platforms and datasets vary in CSI bandwidth and sensing scope. The authors identify a need for an open-source dataset based on a newer platform for more in-depth human-sensing research.
A. The existing open-source test platforms
- Nexmon supports CSI extraction on modern Wi-Fi chips with up to 80 MHz bandwidth, whereas the reviewed 802.11n tools do not support 80 MHz; SDR offers another option at higher hardware cost.The text also notes a later Nexmon extension for 802.11ax with 160 MHz bandwidth.Source: A. The existing open-source test… · p. 3
B. The available datasets
- Available datasets cover gesture recognition, localization, tracking, and activity recognition, but the reviewed Nexmon dataset is limited to activity recognition.Examples include SignFi, WiDar, WiAR, and a Nexmon-based activity-recognition dataset.Source: B. The available datasets · p. 3
- Most reviewed datasets use the Intel 5300 CSI tool and are limited in bandwidth and quantization accuracy, motivating an open-source dataset based on newer platforms.The authors frame the new dataset as support for more advanced machine-learning and signal-processing algorithms for human sensing.Source: B. The available datasets · p. 3
IV. DATA COLLECTION, OBSERVATION, AND PROCESSING
This section introduces the data-collection and processing pipeline, including the Nexmon platform and the CSI phase-shift problem it addresses.
IV. DATA COLLECTION, OBSERVATION, AND PROCESSING
- The section frames data collection, phase-shift analysis, and shift correction as the core of its processing pipeline.The shift is observed in the collected data and motivates a proposed detection and correction scheme.Source: §IV · p. 3
A. Hardware description
- The Asus AP uses Nexmon and a Broadcom card, with a Host and FullMAC chipset structure.The Host configures the radio through Linux Kernel and DMA; the chipset contains the radio layer, D11 Core, and Full MAC.Source: A. Hardware description · p. 3
- The radio layer handles RF-stage operations, while the D11 Core manages time-sensitive tasks and communicates with Full MAC.The D11 Core executes ucode and manages channel access, frame generation, acknowledgments, and scheduling.Source: A. Hardware description · pp. 3–4
- The platform comparison identifies transmitting- and receiving-antenna counts as comparison attributes.NTA denotes transmitting antennas and NRA receiving antennas.Source: Figure 2 · p. 4
- CSI extraction is managed by the D11 Core, which freezes CSI, packages its parts into datagrams, and reports them to the Host.The Full MAC receives CSI parts, packages them with parameters such as IP and UDP headers, and reports the datagram data to the Host.Source: A. Hardware description · p. 4
B. Data collection
The dataset is collected over two transmission links in a real-world scenario, with a cable-connected reference channel and three free-space multipath channels.
B. Data collection
- The collection uses two transmission links, with phase differences among receiver channels first corrected via a power splitter.The transmitter and receiver are positioned as shown in the collection scenario.Source: B. Data collection · p. 4
- The signals use a 5.8 GHz center frequency, 80 MHz bandwidth, 256 subcarriers, and a 400 Hz packet rate.The packet rate is extracted by Ping; the dataset comparison includes the number of subcarriers (NoS).Source: B. Data collection · p. 4, Figure 3 · p. 5
- One Rx2 antenna is cable-connected to the transmitter as a reference, while the other three antennas receive free-space multipath signals.Because synchronization introduces the same phase offset across the four channels, conjugate multiplication can eliminate the phase error in the multipath CSI.Source: B. Data collection · pp. 4–5
- The reference-channel configuration enables more precise analysis of free-space multipath signals, including absolute ToF and Doppler estimation.Source: B. Data collection · p. 5
C. Data analysis and processing
The analysis finds random time-domain shifts in CSI phase differences across receiving antennas and proposes sliding-window detection and phase compensation.
C. Data analysis and processing
- The phase difference can shift randomly in time, either away from the central subcarrier or across all subcarriers.The figure distinguishes these two observed shift cases.Source: C. Data analysis and processing · p. 5, Figure 4 · p. 5
- In the first case, each subcarrier’s phase-shift value is constant over time and varies linearly with subcarrier frequency.The phase difference between symmetrical subcarriers around the center frequency can be the same, as for subcarriers 7 and 251.Source: C. Data analysis and processing · p. 5
- A sliding window detects central-subcarrier shifts by thresholding phase differences, then checks other subcarriers and compensates detected shifts.When the center is stable, detection is best performed on a higher-index subcarrier; correction adds or subtracts the shift according to the chosen boundary reference.Source: C. Data analysis and processing · p. 5
- After the central-subcarrier phase difference shifts, some antenna phase differences can become chaotic and difficult to correct.The paper states this case does not happen very often.Source: C. Data analysis and processing · p. 5
V. CASE STUDY
The case study considers avatar construction from wireless sensing data and evaluates AoA and ToF estimation errors rather than directly comparing avatars before and after phase-shift correction.
V. CASE STUDY
- Avatars can differ when constructed from the same dataset using different models and methodologies.For example, respiratory-frequency estimates from CSI-amplitude-based and CSI-phase-based models may differ.Source: §V · pp. 5–6
- The case study analyzes AoA and ToF estimation accuracy instead of directly comparing avatar construction before and after phase-shift correction.The authors identify AoA and ToF as two important signal parameters for avatar construction.Source: §V · p. 6
A. The impact of phase shift
Phase shift worsens signal-parameter estimates and moves their distribution from its normal position; correction brings estimates closer to the shift-absent case.
A. The impact of phase shift
- ToF estimation error rises from about 1 ns without phase shift to 5 ns with shift, then falls to about 1.5 ns after correction.The corrected error is described as almost the same as the shift-absent case.Source: A. The impact of phase shift · p. 6
- Across accumulated estimates, phase shift drives the parameter distribution away from its normal position, while correction returns the overall distribution to normal.The authors state that the shift has a severe negative impact on avatar construction.Source: A. The impact of phase shift · p. 6
B. Signal parameter estimation accuracy analysis
- At confidence 0.6, phase shift increases active-mode AoA/ToF errors by 1.5 degrees/2.1 ns and passive-mode errors by 1.1 degrees/1.3 ns, with larger ToF impact.Below confidence 0.5, the curves before and after shift almost overlap; above 0.5, they differ significantly.Source: B. Signal parameter estimation… · p. 6
A. Wireless sensing data enhancement
The paper identifies limits in available wireless-sensing datasets and compares platform capabilities, motivating additional public datasets for avatar construction.
- The paper states that platform limitations mean current datasets support only limited types of sensing-algorithm design.Source: A. Wireless sensing data enhancement · p. 6
- Fig. 5 depicts phase-shift effects through AoA-ToF spectra and parameter distributions, while Fig. 6 compares estimation accuracy before and after correction.Source: Figure 5 · p. 6, Figure 6 · p. 6
- Nexmon offers greater bandwidth and more antennas than Intel 5300, but trails SDR and AX-CSI; AX-CSI supports up to 160 MHz in 4 × 4 MIMO.SDR also permits user-specified wireless-parameter configuration.Source: A. Wireless sensing data enhancement · p. 6
- The authors call for more public datasets using these platforms to improve avatar-construction efficiency and support building the surrounding Metaverse environment.Source: A. Wireless sensing data enhancement · p. 7
B. Wearable device data and processing
The paper calls for combining wearable-device data with wireless sensing and for adaptive fusion of cross-modal data; its conclusion presents a Nexmon-based dataset and phase-shift correction.
B. Wearable device data and processing
- Wearable sensors can record subtle physical-world parameters that wireless signals or images may not capture, including pulse, blood pressure, and blood sugar.The authors also call for wearable-data collection and sharing, plus processing models for weak-signal feature extraction and trend prediction.Source: B. Wearable device data and processing · p. 7
C. Multimodal data processing
- Cross-modal fusion should account for data characteristics and adjust in real time to user needs, preferences, situations, and device resources.The authors identify fusion as a key concern for future work.Source: C. Multimodal data processing · p. 7
VII. CONCLUSIONS
- The conclusion presents a Nexmon-collected wireless-sensing dataset, details its collection platform and scenario, and verifies a proposed phase-shift correction using the dataset.It also discusses future research directions and describes wireless sensing as indispensable to Metaverse construction.Source: §VII · p. 7