Source-linked AI summary
See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar
Chris Xiaoxuan Lu, Stefano Rosa, Peijun Zhao, Bing Wang, Changhao Chen, John A. Stankovic, Niki Trigoni, Andrew Markham
TL;DR
milliMap tackles indoor mapping when smoke and other obscurants impair optical sensors, a problem relevant to emergency response. It uses low-cost mmWave radar with generative reconstruction, cross-modal lidar supervision, geometric priors, and semantic features; experiments report dense-map error below 0.2m and semantic classification around 90%.
Problem
Indoor mmWave mapping remains under-explored because single-chip radar measurements are sparse and affected by severe multi-path noise.
Method
milliMap combines cross-modal supervision from co-located lidar with indoor geometric priors and generative learning to reconstruct dense maps from sparse, noisy mmWave data.
Results
milliMap reconstructs dense maps with error less than 0.2m and classifies key semantics with approximately 90% accuracy.
Takeaways & Limitations
milliMap demonstrates low-cost mobile mmWave sensing for dense, semantically annotated indoor maps in smoke-filled and previously unseen environments.
Takeaways & Limitations
The prototype uses a Turtlebot platform that is not rugged enough for real disasters, and further trials are needed across diverse buildings, obscurants, and emergency conditions.
Abstract
from arXiv · showhide
This paper presents the design, implementation and evaluation of milliMap, a single-chip millimetre wave (mmWave) radar based indoor mapping system targetted towards low-visibility environments to assist in emergency response. A unique feature of milliMap is that it only leverages a low-cost, off-the-shelf mmWave radar, but can reconstruct a dense grid map with accuracy comparable to lidar, as well as providing semantic annotations of objects on the map. milliMap makes two key technical contributions. First, it autonomously overcomes the sparsity and multi-path noise of mmWave signals by combining cross-modal supervision from a co-located lidar during training and the strong geometric priors of indoor spaces. Second, it takes the spectral response of mmWave reflections as features to robustly identify different types of objects e.g. doors, walls etc. Extensive experiments in different indoor environments show that milliMap can achieve a map reconstruction error less than 0.2m and classify key semantics with an accuracy around 90%, whilst operating through dense smoke.
1 INTRODUCTION
milliMap addresses the need for indoor mapping in dangerous, low-visibility emergency environments where optical sensors can fail. It uses low-cost mmWave radar and combines learned reconstruction with semantic mapping to produce dense, annotated maps.
- Motivation: 2,775 firefighters died on duty in the USA over a reported 10-year period, motivating improved situational awareness for emergency response.Mobile robots can rapidly survey unknown buildings, support accurate personnel relocalization, and indicate features such as exit routes.
- Motivation: Optical mapping sensors are impaired by smoke, dust, fog, poor illumination, darkness, and glare.These conditions regularly occur in emergency situations, including dense smoke during firefighting.
- Motivation: mmWave radar is low-cost, low-power, and impervious to smoke, fog, and dust, making it suitable for sensing smoke-filled environments.The paper considers single-chip mmWave radar as an alternative to optical sensors for fire response.
- Challenges: Up to 75% of mmWave points can be outliers, while point density is more than two orders of magnitude lower than lidar.These sparsity and multi-path problems make indoor mmWave mapping difficult.
- Contributions: milliMap produces occupancy grid maps with semantic annotations for space-accessibility objects including doors, lifts, glass, and walls.Its contributions include mobile robotic sensing, generative reconstruction, semantic mapping, and real-time evaluation in smoke-filled conditions.
2 PRIMER
The primer explains how mmWave radar measures range and angle, and how GANs can generate dense maps from sparse inputs. Bayesian grid mapping represents observed space as free, occupied, or unknown cells.
- mmWave Radar Principles: FMCW radar estimates target distance from the intermediate-frequency signal produced by mixing received and transmitted chirps.The distance calculation uses the IF signal frequency and chirp frequency slope; FFT peaks separate obstacles at different ranges.
- mmWave Radar Principles: A linear receiver antenna array estimates obstacle angle from phase differences between sampled antenna signals.Beamforming and MUSIC can obtain angle of arrival, which combines with ranging to determine obstacle position.
- Generative Adversarial Networks: GANs train a generator and discriminator simultaneously, with the generator producing samples and the discriminator distinguishing real from generated data.milliMap uses this generative capability to create dense maps from sparse input.
- Bayesian Grid Mapping: Bayesian grid mapping assigns each map cell a free-space, obstacle, or unknown state.The three-state occupancy representation supports interpreting reachability for navigation.
3 MILLIMAP OVERVIEW
milliMap uses co-located radar and lidar on a mobile robot to collect environment information, then outputs a dense occupancy grid map with semantic obstacle annotations.
- System Overview: milliMap takes mmWave reflections captured by a low-cost, single-chip radar and outputs a dense grid map with semantic annotations on obstacles.The system overview comprises mobile robot sensing, map reconstruction, and semantic mapping.
- Mobile Robot Sensing: The mobile robot sensing module collects environment information from a mmWave radar and a co-located lidar.The lidar is used only during offline training rather than as the deployed mapping sensor.
4 GRID MAP RECONSTRUCTION
milliMap reconstructs occupancy-grid maps from sparse, noisy indoor mmWave point clouds by combining generative reconstruction with cross-modal supervision and geometric priors. The framework addresses multipath artefacts, limited density, and practical sensing constraints for mobile mapping.
- Challenges: Indoor mmWave mapping must handle multipath noise and sparse point clouds that differ substantially from dense lidar measurements.Multipath can create ghost points behind walls, while single-chip radar produces far fewer reflective points than lidar.
- Challenges: Single-chip radar provides approximately 100 reflective points per scan, over 100× sparser than lidar.Sparsity arises from specular reflections, low-cost hardware, and restricted sensing range.
- Reconstruction framework: milliMap converts sparse and noisy mmWave patches into dense, clean map images using a generative neural network with a U-Net generator and multi-scale discriminators.The generator fills gaps and removes multipath artefacts using neighbouring spatial context, while discriminators operate at multiple image scales.
- Reconstruction framework: The system uses lidar as offline cross-modal supervision because building maps may be unavailable, outdated, or costly to calibrate manually.Lidar is co-located with the radar on the mobile robot during data collection but supports training rather than deployment.
- Reconstruction losses: The reconstruction objective combines conditional GAN, feature-matching, perceptual, and map-prior losses to balance appearance, stability, and indoor geometry.The map prior uses fixed line or edge detectors to encourage consistency with geometric structures such as walls and doors.
5 SEMANTIC MAPPING
milliMap adds semantic labels to obstacle maps by recognizing four construction-object categories from mmWave reflection patterns, addressing multipath effects caused by layered indoor materials.
- Semantic targets: Four semantic categories describe space accessibility: doors as horizontal access objects, lifts as vertical access objects, glass as alternative access objects, and walls as non-access objects.The system focuses on these predominant construction objects rather than attempting to represent all indoor semantics.
- Recognition challenge: Layered construction objects create diffusion and internal reflections, making peak signal strength insufficient for reliably identifying their types.Earlier methods were designed for single-material objects or very thin layers with mainly specular surface reflections.
- Recognition pipeline: The semantic recognizer extracts a segment of interest from the range FFT profile and applies a classifier to identify obstacle types.The segment is selected after obtaining a scan at a perpendicular angle to the object.
- Recognition challenge: milliMap exploits complicated multipath signatures from layered walls for semantic classification.A simplified three-slab wall model illustrates how diffusion and multiple internal reflections produce these signatures.
- Recognition pipeline: The robot mechanically scans its horizontal field of view and selects the pose with the largest reflection intensity to determine the perpendicular angle.Afterward, it enters static mode and records the range profile for SOI extraction.
6 IMPLEMENTATION
The prototype uses a Turtlebot 2 with synchronized mmWave, lidar, odometry, and camera sensing, while implementation details include a 4GHz radar bandwidth and approximately 4cm ranging resolution.
- Platform: The prototype data-collection platform is a Turtlebot 2 equipped with synchronized mmWave point-cloud, lidar, and wheel-odometry sensors.The mmWave sensor uses a TI AWR1443 board, while lidar data comes from a Velodyne VLP-16.
- Semantic mapping implementation: Figure 7 represents a 16cm-wide segment of interest around a range-FFT peak and compares average SOI patterns aggregated from 27,952 training samples.Different materials exhibit distinct SOI patterns, with the first feature index preceding the detected peak.
- Radar configuration: The radar operates across 77GHz–81GHz with 4GHz bandwidth, yielding approximately 4cm ranging resolution.Its field of view is 120 degrees in azimuth and 30 degrees in elevation.
- Platform: The platform also provides RGB images from a front-facing monocular camera, with the mmWave sensor, lidar, and camera coaxially located along the robot’s vertical axis.This arrangement supports synchronized multimodal data collection.
7 EXPERIMENTAL EVALUATION
Experiments evaluate milliMap’s reconstruction, component choices, smoke robustness, navigation utility, semantic classification, and runtime on indoor datasets and devices. The system reconstructs maps accurately enough for localization, transfers to smoke-filled environments, and classifies known and unknown objects with high reported performance.
- 7.1 Grid Map Reconstruction Performance: Patch representation substantially outperforms scan representation for reconstruction because scan densification overfits to straight lines.With Pix2PixHD, scan inputs have 20% worse L1 errors and over 35% worse IoU scores on both datasets.
- 7.2 Effectiveness of Sub-components: Line-detector map priors improve robustness to imperfect lidar supervision, while edge detectors are more sensitive to lidar noise.The experiments also find that appearance losses stabilize generator training and improve realism.
- 7.3 Testing in Smoke-filled Environments: milliMap reconstructs a corridor accurately across three smoke-filled scenarios, although lidar, depth, and RGB cameras degrade or fail to see through smoke.The model was trained in benign environments and transferred its mapping ability to unseen smoke-filled environments.
- 7.4 Extending to Hand-held Devices: ∼0.83m error is achieved in the hand-held experiment, providing a better sense of space accessibility than raw data alone.The experiment illustrates how teams of robots and people could build a common map despite differences from the training samples.
- 7.5 Downstream Navigation Tasks: 0.285m mean translation accuracy and 0.142 rad orientation accuracy are obtained in A Building, versus 0.178m and 0.140 rad in B Building for localization on reconstructed maps.These results support using milliMap maps to localize and navigate robots or firefighters.
- 7.6 Semantic Mapping Performance: A 0.92 softmax threshold detects over 96% of out-of-set samples as unknowns while producing less than 2.2% false negatives for known samples.Known-class outputs are more concentrated, whereas out-of-set outputs are flatter and more scattered.
8 RELATED WORK
Prior work uses RF sensing for imaging, tracking, and environment sensing, while optical sensors remain established for indoor mapping. milliMap differs by targeting mobile robotic mapping with a single-chip mmWave solution and by addressing out-of-set semantic predictions.
- RF-based Imaging and Tracking: RF sensing has been applied to static-object imaging, human localization, gesture recognition, body-dynamics tracking, speech recovery, and indoor reflector detection.The cited systems span commodity WiFi chips, specialized FMCW radar, and customized mmWave hardware.
- Semantic Mapping: The paper uses softmax-output distributions to distinguish known classes from out-of-set objects, whose flatter distributions yield lower classification scores.Figure 15 compares sample indices horizontally and represents larger values with darker colors.
- RF-based Imaging and Tracking: milliMap is distinguished from paired-transmitter-receiver systems by using a single-chip solution for mobile robotic platforms.This comparison positions milliMap as a mobile-platform alternative within RF-based environment sensing.
- Indoor Mapping/Imaging with non-RF Sensors: Optical sensors such as RGB cameras, laser rangers, and stereo cameras produce accurate indoor maps but are fragile under darkness, glare, and smoke debris.These adverse-vision limitations motivate alternative sensing modalities for low-visibility environments.
9 LIMITATIONS AND FUTURE WORK
The work is presented as a proof-of-principle study, with limitations spanning platform robustness, environmental validation, and single-agent operation.
- Platform limitations: The TurtleBot platform is not rugged enough for real disaster situations.More robust tracked or snake-like robots are identified as alternatives.
- Evaluation scope: Further trials are needed across diverse buildings, obscurants, and real emergency conditions.Examples include dust in factories and other operational environments.
- Future directions: Future work will explore robot swarms for cooperative exploration and map building.The paper specifically mentions cooperative mapping using SLAM.
10 CONCLUSIONS
milliMap uses a mmWave radar on a mobile robot to create dense maps with reachability and object-semantic information, and demonstrates relocalization and generalization under smoke-filled conditions.
- Mapping capabilities: milliMap creates a dense map indicating place reachability and object semantics.The system uses a mmWave radar mounted on a mobile robot.
- System functionality: Another agent can relocalize within the map.
- Evaluation: Experiments in different indoor environments and smoke-filled conditions evaluate reconstruction, semantic classification, and system efficiency.The results demonstrate generalization to previously unseen environments.