Source-linked AI summary
Input-Adaptive Gating of a Dehazing Front-End for On-Device Perception in Smoke-Obscured Environments
Seongjun Kang, Ishaan Garg, Vishnu Bharadwaj
TL;DR
The paper asks whether an enhancement front-end reliably improves downstream perception when computation is constrained on embedded hardware. It measures dehazing’s effect on smoke-obscured edge detection and gates the dehazer using a haze estimate. Gating is more accurate on average than always or never dehazing while substantially reducing Raspberry Pi latency.
Problem
Enhancement networks are often assumed to improve downstream tasks, but this assumption is rarely tested when unnecessary enhancement is costly on embedded hardware.
Method
A Raspberry Pi two-stage pipeline uses a dehazer before a compact edge detector and runs the dehazer conditionally when a dark channel haze estimate exceeds a threshold.
Results
Gating is more accurate on average than always or never dehazing, while the dehazer reaches 18.60 dB PSNR and the edge detector reaches ODS 0.738 versus 0.692 for Canny.
Takeaways & Limitations
The right choice between dehazing and direct edge detection varies with haze density, so input-adaptive execution can improve the accuracy–latency trade-off.
Takeaways & Limitations
The gate and haze crossover were calibrated on synthetic constant-atmospheric-light composites and smoke-machine captures, so τ needs refitting on real-fire data.
Abstract
from arXiv · showhide
Two-stage vision pipelines often place an enhancement network before a task network, on the assumption that a cleaner input produces a better output. We evaluate this in a firefighter assistance pipeline, where a dehazer precedes an edge detector that renders smoke-filled rooms as structural outlines. Both were designed for a Raspberry Pi 4, at 355K and 23K parameters, and quantized to UINT8 via TensorFlow Lite. The float dehazer reaches 18.60 dB peak signal-to-noise ratio (PSNR) on held-out real smoke against 13.60 dB unprocessed and 17.08 dB for an AOD-Net trained on the same data, and the edge detector reaches an F-measure at optimal dataset scale (ODS) of 0.738, outperforming an optimized Canny's result of 0.692. Dehazing improves edge extraction under dense smoke but degrades it on clear and lightly hazed frames, where the dehazer discards more detail than the haze obscures. We therefore run the dehazer only when a dark channel haze estimate exceeds a threshold, a 10.1 ms test that lets the pipeline save 469.6 ms on the dehazing stage. Averaged over four haze levels, gating is more accurate than either fixed decision, at 0.675 mean ODS against 0.664 for always dehazing and 0.630 for never dehazing. It reduces the mean per-frame time on the Raspberry Pi from 569 ms to 321 ms, and on clear frames increases the frame rate fivefold, from 1.8 to 9 frames per second.
1 Introduction
This work tests whether enhancement always improves downstream perception in an offline firefighter-assistance pipeline. It finds that dehazing helps under dense smoke but can hurt on clearer frames, motivating input-adaptive execution.
- Approximately 100 firefighters die on duty annually in the United States, with limited visibility in smoke-filled environments a leading factor.
- The offline Raspberry Pi pipeline uses dehazing to remove smoke and edge detection to outline walls, doorways, and obstacles.
- 83% of measured per-frame latency comes from the dehazer, making unnecessary enhancement costly on embedded hardware.
- The study measures how dehazing affects edge accuracy across haze densities and conditions dehazing on that measurement.
- 43.5% lower per-frame cost is achieved by running the 470 ms dehazer only when a 10 ms dark channel haze estimate exceeds a fitted threshold.
2 Related work
The related work combines classical and learned approaches for dehazing and edge detection, while emphasizing methods suitable for embedded deployment.
- The dark channel prior estimates haze from the darkest pixel in each local patch, while DehazeNet and AOD-Net provide small learned dehazing baselines.
- Canny serves as the quantitative edge-detection baseline because HED and DexiNed exceed the study’s embedded parameter budget.
- Depthwise separable convolutions and integer quantization are used as standard tools for reducing vision models to mobile scale.
3 Method
The method is a Raspberry Pi two-network pipeline with a gate that uses a dark channel haze estimate to decide whether dehazing is needed. Compact architectures and UINT8 deployment target embedded execution.
- The gate either runs the dehazer before edge detection or sends the frame directly to the edge detector.
- Figure 1 bases the gating decision on whether dark channel haze estimate s exceeds threshold τ, with latency measured on a Raspberry Pi 4.
- 355,043 parameters define the dehazer, which uses encoders, decoders, dilated convolution, transposed convolutions, and skip features.
- 23,228 parameters define the U-Net-style edge detector, using depthwise separable convolutions, nearest-neighbor upsampling, and skip concatenation.
- Both networks quantize weights, activations, and inputs to uint8 with TensorFlow Lite for deployment.
- The platform is a Raspberry Pi 4 Model B running 32-bit Raspberry Pi OS and Python 3.7 with a USB webcam and OpenCV.
- The dark channel prior mean s is computed from minimum pixel values within 15 × 15 patches, and dehazing runs only when s exceeds threshold τ.
4 Experimental setup
The experiments train and evaluate dehazing and edge models on real or synthetic smoke and annotated edge data, using PSNR, SSIM, ODS, and OIS.
- The dehazer is trained on RESIDE, SMOKE, and DENSE-HAZE data, while the edge model uses BIPEDv2 images with hand-labeled edges.
- Images are resized to 256 × 256, normalized to [0, 1], augmented with flips and rotations, and trained with Adam, validation splits, and early stopping.
- PSNR and SSIM measure dehazing quality, while ODS and OIS measure edge detection using thinned predictions and ground-truth matching within 2 px.
- Code and the Raspberry Pi deployment kit are planned for release, with dataset licences, training compute, and broader impact documented in Appendix C.
- Table 1 compares smoke removal on 12 held-out SMOKE pairs and edge detection on 50 BIPEDv2 test images against named baselines.
5 Results
The deployed edge detector outperforms optimized Canny, while dehazing helps under dense smoke but hurts on clear and lightly hazed frames. Input-adaptive gating follows these condition-dependent trade-offs, improving mean accuracy while reducing Raspberry Pi latency.
- Smoke removal: 18.60 dB PSNR on held-out real smoke is 5.0 dB above the untouched input and exceeds AOD-Net’s 17.08 dB.The model is the best comparison result on both reported dehazing metrics.
- Edge detection: ODS 0.738 for the deployed 23K-parameter edge model exceeds 0.692 for the best swept Canny operating point.The float model reaches ODS 0.786, while UINT8 conversion gives ODS 0.740 for the edge network.
- Effect of dehazing on edge extraction: Dehazing lowers ODS on clean and lightly hazed frames but raises it under dense smoke, reaching 0.598 versus 0.438 at t=0.35.The crossover occurs between t=0.7 and t=0.5 because lossy reconstruction smoothing can remove more structure than light haze obscures.
- Gated execution: The gate tracks the better fixed policy across haze densities with 43% less arithmetic than always dehazing.Always dehazing is worse at the clear end, never dehazing is worse under dense smoke, and the gate is best in equal-weight mean accuracy.
- On-device latency: 321 ms mean per-frame time with gating compares with 569 ms for always dehazing, while clear-frame throughput rises from 1.8 to 9 FPS.The gate fires on 2%, 14%, 72%, and 100% of frames across the four haze conditions.
6 Limitations and future work
Evaluation is limited by small test sets, synthetic haze calibration, and a mismatch between float evaluation and uint8 latency measurements. Future work includes quantization-aware training, real-fire recalibration, and firefighter testing.
- 6 Limitations and future work: The evaluation uses only 12 real smoke pairs and 50 BIPEDv2 images, limiting precision of absolute ODS estimates.A paired bootstrap places the gate’s 0.011 ODS margin over always dehazing at [0.005, 0.017] with 95% confidence, while single-policy ODS is resolved only to ±0.013.
- 6 Limitations and future work: The four haze conditions receive equal weight, although real-fireground density measurements would replace this modelling choice.
- 6 Limitations and future work: The latency results use the uint8 dehazer, whereas Table 1 and Table 2 policy comparisons use float32; quantization reduces dehazer PSNR by 1.8 dB.The quantized model falls below AOD-Net on both reported metrics, motivating planned quantization-aware training.
- 6 Limitations and future work: The crossover and gate threshold were calibrated on synthetic composites and smoke-machine captures, so τ requires refitting on real-fire data.Fireground testing with firefighters also requires ethics review and departmental safety approval.
7 Conclusion
The study tests whether dehazing consistently improves edge detection in an offline Raspberry Pi smoke-vision pipeline. Because its benefit varies with haze, a low-cost haze test gates the expensive dehazer and improves average accuracy while reducing cost.
- 7 Conclusion: Dehazing improves edge accuracy under dense smoke but degrades it on clear input, making the frame-by-frame choice dependent on haze.
- 7 Conclusion: The haze test costs 2% of the dehazer’s latency and selects whether to run it, achieving higher average accuracy than either fixed policy.
- 7 Conclusion: 43.5% cheaper per frame, the gated pipeline preserves higher average edge accuracy than always or never dehazing.
- 7 Conclusion: The approach is intended for settings where an enhancement stage is expensive and only sometimes needed.
A Network architectures
The two networks use compact architectures designed for embedded processing. The dehazer has 355,043 reported parameters, while the edge network uses separable convolutions and a 23K-parameter design.
- A Network architectures: The dehazer uses LeakyReLU, batch normalization, dropout 0.5 during training, dilated convolution, transposed convolutions, and skip concatenations.
- A Network architectures: 355,043 total parameters comprise 353,251 convolution weights and biases plus 1,792 batch-normalization parameters.The batch-normalization count is 448 channels × 4.
- A Network architectures: 23,228 params, the edge network uses depthwise separable convolutions and skip connections for compact edge prediction.
B Operator support on the deployed runtime
The deployed runtime could not support the edge model’s original dynamic TILE operator. Replacing dynamic upsampling with explicit nearest-neighbor resize preserved float behavior while reducing the exported model’s operator set.
- B Operator support on the deployed runtime: The original export failed during allocate_tensors() because UpSampling2D lowered to an unsupported dynamic TILE operator chain.
- B Operator support on the deployed runtime: The limitation arose from tflite-runtime 2.11, which was pinned by Python 3.7 and the available cp37 armv7l wheel.
- B Operator support on the deployed runtime: An explicit nearest-neighbor resize lowered to RESIZE_NEAREST_NEIGHBOR and transferred exactly because the replacement layers carried no weights.The maximum absolute difference between original and rebuilt float models was 0.0.
- B Operator support on the deployed runtime: 53 KB, the rebuilt model uses six distinct operators instead of ten and reaches ODS 0.741 versus 0.740 for the original export.
C Release, licences, and broader impact
The paper reports its release materials, evaluation setup, licensing constraints, and broader-impact boundaries for this research prototype.
- Release: The release includes training and evaluation scripts, two quantized TensorFlow Lite models, and a Raspberry Pi benchmark kit.The kit includes uint8 models, a bundled tflite-runtime wheel, a sample frame, and commands for reproducing latency figures.
- Release: All four datasets are public and cited, but none are redistributed with the released materials.
- Evaluation setup: Evaluation ran on a workstation CPU, while only Section 5 latency measurements were collected on the Raspberry Pi.Training used a single NVIDIA GeForce RTX 3060 Ti, and individual trials took about five minutes.
- Uncertainty analysis: The confidence interval in Section 6 uses paired bootstrap resampling of 50 BIPEDv2 test images over 2,000 resamples.The procedure recomputes each policy’s mean ODS and re-selects the operating threshold for every resample; it measures variation across test images only, not training seeds.
- Licences: The released models are restricted to research use because the deployed edge network is trained on BIPEDv2, which permits non-commercial use only.
- Broader impact: The prototype is not certified against firefighting equipment standards and requires further testing and development before operational use.
D Output type and quantization cost
Quantization affects the two pipeline stages differently because their outputs have different numerical structure and dynamic range.
- Quantization cost: Uint8 quantization preserves the near-binary edge map more readily than the dehazer’s full-RGB radiance output.Per-tensor scales spanning the color range leave less resolution per level for the dehazer.
- Quantization cost: Quantization cost must therefore be measured separately for the edge detector and dehazer.