PaperlayerView exact revision on arXiv

Source-linked AI summary

Model-Driven Deep Learning for Distributed Detection with Low-Bit Quantization

Wei Guo, Meng He, Chuan Huang, Hengtao He, Shenghui Song, Jun Zhang, Khaled B. Letaief

arXiv:2610.21759v1cs.ITmath.IT

TL;DR

Distributed detection must trade communication cost against accuracy, and existing approaches may rely on unavailable observation statistics. The paper develops model-driven probabilistic quantizers and analytical MAP fusion for low-bit messages. Its analysis establishes a shared-quantizer Chernoff-information result, while the reported finite-sensor surrogate comparison shows a remaining gap between bounds and exact MAP error.

  • Problem

    Low-bit quantization can degrade distributed detection, while existing designs generally assume known observation distributions that may be unavailable.

    Source: §I · pp. 1–2

  • Method

    The framework combines probabilistic binary and tree-structured multi-bit quantizers with model-driven training and analytical MAP fusion based on class-conditional message PMFs.

    Source: §I · p. 2

  • Results

    For the evaluated M = 2, 3 and K = 5, 10, 20 settings, the optimized Chernoff bound to exact MAP error ratio ranges from 1.915 to 3.013.

    Source: A. Detection Performance across… · p. 10

  • Takeaways & Limitations

    Under conditionally i.i.d. observations, independent local randomizations, and a common quantization-level constraint, identical quantizers preserve the supremum of average network Chernoff information.

    Source: §VII · p. 14

Abstract

from arXiv · show

Distributed detection plays a fundamental role in communication-efficient collaborative inference over large-scale networks, where spatially distributed nodes must cooperate using severely quantized messages due to bandwidth constraints. While low-bit quantization is essential for reducing communication overhead, it can significantly degrade detection performance, particularly when prior knowledge of observation noise is unavailable. To address this challenge, we propose a model-driven deep learning framework for distributed detection with low-bit quantization. For fixed local quantizers, we first derive the exact finite-sensor maximum a posteriori (MAP) detection error and prove that, under conditionally independent and identically distributed (i.i.d.) observations, independent local randomizations, and a common quantization-level constraint, identical quantizers preserve the supremum of the average network Chernoff information. Guided by these results, we develop a binary probabilistic quantizer with a trainable neural probability controller and design a tree-structured probabilistic quantizer for scalable multi-bit quantization. The exact finite-sensor MAP error provides the training objective for the binary quantizer, while we derive a computationally tractable Bhattacharyya surrogate objective for training the multi-bit quantizer. For both cases, we derive analytical MAP fusion rules based on the class-conditional probability mass functions (PMFs) of the quantized messages. When the observation statistics are unknown, these PMFs are estimated from held-out labeled data, yielding plug-in MAP detectors. Simulation results show that the proposed method approaches centralized-oracle performance at higher quantization resolutions and remains effective under changes in network size and test-time signal amplitude.

I. INTRODUCTION

Distributed detection must balance low communication cost against detection accuracy, while existing designs can depend on unavailable observation statistics. The paper develops a model-driven approach for quantized sensor messages, grounded in finite-sensor MAP analysis and a shared-quantizer structure.

I. INTRODUCTION

  • Distributed inference must operate under bandwidth and energy constraints, while aggressive quantization can degrade detection performance.The paper positions this accuracy–communication trade-off as a central challenge for distributed detection.Source: §I · p. 1
  • Existing distributed-detection work emphasizes performance analysis and bounds, while current quantizer designs generally assume known hypothesis priors and observation distributions.The paper identifies scalable algorithm design under unknown observation statistics as a gap.Source: §I · pp. 1–2

A. System Model

  • The network comprises K sensors that observe a binary hypothesis through noisy local observations and send finite-alphabet messages to a fusion center.Each sensor uses a common M-bit message constraint, for a total payload of KM bits per detection instance.Source: A. System Model · pp. 2–3

B. Problem Formulation

  • For fixed local quantizers, the MAP detector minimizes finite-sensor detection error, which depends on the quantizers through their induced class-conditional message distributions.This yields an exact finite-sensor objective for quantizer design.Source: B. Problem Formulation · p. 3
  • Under conditionally i.i.d. observations, independent local randomizations, and a common L-level constraint, identical quantizers preserve the supremum of average network Chernoff information.The paper consequently adopts a shared quantizer and optimizes finite-sensor MAP error within that architecture.Source: B. Problem Formulation · p. 3

III. MODEL-DRIVEN DL FOR BINARY QUANTIZATION AND DETECTOR DESIGN

The binary design uses a shared probabilistic quantizer whose neural controller is trained against the exact finite-sensor MAP detection error. Random dithering turns controller probabilities into hard binary messages for transmission.

A. Binary Quantizer Design

  • For conditionally i.i.d. binary messages, the exact MAP detection error depends on the quantizer through the class-conditional probability of message 1.This permits learning the output probability P(u = 1|X) rather than directly optimizing a deterministic decision rule.Source: A. Binary Quantizer Design · p. 4
  • The binary quantizer maps each observation through a probability controller and random dithering to produce a hard binary symbol.The dithering scheme yields P(u = 1|X) = G(X), with the complementary probability for message 0.Source: A. Binary Quantizer Design · p. 4
  • A neural probability controller is trained by directly minimizing the exact binary MAP detection error.The controller approximates the optimal probability mapping from local observations to binary-message probabilities.Source: A. Binary Quantizer Design · p. 4

B. Analytical MAP Detector Design

With conditionally i.i.d. binary messages, the fusion center can make the MAP decision from their average rather than retaining every individual message. The resulting rule uses class-conditional message probabilities.

B. Analytical MAP Detector Design

  • Using the average binary message achieves the same minimum detection error as using all individual messages when messages are conditionally i.i.d. given H.The average is equivalent to the count s of sensors transmitting 1, so it can serve as the detector input.Source: B. Analytical MAP Detector Design · p. 4
  • When the posterior based on the message average is unavailable, the rule is expressed using the class-conditional probabilities of the message count.These low-dimensional probabilities can subsequently be estimated from labeled data.Source: B. Analytical MAP Detector Design · pp. 4–5
  • The fusion center implements the MAP decision using the class-conditional Bernoulli probabilities induced by the quantizer.The decision rule is expressed through log-posterior odds, and the average message is sufficient to evaluate it.Source: B. Analytical MAP Detector Design · p. 5

IV. MODEL-DRIVEN DL FOR MULTI-BIT QUANTIZATION AND DETECTOR DESIGN

The multi-bit design uses a tree-structured probabilistic quantizer to produce quantization levels from M-bit codewords. It trains the controller with a tractable Bhattacharyya surrogate because exact finite-sensor MAP error requires costly enumeration of message histograms.

A. Tree-based Quantizer Design

2) Induced Joint Distribution over Codewords:

3) Codeword Generation with Sequential Dithering:

  • Sequential dithering generates the hard M-bit output by traversing the tree, and its output distribution exactly matches the model-induced distribution used in training.The deployment procedure draws independent dithering variables as it generates bits along the root-to-leaf path.Source: 3) Codeword Generation with Sequential… · pp. 5–6
  • For conditionally i.i.d. sensor outputs, the multi-bit MAP error is computed from the multinomial distribution of level-count histograms across sensors.Evaluating the exact error requires enumerating feasible count vectors; their number grows with sensor count and increases sharply as the number of quantization levels grows.Source: 3) Codeword Generation with Sequential… · p. 6
  • The Bhattacharyya objective provides a computationally tractable surrogate for training the tree-structured quantizer, while the exact finite-K MAP error is intractable to evaluate directly.Its optimized Chernoff counterpart characterizes the asymptotic Bayesian error exponent; at finite K, the Bhattacharyya criterion is an upper-bound surrogate, not the exact MAP error.Source: 3) Codeword Generation with Sequential… · pp. 6–7

B. Analytical MAP Detector Design

For conditionally i.i.d. multi-bit messages, the analytical MAP detector can use the mean one-hot vector instead of retaining all received codewords. Its rule uses class-conditional PMFs of quantization levels.

B. Analytical MAP Detector Design

  • Using the full collection of M-bit codewords or their mean one-hot vector achieves the same minimum detection error under conditionally i.i.d. messages.The count vector of quantization levels is K times the mean one-hot vector.Source: B. Analytical MAP Detector Design · p. 7
  • The analytical MAP rule uses level-wise log-likelihood weights derived from the class-conditional PMFs and the received level-count vector.The multinomial coefficient cancels from the log-posterior odds.Source: B. Analytical MAP Detector Design · p. 7
  • When K is known, the mean one-hot vector is a fixed-dimensional sufficient statistic; with unequal deployment priors, K must also be retained.K determines the relative contribution of accumulated evidence and prior log-odds.Source: B. Analytical MAP Detector Design · p. 7

V. CONTROLLER TRAINING AND POST-TRAINING PMF ESTIMATION

The method separates controller training from detector deployment: after training, it estimates the frozen quantizer’s class-conditional PMFs from independent labeled data and uses them in the analytical MAP rule.

V. CONTROLLER TRAINING AND POST-TRAINING PMF ESTIMATION

  • After controller training, independent labeled data estimate the class-conditional PMFs of quantized outputs, which instantiate the analytical MAP rule for deployment.The estimated PMFs are based on the trained, fixed controller.Source: §V · p. 7

A. Probability Controller Training

The controller is trained from class-balanced labeled samples using differentiable output PMFs rather than hard quantized messages. After training, held-out data estimate the frozen quantizer’s class-conditional PMFs for plug-in MAP fusion.

A. Probability Controller Training

  • Training uses controller-induced conditional output PMFs rather than sampled hard quantization, avoiding non-differentiability while remaining consistent with deployment.Mini-batches are class-balanced, and validation selects the controller checkpoint.Source: A. Probability Controller Training · pp. 7–8
  • For binary quantization, empirical class-conditional probabilities feed the exact finite-K MAP error objective, which is differentiated through the probability controller.The empirical PMFs remain differentiable functions of the controller parameters.Source: A. Probability Controller Training · pp. 7–8
  • For multi-bit quantization, empirical PMFs over levels feed the Bhattacharyya surrogate without enumerating all feasible count vectors.The level PMFs can be estimated without sampling hard codewords.Source: A. Probability Controller Training · p. 8

B. Post-Training PMF Estimation and Plug-In MAP Fusion

  • Post-training PMF estimation uses independent data after the controller is fixed and does not calibrate or modify its outputs.The estimator applies symmetric smoothing, which ensures positive estimated probabilities and prevents unbounded log-likelihood ratios.Source: B. Post-Training PMF Estimation and… · p. 8
  • Estimated PMFs yield a plug-in MAP rule that coincides with the analytical detector if the estimates equal the true class-conditional PMFs.Otherwise, it provides a data-driven implementation under unknown observation statistics.Source: B. Post-Training PMF Estimation and… · p. 8

VI. SIMULATION RESULTS

The simulations evaluate a random-polarity signal-presence task across network sizes and message resolutions, comparing learned quantizers with classical designs and an unquantized oracle. The proposed method performs well at low rates, approaches oracle error at higher resolutions, and remains effective when observation statistics are unknown.

VI. SIMULATION RESULTS

  • The experiment models signal presence with an equiprobable random-polarity component and Gaussian noise, yielding conditionally i.i.d. observations.Controllers are trained with Ktrain = 20 sensors and frozen across deployment sizes; held-out data estimate the class-conditional PMFs for plug-in MAP fusion.Source: §VI · pp. 8–9

A. Detection Performance across Network Sizes

  • At K = 20 with true random-polarity PDFs, one-bit error is 0.0442 for the proposed method versus 0.1435 for Fast Gaussian and 0.1339 for Optimal Intervals.With two-bit messages, the corresponding errors are 0.0251, 0.0441, and 0.0354; the proposed controller can reuse a label over both tails.Source: A. Detection Performance across… · pp. 9–10, Figure 3 · p. 9
  • At K = 20 and M = 4, error probabilities are 0.0209 for the proposed method, 0.0210 for Optimal Intervals, and 0.0197 for the centralized oracle.At K = 50, the corresponding errors are 6.4 × 10−4, 6.7 × 10−4, and 5.95 × 10−4; the proposed-oracle gap becomes small at three and four bits.Source: A. Detection Performance across… · p. 10
  • Across M = 2, 3 and K = 5, 10, 20, the optimized Chernoff bound is 1.915–3.013 times the exact MAP error, while optimizing λ reduces the Bhattacharyya bound by only approximately 0.2%–1.3%.These results support using the training objective as a conservative upper-bound surrogate for finite-sensor MAP error.Source: A. Detection Performance across… · p. 10
  • At K = 20 with unknown observation statistics, error probabilities are 0.0442 for the proposed method versus 0.1675 for Fast Gaussian and 0.1509 for Optimal Intervals at M = 1.At M = 4, the respective errors are 0.0209, 0.0579, and 0.0266.Source: A. Detection Performance across… · p. 10, Figure 4 · p. 10
  • The learned controller incurs offline training and sensor-side neural computation, while quantized fusion remains approximately O(K) and all methods use the same M-bit communication cost per sensor.This trades additional computation for learning from samples without an explicitly specified observation model.Source: A. Detection Performance across… · pp. 10–11

B. Learned Controller Structure

The learned controllers form soft, increasingly fine partitions that approximate the two-sided likelihood structure, while fusion-stage updates address deployment mismatch without retraining the controller. Ablations favor the proposed tree-structured and complete MAP-guided pipelines in the tested settings.

B. Learned Controller Structure

  • For M=1, the learned controller assigns one label to both tails and another to the center; multi-bit tree leaves create finer, approximately symmetric soft partitions.The resulting codeword weights approximate the two-sided likelihood structure and preserve detection-relevant ordering in |x|.Source: B. Learned Controller Structure · pp. 11–12

C. Robustness Under Non-Ideal Deployment

  • Re-estimating FC output PMFs at the target amplitude recovers mismatch-related performance without retraining or changing the frozen local controller.At three and four bits, the recalibrated detector approaches the unquantized oracle in the reported settings.Source: C. Robustness Under Non-Ideal Deployment · p. 12
  • At amplitude spread δ=0.4, sensor-specific likelihood weights reduce error from 0.0492, 0.0265, 0.0222, and 0.0217 to 0.0407, 0.0214, 0.0181, and 0.0172 for M=1 through 4.The shared controller remains frozen; the fusion center uses sensor-specific output PMFs, and the remaining oracle gap is small at three and four bits.Source: C. Robustness Under Non-Ideal Deployment · pp. 12–13
  • At π1=0.1, updating the MAP prior term lowers the M=4 weighted error from 0.0180 to 0.0103, near the unquantized-oracle value 0.00979.At π1=0.9, it lowers error from 0.0243 to 0.0129, versus 0.0122 for the oracle; the local controller and class-conditional PMFs stay fixed.Source: C. Robustness Under Non-Ideal Deployment · p. 13, Figure 9 · p. 13

D. Ablation Study

  • At K=50 and M=4, errors were 6.4×10−4 for Tree + MAP, 8.15×10−4 for Categorical + MAP, and 3.69×10−3 for End-to-end MLP.At K=20 and M=4, Tree + MAP had error 0.0209 versus 0.0445 for End-to-end MLP; the comparison supports the complete MAP-guided design, not either component alone.Source: D. Ablation Study · p. 13

VII. CONCLUSION

  • The conclusion reports that the method outperforms considered classical quantizer designs on the non-Gaussian random-polarity task and supports lightweight fusion-rule adaptation to deployment changes.It also reports a tree-controller benefit over direct categorical control at three and four bits in the tested settings.Source: §VII · p. 14

APPENDIX A PROOF OF LEMMA 1

The proof bounds the average network Chernoff information by the best single-sensor value, then shows identical deployment attains that bound under the stated conditions. The binary MAP proof uses the count of positive messages.

APPENDIX A PROOF OF LEMMA 1

  • Conditional independence makes the quantized messages conditionally independent, allowing the average network Chernoff information to be bounded by the best single-sensor value under the common level constraint.Deploying an attaining single-sensor quantizer identically at every sensor reaches the bound, preserving the supremum.Source: Appendix A · p. 14
  • The proof constructs identical sensor quantizers with independent local randomizations, so their induced conditional message distributions are identical.This construction establishes the matching lower bound for the shared-quantizer supremum.Source: Appendix A · p. 14

APPENDIX B PROOF OF PROPOSITION 1

  • For binary quantization under conditionally i.i.d. observations, message vectors with the same number of ones have equal conditional probability, so the MAP error can be expressed using that count.The appendix completes the proof of the binary-quantization MAP error result.Source: Appendix B · p. 15

APPENDIX C PROOF OF THEOREM 1

The appendix establishes that the binary-message average retains the minimum MAP error of the full message vector. It also verifies the tree quantizer’s sequential dithering probabilities for each output level.

APPENDIX C PROOF OF THEOREM 1

  • The conditional distribution of the binary-message average yields the same minimum detection error as the full quantized data.The appendix concludes PE(H, u) = PE(H, ū).Source: Appendix C · p. 15

APPENDIX D PROOF OF PROPOSITION 2

  • The sequential dithering proof follows the tree traversal and its generated bit prefixes to establish each quantization-level output probability.It concludes that the probability of a level output given X matches the probability of its corresponding bit sequence.Source: Appendix D · p. 15

APPENDIX E PROOF OF THEOREM 2

The proof groups multi-bit message vectors by their level-count histogram, showing that detection from this histogram retains the minimum error of detection from the full messages.

APPENDIX E PROOF OF THEOREM 2

  • The full message collection and the count vector yield the same minimum detection error after grouping vectors with identical counts.The count vector and average message vector are equivalent representations when K is fixed.Source: Appendix E · pp. 15–16
  • Under conditionally i.i.d. observations, message vectors with the same level-count vector have identical likelihood, so grouping by counts preserves the MAP error calculation.The proof notes that each count vector corresponds one-to-one with the average message vector.Source: Appendix E · pp. 15–16
  • The histogram’s level counts determine the grouped likelihood through the class-conditional probability of each quantization level raised to its count.The proof expresses the grouped likelihood using the level probabilities and counts.Source: Appendix E · pp. 15–16
Loading 2610.21759v1…