PaperlayerView exact revision on arXiv

Source-linked AI summary

Joint End-to-End Training of Weighted BP Decoder Ensembles

Felix Krieg, Paul Bezner, Stephan ten Brink

arXiv:2610.33559v1cs.ITcs.LGeess.SPmath.IT

TL;DR

Ensemble decoding seeks reliability through complementary component-decoder failures, while existing approaches commonly obtain diversity from code structure or decoder modifications. This paper jointly trains WBP ensembles and finds best-member training most effective. The resulting ensemble improves over the considered BP-based baselines through specialization, despite higher standalone FER for every trained member.

  • Problem

    Existing ensemble approaches commonly obtain diversity from code-specific structure or targeted decoder modifications, motivating a way to learn complementary behavior through training.

    Source: §I · pp. 1–2

  • Method

    The paper jointly trains WBP decoder ensembles and compares four objectives that distribute per-frame gradients differently across members.

    Source: §I · pp. 1–2

  • Results

    At FER 10^-4, the E=10 PEG ensemble gains approximately 0.66 dB over a single BP decoder, and best-member training achieves the lowest FER among considered BP-based methods across the evaluated SNR range.

    Source: §I · p. 2, §V · p. 6

  • Takeaways & Limitations

    The ensemble gain is associated with member specialization and complementary failure events rather than uniformly improved component decoders.

    Source: §I · p. 2

    Every trained member has a higher standalone FER, despite the improved ensemble FER.

    Source: §I · p. 2

Abstract

from arXiv · show

Ensemble decoding improves decoding reliability by combining component decoders with complementary failure events. Existing approaches typically obtain this diversity from code-specific structure, such as code automorphisms, multiple parity-check representations, or targeted modifications of the message-passing graph. In this work, we instead learn the diversity of an ensemble of weighted belief propagation decoders by jointly optimizing trainable Tanner-graph edge weights. A central design question is how the training objective should distribute gradient among the ensemble members to promote complementary decoding behavior. To this end, we compare four ensemble objectives under direct gradient-based optimization and identify best-member training as the most effective approach. For a (64,32) progressive edge-growth (PEG) low-density parity-check (LDPC) code, the resulting ensemble outperforms the considered trained and untrained belief-propagation-based ensemble baselines. With 10 members, it provides a gain of approximately 0.66 dB over a single belief propagation decoder at a target frame error rate (FER) of $10^{-4}$. Similar behavior is observed for a (128,64) CCSDS LDPC code, indicating that the learned complementarity is not specific to PEG. Although the ensemble performance improves substantially, all trained members show a higher standalone FER. The gain therefore results from learned specialization and complementary failure events rather than uniformly improved component decoders.

I. INTRODUCTION

The paper asks whether joint gradient-based training can create complementary behavior in weighted belief propagation (WBP) ensembles without code- or decoder-specific expert knowledge. It finds that best-member training is most effective, improving ensemble performance through specialization even as each member’s standalone FER worsens.

I. INTRODUCTION

  • Conventional ensemble diversity uses multiple parity-check representations, code automorphisms, or message-passing modifications, while WBP trains decoder parameters [5],.Ensemble decoding combines constituent decoders and selects the most likely candidate from their output list [1].Source: §I · p. 1
  • The proposed ensemble learns complementary member behavior directly from data, without code- or decoder-specific expert knowledge.The approach jointly optimizes member behavior rather than relying on prescribed or sequential training subsets.Source: §I · pp. 1–2
  • The paper compares four ensemble-training objectives and studies their relationship to final ML-in-list FER and their behavior under direct gradient-based optimization.The central question is whether the objective itself can induce member specialization.Source: §I · pp. 1–2
  • Objectives routing each training sample to a selected member outperform objectives that update the entire ensemble simultaneously.The best-member objective assigns each frame to its currently lowest-loss member.Source: §I · pp. 1–2
  • At FER 10^-4, the E=10 PEG ensemble uses 0.23 dB lower SNR than the specialized ensemble baseline and gains approximately 0.66 dB over a single BP decoder.The gains are reported for the PEG(64, 32) code.Source: §I · p. 2
  • The trained members have lower-overlap failure sets and improved ensemble FER, although every member’s standalone FER degrades.The authors attribute the gain to member specialization and complementary failure events, not uniformly improved components.Source: §I · p. 2

II. PRELIMINARIES

The preliminaries establish the notation for code positions and ensemble members, then specify the binary transmission setting and the basis for FER estimates.

A. Code, channel, and notation

  • The index j denotes a code-bit position, e indexes one of E ensemble members, and an indicator equals one when its argument is true.The notation also uses bold lowercase and uppercase for vectors and matrices, respectively.Source: A. Code, channel, and notation · p. 2
  • The setup uses a binary linear (n, k) code and maps coded bits to BPSK symbols for transmission over an AWGN channel.The coded-bit mapping is x_j = 1 − 2c_j.Source: A. Code, channel, and notation · p. 2
  • FER estimates are based on at least 200 observed frame errors.Source: A. Code, channel, and notation · p. 2

B. Weighted belief-propagation decoding

WBP adds trainable weights to conventional BP’s message-passing graph. The ensemble runs related decoders on the same observation and makes its final choice from valid candidates using the channel metric.

B. Weighted belief-propagation decoding

  • WBP retains conventional BP message passing while adding trainable multiplicative weights to the decoding graph.The decoder’s channel-contribution and exchanged-message weights are shared across iterations; unit weights recover untrained BP.Source: B. Weighted belief-propagation decoding · p. 2

C. Ensemble decoding

  • Ensemble members decode the same channel observation, and their diversity comes from differences in parameters and resulting decoding trajectories.Each member has its own parameter vector, posterior LLRs, and candidate.Source: C. Ensemble decoding · p. 2
  • The final decision selects among valid candidates using the channel metric; for BPSK over AWGN, this is ML selection among the list.If the list is empty, the decoder outputs a frame error.Source: C. Ensemble decoding · p. 2

D. Reference methods and experimental setup

The experiments center on the PEG and CCSDS LDPC codes, with up to ten recurrent WBP members and comparisons against single-decoder, ensemble, and approximate-ML references.

D. Reference methods and experimental setup

  • Experiments use the (64, 32) PEG LDPC code and additionally test the (128, 64) CCSDS LDPC code, with ensembles of up to E=10 recurrent WBP decoders.Each member has 256 trainable parameters, with one weight per channel-LLR contribution and exchanged message, shared across iterations.Source: D. Reference methods and experimental… · p. 2
  • Training uses ten decoding iterations, while evaluation uses 25; objective and training-rule comparisons are performed at 4.0 dB before evaluation across the considered SNR range.The evaluation iteration count matches the setup of.Source: D. Reference methods and experimental… · p. 3
  • The single-WBP reference is conventionally trained with bit-wise BCE to distinguish ensemble complementarity from gains obtainable by optimizing one decoder.Source: D. Reference methods and experimental… · p. 3
  • Other references include the specialized BP-RNN ensemble, boosting-based training, row-boosted BP ensembles, and order-3 OSD.OSD-3 is an approximate-ML reference, not a matched-complexity baseline for WBP ensembles.Source: D. Reference methods and experimental… · p. 3

III. TRAINABLE WEIGHTED BP ENSEMBLES

The ensemble objective distributes each frame’s per-member BCE gradients across decoders, with four assignment rules determining which members contribute. These rules range from equal weighting to selection by ensemble decision, metric, or member loss.

A. Ensemble training objective

  • Each frame’s per-member loss is bit-wise BCE, which decreases as posterior LLRs move toward the transmitted-bit signs with increasing magnitude.The ensemble objective combines these losses, while detached assignment coefficients control gradient distribution without receiving gradients themselves.Source: A. Ensemble training objective · p. 3

B. Candidate member assignments

  • The uniform BCE assignment weights every member equally for each frame, without explicitly encouraging complementary decoding behavior.It corresponds to independently training all members on the same channel distribution.Source: B. Candidate member assignments · p. 3
  • The metric-selected assignment gives the frame to the member whose candidate the final ensemble selector returns; only that member receives gradient.If the candidate list is empty, the assignment uses e*=0.Source: B. Candidate member assignments · p. 3
  • The soft metric objective is a differentiable relaxation of member assignment, assigning zero weight to invalid candidates when at least one member converges.The experiments use T = 1.Source: B. Candidate member assignments · p. 3
  • The best-member objective assigns each frame’s gradient to the member with the lowest current loss, the member closest to decoding that noise sample.This assignment is analogous to the hard winner assignment in adaptive mixtures of experts.Source: B. Candidate member assignments · p. 3

C. Ensemble-training

The ensemble is trained jointly with a direct Adam procedure, in which selected per-frame losses are averaged over each batch. Independent initialization perturbations are used to break member symmetry.

C. Ensemble-training

  • Each optimizer step decodes fresh channel realizations with all members, averages selected per-frame losses across the batch, and applies one joint Adam update.A member selected by no frame receives zero gradient in that update.Source: C. Ensemble-training · p. 3
  • Figure 2 plots ensemble FER against loss for 10 PEG-code WBP members, showing interpolation circles, direct-Adam squares, and gray local perturbations.The interpolation runs from the common initialization toward the selected best-member configuration.Source: Figure 2 · p. 4, C. Ensemble-training · p. 4
  • Independent Gaussian perturbations with standard deviation σ = 0.01 are added to unit BP weights to avoid symmetric initialization.Identical members under permutation-symmetric objectives would otherwise receive identical gradients; hard-assignment tie breaking could depend on implementation.Source: C. Ensemble-training · p. 4

IV. RESULTS

The section examines how ensemble objective values relate to FER and how the objectives are optimized. Uniform BCE loss is negatively associated with ensemble FER, while best-member loss has the strongest reported positive correlation.

A. Loss value versus ensemble FER

  • Figure 2 evaluates configurations along an interpolation toward the genie-selected best-member checkpoint and compares them with direct Adam configurations and local perturbations.The interpolation is a controlled parameter-space path, not an optimizer-generated trajectory.Source: A. Loss value versus ensemble FER · p. 4
  • Figure 3 plots validation FER versus training steps for direct Adam optimization of four objectives from a common jittered initialization.The experiments also evaluate checkpoints on a fixed validation set, with 25 decoding iterations.Source: Figure 3 · p. 4
  • The uniform BCE objective has an anticorrelation with FER (r = −0.69), so low average member loss can coincide with comparatively high ensemble FER.Averaging member losses does not distinguish improvements on the same frames from improvements on complementary error events.Source: A. Loss value versus ensemble FER · p. 4
  • Best-member loss has the strongest positive correlation with ensemble FER (r = +0.94), followed by soft metric (r = +0.63) and metric-selected (r = +0.22).The metric-selected objective’s hard selection can switch candidates abruptly as decoder weights change, while ensemble FER changes only marginally.Source: A. Loss value versus ensemble FER · p. 4

B. Optimization with candidate objectives

The four objectives produce different optimization trajectories, and direct best-member optimization can substantially reduce ensemble FER. Scalar loss–FER correlation alone does not predict optimization performance.

B. Optimization with candidate objectives

  • Uniform-BCE validation FER increases, soft-metric FER falls slightly, and the two hard-decision losses perform best.The hard-decision loss outcomes are not distinguished further in the supplied passage.Source: B. Optimization with candidate… · p. 4
  • Training uses 10 decoding iterations but evaluates checkpoints with 25, a mismatch that may explain degradation during longer training.Each update uses fresh channel realizations, so that degradation is not attributed to overfitting a finite training set.Source: B. Optimization with candidate… · p. 5
  • Direct best-member optimization substantially reduces ensemble FER, while metric-selected training also performs strongly despite its weak Pearson coefficient.The loss–FER correlation describes an objective but does not by itself predict how well it optimizes.Source: B. Optimization with candidate… · p. 5, Figure 4 · p. 5

C. Training rules under matched data exposure

The jointly trained best-member ensemble achieves the lowest FER among considered BP-based methods across the evaluated SNR range. At ensemble size 10, its required SNR is 0.23 dB below D10 at FER 10^-4.

C. Training rules under matched data exposure

  • Optimizer sample exposure counts each frame contributing to a gradient update once, without scaling joint updates by ensemble size.Training uses 10 iterations and evaluation uses 25; the authors retain this mismatch for comparability despite possible degradation for some objectives.Source: C. Training rules under matched data… · p. 5
  • The best-member ensemble achieves the lowest FER among considered BP-based methods throughout the evaluated SNR range.The comparison uses checkpoints selected by minimum FER on the PEG code after training.Source: Figure 5 · p. 5, C. Training rules under matched data… · p. 5
  • With E = 10, best-member training reduces validation FER rapidly and reaches the lowest value among considered learned methods for both codes.The setup uses 20,000 joint updates, or 2.56 · 10^6 training samples, with minibatches of 128 at 4.0 dB.Source: C. Training rules under matched data… · p. 5
  • At E = 10, required SNR is 5.02 dB for the proposed method versus 5.25 dB for D10 at FER 10^-4.The required SNR decreases rapidly with increasing ensemble size and remains below untrained specialist ensembles for all considered sizes.Source: Figure 6 · p. 6, C. Training rules under matched data… · p. 6

D. Member specialization and ensemble complementarity

Joint training improves ensemble FER while worsening each member’s standalone FER, alongside substantially reduced overlap between member failure sets. The best-member assignment promotes this specialization by updating the lowest-loss member for each frame.

D. Member specialization and ensemble complementarity

  • At 4.0 dB, mean Jaccard overlap across all 45 member pairs falls from 0.885 to 0.315 after training.The reduced overlap indicates substantially less overlap between member failure sets.Source: D. Member specialization and ensemble… · p. 6
  • At 4.0 dB, ensemble FER improves from 8.09·10^-3 to 2.80·10^-3 even as the members’ standalone FER increases.Fig. 7 distinguishes standalone-member FER markers from dashed lines for the complete ensemble.Source: Figure 7 · p. 6, D. Member specialization and ensemble… · p. 6
  • Best-member assignment promotes specialization by updating each frame’s lowest-loss member rather than improving all members jointly.The ensemble can benefit from complementary frame-specific behavior despite degraded standalone performance.Source: D. Member specialization and ensemble… · p. 6

V. CONCLUSION

The conclusion identifies best-member training as the strongest investigated objective and reports improved ensemble FER despite worse standalone FER for every member. Reduced failure-set overlap supports learned specialization and complementary error events as the source of the gain.

V. CONCLUSION

  • At FER 10^-4, the E = 10 ensemble requires 0.23 dB less SNR than the published specialized ensemble.It also achieves the lowest FER among considered BP-based methods throughout the evaluated SNR range.Source: §V · p. 6
  • At 4.0 dB, ensemble FER improves by a factor of 2.9 although every trained member’s standalone FER increases.The conclusion attributes the gain to learned specialization and complementary error events, not uniformly improved constituent decoders.Source: §V · p. 6
  • Best-member training achieves the strongest correlation and lowest trained FER, but metric-selected training performs competitively despite weak correlation.Thus, scalar loss–FER correlation alone does not predict trainability.Source: §V · p. 6

Related papers

Loading 2610.33559v1…