PaperlayerView exact revision on arXiv

Source-linked AI summary

Task-Oriented Communication for Multi-Device Cooperative Edge Inference

Jiawei Shao, Yuyi Mao, Jun Zhang

arXiv:2109.00172v3eess.SPcs.LG

TL;DR

Cooperative edge inference can overwhelm communication links when multiple devices transmit data, while distributed feature encoding lacks a rigorous rate–inference tradeoff characterization. The paper combines information-bottleneck feature extraction, distributed encoding, and selective retransmission, achieving higher accuracy and a better rate–relevance tradeoff than evaluated alternatives.

  • Problem

    Raw-data transmission creates prohibitive communication overhead, and distributed feature encoding lacks a rigorous characterization of its communication cost–inference performance tradeoff.

  • Method

    The scheme uses IB for task-relevant feature extraction, DIB and DDIB for distributed encoding, variational approximations, and selective retransmission.

  • Results

    Experiments show higher accuracy and a better rate–relevance tradeoff, while the proposed scheme substantially reduces communication overhead versus traditional data-oriented communication.

  • Takeaways & Limitations

    Task-oriented communication provides a practical basis for reducing communication overhead and enabling low-latency multi-device cooperative inference.

  • Takeaways & Limitations

    Variational approximations mean the optimized distributed encoding scheme is not guaranteed to achieve the theoretically optimal rate–relevance tuples.

Abstract

from arXiv · show

This paper investigates task-oriented communication for multi-device cooperative edge inference, where a group of distributed low-end edge devices transmit the extracted features of local samples to a powerful edge server for inference. While cooperative edge inference can overcome the limited sensing capability of a single device, it substantially increases the communication overhead and may incur excessive latency. To enable low-latency cooperative inference, we propose a learning-based communication scheme that optimizes local feature extraction and distributed feature encoding in a task-oriented manner, i.e., to remove data redundancy and transmit information that is essential for the downstream inference task rather than reconstructing the data samples at the edge server. Specifically, we leverage an information bottleneck (IB) principle to extract the task-relevant feature at each edge device and adopt a distributed information bottleneck (DIB) framework to formalize a single-letter characterization of the optimal rate-relevance tradeoff for distributed feature encoding. To admit flexible control of the communication overhead, we extend the DIB framework to a distributed deterministic information bottleneck (DDIB) objective that explicitly incorporates the representational costs of the encoded features. As the IB-based objectives are computationally prohibitive for high-dimensional data, we adopt variational approximations to make the optimization problems tractable. To compensate the potential performance loss due to the variational approximations, we also develop a selective retransmission (SR) mechanism to identify the redundancy in the encoded features of multiple edge devices to attain additional communication overhead reduction. Extensive experiments evidence that the proposed task-oriented communication scheme achieves a better rate-relevance tradeoff than baseline methods.

I. INTRODUCTION … II. SYSTEM MODEL AND PROBLEM DESCRIPTION

The paper develops task-oriented communication for multi-device cooperative edge inference, combining IB-based local feature extraction with distributed feature encoding to reduce communication overhead while preserving task-relevant information. It formulates the system model and two associated design problems, then introduces tractable variational objectives and selective retransmission to balance communication cost and inference performance.

  • I. INTRODUCTION: Conventional edge inference transmits enormous volumes of raw data to an edge server, motivating device-edge co-inference that sends task-relevant features instead.Task-oriented communication avoids reconstructing original inputs when nuisance information is irrelevant to inference.
  • A. Related Works and Motivations: Distributed coding can remove redundancy among correlated device sources, but prior work lacks a rigorous communication-cost versus inference-performance characterization for distributed feature encoding.The DIB framework provides a single-letter rate-relevance characterization using mutual information terms.
  • B. Contributions: The proposed scheme formulates task-relevant feature extraction and distributed feature encoding using the IB principle and DIB framework, respectively.The local extractor discards task-irrelevant correlated information, while DDIB reformulates DIB for practical one-shot coding.
  • B. Contributions: Variational upper bounds yield tractable VIB and VDDIB objectives because the mutual-information terms in the original IB-based objectives are computationally prohibitive for high-dimensional data.These objectives support optimization of local extraction and distributed encoding in practice.
  • B. Contributions: Selective retransmission identifies redundancy among encoded features, reducing communication load and flexibly trading communication overhead against inference performance.The mechanism is introduced within the VDDIB framework to compensate for potential losses from variational approximations.
  • B. Contributions: Experiments on multi-view image classification and object recognition report substantial communication-overhead reductions versus traditional data-oriented communication and outperform existing learning-based feature encoders.The evaluations target cooperative edge inference settings.
  • C. Organization: The paper next presents the system model and IB/DIB problem formulations, followed by variational optimization, DDIB/VDDIB development, selective retransmission, and experimental evaluation.This point summarizes the paper’s stated organization.
  • D. Notations: The notation distinguishes random variables from realizations, uses X1:K for collections of variables, and defines entropy, conditional entropy, mutual information, and KL divergence.These conventions support the subsequent information-theoretic formulations.

A. Probabilistic Modeling · B. Task-Relevant Feature Extraction · C. Distributed Feature Encoding

The paper models cooperative inference through task-relevant local features and distributed encodings, optimizing the tradeoff between inference relevance and communication overhead. Information bottleneck principles and variational approximations provide the framework for tractable feature extraction and encoding.

  • A. Probabilistic Modeling: The system comprises K edge devices and an edge server, where local observations X_k and target Y generate task-relevant features Z_k for cooperative inference.The probabilistic model uses the joint distribution p(x_1, . . . , x_K, y) and represents each device through the Markov chain Y ↔ X_k ↔ Z_k ↔ U_k.
  • A. Probabilistic Modeling: The design minimizes communication overhead while maintaining satisfactory inference performance through local feature extraction and distributed feature encoding.The factorization p(u_k, z_k, x_k|y) = p(u_k|z_k)p(z_k|x_k)p(x_k|y) describes the proposed probabilistic structure.
  • B. Task-Relevant Feature Extraction: The information bottleneck principle makes each Z_k retain task-relevant information about Y while discarding superfluous content from high-dimensional X_k.The IB objective maximizes mutual information between Z_k and Y while penalizing unnecessary information in the encoded feature; variational reformulation makes it tractable.
  • B. Task-Relevant Feature Extraction: Minimal sufficient extracted features satisfy I(Y ; X_k) = I(Y ; Z_k), enabling the conditional-independence assumption required by the DIB theorem.Under these conditions, the paper shows that rate-relevance tuples can be obtained through the distributed encoding framework.
  • C. Distributed Feature Encoding: Distributed feature encoding formalizes the rate-relevance tradeoff: transmitting more task-relevant information can improve inference accuracy but increases communication overhead.The edge server jointly processes device codewords to derive the inference result, while relevance and per-device rates quantify performance and cost.
  • C. Distributed Feature Encoding: The DIB framework characterizes optimal rate-relevance tuples by minimizing a distributed information bottleneck objective when extracted features are conditionally independent given Y.The parameter β determines an optimal tuple (∆_β, R_β), balancing relevance against the sum communication rate.
  • C. Distributed Feature Encoding: DIB optimality depends on minimal and sufficient features, which may fail in general, although feature extraction can reduce correlations among devices’ representations.Discarding task-irrelevant information makes extracted features less correlated than the raw input views, supporting distributed encoding.
  • C. Distributed Feature Encoding: The paper uses variational approximations to enable tractable optimization because achieving DIB rate-relevance tuples may require an excessively large blocklength N.The approximations derive upper bounds for conditional entropy terms, reducing the computational burden of obtaining the encoding solutions.

III. VARIATIONAL APPROXIMATION FOR FEATURE EXTRACTION · A. Variational Information Bottleneck

Section III reformulates the information bottleneck objective using a variational information bottleneck framework to make feature-extraction optimization tractable. In subsection A, variational distributions approximate the intractable marginal and conditional distributions induced by each device’s feature extractor.

  • III. VARIATIONAL APPROXIMATION FOR FEATURE EXTRACTION: The variational information bottleneck framework reformulates the information bottleneck objective to facilitate tractable optimization of feature extraction.This approximation addresses the computational difficulty of directly optimizing the original objective.
  • A. Variational Information Bottleneck: For device k, the information bottleneck objective is L_IB,k(γ) := H(Y |Z_k) + γI(X_k; Z_k).The objective balances conditional uncertainty about Y given Z_k against the mutual-information cost between X_k and Z_k.
  • A. Variational Information Bottleneck: The feature-extraction process is represented by the conditional distribution p_θk(z_k|x_k), parameterized by θ_k.Under the Markov chain Y ↔ X_k ↔ Z_k, this conditional distribution determines the relevant marginal and conditional distributions.
  • A. Variational Information Bottleneck: Given p(x_k, y) and Y ↔ X_k ↔ Z_k, p(z_k) and p(y|z) are fully determined by p_θk(z_k|x_k).These distributions are the quantities needed to evaluate the information bottleneck objective.
  • A. Variational Information Bottleneck: The distributions p(z_k) and p(y|z_k) are generally intractable because of the data’s high dimensionality.Variational inference is therefore used to approximate these target distributions.
  • A. Variational Information Bottleneck: The variational approximation posits distribution families and selects members close to the target distributions.The paper introduces r_k(z_k) and p_φk(y|z_k) for this purpose.
  • A. Variational Information Bottleneck: The detailed derivation of the variational formulation is provided in Appendix B-A.This passage records the location of the derivation rather than introducing another optimization step.

B. DNN Parameterization · IV. VARIATIONAL DISTRIBUTED FEATURE ENCODING

The paper parameterizes task-relevant feature extraction and inference with DNN-based Gaussian distributions, using reparameterized sampling for differentiable optimization. It then develops variational distributed feature encoding through a DDIB objective that explicitly accounts for encoded-feature communication overhead.

  • B. DNN Parameterization: The conditional feature distribution pθk(zk|xk) is modeled as a multivariate Gaussian whose mean and standard deviation are produced by DNNs parameterized by θk.The mean is µk(xk; θk), while σk(xk; θk) determines the diagonal covariance structure.
  • B. DNN Parameterization: The variational marginal rk(zk) is chosen as the centered isotropic Gaussian N(zk|0, I), simplifying the KL-divergence term.This choice is used in the variational approximation of the feature-extraction objective.
  • B. DNN Parameterization: Reparameterized sampling uses zk = µk + σk ⊙ ϵk with ϵk sampled from N(0, I), making the Monte Carlo negative-log-likelihood estimate differentiable with respect to θk.The extracted feature is then supplied to the inference function ˆy(zk; φk).
  • B. DNN Parameterization: Inference results are obtained from the extracted feature zk through the function ˆy(zk; φk), with φk serving as the inference-network parameters.The variational conditional distribution is formulated using pφk(y|zk).
  • B. DNN Parameterization: The VIB objective is empirically estimated at each device, and its training procedures are summarized in Algorithm 1.The supplied passages identify the empirical estimation and algorithmic summary but do not include the full objective expression.
  • IV. VARIATIONAL DISTRIBUTED FEATURE ENCODING: The distributed feature encoding scheme is based on the DIB objective to support efficient task-oriented communication.The method reformulates DIB as DDIB and invokes variational approximations to make optimization tractable.
  • IV. VARIATIONAL DISTRIBUTED FEATURE ENCODING: Algorithm 1 trains task-relevant feature extraction at device k using a training dataset, minibatches, and initialized parameters.The supplied algorithm passages identify these inputs but omit the complete minibatch specification.
  • IV. VARIATIONAL DISTRIBUTED FEATURE ENCODING: Training computes LVIB,k(γ; θk, φk) based on (13), updates θk and φk through backpropagation, and continues until parameter convergence.This procedure operationalizes the variational feature-extraction objective at device k.

A. Distributed Deterministic Information Bottleneck Reformulation · B. Variational Distributed Deterministic Information Bottleneck

The paper reformulates distributed information bottleneck coding with deterministic bitstream encoders whose representational costs are directly controllable. It then introduces a variational relaxation for tractable training, while noting that approximation prevents guaranteed optimality despite strong empirical performance.

  • A. Distributed Deterministic Information Bottleneck Reformulation: The DDIB reformulation measures communication through explicit bit costs rather than mutual-information terms for each encoded feature.It selects deterministic mappings from task-relevant features Z_k to bitstreams U_k.
  • A. Distributed Deterministic Information Bottleneck Reformulation: Each device maps its task-relevant feature through f_ϕk and uniformly quantizes every element into n_k bits, producing u_k.The encoded feature is u_k := Q(f_ϕk(z_k)).
  • A. Distributed Deterministic Information Bottleneck Reformulation: The transmitted representation from device k costs R_bit(u_k) = n_kd_k bits.Uniform quantization is selected because it facilitates gradient approximation and bit-length calculation.
  • B. Variational Distributed Deterministic Information Bottleneck: Because deterministic mappings fix the relevant conditional distributions, the conditional entropy terms remain generally intractable for high-dimensional integrals.The affected terms are H(Y | U_1:K) and H(Y | U_k).
  • B. Variational Distributed Deterministic Information Bottleneck: The variational predictors use received features to predict the target, and minibatch Monte Carlo sampling estimates the VDDIB objective during training.The resulting parameters are optimized through backpropagation until convergence.
  • B. Variational Distributed Deterministic Information Bottleneck: Minimizing VDDIB does not guarantee the optimal rate-relevance tuples because both IB and DIB optimizations use approximations.The paper nevertheless reports that VDDIB notably outperforms existing communication strategies for multi-device cooperative edge inference.

V. DISTRIBUTED FEATURE ENCODING WITH SELECTIVE RETRANSMISSION · A. Selective Retransmission Mechanism · 1) Stopping Policy:

The VDDIB-SR method adds selective retransmission to identify redundancy across devices through sequential communication rounds. A confidence-based stopping policy terminates retransmissions when intermediate inference is sufficiently reliable, reducing latency while controlling the performance–communication tradeoff.

  • V. DISTRIBUTED FEATURE ENCODING WITH SELECTIVE RETRANSMISSION: VDDIB-SR extends VDDIB with selective retransmission and multiple sequential communication rounds between edge devices and the edge server.The mechanism targets redundancy in encoded features to further reduce communication overhead.
  • A. Selective Retransmission Mechanism: Each device transmits an encoded feature vector u_k for at most T attempts, corresponding to T−1 retransmissions.The encoded vector is generated deterministically from Z_k through a DNN-based feature encoder.
  • A. Selective Retransmission Mechanism: The edge server uses T independent predictors, with the t-th predictor inferring from features received after the t-th transmission.This lets the server evaluate inference progressively as additional device features arrive.
  • A. Selective Retransmission Mechanism: The stopping policy determines whether further encoded-feature transmissions are needed after each communication attempt.The policy operates on progressively received feature representations, including the transmitted-feature selection process.
  • 1) Stopping Policy:: The server’s confidence-based policy outputs confidence scores δ_τ after each transmission attempt using T−1 internal predictors.The internal predictors form a cascade-inspired structure that supports sequential termination decisions.
  • 1) Stopping Policy:: If δ_τ exceeds threshold δ_0, retransmission stops; otherwise, the server requests more features to improve inference performance.Adjusting δ_0 controls the tradeoff between inference performance and communication overhead.

2) Attention Module: · B. VDDIB-SR Objective

The attention module selects devices for selective retransmission, while the VDDIB-SR objective combines stopping, informativeness, and communication-overhead terms. Its optimization uses Monte Carlo gradient estimation, and increasing the maximum transmission attempts can improve the rate-relevance tradeoff.

  • 2) Attention Module:: Attention scores a_k,τ ∈ {0, 1} determine which edge devices are activated for retransmission based on previously received features.The scores are constructed by fully connected layers and depend on received features from earlier transmissions.
  • 2) Attention Module:: Hard binarization discretizes attention scores, while a straight-through estimator approximates their gradients during training.This enables discrete device selection within the learned attention module.
  • 2) Attention Module:: All edge devices transmit their features in the first transmission, with a_k,1 = 1 for every device.Subsequent transmissions may deactivate devices according to their attention scores.
  • B. VDDIB-SR Objective: The VDDIB-SR objective accounts for internal-predictor losses across all possible stopping points.This stopping-policy term extends the VDDIB objective.
  • B. VDDIB-SR Objective: An auxiliary loss preserves the informativeness of each encoded feature and improves robustness to dynamic activation by the attention module.The objective also measures each device’s communication overhead as R_bit(ũ_k,t) = a_k,tn_kd_k.
  • B. VDDIB-SR Objective: Monte Carlo sampling provides an unbiased gradient estimate, allowing the VDDIB-SR objective to be minimized by backpropagation.The training procedures extract task-relevant features, compute attention scores, form transmitted features, obtain intermediate inference results, and optimize the loss until convergence.
  • B. VDDIB-SR Objective: Because the VDDIB-SR parameter spaces are nested, its minimum lower-bounds the VDDIB objective, and increasing T can improve the rate-relevance tradeoff.The minimum of L_DDIB(β) over the joint-feature distributions also lower-bounds L_VDDIB-SR for any integer T.

VI. PERFORMANCE EVALUATION

The performance evaluation examines the proposed task-oriented communication methods for cooperative edge inference across two cooperative inference tasks: multi-view image classification and multi-view object recognition.

  • The evaluation assesses the proposed task-oriented communication methods for cooperative edge inference.
  • One evaluated task is multi-view image classification.
  • The other evaluated task is multi-view object recognition.

A. Experimental Setup … 5) Metrics:

The experiments evaluate VDDIB and VDDIB-SR for cooperative edge inference across multi-view image classification and object recognition, using several communication and feature-selection baselines. Performance is assessed through rate–relevance tradeoffs based on transmitted bits and classification accuracy.

  • 1) Datasets:: Experiments use MNIST, CIFAR-10, and Tiny ImageNet for multi-view image classification.MNIST has 60,000 training and 10,000 test examples; CIFAR-10 has 5,000 training and 1,000 testing images per class.
  • 1) Datasets:: WRGBD and ModelNet40 evaluate the methods on multi-view object recognition.WRGBD contains 51 household-object classes captured from three camera heights, with 19,390 training and 2,729 test objects.
  • 2) Proposed methods:: The study compares VDDIB and VDDIB-SR, with VDDIB equivalent to VDDIB-SR (T=1), while γ=10−4 and β∈[10−3,10−1] control training and rate–relevance tuples.VDDIB-SR with t transmissions is denoted VDDIB-SR (T=t).
  • 3) Baselines:: Five baselines comprise server-based inference, NN-REG, NN-GBI, eSAFS, and CAFS.The supplied setup describes these methods as data-oriented transmission, learned quantization, importance-based feature selection, or context-aware feature selection.
  • 3) Baselines:: NN-GBI uses one possible boundary per dimension to reduce its search space and improve training efficiency.The GBI quantization algorithm is computationally prohibitive when the number of bits is too large.
  • 4) Neural Network Architecture:: Each raw sample passes through convolutional NN1 for view-specific feature extraction, while fully connected NN2 processes collected features for inference.Table I reports NN1 and NN2 structures for different tasks.
  • 4) Neural Network Architecture:: Most methods uniformly quantize NN1 outputs, whereas NN-GBI uses non-uniform quantization and server-based inference transmits raw input views.NN-REG, NN-GBI, and VDDIB use vector concatenation after receiving extracted features.
  • 5) Metrics:: The rate–relevance tradeoff uses total transmitted bits, Rsum, as rate and classification accuracy as relevance, while feedback signaling costs are neglected for VDDIB-SR and eSAFS.Cross-entropy measures prediction error, and VDDIB-SR uses maximum predicted probability as its confidence score.

B. Multi-View Image Classification … VII. CONCLUSIONS

Across multi-view classification and object recognition, the proposed task-oriented methods achieve strong accuracy under rate constraints while substantially reducing communication overhead and latency. Ablation results support the roles of retransmission control and attention, while the conclusion identifies IB, DIB, variational approximations, and retransmission as promising tools for cooperative edge inference.

  • B. Multi-View Image Classification: VDDIB achieves comparable or better multi-view image classification accuracy than baselines under different average received-bit constraints.The result supports the effectiveness of the IB and DIB frameworks for task-oriented communication.
  • B. Multi-View Image Classification: Single-view inference is insufficient for good accuracy, whereas the proposed methods greatly reduce communication overhead relative to server-based inference with little performance loss.
  • C. Multi-View Object Recognition: With twelve devices providing distinct views, the proposed methods consistently outperform baselines for multi-view 3D object recognition under all rate constraints.The task recognizes 3D objects from collections of perceived 2D views.
  • C. Multi-View Object Recognition: 560 ms versus 2.5 ms: VDDIB-SR reduces ModelNet40 communication latency while maintaining comparable classification accuracy to data-oriented communication.The comparison concerns the ModelNet40 classification task.
  • D. Ablation Study: The ablation study evaluates maximum transmission attempts T and the attention module, while feature-extraction validation is deferred to Appendix D.
  • 1) Impact of the maximum transmission attempts T:: The maximum transmission attempts T affect the rate-relevance tradeoff, with joint coding outperforming distributed coding methods.The joint coding baseline gives one edge device access to all extracted view-specific features for joint encoding before transmission.
  • 2) Impact of the attention module:: With T set to 2 on ModelNet40, VDDIB-SR achieves a better rate-relevance tradeoff than VDDIB-Cascade, which activates all devices in every retransmission round.
  • VII. CONCLUSIONS: The study concludes that IB, DIB, variational approximations, and retransmission mechanisms enable task-oriented cooperative edge inference focused on downstream tasks rather than data reconstruction.Compared with traditional data-oriented communication, the proposed methods substantially reduce communication overhead; selecting optimal T under feedback costs remains future work.

APPENDIX A PROOF OF REMARK 1 · APPENDIX B DERIVATION OF THE VARIATIONAL UPPER BOUND · A. Variational Information Bottleneck

The appendices prove that minimally sufficient local features are conditionally independent given the target and derive the variational information bottleneck objective as an upper bound on the IB objective. The proof uses auxiliary-variable and Markov-chain arguments, while the variational derivation expands the objective using model and variational distributions.

  • APPENDIX A PROOF OF REMARK 1: An auxiliary random variable Q is defined to establish the conditional-independence proof, and it exists for any task by setting Q := X1:K.Q is introduced through a distribution satisfying the stated equality.
  • APPENDIX A PROOF OF REMARK 1: Because each Zk depends only on its local observation Xk, Y, Q, Xk, and Zk form a Markov chain.This local-dependence property determines the relevant conditional-independence structure.
  • APPENDIX A PROOF OF REMARK 1: The data processing inequality and the minimality and sufficiency conditions imply I(Y, Q; Zk) = I(Y; Zk) for every device k.The argument combines I(Y, Q; Zk) ≤ I(Xk; Zk) with I(Xk; Zk) = I(Y; Zk) and the chain rule.
  • APPENDIX A PROOF OF REMARK 1: Integrating out X1:K and then Q yields the factorization p(z1:K|y) = ∏K_k=1 p(zk|y).The factorization follows from the conditional distribution structure and the Markov chains established in the proof.
  • APPENDIX A PROOF OF REMARK 1: Under the minimality and sufficiency conditions in (3), the extracted features Z1, . . . , ZK are conditionally independent given Y.The proof concludes p(z1:K|y) = ∏K_k=1 p(zk|y).
  • A. Variational Information Bottleneck: The IB objective at device k is LIB,k(γ) = H(Y |Zk) + γI(Xk; Zk), and its variational expansion uses pθk(zk|x), pφk(y|zk), and rk(zk).The expanded formulation is identified as the VIB objective LVIB,k(γ; θk, φk).
  • A. Variational Information Bottleneck: Because KL-divergence is non-negative, the VIB objective LVIB,k(γ; θk, φk) is an upper bound of the IB objective.This provides the variational approximation used for tractable optimization.

B. Variational Distributed Deterministic Information Bottleneck · APPENDIX C DERIVATION OF REMARK 4 · APPENDIX D ABLATION STUDY ON FEATURE EXTRACTION

The paper derives variational objectives for distributed deterministic information bottleneck coding and evaluates task-relevant feature extraction through a corrupted two-view MNIST ablation. The variational DDIB objective upper-bounds DDIB, while the appendix establishes a lower-bound relationship for VDDIB-SR and reports improved accuracy for feature extraction.

  • B. Variational Distributed Deterministic Information Bottleneck: The DDIB objective combines H(Y|U1:K) with β-weighted per-device terms H(Y|Uk)+Rbit(Uk).This objective is denoted LDDIB(β).
  • B. Variational Distributed Deterministic Information Bottleneck: The VDDIB objective is a variational upper bound on the DDIB objective because the relevant KL-divergence is non-negative.The derivation uses variational distributions pψ0(y|u1:K) and pψk(y|uk).
  • APPENDIX C DERIVATION OF REMARK 4: The minimum of LDDIB(β) over P={p(u1:K|z1:K)} lower-bounds the VDDIB-SR objective.The derivation is presented through inequalities (35) and (36).
  • APPENDIX C DERIVATION OF REMARK 4: Inequality (35) follows from variational approximations, whereas inequality (36) follows because encoding functions parameterized by eϕ form a subset of P.These inequalities establish the stated relationship between DDIB and VDDIB-SR.
  • APPENDIX D ABLATION STUDY ON FEATURE EXTRACTION: The feature-extraction ablation compares the proposed method with D-VIB, which performs distributed coding directly on raw input views.The D-VIB baseline is reproduced in PyTorch under the same experimental setup.
  • APPENDIX D ABLATION STUDY ON FEATURE EXTRACTION: The corrupted two-view MNIST experiment uses a masked first view and a noise-corrupted second view for classification.View 1 uses a 15×15 occlusion mask; View 2 adds independent uniform noise from [0,3] before truncation to [0,1].
  • APPENDIX D ABLATION STUDY ON FEATURE EXTRACTION: The dataset contains 50,000 randomly selected training samples and 20,000 testing samples, with convolutional, maxpooling, fully-connected, dropout, and ReLU layers.The neural network structures are provided in Table VII.
  • APPENDIX D ABLATION STUDY ON FEATURE EXTRACTION: The experiments evaluate classification accuracy across β values and construct rate-relevance pairs using neural-network parameters and variational distribution estimates.Conditional target entropy is estimated by cross-entropy, while encoded-feature marginals are approximated variationally; the method achieves higher accuracy.
Loading 2109.00172v3…