PaperlayerView exact revision on arXiv

Source-linked AI summary

Wireless Image Retrieval at the Edge

Mikolaj Jankowski, Deniz Gunduz, Krystian Mikolajczyk

arXiv:2007.10915v2cs.ITcs.LG

TL;DR

The paper addresses image retrieval at the wireless edge when latency, bandwidth, and power constraints make transmitting full-quality images impractical. It compares retrieval-oriented digital compression with analog JSCC of feature vectors, finding JSCC superior and more robust across channel conditions.

  • Problem

    Wireless edge devices must support image retrieval from server-side galleries despite stringent latency and limited wireless bandwidth and power.

  • Method

    The paper compares a DNN-based retrieval-oriented compression scheme with an autoencoder-based JSCC scheme that directly maps feature vectors to channel symbols.

  • Results

    JSCC outperforms digital transmission across SNR, bandwidth, and fading conditions, while degrading gracefully under training-test SNR mismatch and approaching the noiseless bound under harsh constraints.

  • Takeaways & Limitations

    End-to-end retrieval performance can improve significantly when communication and learning algorithms are designed jointly rather than separated.

Abstract

from arXiv · show

We study the image retrieval problem at the wireless edge, where an edge device captures an image, which is then used to retrieve similar images from an edge server. These can be images of the same person or a vehicle taken from other cameras at different times and locations. Our goal is to maximize the accuracy of the retrieval task under power and bandwidth constraints over the wireless link. Due to the stringent delay constraint of the underlying application, sending the whole image at a sufficient quality is not possible. We propose two alternative schemes based on digital and analog communications, respectively. In the digital approach, we first propose a deep neural network (DNN) aided retrieval-oriented image compression scheme, whose output bit sequence is transmitted over the channel using conventional channel codes. In the analog joint source and channel coding (JSCC) approach, the feature vectors are directly mapped into channel symbols. We evaluate both schemes on image based re-identification (re-ID) tasks under different channel conditions, including both static and fading channels. We show that the JSCC scheme significantly increases the end-to-end accuracy, speeds up the encoding process, and provides graceful degradation with channel conditions. The proposed architecture is evaluated through extensive simulations on different datasets and channel conditions, as well as through ablation studies.

I. INTRODUCTION

The paper frames image retrieval at the wireless edge as an inference problem constrained by latency, bandwidth, and power. It compares separate digital transmission with joint analog JSCC, finding JSCC more accurate and robust across channel conditions.

  • Motivation: Large image measurements may not reach an edge server at the required quality within strict latency limits.The motivation includes images, videos, and LIDAR data in IoT applications.
  • Problem setting: Retrieval matches a locally captured person or vehicle image against a server-side gallery because the edge device cannot perform the task locally.The server stores the large database used for identification.
  • Digital approach: The digital scheme compresses retrieval-relevant feature vectors and transmits the resulting bits using channel coding.Its compression combines a retrieval baseline, feature encoder, scalar quantization, and entropy coding, with a Gaussian-mixture density model.
  • JSCC approach: The JSCC scheme directly maps feature vectors to channel symbols, avoiding explicit channel coding and bit conversion.It uses an autoencoder-based architecture for noisy, fading, and bandwidth-limited conditions.
  • Results: JSCC outperforms the digital approach under varied SNR and bandwidth constraints, including fading channels and comparisons against capacity-achieving codes.Its performance also degrades gracefully when test and training SNRs do not match.
  • Implication: The results indicate that communication and learning algorithms generally should be designed jointly for end-to-end inference performance.The paper also analyzes architectures and training strategies as baselines for wireless edge learning.

II. RELATED WORK

Related work spans wireless edge learning, image retrieval, and source-channel coding. This paper focuses on centrally trained remote inference for retrieval rather than distributed training or device computation limits.

  • Wireless edge learning: Much prior wireless edge learning work addresses distributed training under limited communication resources.This paper instead studies the inference phase with centralized training.
  • Wireless edge learning: Prior distributed-inference studies commonly split DNNs across devices and servers to reduce edge-device computation.The present work does not consider computational limitations of the device.
  • Image retrieval: Person and vehicle retrieval research targets recognition for surveillance and commonly uses convolutional neural networks.Representative techniques include part classifiers, bias-invariant features, attention, and multiscale analysis.

C. Joint Source-Channel Coding (JSCC)

The paper studies retrieval over bandwidth-limited wireless channels using both separate digital coding and joint JSCC. Experiments consider AWGN and slow Rayleigh fading, with channel models known during training and inference.

  • Rationale: For remote inference, separation can be suboptimal in general, while optimality holds for the specific testing-against-independence problem.This related result motivates studying JSCC beyond classical asymptotic separation guarantees.
  • Approaches: Feature vectors are transmitted because bandwidth limitations prevent lossless transmission, using either digital compression and channel coding or direct JSCC mapping.The two approaches target low-dimensional identity representations.
  • Channel assumptions: The channel model is assumed known during training and unchanged during inference for both transmission approaches.The considered settings include static and slow fading channels.
  • AWGN channel: In the AWGN model, the received vector equals the transmitted vector plus independent complex Gaussian noise under an average input-power constraint.The channel input contains B complex symbols.
  • Fading channel: The slow-fading model applies one Rayleigh channel gain to all transmitted symbols, with the same average input-power constraint as AWGN.The experiments set the fading-gain variance to 1, giving the same average received power as the static AWGN model.

B. Retrieval Baseline

The retrieval baseline uses a pretrained ResNet-50 to extract features from resized person and vehicle images. Batch normalization follows each convolutional layer.

  • Feature extraction: ResNet-50 pretrained on ImageNet extracts retrieval features for the baseline.The choice follows state-of-the-art retrieval methods and supports comparable results across setups.
  • Input processing: Person images are resized to 256×128 and vehicle images to 128×128 using bicubic interpolation.These resolutions provide the common inputs for the respective dataset types.

C. Digital Transmission of Compressed Feature Vectors

The digital scheme compresses retrieval-relevant feature vectors before channel coding, balancing retrieval accuracy against bit rate under an assumption of reliable communication.

  • Communication assumption: The scheme assumes that a prescribed number of bits can be reliably conveyed, although short-blocklength and fading channels make this difficult in practice.The simulations use capacity-achieving channel codes as an upper bound on practical digital performance.
  • Digital transmission: The digital pipeline extracts features, applies lossy compression and arithmetic coding, then channel-codes the resulting bits for transmission.At the receiver, the quantized latent is classified only during training; retrieval uses feature-vector nearest-neighbor search.
  • Compression: The retrieval-oriented compressor uses dimensionality reduction, quantization, entropy modeling, and arithmetic coding to reduce the transmitted representation.The quantized outputs are modeled with a Gaussian-mixture distribution for arithmetic coding.
  • Model design: The learned mixture model is non-adaptive, avoiding per-image transmission of additional distribution parameters and reserving bandwidth for quantized feature vectors.The authors report that the simpler model performs sufficiently well for their purpose.
  • Training objective: The compression objective combines retrieval cross-entropy with an entropy term representing the average arithmetic-code length.This creates a tunable trade-off between retrieval accuracy and the number of transmitted bits.

D. JSCC of Feature Vectors

The JSCC approach maps retrieval features directly to wireless channel symbols and trains an autoencoder to recover them or support identity prediction under noisy channels.

  • JSCC architecture: JSCC AE directly maps the baseline feature vector to channel inputs through a multilayer fully connected encoder.The channel-input dimension is 2B real symbols, corresponding to B complex channel values.
  • Training strategy: End-to-end training alone is evaluated as an alternative, while T1,2 reconstructs feature vectors without directly accounting for retrieval accuracy.The paper also compares a simpler JSCC FC architecture that maps query images directly to channel inputs.
  • Training strategy: The proposed T1,2,3 strategy pretrains the feature encoder, pretrains the JSCC autoencoder, and then performs end-to-end training.Feature-encoder pretraining uses identity cross-entropy, while autoencoder pretraining reconstructs feature vectors with L1 loss.
  • JSCC architecture: The receiver decodes the noisy signal and uses a fully connected layer to produce identity predictions for cross-entropy training.The architecture is illustrated as feature transmission followed by decoding and classification.
  • JSCC architecture: The JSCC AE encoder reduces feature dimensionality at its first fully connected layer, with the decoder providing the inverse dimensional expansion.The selected architecture uses fully connected layers for the encoder and decoder.

IV. RESULTS

The evaluation compares JSCC AE and JSCC FC with the digital scheme and an ideal unlimited-resource channel baseline across retrieval experiments.

  • Evaluation: The experiments compare the proposed JSCC architectures, the digital scheme, and an ideal channel with unlimited resources.The ideal baseline transmits full, noiseless feature vectors.

A. Experimental Setup

The experiments vary channel conditions and digital compression settings to evaluate retrieval accuracy, encoding cost, and transmission assumptions.

  • Experimental setup: The JSCC schemes are trained across SNRs from −12dB to ∞dB, with matching training and test SNRs unless stated otherwise.The digital scheme varies latent dimensionality from 64 to 512 and adjusts its entropy-control parameter λ.
  • Experimental setup: 25%?
  • Fading channels: The fading-channel evaluation considers perfect channel-state information and a fixed-λ outage scenario.The fixed-λ scenario trades compression distortion against transmission failures when required rate exceeds channel capacity.
  • Training cost: Approximately 20, 3, and 30 minutes are required for JSCC AE training steps T1, T2, and T3, respectively.T1 is performed only once because it does not depend on the channel model.
  • Results overview: At bandwidth B=64, JSCC AE achieves the best retrieval accuracy across the tested SNR range and all three re-ID datasets.The comparison covers AWGN and slow-fading channels.

B. Datasets

The evaluation uses three surveillance retrieval datasets covering person and vehicle identities, with top-1 retrieval accuracy as the common measure.

  • B. Datasets: The evaluation uses CUHK03 and Market-1501 for person retrieval and VeRi for vehicle retrieval.CUHK03 contains 14,096 images of 1,467 identities, Market-1501 contains 32,217 images of 1,501 pedestrians, and VeRi contains over 50,000 images of 776 vehicles.
  • B. Datasets: CUHK03 contains images from two camera views, while Market-1501 and VeRi include images captured across six and 20 cameras, respectively.VeRi also spans different viewpoints, occlusions, resolutions, and lighting conditions.
  • B. Datasets: Top-1 retrieval accuracy measures the fraction of queries whose correct identity appears first in the ranked retrieval list.

C. Performance for Different Methods

Across static and fading channels, JSCC AE generally outperforms the digital scheme, while increased bandwidth improves robustness. Fading reduces accuracy and exposes limitations when channel state information is unavailable.

  • JSCC AE outperforms the digital scheme across all considered static-channel scenarios at bandwidth 64.Bandwidth 64 corresponds to transmitting 64 complex channel symbols.
  • The digital decoder provides no performance improvement over decoding-free digital transmission across all datasets.The decoder maps quantized latents back to the original 2048-dimensional feature space before classification.
  • Fading channels reduce the accuracy of all approaches, but JSCC AE achieves the best performance across the three datasets and considered average SNR values.The digital scheme performs worse without channel state information.
  • At SNR = 15dB, JSCC AE with CSI recovers the noiseless bound, while its gap over the digital approach grows further.
  • Higher bandwidth improves robustness against channel noise, but JSCC AE without channel state information plateaus around SNR = 12dB under fading.The plateau may result from the inability to fully cancel variable channel gain; channel estimation and feedback may mitigate fading.

E. Graceful Degradation

The proposed JSCC AE maintains retrieval accuracy more smoothly across channel conditions than the digital scheme, including when training and test SNRs differ. Its three-step training strategy achieves faster convergence and better final performance than alternatives.

  • E. Graceful Degradation: JSCC AE achieves graceful degradation across test SNRs, unlike the digital scheme’s cliff effect.A model trained at moderate SNR_train performs relatively well across a wide range of SNR_test values.
  • E. Graceful Degradation: A moderate-SNR-trained JSCC AE model avoids training a separate model for every SNR value, with only a moderate accuracy loss.This supports inference across mismatched training and test channel conditions.
  • E. Graceful Degradation: Noise-free training is not robust to channel noise, causing accuracy to decrease faster than for networks trained under noisy conditions.Even so, the noise-free model still exhibits graceful degradation as channel noise increases.
  • F. Training Strategy: The proposed T1,2,3 strategy converges faster and reaches the smallest final cross-entropy loss, outperforming the alternative training strategies by a large margin.The experiment uses B=64 and SNR=0dB; adding training steps improves performance gradually.
  • F. Training Strategy: The autoencoder pretraining phase adds negligible time compared with joint training, taking approximately 3min versus approximately 1hr.The comparison concerns the training-time cost of adding the pretraining phase.

G. Comparison of Different Models

The paper evaluates multiple JSCC autoencoder architectures under fixed CUHK03 channel conditions and selects the architecture used in the main experiments. Differences among candidate models are marginal, with Model D performing slightly better in final retrieval performance.

  • G. Comparison of Different Models: Model D performs slightly better than the other searched architectures in final retrieval performance.It is the architecture presented in Fig. 4 and used for the remainder of the paper.
  • G. Comparison of Different Models: The architecture comparison evaluates CUHK03 retrieval at SNR=0dB and B=64 after applying the three-step training strategy.Mean squared error between original and noisy reconstructed features is also reported after JSCC autoencoder pretraining T2.
  • G. Comparison of Different Models: Differences among the nine candidate architectures are marginal, so the selected model is distinguished mainly by slightly better retrieval performance and implementation considerations.The passage identifies Model D as the architecture used in the rest of the paper.
  • G. Comparison of Different Models: The overall system compares a retrieval-oriented digital compression scheme with a JSCC scheme that directly maps feature vectors to channel symbols.The conclusions also report that JSCC provides graceful degradation when test and training SNRs differ.
Loading 2007.10915v2…