Fast Test-Time Refinement for Robust Learned Image Compression

Jiaming Liang; Chi-Man Pun; Weisi Lin

Source-linked AI summary

Fast Test-Time Refinement for Robust Learned Image Compression

Jiaming Liang, Chi-Man Pun, Weisi Lin

arXiv:2608.15113v1Aug 15, 2026cs.CVcs.AIcs.LG

TL;DR

Learned image compression remains vulnerable to adversarial attacks, while test-time refinement lacks white-box evaluation and can require costly iterations. This paper introduces FTTR using asymmetric adversarial trajectories and reports strong robustness with only one refinement step across diverse adaptive attacks.

Problem
Prior work leaves LIC test-time refinement insufficiently understood and untested against fully informed white-box adversaries, despite standardized codecs requiring strong reliability guarantees.
Method
The paper characterizes asymmetric adversarial trajectories with a Tube Model and develops FTTR, exploiting LIC’s Input-as-Label property to contract adversarial regions.
Results
With 1-step FTTR, average PSNR rises from 9.90 dB to 24.58/25.00 dB under BPDA/Unroll attacks, while average bpp falls from 17.97 to 11.83/11.57.
Takeaways & Limitations
FTTR offers a practical LIC defense that preserves strong robustness while substantially reducing the refinement iterations needed for adversarial recovery.
Takeaways & Limitations
FTTR can introduce excessive perturbations and degrade benign performance, especially when adversarial perturbations are small.

Abstract

from arXiv · show

Learned image compression (LIC) has demonstrated remarkable rate-distortion (RD) performance in benign settings. However, the high representational capacity endowed by deep neural networks (DNNs) comes at the expense of increased adversarial vulnerability. This hinders their adoption as trusted standardized codecs. Recent work has sketched test-time refinement (TTR) as a defense in gray-box scenarios, despite its original purpose of improving benign RD performance. Unfortunately, extensive iterations of TTR incur prohibitive overhead, while the robustness mechanism lacks theoretical understanding. Moreover, TTR has not been evaluated in white-box settings or against attacks beyond $\ell_2$-bounded rate and untargeted distortion objectives. To bridge these gaps, we present a systematic study. Our study reveals an Asymmetric Adversarial Trajectory (AAT) property in LIC systems: transitioning from adversarial to benign regions is significantly easier than the reverse process, where adversarial examples can often be roughly recovered within only 1-2 steps. We provide a two-dimensional Tube Model to explain this phenomenon. Based on AAT, we propose a Fast Test-Time Refinement (FTTR) framework for practical and robust LIC systems. We establish that the robustness arises from the contraction of adversarial regions induced by the Input-as-Label property of LIC systems, rather than from obfuscated gradients. Extensive evaluations with diverse strong adaptive attacks across multiple LIC systems demonstrate the promise of the proposed FTTR framework. The code is available at https://github.com/chinaliangjiaming/FTTR.git.

1 Introduction

Learned image compression offers strong rate-distortion performance but inherits deep networks’ adversarial vulnerability, motivating trustworthy defenses. This study identifies asymmetric refinement behavior and develops FTTR as an efficient, theoretically grounded defense evaluated against diverse adaptive attacks.

Motivation: Imperceptible perturbations can cause rate collapse, reconstruction distortion, manipulated reconstruction, and downstream degradation in LIC systems.Because standardized compression systems disclose technical details, they are exposed to adversaries and require strong reliability guarantees.
AAT and FTTR: Adversarial-example recovery in LIC often requires substantially fewer optimization steps than generation, typically only 1–2 steps when refinement is sufficiently strong.The paper terms this Asymmetric Adversarial Trajectory and explains it with a two-dimensional Tube Model.
AAT and FTTR: FTTR builds on AAT to provide an efficient adversarial-defense framework based on fast test-time refinement.The study presents FTTR as an efficient variant of TTR for practical robust LIC systems.
Evaluation: 1-step FTTR raises average PSNR from 9.90 dB to 24.58/25.00 dB and lowers average bpp from 17.97 to 11.83/11.57 under 400-step ℓ∞ white-box PGD with budget 16/255.The paired values correspond to BPDA/Unroll adversaries, respectively.

2 Related Work

LIC combines neural transforms and entropy modeling to improve rate-distortion performance but remains vulnerable to rate, reconstruction, and downstream-task attacks. Existing defenses mainly rely on adversarial training or input transformations, while TTR has only limited gray-box adversarial evaluation and leaves efficiency, theory, and adaptive-attack questions unresolved.

LIC Background: LIC integrates DNNs into analysis–synthesis transforms and entropy modeling, achieving superior rate-distortion performance over handcrafted codecs.Its transform-coding pipeline consists of transform, quantization, and entropy coding.
Adversarial Threats: Adversarial attacks on LIC target rate collapse, reconstruction fidelity, or downstream task performance.Rate collapse increases storage and transmission overhead, while reconstruction manipulation can impair downstream tasks.
Robustness Defenses: White-box robustness is especially important for LIC, yet existing defenses predominantly use adversarial training and remain scarce.White-box deployment exposes technical details and gives adversaries full system knowledge.
Robustness Defenses: Input-transformation defenses project adversarial examples toward the benign data manifold, but the described approach is effective only in black-box settings.It compares reconstruction losses across candidate transformation pairs and selects the least-loss reconstruction.
Test-Time Refinement: TTR was developed for benign in-distribution and distribution-shifted RD performance through iterative latent optimization, but adversarial use remains underexplored.Its first adversarial application was only sketched as a gray-box benchmark, leaving multi-step overhead, theoretical grounding, and diverse adaptive evaluations unresolved.

3 Theory and Methodology

This section formalizes LIC systems, TTR, and adaptive white-box threats, then explains the Asymmetric Adversarial Trajectory through a Tube Model and derives FTTR as a lightweight defense. It attributes robustness to adversarial-region contraction enabled by Input-as-Label, while noting potential degradation from excessive refinement strength.

LIC Systems and Test-Time Refinement: LIC is modeled as an encoder, quantizer, and decoder optimized by a Lagrangian combining bitrate and distortion.TTR refines latent representations at test time through gradient descent on the rate-distortion objective before quantization and decoding.
Adaptive White-Box Threat Model: The threat model gives adversaries full knowledge of the defended LIC system and evaluates adaptive BPDA and Unroll attacks under input-bounded perturbations.Perturbations use ℓ2 or ℓ∞ constraints and target compression rate, distortion, or downstream-task performance.
Asymmetric Adversarial Trajectory: Large learning rates restore adversarial examples to near-convergent performance within 1–2 TTR steps, revealing the Asymmetric Adversarial Trajectory.Small learning rates converge more stably but slowly, whereas large rates show transient instability followed by recovery; this behavior persists across attack-generation steps.
Tube Model: The Tube Model explains AAT by characterizing adversarial regions as thin, highly curved, tube-like structures with limited thickness along the negated gradient direction.Consequently, a one-step sign-gradient update can escape the adversarial region, while linear adversarial updates struggle to reach effective examples quickly.
Fast Test-Time Refinement: FTTR approximates TTR with few iterations, exploiting Input-as-Label to contract adversarial regions rather than shift them and thereby provide white-box robustness.A relatively large strength s can introduce excessive perturbations, harming adversarial examples with small perturbations and degrading benign performance.

4 Experiments and Results

Experiments across representative LIC systems and multiple adaptive attack settings show that FTTR improves robustness against distortion, bitrate, joint rate-distortion, semantic-manipulation, and downstream classification attacks. The gains persist under strong white-box evaluations and include substantial improvements in reconstruction quality, bitrate, and classification accuracy.

Experimental setup: The benchmark includes adaptive white-box BPDA and Unroll attacks, alongside gray-box evaluation, to test FTTR beyond prior gray-box assessments.BPDA forwards through FTTR while approximating it as identity during backpropagation; Unroll directly attacks the FTTR-equipped system.
ℓ∞ untargeted distortion attacks: At ϵ=16/255, FTTR improves average PSNR over the 9.90 dB w/o setting under ℓ∞ untargeted distortion attacks.The passage reports PSNR results for w/o, Noise, Gray-Box, BPDA, and Unroll settings with T=400 attack iterations.
ℓ∞ compression rate attacks: At ϵ=16/255, FTTR reduces average bpp from 17.97 without defense to 11.83(−6.14) under BPDA and 11.57(−6.40) under Unroll.The experiment uses PGD with bpp loss, and robustness gains vary across LIC systems.
ℓ2 joint R-D attacks: At c=1000, FTTR raises average PSNR from 7.59 dB to 26.71(+19.12) dB under BPDA and lowers bpp from 15.19 to 7.58(−7.61).These results come from ℓ2 joint R-D attacks using C&W with fixed c values of 500, 750, and 1000.
Semantic and downstream attacks: At strength s=0.06, FTTR increases average downstream classification accuracy from 0.4% without defense to 28.1%(+27.7%) under BPDA.FTTR also effectively resists gray-box PGD2-GSM semantic manipulation, while the downstream attacks target ResNet-50 and ViT-B/16 failures after LIC processing.

5 Conclusions and Limitations

This work presents the first systematic study of TTR as a white-box defense for learned image compression, revealing that its effectiveness follows an Asymmetric Adversarial Trajectory property rather than requiring many iterations. The hypothesized Tube Model explains why a few refinement steps suffice for most LIC systems.

Conclusions and Limitations: The study is the first systematic investigation of TTR as a white-box defense for LIC, extending prior robustness work focused on gray-box settings.It reveals the Asymmetric Adversarial Trajectory (AAT) property, under which TTR defense does not require numerous iterations and a few steps suffice for most LIC systems; the phenomenon is explained through a hypothesized Tube Model.

A Proofs · A.1 Deduction of Equation 15

The appendix derives Equation 15 using Adam’s notation while optimizing the purifier δ_d rather than model parameters θ. It specifies the initialization, gradient, moment updates, bias correction, and the k = 1 specialization.

A.1 Deduction of Equation 15: The derivation follows the notation of the original Adam paper without repeating its definitions.
A.1 Deduction of Equation 15: The optimized variable is the purifier δ_d, not the model parameters θ.
A.1 Deduction of Equation 15: The parameters are initialized as specified in the original Adam paper.
A.1 Deduction of Equation 15: The derivation introduces the gradient at time step k.
A.1 Deduction of Equation 15: The first moment follows m_k = β_1 · m_{k−1} + (1 − β_1) · g_k, with bias-corrected estimate m̂_k = m_k / (1 − β_1^k).
A.1 Deduction of Equation 15: Setting k = 1 yields the first-step specialization used in the derivation.

B Supplementary Experiments · B.1 Details of Adversary Benchmarks

The supplementary benchmark details evaluate FTTR as a standalone defense in white-box settings, including strong adaptive Unroll and BPDA attacks. They specify gradient-based attack construction, K=1 refinement, straight-through estimation, and visualization-based PGD and C&W evaluations.

B.1 Details of Adversary Benchmarks: FTTR is evaluated in white-box scenarios as a standalone defense module preceding the LIC system f.
B.1 Details of Adversary Benchmarks: The attacker performs gradient ascent to optimize the attack objective in the white-box benchmark.
B.1 Details of Adversary Benchmarks: The benchmark expands the gradient from Equation 33 to derive the attack gradient used against FTTR.
B.1 Details of Adversary Benchmarks: K=1 is used in the experiments, fixing the number of refinement steps in the analyzed attack formulation.
B.1 Details of Adversary Benchmarks: The experiments use straight-through estimation for Clip(·) and sgn(·), treating their gradients as identity mappings.
B.1 Details of Adversary Benchmarks: Unroll computes second-order derivatives through PyTorch Hessian-vector products, forming a computationally expensive strong adaptive white-box attack against FTTR.The procedure is referred to as unrolled optimization in prior literature.
B.1 Details of Adversary Benchmarks: BPDA provides an approximate gradient and is evaluated alongside Unroll for comprehensive adaptive white-box testing, followed by gray-box benchmarks.