PaperlayerView exact revision on arXiv

Source-linked AI summary

Tent: Fully Test-time Adaptation by Entropy Minimization

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, Trevor Darrell

arXiv:2006.10726v3cs.LGcs.CVstat.ML

TL;DR

Fully test-time adaptation asks how a model can generalize to shifted test data when only its parameters and unlabeled target data are available. Tent answers by minimizing prediction entropy through efficient channel-wise feature modulation, achieving lower error across several shifts and a new ImageNet-C state of the art without altering training. Its scope remains limited on some harder natural or domain shifts.

  • Problem

    Models lose accuracy when testing data differ from training data, yet fully test-time adaptation must operate without source data or supervision.

  • Method

    Tent minimizes prediction entropy on target batches by updating normalization statistics and channel-wise affine feature transformations while leaving training unchanged.

  • Results

    Tent reduces error across corruption, digit, and segmentation shifts, reaching 44.0% online and 42.3% offline error on ImageNet-C versus 50.2% for ANT and 49.9% for test-time normalization.

  • Takeaways & Limitations

    Fully test-time entropy minimization can provide self-supervised adaptation from a model’s own predictions across several shifted-data benchmarks.

  • Takeaways & Limitations

    Tent does not improve natural shifts on CIFAR-10.1 and ImageNetV2 and fails on MNIST-to-SVHN, where error increases to 79.8%.

Abstract

from arXiv · show

A model must adapt itself to generalize to new and different data during testing. In this setting of fully test-time adaptation the model has only the test data and its own parameters. We propose to adapt by test entropy minimization (tent): we optimize the model for confidence as measured by the entropy of its predictions. Our method estimates normalization statistics and optimizes channel-wise affine transformations to update online on each batch. Tent reduces generalization error for image classification on corrupted ImageNet and CIFAR-10/100 and reaches a new state-of-the-art error on ImageNet-C. Tent handles source-free domain adaptation on digit recognition from SVHN to MNIST/MNIST-M/USPS, on semantic segmentation from GTA to Cityscapes, and on the VisDA-C benchmark. These results are achieved in one epoch of test-time optimization without altering training.

1 INTRODUCTION

Fully test-time adaptation addresses dataset shift when only a model’s parameters and unlabeled target data are available. Tent minimizes prediction entropy during testing, using confidence as task-related supervision, and improves error across corruption and domain-shift benchmarks.

  • Fully test-time adaptation uses only model parameters and target data, without source data or supervision during testing.This setting is motivated by source-data availability, computational-efficiency, and accuracy constraints.
  • Tent minimizes prediction entropy during testing because lower-entropy predictions tend to be more accurate and entropy tracks corruption-related shift.On corrupted CIFAR-100-C, increasing corruption raises both loss and entropy, while lower entropy corresponds to lower error.
  • Tent adapts by estimating normalization statistics and optimizing channel-wise affine transformations batch-by-batch without altering training.The method is independent of source data once model parameters are available.
  • Tent supports fully test-time adaptation across corrupted image classification, digit recognition, and semantic segmentation, with analyses covering objectives, sensitivity, and architectures.The reported evaluations include corruption robustness, domain adaptation, and simulation-to-real shift.
  • 44.0% error on ImageNet-C with online adaptation improves on robust-training and test-time-normalization baselines at 50.2% and 49.9%, respectively.Tent reaches 42.3% error with offline adaptation and requires one gradient per test point.

2 SETTING: FULLY TEST-TIME ADAPTATION

Fully test-time adaptation separates testing-time adaptation from settings that require source data, target labels, or training-time losses. It adapts a trained model using only unlabeled target data while inference proceeds.

  • The fully test-time setting requires only the model and unlabeled target data xt for adaptation during inference.Unlike other adaptation settings, it does not require source data or target labels.
  • Fine-tuning requires target labels, while domain adaptation requires both source and target data for cross-domain training losses.These requirements distinguish them from the source-free, label-free setting considered here.
  • Fully test-time adaptation optimizes an unsupervised loss during testing and remains independent of training data and training loss given model parameters.Unlike TTT, it does not jointly optimize the adaptation loss during source-data training.
  • Tent’s method overview keeps training unchanged and minimizes prediction entropy during testing over a constrained modulation using θ and xt.The adaptation objective operates at test time rather than modifying the original training procedure.

3 METHOD: TEST ENTROPY MINIMIZATION VIA FEATURE MODULATION

Tent performs test-time entropy minimization by adapting feature modulations rather than the full parameter vector. It updates normalization statistics and channel-wise affine parameters online using target batches.

  • Tent optimizes prediction entropy during testing by modulating model features, requiring a compatible probabilistic, differentiable supervised-task model.The model requirements support entropy measurement and gradient-based test-time optimization.
  • Entropy is an unsupervised objective because it depends on predictions rather than annotations, while remaining directly tied to the supervised task.Jointly optimizing batched predictions avoids the trivial single-example solution of assigning all probability to one class.
  • 3.2 MODULATION PARAMETERS: Tent updates low-dimensional, channel-wise scales and shifts instead of the full high-dimensional parameter vector for greater test-time stability and efficiency.Directly optimizing θ can be sensitive and inefficient, whereas feature modulations restrict adaptation to linear channel-wise transformations.
  • 3.2 MODULATION PARAMETERS: Normalization computes (x − µ)/σ, then affine transformation produces γx̄ + β; statistics are estimated from target data while γ and β are optimized.The method repurposes normalization layers and updates their statistics and affine parameters across layers and channels.
  • Each test batch updates normalization statistics during the forward pass and affine parameters using prediction-entropy gradients during the backward pass.The update affects subsequent batches, and online adaptation continues while test data arrive.

4 EXPERIMENTS

Across corruption robustness and source-free domain adaptation, tent lowers target error using only target data, often outperforming normalization and methods requiring source-data training or more computation. Analyses show that feature modulation, entropy reduction, and adaptation beyond individual update points support its behavior, while harder shifts remain challenging.

  • 4.1 ROBUSTNESS TO CORRUPTIONS: Tent has the least error on CIFAR-10-C and CIFAR-100-C while using less optimization than domain adaptation and test-time training methods.It also improves on the test-time normalization baseline.
  • 4.1 ROBUSTNESS TO CORRUPTIONS: On ImageNet-C, tent reaches the least error for most corruption types without increasing error on original data, while adapting online at inference efficiency.Figure 5 averages error over corruption levels and compares source, normalization, and tent.
  • 4.1 ROBUSTNESS TO CORRUPTIONS: 44.0% ImageNet-C error with online adaptation and 42.3% offline establishes a new state of the art without altering training.Tent improves on ANT's 50.2% error and test-time normalization, using one gradient per test point.
  • 4.2 SOURCE-FREE DOMAIN ADAPTATION: Tent beats the source model and batch normalization in every digit-adaptation case and achieves the lowest error in 2/3 cases without source data.These results are obtained in one epoch without using source data.
  • 4.2 SOURCE-FREE DOMAIN ADAPTATION: Tent adapts with about 80× less computation than source-using domain-adaptation methods, while additional updates further reduce error.It reaches 8.2% error in 10 epochs and 6.5% in 100 epochs for the reported digit adaptation setting.
  • 4.2 SOURCE-FREE DOMAIN ADAPTATION: On VisDA-C, tent lowers validation error from 56.1% to 45.6%, reaching 39.6% when updating all layers except the final classifier.Offline source-free methods can achieve lower error with more computation and tuning, whereas tent adapts online during testing.
  • 4.3 ANALYSIS: Tent reduces both entropy and task loss for most CIFAR-100-C corruption type-level combinations, while full-model updates never improve over the source model.Ablations show that both normalization and transformation updates are needed for feature modulation.
  • 4.3 ANALYSIS: Adaptation generalizes across target points: CIFAR-100-C error falls from 37.3% to 34.2% and SVHN-to-MNIST error from 8.2% to 6.5%.The update data and test data are separated, indicating the adapted modulation is not point specific.

5 RELATED WORK

Tent is positioned as fully test-time adaptation using entropy as its sole loss, normalization, and lightweight feature modulation, without source data, proxy tasks, or training changes.

  • Tent differs from train-time domain adaptation and transduction because it adapts using only target-time information rather than jointly optimizing source and target data.
  • Unlike several source-free methods, tent needs neither generative modeling nor multiple models and losses, enabling more computationally efficient online adaptation.
  • Unlike test-time training, tent avoids proxy tasks and does not alter training before target adaptation.
  • Tent minimizes entropy during testing as the sole loss for adapting to dataset shifts without additional data.
  • Tent adapts channel-wise affine feature transformations alongside normalization for efficient, stable online updates.

6 DISCUSSION

The discussion finds that tent reduces error across several benchmark shifts, while identifying important boundaries in the shifts, parameters, and losses it can handle.

  • Tent reduces error for image corruptions, digit appearance shifts, and simulation-to-real discrepancies, but does not improve natural dataset-reproduction shifts.
  • Tent’s normalization and transformation update only part of the model, and larger shifts such as VisDA may be better addressed by broader parameterizations.
  • Tent requires batches for entropy optimization and cannot update episodically one point at a time.
  • Tent and related prediction-based updates require full model recomputation, leaving efficiency gains from representation-level losses as an open direction.
  • The authors frame fully test-time adaptation as a setting for developing models that update themselves during inference.

APPENDIX

The supplement provides additional summaries and visual analyses of corruption examples, segmentation adaptation, and feature shifts.

  • The supplement summarizes image corruptions, shows a qualitative instance-wise semantic-segmentation adaptation example, and visualizes feature shifts across more layers.

A ROBUSTNESS TO CORRUPTIONS

The corruption benchmark uses synthetic examples representing natural variation, with results summarized across severe and multiple corruption levels.

  • Table 2 reports errors at corruption severity level 5, while Figure 5 averages errors for each corruption type across levels 1–5.
  • Figure 8 illustrates examples of each synthetic corruption type, including noise, blur, weather, and digital imaging effects.

B SOURCE-FREE ADAPTATION FOR SEMANTIC SEGMENTATION

Tent performs source-free semantic-segmentation adaptation from simulated GTA data to real Cityscapes data using only target images. On a single target image, episodic optimization suppresses noise and recovers missing classes within 10 iterations.

  • B SOURCE-FREE ADAPTATION FOR SEMANTIC SEGMENTATION: Tent adapts semantic segmentation from simulated GTA data to real Cityscapes data using only target data.The supervised model is HRnet-W18, a fully convolutional network.
  • B SOURCE-FREE ADAPTATION FOR SEMANTIC SEGMENTATION: Tent optimizes over a single target image as a dataset of pixel-wise predictions, effectively fitting a custom model to each target-domain image.This is described as episodic optimization.
  • B SOURCE-FREE ADAPTATION FOR SEMANTIC SEGMENTATION: 10 iterations suppress noise and recover missing classes in the qualitative Cityscapes adaptation example.The figure describes completion of a street segment and recovery of motorcycle predictions.

C FEATURE SHIFTS ACROSS LAYERS AND METHODS

On corrupted CIFAR-100-C features, corruption shifts source features away from clean reference features. BN moves them toward the reference, whereas Tent moves them toward an oracle optimized with target labels.

  • C FEATURE SHIFTS ACROSS LAYERS AND METHODS: Corruption shifts CIFAR-100-C source features away from reference features without corruption.The comparison is shown for Gaussian noise.
  • C FEATURE SHIFTS ACROSS LAYERS AND METHODS: BN shifts corrupted features back toward the clean reference features.
  • C FEATURE SHIFTS ACROSS LAYERS AND METHODS: Tent shifts features away from the reference and toward an oracle optimized on target labels.
Loading 2006.10726v3…