PaperlayerView exact revision on arXiv

Source-linked AI summary

The Batch Size Can Be One: Test-Time Adaptation from Classifier Geometry

Jaeho Hwang, Yujin Oh, Juhyeon Shin, Juhye Lee, Sungroh Yoon, Uiwon Hwang

arXiv:2610.36628v1cs.LG

TL;DR

Single-sample and class-ordered test streams make distribution-level statistics unreliable, and confidence-based selection can reinforce wrong predictions. OriTTA builds fixed geometric targets from the source model and adapts selected features toward them. It achieves the highest mean accuracy at batch size 1 in all sixteen evaluated settings without per-setting tuning.

  • Problem

    In single-sample or class-ordered test streams, batch-based statistics can be unreliable, while confidence filtering can admit confidently wrong predictions that updates reinforce.

    Source: §1 · p. 1

  • Method

    OriTTA recenters and projects source-model geometry to define fixed class axes and ideal points, then adapts selected features using per-sample losses and a class-indexed memory.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Without per-setting tuning, OriTTA achieves the highest mean accuracy at batch size 1 in all sixteen combinations of three corruption benchmarks, four backbones, two protocols, and two stream orders.

    Source: §1 · p. 2

  • Takeaways & Limitations

    OriTTA’s reference frame, adaptation losses, and collapse signal are derived from the source model and memory rather than batch statistics or model confidence.

    Source: §1 · p. 2, §5 · p. 9

    The construction assumes a linear head whose centered rows are close to equiangular, and the evaluation covers synthetic corruptions only.

    Source: §5 · p. 10

Abstract

from arXiv · show

Test-time adaptation updates a deployed model on an unlabeled test stream. In practice this stream often arrives one sample at a time, with the classes in no particular order. A single sample then provides limited information for estimating distribution-level statistics, and selecting samples by confidence can admit predictions that are wrong yet confident. A corruption, however, shifts the features while leaving the decision regions of the frozen classifier unchanged. We therefore propose Origin-recentered Test-Time Adaptation (OriTTA), which constructs fixed geometric targets from the source model and adapts selected features toward them. Seen from the origin of the feature space, the classes lean toward a common direction. We choose a new origin to bring class scores closer together while keeping it near the mean feature of one synthetic prototype per class. From this origin, prototype directions spread more evenly, and each class obtains an axis and an ideal point before any test data arrives. During adaptation, OriTTA aligns selected features with their predicted class axis and penalizes excessive radius, keeps one input per predicted class in memory to limit feature changes for classes absent from the stream, and resets the model when re-classifying this memory reveals collapse. Without tuning for each setting, OriTTA achieves the highest mean accuracy at batch size 1 on CIFAR-10-C, CIFAR-100-C and ImageNet-C in all sixteen combinations of four backbones, two protocols and two stream orders. On ImageNet-C fog, recent methods collapse in up to two thirds of one hundred runs, while OriTTA never does.

1 INTRODUCTION

OriTTA uses source-classifier geometry to set fixed adaptation targets before test data arrives, addressing the limits of batch statistics and confidence-based selection in realistic streams. It achieves the highest mean accuracy at batch size 1 across all sixteen evaluated settings.

1 INTRODUCTION

  • Realistic streams make batch-based normalization estimates unreliable at batch size 1 or under class-blocked ordering, while confidence filtering can admit confidently wrong predictions that later updates reinforce.Across continuing shifts, such errors can compound until predictions collapse onto a few classes.Source: §1 · p. 1
  • Although corruption displaces features, the frozen classifier’s decision regions remain unchanged; direct use of their geometry is complicated by shared classifier-row components, decision-invisible directions, and origin-sensitive prototype angles.These geometric complications motivate constructing a recentered reference frame before adaptation.Source: §1 · p. 2
  • OriTTA derives fixed class axes and ideal points from the frozen classifier and synthetic prototypes, recentering the origin to spread prototype directions more evenly.On CIFAR-100, the fraction of prototypes inside their own class cone rises from 47% to 99%.Source: §1 · p. 2
  • Without per-setting tuning, OriTTA achieves the highest mean accuracy at batch size 1 across all sixteen combinations of datasets, backbones, protocols, and stream orders.The evaluation covers CIFAR-10-C, CIFAR-100-C, and ImageNet-C, four backbones, episodic and continual adaptation, and shuffled and class-blocked streams.Source: §1 · p. 2
  • It adapts selected features with per-sample angular and radial losses, while class-indexed memory limits changes for absent classes and triggers source-model restoration when predictions concentrate.The losses require neither batch statistics nor classifier softmax probabilities.Source: §1 · p. 2

2 RELATED WORK

Related work addresses single-sample adaptation, temporally correlated streams, continual drift, and feature geometry. OriTTA instead selects and adapts features using class targets derived from the source model before test data arrive.

2 RELATED WORK

  • Single-sample TTA methods refine entropy-based adaptation through sample filtering, augmentation, or sharpness-aware updates, while temporally correlated-stream methods adjust normalization, use class-balanced memories, or correct outputs.OriTTA selects and adapts test features by their position relative to fixed class ideal points, without batch statistics.Source: §2 · p. 2
  • Continual-adaptation methods counter drift with teacher-student schemes, weight restoration, fixed-interval resets, or divergence from source statistics; OriTTA resets when memory predictions concentrate, without directly using incoming class frequencies.Source: §2 · p. 2
  • Geometric methods promote uniformity among pseudo-label centroids or recenter target embeddings, whereas OriTTA derives its origin and ideal points from the source model before test data arrive.Source: §2 · p. 3

3 METHOD

OriTTA derives class-specific geometric targets from a recentered frozen-classifier feature space, then adapts reliable features while using a class-indexed memory to limit drift and detect collapse.

3.1 NOTATION AND THE RECENTERED FRAME

  • OriTTA recenters near the synthetic-prototype centroid at a point close to classifier hyperplanes, then projects features without changing logits and assigns each class an axis and ideal point.The recentered frame is chosen before test data arrives.Source: §3.1 · p. 4

3.2 LOSSES ON THE RELIABLE REGION

  • For reliable features inside the predicted-class region, angular cross-entropy favors class-axis alignment while a one-sided radial hinge pulls excessive radii inward.The two losses act only on reliable current features.Source: §3.1 · p. 5, §3.2 · p. 5

3.3 MEMORY BANK AND ALIGNMENT LOSS

  • A one-input-per-class bank is periodically aligned to ideal points using seed labels, countering drift for classes absent from class-blocked streams.Stored inputs are selected by predictive entropy; alignment uses the seed label rather than a fresh prediction.Source: §3.3 · p. 5

3.4 COLLAPSE DETECTION AND RESET

  • No source data or per-setting tuning is used; bank reclassification detects prediction concentration and resets the model and bank when a class receives over 25%.The bank has one representative per class, making its healthy share 1/K.Source: §3.4 · pp. 5–6

4 EXPERIMENTS

The experiments evaluate batch-size-1 adaptation across three datasets, four backbones, two protocols, and shuffled or class-blocked streams. They report benchmark results, test the frame’s geometric operations, and assess collapse detection.

4.1 EXPERIMENTAL SETTINGS

  • The evaluation covers CIFAR-10-C, CIFAR-100-C, and ImageNet-C with four backbones, batch size 1, episodic and continual protocols, and shuffled and class-blocked streams.It compares six baselines using their official code and published batch-size-1 settings; OriTTA updates normalization affine parameters, with anchors and the frame fixed before the stream.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • In IN-RN50, OriTTA leads the strongest baseline by about 2.7 points episodically and ends at 1.5 and 1.9 times the best baseline’s accuracy in continual adaptation.The episodic lead holds on either stream order; the continual results are for shuffled and class-blocked streams, respectively.Source: §4.2 · p. 7
  • Across 100 fog runs, DeYO, ReCAP, and SaTeen fall below 5% accuracy in 66%, 33%, and 62% of seeds, while OriTTA never collapses across 300 runs.Table 2 defines collapse as fog accuracy below 5%; OriTTA’s deviation across the same three corruptions is 0.2–0.3.Source: §4.2 · p. 7, Table 2 · p. 7
  • On IN-ViT, OriTTA’s lead narrows to 0.4–1.3 points, while CoTTA falls to chance at batch size 1 on every setting.The passage attributes IN-ViT’s greater robustness to LayerNorm; most baselines remain close to episodic accuracy under continual adaptation.Source: §4.2 · p. 8

4.3 WHAT EACH OPERATION OF THE FRAME CONTRIBUTES

  • Centering matters on IN-RN50: removing a shared classifier-row component reduces row norm by 81% and brings axis deviation from 0.65 to 0.03, while other backbones change little.The shared component shifts every logit equally and does not affect decisions.Source: §4.3 · p. 8
  • Projection removes directions the classifier cannot read; on IN-RN50, 1,049 of 2,048 dimensions are invisible, and removing projection costs 2.6 points.Without projection, the cone can admit samples based on unused variation and the radial term measures an unreadable radius; projection is absent on IN-ViT, where P = I.Source: §4.3 · p. 8
  • Recentering changes cone membership where relevant: on C100 it shifts the admitted prototype fraction from 47% to 99%, and moving the origin to zero costs 2.0 points on the class-blocked stream.On IN-RN50, the cone admits all prototypes either way; on C10, the shift is only 0.05 s and the row space spans nine of 640 feature dimensions.Source: §4.3 · p. 9

4.4 COLLAPSE DETECTION

  • Replacing SAR’s recovery with OriTTA’s detector raises continual IN-RN50 accuracy from 21.6% to 38.5% with three resets per run, versus about 9,000 with SAR’s recovery.The detector fires once on episodic elastic transform, recovering seven points, and 4.6 times per run for OriTTA in continual adaptation.Source: §4.4 · p. 9, Table 7 · p. 9

5 DISCUSSION

OriTTA derives its adaptation frame from the source classifier and synthetic prototypes, and is reported as the most accurate method in every evaluated setting. Its performance depends on the geometry of the ideal subspace, and its evaluation is limited to synthetic corruptions.

5 DISCUSSION

  • OriTTA is the most accurate method in every evaluated setting, with accuracy varying by a fraction of a point across seeds while recent methods vary by several.The paper attributes its shared constants across datasets, backbones, and stream orders to measured classifier structure.Source: §5 · p. 9
  • The frame works best when the ideal subspace is neither the whole feature space nor a small part of it.The lead over baselines narrows to about a point for ViT-B/16, while on CIFAR-10 recentering is nearly invisible and the cone rule becomes too narrow.Source: §5 · p. 10
  • The construction assumes a linear head with centered rows close to equiangular, a condition the examined models meet but a long-tailed-trained head would not.Evaluation covers synthetic corruptions only; natural shifts such as ImageNet-R and ImageNet-Sketch remain untested.Source: §5 · p. 10
  • The outlook proposes restoring the most recent detector-judged healthy state instead of source weights, recalibrating the cone rule for small row spaces, and extending the frame to dense prediction.The release includes the method, anchor-generation script, anchors used for reported results, and stream construction.Source: §5 · p. 10, REPRODUCIBILITY STATEMENT · p. 10

A IMPLEMENTATION DETAILS

The appendix specifies the adapted parameters, source-derived geometric frame, online update mechanics, stream construction, baseline configurations, and runtime. It also reports that frame construction is stable to anchor resampling and that most constants are shared across settings.

A.1 BACKBONES AND ADAPTED PARAMETERS

  • OriTTA adapts only normalization affine parameters while freezing the classifier and convolutional and attention weights; ViT-B/16's final LayerNorm is also frozen.CIFAR models normalize inputs internally, whereas ImageNet models use external ImageNet normalization.Source: §A.1 · p. 13, Table 8 · p. 13

A.2 ANCHOR GENERATION AND ITS ALGORITHM

  • Each class anchor is synthesized from the source model by minimizing class cross-entropy plus a small total-variation penalty, and the frame is built once from those anchors.Regenerated anchors varied substantially, but the IN-RN50 frame changed little across ten further seeds and its reported geometry was unchanged.Source: §A.2 · p. 13, Table 9 · p. 13

A.3 DERIVED QUANTITIES

  • Derived geometric quantities are computed before the stream, while the remaining constants are shared across settings and were selected on IN-RN50 rather than tuned per setting.The appendix lists derived quantities including prototype radius, ideal-point spacing, radial slack, temperature, cone slope, and ridge weight.Source: §A.3 · p. 14, Table 10 · p. 14, §A.4 · p. 14

A.5 RELIABLE REGION AND QUANTILE TRACKING

  • For each class, the reliable-region radius tracks the 0.2-quantile of distances from reliable cone samples, with a twenty-sample warm-up; absent classes' radii do not move.The radii are cleared on reset.Source: §A.5 · p. 14

A.6 MEMORY BANK, ALIGNMENT AND RESET SCHEDULE

  • The bank begins with anchors, admits low-entropy inputs by predicted class, and periodically aligns stored inputs; bank reclassification detects concentrated predictions and can reset adaptation.Admission may include confidently mislabeled inputs, which the detector does not measure directly but tracks through the drift that remaps bank entries.Source: §A.6 · p. 15

A.7 BATCH SIZES ABOVE ONE

  • Schedules are defined in frames and converted to steps for batch size B, while the learning rate scales with √B; admission and quantile updates are applied per sample within batches.The main algorithm combines cone and quantile gates, angular and radial losses, periodic bank alignment, reclassification probes, and reset checks.Source: §A.7 · p. 15, §A.9 · p. 16

A.8 TEST STREAMS

  • Each seed determines shared shuffled or class-blocked streams for all methods, and ten seeds provide independent repetitions of both stream and method.The class-blocked stream was corrected to vary with the seed before the reported results were produced.Source: §A.8 · p. 15

A.10 BASELINES

  • Baseline runs use released implementations with protocol-specific adjustments, and OriTTA is faster than SAR, ReCAP, and SaTeen but slower than TENT, EATA, and DeYO in the reported runtime setting.OriTTA performs an input forward/backward pass, periodic bank alignment passes, and batched bank reclassification.Source: §A.10 · pp. 16–17, §A.11 · p. 17

B FURTHER RELATED WORK

The related-work discussion distinguishes OriTTA from methods that estimate class structure from test streams: its reference frame comes from the frozen classifier and synthetic prototypes before test data arrive. It also situates the frame among geometric adaptation, source references, and synthetic-input methods.

B FURTHER RELATED WORK

  • Unlike TCA and NEO, OriTTA fixes its origin and ideal points from the source model before test data, rather than estimating class structure from test batches.On class-blocked streams, a running feature mean describes the currently presented class rather than the distribution shift.Source: B FURTHER RELATED WORK · p. 18
  • OriTTA measures angular geometry from a classifier-and-prototype-derived center within the classifier-readable subspace, rather than from the parameterization's origin.The classifier rows share a common component that no decision reads; on IN-RN50 it is 81% of a row's norm.Source: B FURTHER RELATED WORK · p. 18
  • Unlike SHOT, T3A, and SaTeen, OriTTA takes its reference from the classifier alone and uses the centered classifier's row space rather than a subspace estimated from reliable test samples.Its fixed subspace cannot follow the drift it is intended to correct and requires no test samples of any class to form.Source: B FURTHER RELATED WORK · p. 18
  • OriTTA's synthetic anchors are needed for their feature locations, not their appearance, so they use class-score optimization with a small total-variation term and no statistical prior.The frame is reported not to depend on the particular anchor draw.Source: B FURTHER RELATED WORK · p. 18
  • The bank-based detector answers whether predictions have merged onto a few classes, not whether accuracy is high, and its one-input-per-class design is insensitive to stream label order.The frame requires a linear head; vision-language models with text-embedding classifier rows are outside the work's scope.Source: B FURTHER RELATED WORK · pp. 18–19

C FIRING RATE ACROSS SETTINGS

The detector is silent in nearly all episodic runs but fires several times per run in continual adaptation, with class-count and stream-order exceptions. The reported bank-share threshold separates healthy from collapsing runs across settings.

C FIRING RATE ACROSS SETTINGS

  • The 0.25 bank-share threshold separates healthy runs from collapsing ones in every setting, with a wide margin on ImageNet and one bank-resolution step on CIFAR-10.The table reports reset counts per run averaged over ten seeds and per corruption.Source: C FIRING RATE ACROSS SETTINGS · p. 19, Table 14 · p. 19, Table 13 · p. 19
  • In episodic adaptation, the detector never fires across ten seeds except for class-blocked CIFAR-10, where it fires in four runs and averages 0.5 resets per run.The reported bank share in runs without resets stays below 0.04 on ImageNet and reaches 0.15 on CIFAR-100.Source: C FIRING RATE ACROSS SETTINGS · p. 19, Table 14 · p. 19
  • In continual adaptation, the detector fires 0.9–5.2 times per run in every setting except shuffled CIFAR-10, where it never fires.At least nine of ten seeds see a reset in nearly every continual setting, and no setting exceeds 5.2 resets per run against 15 corruption boundaries.Source: C FIRING RATE ACROSS SETTINGS · pp. 19–20
  • Reset rates are similar across stream orders on ImageNet and CIFAR-100, while class-blocked CIFAR-10 is an exception associated with its long class blocks.The bank holds one representative per class regardless of stream order.Source: C FIRING RATE ACROSS SETTINGS · p. 20

D.1 OPTIMIZER ABLATION

The optimizer ablation finds OriTTA’s advantage over baselines is not an artifact of Adam, although SGD performs better on average. Adam is retained for stability and scale invariance, supporting one learning rate across settings without per-setting tuning.

D.1 OPTIMIZER ABLATION

  • SGD is ahead by about two points on average and on thirteen of fifteen corruptions, but OriTTA’s baseline advantage is not an Adam artifact.With the baselines’ optimizer, OriTTA’s advantage is larger still.Source: Table 15 · p. 20, §D.1 · p. 20
  • Under SGD, the average seed-to-seed deviation triples, while fog’s deviation rises from 0.36 to 1.03 as its mean falls by two points.Three times the selected SGD rate triggers resets within a single corruption; Adam’s normalization bounds each single-sample step.Source: §D.1 · p. 20
  • Adam’s scale invariance makes loss factors that vary across datasets and backbones irrelevant to the learning rate, allowing one rate per normalization type.Under SGD, the rate must absorb objective scale and be chosen again for each dataset and backbone.Source: §D.1 · p. 20
  • The authors accept SGD’s average two-point gain as the price of a single procedure without per-setting tuning and retain Adam.The batch-size schedule is described as appropriate for Adam, while SGD has no rule that holds down to batch size 1 without additional corrections.Source: §D.1 · p. 20

D.2 LOSSES ABLATION

The ablation shows the angular loss does most of the work, while radial and alignment losses make smaller, distinct contributions.

D.2 LOSSES ABLATION

  • Removing the angular loss lowers IN-RN50 accuracy by fourteen points, with losses of a third to a half on most corruptions.The reduction is smallest on brightness, frost, and JPEG compression, where the source model is already accurate; the other terms cannot replace the angular loss’s class assignment and separation role.Source: §D.2 · p. 21
  • The radial loss contributes about a point, spread evenly across corruptions.It penalizes excessive radius as part of OriTTA’s feature adaptation.Source: §D.2 · p. 21
  • The alignment loss contributes least on IN-RN50, where a class is absent only while fifty images of another class pass.Its role is to hold in place classes the stream is not presenting.Source: §D.2 · p. 21

D.3 ALIGNMENT LOSS ON LONG CLASS BLOCKS

The alignment loss matters more as class blocks lengthen: on C10, removing it leads to repeated collapse and a larger accuracy loss, which the detector does not prevent.

D.3 ALIGNMENT LOSS ON LONG CLASS BLOCKS

  • Without Lalign, C10 accuracy falls by eight points and the detector fires 37 times per run rather than 0.5, across every seed rather than four of ten.The losses concentrate on the three noise corruptions and pixelate, where accuracy drops by 13 to 22 points.Source: §D.3 · p. 21, Table 18 · p. 21, Table 17 · p. 21
  • Removing Lalign costs 0.8 points with 50-image IN-RN50 blocks and 1.4 points across C100 corruptions with 100-image blocks; the detector never fires on C100.The cost reaches eight points with repeated collapse on C10’s 1,000-image blocks.Source: §D.3 · p. 21, Table 17 · p. 21
  • The alignment loss keeps absent classes near their ideal points, while the detector restores the model after collapse; neither mechanism replaces the other.On C10, frequent resets leave the model near source weights and accuracy consistently lower despite thirty-seven recoveries.Source: §D.3 · pp. 21–22

D.4 BATCH SIZE ABLATION

OriTTA remains the most accurate method across evaluated batch sizes. The ridge-weight ablation shows why its rule changes with class count: the hyperplane constraints and prototype centroid have different reliability at small and large K.

D.4 BATCH SIZE ABLATION

  • OriTTA is most accurate at every batch size from 1 to 64 on both stream orders, with accuracy moving within 0.8 points on shuffled and 0.5 on class-blocked streams.A larger batch does not close the gap; EATA falls below 20% at batch sizes 4 to 8 on the class-blocked stream.Source: §D.4 · p. 22

D.5 RIDGE WEIGHT λ

  • The ridge balances hyperplane constraints against the prototype centroid: the centroid is more reliable when K is small, while the hyperplane system is more informative when K is large.At K = 10, the system constrains only nine directions in a 640-dimensional space; for ViT-B/16, K−1 > D and the unregularized solution has norm 1.7 against feature scale 80.Source: §D.5 · p. 22
  • On C10, using λ = 10 instead of the rule’s λ = 0.1 costs ten points on both streams and every corruption.On ImageNet, raising λ from 10 to 100 or dropping the hyperplane term moves ω by less than 0.06 s and leaves accuracy unchanged.Source: §D.5 · p. 22, Table 19 · p. 23

E FULL RESULTS

The appendix provides full accuracy results across corruption types, datasets, backbones, and adaptation settings, including an episodic batch-size-64 setting.

E FULL RESULTS

  • Tables 20–23 report accuracy for every corruption in each setting and protocol, as ten-seed means with standard deviations.Within each block, bold marks the best result per column and underlining marks the second best; continual adaptation proceeds across corruptions without boundary resets.Source: E FULL RESULTS · p. 23
  • Full results cover ImageNet-C with ResNet-50-GN and ViT-B/16, and CIFAR-100-C and CIFAR-10-C with WRN-28-10.Source: Table 20 · p. 24, Table 21 · p. 25, Table 22 · p. 26
  • Table 24 reports full episodic ImageNet-C results for ResNet-50-GN at batch size 64.Source: Table 24 · p. 28

Related papers

Loading 2610.36628v1…