PaperlayerView exact revision on arXiv

Source-linked AI summary

Image-Scale Robustness and Visual Recognition Performance: A Cross-Architecture Analysis

Anish Monsley Kirupakaran

arXiv:2609.06051v1cs.CVcs.AI

TL;DR

Whether scale robustness has a common quantitative structure across modern vision architectures, and what model properties explain its variation, remains unclear. The study measures characteristic scale across pretrained ImageNet-1K classifiers and finds it strongly associated with baseline accuracy, with the relationship stable under multiple checks.

  • Problem

    It remains unclear whether scale robustness is a common quantitative property across heterogeneous architectures and whether it is associated with recognition accuracy rather than model size or family.

    Source: §1 · p. 2

  • Method

    The study defines characteristic scale from classifiers’ scale-response curves and examines it across 20 pretrained architectures, testing accuracy, parameter count, family, and representation stability.

    Source: §1 · p. 2, §2.1 · p. 2

  • Results

    Across 20 evaluated architectures, characteristic scale was strongly negatively associated with baseline Top-1 accuracy (Pearson r = −0.890, p = 1.52 × 10−7), with R2 = 0.792.

    Source: §3.1 · p. 4, §5 · p. 8

  • Takeaways & Limitations

    Across the evaluated models, characteristic scale is strongly organized by baseline recognition performance rather than simply by model size, architectural family, or representation stability.

    Source: §3.2 · p. 5, §3.5 · p. 7, §5 · p. 8

    The observed accuracy–scale relationship is a statistical association and does not by itself identify the underlying causal mechanism.

    Source: §4 · p. 8

Abstract

from arXiv · show

The sensitivity of visual recognition models to changes in image scale is well established, yet the factors governing this sensitivity across heterogeneous architectures remain unclear. In this work, we investigate whether scale robustness exhibits a common quantitative structure across modern vision models. We evaluate 20 pretrained ImageNet-1K classifiers spanning seven architectural families, including convolutional, mobile, efficient, and Transformer-based architectures. By systematically reducing input image scale, we construct scale-accuracy response curves and define a characteristic scale as a compact measure of the onset of substantial recognition degradation. We then examine the relationship between characteristic scale and baseline recognition accuracy, model parameter count, architectural family, and representation stability. A strong inverse association is observed between baseline accuracy and characteristic scale (Pearson r = -0.890, R^2= 0.792, p < 10^-6). This relationship remains stable under bootstrap resampling, leave-one-architecture-out analysis, and leave-one-family-out analysis. In contrast, parameter count provides negligible additional explanatory power after controlling for baseline accuracy (p = 0.80), while architectural family does not provide significant incremental explanatory power. Furthermore, characteristic scale shows essentially no association with representation stability (r = -0.003, p = 0.991). These results indicate that, across the studied models, scale robustness is strongly organized by baseline recognition performance rather than simply by model size, architectural family, or representation stability. The study provides an empirical framework for characterizing scale robustness across vision architectures and identifies a reproducible accuracy-scale regularity that warrants further theoretical investigation.

1 Introduction

Prior work documents scale sensitivity and proposes mechanisms for improving scale robustness, but whether robustness follows a common quantitative structure across modern architectures remains unclear. This study compares characteristic scale with recognition accuracy and other model properties across architectures.

1 Introduction

  • Earlier work reports scale sensitivity and investigates multi-scale processing, shared representations, scale-aware architectures, and resolution robustness in contemporary models.These studies include improved robustness from weight-shared multi-stage convolutional networks and scale-channel architectures for invariant representations.Source: §1 · pp. 1–2
  • Existing studies examine robustness within individual architectures, compare selected models, or develop scale-invariance mechanisms, leaving cross-architecture structure and its explanatory factors unclear.The unresolved question is whether characteristic scale is architecture-specific or systematically associated with recognition capability.Source: §1 · p. 2
  • The study measures classifiers continuously as input scale decreases, defines characteristic scale from the response curve, and evaluates it across 20 pretrained ImageNet-1K architectures in seven families.Rather than proposing a new architecture or training procedure, it tests reproducible relationships with model properties.Source: §1 · p. 2
  • The analysis tests baseline accuracy, parameter count, architectural family, and representation stability as potential explanations of characteristic-scale variation across a population of models.This frames robustness as a statistical property across models rather than a performance characteristic of one architecture.Source: §1 · p. 2

2 Methods and Experimental Framework

The study compares characteristic scale across 20 pretrained classifiers using a common scale-reduction protocol, then tests its association with accuracy, model size, family, and representation stability. It also uses matched comparisons and robustness checks to assess the relationships.

2.1 Research Question

  • Characteristic scale is examined against baseline recognition capability, capacity, and architecture, alongside representation stability across image scales.The framing treats scale robustness as a population-level statistical property rather than a single-model performance characteristic.Source: §2.1 · p. 2

2.2 Models and Dataset

  • Twenty pretrained classifiers from seven architectural families are evaluated on the same ImageNet-1K validation data and preprocessing protocol, spanning about 3.5–88.6 million parameters.Baseline Top-1 accuracy is measured at the reference scale before scale perturbations.Source: §2.2 · p. 2

2.3 Image-Scale Perturbation and Characteristic Scale

  • Input images are tested at systematically reduced scales, and characteristic scale is the smallest relative scale at which performance retains a predefined fraction of baseline.A smaller characteristic scale indicates greater tolerance to scale reduction; the procedure is applied uniformly across architectures.Source: §2.3 · pp. 2–3

2.4 Determining the Factors Associated with Scale Robustness

  • The analysis uses correlation and OLS regression, then tests whether parameter count and architecture family add explanatory power beyond baseline accuracy.The extended model includes log parameter count; family is represented categorically and assessed with partial F-tests and permutation testing.Source: §2.4 · p. 3

2.5 Accuracy-Matched and Robustness-Residual Analysis

  • Models with similar baseline accuracy are paired, and robustness residuals compare observed characteristic scale with the accuracy-based prediction.Positive residuals indicate greater-than-expected scale tolerance, while negative residuals indicate lower-than-expected tolerance.Source: §2.5 · p. 3

2.6 Statistical Robustness Analysis

  • Bootstrap resampling, permutation tests, and leave-one-architecture-out and leave-one-family-out analyses assess the stability of the accuracy–scale relationship.Figure 1 plots one point per architecture and reports a strong negative association between baseline accuracy and characteristic scale.Source: §2.6 · p. 3, Figure 1 · p. 4

2.7 Representation Stability Analysis

  • Representation stability is summarized with an AUC measure and compared with characteristic scale using correlation, permutation, bootstrap, and exclusion analyses.The analysis tests whether more stable internal representations under scale reduction necessarily accompany greater image-scale robustness.Source: §2.7 · p. 4

3 Results and Discussion

Across the evaluated architectures, characteristic scale is strongly associated with baseline recognition accuracy, while parameter count, architectural family, and representation stability provide little independent explanation. Architecture-specific residuals capture variation above or below accuracy-based expectations.

3.1 Characteristic Scale is Strongly Associated with Baseline Recognition Accuracy

  • Pearson r = −0.890 (p = 1.52 × 10^-7) and R2 = 0.792 describe the strong baseline-accuracy association; bootstrap 95% CI for r was [−0.961, −0.776].The association remained negative in all bootstrap samples, and higher-accuracy models generally reached the characteristic-scale criterion at smaller image scales.Source: §3.1 · pp. 4–5, Figure 2 · p. 5

3.2 Model Capacity Provides Little Additional Explanatory Power

  • Adding parameter count increased R2 by only approximately 0.001 and its coefficient was nonsignificant (p = 0.80), despite a size-only model explaining 37.3% of variance.The combined accuracy-and-size model achieved R2 = 0.793.Source: §3.2 · p. 5

3.3 The Accuracy–Scale Relationship is Not Driven by Individual Architectures or Families

  • Architecture exclusions kept Pearson r between −0.914 and −0.857, and family exclusions kept it between −0.915 and −0.845; adding family was nonsignificant (p = 0.550).The family model's nominal R2 increase from 0.792 to 0.859 was not supported by the partial F-test or permutation test.Source: §3.3 · pp. 5–6

3.4 Architecture-Specific Deviations Reveal Models That Exceed Accuracy-Based Expectations

  • Residuals identify above-expectation scale tolerance (Swin-T +0.057) and below-expectation tolerance (MobileNetV2 −0.055); residual robustness was unrelated to parameter count (r = −0.046, p = 0.846).Positive and negative residuals indicate greater- and lower-than-expected tolerance relative to baseline-accuracy predictions.Source: §3.4 · p. 6, Figure 3 · p. 6

3.5 Representation Stability Does Not Explain Characteristic Scale

  • Characteristic scale and representation-stability AUC were unassociated (Pearson r = −0.003, p = 0.991); the bootstrap 95% CI [−0.528, 0.480] spans both signs.The lack of association persisted under model and family exclusions.Source: §3.5 · pp. 6–7, Figure 4 · p. 7

3.6 Findings

  • Across the evaluated models, baseline recognition capability was the dominant empirical predictor, while architecture-specific residuals accounted for additional variation beyond accuracy alone.The findings also report negligible added explanatory power from parameter count and no measurable cross-architecture association with representation stability.Source: §3.6 · p. 7

4 Limitations

The study identifies limits on what its scale-robustness findings establish: characteristic scale is an empirical descriptor, and the observed associations do not establish mechanisms or universal scope.

4 Limitations

  • Characteristic scale is an empirical descriptor from the scale–response curve, not a universal architectural constant.Its interpretation is limited to the empirical characterization used in this study.Source: §4 · p. 8
  • The accuracy–scale association does not identify the underlying causal mechanism.The statistical relationship is robust, but that alone does not establish causality.Source: §4 · p. 8
  • The absent association with aggregate representation stability does not rule out localized or layer-specific mechanisms.Source: §4 · p. 8
  • The relationship’s consistency supports reliability within this study, while broader validation across datasets and visual tasks remains future work.Source: §4 · p. 8

5 Conclusion

Across 20 pretrained architectures, baseline recognition accuracy was strongly and consistently inversely related to characteristic scale. The findings position recognition capability, rather than model size or architectural family alone, as the dominant empirical correlate of scale tolerance.

5 Conclusion

  • Across 20 pretrained architectures, higher baseline accuracy was associated with smaller characteristic scales, with the relationship stable across multiple robustness analyses.The study reports bootstrap, permutation, leave-one-architecture-out, and leave-one-family-out analyses.Source: §5 · p. 8
  • Parameter count contributed substantially less explanatory power, while architectural family added limited explanatory power after recognition accuracy was considered.Source: §5 · p. 8
  • The characteristic-scale formulation provides a quantitative framework for comparing scale robustness across heterogeneous vision models.The findings suggest scale tolerance may be associated with recognition performance rather than directly caused by parameter count or architecture family.Source: §5 · p. 8

Declarations

The declarations report no specific funding or competing interests. The study uses publicly available ImageNet-1K data, and supporting materials are available upon request.

Funding

  • The author received no specific funding for this work.Source: Funding · p. 8

Conflict of Interest

  • The author declares no known competing financial interests or personal relationships that could have influenced the reported work.Source: Conflict of Interest · p. 8

Data Availability

  • ImageNet-1K was used under its licensing and access policies; processed data and additional results are available from the corresponding author upon reasonable request.Source: Data Availability · p. 8

Reproducibility Statement

  • Source code, evaluation and statistical analysis scripts, and figure-generation pipelines can be shared by the corresponding author upon reasonable request.Source: Reproducibility Statement · p. 8

Related papers

Loading 2609.06051v1…