PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Balanced Accuracy: A Resolution and Parity-Controlled Benchmark for Vision-Language and Vision-Only Defect Assessment in UAV Power-Line Inspection

Linghao Zhang, Siyu Xiang, Junwei Kuang, Peiyu Yi

arXiv:2609.27457v1cs.CV

TL;DR

VLMs are often reported to outperform task-specific vision backbones in UAV power-line defect assessment, but the evidence depends on how comparisons are set up. This study audits those choices on ElecVQA-Bench; after split-specific retraining and replication, no split regime shows a corrected family-level advantage.

  • Problem

    VLMs are often reported to outperform purpose-trained vision backbones for UAV power-line defect assessment, motivating a controlled comparison of the two families.

    Source: §1 · p. 1

  • Method

    The study audits partition, item set, label space, replication, resolution, and side information in comparisons on ElecVQA-Bench.

    Source: §1 · p. 2

  • Results

    After split-specific retraining and correction, no split regime shows a significant Qwen–ResNet advantage; crop and image comparisons favour ResNet-50.

    Source: §7 · p. 33

  • Takeaways & Limitations

    A VLM-versus-vision-only comparison requires reporting partition, evaluated item set, label space, seed, split construction, resolution, and side information together.

    Source: §7 · p. 33

    Filename-derived parent_id disjointness does not guarantee source-photograph disjointness, and the primary global results have only two seeds each.

    Source: §6.4 · p. 32

Abstract

from arXiv · show

Vision-language models (VLMs) are often reported to outperform task-specific vision backbones for unmanned aerial vehicle (UAV) power-line defect assessment. We test that claim on ElecVQA-Bench, a 56,972-item benchmark derived from the public InsPLAD dataset, across six evaluation choices: partition, evaluated item set, label space, replication, input resolution, and side information. On a matched partition, a Swin Transformer and the strongest adapted VLM differ by only 0.03 points at binary screening. At seven-way defect typing, increasing the vision backbones from 224 px to the measured pixel budget of the VLM preprocessor narrows the gap against InternVL3.5-8B from +20.53 to -0.57 points for ResNet-50 and from +23.67 to +4.70 points for Swin-T. A pixel-budget audit shifts Qwen3-VL-8B macro recall by 10.78 points, yet a source-pixel-matched InternVL control still leaves Qwen ahead by 7.43 to 13.61 points while using 56% fewer visual tokens, so neither source pixels nor token budget explains the difference between the two VLMs. A two-seed global replication changes Qwen binary accuracy and seven-way macro recall by 0.86 and 1.02 points. After split-specific retraining, Qwen does not lead at crop or image level, and a 14-tower, three-seed replication reverses the sign across seeds, giving mean common-six macro recall of 0.9085 for Qwen against 0.9509 for ResNet-50. No split regime yields a family-level advantage that survives multiple-comparison correction. The study supports a benchmark-audit contribution rather than a general claim of VLM superiority.

1. Introduction

The paper audits claims that VLMs outperform vision-only backbones by controlling six comparison choices. After those controls, it reports no stable family-level ordering on ElecVQA-Bench.

1. Introduction

  • Once six experimental choices are controlled, no stable family-level ordering remains on this benchmark.The authors frame the contribution as a benchmark audit, not a general claim of VLM superiority.Source: §1 · p. 2
  • A mismatched test file produced a 13–15-point VLM lead, but rescoring both families on the same 2,021-item file removed the detectable difference.The initial comparison used a seven-class file for vision backbones and a binary file for VLMs.Source: §1 · p. 2
  • At seven-way typing, resolution matching narrowed the InternVL3.5-8B gap from +20.53 to −0.57 points for ResNet-50 and from +23.67 to +4.70 for Swin-T.The pixel-budget control did not close the gap against Qwen3-VL-8B.Source: §1 · p. 2
  • A 14-tower, three-seed replication reversed the tower-level comparison across seeds, with ResNet-50 leading in two of three matched seeds.The reported seed standard deviations were 0.5 points for ResNet-50 and 6.2 for Qwen.Source: §1 · p. 2
  • The audit covers partition, evaluated item set, label space, replication, input resolution, and side information; VLMs also receive asset names, a decision tree, and candidate options.Cross-family comparisons are therefore labeled system-level comparisons.Source: §1 · p. 2
  • Balanced accuracy can conceal miss-, hallucination-, or abstention-dominated errors; zero-price abstention leaves 20% of defective items unjudged despite a zero miss rate.The paper extends its cost-sensitive risk function to price refusal.Source: §1 · p. 2
  • A cost-ratio crossover derived on the constructed benchmark does not transfer directly to field prevalence two orders of magnitude lower.The paper provides a closed-form rebasing and resulting surface.Source: §1 · pp. 2–3
  • Two classes with nine test instances between them can move the seven-class macro-average by 22.9 points in opposite directions.ElecVQA-Bench contains 56,972 items and is built from public InsPLAD data [1].Source: §1 · p. 3

2. Related Work

The related-work review situates the study in power-line inspection, cost-sensitive evaluation, long-tailed recognition, VLMs, and parameter-efficient adaptation. It identifies evaluation methodology—not model architecture—as its focus.

Power-line inspection from UAV imagery

  • Power-line inspection studies report detection or classification accuracy on their own partitions; the review found no study verifying partition and label-space equivalence before family comparisons.The authors characterize their review as selective, not systematic.Source: §2 · p. 3

Cost-sensitive evaluation

  • Cost-sensitive evaluation motivates assessing classifiers across plausible cost ratios rather than relying on a single aggregate accuracy [11].The paper adapts this reasoning to misses, hallucinations, and abstentions in industrial visual question answering.Source: §2 · p. 3

Class imbalance and long-tailed recognition

  • The seven-way task is long-tailed, with a class having four test instances; standard class-balancing mitigations are not applied here [16] [17].The paper quantifies the consequences of that choice for its headline comparison.Source: §2 · p. 3

Vision–language models and hallucination evaluation

  • The review treats hallucination and refusal separately because the benchmark has a closed answer space and an explicit abstention token.It relates the hallucination measure to industrial-inspection analogues of CHAIR [23] and POPE.Source: §2 · p. 4

Parameter-efficient adaptation and statistical testing

  • The study uses LoRA [25] for adaptation and ResNet-50 [29] and Swin-T [30] as its vision-only baselines.Paired classifier comparisons use McNemar’s exact test.Source: §2 · p. 4

3. Materials: ElecVQA-Bench

ElecVQA-Bench is constructed from InsPLAD subsets through audited crop generation, task-specific questions, and grouped splits. Its protocol specifies benchmark statistics, prompts, and the asymmetries that make cross-family results system-level comparisons.

3.1. Source Data and Construction

  • The construction audits source data, expands annotated boxes at four ratios, builds T1–T3 items, and groups items by parent_id for a stratified split.The pipeline also asserts split integrity and writes digests for splits, scripts, and prompts.Source: §3.1 · pp. 4–5

3.2. Benchmark Statistics

  • The released Table 1 presents ElecVQA-Bench composition; realized split counts, rather than target ratios, are authoritative because group sizes are uneven.Realized proportions can deviate from the 70/10/20 target by up to 1.5 percentage points.Source: Table 1 · p. 5, §3.2 · p. 5
  • T3 contains 36,370 audited crops; T2’s test set has 1,255 normal and 766 defective items, giving a realized defect prevalence of 0.379.T3 uses seven ground-truth classes, including peeling-paint and torned-up with four and five test instances, respectively.Source: §3.2 · pp. 5–6

3.3. Prompts and Evaluation Protocol

  • P3 prompts supply an asset name, description, decision tree, question, and candidate options; the main experiments use P3 for training and inference.Guided decoding constrains VLM outputs to JSON, with format-failure rate reported separately from accuracy.Source: §3.3 · p. 6
  • VLMs receive asset-specific candidate options and textual side information, while vision-only classifiers receive pixels and a fixed label set; comparisons are therefore system-level.Impossible VLM predictions affect 7 of 7,284 items, and oracle correction changes balanced accuracy by at most 0.55 points.Source: §3.3 · p. 6

4. Methods

The methods define miss, hallucination, and abstention rates, then use them to expose balanced accuracy’s blind spot and calculate risk under refusal costs and prevalence changes. A six-factor parity audit and controlled model comparisons accompany cluster-aware statistical testing.

4.1. Notation

  • Balanced accuracy is expressed through miss, hallucination, and class-conditional abstention rates, with true-positive and true-negative rates defined from those quantities.The identity is also used to cross-validate measured abstention rates.Source: §4.1 · p. 6

4.2. The Balanced-Accuracy Blind Spot

  • For a fixed balanced accuracy, feasible error profiles can be dominated by misses, hallucinations, or abstentions, so the metric alone cannot distinguish them.The argument identifies a blind spot in a symmetric two-class summary [11].Source: §4.2 · pp. 6–7

4.3. Cost-Sensitive Risk with Abstention

  • The cost-sensitive risk function prices misses, false alarms, and abstentions relative to false alarms; the analysis restricts abstention cost to 0 ≤ κ ≤ ρ.This is an inspection-setting assumption, not a universal claim; the model comparison can yield a crossover or dominance verdict.Source: §4.3 · p. 7

4.4. Prevalence Sensitivity

  • A crossover cost ratio depends on defect prevalence, so a benchmark-derived ratio carries an implicit prevalence assumption and requires rebasing for another prevalence.Equation (4) applies to the zero-cost abstention case; the general surface is given by Equation (3).Source: §4.4 · p. 8

4.5. Parity Audit

  • The parity audit checks test files, item counts, label spaces, evaluated item identities, seed counts, and side-information sets.Different side information permits only a system-level comparison, not a claim of equivalent input information.Source: §4.5 · p. 8

4.6. Models, Adaptation, and Statistical Testing

  • The comparison uses ResNet-50 [29] and Swin-T [30] as vision-only baselines alongside adapted VLMs; LoRA [25] adaptation trains a small parameter fraction.The vision encoder is frozen during adaptation, while vision-only baselines use ImageNet initialization.Source: §4.6 · pp. 8–9
  • Seven-way and three-way macro recall average over different label partitions and weights, so neither score is constrained to exceed the other.Input resolution is controlled; the InternVL3.5-8B audit records a 768 px-square-equivalent pixel budget, between the 672 px and 896 px backbone settings.Source: §4.6 · p. 9
  • Paired item contrasts use McNemar’s exact test, while primary macro-recall inference uses cluster-level sign-flip randomization with Holm correction.The cluster unit varies by split, and all experiments used approximately 205 GPU-hours.Source: §4.6 · p. 9

5. Results

On matched full-data binary screening, the best vision-only configuration and best adapted VLM are separated by just 0.03 percentage points, while zero-shot profiles and cost rankings depend on abstention and prevalence. Adaptation and training choices affect the measured comparisons, and unequal few-shot evaluations do not support a cross-family ranking.

5.1. Zero-Shot Error Profiles

  • 42.0% to 74.3% is the zero-shot T2 balanced-accuracy range across seven models; 2B models are hallucination-heavy, while 8B–27B models are miss-heavy.InternVL3.5-2B's zero miss rate still leaves 20.1% of defective items unjudged.Source: §5.1 · pp. 10–11

5.2. Cost-Sensitive Ranking and Its Assumptions

  • At the benchmark's 37.9% defect prevalence, the accuracy and risk rankings invert at a cost ratio of 1.8 to 2.7, depending on review cost.At 1% prevalence, the 42%-accuracy model is preferable only if a missed defect costs roughly a hundred false alarms; this is a sensitivity result, not a routine operating assumption.Source: §5.2 · p. 12

5.3. Adaptation

  • Qwen3-VL-8B is 11.18 points above InternVL3.5-8B at seven-way typing but only 1.3 points above it at binary screening.The two-seed replication changes Qwen's T2 and seven-way macro recall by 0.86 and 1.02 points; two seeds remain too few for a precise seed-variance estimate.Source: §5.3 · pp. 12–14

5.4. Multi-Task Mixing and Epoch Budget

  • The mixed corpus exceeds both specialists on T2 by 2.4 and 5.8 points, but training-volume effects cannot be separated without a volume-matched control.Training from three to ten epochs leaves T2 essentially unchanged as evaluation loss rises from 0.035 to 0.099 and the five-class T3 diagnostic falls by 10.1 points; the S200 T3 diagnostic is not comparable to seven-class results.Source: §5.4 · pp. 14–15, Table 7 · p. 14

5.5. Partition Parity at Binary Screening

  • Swin-T leads ResNet-50 at every training budget, with the margin closing to 0.6 points at full data; no cross-family few-shot comparison is reported.The available VLM ladder used a 200-item subsample while the backbones used the full partition, with unequal replication.Source: §5.5 · p. 15
  • 0.03 percentage points separate the best full-data vision-only configuration and best adapted VLM on matched binary screening; p = 1.000 is not an equivalence test.An earlier 13–15-point VLM lead came from a comparison that failed the parity audit.Source: §5.5 · p. 15

5.6. Label-Space Parity: Seven-Way Defect Typing

Across seven-way typing, resolution, rare-class sampling, and prompt information materially shape the comparisons between VLMs and vision-only backbones. The controlled-factor summary leaves the difference between the two adapted VLMs unresolved and identifies no mechanism that explains it.

5.6. Label-Space Parity: Seven-Way Defect Typing

  • At 224 px, seven-way typing reverses the binary-row ordering: the vision-backbone gaps are 20.53 and 23.67 points against InternVL, and 31.71 and 34.85 against Qwen.The passage reports the gaps for ResNet-50 and Swin-T, respectively.Source: §5.6 · p. 15, Table 10 · p. 16

5.7.1. Resolution and Pixel-Budget Control

  • At 672 px, the gaps against InternVL narrow from +20.53 to −0.57 points for ResNet-50 and from +23.67 to +4.70 for Swin-T.The measured VLM pixel budget is equivalent to a 768 px square image; 672 and 896 px are the bracketing vision-backbone settings.Source: §5.7.1 · pp. 16–17, Table 11 · p. 17
  • 10.78 points: lowering Qwen’s pixel budget reduces macro recall, yet source-pixel-matched tests leave Qwen 7.43–13.61 points ahead of InternVL while using 56% fewer visual tokens.The source-pixel control does not isolate the mechanism behind the VLM difference.Source: §5.7.1 · pp. 18–19, Table 16 · p. 19, Figure 4 · p. 20

5.7.2. Per-Class Decomposition

  • In the seven-class macro-average, this VLM’s entire 2.01-point advantage over this vision-only run resides in peeling-paint, while the vision backbone leads by 12.3 points in aggregate on the other six classes.The two rarest classes hold nine test items between them, which can move the headline number by 22.9 points in opposite directions.Source: §5.7.2 · pp. 20–21

5.7.3. Prompt Ablation

  • Without the decision tree, both VLMs fall to 14.29% macro recall through majority-class prediction; restoring it yields inference-time gains of 54.97 points for InternVL and 70.07 for Qwen.The ablation identifies language grounding as necessary for these VLMs’ fine-grained typing, but does not explain the difference between models.Source: §5.7.3 · p. 22, Table 19 · p. 22, Figure 6 · p. 23

5.7.4. Summary of the Controlled Factors

  • At native settings, the adapted VLMs differ by 1.3 points at binary screening and 11.18 points at seven-way typing; pixel-budget controls leave the difference intact without explaining it.Tokenizer design, patch formation, pre-training composition, and visual encoding remain candidate explanations; the study lacks a matched-visual-token comparison.Source: §5.7.4 · p. 23

5.8. Quantization and Deployment Envelope

Quantization and deployment measurements characterize operating trade-offs, while split-specific retraining and replication test whether the model-family ordering is stable. No split regime yields corrected evidence that Qwen outperforms ResNet-50, and the larger replication shows marked seed sensitivity.

5.8. Quantization and Deployment Envelope

  • NF4 reduces memory by 61% at a 3% throughput cost, while INT8 reduces memory by 44% at a 24% throughput cost; the NF4 preference is limited to this single-GPU, 50-item measurement.The study treats this as a measured operating point, not a general deployment recommendation.Source: §5.8 · p. 23, Table 21 · p. 25
  • At 448 px, ResNet-50 has the highest throughput and Swin-T the lowest memory; workstation results omit disk input, decoding, energy, thermal effects, and embedded accelerators.The two families’ throughput measurements use different harnesses and batch policies, so the paper reports them separately rather than as one ranking.Source: §5.8 · p. 25, Table 22 · p. 25, Figure 7 · p. 26

5.9. Split-Level Retraining and Tower Replication

  • After split-specific retraining, Qwen is slightly behind at crop level and 4.40 points behind at image level; its +8.02-point tower-level lead does not survive Holm correction.No split regime provides corrected evidence of Qwen outperforming ResNet-50 under the common-six-class protocol.Source: Table 23 · p. 27, §5.9 · p. 28
  • At the nine-tower split, Qwen’s two-seed mean is 0.7604 versus ResNet-50’s 0.7044, but the seed-matched difference varies by 4.73 points and its inferential status is seed-sensitive.The larger tower14 replication reverses the sign across seeds; Qwen’s common-six mean is 0.9085 versus ResNet-50’s 0.9509, with seed standard deviations of 6.2 versus 0.5 points.Source: §5.9 · pp. 28–29, Table 27 · p. 28, Figure 8 · p. 29
  • The single-seed Swin-T control reaches 0.9695 common-six macro recall, but tower ordering varies with backbone, seed, and label space and identifies no stable winner.The control fills baseline coverage; it does not add a replicated family-level claim.Source: §5.9 · p. 29, Table 28 · p. 29, Figure 8 · p. 29
  • Seed 44’s lower common-six score reflects two torned-up errors; excluding that three-item class yields 0.9377 five-class macro recall, supporting rare-class seed sensitivity rather than general degradation.The rerun reproduced the errors, and both images remained classified as defective under the binary prompt.Source: §5.9 · pp. 29–30

6. Discussion

Controlled comparisons remove the apparent binary-screening lead and show that seven-way results depend on evaluation choices rather than establishing a family-level ordering. The discussion identifies practical trade-offs, data and deployment limits, and controls needed to resolve remaining questions.

6.1. What the Controlled Comparisons Establish

  • 0.03 points separate the best vision-only backbone and adapted VLM at matched binary screening, while seven-way comparisons support no family-level ordering.The binary result is a balanced discordant-pair finding, not a formal equivalence test; split-specific and tower-level comparisons vary in direction.Source: §6.1 · p. 30
  • 11.18 points at seven-way typing and 1.3 points at binary screening separate Qwen3-VL-8B from InternVL3.5-8B, but the tested controls do not explain the difference.Source-pixel matching does not remove the gap, and Qwen leads while using 56% fewer visual tokens; the evidence does not exclude resolution, language grounding, or scale as contributors.Source: §6.1 · p. 30

6.2. Why the Seven-Way Result Is Not an Architecture Result

  • Pixel budget accounts for most of the gap against InternVL, while language grounding is shared by both adapted VLMs and scale contributes at most 3.74 points.Both VLMs fall to 14.29% without the decision tree; Swin-T rules out only the coarse convolutional-versus-transformer contrast.Source: §6.2 · pp. 30–31
  • Two classes with nine test items can move the seven-class macro-average by 22.9 points in opposite directions, with the influential rare class changing by split.The same mechanism appears across configurations; unweighted macro-averaging turns rare-class sampling into headline differences.Source: §6.2 · p. 31

6.3. Practical Implications

  • For binary screening, a fine-tuned vision backbone matches the best adapted VLM; for fine-grained typing, results are split- and seed-dependent, leaving deployment choice unresolved.The VLM’s supported practical distinction is text-editable side information: restoring the decision tree recovers 54.97 to 70.07 points, depending on the model.Source: §6.3 · p. 31

6.4. Limitations

  • The study is limited to T2 and T3 on one dataset without cross-domain evaluation, and filename-derived grouping does not guarantee source-photograph disjointness.Other constraints include unquantified annotation noise, small or incomplete replication, non-unseen-tower crop and image regimes, and workstation-only deployment measurements on ground-truth crops.Source: §6.4 · p. 32

6.5. Future Work

  • Future comparisons should match visual-token budgets, preregister larger tower replications, and measure all metrics and deployment costs in common instrumented settings.Further directions include calibrated VLM confidence for full cost curves and replacing ground-truth boxes with a trained detector.Source: §6.5 · pp. 32–33

7. Conclusions

The study concludes that most headline differences reflect evaluation protocol choices, not a general advantage for either model family. Its controlled results leave a VLM-to-VLM difference unexplained and support an audited benchmark protocol as the main contribution.

7. Conclusions

  • Most headline differences are properties of the evaluation protocol rather than of the models, and the study supports an audited benchmark protocol rather than a family-level winner.Partition, evaluated item set, label space, seed, split construction, input resolution, and side information must be reported together for cross-family comparisons to be interpretable.Source: §7 · p. 33
  • At matched binary screening the partition tie is 97.17% against 97.14%, while raising vision-backbone resolution narrows the seven-way gaps against InternVL3.5-8B.The reported resolution changes are +20.53 to −0.57 points for ResNet-50 and +23.67 to +4.70 points for Swin-T.Source: §7 · p. 33
  • Qwen3-VL-8B macro recall shifts from 78.61% at a 65,536-pixel budget to 89.39% at native resolution, but source pixels and token budget do not explain its gap over InternVL.In the source-pixel-matched control, Qwen remains 7.43 to 13.61 points ahead while using 56% fewer visual tokens.Source: §7 · p. 33
  • No split regime yields a significant Qwen–ResNet advantage after correction; the 14-tower replication favors ResNet-50 in two of three seeds.Mean common-six macro recall is 0.9085 for Qwen and 0.9509 for ResNet-50, with greater seed variability for Qwen.Source: §7 · p. 33
  • Balanced accuracy hides materially different miss, hallucination, and abstention profiles, and zero-priced review selects a model that leaves 20% of defective items unjudged.That selection no longer holds with realistic review costs and field prevalence.Source: §7 · p. 33
  • Rare-class support can move an unweighted macro-average by more than ten points, and adapted-VLM run-to-run instability can exceed the family-level difference being measured.The difference between adapted VLMs remains real in magnitude but unresolved in mechanism.Source: §7 · pp. 33–34

Data Availability Statement

The authors plan to release construction and evaluation materials and adapter weights, while not redistributing the source imagery.

Data Availability Statement

  • Construction scripts, split manifests, prompts, evaluation materials, derived metadata, and LoRA weights will be released; public InsPLAD imagery will not be redistributed.The materials are planned for a public repository and DOI archive upon acceptance, under licences compatible with the source dataset’s public release terms.Source: Data Availability Statement · pp. 34–35

Abbreviations

The paper uses abbreviations for model families, metrics, training methods, and evaluation quantities.

Abbreviations

  • CNN means convolutional neural network; ViT, vision transformer; VLM, vision–language model; UAV, unmanned aerial vehicle; VQA, visual question answering.ROI means region of interest.Source: Abbreviations · p. 35
  • BalAcc means balanced accuracy; TNR, true negative rate; TPR, true positive rate; EAR, error-asymmetry ratio; SD, standard deviation; CI, confidence interval.S200 means the shared 200-item evaluation subsample.Source: Abbreviations · p. 35
  • LoRA means low-rank adaptation; QLoRA, quantized low-rank adaptation; NF4, 4-bit NormalFloat; INT8, 8-bit integer; FP8, 8-bit floating-point; bf16, 16-bit brain floating-point.Source: Abbreviations · p. 35

Appendix A

Appendix A documents the experiment crosswalk, supplementary diagnostics, and configuration and provenance details. Its results include a mixed-corpus diagnostic gain alongside an over-fitting signal, and its notes delimit which test-set comparisons are valid.

Appendix A

  • Appendix tables provide experiment crosswalks, notation and label references, test-partition class support, software and API provenance, an experiment ledger, and smoothing-constant sensitivity.The crosswalk distinguishes the main adaptation pass from the instrumented pixel-budget pass; the S200 five-class diagnostic is not comparable to seven-class results.Source: Table A2 · p. 38, Table A1 · p. 41, Table A3 · p. 41
  • The fixed S200 subsample is used only in Tables 7 and A8; crop, image, tower, and tower14 tests are separate diagnostics, not comparable to one another or original T3.S200 contains 200 items drawn once with seed 42; Table 4's InternVL-2B value uses the full 2,021-item test set, unlike Table 7's S200 value.Source: Appendix A · p. 40
  • Hosted-service zero-shot results are provenance snapshots, not version-pinned reproductions, because stable weight-revision hashes are unavailable.Source: Appendix A · p. 40
  • InternVL3.5-8B varies by a factor of fourteen across the tested range, and the 27B and 122B models change order.Source: Appendix A · p. 40
  • The mixed corpus exceeds the T3-only specialist by 8.6 points and the T2-only specialist by 12.5 points; extending training from three to ten epochs reduces it by 10.1 points as evaluation loss rises from 0.035 to 0.099.The authors identify the rising evaluation loss as the over-fitting signal.Source: Appendix A · p. 40
  • The appendix also documents the experiment ledger and error-asymmetry-ratio sensitivity, while Table A8 reports a five-class S200 diagnostic that is not comparable to seven-class results.Table A8's S200 set has no peeling-paint or torned-up instances.Source: Table A6 · p. 45, Table A7 · p. 46, Table A8 · p. 46
Loading 2609.27457v1…