PaperlayerView exact revision on arXiv

Source-linked AI summary

When Does Geometric View Synthesis Help Wine Label Retrieval? A Public One-Shot Benchmark Across Self-Supervised and Vision-Language Backbones

Yueh-Cheng Huang

arXiv:2609.33359v1cs.CV

TL;DR

It is unclear whether geometric view synthesis adds value when one-shot retrieval starts from strong pretrained image encoders. This paper evaluates synthesis and adaptation on a public wine-label benchmark, finding a large benefit for the tested self-supervised model and a smaller recipe-dependent benefit for the text-supervised model.

  • Problem

    The contribution of geometric view synthesis when retrieval starts from strong pretrained encoders, and the backbone and adaptation conditions under which it helps, remain unclear.

    Source: IEEE ACCESS 1 · p. 2

  • Method

    The paper evaluates geometric synthesis and adaptation recipes on a public benchmark of wine-label retrieval using pretrained self-supervised and text-supervised encoders.

    Source: §VII · p. 15, §VIII · p. 17

  • Results

    Geometric synthesis adds about 30 points over frozen DINO, while frozen SigLIP 2-B reaches 94.7% and gains about one point only through an adaptation that preserves its features.

    Source: §VII · p. 15, §VIII · p. 17

  • Takeaways & Limitations

    The tested results favor preserving pretrained features: synthesis is a substitute for a strong prior and a small, recipe-dependent supplement to one.

    Source: §VII · pp. 15–16

    The benchmark inherits unverified crowd-sourced class identities, possible shared photo sessions, unchecked pretraining overlap, and a one-annotator audit of 50 of 226 errors.

    Source: §VII · p. 16

Abstract

from arXiv · show

Geometric view synthesis can expand a single wine-label photograph into a training set, but its value with pretrained image encoders is unclear. We study this on a public WineSensed-derived benchmark of 1,000 classes, one enrollment photograph per class, and 4,295 real queries. With the earlier DINO vision transformer (ViT-S/16) recipe, geometric views raise top-1 accuracy from 34.1% to 62.6-63.7%, about three times the gain from two-dimensional (2D) augmentation. Frozen SigLIP 2-B already reaches 94.7%. A linear head over its frozen features gains 1.2-1.3 percentage points with the two geometric pipelines localized by the Segment Anything Model (SAM), while the other pipelines gain an inconclusive 0.3-0.6 points. Low-rank adaptation (LoRA) and validation-selected full fine-tuning show no clear gain within the reported confidence intervals; fixed-budget full fine-tuning loses 9-24 points. SAM localization supplies all six views for 99% of sources, compared with 43% for the edge-based front end. Recognition differences between the two cylinder constructions depend on the training recipe and are confounded by their crop and canvas conventions. Rendered-cylinder tests show different responses to source tilt, but an uncalibrated rim-ratio proxy establishes no corresponding trend in recognition on real photographs. An author-confirmed audit of 50 residual errors identifies 21 query-enrollment appearance mismatches, without establishing an irreducible error rate. These results support geometric synthesis for the tested self-supervised recipe and a smaller benefit through frozen-feature adaptation of the text-supervised encoder.

I. INTRODUCTION

The paper asks whether geometric view synthesis still helps once retrieval starts from strong pretrained encoders, and tests that question on a public one-shot wine-label benchmark. Results separate by backbone and adaptation recipe: synthesis adds substantially to the tested self-supervised model but only modestly to the text-supervised model.

IEEE ACCESS 1

  • The public benchmark contains 1,000 label classes, one enrollment photograph per class, and 4,295 real queries, addressing the limited reproducibility of prior results on private datasets.It is derived from the WineSensed release.Source: §I · p. 1, IEEE ACCESS 1 · p. 2
  • Geometric synthesis adds about 30 points over frozen DINO, while the frozen SigLIP 2-B starts at 94.7% and gains 1.2–1.3 points with a linear head and SAM-localized views.LoRA shows no clear gain within the reported confidence intervals, and fixed-budget full fine-tuning loses 9–24 points.Source: IEEE ACCESS 1 · pp. 2–3
  • SAM localization supplies all six views for 99% of sources, compared with 43% for the edge-based front end.The two cylinder constructions’ recognition differences depend on the training recipe and are confounded by crop and canvas conventions; an audit found 21 visibly mismatched query-enrollment pairs among 50 residual errors, without establishing an irreducible error rate.Source: IEEE ACCESS 1 · p. 2
  • The study evaluates four view-synthesis families and multiple adaptation strategies against frozen encoders on two backbones, with multi-view enrollment as a training-free alternative.The benchmark uses one enrollment photograph per label and 4,295 real queries; the design includes matched budgets, three seeds, and class-cluster confidence intervals.Source: IEEE ACCESS 1 · pp. 2–3, Figure 1 · p. 2
  • The work builds on earlier conference papers,, while identifying the benchmark, experiments, and text reported here as new.Its contributions also include a construction comparison and analysis of synthesized-view distances and residual errors.Source: IEEE ACCESS 1 · p. 3

II. RELATED WORK

Prior wine-label systems report strong results but rely on private collections, while geometric synthesis and pretrained embeddings offer distinct routes to one-shot retrieval. This study places these approaches in context and compares synthesis pipelines that vary in localization and cylinder construction.

A. WINE-LABEL RECOGNITION

  • Earlier wine-label recognizers combined a CNN producer search with local-feature matching, reporting 82–84% sub-brand accuracy on a proprietary collection unavailable for independent comparison.An OCR-and-database alternative was evaluated on 45 bottles; commercial applications publish no methodology.Source: A. WINE-LABEL RECOGNITION · p. 3

LABELS

  • Geometric synthesis re-projects a label photographed on a cylinder; Li et al. assume a frontal input, while the earlier pipeline estimates rims and a vanishing point.Li et al. reported main-brand accuracy rising from 0.716 with image copying to 0.757; generative augmentation is outside this paper’s scope.Source: LABELS · p. 3

C. RETRIEVAL WITH PRETRAINED EMBEDDINGS

  • Frozen self-supervised and image–text embeddings are established retrieval approaches, and prior work reports benefits from linear adaptation and local-feature re-ranking.The paper evaluates image-only retrieval, with OCR and local-feature re-ranking only as controls; catalog-to-real retrieval is not pursued because there is one enrollment image per class.Source: C. RETRIEVAL WITH PRETRAINED EMBEDDINGS · p. 3

III. VIEW-SYNTHESIS PIPELINES

  • The four pipelines comprise a 2D control and three geometric variants, enabling pairwise comparisons of localization and cylinder construction.The edge-based and SAM pipelines share the construction of, while the frontal-assumption pipeline uses SAM localization and the construction of; comparisons account for fallback coverage and rendering conventions.Source: §III · pp. 3–4, Figure 2 · p. 4

A. CYLINDRICAL MODEL AND CROSS-RATIO

The rim-estimating construction maps label pixels between source and target cylinder poses using projective geometry. Its front end can use edge-based or SAM localization, while the compared cylinder constructions also differ in crop and canvas conventions.

RESAMPLING

  • The rim-estimating construction detects upper and lower rims and a vanishing point, then resamples source pixels onto a cylinder at the desired pose.The mapping uses corresponding source and target line endpoints and the vanishing point; the cylinder’s visible surface angle is fixed at 160 degrees.Source: RESAMPLING · p. 4
  • Cross-ratio equality maps points along each source label line to a corresponding target line, with the central-line mapping computed exactly and scaled to other lines.The visible surface angle is fixed at 160 degrees, and the paper measures the cost of the central-line simplification.Source: RESAMPLING · p. 4

B. FRONT ENDS

  • SAM localization uses a prompted label mask to fit the rims and infer the edges, while edge-based localization fits rim ellipses and finds common external tangents.The edge detector was designed for higher-resolution photographs than this benchmark; the two cylinder constructions also use different crop and canvas conventions, limiting fairness of their comparison.Source: B. FRONT ENDS · p. 5

C. 2D CONTROL

The 2D control applies stochastic transformations to the whole enrollment photograph, while geometric failures receive same-source 2D fallbacks so classes remain represented consistently. Rendered-cylinder validation finds that central-line sampling is close to exact projection and that the constructions respond differently to source tilt.

C. 2D CONTROL

  • The 2D control makes six whole-photograph views per source using geometric, blur, color, channel, and dropout or noise operations.A flip-free bank checks the role of horizontal flipping, but the random draws of later operations differ, so the banks are not image-by-image comparisons.Source: C. 2D CONTROL · p. 5

D. SLOTS AND FALLBACK

  • Each source keeps one original and six synthesis slots, with failed geometric slots filled by fixed same-source 2D views rather than dropping classes.The edge-based pipeline natively synthesized 2,588 of 6,000 views, compared with 5,958 for SAM and 5,988 for frontal-assumption.Source: D. SLOTS AND FALLBACK · p. 5

E. GEOMETRIC VALIDATION

  • The central-line cross-ratio simplification has 0.60 px mean displacement, close to 0.54 px for exact per-line projection, while equidistant sampling has 4.0 px mean displacement.The test covered 22 pose pairs where all three sampling variants were defined.Source: E. GEOMETRIC VALIDATION · p. 6
  • After correcting near-vertical-line handling, the geometric pipeline’s maximum displacement is 18.6 px and its mean over poses is 4.1 px.The renderer used ground-truth masks, so these measurements assess geometry and sampling rather than segmentation.Source: E. GEOMETRIC VALIDATION · p. 6
  • Under source tilt from 0 to 40 degrees, rim-estimating displacement stays between 4.1 and 5.3 px, while frontal-assumption displacement rises from 26.8 to 79.6 px.The frontal-assumption curve has a fixed offset from crop, canvas, and radius adaptation, so the comparison is between slopes rather than levels.Source: E. GEOMETRIC VALIDATION · p. 6, Figure 3 · p. 6

IV. RETRIEVAL, ADAPTATION AND BASELINES

The retrieval setup compares frozen DINO ViT-S/16 and SigLIP 2-B embeddings and tests synthesized views through several adaptation recipes and training-free multi-view enrollment. The recipes differ in how much of the encoder they update, while retrieval remains one enrollment photograph per class.

A. BACKBONES AND FROZEN RETRIEVAL

  • SigLIP 2-B uses L2-normalized pooled features and cosine ranking, while DINO uses unnormalized CLS features and Euclidean ranking; no crop, fusion, or re-ranking is used in the main results.Source: A. BACKBONES AND FROZEN RETRIEVAL · p. 6

B. ADAPTATION WITH SYNTHESIZED VIEWS

C. MULTI-VIEW ENROLLMENT

  • Training-free multi-view enrollment adds six synthesized views to the gallery and compares maximum, mean, cosine-weighted mean, and views-only scoring; a 2D pipeline controls for adding vectors.Source: C. MULTI-VIEW ENROLLMENT · p. 7

A. SOURCE DATA

The study uses WineSensed vintage identifiers to build a public one-shot benchmark, reports frozen-backbone results, and compares geometric synthesis with 2D augmentation for DINO. Its data and evaluation procedures also define important scope boundaries for interpreting the results.

A. SOURCE DATA

  • WineSensed vintage identifiers are the only label available at scale, but can group different editions of a label and do not support producer-level labels across the full benchmark.Source: A. SOURCE DATA · p. 7

B. CLASSES AND SPLITS

  • The benchmark contains 1,000 classes, one enrollment image per class, and 4,295 queries; a nested 100-class subset supports ablations, while a separate 1,384-class split supports development and selection.Source: B. CLASSES AND SPLITS · p. 8

C. METRICS AND STATISTICS

  • Top-1 is primary, with query-weighted results reported by default; class-cluster intervals reflect benchmark class composition, not variation between training seeds.Source: C. METRICS AND STATISTICS · p. 8

D. RELEASE

  • The public package includes image identifiers, hashes, rankings for 73 runs, and rebuild scripts, but excludes images, features, and weights.Source: D. RELEASE · p. 8

A. FROZEN BACKBONES

  • Frozen SigLIP 2-B reaches 94.7% top-1 on the benchmark, while frozen DINO ViT-S/16 reaches 34.1%.Source: A. FROZEN BACKBONES · p. 9, Table I

DECISIVE

  • After 4,000 updates, DINO ViT-S/16 top-1 accuracy reaches 62.6–63.7% across three geometric pipelines, gaining 28.6–29.6 points over frozen; the SAM-based pipeline gains 19.3 points over 2D augmentation.Source: DECISIVE · p. 9, Figure 4 · p. 9

Pipeline

For DINO, the geometric pipelines outperform 2D augmentation, while differences among the geometric pipelines are not distinguishable from zero at benchmark scale. Removing horizontal flips does not explain the weaker 2D result.

Pipeline

  • The SAM-based and frontal-assumption pipelines are 0.9 and 1.0 points above the edge-based pipeline, respectively, but neither difference is distinguishable from zero.Source: Pipeline · p. 10
  • On the 100-class subset after 1,000 updates, SAM-based synthesis reaches 83.0% versus 79.1% for edge-based synthesis, a 3.9-point difference.Source: Pipeline · p. 10
  • Removing horizontal flips lowers 2D accuracy to 42.1%, 2.2 points below the original control and 21.5 points below the SAM-based pipeline.Source: Pipeline · p. 10
  • The DINO result reproduces the finding of on public data and the direction of that a cylinder model beats 2D augmentation.Source: Pipeline · p. 10

C. TEXT-SUPERVISED BACKBONE: THE GAIN DEPENDS ON THE ADAPTATION RECIPE

With SigLIP 2, the value of synthesized views depends on the adaptation recipe: a linear head over frozen features yields the clearest gain, while full fine-tuning and multi-view enrollment do not improve on the frozen model. The fixed-budget full fine-tuning results lose accuracy, whereas validation-selected geometric runs return near parity.

C. TEXT-SUPERVISED BACKBONE: THE GAIN DEPENDS ON THE ADAPTATION RECIPE

Pipeline

  • Validation-selected geometric pipelines finish within 0.7 points of frozen accuracy, while 2D remains 4.8 points behind; the selection rule chooses checkpoints at 100 or 250 updates.Source: Pipeline · p. 10, Table IV · p. 11
  • Validation accuracy declines alongside embedding drift, but similar early drift accompanies different accuracy, so drift is descriptive rather than a demonstrated mechanism.Source: Pipeline · p. 11, Figure 5 · p. 11
  • Maximum aggregation in training-free multi-view enrollment changes top-1 by only +0.05 points for SAM-based views and +0.02 for frontal-assumption views; averaging lowers accuracy for every pipeline.Source: Pipeline · p. 11, Figure 6 · p. 11
  • A linear head over frozen features gains 1.24 points with SAM-based views and 1.30 with frontal-assumption views; LoRA intervals include zero, and fixed-budget full fine-tuning loses 9–24 points.Source: Pipeline · pp. 11–12, Table V · p. 12

D. FRONT-END RELIABILITY

The SAM front end supplies all six views for nearly every source, unlike the edge-based front end. Recognition comparisons show recipe-dependent pipeline differences, but do not isolate cylinder geometry from rendering conventions.

D. FRONT-END RELIABILITY

  • SAM supplies all six views for 993 of 1,000 sources, versus 431 for the edge-based front end.An earlier development study likewise found complete coverage for 98 of 100 sources with SAM and 41 with the edge-based front end.Source: D. FRONT-END RELIABILITY · p. 12
  • On edge-based queries, trained accuracy is similar for sources with native views and fallback views, but those subsets differ in frozen-model difficulty.For SigLIP 2, trained accuracy is 81.1% on both groups, while frozen accuracy is 96.0% for native-source queries and 93.6% for the rest.Source: D. FRONT-END RELIABILITY · p. 12
  • The SAM-versus-edge comparison on sources where both localizers succeeded does not establish a better-view benefit: validation-selected models are indistinguishable on those queries.At fixed budget, SAM-trained models lead on both native and other-source queries, so the difference cannot be attributed to views from jointly successful sources alone.Source: D. FRONT-END RELIABILITY · p. 12
  • SAM masks average 0.927 intersection over union on 50 annotated photographs, but 150 of 200 photographs were not annotated.This is a sample-only mask check, not a benchmark-wide estimate.Source: D. FRONT-END RELIABILITY · p. 13
  • With SigLIP 2, the frontal-assumption construction is 1.67 points below rim-estimating under SAM localization; construction differences vary by recipe and are confounded by crop, canvas, and scale.Under the edge-based front end, frontal-assumption is 0.23 points above rim-estimating; with DINO, construction differences are not distinguishable from zero.Source: D. FRONT-END RELIABILITY · p. 13
  • After 4,000 updates, SAM raises SigLIP 2 accuracy over the edge-based front end by 4.41 points with rim-estimating and 2.52 with frontal-assumption construction.For DINO, the corresponding differences are 0.94 and 1.96 points, with intervals including zero.Source: D. FRONT-END RELIABILITY · p. 13, Table VI · p. 13

E. RE-RANKING AND OCR

Development controls find no clear accuracy gain from local-feature re-ranking or OCR over the frozen SigLIP 2 embedding. Re-ranking can favor shared label templates over product-specific differences.

E. RE-RANKING AND OCR

  • Direct local-feature promotion in development ranged from 90 to 95 correct across methods, versus 94 without re-ranking.The best setting rescued two queries and broke one; broken cases often shared a producer template but differed in product name or year.Source: E. RE-RANKING AND OCR · pp. 13–14
  • Re-ranking produced a difference of +0.7 points, with a 95% interval of −0.67 to +2.00; the cited passage does not state the underlying correct-answer counts.By the stopping rule fixed in advance, no further variants were tried.Source: E. RE-RANKING AND OCR · p. 14
  • OCR re-ordering reached 88 correct at K = 5 and 84 at K = 10 and 20, versus 94 for the embedding alone.These were single development studies of particular implementations, and OCR did not enter the main results.Source: E. RE-RANKING AND OCR · p. 14

F. ERROR ANALYSIS

The frozen SigLIP 2 model’s residual errors are often near the top of the ranking, and an audit finds recurring query–enrollment appearance mismatches. The audit is descriptive and does not establish an accuracy ceiling.

F. ERROR ANALYSIS

  • Of 226 frozen SigLIP 2 errors, 107 rank the true class second, 147 place it in the top five, and the median true-class rank is 3.Thirty-six errors rank the true class beyond 100.Source: F. ERROR ANALYSIS · p. 14, Figure 7 · p. 14
  • The near-duplicate flag marks 129 of 226 errors but also 2,280 of 4,069 correct queries’ second-ranked enrollments, so it does not distinguish error categories.It flags 10 of 11 sibling confusions and 15 of 21 appearance mismatches.Source: F. ERROR ANALYSIS · pp. 14–15
  • In an author audit of 50 errors, 21 involve visibly different query and enrollment labels, while 11 are same-layout sibling-label confusions and 16 have a top-ranked label with a different layout.Visual inspection alone does not verify wine identity.Source: F. ERROR ANALYSIS · p. 14
  • Nine of the 21 appearance mismatches belong to one class whose supermarket-branded enrollment label differs from the producer labels in its queries.The multimodal language model flagged these nine cases, and the author confirmed them individually.Source: F. ERROR ANALYSIS · p. 14, Figure 7 · p. 14
  • The 50-case audit is descriptive, identity was not independently verified, and its counts should not be extrapolated to an irreducible error rate.Visibly different labels need not be impossible to match.Source: F. ERROR ANALYSIS · pp. 14–15

G. DISTANCE BETWEEN SYNTHESIZED VIEWS AND PHOTOGRAPHS

In frozen SigLIP 2 features, geometric views are farther from their source photographs than real query–enrollment pairs, and the uncalibrated rim-ratio proxy does not establish a corresponding recognition trend. The discussion places this finding alongside the tested adaptation recipes, front-end coverage, construction differences, and study limitations.

G. DISTANCE BETWEEN SYNTHESIZED VIEWS AND PHOTOGRAPHS

  • Mean cosine distance is 0.24 for frontal-assumption views and 0.29 for SAM-based views, versus 0.10 for real query–enrollment pairs and 0.18 and 0.15 for 2D and flip-free 2D copies.Geometric views are thus farther from their source photographs in frozen SigLIP 2 space; this is a feature-space observation, not evidence about training dynamics.Source: Figure 8 · p. 15, G. DISTANCE BETWEEN SYNTHESIZED VIEWS… · p. 15
  • The proxy-dependent feature-distance association differs between pipelines, but no corresponding proxy-dependent trend in recognition contrast is established.The rim-ratio proxy is uncalibrated, and the authors say the results preclude a causal interpretation.Source: G. DISTANCE BETWEEN SYNTHESIZED VIEWS… · p. 15

H. COST

  • On the RTX 3080, median query embedding takes 12.0 ms for SigLIP 2-B and 9.3 ms for DINO ViT-S/16; exact search takes 0.09 and 0.05 ms, respectively.These are separately timed implementation-specific measurements, not end-to-end figures.Source: H. COST · p. 15

VII. DISCUSSION

  • The tested results favor preserving pretrained features: geometric views give a small, recipe-dependent supplement to a strong prior, while serving as a substitute when the backbone lacks one.The authors do not claim that no adaptation recipe could gain more than a point, and the backbones were not compared under a common recipe.Source: §VII · pp. 15–16
  • Removing horizontal flips improves the 2D pipeline for SigLIP 2 but lowers accuracy by 2.2 points under the tested self-supervised recipe; geometric synthesis retains more accuracy than 2D augmentation under fine-tuning.The authors note that this comparison does not establish the proposed mismatch explanation as the mechanism.Source: §VII · p. 16
  • SAM supplies all six views for 99% of sources versus 43% for the edge-based front end, but the source of recognition differences between front ends was not isolated.On sources where both succeed, selected comparisons do not distinguish the models; the 100-class subset is not an independent replicate.Source: §VII · p. 16
  • Rendered tests show greater displacement growth with tilt for the frontal-assumption construction, but its crop-and-canvas adaptation leaves a 27 px zero-tilt error.On real photographs, recognition contrasts depend on recipe, and the uncalibrated proxy establishes no change across tertiles.Source: G. DISTANCE BETWEEN SYNTHESIZED VIEWS… · p. 15, §VII · pp. 16–17
  • The audit of 50 errors found visibly different query and enrollment labels, but neither it nor the unverified crowd-sourced vintage metadata establishes an irreducible error rate.The human audit covered 50 of 226 errors and was performed by one annotator.Source: §VII · p. 16

VIII. CONCLUSION

Frozen SigLIP 2-B substantially outperforms frozen DINO on the public one-shot benchmark, while geometric synthesis benefits depend on the backbone and adaptation recipe. The conclusion also identifies limits on interpreting geometry comparisons and makes benchmark materials available for further testing.

VIII. CONCLUSION

  • 94.7% top-1 for frozen SigLIP 2-B and 34.1% for frozen DINO ViT-S/16 on 1,000 classes with one enrollment photograph each.The benchmark includes 4,295 real queries.Source: §VIII · p. 17
  • About 30 percentage points from geometric synthesis versus about 10 from the original 2D control under the recovered DINO recipe; SAM-based linear-head adaptation reaches about 96.0% for SigLIP 2.Source: §VIII · p. 17
  • LoRA and validation-selected full fine-tuning show no clear gain within reported intervals, fixed-budget full fine-tuning loses accuracy, and multi-view enrollment changes accuracy by less than a quarter point.Source: §VIII · p. 17
  • SAM localization raises complete synthesis coverage from 43% to 99%, but cylinder-construction differences depend on recipe and rendering conventions, and the pose proxy shows no real-image recognition trend.The controlled renderer shows different responses to source tilt; the error audit finds appearance mismatches without establishing an accuracy ceiling.Source: §VIII · p. 17
  • Public identifiers, splits, rankings, and evaluation tools support further adaptation tests and benchmarks with independently verified identities.Source: §VIII · p. 17

DATA AVAILABILITY

  • The evidence release provides identifiers, splits, hashes, synthesis parameters, per-query rankings, and evaluation code; images remain available from WineSensed and are not redistributed.Source: DATA AVAILABILITY · p. 17

Related papers

Loading 2609.33359v1…