PaperlayerView exact revision on arXiv

Source-linked AI summary

FDSML: Formula-Driven Supervised Metric Learning with Parameter-Aware Triplet Loss

Hina Otake, Shinei Arakawa, Keitaro Tanaka, Yoshihiro Fukuhara, Hirokatsu Kataoka, Shigeo Morishima

arXiv:2610.39423v1cs.CVcs.AIcs.LGcs.NE

TL;DR

Classification-based FDSL discretizes generation parameters, discarding continuity in the generative process. FDSML combines FineFractalDB with a triplet objective that derives relational supervision from parameter space. It improves image retrieval over classification-based FDSL while remaining competitive on classification and improving ResNet-50 corruption robustness, with architecture-dependent trade-offs for ViT.

  • Problem

    Classification-based FDSL discretizes continuous generation parameters, losing fine-grained geometric relationships, while representation quality matters for retrieval beyond classification accuracy.

    Source: §3.2 · p. 3

  • Method

    FDSML pairs FineFractalDB's increased intra-class diversity with PA-Triplet Loss, which orders positives and negatives using distances in the generative parameter space.

    Source: §1 · p. 1, §3.3 · p. 3

  • Results

    FDSML substantially improves image retrieval over classification-based FDSL while remaining competitive on classification and improving ResNet-50 corruption robustness, with architecture-dependent trade-offs for ViT.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    Parameter-derived relative orderings provide a synthetic pre-training signal that can improve transfer beyond recognition.

    Source: §3.3 · p. 3, §6 · p. 8

    Validation is limited to IFS-based fractal pre-training; applying the approach to other generators requires generator-specific similarity definitions and empirical validation.

    Source: §5.6 · p. 8

Abstract

from arXiv · show

Formula-Driven Supervised Learning (FDSL) enables vision pre-training without real images, but existing methods cast pre-training as classification on discretized labels, discarding continuity in the generative process. We propose Formula-Driven Supervised Metric Learning (FDSML), which derives relational supervision from image-generation parameters beyond discrete class labels. FDSML combines FineFractalDB, our refined fractal dataset with higher intra-class variation and more stable structures, with a Parameter-Aware Triplet (PA-Triplet) Loss that assigns positives and negatives according to distances in the underlying Iterated Function System parameter space. This objective trains embeddings to respect parameter-derived relative similarity orderings. Experiments show that FDSML improves transferability to image retrieval while remaining competitive on downstream classification and improving robustness under several corruption settings. On ResNet-50, Recall@1 improves from 65.6% to 74.0% on Stanford Online Products and from 56.2% to 78.4% on Stanford Cars over FDSL pre-training. These results suggest that parameter-derived relational supervision provides a stronger synthetic pre-training signal than label-based classification for transfer beyond recognition.

1. Introduction

Formula-driven pre-training avoids reliance on real images, but classification-based FDSL can learn representations that remain limited for retrieval and robustness. FDSML uses parameter-derived relationships to train structured embeddings, and improves transfer beyond classification-based FDSL.

1. Introduction

  • Real-image pre-training faces annotation costs, privacy concerns, and social biases, motivating synthetic data as a scalable, privacy-compliant alternative.FDSL generates images from mathematical formulas and assigns labels from their generation parameters.Source: §1 · p. 1
  • FDSL can achieve high classification accuracy yet learn representations that differ from real-data representations, limiting retrieval and sometimes underperforming from-scratch training in target domains.The introduction also notes sensitivity to perturbations such as blur.Source: §1 · p. 1
  • The paper attributes this limitation to synthetic-category recognition that can overfit localized, low-level geometric textures rather than learn relationships among visual structures.It proposes shifting FDSL from recognition toward relational comparison.Source: §1 · p. 1
  • FDSML combines FineFractalDB, with more parameter perturbations and higher intra-class diversity, and PA-Triplet Loss, which assigns positives and negatives using parameter-space relative orderings.The resulting embeddings are trained to respect parameter-derived similarity orderings.Source: §1 · pp. 1–2
  • FDSML improves transfer beyond classification-based FDSL, notably on image retrieval, while improving classification benchmarks and ResNet-50 corruption robustness with architecture-dependent ViT trade-offs.It also produces qualitative differences in activation and attention patterns.Source: §1 · p. 2

2. Related Work

The related work situates FDSML among synthetic-data approaches and metric-learning methods. It contrasts formula-driven data with other generators and motivates retrieval as a probe of embedding geometry.

2.1. Learning from Synthetic Data

  • Real-image-free visual learning can use formula-driven datasets such as FractalDB, which generates images from Iterated Function Systems and has been extended to other mathematical structures.The broader synthetic-data motivation includes avoiding costs, licensing restrictions, privacy violations, and implicit biases associated with real-image corpora.Source: §2.1.1 · p. 2, §2.1 · p. 2

2.1.1. Formula-Driven Supervised Learning (FDSL).

  • Most FDSL pipelines discretize continuous generation parameters into classes, discarding neighborhood structure; FDSML instead uses parameter-space relations as ordinal supervision for structured embeddings.Formula-driven datasets synthesize both images and supervision from explicit mathematical rules.Source: §2.1.1 · p. 2

2.1.2. Data-Free Procedural Programs and Stochastic Generative Processes.

  • Procedural programs and stochastic generative processes can support representation learning without real-world data, but may lack a continuous, interpretable parameter space for fine-grained geometric control.Source: §2.1.2 · p. 2

2.1.3. Model-Generated Synthetic Data (Data-Dependent Generators).

  • Diffusion-based synthetic datasets and text-to-image corpora support representation learning, but their generators are trained on real-image corpora and may inherit those data's biases.This work instead focuses on formula-driven data independent of real-image-trained generators.Source: §2.1.3 · p. 2

2.2. Metric Learning and Image Retrieval as Representation Evaluation

  • Retrieval probes whether embedding geometry reflects meaningful neighborhood structure, while FDSML derives its relative-similarity supervision from continuous IFS parameters rather than only discrete labels.Prior metric-learning approaches include label-based constraints and augmentation-based self-supervision.Source: §2.2 · pp. 2–3

3. Method

FDSML reframes formula-driven pre-training as relational comparison, using continuous IFS parameter distances to supervise embeddings rather than relying only on discrete categories. FineFractalDB supplies greater structural diversity and stability for this metric-learning objective.

3. Method

  • The method combines metric-learning reformulation, PA-Triplet supervision, and a refined fractal dataset to support relational pre-training.The section presents these components as a framework tailored to the continuous generative nature of fractal images.Source: §3 · p. 3

3.1. Background: Fractal Generation with Iterated Function System

  • An IFS generates fractals through probabilistically selected contraction mappings; in FractalDB, each mapping is an affine transformation in R2.The six affine parameters and selection probabilities determine the generated attractor geometry.Source: §3.1 · p. 3

3.2. FDSML: Formula-Driven Supervised Metric Learning

  • FDSML replaces classification over discretized labels with relative-order supervision suited to fractals’ continuous parameter space.The paper argues that classification may encourage memorization of instance-specific patterns that need not generalize to downstream tasks.Source: §3.2 · p. 3

3.3. Parameter-Aware Triplet Loss

  • PA-Triplet assigns positives and negatives by relative IFS parameter-space distance, training embedding distances to respect the resulting similarity orderings.The parameter distance is used ordinally to decide which candidate is closer, not as a perceptual distance.Source: §3.3 · p. 3, Figure 1 · p. 4

3.4. Filtering Triplets with Discrete Labels

  • Discrete labels restrict triplet candidate pools, while parameter-space distance orderings still determine positive and negative assignments.The all, each, same, and diff strategies vary whether candidates come from the whole dataset, the anchor’s class, or other classes.Source: §3.4 · p. 4, Figure 2 · p. 4

3.5. Fine-Grained Fractal Database

  • FineFractalDB increases perturbations to 1,000 per class, enforces strict contraction constraints, and rejects samples with pixel occupation below 0.05%.The refined dataset exhibits greater structural variation and fewer degraded samples than FractalDB.Source: §3.5 · pp. 4–5

4. Experiments

The experiments use FineFractalDB for pre-training and evaluate classification, retrieval, and corruption robustness. They cover ResNet-50 and ViT backbones with separate optimization schedules for convolutional and transformer models.

4.1. Pre-training Dataset

  • FineFractalDB pre-training uses 1,000 categories with 1,000 synthesized images per class, totaling 1 million samples.Source: §4.1 · p. 5

4.2.1. Image Classification.

  • Classification is evaluated by fine-tuned Top-1 accuracy on CIFAR-10/100, ImageNet-1k/100, Places-30, Stanford Cars, and Flowers-102.ImageNet-100 uses a predefined 100-class subset, while Places-30 selects 30 Places365 categories.Source: §4.2.1 · p. 5

4.2.2. Image Retrieval.

  • Retrieval is assessed with Recall@K on Stanford Online Products, Stanford Cars, and In-Shop Clothes Retrieval.The reported K values are {1, 10, 100} for SOP, {1, 4, 16} for Cars, and {1, 10, 20} for In-Shop.Source: §4.2.2 · p. 5

4.2.3. Robustness.

  • Robustness evaluation uses CIFAR-10-C and ImageNet-C to test distribution shifts and out-of-distribution corruptions.Source: §4.2.3 · p. 5

4.3.1. Network Architectures.

  • The evaluated backbones are ResNet-50, ViT-Tiny, and ViT-Base, with their classification heads replaced by embedding or projection layers.ResNet-50 uses a 512-dimensional linear embedding; ViTs use two-layer MLP projection heads.Source: §4.3.1 · p. 5

4.3.2. Optimization and Training Schedule.

  • ResNet-50 is pre-trained for 134 epochs with SGD, while ViT models use AdamW, a 10,000-step warm-up, cosine decay, and 536 epochs.The ViT schedule accounts for their slower convergence on synthetic data.Source: §4.3.2 · p. 5

5. Experimental Results

Experiments test whether parameter-derived supervision yields reliable similarity orderings and improves transfer across classification, robustness, retrieval, and qualitative representation analyses. The results also identify limits in parameter-descriptor reliability and evaluation scope.

5.1. Validating the Parameter Descriptor

  • The descriptor ordering matches structural closeness in 89.2% of each-filtered and 80.6% of same-filtered triplets, but falls below chance for all (47.2%) and diff (46.8%).For same, labels cannot supply this within-class ordering; for each, the label rule reaches 89.4% and agrees with the descriptor on 99.6% of triplets.Source: §5.1 · p. 6

5.2. Downstream Classification

  • FDSML improves synthetic-pretraining classification across architectures: ResNet-50 is competitive, ViT-Tiny reaches 88.90% on ImageNet-100 and 98.00% on CIFAR-10, and ViT-Base leads overall.ImageNet-1k pre-training remains the strongest ResNet-50 baseline; ViT-Tiny's CIFAR-10 result slightly exceeds its 97.92% ImageNet-1k supervised baseline.Source: §5.2 · pp. 6–7, Table 3 · p. 6, Table 4 · p. 6

5.2.1. Ablation Studies.

  • With data fixed to FractalDB-1k, PA-Triplet improves all three benchmarks; FineFractalDB adds gains under FDSML but not under FDSL, supporting complementarity.Cross-entropy does not directly exploit FineFractalDB's increased within-class variation, whereas PA-Triplet constructs relational supervision from sample orderings.Source: Table 5 · p. 6, §5.2.1 · p. 7

5.3. Robustness Analysis

  • For ResNet-50, distance-based pre-training beats synthetic FDSL across all five corruption categories, with FDSML (same) best in three.ViT-Tiny shows trade-offs: Weather improves by 6.3% relatively (47.64 vs. 50.84), while Noise and Digital errors worsen.Source: §5.3 · p. 7, Table 6 · p. 7

5.4. Image Retrieval

  • FDSML consistently outperforms scratch and FDSL pre-training on retrieval across three datasets, while ImageNet-1k remains an upper baseline.FDSML (same) performs best on SOP and Stanford Cars and is on par with the best variant on In-Shop.Source: Table 7 · p. 7, §5.4 · pp. 7–8

5.5.1. Grad-CAM Visualization on ResNet-50.

  • Grad-CAM maps from FDSML are more spatially concentrated on object-relevant regions and qualitatively closer to ImageNet maps than FDSL maps.FDSL responses are often more diffuse and less localized.Source: Figure 4 · p. 7, §5.5.1 · p. 8

5.5.2. Visualizing Attention Maps.

  • ViT FDSML attends to textured and interior regions as well as prominent boundaries, unlike FDSL's boundary-focused attention.The broader distribution suggests sensitivity to fine-grained appearance variation in addition to coarse structure.Source: §5.5.2 · p. 8, Figure 5 · p. 8

5.6. Limitations

  • Evaluation is limited to IFS-based fractal pre-training and FineFractalDB's one-million-image scale; descriptor and triplet-construction overhead also remains unanalyzed.Other generators would need their own parameter-space similarity definitions and empirical validation; persistence of gains at larger scales remains open.Source: §5.6 · p. 8

6. Conclusion

FDSML uses continuous generative parameters for relational rather than category-based synthetic pre-training. The paper reports stronger retrieval transfer than classification-based FDSL while remaining competitive in classification, and supports its parameter descriptor with structural-distance validation.

6. Conclusion

  • FDSML reformulates synthetic pre-training as relational learning, using PA-Triplet and FineFractalDB to supervise embeddings with generative-parameter orderings.The paper reports improved retrieval over classification-based FDSL, competitive downstream classification, and ResNet-50 corruption-robustness gains with architecture-dependent ViT trade-offs.Source: §6 · p. 8

A. Empirical Validation of the Parameter Descriptor

  • The descriptor distance dΨ ranks candidates against an anchor, and symmetric Chamfer distance D supplies the structural reference for validating that ranking.D avoids tunable perceptual-metric parameters and does not assume a prior over natural images.Source: A. Empirical Validation of the… · p. 12

A.a. Response to Controlled Generative Perturba-

Controlled parameter sweeps test whether descriptor distance tracks structural change in rendered fractals. It generally preserves the ordering under contractive perturbations, while results weaken when the contraction condition is violated.

A.a. Response to Controlled Generative Perturba-

  • Each sweep rescales one affine coefficient across all maps by s∈[0.6, 1.4] at 33 values and recomputes selection probabilities from determinant magnitudes.D averages nearest-counterpart foreground-pixel distances; seed-only rerenderings estimate its noise floor.Source: A.a. Response to Controlled Generative… · p. 12
  • Within each of six directions, dΨ and D had ρ≥0.94, while pooling directions yielded ρ=0.831 because response slopes differed by roughly fourfold.Thus dΨ orders structural change within directions but is not calibrated in magnitude across directions.Source: A.a. Response to Controlled Generative… · p. 12
  • 91.4% of contractive sweeps had nondecreasing D; allowing decreases within each sweep’s noise floor raised this to 99.0%, with median ρ=0.977.The evaluation covered 300 sweeps across 50 base codes and six affine coefficients.Source: A.a. Response to Controlled Generative… · p. 12
  • Figure A shows D increasing monotonically on both sides of the base parameter in all six directions, with only its immediate neighborhood within the 0.53 px noise floor.The largest displacement per direction was 2.2–3.6 px, four to seven times the noise floor.Source: A.a. Response to Controlled Generative… · pp. 12–13
  • 49.0% of sweeps violating contraction had nondecreasing D, and 10 of 102 sampled sweeps produced empty renderings.These results support contraction as a condition for locally meaningful parameter distances.Source: A.a. Response to Controlled Generative… · p. 13

A.b. Qualitative Examples of Filtered Triplets

Filtered-triplet examples show that descriptor-based rankings depend on the candidate regime: they capture fine within-class differences better than comparisons among structurally unrelated samples. FineFractalDB also presents clearer structures than the original FractalDB examples.

A.b. Qualitative Examples of Filtered Triplets

  • Under same, both candidates share the anchor’s class and differ in fine structure; under each, the descriptor almost always selects the within-class candidate, while all and diff mostly pair unrelated structures.Figure C visualizes the anchor and two candidates in each filtering strategy.Source: A.b. Qualitative Examples of Filtered… · p. 13
  • Within each perturbation direction, dΨ tracks structural change nearly perfectly by rank, but differing slopes prevent magnitude calibration across directions.Figure B plots Chamfer distance D against descriptor distance dΨ for the perturbation sweeps.Source: A.b. Qualitative Examples of Filtered… · p. 13

B. FineFractalDB

  • Original FractalDB samples can have collapsed, visually indistinct patterns because affine transformations are not strictly constrained to be contractive.Figure D displays randomly selected samples from the original dataset.Source: B. FineFractalDB · p. 13
  • FineFractalDB samples use continuous parameter perturbations with contraction constraints, yielding clearer fractal structures than the original examples.Displayed instances may look similar because only a limited number of continuously varied samples are shown.Source: B. FineFractalDB · p. 13

C.a. Grad-CAM Visualization

Grad-CAM comparisons show FDSML activations more closely resemble the ImageNet-pretrained reference than FDSL activations, and often focus on the target object when the reference highlights background. ViT attention visualizations distinguish the cues emphasized by FDSML and its variants.

C.a. Grad-CAM Visualization

  • FDSML activations more closely resemble the ImageNet-pretrained model than FDSL activations, and FDSML often focuses on the object where the reference highlights background.The authors interpret these examples as indicating more consistent reliance on structurally meaningful cues for recognition.Source: C.a. Grad-CAM Visualization · p. 14
  • Mean Grad-CAM similarity to the ImageNet-pretrained reference rose from 0.7892 for FDSL to 0.8434 for FDSML (each) and 0.8462 for FDSML (same).The similarity distributions in Fig. G quantitatively support the qualitative comparisons in Fig. F.Source: C.a. Grad-CAM Visualization · p. 17

C.b. Visualization of Attention Maps

  • FDSL attention concentrates mainly on object boundaries, whereas FDSML attends to both edges and textured regions; FDSML (each) emphasizes texture more than FDSML (same).The authors suggest that same emphasizes subtle structural differences, while each emphasizes larger variations and appearance cues.Source: C.b. Visualization of Attention Maps · p. 17

C.c. Visualization of Learned Embedding Space

UMAP visualizations compare class organization and interpolated parameter trajectories in learned embeddings. FDSML variants show different patterns of intra-class structure and overlap, while FDSML (same) follows gradual parameter variation more smoothly in the projection than FDSL.

C.c. Visualization of Learned Embedding Space

  • The UMAP comparison projects pre-final-layer ResNet-50 features for 10 randomly selected FineFractalDB test classes into two dimensions.For the continuity visualization, two parameter sets are linearly interpolated, rendered across 5000 evenly spaced α values, encoded, and projected with UMAP.Source: C.c. Visualization of Learned… · p. 17
  • FDSML (each) forms more structured within-class trajectories while remaining separated across classes, whereas FDSML (same) shows greater overlap across classes.The authors note that inter-class separation under each is expected because its assignments nearly coincide with class-label supervision; same uses intra-class variation and may extend continuous structure beyond class boundaries.Source: C.c. Visualization of Learned… · p. 17
  • FDSL’s UMAP trajectory shifts abruptly around α≈0.6, whereas FDSML (same) follows the interpolation progression smoothly in the projection.The projection alone does not establish continuity in the original high-dimensional embedding space.Source: C.c. Visualization of Learned… · pp. 17–18

D. Robustness Analysis

On ResNet-50, every FDSML variant reduces classification error in four CIFAR-10-C corruption groups, while Noise improves only for FDSML (all); on ViT-Tiny, all variants improve across all five groups. The visualizations also show differences in activation similarity and parameter-interpolation trajectories.

D. Robustness Analysis

  • For ResNet-50, every FDSML variant reduces classification error in Blur, Weather, Digital, and Geometric groups, but only FDSML (all) improves in Noise.FDSML (each) reduces Blur error from 35.49% to 27.08%; FDSML (same) reduces Geometric error from 21.55% to 18.47%.Source: D. Robustness Analysis · p. 18
  • On CIFAR-10-C, every ViT-Tiny FDSML variant reduces classification error in all five corruption groups, unlike its ImageNet-C results in Noise and Digital.FDSML (all) reduces Blur error from 13.70% to 9.94%; FDSML (each) reduces Digital error from 10.09% to 7.30%.Source: D. Robustness Analysis · p. 18
  • FDSML (all) ranks first or second in nine of ten architecture–corruption combinations; FDSML (same) does so in four of five ResNet-50 groups.FDSML (same) has the lowest ResNet-50 error in Weather and Geometric, while FDSML (each) has the lowest ViT-Tiny error in Noise and Digital.Source: D. Robustness Analysis · p. 18
  • On ResNet-50, FDSML (diff) yields the smallest gains in most groups, with its cross-class comparisons excluding within-class comparisons whose descriptor ordering is reliable.For Noise, the authors suggest the broad candidate pool of FDSML (all) may promote stability across instances, while narrower pools may increase sensitivity to high-frequency perturbations; ViT-Tiny FDSML (each) nevertheless has the lowest Noise error.Source: D. Robustness Analysis · p. 18

FDSL FDSML (each) FDSML (same)

  • FDSML variants’ Grad-CAM similarities to the ImageNet-pretrained reference are higher than FDSL’s, and Figure G marks each method’s mean similarity.The caption describes per-image cosine similarity between Grad-CAM maps and the reference model’s maps.Source: D. Robustness Analysis · p. 20
  • Under linear interpolation between two FineFractalDB parameter sets, FDSML (same) produces a smoother embedding trajectory than FDSL, whose trajectory changes abruptly near α ≈ 0.6.In the UMAP visualization, FDSML (each) also forms more structured within-class trajectories while maintaining relative separation across classes; FDSML (same) shows more between-class overlap.Source: Figure I · p. 22, FDSL FDSML (each) FDSML (same) · p. 23

Related papers

Loading 2610.39423v1…