PaperlayerView exact revision on arXiv

Source-linked AI summary

Amortized Set Prediction for Inverse IFS Reconstruction from Density Maps

Yutaka Yamaguti

arXiv:2608.24175v1cs.CV

TL;DR

Recovering affine maps from IFS density images is difficult and traditionally requires separate optimization for each image. This paper amortizes reconstruction with a learned set predictor and finds a better quality–speed trade-off with more reliable convergence than random-start optimization.

  • Problem

    Inverse IFS reconstruction is difficult because density maps may not uniquely identify the underlying affine-map set, while existing methods optimize separately for each image.

  • Method

    A learned estimator predicts unordered affine-map sets from density maps, using reconstruction-based evaluation and optional image-only refinement with the known renderer.

  • Results

    Amortized initialization with light refinement achieves a better quality–speed trade-off than random-start per-image optimization, including when the latter receives twice the time, and converges more reliably.

  • Takeaways & Limitations

    Amortized initialization can improve both the efficiency and reliability of inverse IFS reconstruction within the evaluated settings.

  • Takeaways & Limitations

    The framework assumes selection probabilities proportional to |det W| and cannot represent IFSs with independently specified probabilities.

Abstract

from arXiv · show

Iterated Function Systems (IFS) generate self-similar fractals from a few contractive affine maps. The forward map from parameters to images is computationally inexpensive and well understood, whereas the inverse problem of estimating maps from an image is difficult and is typically handled by per-image optimization. We replace this loop with a single forward pass of a learned estimator that predicts the affine-map set directly from a visit-frequency density map, thereby amortizing the inverse problem. The design follows two constraints. First, density maps do not uniquely identify IFS parameters, so evaluation is based on reconstruction rather than parameter recovery; unordered map sets are handled by Hungarian matching, and ground-truth parameters provide a stable training surrogate. Second, the fully known forward model lets us generate exact synthetic training pairs and also supports image-only test-time refinement. On in-distribution tests, amortized initialization plus a few refinement steps lies on a better quality--speed frontier than equal-budget random-initialized per-image optimization, and a 30-step refinement (about $0.56$ s per sample) remains better than a doubled-budget baseline. Extending optimization to 1000 steps shows that the benefit is not only speed: amortized initialization reaches high-quality reconstructions more frequently than random starts. On real images (MNIST and Fashion-MNIST), it improves density metrics on average over a published per-image optimizer while being roughly 12 to 2600 times faster.

1. Introduction

The paper amortizes inverse IFS reconstruction by predicting affine-map sets from density maps in one feed-forward pass, optionally followed by refinement. Because density maps are non-identifying, it evaluates reconstruction with permutation-invariant set comparisons and exploits the known renderer for exact synthetic supervision.

  • Introduction: Inverse IFS reconstruction is difficult despite inexpensive, well-understood forward rendering from contractive affine maps.
  • Introduction: A learned estimator predicts n affine maps from a visit-frequency density map in one feed-forward evaluation, either directly or as a refinement initialization.
  • Introduction: Density maps can arise from different parameter sets, so success is defined by reconstruction rather than recovery of the true parameters.
  • Introduction: Hungarian set loss removes penalties for map permutations, while the known renderer supplies exact self-generated parameter–density pairs and stable ground-truth supervision.
  • Introduction: A few refinement steps from amortized predictions outperform equal-budget random-start optimization and remain better than a random-start optimizer given twice the compute.
  • Introduction: Longer optimization shows the advantage is also reliability: amortized initialization reaches high-quality solutions more reliably than random starts.

2. Related Work

Prior inverse-IFS methods rely on per-image optimization, while earlier neural estimators do not jointly handle unordered map sets, reconstruction-based objectives, and refinement. This work combines Hungarian set prediction, differentiable rendering, and amortized inference to provide learned initialization followed by lightweight per-image optimization.

  • Inverse IFS methods: Classical inverse-IFS methods include moment matching, wavelet–moment methods, evolutionary computation, and partitioned-IFS image compression.The Collage Theorem motivates matching the maps’ image union to the target.
  • Differentiable rendering: Tu et al.’s differentiable renderer enables gradient-based fitting, but clamping makes pixel MSE measure attractor occupancy more than visit-frequency density.The renderer uses chaos-game samples, RBF splatting, and pixel-value clamping.
  • Neural inverse methods: Earlier feed-forward estimators regress canonically ordered affine parameters with parameter MSE, using predictions only to initialize a subsequent search.Their reported parameter proximity can still yield markedly different reconstructed attractors.
  • This work: The proposed approach applies Hungarian optimal-assignment loss to unordered IFS map sets and amortizes inverse inference before light per-image refinement.It combines the set-prediction principle with differentiable reconstruction and the cheap, fully known forward model.

3. Problem Formulation

The paper formulates inverse IFS reconstruction as unordered set prediction from density maps generated by a known stochastic forward model. Because parameters are not uniquely identifiable, success is defined by reconstruction quality, while parameter supervision remains useful during training.

  • 3.1. Iterated function systems and attractors: An affine IFS applies maps x 7→xW +b, whose contractions define a unique attractor independent of map ordering.Each map satisfies ∥W_i∥_2 < 1, making the Hutchinson operator contractive.
  • 3.2. Selection probabilities and the invariant measure: The chaos game samples the invariant measure on the attractor, making a density map a finite-sample approximation of that measure.Under contraction, the invariant probability measure is unique and supported on the attractor.
  • 3.2. Selection probabilities and the invariant measure: Selection probabilities are proportional to | det W_i|, so visit frequencies are determined by geometric map parameters rather than estimated separately.The setting excludes IFS models whose probabilities are independent of determinants.
  • 3.3. Forward problem: density-map generation: The renderer converts parameters into a normalized 128 × 128 histogram over the fixed domain Ω= [−1.5, 1.5]2.Points outside the domain are discarded, and densities are normalized to unit mass for direct comparison.
  • 3.3. Forward problem: density-map generation: The fully known forward model generates arbitrarily many exactly labeled parameter–density pairs for training and test-time refinement.Gradient-based refinement uses a differentiable bilinear-kernel surrogate for the nondifferentiable histogram renderer.
  • 3.4. Inverse problem: formulation as set prediction: The inverse task estimates an unordered map set with one learned forward pass, using Hungarian matching to remove permutation ambiguity.The number of maps is fixed per model: n=4 in main experiments and n=10 for real-image experiments.
  • 3.5. Reconstruction as the success criterion: Because density maps can arise from distinct parameter sets, evaluation measures reconstruction distance between R(ˆθ) and y rather than parameter error.Known ground-truth parameters from self-generated pairs nevertheless provide the parameter-discrepancy training objective.

4. Method

The method predicts an unordered set of affine maps directly from a normalized density map, using supervised assignment-aware training and optional image-only refinement. A known renderer supports both synthetic pair generation and differentiable reconstruction.

  • Estimator: The estimator maps a three-channel 128×128 density input to n affine-map parameters in one forward pass.The input combines the normalized density map with normalized x and y coordinate channels.
  • Training objective: Hungarian matching removes arbitrary map ordering, while a fixed-point term with λfp=0.05 supplements matched parameter distances.The fixed point directly captures map location and corrects coupled errors in W and b.
  • Inference and refinement: Optional refinement starts from the 0-step prediction and performs K ∈{10, 20, 30, 100} AdamW steps using an objective computed from the observed image alone.Refinement trades additional computation for higher reconstruction quality.
  • Rendering: The attractor renderer uses 16 trajectories of 1024 steps, burn-in 128, and smoothing σ=2 for hard training targets.Differentiable soft splatting with matched settings supports auxiliary reconstruction and refinement, while evaluation uses the matched renderer.

5. Identifiability: parameter recovery is ill-posed

Density maps do not uniquely identify the generating IFS parameter set, an intrinsic ambiguity demonstrated analytically and observed in the synthetic test distribution. Evaluation therefore prioritizes reconstruction, with parameter error retained as a secondary diagnostic.

  • 5. Identifiability: parameter recovery is ill-posed: An analytic Sierpinski-triangle construction produces identical density maps from parameter sets that remain far apart after optimal Hungarian assignment.Composing each map with the triangle’s 120° rotational symmetry preserves the invariant measure while changing the linear parts from 1/2I to 1/2 Rot.
  • 5. Identifiability: parameter recovery is ill-posed: 59 near-image, far-parameter pairs satisfy density distance ≤0.0168 and parameter-set distance ≥1.299 in test256.Across all 32,640 test256 pairs, density distance is measured on normalized 128 × 128 maps and parameter sets are matched over all 24 permutations.
  • 5. Identifiability: parameter recovery is ill-posed: At density L2 ≈0.013, the median parameter-set distance remains about 1.2, versus an overall value of ≈1.06.The parameter distance does not decrease as the density tolerance ε shrinks, indicating persistent non-identifiability among nearly identical images.
  • 5. Identifiability: parameter recovery is ill-posed: The ambiguity appears near the reconstruction accuracy achieved by refinement, rather than only in an asymptotic regime.This links the observed non-identifiability to the accuracy range relevant for the paper’s reconstruction procedure.
  • 5. Identifiability: parameter recovery is ill-posed: The study therefore evaluates success by reconstruction and uses (W, b) error only as a secondary indicator of true-value recovery and failure modes.Unique recovery of the true parameter set from a density map is not well posed in the synthetic distribution.

6. Experiments

Experiments show that amortized initialization improves the reconstruction quality–speed tradeoff over random initialization, with advantages persisting under extended refinement and several distribution shifts. Real-image transfer also compares favorably with a published per-image optimizer, while reconstruction quality remains distinct from parameter recovery.

  • Quality–speed comparison: At 0 steps, feed-forward prediction improves over random-r4 at comparable time, with Chamfer 0.0547 versus 0.1546 and coverage@2px 0.665 versus 0.406.At about 0.56 s/sample, base+30 improves every metric over random-r4+30 and wins on 239–253 of 256 cases, depending on metric.
  • Parameter recovery: Refinement substantially improves reconstruction but does not imply parameter recovery, so (W, b) error is treated as a diagnostic rather than a success criterion.For base, Chamfer changes from 0.0547 to 0.0259 while (W, b) error changes from 0.405 to 0.395; in the extended experiment, Chamfer changes from 0.055 to 0.019 while (W, b) error increases from 0.405 to 0.453.
  • Extended refinement: After 1000 steps, amortized initialization still outperforms random initialization in mean Chamfer, tail quality, HD95, coverage@2px, and success rate.Mean Chamfer is random-r4 0.0276 and random-r1 0.0331 versus base 0.0195; success rates at 1000 steps are 0.824 for base, 0.301 for r1, and 0.445 for r4.
  • Out-of-distribution robustness: Under T1 location shift, amortized 30-step refinement exceeds random-r4 at equal time and random-r4-60 at double time, whereas its advantage fades for S1’s novel space-filling attractors.For T1, it wins per sample on Chamfer in 208/256 cases and HD95 in 215/256 cases; for S1, per-instance optimization becomes favorable given enough budget.
  • Real-image transfer: Transfer experiments compare the estimator directly with Tu et al. (2023)’s per-image optimizer on MNIST and Fashion-MNIST, including occupancy-style evaluation for saturated rendered images.MNIST is close to occupancy, Fashion-MNIST contains filled regions, and occupancy MSE is the minimum MSE over 100 sampling sequences.

7. Discussion

The discussion separates reconstruction quality from parameter recovery and emphasizes that objective design, renderer consistency, and distributional alignment determine amortized inference’s effectiveness. It also identifies unresolved non-identifiability and modeling limitations involving probabilities, map counts, and reflections.

  • Reconstruction improvements need not reduce parameter error: refinement improves Chamfer while parameter error stays nearly unchanged or increases, whereas reconstruction supervision improves one-shot output but worsens validation loss.
  • Matched rendering is necessary because mismatched generation and evaluation or optimization settings can make the true parameters fail to minimize the measured objective.
  • Density targets connect inverse IFS reconstruction to invariant-measure matching and motivate future optimal-transport distances or training objectives beyond pixel SSE and point-cloud Chamfer.
  • Amortized initialization outperforms random-from-scratch optimization at equal and double budgets, especially when test data preserves training-distribution structure; with n=10, it is competitive on real images.
  • Non-identifiability is quantified to density L2 ≈ 0.013, but persistence at the reconstruction noise floor of ∼2.7 × 10−6 density SSE remains undetermined.
  • The framework fixes selection probabilities to p ∝ | det W|, the number of maps per model, and orientation-preserving maps, excluding independently specified probabilities, variable map counts, and reflections.

8. Conclusion

The paper frames inverse IFS reconstruction as amortized set prediction from density maps, using stable parameter matching for training but reconstruction for evaluation and refinement. Its advantage over random-start optimization persists across efficiency, convergence reliability, distribution shifts, and real-image transfer, while metric choice affects comparisons with prior work.

  • Conclusion: The method predicts IFS map sets from density maps, then evaluates and refines them through reconstruction because parameters are non-identifiable.The known renderer provides unlimited labeled synthetic training pairs and an image-only refinement objective.
  • Conclusion: Amortized prediction with light refinement offers a better quality–speed trade-off and more reliable convergence than random-start per-image optimization.The advantage remains when random-start optimization receives twice the time, persists under structure-preserving shifts, weakens under stronger geometric shifts, and transfers to real images with more maps.
  • Conclusion: Compared with Tu et al., the proposed method is stronger under density criteria, whereas the per-image optimizer is preferable under saturated occupancy.The comparison is deliberately reported under both metric families.
  • Conclusion: The strategy may extend to inverse problems with inexpensive simulators by combining synthetic-pair training, simulator-based reconstruction refinement, and observable-based evaluation.This avoids relying on non-identifiable latent parameters as the evaluation target.

CRediT authorship contribution statement

Yutaka Yamaguti contributed across the paper’s conceptual, technical, analytical, and writing activities.

  • Yamaguti led conceptualization, methodology, software, and visualization.
  • Yamaguti conducted validation, formal analysis, investigation, and data curation.
  • Yamaguti wrote the original draft and participated in review and editing.

Funding

The research received no specific grant from public, commercial, or not-for-profit funding agencies.

  • The research received no specific grant from public, commercial, or not-for-profit funding agencies.

A. Point-cloud view of the main result (test256)

The section presents four qualitative examples as attractor point clouds, linking them to the density-map examples and defining the plotted points through the matched renderer.

  • A. Point-cloud view of the main result (test256): Four examples from Section 6.3 are revisited as attractor point clouds.They correspond to the qualitative density-map examples shown in Figure 3 and the point-cloud visualizations in Figure A.1.
  • A. Point-cloud view of the main result (test256): Figure A.1 visualizes the same examples previously shown as density maps in Figure 3.The correspondence is explicitly stated with the qualitative examples of Section 6.3.
  • A. Point-cloud view of the main result (test256): The plotted point clouds contain trajectory points from the matched renderer within the drawing domain [−1.5, 1.5]2.Thus, the visualization represents renderer trajectories restricted to the stated drawing domain.

B. Additional examples (MNIST / Fashion-MNIST)

This section adds qualitative MNIST and Fashion-MNIST examples in occupancy and density-map views. The occupancy comparisons show that Tu often shapes outlines clearly, but sometimes drops thin lines or concentrates mass.

  • Additional qualitative examples: Additional examples cover MNIST and Fashion-MNIST in saturated occupancy and common-grayscale density-map views.The occupancy view contains 10 examples per dataset, while the density galleries contain all balanced 50 examples.
  • Occupancy view: Tu often shapes outlines clearly, but some examples show dropped thin lines or concentrated mass.This observation is reported for the occupancy view of the additional qualitative comparison.
  • Occupancy view: The occupancy comparisons use columns for target, ours-0, ours-30, ours-100, occupancy-GD-100, and Tu (n=10).MNIST and Fashion-MNIST are rendered with Tu’s 32×32 saturated occupancy renderer.
Loading 2608.24175v1…