PaperlayerView exact revision on arXiv

Source-linked AI summary

GigaBench: Benchmarking Learnable Representations for Gigapixel Image Compression and Super-Resolution

Matteo Fusconi, Alex Costanzino, Pierluigi Zama Ramirez, Luigi Di Stefano

arXiv:2610.31644v1cs.CV

TL;DR

Gigapixel imagery is increasingly common, but LR methods lack comprehensive evaluation at this scale. GIGABENCH benchmarks three LR families on compression and super-resolution; results show task- and constraint-dependent trade-offs, while the findings are bounded by a ten-scene benchmark.

  • Problem

    Gigapixel images pose fitting, memory, and partitioning challenges, yet comprehensive evaluation of LR methods at this scale is lacking.

    Source: §1 · p. 1

  • Method

    GIGABENCH curates gigapixel images and systematically evaluates Implicit, Hybrid, and Explicit representations for compression and super-resolution.

    Source: §1 · p. 1, §3.1 · p. 3, §3.2 · p. 3

  • Results

    No LR family dominates uniformly: Hybrid methods lead in fidelity and super-resolution, Implicit methods offer higher compression at a fidelity cost, and outcomes depend on task and quality-speed trade-offs.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Choosing an LR depends on the task, target compression or super-resolution ratio, and application constraints.

    Source: §1 · p. 2, §5 · p. 8

    The findings characterize the evaluated methods on ten scenes and should not be interpreted as broad general conclusions.

    Source: §5 · p. 8, A. Limitations and Intended Scope · p. 12

Abstract

from arXiv · show

Learnable Representations (LRs) can fit parametric families of functions to diverse kinds of signals. These emergent neural-native data formats are gathering interest in the scientific community due to their compactness, resolution-agnosticism and amenability to direct neural processing. On the other hand, gigapixel images are becoming increasingly widespread across a variety of scientific, cultural, and industrial domains. In this paper, we present GigaBench, the first benchmark and experimental evaluation focused on deploying LRs with gigapixel imagery. GigaBench includes images collected, acquired, and synthetically rendered across diverse domains, and addresses image compression and super-resolution. We adapt a representative set of LRs, spanning Implicit, Hybrid, and Explicit methods, so as to operate reliably at the gigapixel scale on consumer hardware. Through systematic evaluation across reconstruction quality, model size, compression and inference times, we identify consistent trends across methods and suggest actionable guidance for future investigation.

1. Introduction

Learnable Representations (LRs) encode images as continuous functions, but gigapixel use lacks comprehensive evaluation and brings fitting, memory, and partitioning challenges. GIGABENCH benchmarks three LR families on compression and super-resolution, finding distinct trade-offs rather than one uniformly best approach.

1. Introduction

  • LRs represent images as continuous coordinate-to-color functions with optimizable parameters, rather than as discrete pixel grids.Their cited advantages include compactness, resolution-agnosticism, and amenability to direct neural processing.Source: §1 · p. 1
  • Prior LR evaluations lack comprehensive gigapixel-scale comparisons, despite growing use of gigapixel images across scientific, cultural, and industrial domains.At this scale, fitting may take hours, naive partitioning can introduce boundary artifacts, and memory constraints force architectural compromises.Source: §1 · p. 1
  • Hybrid representations offer the highest reconstruction fidelity and strongest super-resolution performance, while Implicit representations support higher compression at a fidelity cost.Hybrid grid components impose a memory floor at extreme compression rates; Explicit methods vary substantially with parametrization, and no family dominates across tasks and quality-speed trade-offs.Source: §1 · p. 2
  • GIGABENCH systematically compares Implicit, Hybrid, and Explicit representations for gigapixel compression and super-resolution, and characterizes their fidelity, speed, and downstream-usability trade-offs.The benchmark covers 10 scenes across different domains and is released publicly to foster future research.Source: §1 · pp. 1–2

2. Related Work

Prior work develops Implicit, Hybrid, and Explicit representations for signal fitting and downstream learning, but evaluations remain inconsistent and do not systematically address gigapixel imagery. GIGABENCH addresses this evaluation gap.

2. Related Work

  • Implicit representations use MLPs, while Hybrid methods combine an MLP decoder with learnable local structures such as grids, hash tables, or basis functions.The cited Hybrid examples include adaptive blocks, Laplacian pyramids, multi-resolution hash grids, radial basis functions, and metric grids.Source: §2 · p. 2
  • Learnable Representations fit a function from coordinates to target-signal values using optimized parameters, and their weights can support downstream tasks without reconstructing the signal.The cited applications include classification and retrieval.Source: §2 · p. 2
  • Explicit representations place optimizable graphic primitives directly in the signal domain, with Gaussian primitives among the forms investigated for image fitting.The cited work describes GaussianImage as using eight parameters per primitive and a tile-based rasterizer.Source: §2 · pp. 2–3
  • Existing evaluations use inconsistent datasets, resolutions, compression definitions, and metrics, and no prior work systematically evaluates these methods at gigapixel scale.Gigapixel fitting cost, memory, and rate-distortion behavior introduce distinct challenges.Source: §2 · p. 3

3. Benchmark

GIGABENCH curates gigapixel images, selects representative methods and configurations across three LR families, and defines quality, efficiency, and human-evaluation measures for compression and super-resolution.

3.1. Data Collection

  • The benchmark contains 10 images at 1.0–1.19 GPx, spanning diverse domains, excluding images with pre-existing JPEG artifacts, and retaining images licensed for research use and redistribution.The set includes panoramas, industrial scans, histopathology slides, satellite imagery, and synthetic images.Source: §3.1 · p. 3

3.2. Methods Selection

  • Method selection covers seminal and newer Implicit, Hybrid, and Explicit representations in configurations spanning compression ratios of 10^1, 10^2, and 10^3.Gigapixel fitting uses image tiling and gradient accumulation to reduce memory use while maintaining theoretical accuracy; Gaussian splatting is otherwise unfeasible on consumer GPUs at this scale.Source: §3.2 · p. 3

3.3. Metrics

  • Compression is assessed with PSNR, SSIM, compression ratio, compression time, and inference time; super-resolution uses ×4 and ×8 upsampling and includes human rankings of selected methods.The human study asks participants to rank perceived detail sharpness, artifact absence, and overall visual fidelity; compression ratio compares uncompressed-image size with stored-representation size.Source: §3.3 · pp. 4–5

4. Experiments

Compression reveals a fidelity–rate–speed trade-off, while super-resolution favors hybrids quantitatively but human evaluations favor classical methods at ×4 and find hybrids comparable at ×8.

4.1. Compression

  • Hybrids lead reconstruction fidelity (MetricGrids T = 24: 33.59 dB; I-NGP T = 24: 33.19 dB) but remain at CR ≈5–16×, while explicit methods span wider compression ratios.At extreme compression, SmartSplat at CR = 3000 maintains a clear PSNR advantage over implicit methods at similar ratios.Source: §4.1 · p. 5, Figure 2 · p. 6
  • Explicit methods decode fastest but take longer to fit, while hybrids offer stable 10–40-second decoding and I-NGP fits in under 1.7 minutes on average.The larger implicit variants approach 7–11 minutes to decode a gigapixel image.Source: §4.1 · pp. 5–6, Figure 3 · p. 6

4.2. Super-Resolution

  • At ×4, I-NGP (T = 24) leads with 27.11 dB (+1.11 dB over bicubic interpolation), but bicubic outperforms all implicit methods and hybrids alone consistently surpass classical baselines.At ×8, I-NGP’s advantage over classical methods narrows to 0.68 dB.Source: Table 3 · p. 6, §4.2 · p. 6
  • Human ratings favored classical methods at ×4, while hybrids reached essentially equivalent perceptual quality at the top of the ×8 ranking; implicit and explicit methods ranked lower.The study involved 170 participants and excluded Andromeda because its crops did not provide sufficient context for reliable assessment.Source: Figure 4 · p. 7, §4.2 · p. 7

4.3. Qualitative Analysis

  • Qualitative crops show implicit blurring of fine structures, Gaussian ringing, ghosting, or blob-like textures, while hybrids remain close to ground truth across tested images and tasks.Classical interpolation avoids artifacts but slightly smooths fine structural detail at ×8.Source: Figure 5 · p. 7, §4.3 · p. 8

4.2. Super-Resolution

  • Family-level human rankings were broadly consistent across domains, with Venice at ×4 an outlier where classical methods dominated, likely due to smoothing artifacts on fine architectural detail.Implicit and explicit outputs may score competitively on PSNR yet be perceptually penalized for suppressing fine textures.Source: §4.2 · pp. 7–8

5. Discussion

GIGABENCH evaluates learnable representations for gigapixel compression and super-resolution, finding that the best choice depends on task, compression ratio, and application constraints. Its results offer practical trade-offs while remaining limited to the benchmark’s ten scenes.

5. Discussion

  • Hybrid representations are recommended for compression at moderate ratios, while Implicit representations and SmartSplat are viable at extreme compression.SmartSplat is fastest at inference among the benchmarked implementations, making it a latency-sensitive option when fitting cost is not a bottleneck.Source: §5 · p. 8
  • For super-resolution, classical methods retain a human-preference advantage at ×4, while Hybrid representations match them at ×8.PSNR and human-preference rankings diverge, so quantitative metrics alone are insufficient in perceptually demanding regimes.Source: §5 · p. 8
  • Learnable representations can be deployed at gigapixel scale with dedicated adaptations such as tiling and gradient accumulation, but computational cost is a decisive selection criterion.Source: §5 · p. 8
  • The findings characterize ten benchmark scenes and should not be interpreted as broad, general conclusions.Source: §5 · p. 8

Super-Resolution

This material describes the benchmark’s scope and image provenance, then characterizes image diversity using complementary low-level statistics. The evidence covers ten scenes and does not claim exhaustive coverage of gigapixel imagery.

A. Limitations and Intended Scope

  • GIGABENCH’s findings are comparative statements about eight methods in sixteen configurations across ten scenes, not estimates of representation-family behavior over the population of gigapixel images.With only one or two images per domain, a single scene can shift family averages; the benchmark supports no domain-level conclusions.Source: A. Limitations and Intended Scope · p. 12

B. Provenance and Attribution of the GIGABENCH images

C. Quantitative Characterization of Image Diversity

  • Image diversity is measured with spectral, edge, entropy, and texture statistics because semantic domain labels alone are insufficient.The measures include power-spectrum slope and high-frequency energy, gradient and edge density, histogram entropy and JPEG XL rate, and GLCM contrast and homogeneity.Source: C. Quantitative Characterization of… · p. 13
  • β spans 1.73–4.28, EHF spans two orders of magnitude, entropy ranges from 3.78 to 7.75 bits/px, and edge density and GLCM contrast vary by factors of ∼235 and ∼144.The statistics are not redundant: House of Chores combines high entropy with near-minimal edge density, while the spectra also differ in shape.Source: C. Quantitative Characterization of… · pp. 13–14, Table 4 · p. 14, Figure 7 · p. 14
  • The ten images occupy widely separated, partially decorrelated positions in low-level statistics space, but do not establish exhaustive coverage of gigapixel imagery.Source: C. Quantitative Characterization of… · p. 14

D. Structural Similarity Analysis

The SSIM analysis largely corroborates the PSNR findings but changes some family rankings, especially in compression. In super-resolution, neither metric reproduces the human-evaluation ranking.

D. Structural Similarity Analysis

  • Compression SSIM changes the category order from Hybrid–Explicit–Implicit under PSNR to Explicit–Hybrid–Implicit, while preserving nearly all within-family orderings.The high-frequency images that are difficult under PSNR remain the hardest under SSIM across categories.Source: D. Structural Similarity Analysis · pp. 14–15, Table 5 · p. 14
  • Under SSIM, SmartSplat (CR= 15) scores best on 8 of 10 images and has the highest dataset average, 0.886, swapping the compression category order to Explicit–Hybrid–Implicit.The authors attribute this swap to SmartSplat’s training objective, which includes an SSIM term.Source: Table 5 · p. 14, D. Structural Similarity Analysis · p. 15
  • SSIM preserves the PSNR category order for super-resolution at ×4, but Explicit and Hybrid swap at ×8; both metrics differ from the human ranking, Classic–Hybrid–Implicit–Explicit.The Implicit family ranks last with large gaps, while category margins are otherwise narrow.Source: D. Structural Similarity Analysis · p. 15, Table 6 · p. 15

E. Quantitative Perceptual Similarity on Super Resolution

The paper evaluates LPIPS for super-resolution using tiled inference and finds that its rankings only partly align with PSNR and human judgments. The results underscore that quantitative scores should be interpreted alongside human evaluation.

E. Quantitative Perceptual Similarity on Super Resolution

  • LPIPS is evaluated on non-overlapping 8192 × 8192 tiles because full-image evaluation is infeasible at gigapixel scale.The authors argue that tile-boundary discrepancies are negligible because LPIPS is local, with receptive fields spanning only a few hundred pixels.Source: E. Quantitative Perceptual Similarity… · p. 15, Table 7 · p. 16
  • LPIPS preserves the PSNR family order, Hybrid–Classic–Explicit–Implicit, at both scales, but does not reproduce the human study’s Classic-first ranking.Source: E. Quantitative Perceptual Similarity… · p. 15, Table 7 · p. 16
  • At ×4, MetricGrids scores 0.312 versus I-NGP’s 0.368 on LPIPS, while SmartSplat (CR=100) surpasses its CR=3000 counterpart.At ×8, Nearest Neighbour overtakes Bicubic; LPIPS also reports a 0.669 vs. 0.232 gap where PSNR reports only 1.5 dB, illustrating content-specific effects.Source: E. Quantitative Perceptual Similarity… · p. 16
  • The authors conclude that quantitative metrics, including perceptual ones, are informative but insufficient at gigapixel scale, and close method rankings warrant joint interpretation with human judgment.Source: E. Quantitative Perceptual Similarity… · p. 16

F. Additional motivation for the exclusion of I-NGP in the Human Evaluation

I-NGP’s ×4 and ×8 super-resolution outputs are near-constant within each low-resolution cell, so their high PSNR does not reflect synthesized detail. MetricGrids instead varies nonlinearly within cells and produces more plausible structure, though with mild wavy artifacts.

F. Additional motivation for the exclusion of I-NGP in the Human Evaluation

  • Figure 10 compares ×4 Venice-crop outputs from I-NGP (T=24), MetricGrids (T=24), and nearest-neighbor upsampling.The comparison illustrates the qualitative motivation for choosing MetricGrids over I-NGP in the human evaluation.Source: Figure 10 · p. 16, F. Additional motivation for the… · pp. 16–17
  • I-NGP’s bilinear hash features and pointwise ReLU MLP yield nearly flat in-cell plateaus, so it cannot synthesize detail at new high-resolution coordinates.It fits low-resolution samples faithfully but behaves like nearest-neighbor upsampling between them.Source: F. Additional motivation for the… · p. 17
  • MetricGrids adds nonlinear sub-pixel features and a high-order decoder, enabling plausible fine structure but sometimes producing wavy artifacts.Unlike I-NGP’s near-linear interpolation, it varies the signal within each low-resolution cell.Source: F. Additional motivation for the… · p. 17
  • I-NGP was excluded from the human study because high-resolution PSNR rewards matching fitted samples but does not penalize missing synthesized detail.MetricGrids (T=24) was selected as the Hybrid representative for both ×4 and ×8 settings.Source: F. Additional motivation for the… · p. 17

G. Additional Aggregate Results

Across the ten benchmark images, dispersion statistics largely preserve the reported family rankings, but the mean and median can differ because image-score distributions are skewed. In particular, difficult high-frequency scenes affect Implicit results, and some adjacent rankings shift.

G. Additional Aggregate Results

  • The aggregate statistics are means over N = 10 images, so a mean alone does not show score concentration or how much rankings reflect a few scenes.The paper also reports standard deviation, median, IQR, minima, and maxima for compression and super-resolution metrics.Source: G. Additional Aggregate Results · p. 17
  • Median family rankings match mean rankings on both tasks: Hybrid–Explicit–Implicit for PSNR and Explicit–Hybrid–Implicit for SSIM.Bicubic also remains above every Implicit configuration at both super-resolution factors on the median.Source: G. Additional Aggregate Results · p. 17
  • MetricGrids (T=24) and I-NGP (T=24) swap first place in compression, while T=22 and T=24 variants swap at ×4; both pairs are treated as indistinguishable at N = 10.SmartSplat (CR=15) moves above every T=22 Hybrid configuration on the median, consistent with strong Explicit SSIM results.Source: G. Additional Aggregate Results · p. 17
  • Hybrid medians sit 1.4–1.9 dB below their means, while the three (8, 1024) Implicit architectures’ medians sit 0.6–1.6 dB above their means.The paper attributes this skew to image content rather than model capacity.Source: G. Additional Aggregate Results · p. 17
  • The mean-to-median gap between the best methods of each family shrinks from 10.9 to 8.1 dB because Implicit results have a lower tail on high-frequency scenes.Every Implicit variant bottoms out at about 13 dB (0.05 SSIM) in compression and at both super-resolution factors, while Hybrid methods lose quality on those scenes without collapsing.Source: G. Additional Aggregate Results · p. 17

H. Additional Human Evaluation scores

The additional human-evaluation results report win rates and bootstrapped Elo scores as complementary views of participant judgments. They agree at the top of the ranking but diverge at the bottom.

H. Additional Human Evaluation scores

  • Win rate counts how often a family was chosen best; Elo also accounts for opponent strength, and the two metrics diverge at the bottom of the ranking.Table 10 lists both measures, while Figure 11 visualizes per-image win rates.Source: H. Additional Human Evaluation scores · p. 17
  • Classical and Hybrid methods receive the large majority of wins and the highest Elo scores across images, agreeing at the top of the ranking.The win-rate and Elo patterns are consistent with the family-level conclusions reported in Table 3.Source: H. Additional Human Evaluation scores · pp. 17–18

I. Additional Implementation Details

  • The experiments used one 32 GB NVIDIA RTX 5090 and 128 GB RAM, requiring approximately 418 GPUh in total.The stated split is 182 GPUh for compression and 236 GPUh for super-resolution at both scales.Source: §I · p. 18

I.1. Training Details

The supplementary details describe family-specific training protocols, shared runtime measurements, and additional qualitative reconstructions. These protocols include tiled Gaussian Splatting training for gigapixel images.

I.1. Training Details

  • Implicit compression training uses eight image passes, about 30,000–35,000 steps, while super-resolution uses 35,000 steps; Explicit methods use 10,000 full-image optimizer steps.Implicit models use shuffled pixel batches; Explicit tiled training accumulates gradients and updates once per full-image pass.Source: §I.1 · pp. 18–19

I.2. Timing Procedure

  • Runtime measurements use a shared machine and protocol; compression time covers fitting, while inference time covers full-resolution decoding and output assembly.The reported end-to-end times reflect both the algorithm and reference-code engineering, including I-NGP’s fused CUDA kernels versus mostly PyTorch implementations.Source: §I.2 · pp. 19–20

J. Additional qualitative results

  • On GIGABENCH, Figures 12–13 show additional image-compression qualitative results, while Figures 14–17 show additional image-super-resolution results; for each model, the reported version is its most performant by PSNR.Figures 14 and 15 show ×4 super-resolution results as 1228 × 2048 crops from full-resolution images.Source: J. Additional qualitative results · p. 20, Figure 14 · p. 23, Figure 15 · p. 24

SIREN FDHO FINER

  • Figure 18 shows 2048 × 2048 Venice reconstruction crops for all considered method variants.The passage identifies the variants as SIREN, FDHO, and FINER.Source: Figure 18 · p. 27

Related papers

Loading 2610.31644v1…