PaperlayerView exact revision on arXiv

Source-linked AI summary

GraLoD: Graphics-Inspired Continuous Level-of-Detail Learning for Image Restoration

Hu Gao, Lizhuang Ma, Yulong Chen

arXiv:2609.16578v1cs.CV

TL;DR

Image restoration needs different spatial support across degradations, image regions, and reconstruction stages, but existing architectures largely fix representation scales through discrete hierarchies. GraLoD reuses and aligns native multiscale features into a continuous LOD space, predicts stage-conditioned scale fields, and queries neighboring levels with MSFC and SAR constraints. Experiments report consistent gains in task-specific and all-in-one restoration.

  • Problem

    Existing restoration models use predefined discrete representation scales even though required spatial context varies across regions, degradations, and reconstruction stages.

  • Method

    GraLoD aligns native encoder features into a shared LOD space, predicts stage-conditioned LOD fields, and continuously queries neighboring levels while preserving the backbone.

  • Results

    GraLoD consistently improves task-specific backbones and unified restorers across five degradation types, including gains of up to 0.69 dB in task-specific restoration.

  • Takeaways & Limitations

    Continuous representation-scale adaptation extends existing restoration backbones toward competitive all-in-one restoration without redesigning their fundamental feature-processing blocks.

Abstract

from arXiv · show

The spatial support required for image restoration varies across degradation types, image regions, and reconstruction stages. However, most existing methods rely on predefined multi-scale hierarchies and aggregate features through fixed fusion or attention, leaving the representation scale itself largely determined by the network architecture. This limitation becomes more pronounced when a task-specific backbone is extended to heterogeneous degradations in all-in-one restoration. Inspired by level-of-detail (LOD) rendering in computer graphics, we propose GraLoD, a plug-and-play framework that treats restoration scale as a spatially varying and stage-dependent continuous variable. GraLoD reuses the native encoder hierarchy, aligns its multi-scale features into a shared LOD representation space, and predicts a stage-conditioned LOD field at each decoder stage. Each spatial location then continuously queries only two neighboring representation levels, enabling the effective restoration scale to adapt to both local image content and reconstruction progress. To prevent degenerate or arbitrary scale selection, we further introduce minimal-sufficient footprint calibration (MSFC) together with structure-aware regularization (SAR) to encourage restoration-effective and spatially coherent LOD assignments. GraLoD can be directly integrated into existing restoration backbones without redesigning their fundamental feature-processing blocks. Extensive experiments demonstrate consistent improvements in task-specific and all-in-one restoration.

1 INTRODUCTION

Existing restoration architectures use fixed discrete representation scales even though required spatial context varies across regions, degradations, and reconstruction stages. GraLoD introduces continuous, adaptive scale selection while preserving the backbone, improving task-specific and all-in-one restoration.

  • 1 INTRODUCTION: Fixed feature hierarchies let models combine predefined scales but do not determine which representation scale each location requires, especially for heterogeneous all-in-one degradations.Different degradations, textures, boundaries, and thin structures require different spatial support.
  • 1 INTRODUCTION: GraLoD reuses native encoder features, aligns them into a shared LOD space, and predicts stage-conditioned LOD fields that continuously query neighboring levels without redesigning the backbone.The queried representation is injected through a lightweight residual adaptor while the original encoder, decoder, and skip connections remain unchanged.
  • 1 INTRODUCTION: MSFC and SAR constrain LOD learning toward minimal restoration-effective footprints and spatially coherent assignments rather than arbitrary gating or collapse to one level.SAR encourages coherence in homogeneous regions while allowing transitions near structural boundaries.
  • 1 INTRODUCTION: Up to 0.69 dB improvement is reported for representative task-specific backbones, while unified restorers improve across five degradation types with competitive or superior all-in-one performance.Figure 1 summarizes gains in task-specific and all-in-one restoration settings.
  • 1 INTRODUCTION: GraLoD treats restoration scale as a spatially varying and stage-dependent continuous representation variable instead of a predefined architectural choice.It is inspired by LOD rendering and adapts scale across image regions and reconstruction stages.

2 RELATED WORK

Prior restoration and all-in-one methods adapt features, degradation representations, or routing within predefined multiscale hierarchies. GraLoD instead makes representation scale itself continuously queryable while remaining compatible with existing backbones.

  • 2 RELATED WORK: Existing all-in-one methods mainly adapt degradation representations, prompts, routing, or feature modulation while retaining discrete backbone resolutions.This leaves representation scale largely fixed despite differing degradation spatial extents.
  • 2 RELATED WORK: Conventional multiscale restoration combines fine spatial details with broader context through hierarchical encoder–decoder features and multiscale attention or weighting.These approaches adjust contributions from available levels rather than replacing the underlying hierarchy.
  • 2 RELATED WORK: Graphics and neural-rendering research motivates LOD representations that support resolution selection or smooth scale changes according to spatial support.GraLoD transfers this principle from rendering to local image reconstruction.
  • 2 RELATED WORK: GraLoD predicts a continuous LOD coordinate, queries neighboring ordered levels, and remains compatible with existing restoration backbones.The architecture aligns native encoder features into a shared LOD representation space and shares them across decoder stages.

3 METHOD

GraLoD converts native hierarchical features into a shared continuous LOD space and predicts stage-conditioned spatial scale fields for adaptive decoder queries. MSFC and structure-aware regularization constrain these assignments toward restoration-effective and spatially coherent scales while preserving the original backbone.

  • 3.2 GRALOD: PLUG-AND-PLAY CONTINUOUS LOD ADAPTATION: The framework preserves the original encoder, decoder, and skip connections by injecting queried representations through lightweight stage-specific residual adaptors.The aligned LOD hierarchy is shared across stages, while the LOD estimators and adaptors remain stage-specific.
  • 3.2 GRALOD: PLUG-AND-PLAY CONTINUOUS LOD ADAPTATION: GraLoD aligns native encoder features into a shared LOD representation space and reuses that hierarchy across decoder stages.Lightweight refinement and scale alignment convert discrete backbone features into ordered LOD levels, with anti-aliased resampling when needed.
  • 3.2 GRALOD: PLUG-AND-PLAY CONTINUOUS LOD ADAPTATION: Each decoder stage predicts a spatial LOD field conditioned on the current decoder feature, allowing scale selection to depend on image content and reconstruction progress.The predicted coordinate determines the representation scale required at each spatial location and stage.
  • 3.2 GRALOD: PLUG-AND-PLAY CONTINUOUS LOD ADAPTATION: GraLoD continuously interpolates between two neighboring ordered LOD levels instead of independently weighting every feature scale.This preserves scale ordering while allowing queried representations to vary continuously across locations and decoder stages.
  • 3.3 LOD CALIBRATION AND STRUCTURAL REGULARIZATION: MSFC calibrates LOD coordinates toward the smallest representation footprint that provides sufficient local restoration benefit.It compares local restoration utility and penalizes unnecessarily coarse representations; training-only probes are removed at inference.
  • 3.3 LOD CALIBRATION AND STRUCTURAL REGULARIZATION: Structure-aware regularization encourages similar LOD coordinates within homogeneous regions while permitting changes across structural boundaries.The overall objective combines the backbone reconstruction loss with training-only probe supervision and the regularization terms.

4 EXPERIMENTS

GraLoD consistently improves task-specific and all-in-one restoration while preserving existing backbones, and its components support adaptive, calibrated scale selection.

  • PERFORMANCE IMPROVEMENT ON TASK-SPECIFIC RESTORATION: 0.69 dB improves FSNet deraining, while StarIR, ECFNet, and ALGNet gain 0.22 dB, 0.26 dB, and 0.47 dB across other task-specific restorations.Structural similarity is maintained or slightly improved, and the original backbone architecture is preserved.
  • CAPABILITY EXPANSION TOWARD ALL-IN-ONE RESTORATION: 1.10 dB average improvement enables ALGNet to address all five degradation tasks without degradation-specific branches or a redesigned all-in-one architecture.Gains of approximately 1.3–2.0 dB occur for deraining, desnowing, and dehazing.
  • CAPABILITY EXPANSION TOWARD ALL-IN-ONE RESTORATION: 0.21 dB further improves Perceive-IR on average, producing the best overall performance among compared methods with consistent gains across five degradation types.GraLoD-enhanced Perceive-IR remains competitive with or superior to dedicated all-in-one methods qualitatively.
  • ZERO-SHOT GENERALIZATION TO REAL-WORLD DEGRADATIONS: 0.30 dB improves Perceive-IRG PSNR on RealRain-1k-L while also improving SSIM and LPIPS under real-world rain.The evaluation targets real-world rain, haze, and noise datasets not used during training.
  • EFFECT OF EACH COMPONENT: 0.41 dB from LOD alone and cumulative 1.10 dB after LFA, MSFC, and SAR show progressively stronger component contributions.LFA, MSFC, and SAR add 0.27 dB, 0.21 dB, and 0.21 dB, respectively; MSFC targets the smallest sufficient footprint and SAR promotes spatially meaningful assignments.
  • MULTI-SCALE ADAPTATION STRATEGIES: Fixed finest, coarsest, or uniformly averaged representations provide limited gains, whereas continuous multi-scale adaptation is presented as the more effective strategy.The comparison motivates selecting representation scale rather than merely increasing access to fixed-resolution features.

5 CONCLUSION

The conclusion presents GraLoD as a plug-and-play framework for spatially varying, stage-dependent continuous restoration scale adaptation.

  • 5 CONCLUSION: GraLoD reuses and aligns native encoder features, continuously queries decoder-stage representation scales, and uses MSFC and SAR to keep assignments effective and spatially meaningful.Experiments report consistent improvements for task-specific models and competitive all-in-one restoration from conventional backbones.

AI USE STATEMENT

The authors used generative AI tools for manuscript language editing and report that the scientific content and figure data were not changed.

  • AI USE STATEMENT: GPT-based tools assisted with language, grammar, and readability editing, while authors manually reviewed the outputs and retained responsibility for the final manuscript.The statement says the underlying scientific content, figures, experimental results, claims, and conclusions were not altered by AI.

A.1.1 PROPERTIES OF CONTINUOUS LOD QUERYING

GraLoD predicts a continuous LOD coordinate at each decoder stage and spatial location, then interpolates between only two neighboring ordered representation levels.

  • Continuous interpolation along the LOD axis: A coordinate λ_s(p) in [0, L−1] selects two neighboring levels on the ordered LOD axis for interpolation.This local query differs from scale attention that can assign independent weights to all feature levels.
  • Continuous interpolation along the LOD axis: The queried representation is a convex combination because interpolation coefficients are non-negative and sum to one.At most two adjacent weights are non-zero.
  • Continuity: The continuous query remains continuous across integer LOD boundaries, avoiding abrupt feature switching from hard discrete-scale selection.The reconstruction objective can propagate gradients through the continuous query to the LOD estimator, with a piecewise derivative.
  • Stage-conditioned LOD prediction: Each decoder stage predicts its own LOD field from the current reconstruction state rather than sharing one global scale assignment.The formulation does not impose a monotonic coarse-to-fine constraint.

A.1.2 INTEGRATION INTO EXISTING RESTORATION BACKBONES

GraLoD integrates continuous, stage-conditioned LOD querying into existing restoration backbones while preserving their native encoder–decoder processing and skip topology.

  • Integration design: GraLoD reuses native hierarchical features, aligns them into a shared representation space, and leaves the original feature-processing blocks and skip connections unchanged.The shared aligned hierarchy is constructed once and reused across decoder stages.
  • Stage-wise querying: Each decoder stage maps aligned features to its spatial lattice, predicts a stage-specific LOD field, and queries neighboring levels for the current reconstruction state.Only two neighboring representation levels contribute to the interpolated query.
  • Residual integration: The queried feature is injected through a residual adaptor, augmenting the native decoder representation without replacing backbone-specific blocks or skip connections.The enhanced feature is subsequently processed by the original decoder block with its native skip feature.

A.1.3 FURTHER ANALYSIS OF MINIMAL-SUFFICIENT FOOTPRINT CALIBRATION

MSFC calibrates LOD selection using local restoration utility and a cost for coarse representations, producing auxiliary supervision that favors minimal sufficient spatial support.

  • Minimal-sufficient criterion: MSFC compares local reconstruction utility across aligned LOD levels while penalizing unnecessarily coarse representations.A coarser level is preferred only when its reconstruction benefit compensates for the additional scale cost.
  • Scale probes: Scale probes estimate local reconstruction error using a shared lightweight probe and neighborhood averaging, making scale requirements depend on spatial regions rather than isolated pixels.The probes are auxiliary modules used only to compare the relative restoration utility of representation scales.
  • Soft target construction: MSFC converts scale costs into a soft target distribution, with temperature controlling concentration and lower temperature approaching minimum-cost discrete selection.The stop-gradient operation prevents the predicted LOD field from changing its own calibration target.
  • Auxiliary training: Probe inputs are detached during auxiliary optimization, and the probes are discarded after training so they do not alter backbone representations or inference.This keeps probe learning auxiliary to the restoration objective.

A.1.4 ANALYSIS OF STRUCTURE-AWARE LOD REGULARIZATION

Structure-aware regularization encourages coherent LOD fields within image structures while preserving transitions at boundaries, and GraLoD retains its reported gains across varied restoration settings.

  • Structure-aware regularization: Structure-aware weighting penalizes LOD differences more strongly between similar neighboring locations while allowing changes across image boundaries.The regularizer acts as an edge-aware total-variation constraint rather than enforcing global smoothness.
  • Training strategy: Training jointly optimizes reconstruction, calibration, and structure losses, while clean targets and target-dependent regularization are removed at inference.The same procedure serves task-specific and multi-degradation settings without degradation labels or task-dependent GraLoD branches.
  • Unseen degradation types: Zero-shot underwater evaluation improves Perceive-IR on UIEB by 0.21 dB PSNR and yields the best overall performance across UIEB and C60.These gains use no underwater-specific supervision, task identifiers, or additional adaptation modules.
  • Unseen degradation severities: At unseen noise levels, GraLoD improves CBSD68 over the baseline by 0.12 dB at noise level 60 and 0.14 dB at noise level 100.Similar gains occur on Urban100, with the best performance at the more challenging noise level 100.
  • Feature-space analysis: Feature analysis shows GraLoD makes ALGNet degradation clusters more compact and separable, while Perceive-IR retains its existing cluster structure with only slight local reorganization.This contrasts task-specific feature reorganization with the behavior of an already unified restoration model.
Loading 2609.16578v1…