PaperlayerView exact revision on arXiv

Source-linked AI summary

Shared-Factor-Conditioned Diffusion for Low-Rank Tensor Recovery

Zhengyun Cheng, Guanwen Zhang, Wei Zhou

arXiv:2610.35368v1cs.CVcs.AI

TL;DR

Tensor recovery must handle heterogeneous tensor structures and observation processes, while existing formulations can be limited to particular output units or coordinate domains. DiffTR uses shared-factor-conditioned diffusion with task-specific observation models for completion, denoising, and HSI super-resolution. It achieves the best PSNR, SSIM, and NRMSE across 30 tested modality, sampling-rate, and observation-pattern combinations.

  • Problem

    Tensor recovery spans heterogeneous structures and observations, while existing formulations often target specific output units or coordinate domains, complicating joint support for entries, tubes, and side information.

    Source: §I · p. 1

  • Method

    DiffTR jointly learns coordinate-indexed shared factors and a conditional score network, using scalar or vector-valued atoms and task-specific observation models for recovery.

    Source: §I · pp. 1–2, B. Shared-Factor-Conditioned Diffusion · p. 3

  • Results

    DiffTR achieves the best PSNR, SSIM, and NRMSE across all 30 tested modality, sampling-rate, and observation-pattern combinations; at 5% entry-wise sampling, its PSNR exceeds the strongest baseline by 1.28 dB on RGB images.

    Source: 1) Multidimensional Visual Data… · p. 7

  • Takeaways & Limitations

    Across tabular tensors, images, and videos, the shared representation supports completion, denoising, and HSI super-resolution with competitive recovery quality.

    Source: §VI · p. 14

Abstract

from arXiv · show

Tensor recovery spans heterogeneous data structures and observation processes, calling for a representation that can adapt across modalities and tasks. We propose DiffTR, an unsupervised framework that unifies tensor completion, denoising, and hyperspectral super-resolution through coordinate-conditioned diffusion. Shared factors indexed by tensor coordinates condition a compact score network, and score matching jointly learns the factors and an implicit tensor representation from the available observations. Scalar and vector-valued outputs accommodate individual entries and entire tubes, enabling a common formulation for tabular tensors, images, and videos. The framework separates representation learning from task-specific observation modeling: reverse diffusion recovers missing values; estimation via Tweedie's formula, soft thresholding, and optional spatial consistency support mixed-noise denoising; and measurement-guided sampling transfers a prior learned from low-resolution hyperspectral observations to high-resolution reconstruction, followed by post-sampling refinement with data consistency and spectral subspace regularization. This separation makes the representation compatible with different corruption models and permits task-specific constraints to be incorporated within the same recovery framework. Experiments across multiple tensor modalities and recovery tasks demonstrate competitive reconstruction quality and the flexibility of a unified conditional generative representation. The code is available at https://github.com/CZY-Code/DiffTR

I. INTRODUCTION

DiffTR addresses the need for tensor recovery that can accommodate varied structures and observations with a shared conditional-diffusion framework. It supports scalar entries and vector-valued tubes, with task-specific procedures for completion, denoising, and hyperspectral image super-resolution.

I. INTRODUCTION

  • DiffTR conditions a score network on shared mode-wise factors to learn interactions for scalar entries or vector-valued tubes, while task-specific observation models support completion, denoising, and HSI super-resolution.The framework learns from each tensor’s own observations and uses an implicit representation rather than a prescribed multilinear contraction.Source: §I · pp. 1–2

A. Tensor Completion

  • DiffTR supports scalar entries and vector-valued tubes in one conditional-diffusion formulation, with factors retrieved for discrete indices or generated by implicit neural representations for continuous coordinates.Its score network learns factor interactions without a prescribed multilinear contraction.Source: A. Tensor Completion · p. 2

B. Tensor Denoising

  • For mixed-noise denoising, DiffTR alternates score learning, clean-value estimation via Tweedie’s formula, and sparse-noise separation, with optional spatial consistency for local smoothness.The method addresses dense noise and sparse outliers because fitting corrupted values directly can embed noise in the learned representation.Source: B. Tensor Denoising · p. 2

C. Hyperspectral Image Super-Resolution

  • For HSI super-resolution, DiffTR learns a conditional score from upsampled LR-HSI, guides reverse diffusion with HR-MSI measurements, and refines samples using both observation operators and spectral subspace regularization.The observation constraints account for spatial blur and downsampling that couple neighboring locations.Source: C. Hyperspectral Image Super-Resolution · p. 2

III. METHODOLOGY

DiffTR uses coordinate-indexed shared factors to condition diffusion over scalar entries or vector-valued tubes. Its preliminary diffusion formulation perturbs data with Gaussian noise and recovers samples through a learned reverse process.

III. METHODOLOGY

  • The DiffTR overview uses coordinate-indexed shared factors to condition diffusion for completion, denoising, and HSI super-resolution, with task-specific learning and recovery procedures.Figure 1 uses green for completion, orange for denoising constraints, and purple for super-resolution guidance; matched circles mark the same spatial location.Source: §III · p. 2, Figure 1 · p. 3
  • An atom is indexed by its coordinate modes: scalar entries use all tensor modes, while tubes use one fewer coordinate mode and retain the remaining mode as a vector.The observed atom coordinates form Ω, and their complement identifies unobserved coordinates.Source: §III · p. 2

A. Preliminaries

  • The preliminary DDPM process gradually perturbs clean samples with Gaussian noise, then parameterizes a reverse process whose score network is trained by denoising score matching.Starting from a standard Gaussian, ancestral sampling applies the learned reverse process.Source: A. Preliminaries · p. 3

B. Shared-Factor-Conditioned Diffusion

DiffTR learns a conditional score over tensor atoms using shared factors indexed by coordinate modes. The same approach accommodates discrete or continuous coordinates and supports reverse-diffusion recovery at target coordinates.

B. Shared-Factor-Conditioned Diffusion

  • For each coordinate mode, DiffTR associates an R-dimensional shared factor with each index or coordinate, defining representation rank R as the factor’s feature dimension.Atoms sharing an index reuse that mode’s factor, coupling their representations across the tensor.Source: B. Shared-Factor-Conditioned Diffusion · p. 3
  • DiffTR models the conditional distribution of each scalar or tube-valued atom rather than defining tensor values through a deterministic factor contraction.For visual tensors it uses continuous coordinate features; sparse tabular tensors optimize factor rows directly.Source: B. Shared-Factor-Conditioned Diffusion · p. 3
  • Completion samples observed atoms for training and reverse-diffuses at missing coordinates, combining generated atoms with observations; scalar atoms serve entry-wise completion and vector atoms serve tube-wise recovery.The recovered atoms are folded into the tensor, retaining observations where required by the task.Source: B. Shared-Factor-Conditioned Diffusion · pp. 3–4
  • The score network separately embeds the noisy atom, shared-factor condition, and diffusion index, concatenates those features, and predicts an output with the atom dimension.The jointly optimized score network and shared factors use a score-matching loss.Source: B. Shared-Factor-Conditioned Diffusion · p. 4

C. Sparse-Noise Separation and Spatial Consistency

DiffTR’s denoising procedure separates corruption from training targets while learning an implicit tensor representation. It combines clean-value estimation and sparse-noise updates with optional coordinate-based spatial consistency.

C. Sparse-Noise Separation and Spatial Consistency

  • DiffTR models noisy observations as clean signal plus sparse or structured outliers and dense residual noise, alternating clean-atom estimation, sparse-noise separation, and score-model updates.A Gaussian data term models dense residual noise, while a Laplace prior promotes sparse corruption.Source: C. Sparse-Noise Separation and Spatial… · p. 4
  • Tweedie’s formula uses the learned score to estimate clean values, and soft thresholding updates the sparse-noise component.The procedure recovers the tensor by reverse diffusion and computes the total residual as the difference between the observation and recovered tensor.Source: C. Sparse-Noise Separation and Spatial… · p. 4
  • Optional spatial consistency penalizes changes in conditional score after perturbing an atom’s coordinate, adding local smoothness for tensors with spatial correlations.The denoising objective adds λD times this loss; λD can be set to zero when local continuity is not required.Source: C. Sparse-Noise Separation and Spatial… · p. 4
  • The same coordinate-conditioned network can fit a deterministic clean estimate using a Gaussian–outlier mixture, with optional proximal refinement balancing noise-scaled likelihood and spectral-subspace correction.A two-stage variant fits a deterministic estimate first, then trains diffusion on that fixed target and applies proximal guidance during DDIM sampling.Source: C. Sparse-Noise Separation and Spatial… · p. 4

D. Likelihood-Guided Sampling for HSI Super-Resolution

DiffTR learns an HSI score from spatially upsampled LR-HSI observations, then uses HR-MSI measurements to guide reverse diffusion. A post-sampling refinement step enforces observation consistency and spectral-subspace regularization.

D. Likelihood-Guided Sampling for HSI Super-Resolution

  • At each spatial coordinate, the spectral signature is treated as a tube-valued atom for reconstruction.The LR-HSI preserves spectral resolution, whereas the HR-MSI preserves spatial resolution.Source: D. Likelihood-Guided Sampling for HSI… · pp. 4–5
  • The LR-HSI supplies the learned prior, while HR-MSI spectral residuals guide reverse diffusion to produce an initial HR-HSI estimate.The likelihood correction evaluates the measurement term at a denoised estimate.Source: D. Likelihood-Guided Sampling for HSI… · p. 5
  • Post-sampling refinement fits both full observation operators and penalizes spectral components outside the LR-HSI-derived subspace.The quadratic problem is solved by conjugate gradients, then the estimate is clipped to [0, 1].Source: D. Likelihood-Guided Sampling for HSI… · p. 5

IV. EXPERIMENTS

Experiments evaluate DiffTR on sparse and continuous-time tensor completion, alongside other tensor recovery tasks. On both completion settings, it achieves the lowest mean RMSE across the reported comparisons and leads most MAE comparisons.

1) Sparse Tensor Completion:

  • The sparse-completion evaluation uses five-fold cross-validation on Alog and ACC, with factor dimensions R ∈ {3, 5, 8, 10} for applicable methods.POND is omitted from ACC because of its reported computational and memory requirements in this setting.Source: 1) Sparse Tensor Completion · p. 5
  • At R = 10, DiffTR reduces RMSE relative to ScoreTR by 3.2% on Alog and 8.8% on ACC.It has the lowest mean RMSE in all eight dataset–rank comparisons and the lowest mean MAE in seven; ScoreTR leads MAE on Alog at R = 3.Source: 1) Sparse Tensor Completion · p. 5, Table I · p. 6

2) Continuous-Time Tensor Completion:

  • The continuous-time evaluation compares DiffTR with trajectory, Gaussian-process, recurrent, ODE, and generative-model baselines on Air and Click.The datasets contain timestamped observations, and the study reports five-fold cross-validation results.Source: 2) Continuous-Time Tensor Completion · pp. 5–6
  • On Air at R = 5, DiffTR reduces RMSE and MAE relative to ScoreTR by 10.8% and 15.9%, respectively.It achieves the lowest mean RMSE and MAE on both Air and Click at every tested factor dimension.Source: 2) Continuous-Time Tensor Completion · p. 6, Table II · p. 6

B. Image Recovery

DiffTR is evaluated on visual inpainting, mixed-noise denoising, and hyperspectral super-resolution. Results show strong recovery across modalities, with performance varying by task and setting.

1) Multidimensional Visual Data Inpainting:

  • DiffTR achieves the best PSNR, SSIM, and NRMSE across all 30 visual inpainting combinations, exceeding the strongest baseline by 1.28 dB on RGB, 0.53 dB on MSIs, and 0.50 dB on videos at 5% entry-wise sampling.For tube-wise MSI recovery, it exceeds FA-INR by 2.63–2.80 dB across tested sampling rates; Figure 2 illustrates preservation of boundaries and local detail.Source: 1) Multidimensional Visual Data… · p. 7, Table III · p. 7, Table IV · p. 7

2) Image Denoising:

  • In Case 4, DiffTR has the highest PSNR and lowest NRMSE across WDC, PaviaU, and CAVE, with PSNR gains of 0.72, 0.85, and 0.94 dB over the strongest baseline, respectively.It also leads both metrics on CAVE in Cases 3 and 5, while ScoreTR performs better on the HSI datasets in Cases 1–3; Figure 3 illustrates contour and local-detail preservation.Source: 2) Image Denoising · pp. 7–8, Table V · p. 9, Figure 3 · p. 9

C. Hyperspectral Image Super-Resolution

  • DiffTR achieves the highest PSNR in 8 of 15 HSI dataset–setting comparisons, with gains of 1.43 dB on PaviaU and 1.92 dB on CAVE at (s, σ) = (4, 2).At (16, 0.5), it leads all three metrics on WDC and PaviaU, while GTNN leads on the remaining WDC settings and on CAVE at s = 8, 16 with weak blur.Source: C. Hyperspectral Image Super-Resolution · p. 10, Table VI · p. 10
  • DiffTR matches or exceeds GTNN on most bands of PaviaU and CAVE at s = 4, and avoids several pronounced PSNR drops on WDC at s = 16.At s = 16, it also shows fewer regions of large spectral-angle error than GNLTN and EDIP-Net on WDC and PaviaU.Source: C. Hyperspectral Image Super-Resolution · p. 10, Figure 4 · p. 10
  • DiffTR uses 0.54–0.97 GiB of GPU memory, lower than CS2DIPs and EDIP-Net on every dataset, while GTNN and GNLTN have shorter total runtimes.On CAVE, DiffTR uses 0.54 GiB versus 11.45 GiB and 19.40 GiB; sampling and refinement add 8.03–16.33 seconds.Source: C. Hyperspectral Image Super-Resolution · p. 10

V. ABLATION AND DISCUSSION

The ablations examine coordinate conditioning, sparse-noise separation, spatial consistency, and noise-aware fitting. Their results show that component benefits depend on the corruption and setting.

A. Denoising Component Ablation

  • Removing coordinate conditioning lowers PSNR by 6.46 dB on PaviaU and 12.99 dB on WDC, while disabling sparse separation lowers PSNR by 0.08 and 1.99 dB, respectively.The sparse-update removal also reduces SSIM by 0.0257 on PaviaU and 0.1448 on WDC; removing spatial consistency raises PSNR by only 0.024 and 0.048 dB, changes smaller than reported standard deviations.Source: A. Denoising Component Ablation · pp. 10–12, Table VIII · p. 12

B. Noise-Aware Denoising and Proximal Guidance

  • Likelihood fitting improves PSNR over BCD in five of six cases, and proximal refinement with g = 1 raises WDC Case 2 PSNR from 35.84 to 36.96 dB.In two-stage diffusion, changing g from 0 to 1 raises WDC Case 2 PSNR from 34.14 to 36.17 dB, whereas g = 4 reduces PSNR throughout both comparisons.Source: B. Noise-Aware Denoising and Proximal… · p. 12, Table IX · p. 12, Table X · p. 12
  • Huber fitting performs best on both WDC cases, and an additional Case 2 run reproduces gains from likelihood fitting over BCD and proximal refinement with g = 1.The compared ablations use matched crops and one training run per configuration, with an additional independent Case 2 run.Source: B. Noise-Aware Denoising and Proximal… · p. 12

C. Sampling Depth

The section tests DDIM sampling depth, HR-MSI measurement guidance, and post-sampling refinement. It reports quality gains with more sampling steps and stronger guidance, while refinement benefits vary by scene and degradation.

C. Sampling Depth

  • Increasing DDIM steps from 5 to 20 raises PSNR from 31.11 to 37.70 dB on PaviaU and from 30.35 to 38.81 dB on WDC.Going from 20 to 40 steps adds only 0.38 and 0.13 dB, while mean sampling time rises from 4.44 to 8.67 seconds on PaviaU and from 4.40 to 8.97 seconds on WDC.Source: C. Sampling Depth · p. 12, Figure 6 · p. 13

D. Measurement Guidance

  • Increasing measurement guidance g from 0 to 15 raises PSNR from 20.03 to 38.08 dB on PaviaU and from 19.84 to 38.94 dB on WDC.The gains from g = 5 to g = 15 are only 0.68 and 0.08 dB, respectively.Source: D. Measurement Guidance · p. 13, Figure 6 · p. 13

E. Data Consistency and Spectral Subspace Regularization

  • Spectral subspace refinement improves PSNR over DC-100 in five of six configurations, but its gains and the effect of the reference reconstruction depend on scene and degradation.Gains include 1.97 dB on WDC with weak blur and 2.61 dB on CAVE Beers at s = 16, while Beers at s = 4 decreases by 0.32 dB; increasing the spectral penalty or retained energy does not consistently improve PSNR.Source: E. Data Consistency and Spectral… · pp. 13–14, Table XIII · p. 13, Table XIV · p. 13

VI. CONCLUSION

  • The ablations show that the shared representation accommodates coordinate information, corruption models, spatial constraints, and spectral measurements through task-specific observation models.The conclusion describes this separation of representation learning and observation modeling as a foundation for extending DiffTR to additional tensor structures and inverse problems.Source: E. Data Consistency and Spectral… · p. 14, §VI · p. 14

APPENDIX A GENERAL IMPLEMENTATION SETTINGS

The appendix records shared implementation settings and details two denoising fitting procedures: alternating sparse-noise separation with model updates, and deterministic likelihood fitting with observation-inferred masking and learned noise parameters.

APPENDIX A GENERAL IMPLEMENTATION SETTINGS

  • The general setup uses two-layer MLP embedding and score subnetworks with equal hidden widths unless otherwise specified.Table I collects training settings for main comparisons and denoising ablations.Source: Appendix A · p. 16, Table I · p. 16

APPENDIX B DENOISING: OPTIMIZATION AND EVALUATION

  • The denoising formulation defines a clean-value prediction from a perturbed observation and derives its corresponding score.The appendix denotes network and coordinate-factor parameters collectively by ψ.Source: Appendix B · p. 16

A. BCD: Sparse-Noise Separation and Model Fitting

  • BCD refreshes sparse estimates on each sampled mini-batch, then updates model parameters using clean-sample prediction loss.It predicts without tracking gradients, uses δ = 0.04 for λ/2, and applies one Adam update with a one-level diffusion setup.Source: A. BCD: Sparse-Noise Separation and… · p. 16

B. Likelihood: Fitting a Deterministic Estimate

  • Likelihood fitting uses an observation-inferred mask and a clipped Gaussian–outlier mixture, while Adam learns network parameters and per-band noise scales and outlier probabilities.Noise scales and probabilities are constrained to [0.02, 0.5] and [0.001, 0.4], respectively.Source: B. Likelihood: Fitting a Deterministic… · p. 17

C. Proximal Refinement and Two-Stage Diffusion

This section describes proximal correction within an estimated spectral subspace, a two-stage diffusion alternative, experimental protocols, and post-sampling refinement under observation and spectral constraints.

C. Proximal Refinement and Two-Stage Diffusion

  • Proximal refinement represents corrections in the leading r = 8 eigenvector subspace and approximates the solution with ten gradient steps.The post-fit variant uses the Likelihood output as its starting estimate; g = 0 leaves that output unchanged.Source: C. Proximal Refinement and Two-Stage… · p. 17
  • Two-stage diffusion trains on frozen Likelihood predictions and applies correction during 20-step deterministic DDIM sampling.The fitting and guidance comparisons use 96 × 96 crops, per-voxel zero masks, and single-run entries, with an additional independent Case 2 run in Table XII.Source: C. Proximal Refinement and Two-Stage… · p. 17, D. Experimental Protocols · p. 17

A. Data and Observation Operators

  • The HSI data protocols specify patch and image sets, degradation operators, sensor responses, and normalization for WDC, PaviaU, and CAVE.The WDC spectral-response correspondence assumes the stored HSI band order because wavelength metadata are unavailable.Source: A. Data and Observation Operators · p. 18

B. Post-Sampling Refinement

  • Post-sampling refinement uses the leading spectral basis and full observation operators, with conjugate gradients solving for a correction before clipping the reconstructed estimate.The main comparison uses (τ, γ) = (1000, 10), at most 400 iterations, and all 150 reconstructions meet the stated tolerance.Source: B. Post-Sampling Refinement · p. 18

C. Training, Selection, and Evaluation

The section specifies training and checkpoint selection without clean HR-HSI, benchmark-informed refinement selection, ablation protocols, and resource-matched baseline profiling.

C. Training, Selection, and Evaluation

  • Network fitting and checkpoint selection use no clean HR-HSI, while three reconstructions per checkpoint vary only the sampling seed.Targets are nearest-neighbor-upsampled LR-HSI values.Source: C. Training, Selection, and Evaluation · p. 18
  • Refinement settings were selected using mean PSNR on eight diagnostic combinations and applied to the remaining 42, with some scenes shared across the sets.The authors characterize refinement selection as benchmark-informed; all 50 combinations enter the final comparison.Source: C. Training, Selection, and Evaluation · p. 18

D. Ablation Protocols

  • Sampling-depth and guidance studies vary reverse-step count or guidance strength, while refinement ablations test specified settings and report three-sample or paired PSNR comparisons.The sampling studies use three matched initial-noise seeds without post-sampling refinement.Source: D. Ablation Protocols · p. 19

E. Baseline Budgets and Resource Measurement

  • Baselines receive DiffTR’s measured initialization-and-training time, and profiling includes DiffTR’s basis estimation, operator setup, sampling, and refinement.Loading, evaluation, and file I/O are excluded from timing; peak GPU memory excludes CUDA context memory and unused allocator reservations.Source: E. Baseline Budgets and Resource… · p. 19

Related papers

Loading 2610.35368v1…