PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoSPRINT: Geometric Redundancy-Aware Step Pruning for Inference in Diffusion Trajectories

Arpita Joshi

arXiv:2609.02160v1cs.LGcs.AI

TL;DR

Diffusion inference remains costly because sampling requires many sequential neural evaluations, and existing acceleration methods lack a global geometric criterion for step importance. GeoSPRINT is a training-free framework that detects redundant denoising steps from trajectory geometry, builds curvature-aware schedules, and measures trajectory straightness with αtraj. Across CIFAR-10, LSUN Church, and Stable Diffusion v1.5, it consistently improves matched-budget DDIM results, including a 0.22 FID reduction on LSUN Church at 52 steps and superiority to DPM-Solver++ on CIFAR-10 at NFE≥30.

  • Problem

    Diffusion sampling requires many sequential neural evaluations, while existing training-free methods lack a global trajectory-level criterion for geometrically important steps.

  • Method

    GeoSPRINT uses arbitrary-dimensional hyperplanarity tests to detect redundant trajectory steps, blends curvature density with log-SNR spacing, and introduces αtraj as a residual-variance measure of trajectory non-straightness.

  • Results

    GeoSPRINT consistently improves matched-budget DDIM schedules across CIFAR-10, LSUN Church, and Stable Diffusion v1.5, and first-order DDIM+GeoSPRINT outperforms DPM-Solver++ at NFE≥30 on CIFAR-10.

  • Takeaways & Limitations

    Trajectory geometry provides a useful global signal for allocating inference steps and improving sampling efficiency without retraining.

  • Takeaways & Limitations

    The original geometric instance reduction operates only in 2–3 PCA dimensions, and αtraj measures orthogonal reconstruction residual rather than retained-set covariance.

Abstract

from arXiv · show

Diffusion models achieve high sample quality but remain expensive at inference time because sampling requires many sequential neural function evaluations (NFEs). Existing acceleration methods either use fixed step-skipping schedules, adapt step sizes based on local numerical error, or require additional training. We introduce GeoSPRINT (Geometric Step Pruning for Inference in Trajectories), a training-free framework for constructing non-uniform sampling schedules from the geometry of denoising trajectories. GeoSPRINT detects geometrically redundant steps using a hyperplanarity test in latent space, implemented efficiently via QR factorization, and converts the resulting redundancy profile into a sampling schedule that allocates more steps to high-curvature regions of the trajectory. In addition, we introduce the trajectory projection score $α_{\mathrm{traj}}$, a residual-variance metric that quantifies trajectory straightness and serves as a model-free diagnostic for rectified flow quality. Across CIFAR-10 ($32{\times}32$), LSUN Church ($256{\times}256$), and Stable Diffusion v1.5 ($512{\times}512$ latent), GeoSPRINT consistently improves over uniform DDIM (Denoising Diffusion Implicit Models) schedules at matched NFE budgets. On CIFAR-10, GeoSPRINT improves FID (Fréchet Inception Distance) by 0.7-1.1 over DDIM across 49-89 NFEs and surpasses DPM-Solver++ at NFE${\geq}30$ despite using a first-order DDIM solver. On LSUN Church, it reduces FID from 1.48 to 1.26 at 52 steps, and on Stable Diffusion v1.5 it achieves up to 1.93 FID improvement over DDIM. These results show that trajectory geometry provides a useful global signal for allocating inference steps and that schedule quality can substantially improve diffusion sampling efficiency without retraining.

1 Introduction

Diffusion sampling is expensive because reverse trajectories require many sequential neural evaluations, while existing acceleration methods either skip uniformly, adapt locally, or require training. GeoSPRINT instead uses global denoising-trajectory geometry to identify redundant steps and allocate inference effort where trajectories curve.

  • Tens to thousands of sequential neural evaluations typically make single-sample diffusion generation computationally expensive.
  • Existing acceleration methods include uniform DDIM step-skipping, locally error-adaptive DPM-Solver methods, and training-based distillation or consistency approaches.
  • Training-free methods lack a global criterion for determining which trajectory steps contribute geometrically important information.
  • Ordered denoising trajectories can be treated as geometric data sequences, allowing approximately redundant points on shared linear or planar subspaces to be removed.
  • GeoSPRINT retains more steps where denoising trajectories curve sharply and prunes steps where slowly changing score fields produce nearly linear paths.
  • GeoSPRINT combines arbitrary-dimensional hyperplanarity testing, curvature-aware schedule construction, and the trajectory projection score αtraj for evaluating trajectory non-straightness.

2 Background and Related Work

Diffusion sampling solves a reverse-time dynamical system through sequential neural evaluations, motivating methods that reduce steps through skipping, higher-order integration, or learned mappings. Geometric instance reduction provides the conceptual basis for identifying redundant ordered trajectory points, but its original formulation is limited to low-dimensional PCA spaces.

  • Diffusion models and sampling: The score-based SDE framework describes diffusion through forward and reverse stochastic processes, with the score function approximated by a neural network.
  • Diffusion models and sampling: Sampling converts the reverse dynamics into an ODE discretized into N steps, with each step requiring one neural function evaluation.
  • Acceleration methods: DDIM reduces sampling through uniform timestep skipping, whereas DPM-Solver methods use exponential integration and local truncation-error adaptation.
  • Acceleration methods: Consistency models and consistency flow matching require additional training for target NFE budgets, while adaptive timestep methods operate on local timestep properties.
  • Geometric instance reduction: Geometric instance reduction removes ordered points that are approximately collinear or coplanar with neighbors while quantifying information loss through a projection score.
  • Geometric instance reduction: The original geometric reduction algorithm operates in 2–3 PCA dimensions, whereas diffusion latent spaces commonly range from 32–512 dimensions.

3 Method: GeoSPRINT

GeoSPRINT identifies geometrically redundant points in denoising trajectories and converts their curvature profile into a fixed, non-uniform sampling schedule. It also uses residual variance to quantify trajectory non-straightness and pruning information loss.

  • Hyperplanarity-based pruning: GeoSPRINT tests whether candidate latent states lie near the affine subspace spanned by a window of retained neighbors, removing approximately redundant points.The reference window uses the most recently retained, not necessarily consecutive, points.
  • Hyperplanarity-based pruning: QR factorization computes hyperplanarity residuals in O(d · k^2) time, which is O(d) when the window size k is typically 2.The implementation avoids direct pseudoinverse computation.
  • Schedule construction: Reference trajectories are generated with the full schedule, pruned individually, and aggregated into a universal schedule without per-sample inference overhead.The retained timestep sets from multiple trajectories provide the basis for fixed schedule construction.
  • Schedule construction: The schedule blends log-SNR density with smoothed, floored curvature density so that step allocation reflects both solver error characteristics and trajectory geometry.Pure curvature weighting can leave coverage gaps at low NFE, while log-SNR spacing ignores trajectory geometry.
  • Trajectory projection score: The trajectory projection score α_traj measures the fraction of trajectory variance lost through residuals orthogonal to locally retained affine structure.Low scores indicate that pruned steps add little geometric structure beyond what neighboring retained points reconstruct.
  • Trajectory projection score: For perfectly straight trajectories, collinearity pruning retains only the initial window and yields α_traj = 0; for rectified flow, α_traj decreases across reflow rounds.This makes α_traj a training-free diagnostic of trajectory non-straightness and rectification quality.

4 Theoretical Analysis

GeoSPRINT provides a reconstruction-error guarantee for pruning geometrically redundant trajectory points, with error controlled by the hyperplanarity threshold, pruning rate, and trajectory variance. Its residual test approximates orthogonal curvature and has manageable computational cost in high-dimensional latent spaces.

  • Reconstruction Error Bound: GeoSPRINT bounds reconstruction error through the hyperplanarity threshold τ, pruning rate |S|/(N+1), and total trajectory variance.The bound concerns orthogonal residuals of pruned points relative to the full trajectory.
  • Trajectory Projection Score: αtraj measures the fraction of trajectory variance represented by orthogonal reconstruction residuals from pruned points.It does not bound the retained set’s sample covariance, because pruning can change the sample mean and covariance even when residuals vanish.
  • Reconstruction Error Bound: For a target αtraj ≤ ϵ, the pruning threshold can be selected using the trajectory variance, trajectory length, and number of pruned steps.The sufficient threshold condition follows directly from the reconstruction-error bound.
  • Computational Cost: The multi-level test costs O(N · d · ℓ^2) per trajectory in the worst case, while Level 1 costs O(N · d) for k=2.Cached QR factorizations further reduce amortized overhead relative to neural function evaluations.
  • Geometric Interpretation: The hyperplanarity residual tracks orthogonal curvature, while tangential acceleration along a straight path contributes no residual.Removing points with residual r_i < τ therefore removes steps with small orthogonal curvature.

5 Experiments

Experiments across CIFAR-10, LSUN Church, and Stable Diffusion show that curvature-aware GeoSPRINT schedules generally improve FID at matched NFE budgets. The gains arise from combining log-SNR spacing with curvature information and concentrating steps in early and late denoising regions.

  • Experimental Setup: GeoSPRINT evaluates pretrained CIFAR-10, LSUN Church, and Stable Diffusion v1.5 models at matched NFE budgets against DDIM and, on CIFAR-10, DPM-Solver++.FID uses 10,000 samples for CIFAR-10 and LSUN Church and 5,000 for Stable Diffusion v1.5.
  • Main Results: 1.93 FID improvement is the largest reported SD v1.5 gain, while the SD comparison uses a 50-step DDIM reference rather than ground-truth images.This protocol measures relative trajectory fidelity rather than absolute generation quality.
  • Main Results: 1.26 versus 1.48 FID at 52 steps on LSUN Church demonstrates a −0.22 GeoSPRINT improvement over DDIM.The reported values are means with standard deviations over three seeds.
  • Schedule Analysis: Retention is bimodal: the late denoising zone has 2× the middle zone’s mean retention, and the early zone also receives elevated retention.Figure 3 reports that 50% of retained steps lie in the final 20% of denoising.
  • Trajectory Diagnostic: αtraj drops 450× from DDPM to DDIM across 50 CIFAR-10 trajectories, supporting its use as a training-free rectification diagnostic.The diagnostic is intended to quantify trajectory straightness without retraining.

6 Discussion

GeoSPRINT combines log-SNR spacing with trajectory curvature because DDIM update error depends on both noise-prediction gaps and curvature. Its geometric signal also links straighter trajectories with greater compressibility and improved rectification diagnostics.

  • Trajectory diagnostics: 450× lower αtraj from DDPM to DDIM supports αtraj as a training-free rectification diagnostic.
  • Trajectory geometry: Straighter trajectories are more compressible, connecting trajectory geometry to potential step reduction.
  • Schedule quality: GeoSPRINT blends log-SNR spacing with curvature density to address both noise-prediction error and trajectory curvature.Uniform-t spacing equalizes neither factor, while log-SNR spacing addresses only the first.
  • Schedule quality versus solver order: At NFE≥30 on CIFAR-10, first-order GeoSPRINT+DDIM outperforms second-order DPM-Solver++, whereas DPM-Solver++ leads at NFE≤10.The discussion attributes the moderate-budget pattern to schedule quality versus solver accuracy, while noting that broader validation remains future work.

7 Conclusion

GeoSPRINT constructs training-free non-uniform schedules from denoising-trajectory geometry, using hyperplanarity pruning, cached QR factorization, and blended density mapping. The resulting schedules improve sampling quality across the reported settings without retraining.

  • Framework: GeoSPRINT analyzes denoising-trajectory geometry to construct non-uniform sampling schedules without retraining.
  • Empirical outcome: The conclusion reports 0.7–1.1 FID improvement over uniform DDIM on CIFAR-10 and up to −1.93 FID on Stable Diffusion v1.5.It also reports that GeoSPRINT+DDIM outperforms DPM-Solver++ at NFE≥30 on CIFAR-10.
  • Schedule construction: The blended density is formed from log-SNR and curvature terms using ρ(t) = (1 − β) · ρlogSNR(t) + β · ρcurv(t).
  • Hyperplanarity pruning: HyperplanarityPrune returns pruned indices and residuals from an ordered point sequence using a sliding window.The algorithm initializes a window, tests points, updates the window, and returns both outputs.
  • Implementation: Thin QR factorization is cached until the window updates, then recomputed to reflect the algorithm’s amortized cost.

B Proof of Theorem 1

The proof uses hyperplanarity residuals to bound the variance contribution of pruned trajectory points. The interpretation is explicitly limited: the bound does not equal retained-set variance relative to full-set variance.

  • Residual condition: Each pruned point satisfies a hyperplanarity residual condition r_sj ≤ τ relative to the affine subspace of retained neighbors.
  • Projection score: The trajectory projection score is defined from residuals of the pruned set and quantifies orthogonal reconstruction variance.
  • Interpretation and limitation: The bound does not control tr(C_R)/tr(C_Z), because pruning changes the sample mean and can reduce retained covariance even when all residuals are zero.Removing interior points from a collinear set can yield αtraj = 0 while retained spread is smaller than full-set spread.

C Extended Experimental Details

The extended experimental-details section points readers to Figures 5 and 6.

  • Figures: Figures 5 and 6 provide the referenced extended experimental details.

C.2 Implementation Details

The implementation manually applies DDIM updates for non-uniform schedules and carefully configures DPM-Solver++ experiments. Trajectory normalization is applied before curvature analysis.

  • Non-uniform DDIM schedules use a manually implemented update with correct α_tprev lookup because Diffusers assumes uniform spacing internally.DPM-Solver++ uses order 2 with lower_order_final=True and a fresh scheduler per batch.
  • Trajectory normalization uses zero mean and unit variance per dimension before curvature analysis.

C.3 Hyperparameter Sensitivity

GeoSPRINT is relatively insensitive to key hyperparameters and reference-set size, while universal schedules accommodate diverse per-sample curvature profiles. Its offline cost is dominated by reference trajectory generation and is amortized across later samples.

  • The default collinearity test uses k=2; testing k=3 and k=4 produced no schedule-quality improvement, with FID differences below 0.1.B=100 is sufficient for CIFAR-10, while B=50 is used for LSUN Church and Stable Diffusion v1.5.
  • B=75 reference trajectories reach fuzzy Jaccard similarity 0.84 and retention-frequency correlation 0.86 relative to the B=100 schedule.
  • Reference trajectory generation costs 20,000 NFEs for CIFAR-10 and 10,000 NFEs for LSUN Church and Stable Diffusion v1.5, with the offline cost amortized over subsequent samples.Curvature analysis takes approximately 40 seconds on CPU for 100 trajectories of 200 steps in 3,072 dimensions.
Loading 2609.02160v1…