Source-linked AI summary
ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching
Chee-En Yu, Xiao-Xi Tan, Yi-Cheng Lin, Yun-Shao Tsai, Chee-An Yu, Hung-yi Lee
TL;DR
Sequential denoising makes diffusion sampling costly, while PinT methods can repeat network evaluations at similar inputs during refinement. ParaAnya caches outputs for similar timestep inputs and reuses them within parallel samplers. Across the evaluated settings it reduces evaluations and improves speed while maintaining comparable CLIP scores.
Problem
Diffusion sampling is sequential, and PinT refinement can repeatedly evaluate the same timestep even when its latent input changes only slightly.
Source: §1 · pp. 1–2Method
ParaAnya caches timestep-specific input-output pairs and reuses an output when the current latent remains sufficiently close to the cached input.
Source: §1 · p. 2Results
Takeaways & Limitations
The evaluated PinT samplers contain redundant network evaluations that output caching can reduce without changing their update rules.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Diffusion models have achieved remarkable success in generative tasks, but their inherently sequential sampling process introduces a severe computational bottleneck. Recent Parallel-in-Time (PinT) solvers attempt to mitigate this by parallelizing generation across a sliding window of timesteps, advancing the window only when step-wise changes stabilize. However, this overlapping window mechanism forces the network to repeatedly evaluate the same timesteps. When the input variations between iterations are minimal, these redundant evaluations lead to significant computational waste. To address this inefficiency, we propose ParaAnya, an output cache mechanism agnostic to the parallel sampling algorithm that can reduce the number of function evaluations (NFE). ParaAnya caches input-output pairs of diffusion models and reuses the cached output at overlapping timesteps. By dispatching only cache-miss timesteps to GPU workers, our approach eliminates redundant computation while preserving the structure of the underlying algorithms' update rules. We integrate ParaAnya into four representative parallel sampling algorithms and evaluate its performance on Stable Diffusion v1.5. Across four parallel samplers evaluated with DDIM on eight GPUs, ParaAnya provides $1.30$--$2.43\times$ speedups over their uncached counterparts and reduces NFE by up to 70.1\%, reaching up to a $5.62\times$ speedup over single-GPU serial sampling while maintaining comparable CLIP scores.
1. INTRODUCTION
Parallel-in-Time (PinT) methods address diffusion sampling’s sequential latency, but iterative refinement can repeatedly evaluate timesteps whose latent inputs have barely changed. ParaAnya reuses outputs for similar inputs, reducing evaluations and improving speed while maintaining comparable CLIP scores.
1. INTRODUCTION
- PinT methods parallelize across diffusion timesteps to address single-sample latency, which conventional multi-GPU batch inference does not reduce.Diffusion sampling is sequential because each denoising step depends on the previous timestep, and more steps often improve quality at the cost of inference time.Source: §1 · p. 1
- In a 100-step DDIM study over 30 prompts, 39.93% of requested drift evaluations revisited a timestep with mean latent change below 0.05.Unlike DeepCache, which reuses information across adjacent denoising timesteps, ParaAnya targets redundancy across PinT iterations at the same timestep.Source: §1 · p. 2
- Up to 5.62× speedup over single-GPU sequential sampling and up to 70.1% fewer network evaluations, with comparable CLIP scores, are reported for ParaAnya on Stable Diffusion v1.5.Across four representative PinT samplers, it also provides up to a 2.43× additional speedup over uncached counterparts.Source: §1 · p. 2
- ParaAnya reuses a timestep’s cached network output when its current input latent remains sufficiently close, and reevaluates it when the change exceeds a threshold.The cache is designed for PinT methods without retraining the underlying diffusion model.Source: Figure 1 · p. 1, §1 · p. 2
2. METHOD
PinT samplers refine estimates across timesteps, often revisiting network evaluations as the trajectory changes. ParaAnya caches outputs for similar inputs and supplies hits and newly computed misses to the solver without changing its update rules.
2.1. Parallel Sampling Preliminaries
- Picard-based PinT methods refine timestep states iteratively, and drift evaluations can run in parallel because they depend on states from the previous iteration.GPU memory constraints lead parallel samplers to use sliding windows that advance when leading states satisfy a convergence criterion.Source: §2.1 · p. 2
- Both Picard- and Parareal-based samplers revisit network evaluation points across refinement iterations as successive inputs can become increasingly similar.Parareal methods such as SRDS use coarse and fine solvers, but also revisit evaluations as trajectory estimates are refined.Source: §2.1 · p. 2
2.2. ParaAnya: An Output Caching for PinT diffusion solvers
- ParaAnya places a timestep-indexed cache before drift evaluation and compares the current latent with the most recently evaluated cached input.The mean absolute deviation is normalized by latent dimension, making the cache threshold comparable across latent resolutions; a timestep entering the window for the first time is a miss.Source: §2.2 · p. 2
- The coordinator reuses outputs for cache hits, batches misses for network evaluation, and assembles the drift values before the sampler continues its original update and convergence rules.The cache is updated only for misses, preserving the invariant that each stored output corresponds to its stored input.Source: §2.2 · p. 2
- ParaAnya applies the same reuse mechanism to Parareal coarse- and fine-solver evaluations while leaving the Parareal update rule unchanged.SRDS is given as an example of a Parareal-based sampler.Source: §2.2 · p. 2
3. EXPERIMENTAL SETUP
The evaluation measures function evaluations and CLIP score for Stable Diffusion v1.5 text-to-image generation, using paired prompt and noise conditions. It tests DDIM step counts with fixed sampler settings and reports results from specified GPU configurations.
3.1. Metrics
- The evaluation uses number of function evaluations (NFE) and CLIP score as metrics for ParaAnya.Source: §3.1 · p. 3
3.2. Models and Testing Prompts
- Stable Diffusion v1.5 generates 512 × 512 images from 1,000 shuffled COCO 2017 validation prompts, with methods sharing each prompt’s seed and initial noise.The model uses a half-precision UNet with xFormers attention.Source: §3.2 · p. 3
3.3. Algorithms
- DDIM denoising steps are swept over N ∈ {25, 50, 100} with window size P = 16 and fixed per-method convergence settings.ParaAnya uses τcache = 0.05 in the main sweep; cache tolerance is analyzed separately.Source: §3.3 · p. 3
3.4. Implementation Details
- Main and GPU-scaling comparisons use eight V100 GPUs, while window-size and cache-tolerance ablations use eight H100 GPUs.Parallel methods use all eight GPUs, the serial baseline uses one, and each method receives an untimed warm-up generation.Source: §3.4 · p. 3
4. RESULTS
Across four PinT samplers, ParaAnya improves efficiency, while experiments show that window size, cache tolerance, and GPU count shape measured outcomes.
4. RESULTS
- ParaAnya matches or improves wall-clock speedup in every tested configuration, reaching 5.62× with ParaTAA, while CLIP-score changes remain negligible.The results attribute consistent speedups to reduced NFE/img.Source: §4 · p. 3
4.1. The Effect of Window Size
- The best window size depends on the sampler and schedule: DDPM peaks at P = 64, while DDIM peaks at P = 8 for ParaSolver and P = 16 for ParaAnya.Both methods slow at P = 128 for DDPM and P = 32 for DDIM.Source: §4.1 · p. 3
4.2. The Effect of Cache Tolerance
- At τcache = 0.1, NFE falls from 176 to 57 and runtime to 0.299 s per image, yielding 3.56× speedup; τcache = 0.05 achieves 3.08× without CLIP loss.At τcache = 0.001 and 0.005, runtimes of 0.579 and 0.582 s per image slightly exceed the uncached 0.551 s.Source: §4.2 · p. 4
4.3. The effect of the number of GPUs
- On 8 GPUs, ParaAnya raises speedup from 1.25× to 3.13× for ParaDiGMS, 1.90× to 3.70× for ParaSolver, and 2.31× to 3.79× for ParaTAA.On one GPU, ParaAnya improves ParaSolver from 0.41× to 1.02×.Source: Table 4 · p. 4, §4.3 · p. 4
5. CONCLUSION
ParaAnya caches network outputs for sufficiently similar input latents without changing the sampler’s update rule. Across four PinT algorithms, it matches or improves speedup in every tested configuration while maintaining CLIP scores comparable to uncached baselines.
5. CONCLUSION
- ParaAnya reuses cached network outputs when the current latent is sufficiently close to the cached input, without modifying the underlying algorithm’s update rule.The cache is agnostic to the parallel sampling algorithm.Source: §5 · p. 4
- Integrated into four PinT algorithms, ParaAnya matches or improves wall-clock speedup in every tested configuration, reaching 5.62× on 8 GPUs with comparable CLIP scores.The conclusion attributes substantial speedups over serial sampling to the paired use of ParaAnya and parallel sampling algorithms on eight GPUs.Source: §5 · p. 4
- The results indicate that some excess network evaluations in PinT sampling are redundant computation that a cache can eliminate while preserving update rules.The authors identify online adaptation of τcache, higher-order solvers, and video and audio diffusion as possible extensions.Source: §5 · p. 4