JIVE: Jacobian-Informed Volume Expansion for Diverse Generative Sampling

1 Introduction

• Low diversity limits generative models’ usefulness in data augmentation, content creation, and exploration, while prior batch-repulsion methods push jointly sampled trajectories apart.The introduction identifies this gap as especially problematic for few-step models, which leave less opportunity for trajectories to diverge.Source: §1 · p. 1
• JIVE samples velocity perturbations from the endpoint Jacobian’s leading right singular subspace to explicitly maximize endpoint change.The endpoint Jacobian measures how local perturbations affect the estimated endpoint.Source: §1 · p. 1, Figure 1 · p. 2
• Theorem 1 connects maximizing local endpoint change to increased pairwise distance and spread among multiple generated endpoints.The method selects perturbations based on their effect on a single trajectory, while the theoretical result concerns multiple generations.Source: §1 · pp. 1–2
• JIVE avoids forming the full Jacobian or its singular value decomposition, using Jacobian and Jacobian-transpose vector products to approximate a top right singular vector.The paper reports that a single power-iteration step can work without full convergence.Source: §1 · p. 2
• Across standard benchmarks, JIVE consistently enhances diversity across metrics including Feature Vendi and Pixel Vendi, with superior or on-par performance against baselines and existing methods.With matched initial seeds for “A scientist,” Figure 2 shows JIVE producing distinct subjects and backgrounds where the original samples have similar compositions.Source: §1 · p. 2, Figure 2 · p. 2

2 Method

• Flow matching transports Gaussian noise toward data, and the one-step predictor approximates a trajectory’s endpoint for analyzing local generator behavior.The predictor is Ft(x) = x + (1 − t)uθ(x, t).Source: §2 · p. 3

2.1 Motivation: independent direction selection

• Batch-repulsion directions need not maximize endpoint spread; in the toy example, JIVE uses amplified rotational directions to send samples toward new modes.OSCAR’s volume gradient points outward from the batch mean, while JIVE selects directions independently using the endpoint Jacobian’s local geometry.Source: §2.1 · p. 3

2.2 Algorithm

• JIVE samples a velocity perturbation from the endpoint Jacobian’s leading right singular subspace, which produces the greatest endpoint change, and perturbs the trajectory once.The subspace is spanned by the right singular vectors with the k largest singular values.Source: §2.2 · p. 4

2.3 Theoretical guarantee of gained diversity

• Theorem 1 connects zero-mean local perturbations to increased expected squared pairwise distance between linearized endpoints.The expected spread increase ξ grows quadratically with perturbation magnitude ε, and the top right singular subspace maximizes the local objective that also optimizes overall spread increase.Source: §2.3 · p. 4

2.4 Efficient computation of velocity perturbations

• JIVE avoids forming the Jacobian by using matrix-free Jacobian-vector and transpose-vector products to estimate its leading singular directions.The Jacobian-vector product uses finite differences; the transpose product uses reverse-mode automatic differentiation, and block power iteration yields a basis for the leading subspace.Source: §2.4 · pp. 4–5
• A single JVP is a low-cost alternative, while JᵀJ iterations are more reliable; JVP also outperforms isotropic random velocity perturbations.A single JVP requires roughly twice the runtime of a standard forward pass; the lower-budget variants use one isotropic probe and a random sign.Source: §2.4 · p. 5

3.1 Benchmark diversity

• Both JIVE variants improve F-Vendi, P-Vendi, and pairwise L2 over OSCAR at every prompt-difficulty level, with modestly lower text alignment and larger image-quality and preference costs.Against CADS, standalone JIVE yields greater pixel-space diversity but lower feature-space Vendi.Source: §3.1 · p. 6
• Adding JIVE to CADS with perturbation norm 4 improves all three diversity measures over CADS in every band, with small decreases in quality and alignment.The authors describe this combination as an effective plugin for existing diversity guidance.Source: §3.1 · p. 6

3.2 Low cost direction estimation

• Across both models, JIVE(J⊤t Jt) generally achieves higher Vendi-DINO at comparable CLIPScore or HPSv2 than JIVE(JVP).The paper attributes this to more perturbation energy in high-gain right singular directions; JVP remains a cheaper direction-estimation alternative.Source: §3.2 · p. 7
• On one-step FLUX.1-schnell, JIVE(J⊤t Jt) has a more favorable diversity tradeoff than CADS for CLIPScore and GenEval, while CADS is stronger for HPSv2 and competitive for PickScore.Both JIVE variants offer stronger diversity-quality tradeoffs than OSCAR in the evaluated settings.Source: §3.2 · p. 8
• On four-step SD3.5-Large-Turbo, JIVE(J⊤t Jt) shows the strongest observed diversity-quality tradeoff across CLIPScore, HPSv2, PickScore, and GenEval.JIVE’s advantage over CADS is more consistent across quality metrics on Turbo than on FLUX; JVP also generally improves on CADS and OSCAR while requiring less direction-estimation computation.Source: §3.2 · p. 8

3.3 Improving diversity in inversion-based generation

• In inversion-based generation, JIVE improves F-Vendi, P-Vendi, and LPIPS across all five datasets, while KID generally improves.Outputs move farther from the source while generally preserving or improving quality, suggesting meaningful alternative reconstructions rather than arbitrary distortion.Source: §3.3 · pp. 8–9

4 Related work

• OSCAR, DiverseFlow, and Particle Guidance use repulsive interactions among sampling trajectories, whereas JIVE selects each perturbation independently using the generator’s Jacobian [Wu et al., 2026, Morshed & Boddeti, 2025, Corso et al., 2024].Noise optimization instead optimizes initial latents using output-diversity objectives and quality rewards, jointly or sequentially.Source: §4 · p. 9
• CADS perturbs text conditioning, STRIDE perturbs intermediate transformer features, and LOCO Edit uses leading Jacobian singular directions for localized editing; JIVE perturbs sampling latents to induce diversity [Sadat et al., 2024, Yadav et al., 2026, Chen et al., 2024].JIVE can also be combined with CADS and does not require modifying internal features or choosing architecture-specific injection layers.Source: §4 · p. 9

5 Conclusion

• JIVE uses the generator’s Jacobian to identify velocity perturbations that alter a trajectory’s endpoint.The analysis connects this local criterion to greater expected spread among independently generated outputs.Source: §5 · p. 9
• JIVE improves diversity across evaluated generators and remains effective in low-cost settings.Its inversion-based application demonstrates usefulness beyond text-to-image generation.Source: §5 · p. 9
• A general norm-selection principle for latent perturbations remains open, despite a theoretical bound and experimental test of the perturbation norm.The authors identify developing such a principle as future work.Source: §5 · p. 9

A.1 Ring of Gaussians toy experiment

• The target is sixteen equally likely Gaussian modes centered evenly on a radius-2 circle, each with standard deviation 0.08.The source is a centered two-dimensional Gaussian with standard deviation 0.25.Source: §A.1 · p. 12
• The analytic velocity is the conditional average displacement Y − Z for source-target paths passing through the current location.Near the center, several target modes contribute; nearer the ring, nearby modes increasingly determine the final direction.Source: §A.1 · p. 12
• The comparison uses eight shared source samples and 800 steps; JIVE perturbs each sample once after step 100 without coordinating the others.The middle panel applies deterministic log-determinant repulsion at every step, with its force vanishing near the endpoint.Source: §A.1 · p. 12

A.2 Proofs of the main results

• The cross-covariance between the original endpoint and the Jacobian-transformed perturbation vanishes because the perturbation has conditional mean zero.The proof then invokes the law of total covariance.Source: §A.2 · p. 12
• For independent identically distributed vectors with finite second moments, expected squared pairwise distance equals twice the trace of their covariance.The proof applies this identity to the perturbed endpoints and uses trace cyclicity.Source: §A.2 · p. 12
• Among orthonormal k-dimensional subspaces, the top right singular vectors achieve the largest weighted sum of squared singular values.The proposition considers a Jacobian with ordered singular values and Q satisfying Q⊤Q = I_k.Source: §A.2 · p. 12

A.3 Direction estimation algorithm

• Algorithm 1 is the concrete rank-one implementation of Jacobian-guided trajectory perturbation for rectified-flow generation.It takes the velocity field, denoising-step count, initial noise, power-iteration count, finite-difference step, and perturbation norm as inputs.Source: §A.3 · p. 13
• The direction estimate starts from a random vector and uses finite-difference power iterations.The iteration count is Kpow.Source: §A.3 · p. 13
• The algorithm adds a randomly signed perturbation of norm γ to the initial sample, then integrates the velocity field for N steps to return x_N.The sign is sampled uniformly from {−1, +1}.Source: §A.3 · p. 13

A.4 Eigenvector and singular vector alignment

• The reported alignment observation is limited to the models tested.Source: §A.4 · p. 13
• The top eigenvector and top singular vectors are strongly aligned in multi-step models, but less aligned in distilled models, especially for the right singular vector.Figure 7 reports absolute cosine-similarity histograms at t = 0, with 500 samples per model.Source: Figure 7 · p. 13, §A.4 · p. 13

• Random perturbation directions generally fail to meaningfully alter the final image.Source: §A.5 · p. 13
• Random directions place much of their energy in right singular vectors with small singular values, making the overall endpoint displacement small.Source: §A.5 · p. 13
• Both distilled models have rapidly decaying singular-value spectra, so alignment with top singular directions yields greater endpoint displacement per unit perturbation.Source: §A.5 · p. 13, Figure 8 · p. 14

A.6 Runtime and Approximation Power for Perturbation Computation Methods

• More than 91% of direction energy falls in the top four right singular directions for both models with one J⊤tJ product; one-iteration JVP beats random in leading-subspace energy and endpoint-diversity gain.J⊤tJ is the stronger low-cost estimator; JVP may not align with a top singular vector, but is the cheapest alternative.Source: §A.6 · p. 14, Table 5 · p. 14

A.7 Additional tradeoff curves

• Both low-cost JIVE variants improve diversity–GenEval tradeoffs over CADS or OSCAR; finite-difference and exact estimates yield similar frontiers, while random directions yield almost none of the diversity gain.The frontier comparisons use Flux.1-schnell at one step with four images per prompt; the CADS/OSCAR comparison covers FLUX.1-schnell and SD3.5-Large-Turbo.Source: Figure 9 · p. 15, Figure 10 · p. 16, Figure 11 · p. 16

A.8 Additional qualitative samples

• Across twelve prompts and matched initial seeds, JIVE produces broader changes in appearance, viewpoint, and style while preserving prompt content.The examples compare unperturbed samples in upper rows with JIVE samples in lower rows.Source: Figure 13 · pp. 18–20

A.9 Category-level results

• Category-level results average 10 prompts and 128 images per arm, while detailed and general prompts are matched by category and index to control the specificity contrast.The table caption states that higher values are better; the supplied passages do not provide category-level metric values.Source: Table 8 · p. 21, §A.10 · p. 21, §A.11 · p. 26

A.11.2 H — Human/animal

• Human/animal prompts include people and animals in scenes such as a ballerina stretching at a barre and an owl perched on a fence post.Source: §A.11.2 · p. 26

A.11.3 N — Natural

• Natural prompts depict environments and weather, including a mountain lake, a tropical forest waterfall, and a thunderstorm over a canyon.Source: §A.11.3 · p. 26

A.11.4 S — Structured

• Structured prompts feature built environments such as a living room, a city street, a castle banquet hall, and a subway station.Source: §A.11.4 · p. 26

A.11.5 C — Compositional

• Compositional prompts combine unexpected subjects or actions, such as a panda in a spacesuit on the moon and an octopus playing piano underwater.Source: §A.11.5 · p. 26

A.11.6 F — Food

• Food prompts cover dishes and dining scenes, from ramen and pancakes to a breakfast table with coffee and croissants.Source: §A.11.6 · p. 26

A.11.7 A — Artistic

• Artistic prompts specify media or visual styles, including watercolor, charcoal, paper-cut artwork, stained glass, mosaic, and low-poly 3D rendering.Source: §A.11.7 · p. 26

A.11.8 T — Text/signage

• Text/signage prompts include signs, posters, menus, book covers, murals, and displays bearing words or titles.Source: §A.11.8 · p. 26