Joint Branch-Space Transform Coding for Diffusion Activation Quantization with Classifier-Free Guidance

1. Introduction

• Activation quantization typically operates independently on conditional and unconditional coordinates, fixing the branch basis and overlooking their matched-pass relationship.The branches arise from passes through the same model weights.Source: §1 · p. 2
• Matched CFG activations have median uncentered correlation 0.86, versus 0.33 for mismatched pairs across DiT-XL/2 layers.The paper identifies this cross-branch structure as relevant to activation quantization under a fixed bit budget.Source: §1 · p. 2
• Branch-space coding rotates each matched conditional/unconditional pair, applies the existing low-bit quantizer, and then inverts the rotation.It changes branch coordinates, not model parameters, nominal activation precision, or the number of quantized values.Source: §1 · pp. 2–3
• Across diverse diffusion PTQ backbones, models, and datasets, GCBT consistently improves reconstruction and perceptual fidelity.It is applied on top of existing activation quantizers without modifying the underlying quantization method.Source: §1 · p. 3
• GCBT combines the CFG guidance direction with per-layer cross-branch second moments to determine the coding basis.Under an equal-rate quantization-noise surrogate, it yields a closed-form per-layer solution without gradient optimization or angle search.Source: §1 · p. 3
• The paper treats the CFG branch axis as an explicit coding space, where cross-branch structure between matched activations governs quantization fidelity.The authors describe this as a complementary design axis for diffusion PTQ.Source: §1 · p. 3

2. Related Work

• Diffusion PTQ methods address timestep-dependent distributions, architectural outliers, spatial and temporal variation, and activation outliers, but retain native CFG branch coordinates.The passage names Q-Diffusion, Q-DiT, ViDiT-Q, SVDQuant, and DeltaQuant as examples.Source: §2 · p. 3
• Rotation-based quantizers transform feature or hidden-state coordinates; this work transforms across the two-dimensional CFG branch axis while leaving token and feature coordinates unchanged.The cited approaches include orthogonal rotations, learned transforms, and timestep-dependent feature rotations.Source: §2 · p. 3, Figure 3 · p. 4
• The overview contrasts direct quantization of (X_c, X_u) with rotating the matched pair, quantizing in transformed coordinates, and rotating back.The coding basis is determined jointly by activation geometry and the CFG branch direction.Source: Figure 3 · p. 4
• CFG-aware methods exploit branch redundancy or guided-prediction sensitivity while retaining the native conditional/unconditional basis; this work selects the branch basis itself.The passage identifies TR-DQ and GAMP as examples of CFG-aware approaches.Source: §2 · p. 4

3.1. Problem Setup

• The method stacks matched conditional and unconditional activations along a CFG branch axis, while standard PTQ quantizes them in the native branch coordinates.Each token-feature coordinate corresponds to a matched two-dimensional branch-activation pair.Source: §3.1 · p. 4

3.2. Branch-Space Structure and Transform Coding

• Matched CFG activations have median correlation 0.859 versus 0.328 for mismatched pairs, and changing only the branch basis improves PSNR and SSIM while reducing LPIPS at CFG = 4.5.Branch-space transform coding applies an orthogonal transform to matched conditional/unconditional pairs; the identity transform recovers native branchwise quantization.Source: §3.2 · p. 5

3.3. Guidance-Correlation Branch Transform

• GCBT chooses a single coding basis jointly from the CFG branch direction and each layer’s uncentered second-moment geometry; calibration needs only three distinct entries of each 2 × 2 matrix, without gradient optimization or angle search.Source: §3.3 · pp. 5–6

3.4. Compatibility with Rotation-Based and CFG-Aware PTQ Backbones

• On a fixed CFG-aware W4A4 PixArt-Σ host, GCBT adds consistent LPIPS and SSIM gains on MJHQ and sDCI, while PSNR changes are not statistically conclusive.Branch-space and shared feature-space transforms commute, and GCBT changes the coding basis independently of CFG-aware calibration.Source: §3.4 · pp. 6–7

4.1. Experimental Settings

• Across image and video models, the evaluation uses multiple PTQ hosts, fixed paired prompt sets, and paired 95% confidence intervals to assess significance.Image hosts include SVDQuant and Q-DiT; video hosts include OrbitQuant and ViDiT-Q.Source: §4.1 · pp. 7–8

4.2. Main Results

• Image-generation comparisons show statistically significant gains in most paired metrics, with no significant degradation, across architectures, PTQ hosts, datasets, and model scales.The remaining comparisons are statistically inconclusive.Source: §4.2 · p. 8, Table 2 · p. 8
• Video evaluations show significant LPIPS and SSIM improvements for every model, alongside gains in subject consistency and aesthetic quality.The effects persist on the larger Wan2.1-T2V-14B model and extend to generation-level quality.Source: §4.2 · pp. 8–9, Table 3 · p. 9

4.3. Ablations and System Analysis

• Adding measured branch geometry to a guidance-aligned basis yields further gains across all three fidelity metrics over guidance alignment alone.The comparison uses PixArt-Σ with SVDQuant W4A4 on MJHQ.Source: §4.3 · p. 10, Table 4 · p. 10
• GCBT’s fidelity gains over the host quantizer grow monotonically as guidance scale increases, with marginal perceptual improvement at s = 2.From s = 3 onward, LPIPS, SSIM, and PSNR improve consistently.Source: §4.3 · p. 10, Figure 5 · p. 10
• GCBT adds 1–2% runtime overhead while leaving peak GPU memory unchanged relative to host implementations.Measurements cover representative SVDQuant and OrbitQuant deployments; offline calibration is excluded.Source: §4.3 · p. 10, Figure 6 · p. 11
• A 64-sample geometry pool is sufficient for stable calibration: increasing it to 128 changes median principal-axis orientation by only 0.04°.Matched-branch correlation remains approximately 0.86, and the source remains strongly anisotropic.Source: §4.3 · p. 11, Table 5 · p. 11

• GCBT is a closed-form transform that jointly exploits matched-branch geometry and guidance sensitivity.Source: §5 · p. 11
• Across many settings, GCBT yields favorable fidelity and generation-quality improvements without modifying model parameters or the underlying quantization pipeline.Source: §5 · p. 11
• Branch-space coding is a lightweight, complementary design axis for diffusion PTQ, with negligible runtime and memory overhead.Source: §5 · p. 11

• An orthogonal rotation R maps a matched branch-activation pair x to transformed coordinates z = Rx.Source: §A.1 · p. 15
• The transformed-coordinate quantization error is defined as δ = b_z − z, and after rotation inversion the local branch-space error is R^Tδ.Source: §A.1 · pp. 15–16
• With the same κ_l for both transformed coordinates and κ_l > 0 independent of the basis, minimizing expected squared distortion is equivalent to minimizing D_sur(R).The derivation does not require a zero-mean quantization-error assumption.Source: §A.1 · pp. 15–16

A.2. Proof of Theorem 3.3

• When the covariance eigenvalues are equal, the objective is independent of the rotation angle and the surrogate is rotationally invariant.The proof expresses the rotation, dominant-eigenvector direction, and guidance direction as angles measured from the native conditional axis.Source: §A.2 · pp. 16–17

A.3. Commutation with Shared Linear Operators

• A shared linear operator can act directly on the quantized activation in transformed branch coordinates, avoiding reconstruction in the native branch basis first.For a shared affine bias, the bias is added after the inverse branch transform.Source: §A.3 · p. 17

B.1. Calibration-Size Stability

• Beyond 64 calibration samples, the principal-axis orientation changes negligibly, and matched-branch correlation remains near 0.855, above the mismatched reference.The estimated branch geometry is already stable at modest calibration sizes; the source remains strongly anisotropic across the sweep.Source: §B.1 · p. 17
• +0.031 LPIPS reduction, +0.044 SSIM gain, and +0.445 PSNR gain are reported for GCBT vs. Native.The paired 95% confidence intervals are [0.024, 0.038], [0.039, 0.050], and [0.287, 0.607], respectively.Source: §B.1 · p. 18, Table 7 · p. 18

B.2. Paired Statistics for CFG-Aware Calibration

• After CFG-aware calibration, paired intervals favor GCBT for LPIPS and SSIM on both datasets, while PSNR intervals overlap zero.The study uses a separate BF16 execution setup; its absolute values are not comparable with those in Table 2, so the evidence is the paired Host versus +GCBT comparison under identical conditions.Source: §B.2 · p. 18

B.3. Additional Statistics for Branch-Basis Ablation

• All branch-basis variants are compared on the same 256 MJHQ prompts with identical generation seeds.Bootstrap 95% confidence intervals use 10,000 prompt-level resamples.Source: §B.3 · p. 19
• Guidance alignment significantly improves over Native in LPIPS, SSIM, and PSNR, and GCBT further improves over the guidance-aligned basis on all three metrics.The full GCBT–Native comparison is favorable on all three metrics.Source: §B.3 · p. 19
• The comparisons use paired intervals in Table 8(b), rather than overlap between marginal confidence intervals of absolute means.Source: §B.3 · p. 19

B.4. Full Experimental Protocol & Details

• Image evaluations use 1024 × 1024 resolution; PixArt-Σ and SANA-1.6B use 20 steps, while SD3.5-Large uses 28 steps.All use CFG = 4.5, official non-distilled checkpoints, and fixed empty negative prompts; within each model, reference and quantized variants share prompts, seeds, samplers, and settings.Source: §B.4 · p. 19
• Video evaluations cover Wan2.1, CogVideoX-2B, and OpenSora v1.2 with model-specific resolutions, frame counts, sampling steps, and CFG settings.Each model uses non-distilled checkpoints with active two-branch CFG, and reference and quantized variants share generation settings and seeds.Source: §B.4 · p. 19
• Video comparisons use a fixed, category-stratified 256-prompt VBench subset, with prompts and seeds reused across the full-precision reference, Host PTQ, and Host+GCBT.Framewise metrics are averaged within each clip, with prompts as the paired statistical units; Table 9 reports paired improvement intervals, not uncertainty intervals around absolute means.Source: §B.4 · p. 20, Table 9 · p. 20
• For Wan, high BF16-referenced LPIPS is also present in the OrbitQuant baseline and can reflect trajectory shifts to different prompt-consistent realizations.The paper treats LPIPS as BF16-reference fidelity rather than standalone generation quality and reports VBench metrics alongside it.Source: §B.4 · p. 20

C. Limitations

• GCBT is optimal for the equal-rate quantization-noise surrogate, which neglects cross-coordinate error terms and does not guarantee minimum exact host distortion or end-to-end generation error.The surrogate assumes transformed-coordinate quantization error scales with projected second moments.Source: C. Limitations · p. 20
• GCBT directly applies to inference pipelines with an explicit matched conditional–unconditional pair; distilled or single-pass guidance and timestep-varying or dynamic CFG schedules are not studied.Source: C. Limitations · p. 20
• The CFG-aware comparison uses an author-implemented GAMP-inspired calibration, not an official GAMP reproduction, and results with the original implementation may differ.Source: C. Limitations · pp. 20–21