Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers

1 Introduction

• Pixel-space Transformers commonly use one token per patch, so finer detail increases global-attention cost; on 256 × 256 images, changing from 16 × 16 to 8 × 8 patches makes dense attention’s quadratic term 16× larger.The patch grid determines the model’s input representation, predicted output, and global token count.Source: §1 · p. 1
• The paper’s observation is that broad low-frequency structure benefits from long-range exchange, whereas fine detail is largely local.This motivates assigning communication according to a token’s output role.Source: §1 · p. 2
• SpecSlot assigns each anchor K slots with disjoint DCT coefficient groups, and only the lowest-frequency slot from each anchor enters global attention.All slots interact locally, while the same slot states also decode an anchor-aligned spatial correction.Source: §1 · p. 2
• Splitting the representation into W anchors and K slots makes anchor density and spectral subdivision separate allocation variables, while the global sequence contains only W carriers.The total slot count grows with WK, but global attention remains over W carriers.Source: §1 · p. 2
• ImageNet-512 W256/K4 improves FID by 2.84 over JiT-CM with 3.9% fewer GFLOPs, while text-to-image surpasses JiT-B/8 on DPG-Bench by 1.06 at 40.6% lower forward cost.SpecSlot is comparable to the FLOPs-matched JiT control on ImageNet-256 and converges faster.Source: §1 · pp. 2–3

Diffusion and pixel-space generation

• Pixel-space methods remove the autoencoder but process full-resolution states; approaches distribute compute through state reduction, cascades, extra detail components, or direct large-patch Transformers such as JiT [1].SpecSlot instead uses one backbone with indexed slots carrying fixed output and communication roles.Source: §2 · p. 3

Frequency structure and transform-domain generation

• Frequency has shaped generation representations, ordering, conditioning, objectives, and denoisers, but the cited lines do not assign fixed spectral groups both output ownership and communication eligibility within one pixel-space backbone.SpecSlot keeps the state, target, and loss in pixel space.Source: §2 · p. 3

Visual tokenization and allocation

• Token-allocation methods change how many spatial tokens are retained or processed, but keep the spatial patch as the allocation unit; SpecSlot adds K spectral slots to each of W anchors.The fixed budget N=WK becomes a split between anchor density and spectral subdivision.Source: §2 · p. 3, Figure 3 · p. 4

Global-local and sparse communication

• Existing global-local and sparse methods use windows, global tokens, bottlenecks, or learned interaction patterns [5] [6] [7] [8] [45] [46], whereas SpecSlot assigns global eligibility to one lowest-frequency carrier per anchor.All slots interact locally, so cross-anchor communication is spent on the low-frequency scaffold.Source: §2 · p. 3

3 Methodology

• SpecSlot keeps the rectified-flow state, target, and loss in pixel space while treating tokenization as an allocation of output roles and global communication.Anchor density and spectral subdivision are separate variables, and global attention is spent on the role that needs it.Source: §3 · p. 4

3.1 Shared Pixel-Space Rectified-Flow Objective

• The shared objective mixes image x with noise ε along a linear path and trains an image-shaped prediction converted to velocity using velocity MSE.The flow time is sampled as t = σ(g), with g ∼ N(−0.8, 0.82); the clamp avoids the singularity at t = 1.Source: §3.1 · p. 4

3.2 SpecSlot Architecture

• At fixed token budget N, W controls anchor density and K controls spectral subdivision, while the global sequence remains at W carriers.Equal N does not imply equal cost because communication and projection scale differently with the split.Source: §3.2 · pp. 4–5

3.2.1 Read: Spatial Anchors and Slot Expansion

• Read embeds each input cell as an anchor, mixes neighboring evidence, then expands each anchor into K slot states whose roles come from slot positions, output heads, and routing.All WK slots are evaluated at every flow time.Source: §3.2.1 · p. 5

3.2.2 Allocate: Radial Coefficients and Pixel Synthesis

• Allocate assigns each slot a disjoint radial group of DCT coefficients, then synthesizes overlapping windows with normalized overlap-add to mitigate cross-window discontinuities.Applying the matched synthesis frame to its analysis coefficients reconstructs x up to floating-point error.Source: §3.2.2 · pp. 5–6

3.2.3 Route: Carrier and Local Attention

• Route sends only slot 0, the lowest-frequency carrier from each anchor, through global attention while all slots mix locally within their anchor.The global quadratic term therefore grows with W rather than WK.Source: §3.2.3 · p. 6

4.1 Experimental Setup

• ImageNet models use the aligned training recipe; text-to-image evaluation uses a frozen OpenCLIP ViT-L/14 encoder, a trainable 32-query adapter, and a 50-step Heun sampler.Compute is measured as forward cost on one H800 in BF16; the text-to-image SpecSlot uses CFG 4.0 and JiT uses CFG 2.9.Source: §4.1 · pp. 6–7, §4.3 · p. 8

4.2 ImageNet Results

• At ImageNet-512, SpecSlot-K4 improves FID by 2.84 over JiT-CM at slightly lower cost and trails JiT-B/16 by 0.69 FID at 38.2% lower cost.Table 1 reports ImageNet quality and compute; the control is resolution-matched.Source: Table 1 · p. 7, §4.2 · p. 8
• On ImageNet-256, SpecSlot-K4 matches JiT-CM’s mean FID within the five-seed 95% interval at nearly identical FLOPs and throughput, while descending faster at early-to-mid checkpoints.SpecSlot-K2 and K4 are nondominated operating points; K4 uses 40.9% fewer FLOPs than quality-leading JiT-B/8.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 Text-to-Image Generation

• On text-to-image generation, SpecSlot attains the highest DPG-Bench score, improving over both JiT baselines at 40.6% lower forward cost than JiT-B/8.On GenEval, SpecSlot improves over JiT-B/16, while JiT-B/8 remains the quality leader at 1.7× the forward cost.Source: §4.3 · pp. 8–9, Table 2 · p. 8

5.1.1 Anchor Density and Coefficient Subdivision

• At fixed token budget, shifting allocation toward more anchors lowers FID monotonically, while increasing K has diminishing returns; the tested N=2,048 configurations favor anchors, without establishing a broader rule.The last doubling of K buys only 0.31 FID.Source: §5.1.1 · p. 9

5.1.2 Input and Output Representations

• At the same spectral output, spatial input surpasses frequency input, while the dual-basis hybrid achieves FID 6.59, 0.40 below the best single-basis variant at 6.99, without extra tokens or cross-anchor edges.The authors interpret the margin as evidence that the spatial correction supplies structure missed by the spectral base alone.Source: §5.1.2 · pp. 9–10

5.1.3 Carrier Routing

• At essentially identical FLOPs, GGLL is better and faster than Dense-global-CM; full-global attention reaches lower FID at 74.8% higher cost, while GLGL degrades FID at identical compute.The carrier design saves 42.8% of full-global cost.Source: §5.1.3 · p. 10

5.2 Component Ablations

• Among tested evidence widths, B=1536 gives the best FID; the residual mixer and 50% overlap improve FID at nearly unchanged forward cost.These findings support their inclusion in the final contract.Source: §5.2 · p. 10, Table 6 · p. 10

5.3 Sampling Efficiency

• Across 20, 30, and 50 Heun intervals, GGLL throughput is 1.47–1.49× higher than the compute-matched dense-global control.The measurements use fixed-batch BF16 sampling at batch 512.Source: §5.3 · p. 10

6.1 Spectral Base and Spatial Correction

• The spectral term has larger self-energy; spatial share rises from 0.07 to 0.23, and removing spatial or spectral terms raises velocity MSE 32.1× or 64.1× at t=0.90.The cross term stays positive, and the spatial term dominates higher radial groups late in the flow.Source: §6.1 · p. 11

6.2 Carrier Identity and Routed Communication

• Replacing the lowest-radial carrier with slot 1, 2, or 3 raises mean FID; the lowest-radial group emerges earliest, and removing carrier edges costs most early while removing local cross-band edges costs most late.Table 7 varies only which slot receives cross-anchor access; Figure 6 probes route-edge interventions inside the shared Transformer.Source: §6.2 · p. 11, Table 7 · p. 11

7 Conclusion

• SpecSlot splits a fixed token budget between anchor density and spectral subdivision, assigning global attention by role.Source: §7 · p. 11
• On ImageNet-256, SpecSlot reaches comparable final FID to the FLOPs-matched JiT control and descends faster early; on ImageNet-512, it improves FID by 2.84 over the control and approaches JiT-B/16 at 38% lower cost.Source: §7 · p. 11
• On text-to-image generation, SpecSlot scores 62.21 vs. 61.15 on DPG-Bench against JiT-B/8 at 40.6% lower forward cost.Source: §7 · p. 11

• Tables 8 and 9 record the implementation and training contracts for the final class-conditional and text-to-image models.Source: A Implementation Contract · p. 15

• The coarse-grid optimum, CFG 2.90, was also the fine-grid optimum.The fine search examined a local ±1 window in increments of 0.25.Source: B Classifier-free Guidance Sweep · p. 15
• The search held the epoch-120 EMA1 checkpoint and evaluation protocol fixed, changing only CFG.It evaluated 50K images with the legacy-uniform 50-step Heun sampler and generation seed 0.Source: B Classifier-free Guidance Sweep · p. 15
• JiT uses CFG 2.9 in the main comparisons, matching the scale used by Spectral Forcing [9].Source: B Classifier-free Guidance Sweep · p. 15

• Latency measurements cover 20, 30, and 50 Heun intervals, corresponding to 73, 113, and 187 network evaluations per image.Measurements used one H800, BF16, a fixed sampling batch of 512, one warm-up repeat, and three timed repeats.Source: C Sampling-step Latency Scaling · p. 17
• GGLL throughput is 1.47–1.49× higher than Dense-global-CM and 1.70–1.71× higher than full-global across all three tested budgets.Dense-global-CM is the primary routing comparison because its per-evaluation FLOPs are nearly the same as GGLL.Source: C Sampling-step Latency Scaling · p. 17
• Figure 8 plots sampling latency against profiler-observed network evaluations per image, with nominal Heun intervals shown in parentheses.The latency measurements isolate fixed-batch model sampling under per-network compilation; main-table quality results use 50-step evaluations.Source: Figure 8 · p. 17, C Sampling-step Latency Scaling · p. 17

D Five-seed ImageNet-256 Results

• The five-seed evaluation fixes generation seed zero and uses epoch-120 EMA1 weights, 50K images, and the legacy-uniform 50-step Heun sampler.Training seeds identify independently trained checkpoints; K2 and K4 use CFG 5.0 and 4.0, respectively.Source: D Five-seed ImageNet-256 Results · p. 18
• Figure 4's aligned five-seed mean trajectories complement the endpoint statistics.Source: D Five-seed ImageNet-256 Results · p. 18

E Additional Mechanistic Diagnostics

• The mechanistic probes intervene on checkpoint states without parameter updates, characterizing representation and communication rather than end-to-end quality gains.The route probe targets the spectral-slot/GGLL submodule after slot formation and before the output head.Source: E Additional Mechanistic Diagnostics · p. 18

E.1 Evidence timing and route interventions

• The empirical DNR crossing time marks when each radial group’s noisy input first has at least as much signal as noise; it is descriptive, not a learned gate.Table 14 reports crossings for paired W256/K8, W256/K4, and W1024/K2 checkpoints.Source: §E.1 · pp. 18–19, Table 14 · p. 19
• Removing the global carrier path primarily raises low-group and early error, whereas removing local mixing raises detail-group error later in the trajectory.For W256/K8 and W256/K4, mid-time log-MSE penalties are approximately 0.14/1.66 and 0.20/2.14 (global/local).Source: §E.1 · p. 19

E.2 Allocation and effective use of additional slots

• As within-anchor sequence grows, effective-slot and effective-local-key fractions decline despite little change in teacher-forced velocity error; added slots need not add equal independent information.The analysis compares K ∈{2, 4, 8} at fixed W = 256 and B = 1536.Source: §E.2 · p. 19

F.1 ImageNet examples

• Under matched class and noise draws, SpecSlot W256/K4 shows clearer class-specific fine structure and fewer visible local distortions than JiT-CM.Examples include clearer feather boundaries, cornet geometry, and boathouse structure.Source: §F.1 · p. 19, Figure 9 · p. 20

F.2 Text-to-image examples

• DPG-Bench prompt samples span portraits, creatures, environments, architecture, and interiors; Appendix Figure 10 adds W256/K4 examples.The prompt paired with each additional sample is included in Figure 10.Source: §F.2 · p. 19, Figure 10 · pp. 21–22