FlexTok: Resampling Images into 1D Token Sequences of Flexible Length

• Traditional 2D and recent 1D tokenizers use fixed-length representations, making token counts depend on image size rather than semantic complexity.Conditional generators must therefore predict the full token set even for simple conditions.Source: §1 · p. 2
• Earlier FlexTok tokens capture high-level semantic and geometric information, while later tokens progressively add finer visual details.This hierarchical ordering replaces rigid raster-scan organization with coarse-to-fine generation.Source: §1 · p. 2
• FlexTok encodes images into ordered sequences of 1 to 256 tokens, with nested dropout and causal masking arranging information from coarse to fine.Truncated sequences of lengths 1, 2, 4, ..., 256 remain decodable into plausible images.Source: §1 · p. 2
• 8 tokens can generate ImageNet-1k classes, whereas complex text conditions can benefit from predicting up to 256 tokens.The visual vocabulary therefore supports shorter sequences for coarse conditions and longer sequences for more detailed prompts.Source: §1 · p. 3
• A rectified-flow decoder enables high-quality reconstruction across token lengths, including extreme compression rates.The decoder is trained end-to-end to maintain plausible reconstructions from variable-length sequences.Source: §1 · pp. 2–3

• Image tokenizers remove imperceptible details through compression and enable generative models to produce categorical per-token distributions.This maps challenging continuous, high-dimensional image spaces into compact discrete latent spaces.Source: §2 · p. 3
• VQ-VAEs use an encoder, quantizer, and decoder to transform images into discrete codebook representations and reconstruct them.They support image, audio, video, novel-view synthesis, and multimodal pretraining applications.Source: §2 · p. 3
• Follow-up work explores architectures, objectives, codebooks, diffusion decoders, and finite scalar quantization (FSQ) (Mentzer et al., 2023).FSQ replaces learned codebooks with low-dimensional projections and fixed per-dimension bins while maintaining reconstruction quality and simplifying training.Source: §2 · p. 3
• TiTok (Yu et al., 2024a) produces compact 1D sequences from learned register tokens, but requires two-stage training and a fixed token count.Its decoder reconstructs images by transforming learnable spatial mask tokens conditioned on quantized register embeddings.Source: §2 · p. 3
• FlexTok addresses complexity-independent sequence length by supporting as few as one token, while ElasticTok (Yan et al., 2024) and ALIT (Duggal et al., 2024) require at least 256 and 32 tokens, respectively.Unlike ElasticTok, ALIT, and One-D-Piece (Miwa et al., 2025), FlexTok is shown to support strong autoregressive generation; ViLex (Wang et al., 2024b) and CAT (Shen et al., 2025) focus more on continuous tokens for diffusion models.Source: §2 · p. 3

3. Method

• FlexTok maps 2D image patches to FSQ-quantized 1D register tokens, then conditions a rectified-flow decoder to reconstruct images from nested token subsets.A ViT encoder produces the register bottleneck, while encoded patches are discarded after operating in VAE-GAN latent space.Source: §3 · p. 3, Figure 3 · p. 4, §3.1 · p. 4

3.1. 1D tokenization with a rectified flow decoder

• A rectified-flow decoder predicts noise-to-latent flow from partially noised VAE latents and quantized registers, while REPA supervision improves convergence and downstream generation.The optimized objective is LFlexTok = LRF + λREPA · LREPA with λREPA = 1.Source: §3.1 · p. 4

3.2. Learning ordered 1D token sequences of flexible length

• Nested dropout randomly retains token prefixes, inducing an emergent coarse-to-fine ordering in which simple images use fewer tokens and complex images use longer sequences.Causal register attention reinforces prefix dependencies, and exponential nested-dropout sampling addresses undertraining of later tokens.Source: §3.2 · pp. 4–5

3.3. Autoregressive image generation

• Autoregressive Transformers evaluate FlexTok through class-conditional ImageNet-1k generation and text-to-image generation on DFN-2B.The experiments measure both reconstruction and generation performance, following LlamaGen for ImageNet-1k class-conditional generation.Source: §3.3 · p. 5

Stage 0: VAE training

• Stage 0 trains SDXL-VAE-style models to compress images into continuous 2D latent grids before FlexTok training, with later experiments using 16 channels and 8× downsampling.Versions with 4, 8, and 16 channels are trained on DFN.Source: §4 · p. 5

Stage 1: FlexTok training

• Stage 1 trains FlexTok to resample 2D VAE latents into discrete 1D sequences using nested dropout, rectified-flow decoding, and a 64,000-entry FSQ vocabulary.The models use up to 256 register tokens, 2×2 patchification, and 16× total pixel downsampling at 256×256 resolution.Source: §4 · p. 5

Stage 2: AR Transformer training

• Stage 2 trains LlamaGen-style autoregressive Transformers with learned absolute positions for FlexTok’s non-grid token sequences.Class conditioning adds a learned embedding to an [SOI] token, while models range from 49M to 1.3B parameters.Source: §4 · pp. 5–6

Reconstruction behavior

• More tokens improve reconstruction fidelity, while tokenizer scaling substantially improves reconstruction FID but affects MAE and DreamSim less strongly.FlexTok also reconstructs images semantically and in an ordered manner down to a single token within one model.Source: Figure 5 · p. 6, Figure 4 · p. 6

Stage 2: AR Transformer training

• Text-conditioned generation uses cross-attention to FLAN-T5-XL embeddings, scales to 3B parameters with µP, and applies conditioning dropout for classifier-free guidance.Text embeddings are projected to the autoregressive model dimension through an MLP, with text inputs randomly replaced by empty strings during training.Source: §4 · p. 6

5.1. Flexible-length tokenization

• FlexTok produces plausible reconstructions with as little as 1 token, while additional tokens rapidly improve fine-grained alignment, MAE, and DreamSim.Nested dropout orders high-level concepts first, and the rectified flow decoder allows any token subsequence to decode into a plausible image.Source: §5.1 · pp. 6, 8

5.2. Coarse-to-fine generation with increasing specificity

• Conditioning alignment improves with token count, while class accuracy plateaus around 32 tokens and generation quality remains consistent across sequence lengths.Alignment is measured with DINOv2-L classification accuracy for class conditioning and CLIPScore for text conditioning; gFID remains stable across lengths.Source: Figure 7 · p. 7, §5.2 · p. 8
• Simple prompts can produce satisfactory images with 4–16 tokens, whereas detailed prompts benefit from the full 256-token sequence and show greater low-token variation.The number of tokens needed to fulfill conditioning therefore depends on prompt complexity, with variation across random seeds decreasing faster for simple prompts.Source: §5.2 · pp. 8–9, Figure 8 · p. 8

5.3. Scaling autoregressive model size

• For 1–8 tokens, decoded-image gFID is effectively independent of autoregressive model size, but sequences longer than 128 tokens scale strongly with model size.With few tokens, the rectified flow decoder drives most generation; with more tokens, an equally powerful autoregressive model is needed to maintain performance.Source: §5.3 · p. 9

5.4. System-level comparison

• FlexTok achieves superior reconstruction and generation quality at each token budget compared with previous 1D approaches, using one model for variable-length sequences.The system-level comparison evaluates ImageNet-1k class-conditional generation with autoregressive models and FlexTok sequences from 1 to 256 tokens.Source: §5.4 · p. 9, Table 1 · p. 9
• Text-conditioned FlexTok generation outperforms a matched 2D grid baseline from 2 to 128 tokens and achieves better full-length CLIPScore despite a slight gFID regression.The 2D baseline matches FlexTok d18-d28 in parameter count, training steps, and source dataset, while FlexTok uses 256 tokens for the full-length comparison.Source: §5.4 · pp. 9–10

6. Discussion & Conclusion

• FlexTok token sequences form a coarse-to-fine visual vocabulary for class- and text-conditional autoregressive image generation.The sequences support high-fidelity reconstruction and generation with very few tokens.Source: §6 · p. 10
• The required sequence length depends on task complexity: simple conditions may permit early stopping, whereas dense content such as text requires more tokens and semantically focused objectives.FlexTok can compress images to a single token, but structured content requires longer representations.Source: §6 · p. 10
• FlexTok-like adaptive tokenizers may extend to redundant domains such as audio and video, supporting representations that range from compact semantic codes to long detailed sequences.The discussion highlights possible applications to long-horizon video generation, understanding, and visual reasoning.Source: §6 · p. 10

A. Ablation of FlexTok Design Choices

• The FlexTok ablation study varies the VAE, token-ordering structure, decoder loss, and inductive-bias losses to develop a compact variable-length 1D tokenizer.These choices are evaluated toward high-quality image resampling across sequence lengths.Source: A. Ablation of FlexTok Design Choices · p. 16
• The default ablation setup uses 256x256 images, a 16-channel sampled VAE, FSQ with six dimensions and effective vocabulary size 64 000, and 2x2 latent patchification.The resulting effective spatial downsample factor is 16 from pixels to patch tokens.Source: A. Ablation of FlexTok Design Choices · p. 16
• The ablation models use d12-d12 encoders and decoders trained for 50B patch tokens, with 393M-parameter autoregressive Transformers trained for 94B ImageNet-1k tokens.One patch token corresponds to a 16x16 pixel patch, and the autoregressive models are trained for 300 epochs.Source: A. Ablation of FlexTok Design Choices · p. 16
• Ablation evaluation measures reconstruction with rFID, DreamSim, and MAE, and class-conditional generation with gFID under classifier-free guidance scale 1.5.Reconstruction metrics are computed on ImageNet-1k validation images, while gFID uses 50K generated samples against training-set reference statistics.Source: A. Ablation of FlexTok Design Choices · p. 16

A.1. VAE choice ablation

• Increasing VAE latent channels above 4 significantly improves FlexTok reconstruction quality, leading to the selection of a 16-channel VAE for the final setup.The choice optimizes performance for the largest-sized tokenizer.Source: §A.1 · p. 16

A.2. Resampling strategy ablation

• Rectified-flow decoding improves rFID and gFID but worsens MAE, while switching from 2D grids to 1D registers improves all evaluated metrics, especially rFID.The ablation also finds that logit-normal noise performs strongest but is unstable at extreme timesteps, leading the authors to select mode sampling for better reconstruction performance.Source: §A.2 · p. 17, Table 3 · p. 17

A.3. Use of inductive bias loss.

• REPA significantly improves DreamSim, rFID, and gFID, producing higher-fidelity, more semantic reconstructions and motivating its use in all subsequent experiments.REPA adds a cosine loss between predicted decoder features and DINOv2-L reference features, using a three-layer MLP to project decoded patch activations.Source: §A.3 · p. 17, Table 3 · p. 17
• REPA accelerates convergence to high-quality reconstructions, although it does not improve the tokenizer’s reconstruction-loss convergence.This distinction is reported alongside evaluation curves and reconstruction-loss curves for FlexTok d12-d12 models trained on ImageNet-1k.Source: §A.3 · p. 17, Figure 12 · p. 18, Figure 11 · p. 18

A.4. Structuring the 1D tokens and inducing an ordering

• The ordering ablation compares causal masking with uniform, pow2, and unifpow2 nested-dropout schedules, together with guidance-scale sweeps across the ablation models.These evaluations use FlexTok d12-d12 models on the ImageNet-1k validation set.Source: Table 4 · p. 19, §A.4 · p. 19, Figure 15 · p. 20
• A causal attention mask improves rFID and gFID over unstructured registers but does not support flexible-length tokenization.The mask implicitly orders register tokens, yet the authors therefore investigate nested dropout to enable variable sequence lengths.Source: §A.4 · p. 19, Table 4 · p. 19
• Pow2 nested dropout provides a more balanced quality profile across sequence lengths, whereas unifpow2 favors longer sequences but undertrains shorter ones.Uniform sampling trains each valid length on roughly 1/256 of samples, while pow2 and unifpow2 reduce possible lengths from 256 to 9.Source: §A.4 · p. 19, Figure 14 · p. 19
• Unifpow2’s preference for longer sequences improves suitability for complex images, but its lower-token performance is worse and pow2 sacrifices some quality at the full 256-token length.The trade-off reflects how each schedule weights different subsets of token sequences during training.Source: §A.4 · p. 19, Figure 14 · p. 19

Evaluation protocol

• Linear evaluation concatenates quantized 6-dimensional FSQ register tokens into one feature vector while keeping the tokenizer frozen.A linear classifier is trained on ImageNet-1k and evaluated on its test set; feature size is 6 × num register tokens.Source: B. Evaluating the Representation… · p. 21
• The study sweeps register-token counts across FlexTok d12-d12, d18-d18, and d18-d28 using tuned linear-probing hyperparameters.The sweep uses learning rate 5 × 10^-4, weight decay 0.05, crop scale [0.4, 1.0], and batch size 1024.Source: B. Evaluating the Representation… · p. 21, Figure 16 · p. 21

Results

• Performance consistently improves with additional register tokens, unlike TiTok, because FlexTok retains earlier tokens and adds information within one unified model.The linear layer can ignore unhelpful added tokens, so separability is expected to increase monotonically.Source: B. Evaluating the Representation… · p. 21
• 64.6% top-1 accuracy is achieved by FlexTok’s best configuration on ImageNet, significantly surpassing TiTok.The representations are evaluated with a frozen tokenizer trunk and a linear classifier.Source: B. Evaluating the Representation… · p. 21
• A larger encoder improves linear separability when all register tokens are active, whereas a larger decoder slightly degrades it.These trends compare FlexTok variants with different encoder and decoder sizes.Source: B. Evaluating the Representation… · p. 21

C.1. VAE details

• 0.354 rFID is achieved by the 16-channel VAE, which is therefore used to avoid limiting subsequent FlexTok reconstruction performance.Increasing latent channel dimension improves VAE reconstructions, while low-channel Stage 0 VAEs can harm Stage 1 FlexTok reconstruction.Source: §C.1 · p. 22

C.2. FlexTok details

• FlexTok uses Transformer resamplers with causal register ordering, FSQ quantization, nested-dropout mask tokens, and rectified-flow decoders conditioned on register tokens.The FSQ levels [8,8,8,5,5,5] yield an effective codebook size of 64 000; the decoder uses full register-patch attention and classifier-free null conditioning with probability 0.2.Source: §C.2 · p. 22

C.3. Autoregressive model details

• The class-conditional and text-conditional autoregressive models use causal decoder Transformers but differ in scaling, regularization, cropping, and augmentation choices.Class-conditional models omit µP and use dropout plus random crops and flips, whereas text-conditional models use µP, a 1e-2 learning rate, square center crops, and no augmentation.Source: §C.3 · p. 24

D. Related Work

• FlexTok extends prior adaptive tokenizers by supporting stable single-token operation, high-fidelity rectified-flow reconstruction across lengths, single-pass ordered encoding, and downstream generative modeling.Compared with ElasticTok, ALIT, and One-D-Piece, the supplied passages emphasize lower-token stability, efficient ordering, end-to-end Stage 1 training, and demonstrated generative scaling; ViLex instead targets variable-sized soft text tokens.Source: D. Related Work · p. 27

E. Additional ImageNet Reconstruction and Generation Metrics

• The supplementary ImageNet evaluations add PSNR, SSIM, LPIPS, DreamSim, CLIP score, gFID, sFID, Inception score, precision, and recall to the main reconstruction and generation analyses.Table 8 covers image-wise and distribution-level reconstruction metrics, while Table 9 reports generation metrics for a 1.33B class-conditional AR model using FlexTok d18-d28.Source: E. Additional ImageNet Reconstruction… · p. 29, Table 8 · p. 29, Table 9 · p. 29

F.1. FlexTok flow decoder timestep sweeps

• 25 denoising steps provide the selected balance between inference speed and reconstruction quality, because improvements diminish beyond 25 steps while MAE can worsen at low token counts.rFID and DreamSim plateau beyond 25 denoising steps, so 25 is used for subsequent ablations and model sizes.Source: §F.1 · p. 30, Figure 19 · p. 30

F.2. Ablating standard classifier-free guidance

• Standard classifier-free guidance produces narrow, token-count-dependent optimality basins, motivating the use of adaptive projected guidance (APG).This behavior is shown for FlexTok d18-d28 trained on ImageNet-1k.Source: §F.2 · p. 30, Figure 20 · p. 30

F.3. FlexTok normalized guidance sweeps

• APG produces shallower, more aligned guidance basins with improved rFID, while higher guidance improves and plateaus DreamSim but slightly degrades MAE.The sweep uses r = 2.5, η = 0, and β = −0.5; smaller models have less-aligned basins and higher optimal guidance values.Source: §F.3 · pp. 30–31, Figure 21 · p. 31

F.4. 1D vs 2D tokenizer classifier-free guidance ablation

• 1D FlexTok outperforms the controlled 2D tokenizer by a large margin on rFID and DreamSim across guidance scales, while 2D performs better on MAE.The comparison uses FlexTok d18-d28 trained on DFN-2B, the COCO 30.5k validation set, and APG for both models.Source: §F.4 · p. 32
• The ablation compares the largest tokenizer size, FlexTok d18-d28, trained on DFN-2B.Both the 1D and controlled 2D models use APG in the COCO 30.5k validation comparison.Source: §F.4 · p. 32
• The guidance-scale sweep evaluates FlexTok d18-d28 with 256 tokens against the controlled 2D tokenizer baseline.The sweep uses APG guidance scales and the 1D model is evaluated with its full 256-token sequence.Source: Figure 22 · p. 32

F.5. Final FlexTok inference hyperparameters

• Final inference hyperparameters are selected from the ablation observations and used in all subsequent image-generation experiments.The chosen configurations are reported separately for the model sizes and training datasets.Source: §F.5 · p. 32
• Timestep and guidance-scale distillation techniques applicable to diffusion and flow models could substantially reduce FlexTok decoder computation.The paper presents this as an expected computational benefit rather than a reported experiment in this section.Source: §F.5 · p. 32
• Table 10 specifies the selected inference configurations for three FlexTok model sizes trained on ImageNet-1k.The table covers tokenizer inference settings chosen for the ImageNet-1k models.Source: Table 10 · p. 32
• Table 11 specifies the selected inference configurations for three FlexTok model sizes trained on DFN-2B.The table covers tokenizer inference settings chosen for the DFN-2B models.Source: Table 11 · p. 32

G. Autoregressive Class-Conditional Image Generation Hyperparameter Sweeps

• With a 1.33B AR model and FlexTok d18-d28, CFG scale s = 1.0 yields the best gFID regardless of top-k threshold or generated-token count.This differs from LlamaGen class-conditional models, whose optimal CFG value was 2.0 (Sun et al., 2024), and avoids doubling the inference batch for unconditional samples.Source: G. Autoregressive Class-Conditional… · p. 33, Figure 23 · p. 33, Figure 24 · p. 33
• Increasing the top-k threshold improves gFID, while optimal generation FID uses no top-k sampling.The sweep is conducted with classifier-free guidance and varying numbers of generated tokens before decoding to image space.Source: G. Autoregressive Class-Conditional… · p. 33, Figure 23 · p. 33

H. Autoregressive Class-Conditional Image Generation Model Size

• The AR model-size sweep shows consistent training-loss improvements as parameter count increases under the FlexTok d18-d28 tokenizer.Models of different sizes are trained for 94B tokens on ImageNet-1k.Source: H. Autoregressive Class-Conditional… · p. 34, Figure 25 · p. 34
• The model-size evaluation uses no CFG and no top-k sampling, following the optimal inference parameters identified in the hyperparameter sweep.Generation is evaluated on the FlexTok d18-d28 tokenizer across different numbers of decoded tokens.Source: H. Autoregressive Class-Conditional… · p. 34, Figure 26 · p. 34
• Full-length generation improves significantly with larger AR models, although even the 1.33B model shows slight gFID regression beyond about 128 decoded tokens.At short sequence lengths, generation performance is largely independent of model size; larger models increasingly limit token-decoding errors as sequences lengthen.Source: Figure 26 · p. 34, H. Autoregressive Class-Conditional… · p. 34
• Scaling FlexTok from d12-d12 to d18-d28 improves measured gFID with a fixed 1.33B AR model, though tokenizer reconstruction can upper-bound generation quality.The comparison evaluates tokenizer sizes across varying generated-token counts and at 256 tokens.Source: H. Autoregressive Class-Conditional… · p. 34, Figure 27 · p. 35

I. Autoregressive Text-Conditional Image Generation Inference Hyperparameter Sweeps

• Classifier-free guidance improves gFID for every generated sequence length in the 3.06B FlexTok model and increases CLIPScore with stronger guidance.This differs from class-conditioned FlexTok models, where CFG harmed gFID; conditioning complexity and DFN-to-COCO distribution shift may contribute.Source: §I · p. 36, Figure 28 · p. 36
• CFG 2.5 optimizes gFID across generated sequence lengths, while CLIPScore improves monotonically as more tokens are generated.The gFID curve drops sharply before gradually increasing with token count, whereas text-image alignment continues improving.Source: §I · p. 36, Figure 29 · p. 36
• The 2D grid tokenizer shows a similar dependence on CFG scale, so CFG 2.5 is selected to balance gFID and CLIPScore across both approaches.Using the same scale enables a balanced comparison between FlexTok and 2D-grid generation.Source: §I · p. 36, Figure 30 · p. 37

J. Autoregressive Text Conditional Image Generation Model Size

• Scaling from 113M to 3.06B parameters lowers final loss for both tokenizers and significantly improves gFID when generating 256 tokens.These comparisons use the FlexTok d18-d28 tokenizer and COCO validation evaluation under the optimal inference parameters.Source: J. Autoregressive Text Conditional… · p. 38, Figure 33 · p. 38
• For both FlexTok and 2D-grid tokenizers, increasing autoregressive model size produces lower final loss after training on 284B DFN tokens.The scaling study fixes the FlexTok d18-d28 tokenizer for the main comparison and evaluates model-size loss curves for both tokenization schemes.Source: J. Autoregressive Text Conditional… · p. 38, Figure 31 · p. 38, Figure 32 · p. 38
• The scaling comparison generates 256 tokens and measures gFID on COCO validation, using the optimal inference parameters from Appendix I.All autoregressive models use tokenizers trained on DFN, enabling direct comparison between FlexTok and 2D-grid tokenization.Source: Figure 33 · p. 38

K.1. FlexTok image reconstruction for different numbers of tokens – multiple samples per token sequence

• More tokens make FlexTok’s sampled reconstruction distribution narrower and more specific to the original image.Five rectified-flow decoder samples are drawn from the same token sequence for ImageNet-1k validation images.Source: Figure 34 · p. 39, Figure 35 · p. 40

K.2. FlexTok image reconstruction for different numbers of tokens and tokenizer sizes

• FlexTok reconstructions are visualized across token counts and tokenizer model sizes d12-d12, d18-d18, and d18-d28.The examples use ImageNet-1k validation samples.Source: Figure 36 · p. 41, Figure 37 · p. 42

K.3. Image reconstruction comparison between FlexTok, TiTok, and ALIT

• FlexTok reconstructions are compared with three TiTok models and ALIT across different token counts.The compared TiTok configurations are TiTok L-32, TiTok B-64, and TiTok S-128; the cited works are Yu et al., 2024a and Duggal et al., 2024.Source: Figure 38 · p. 43, Figure 39 · p. 44

K.4. Class-conditional image generation visualizations

• Class-conditional FlexTok generations increase token counts from 1 to 256 in powers of 2 while using a 1.33B-parameter autoregressive Transformer.The visualizations cover ImageNet-1k class conditioning, including golden retriever (207), flamingo (130), and Macaw (88), with varying random seeds.Source: Figure 40 · p. 45, Figure 41 · p. 46, Figure 42 · p. 47

K.5. Text-conditional image generation visualizations

• Text-conditional FlexTok generations increase token counts from 1 to 256 in powers of 2 using DFN-trained tokenizer and 3.06B-parameter autoregressive Transformer models.Rows are conditioned on text embeddings of prompts, with some visualizations varying the random seed.Source: Figure 44 · p. 49, Figure 45 · p. 50, Figure 46 · p. 51
• Text-conditioned examples include a black-bear prompt, a blue Porsche 356 prompt, and different PartiPrompts from Yu et al., 2022.The black-bear and Porsche visualizations use different random seeds, while the PartiPrompt figures vary the prompt.Source: Figure 45 · p. 50, Figure 46 · p. 51, Figure 47 · p. 52