PaperlayerView exact revision on arXiv

Source-linked AI summary

One-D-Piece: Image Tokenizer Meets Quality-Controllable Compression

Keita Miwa, Kento Sasaki, Hidehisa Arai, Tsubasa Takahashi, Yu Yamaguchi

arXiv:2501.10064v1cs.CVcs.LG

TL;DR

Fixed-length image tokenizers cannot adjust token allocation to image information or control reconstruction quality, despite the compression-quality trade-off. One-D-Piece uses Tail Token Drop to enable variable-length tokenization, with experiments reporting improved perceptual quality over image formats and performance across downstream tasks. The evaluation is limited to a maximum of 256 tokens.

  • Problem

    Most image tokenizers produce fixed-length sequences, limiting flexibility despite varying image information and a trade-off between compression rate and reconstruction quality.

    Source: §1 · p. 2, §2.1 · p. 3

  • Method

    One-D-Piece applies Tail Token Drop to 1D token sequences, encouraging important information to concentrate at the head so token length can be adjusted.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    One-D-Piece achieves an rFID of 1.08 with 256 tokens and excels in image classification, object detection, and semantic segmentation.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The results indicate that One-D-Piece is an efficient, adaptive tokenizer with potential for a range of computer vision applications.

    Source: §5 · p. 8

    Evaluation was limited to a maximum of 256 tokens; the effects of higher token counts on reconstruction quality and pixel-level performance remain unexplored.

    Source: §5 · p. 8

Abstract

from arXiv · show

Current image tokenization methods require a large number of tokens to capture the information contained within images. Although the amount of information varies across images, most image tokenizers only support fixed-length tokenization, leading to inefficiency in token allocation. In this study, we introduce One-D-Piece, a discrete image tokenizer designed for variable-length tokenization, achieving quality-controllable mechanism. To enable variable compression rate, we introduce a simple but effective regularization mechanism named "Tail Token Drop" into discrete one-dimensional image tokenizers. This method encourages critical information to concentrate at the head of the token sequence, enabling support of variadic tokenization, while preserving state-of-the-art reconstruction quality. We evaluate our tokenizer across multiple reconstruction quality metrics and find that it delivers significantly better perceptual quality than existing quality-controllable compression methods, including JPEG and WebP, at smaller byte sizes. Furthermore, we assess our tokenizer on various downstream computer vision tasks, including image classification, object detection, semantic segmentation, and depth estimation, confirming its adaptability to numerous applications compared to other variable-rate methods. Our approach demonstrates the versatility of variable-length discrete image tokenization, establishing a new paradigm in both compression efficiency and reconstruction performance. Finally, we validate the effectiveness of tail token drop via detailed analysis of tokenizers.

1. Introduction

Discrete tokenizers can integrate with Transformer-based models, but fixed-length sequences limit flexibility when compression and reconstruction quality must be balanced. One-D-Piece introduces variable-length tokenization through Tail Token Drop and evaluates reconstruction quality and downstream task performance.

1. Introduction

  • Tail Token Drop concentrates critical information at the sequence head, enabling adaptive token lengths and quality control.The approach supports token lengths from 1 to 256, with visually accurate reconstructions reported even at 8 or 16 tokens.Source: §1 · p. 2
  • At 384 bytes, One-D-Piece-L-256 achieves superior visual quality and compression efficiency to image formats.One-D-Piece also shows improved reconstruction quality as token length increases, including at 16, 32, 64, and 128 tokens.Source: Figure 2 · p. 2
  • One-D-Piece outperforms existing variable-length compression methods, including JPEG, JPEG 2000, and WebP, in perceptual quality, especially at low token counts.The evaluation also assesses image classification, object detection, and semantic segmentation.Source: §1 · p. 2
  • The paper introduces One-D-Piece as a variable-length discrete image tokenizer and reports competitive reconstruction quality across flexible token lengths.Its contributions include detailed analysis of how Tail Token Drop concentrates important information at the sequence head.Source: §1 · pp. 2–3

2. Related Work

Related work covers discrete image tokenizers and variable-rate compression. It describes the limits of fixed token lengths and the difficulty of adapting tail-drop regularization to spatially structured token sequences.

2. Related Work

  • Discrete tokenization represents images as sequences used in compression, image and video generation, and vision-language models [4] [5] [9] [36] [12] [35].AutoEncoder-based methods such as VAE [17] extract information into lower-dimensional latent spaces; VQ-VAE [30] is one basis for discrete tokenization.Source: §2.1 · p. 3

2.1. Image Tokenization

  • CNN-based 2D tokenizers capture local information effectively but struggle to integrate global information, often resulting in low compression efficiency; typical methods consume 256 tokens for 256×256 images [5] [9] [24] [30] [34] [36] [40].SEED-Tokenizer [11] uses Vision Transformer (ViT) [8] to reduce an image to a sequence of semantic tokens.Source: §2.1 · p. 3

2.2. Variable-Length Coding

  • High compression creates a reconstruction-quality trade-off, especially for complex images, and existing tokenizers do not allow quality adjustment [3].JPEG, JPEG 2000, and WebP provide variable-rate compression; neural Tail Drop [18] concentrates information toward the embedding head, but spatial dependencies complicate its use with 2D tokens.Source: §2.1 · p. 3, §2.2 · p. 3

3. Method

One-D-Piece is a discrete tokenizer designed to produce variable-length image token sequences. Its method adapts tail-drop regularization to 1D tokens and uses a decoder and training procedure that support variable token counts.

3. Method

  • One-D-Piece produces variable-length image token sequences rather than the fixed-length sequences typical of existing tokenizers.The tokenizer is designed for efficient image compression.Source: §3 · p. 3

3.1. Tail Token Drop

  • Tail Token Drop randomly truncates the tail of a 1D token sequence during training, encouraging essential information to accumulate at the head.The token count can then be adjusted at inference by cutting the tail according to image information content.Source: §3.1 · p. 4

3.2. Architecture

  • The architecture requires 1D tokens without explicit 2D structure and a detokenizer that can handle variable-length sequences.One-D-Piece builds on TiTok [41]: its encoder produces latent tokens, a quantizer discretizes them, and a decoder processes the tokens alongside mask tokens to reconstruct the image.Source: §3.2 · p. 4

3.3. Training

  • Training follows TiTok’s two-stage strategy, applying Tail Token Drop in the reconstruction stage and uniformly sampling a token cutoff from 1 to 256 for each batch.Evaluation and training use ImageNet-1K.Source: §3.3 · p. 4

4. Experiments

Experiments compare One-D-Piece with image tokenizers and compression formats on reconstruction quality, variable token lengths, and downstream vision tasks. They also analyze how information is distributed across tokens and how that relates to image content.

4.1. Reconstruction

  • At 256 tokens, B-256 and L-256 achieve the lowest rFID scores, 1.11 and 1.08, while traditional formats perform worse on rFID but better on PSNR.The authors attribute the PSNR difference to GAN and perceptual losses that prioritize perceptual quality over pixel-level distortion, consistent with the rate-distortion-perception trade-off [3].Source: §4.1 · p. 5, Table 1 · p. 5
  • rFID and other reconstruction metrics improve as token length increases; One-D-Piece is more efficient than algorithm-based formats, but TiTok leads at 32, 64, and 128 tokens for equivalent model sizes.The authors identify perceptual quality at smaller token sizes as a remaining challenge.Source: §4.1 · p. 5, Figure 4 · p. 6, Figure 5 · p. 6

4.2.1 Task Settings

  • The downstream evaluation uses classification, detection, segmentation, depth estimation, and CLIP embedding tasks, with task-specific datasets, models, and reference predictions.Image classification and segmentation use dataset labels; detection, depth, and CLIP embedding reconstruction use model predictions as references.Source: §4.2 · p. 6, §4.2.1 · p. 6, Table 3 · p. 7

4.2.2 Results

  • At 128 tokens—about 10% of WebP’s byte size—One-D-Piece outperforms WebP across tasks and also beats JPEG across tasks with 16 tokens.Table 3 covers classification, detection, segmentation, depth estimation, and CLIP embedding reconstruction.Source: §4.2.2 · pp. 6–7, Table 3 · p. 7

4.3. Analysis

  • Random token replacement and reconstruction-error measurements show a strong contribution peak at the sequence head; Figure 6 also depicts early tokens as global and later tokens as more localized.The analysis uses ImageNet validation data and pixel-wise L1 error to measure token contributions and their spatial effects.Source: §4.3 · p. 7, Figure 6 · p. 8
  • Some middle and later tokens correspond to specific image regions, while the final tokens contribute almost nothing, leaving part of the latent space underused.Packing more information into less-important tokens is left for future research.Source: §4.3 · p. 7, Figure 6 · p. 8
  • Clustering by the first token indicates that it captures overall image similarity, and replacing it can substantially change reconstructions, especially for shorter sequences.Source: §4.3 · p. 7
  • One-D-Piece achieves higher linear-probing accuracy than pretrained TiTok models, which the authors attribute to Tail Token Drop emulating a compact latent space.They interpret this result as further evidence of global information aggregation.Source: Table 2 · p. 6, §4.3 · p. 8

5. Conclusion

One-D-Piece supports variable-length image tokenization through Tail Token Drop, and the reported results span reconstruction quality and downstream tasks. Its evaluation used at most 256 tokens.

5. Conclusion

  • The first token corresponds to global image information, especially the background, and changing it changes the background, particularly in reconstructions using fewer tokens.Source: Figure 7 · p. 8
  • Tail Token Drop supports dynamic token counts from 1 to 256 by concentrating critical information at the sequence start, enabling high reconstruction quality with fewer tokens.The conclusion reports significant perceptual-quality improvements over JPEG at comparable byte sizes.Source: §5 · p. 8
  • At 256 tokens, One-D-Piece achieves an ImageNet-1K rFID of 1.08, surpasses prior methods, and excels at image classification, object detection, and semantic segmentation.The conclusion also reports that analysis verified Tail Token Drop concentrates important information at the sequence head.Source: §5 · p. 8
  • Evaluation was limited to 256 tokens; effects of higher token counts on reconstruction quality and pixel-level performance remain unexplored.Source: §5 · p. 8

A. Training Details

Experiments followed TiTok settings for model architecture and training, using a specified base implementation and commit.

A. Training Details

B.1. Reconstruction

The appendix identifies sample reconstructions and reports where detailed rFID values and image-compression comparisons are documented.

B.1. Reconstruction

  • Sample reconstructions for L-256, B-256, and S-256 appear in Figures 8, 9, and 10, respectively.Source: §B.1 · p. 11
  • Table 4 provides detailed rFID values corresponding to Figure 4; JPEG, JPEG 2000, and WebP files were generated using OpenCV 4.10.0.84.Source: §B.1 · p. 11
  • Table 4 compares image quality against compression quality for JPEG, JPEG 2000, and WebP; BpI means Byte-per-Image.Source: Table 4 · p. 11

B.2. Downstream Tasks

Downstream-task results vary by task: semantic tasks can surpass WebP with fewer tokens, while depth estimation requires 128 tokens to do so.

B.2. Downstream Tasks

  • Semantic segmentation and CLIP embedding reconstruction surpass WebP with fewer tokens for One-D-Piece-S-256 and B-256.Source: §B.2 · p. 11
  • Object detection and image classification tend to show better results with a moderate number of tokens.Source: §B.2 · p. 11
  • Both One-D-Piece-S-256 and B-256 require 128 tokens to outperform WebP on depth estimation.Source: §B.2 · p. 11

B.3. Generation

One-D-Piece is evaluated for class-conditioned generation and inference speed. Generation quality is competitive with TiTok variants, while Tail Token Drop reduces detokenization work but not tokenization complexity.

B.3. Generation

  • One-D-Piece models show competitive generation performance against TiTok variants, and Tail Token Drop does not damage generation quality.The evaluation uses MaskGIT and gFID; TiTok-S-128's Inception Score is unreported because its pretrained MaskGIT model was not released.Source: §B.3 · p. 11, Table 5 · p. 11

B.4. Inference Speed

  • Tail Token Drop leaves tokenization time complexity unchanged, although processing fewer tokens can improve detokenization speed.Tokenization still generates the maximum number of tokens, so its computational cost does not differ from TiTok's architecture.Source: §B.4 · p. 11
  • Table 6 compares downstream-task evaluations across token lengths and marks where One-D-Piece-S-256 or One-D-Piece-B-256 surpasses WebP.The caption does not provide the underlying metric values.Source: Table 6 · p. 12

C. Analysis Details

The analysis visualizes how token contributions are distributed across One-D-Piece and TiTok variants, while the supplementary material documents licenses, model settings, and task outputs.

D. Licenses

  • ImageNet-1K is used for non-commercial research and education, ImageNet-S follows ImageNet licensing, and COCO use follows COCO and Flickr terms.The cited descriptions specify these dataset usage terms.Source: D. Licenses · p. 12
  • Ultralytics YOLO11x is used for object detection, and YOLO11 is released under AGPL-3.0, which permits research use.Source: D. Licenses · p. 12
  • The One-D-Piece and MaskGIT hyperparameters follow TiTok settings.Source: Table 7 · p. 13, Table 8 · p. 13
  • Depth-estimation and semantic-segmentation results on reconstructed images approach results on originals as token count increases.Source: Figure 11 · p. 17

C. Analysis Details

  • One-D-Piece concentrates global information near the sequence head, unlike TiTok, but normalized log-scaled colors are not comparable across models.Each grid is normalized using its own global minimum and maximum.Source: C. Analysis Details · p. 12, Figure 13 · p. 18

Related papers

Loading 2501.10064v1…