Source-linked AI summary
Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation
Qian Yu, Zongyang Qiu, Juncheng Yan, Hongliang Li, Qi Qin, Keqi Wang, Siqi Luo, Junkui Ju, Yapeng Mi, Tiantian Lu, Tian Liu, Pengyu Xie, Chenhao Ji, Minghui Zhang, Yiang Zhao, Siyu Yan, Haiying Xu, Yanbo Wang, Jianghan Shen, Yuandong Pu, Jiayi Lei, Borui Zhang, Jinxin Shi, Bo Zhang, Yuhang Zang, Shenglong Ye, Lixin Gu, Hongjie Zhang, Linghan Liu, Xinyuan Chen, Yuewen Cao, Lihe Zhang, Xiaohong Liu, Junjun He, Qiangwei Liu, Qi Zhang, Yu Cheng, Tianfan Xue, Hui Xiong, Chang Xu, Yu Qiao, Yihao Liu, Bin Fu
TL;DR
Intern Lumina U2 addresses the challenge of combining high-capacity visual representations with text in a shared optimization space. It models multi-codebook visual tokens in a fully discrete diffusion language model, achieving leading performance among unified models across evaluations and reducing image-generation latency by 4.45× without degrading quality.
Problem
Single-codebook discrete models have limited visual representation capacity, while continuous and hybrid approaches use visual representations and objectives distinct from text modeling.
Source: §1 · p. 3Method
Intern Lumina U2 uses multi-codebook discrete visual states in a shared diffusion backbone for image, video, and 3D inputs, with parallel spatial refinement and autoregressive code prediction within each position.
Source: §1 · p. 3Results
Intern Lumina U2 achieves leading performance among unified models across evaluations and reduces image-generation latency by 4.45× without degrading generation quality.
Source: §1 · p. 4, §8 · p. 33Takeaways & Limitations
The results demonstrate the potential of fully discrete diffusion language modeling to unify broad visual understanding and high-quality generation within a single framework.
Source: §8 · p. 33The multi-codebook autoregressive head adds decoding overhead, high-resolution generation produces long visual sequences, and generation has not yet been extended to video or 3D.
Source: §8 · p. 33
Abstract
from arXiv · showhide
Unified multimodal generation covers discrete, continuous, and hybrid paradigms. Discrete approaches with single-codebook visual tokenizers align naturally with text modeling but have limited representational capacity. Continuous and hybrid approaches retain richer visual representations, but they model text and images in heterogeneous spaces with different training objectives, potentially introducing optimization conflicts between modalities with misaligned representation spaces. These choices expose a central question: how to efficiently model high-capacity visual representations alongside textual content in a shared optimization space within a single backbone. We present Intern Lumina U2, to our knowledge the first diffusion language model to represent multi-codebook visual tokens in a fully discrete space. Each visual position is represented by a group of codes from complementary codebooks, increasing its representational capacity while preserving compatibility with discrete text modeling. At the input, the embeddings are concatenated at each position; at the output, a lightweight multi-codebook autoregressive head predicts the codes within each position sequentially, while the diffusion backbone processes all positions in parallel. Built on the unified 4D representation of AToken, Intern Lumina U2 supports image, video, and 3D inputs within a single backbone and achieves leading performance among unified models in visual understanding, image generation, and image editing. We further build a full-stack training, inference, and evaluation system on both Ascend and NVIDIA platforms to verify the cross-platform effectiveness of our model. We also improve the inference efficiency through a stability-aware sampling strategy, which combines confidence-plateau-guided unmasking approach with drift-triggered cache refreshing mechanisms, accelerating image generation by 4.45x.
1. Introduction
Intern Lumina U2 addresses how to model high-capacity visual representations with text in a shared optimization space while respecting their different structures. It uses multi-codebook discrete visual states in a shared diffusion backbone and reports leading results across visual tasks.
1. Introduction
- Discrete masked diffusion gives text and visual tokens a shared masked-prediction objective, while refining visual positions in parallel with bidirectional context.Existing single-codebook unified models have limited visual representation capacity, motivating a high-capacity discrete framework in one backbone.Source: §1 · p. 3
- Each visual position carries complementary codes, with concatenated codebook embeddings at input and a lightweight autoregressive head predicting codes within each position.The diffusion backbone processes visual positions in parallel, while task-specific masking trains understanding, generation, and editing jointly under one objective.Source: §1 · p. 3
- The system supports training, inference, and evaluation on 1,024 Ascend 910B NPUs, with a smaller-budget NVIDIA reference run using the same architecture and optimization recipe.The reference run is used for cross-platform validation.Source: §1 · p. 3
- Intern Lumina U2 ranks first on six of ten image-understanding benchmarks and improves on eight of ten over same-backbone LLaDA2.0-Uni.The model also reports superior performance on all three video benchmarks and 3D MM-Vet, with leading performance among unified models across its evaluations.Source: §1 · p. 4
- Confidence-plateau commits with cache refresh on hidden-state drift reduce end-to-end image-generation latency by 4.45×.The paper reports a favorable quality–efficiency trade-off for this stability-aware sampling strategy.Source: §1 · p. 4
2. Related Work
Prior unified models use discrete, continuous, or hybrid approaches, with trade-offs in representation capacity, objectives, and decoding. Intern Lumina U2 combines multi-codebook discrete visual states with masked diffusion and parallel spatial refinement.
Unified multimodal understanding and generation
- Table 1 groups representative systems by backbone paradigm and design profile; Intern Lumina U2 uniquely combines image generation, image/video/3D perception, and masked-diffusion decoding over multi-codebook visual states.Its images are decoded through backbone tokens without a separate continuous diffusion or flow generator.Source: §2 · p. 4, Table 1 · p. 6
- Single-codebook discrete models constrain visual detail, while continuous and hybrid designs retain richer image synthesis but use a continuous objective distinct from text prediction.Intern Lumina U2 instead trains text prediction and image generation with multi-codebook discrete representations under a shared objective.Source: §2 · p. 5
Diffusion language models
- Intern Lumina U2 extends LLaDA2.0-based diffusion language modeling with multi-codebook visual states and image, video, and 3D inputs.MMaDA and Lumina-DiMOO also use masked prediction for text and image tokens; LLaDA2.0-Uni uses block-wise masked prediction.Source: §2 · p. 5
Multi-codebook representations and depth decoding
- Unlike prior autoregressive multi-codebook systems, Intern Lumina U2 refines spatial positions in parallel and models dependencies among codes within each position.This factorization keeps backbone sequence length independent of the number of codebooks without treating codes as conditionally independent.Source: §2 · p. 5
Accelerating masked-diffusion decoding
- Training-free masked-generation accelerators reshape commit schedules or cache refreshes; Intern Lumina U2 adapts both using confidence plateaus and measured hidden-state drift.Its approach responds to visual-token confidence being below Fast-dLLM-style absolute thresholds.Source: §2 · p. 6
3. Model Architecture
Intern Lumina U2 represents visual and text inputs as discrete tokens in a packed sequence processed by a masked-diffusion backbone. AToken provides a shared visual representation, while a multi-codebook head predicts visual codes within each position.
3.1. Overview
- The masked-diffusion backbone processes text and visual tokens in one sequence, generating text block by block and predicting visual codes sequentially within positions processed in parallel.AToken encodes visual inputs; the tokenizer decodes predicted visual codes back to pixels.Source: §3.1 · pp. 6–7, Figure 3 · p. 7
3.2. Unified Visual Tokenizer
- AToken maps images, videos, and 3D assets into a shared 4D latent space, using eight 4096-entry codebooks per visual position to retain capacity for fine visual detail.Its discrete tokenizer is trained for reconstruction and understanding, then kept frozen during model training.Source: §3.2 · p. 8
3.3. Backbone
- The LLaDA2.0-mini masked-diffusion backbone retains its text components, adds visual interfaces, and uses a task-aware attention mask for visual conditions and answers.It retains native 1D positional encoding; structural delimiter tokens indicate visual spatial or spatio-temporal layout.Source: §3.3 · p. 8
3.4. Multi-Codebook Input and Output
- Each visual position becomes one backbone token from concatenated codebook embeddings, and a causal head predicts its K codes sequentially while processing positions in parallel.Separate embedding tables avoid expanding the text vocabulary or multiplying sequence length by K.Source: §3.4 · pp. 9–10
3.5. Unified Data Format with Data Packing
- Packed dialogue sequences annotate token roles, turns, noise states, blocks, and positions; task-aware masks distinguish block-diffused text from bidirectional visual answers and expose clean prior answers to later turns.Packed samples cannot attend to one another; visual answers have no clean copy because no later turn conditions on them.Source: §3.5 · pp. 10–12
3.6. Mask Schedule and Training Objective
- Text uses 32-token blocks with independently sampled masks; visual masks share a ratio and average K code losses per position, with both losses normalized together and MoE balancing added.The visual-position loss averages the K per-codebook cross-entropies, so a visual position carries the same weight as a text token.Source: §3.6 · pp. 12–13
4. Data Construction
The training corpus combines six visual and text categories in a common dialogue format, with category-specific construction and stage-specific mixtures. Specialized curation addresses multimodal understanding, text-to-image generation, and image editing.
4. Data Construction
- Six categories span image, video, and 3D understanding, text-to-image generation, image editing, and text-only data, with category-specific sources and quality checks.Samples are converted to a common dialogue format and mixed in stage-specific proportions.Source: §4 · p. 13
4.1. Data Taxonomy and Unified Protocol
- Each sample is represented as conditions and targets under a task type that determines their ordering, letting new sources use an existing sequence format.Understanding samples pair visual conditions with text responses; generation samples target images.Source: §4.1 · pp. 13–14
4.2. Text and Reasoning Data
- Text-only data combines post-training corpora, reasoning datasets, and private instruction data, with deduplication and reweighting by domain and supervision density.Reasoning traces are marked with <think> and </think> before the final text or image.Source: §4.2 · p. 15
4.3. Multimodal Understanding Data
- Image-understanding data are grouped by capability into perception, text-rich image understanding, and knowledge-intensive reasoning, while video and 3D data include captioning and question answering.Video data also cover temporal reasoning; 3D examples pair object-centric assets with captions or question-answering annotations.Source: §4.3 · p. 15
4.4. Text-to-Image Data
- Text-to-image corpus construction addresses concept coverage, caption quality, and image quality through source expansion, prompt reconstruction, and filtering, deduplication, and balancing.Sources include web pairs, curated photographs, and synthetic images; dedicated datasets target identified weak areas.Source: §4.4 · pp. 15–16
4.5. Image Editing Data
- The editing corpus combines public datasets and in-house synthetic data across ten edit types, requiring targets to reflect requested changes while preserving unmentioned content.Quality checks verify visible, instruction-matched changes, consistency in unreferenced regions, and adequate target-image quality.Source: §4.5 · pp. 16–17
5. Training Strategy
Training proceeds through five stages that expand visual resolution, task coverage, and modality scope while retaining a shared objective. The same strategy is implemented on Ascend and NVIDIA platforms with different hardware settings and token budgets.
5.1. Training Stages
- The five-stage curriculum progresses from vision-language alignment and perception to resolution transition, expanded omni-visual training, and instruction fine-tuning.The model begins from LLaDA2.0-mini; visual modules are trained from scratch, AToken remains frozen, and each stage starts from the preceding checkpoint.Source: §5 · p. 17, §5.1 · pp. 17–18
- Expanding modality coverage alongside the 1024 × 1024 resolution increase degraded previously acquired capabilities, so Stage 3 isolates the resolution transition before adding new tasks.Stage 4 then adds image editing, video understanding, and 3D understanding while retaining earlier image and text data in the mixture.Source: §5.1 · pp. 17–18
5.2. Training Configurations
- The Ascend run uses 1,024 Ascend 910B NPUs, while the NVIDIA reference uses 192 GPUs; both process 50B and 200B tokens in Stages 1 and 2, respectively.Later token budgets differ across platforms: Ascend uses 500B for Stages 3–4 and 300B for SFT, versus 200B for each on NVIDIA.Source: §5.2 · p. 18, Table 3 · p. 19
5.3. Optimization and Parallel Training
- Muon optimizes trainable matrix weights and AdamW optimizes embeddings, one-dimensional parameters, and output heads; dynamic packing reduces inter-sample padding while preserving each sample’s objective.Both platforms use FSDP2 with full parameter sharding in an HSDP layout, and follow the same five-stage schedule and unified objective despite hardware-specific settings.Source: §5.3 · pp. 18–20
6. Inference
Text is generated in left-to-right blocks with parallel denoising within each block, while images are refined across all spatial positions in parallel and decoded through sequential codebooks. The inference system adds cache reuse and PHV sampling; its selected configuration reduces image-generation latency while improving GenEval under the reported protocol.
6.1. Inference Procedure
- Text decoding extends the completed prefix one block at a time, denoises positions within each block in parallel, and stops at <EOA>, reusing the completed prefix’s key-value cache.Within a block, unmasked tokens remain visible as bidirectional context for still-masked positions.Source: §6.1 · p. 20
- Image inference refines all masked canvas positions in parallel, samples the K = 8 codes sequentially within each position, and remasks positions according to a cosine schedule.The multi-codebook head preserves within-position code dependencies while unresolved positions can be revised during generation.Source: §6.1 · p. 21
6.2.1. Batching and Cache-Aware Reuse
- Inference acceleration shares conditional and unconditional CFG computation and reuses AR-head and backbone states when their inputs remain unchanged; a normalized drift test triggers cache refreshes.Warm-up and fixed-interval refreshes remain in effect regardless of the drift threshold.Source: §6.2.1 · p. 22
6.2.2. Plateau-Halton Sampling with V-verify (PHV)
- PHV treats confidence as a temporal signal: positions become plateau-eligible after confidence changes remain small for consecutive steps and their EMA exceeds a floor, then uses Halton ordering for remaining commits.The offline matched-control test reports lower flip rates for plateau tokens; at H = 8, 157 of 200 prompts show steadier plateau tokens.Source: §6.2.2 · pp. 22–24, Figure 9 · p. 25
- 4.45× lower latency, from 159.9 to 35.9 seconds per image, accompanies a reduction from 96 to 60.4 refinement steps and a GenEval increase from 86.6 to 89.3.The selected PHV setting is evaluated under the matched 1024 × 1024 text-to-image protocol; its GenEval result is not comparable to the four-seed protocol of Table 7.Source: Table 4 · p. 23, §6.2.2 · p. 25
7. Evaluation
Evaluation covers image, video, and 3D understanding, image generation and editing, and scientific-diagram generation, alongside interface, output-head, and sampling ablations. Intern Lumina U2 performs strongly across these tasks, while the ablations identify design choices associated with accuracy, efficiency, and generation quality.
Evaluation setup
- Evaluation compares an Ascend-trained primary model with an NVIDIA reference model sharing its architecture, tokenizer, and five-stage schedule, using official benchmark protocols.The reference model has a smaller training budget; the reported ablations cover visual input, the multi-codebook autoregressive head, and inference sampling.Source: §7 · p. 26
Image understanding and reasoning
- Intern Lumina U2 achieves the best score on six of ten image-understanding benchmarks and outperforms LLaDA2.0-Uni on eight of ten.It ties BAGEL on MME-C; the clearest leads and largest gains over LLaDA2.0-Uni occur on benchmarks requiring fine-grained visual detail.Source: Table 5 · p. 26, §7 · p. 27
Video and 3D understanding
- Intern Lumina U2 leads all three video-understanding benchmarks and outperforms 3D-specialist models on 3D MM-Vet.The same checkpoint supports video and 3D understanding alongside image generation, a combination absent from the other unified baselines described.Source: §7 · p. 27, Table 6 · p. 27
Image generation and editing
- Among unified models, Intern Lumina U2 leads on DPG-Bench, UniGenBench, and TIIF-Bench, matches the best GenEval score, and is close to the best on two editing benchmarks.It ranks second on WISE without reasoning-augmented prompting; on OneIG-Bench it has the highest style score but lower sample diversity than baselines.Source: §7 · p. 27, Table 7 · p. 28, Table 8 · p. 28
Scientific-diagram generation
- Intern Lumina U2 obtains the highest relaxed GenExam score among general-purpose unified models and performs strongly across VeriphyT2IBench domains.It ranks first in Local E&M and Global mechanics; its largest VeriphyT2IBench margin is on hard E&M prompts, where it scores 81.30.Source: §7 · pp. 27–28, Table 11 · p. 30, Table 12 · p. 30
Input-interface ablation
- D3, with separate codebook embedding tables followed by concatenation and projection, has the highest next-code accuracy and is preferred for accuracy and efficiency.It exceeds D1 and D2 by more than ten standard errors, with widening gaps on later codebooks; D1 uses 2.35× D3’s memory, while D3 matches D2’s memory with 1.9% more interface parameters.Source: §7 · pp. 28–29, Table 15 · p. 31, Figure 10 · p. 32
Output-head ablation
- The multi-codebook autoregressive head achieves the best results on all four generation benchmarks, while independent parallel prediction loses 14 GenEval points.The reported advantage is attributed to modeling inter-code dependencies; the parallel head’s largest drops involve counting, position, and attribute binding.Source: §7 · pp. 30–32, Table 16 · p. 32
Inference-sampling ablation
- The selected sampling setting, δp=0.18 and fp=0.005, reaches GenEval 89.3 at 35.9 seconds per image, a 4.45× speedup.The two plateau controls act as a serial gate: δp determines eligibility for early commitment, while fp limits how many eligible positions commit.Source: §7 · p. 32, Figure 11 · p. 33
8. Conclusion
Intern Lumina U2 combines fully discrete multi-codebook visual states with a shared diffusion-language-model backbone for understanding and generation across images, videos, and 3D assets. Its reported evaluations and ablations support the design, while decoding overhead and the absence of video and 3D generation remain limitations.
Unified design
- Intern Lumina U2 models multi-codebook visual states in a fully discrete space and extends visual understanding across images, videos, and 3D assets.Its input interface uses independent codebook embeddings combined by concatenation and projection; its output head predicts codes autoregressively within each position while diffusion refines positions in parallel.Source: §8 · p. 33
Ablation findings
- Ablations support both interface choices: codebook-specific input embeddings improve next-code accuracy, and the multi-codebook autoregressive head performs best on all four generation benchmarks.The reported head gains are most pronounced in counting, position, and attribute binding.Source: §8 · p. 33
Evaluation and system
- The system reports results across omni-visual understanding, image generation and editing, and physically grounded scientific-diagram generation, with a 4.45× image-generation latency reduction without degraded quality.The main model was trained on 1,024 Huawei Ascend 910B NPUs; an independently trained NVIDIA reference model assesses cross-platform consistency.Source: §8 · p. 33
Limitations
- The autoregressive output head adds decoding overhead, high-resolution generation creates long visual sequences, and the current system does not support video or 3D generation.The authors identify extending generation to videos and 3D assets as future work.Source: §8 · p. 33
A. Special Tokens and Sequence Format Examples
The appendix specifies the model configuration and special-token inventory, then illustrates how text, conditions, and answers are packed and supervised. Task examples reuse this sequence format with modality-specific content.
A.1. Full model configuration
- Table 17 separates backbone settings from the visual interface modules added in this work.It lists the complete checkpoint configuration.Source: §A.1 · p. 42, Table 17 · p. 42
A.2. Special tokens
- The text vocabulary contains 157,184 tokens, while each of the K visual embedding tables has 5,010 entries, including codes and visual mask and delimiter tokens.The 4,096 code entries come first, followed by tokens marking image rows, video rows or frames, and 3D rows or layers.Source: §A.2 · p. 42
- Packed sequences use an [L, K] tensor with one column per codebook; text and structural tokens are written into all columns and read from the first.Visual-related token IDs index per-codebook embedding tables; other IDs index the text vocabulary.Source: §A.2 · p. 42, Table 18 · p. 43
A.3. A complete packed sequence
- The image-understanding example combines a clean user condition and answer in one turn, with token types distinguishing text conditions, media conditions, text answers, and visual answers.The illustrated 2 × 2 visual grid is a readable substitute for the 16 × 16 grid of an actual 256 × 256 image.Source: §A.3 · p. 42, Table 19 · p. 43
- Only masked answer positions contribute to loss; the clean answer copy is unpadded and reuses the noisy answer’s position IDs.Before masking, the answer is padded with <EOA> tokens to block length B=32.Source: §A.3 · p. 42, Table 19 · p. 43
- Text-to-image samples place the prompt before the target image and may replace prompt tokens with <uncondition>; they add no clean copy.The prompt has type 1, grid positions type 4, and all answer tokens belong to block 0.Source: §A.3 · p. 44
- Editing samples pair a type-2 source image and type-1 instruction with a target image, while plain-text samples contain only an answer and its clean copy.These task samples retain the same general format while changing the content of their turns.Source: §A.3 · p. 44
B. Visual Token Layouts and Positions
Visual modalities are flattened into one-dimensional sequences for the backbone, with modality-specific layouts and delimiters. The appendix gives image sequence lengths and the position and encoding conventions used across modalities.
Position encoding
- AToken applies its own 4D rotary position embedding, while the backbone receives a 1D sequence and applies partial RoPE to sequential position IDs.Position IDs increase across conditions and noisy answers in dialogue order; clean copies reuse the IDs of their noisy counterparts.Source: B. Visual Token Layouts and Positions · p. 44
Image layout
- A 256 × 256 image occupies 272 positions, a 512 × 512 image 1,056 positions, and a 1024 × 1024 image 4,160 positions, excluding boundary tokens.A row delimiter follows each row of visual tokens.Source: B. Visual Token Layouts and Positions · p. 44
Video and 3D layouts
- For video and 3D assets, only non-empty AToken positions are included; row delimiters mark non-empty rows, followed by frame or layer delimiters.At each frame or layer’s end, the row delimiter precedes the frame or layer delimiter.Source: B. Visual Token Layouts and Positions · p. 44, Table 20 · p. 44
C. Data Construction, Sources, and Audit Details
The section describes how training data are sourced, encoded, validated, and filtered before use. It also explains how mixture weights preserve representation for underrepresented or harder capabilities.
C. Data Construction, Sources, and Audit Details
- Table 21 organizes public and private data sources by category, encoding resolution, and training stage.Image and editing resolution is specified by long side, while video resolution uses frame-area budget; “native” resolution is clamped to [256, 1024].Source: C. Data Construction, Sources, and… · p. 44, Table 21 · p. 45
- Visual inputs use frozen AToken encoding, with distinct image resizing, 64-segment video sampling, and sparse-voxel encoding for 3D assets.Video frames are resized to a 448^2-pixel area budget; 3D assets are voxelized at 48^3, and empty assets are discarded.Source: C. Data Construction, Sources, and… · pp. 44–45
- Before training, data mixtures are checked for source and sample counts, parseable nonempty samples, matching recomputed sequence lengths, and successful packing.Unchanged sources are reused after a first-sample consistency check; changed or new sources are processed again.Source: C. Data Construction, Sources, and… · p. 46
- Text-to-image filtering removes images with illegible required text or prompt-following scores below 60, then keeps the global top 70% by aesthetic score.The aesthetic threshold is about 74; Table 22 reports removal counts for two curated photograph sources.Source: C. Data Construction, Sources, and… · p. 46, Table 22 · p. 46
- Training mixtures assign different sampling weights across stages, giving underrepresented or harder capabilities higher weights so larger sources do not reduce their share.Source: C. Data Construction, Sources, and… · p. 46
D. Ascend Adaptation Details
The Ascend adaptation changes execution, kernels, attention handling, and memory settings to accommodate differences from the GPU stack. Its inference path uses the stacked expert weights directly.
D. Ascend Adaptation Details
- The execution path remaps torch.cuda calls to the NPU runtime, isolates module caches by rank, and rebuilds the decoder’s attention modules on native kernels.The rebuilt attention modules serve image decoding; understanding and text paths do not reach that code.Source: D. Ascend Adaptation Details · pp. 46–47
- The Ascend MoE path uses grouped matrix multiplications and an explicit backward pass, with an expert-parallel variant using all-to-all dispatch and combine.Source: D. Ascend Adaptation Details · p. 47
- Packed training uses dense attention because the variable-length Ascend interface ignores attention masks and cannot express each packed segment’s task-aware pattern.The implementation also explicitly converts mask polarity because the kernel treats True as masked.Source: D. Ascend Adaptation Details · p. 47
- Per-NPU memory is the training configuration’s binding constraint, so checkpointing, sharding, and cache-release settings lower peak memory and increase the longest sequence each NPU can hold.The data-parallel shard group is 24 or 32, and cache release occurs every 20 rather than 100 steps.Source: D. Ascend Adaptation Details · p. 47
- Ascend inference loads stacked expert weights directly, whereas the GPU path splits them into per-expert weights, requiring a conversion pass and a second parameter copy.Source: D. Ascend Adaptation Details · p. 47
E. Additional Examples
The additional figures provide qualitative examples across generation, omni-visual understanding, and editing. Figure 13 specifically presents image-editing examples.
E. Additional Examples
- Figures 12 and 13 provide qualitative examples for generation, omni-visual understanding, and editing.Source: E. Additional Examples · p. 47
- Figure 12 presents qualitative examples for text-to-image generation and omni-visual understanding.Source: Figure 12 · p. 48
- Figure 13 presents qualitative examples for image editing.Source: Figure 13 · p. 49
Related papers
This paper cites
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models2023
- RoFormer: Enhanced Transformer with Rotary Position Embedding2021
- Classifier-Free Diffusion Guidance2022
- DeepSeek-V3 Technical Report2024
- Sigmoid Loss for Language Image Pre-Training2023
- GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment2023