Source-linked AI summary
Thinking with Generated Images
Ethan Chern, Zhulin Hu, Steffi Chern, Siqi Kou, Jiadi Su, Yan Ma, Zhijie Deng, Pengfei Liu
TL;DR
The paper addresses the limits of visual reasoning systems that either process fixed user images or reason through text-only chains of thought. It introduces Thinking with Generated Images through a native long-multimodal thought process, and reports improved vision-generation performance, including gains from self-critique. The approach is demonstrated on Anole-7b, while broader benchmarking and other architectures remain future work.
Problem
Existing LMMs are limited by fixed user-provided images or text-only reasoning, restricting intermediate visual thinking for tasks that require visual representations.
Method
The paper implements Thinking with Generated Images using a native long-multimodal thought process that generates interleaved text and vision tokens, including visual subgoals and self-critique.
Results
The self-critique model improves GenEval from 0.45 to 0.48 and DPG-Bench from 62.83 to 67.14 after revising its initial visual hypothesis.
Takeaways & Limitations
Generated visual thoughts let a single model construct, inspect, and refine visual intermediate steps within its reasoning process.
Takeaways & Limitations
The approach is demonstrated on Anole-7b and autoregressive next-token-prediction LMMs, while diffusion-based and mixed architectures are left for future work.
Abstract
from arXiv · showhide
We present Thinking with Generated Images, a novel paradigm that fundamentally transforms how large multimodal models (LMMs) engage with visual reasoning by enabling them to natively think across text and vision modalities through spontaneous generation of intermediate visual thinking steps. Current visual reasoning with LMMs is constrained to either processing fixed user-provided images or reasoning solely through text-based chain-of-thought (CoT). Thinking with Generated Images unlocks a new dimension of cognitive capability where models can actively construct intermediate visual thoughts, critique their own visual hypotheses, and refine them as integral components of their reasoning process. We demonstrate the effectiveness of our approach through two complementary mechanisms: (1) vision generation with intermediate visual subgoals, where models decompose complex visual tasks into manageable components that are generated and integrated progressively, and (2) vision generation with self-critique, where models generate an initial visual hypothesis, analyze its shortcomings through textual reasoning, and produce refined outputs based on their own critiques. Our experiments on vision generation benchmarks show substantial improvements over baseline approaches, with our models achieving up to 50% (from 38% to 57%) relative improvement in handling complex multi-object scenarios. From biochemists exploring novel protein structures, and architects iterating on spatial designs, to forensic analysts reconstructing crime scenes, and basketball players envisioning strategic plays, our approach enables AI models to engage in the kind of visual imagination and iterative refinement that characterizes human creative, analytical, and strategic thinking. We release our open-source suite at https://github.com/GAIR-NLP/thinking-with-generated-images.
1. Introduction
The paper argues that text-only reasoning and fixed-image processing limit multimodal cognition because visual intermediate thoughts can support alternative exploration, decomposition, critique, and refinement. It proposes Thinking with Generated Images, where models proactively generate visual steps within a unified multimodal thought process.
- Motivation: Human cognition uses visual thinking to explore possibilities and connections that text reasoning alone may not discover.Examples include protein-structure exploration, crime-scene reconstruction, and architectural revision.
- Motivation: Text-only AI reasoning cannot freely explore visual alternatives, verify visual evidence, or revise design configurations in place.The paper frames these capacities as part of human creativity and analysis.
- Limitations of Existing Approaches: Standard LMMs process images as fixed inputs, while prior image-thinking methods mainly revisit or transform those inputs through limited operations.These approaches expand access to user-provided images but constrain the model’s thinking space.
- Proposal: Thinking with Generated Images lets models spontaneously produce their own visual steps or subgoals instead of relying entirely on user-supplied images.The proposed paradigm is intended to support generalized reasoning across text and vision modalities.
- Approach: The paper introduces a native long-multimodal thought process that interleaves multimodal tokens within a single model’s reasoning.The approach is presented as an end-to-end implementation of Thinking with Generated Images on unified autoregressive LMMs.
2. Thinking with Generated Images
Thinking with Generated Images extends visual reasoning from fixed or transformed inputs to dynamically generated visual intermediate states. The paper implements this through a native long-multimodal thought process that interleaves modality tokens and supports test-time scaling.
- Unified Autoregressive LMMs: The approach uses unified autoregressive LMMs to generate multimodal thought tokens and naturally perform test-time scaling.The paper presents this as a single-model alternative to fragmented multi-component systems.
- From Seeing to Thinking: Standard LMMs see an image once, whereas image-thinking pipelines revisit or transform fixed user inputs through multi-step systems or external tools.These methods support spatial puzzles, visual question answering, and chart or diagram interpretation, but remain tied to supplied images.
- Thinking with Generated Images: Thinking with Generated Images enables models to dynamically create and manipulate visual representations during reasoning, supporting domains where visual intermediate steps are essential.The paper identifies 3D modeling, visual foresight, design, and intuitive physics as examples.
- Native Long-Multimodal Thought Process: The native long-multimodal thought process is a sequence of interleaved multimodal tokens drawn from a unified vocabulary across modalities.The formulation permits discrete or continuous modality-specific token spaces.
- Native Long-Multimodal Thought Process: Vision tokens may appear in fixed-size blocks, with delimiter tokens marking boundaries and transitions between modality spaces.One example uses exactly 1024 consecutive tokens for each image representation.
3. Experiments
The experiments use Anole as the base unified autoregressive LMM and train it with curated multimodal reasoning data. The data elicits both intermediate visual subgoals and textual self-critique of generated visual steps.
- Base Model: Anole is used as the base model for generating native long-multimodal thought processes for test-time scaling.It directly predicts the next multimodal text or image token and supports interleaved generation.
- Base Model: Anole’s 1024-token image representation makes native long-multimodal test-time scaling viable within a reasonable inference compute budget.Its interleaved text-image pretraining and LLM-like modeling strategy provide additional implementation advantages.
- Data Curation: The supervised fine-tuning dataset contains diverse vision-generation prompts curated for high-quality alignment.The training data is designed to elicit spontaneous visual reasoning behaviors.
- Data Curation: The multimodal reasoning chains train models to critique their own generated visual steps and generate intermediate visual subgoals.These are the two targeted mechanisms for native long-multimodal thought.
3 Experiments
The experiments instantiate Thinking with Generated Images through intermediate visual subgoals and self-critique, supported by synthetic data construction and multimodal image-refinement components. Evaluations on GenEval and DPG-Bench test complex visual generation, while ablations examine reconstruction loss.
- 3 Experiments: Two native long-multimodal thought processes generate intermediate visual subgoals or critique and refine visual hypotheses.Subgoal reasoning decomposes multi-object prompts; self-critique evaluates an initial image and produces a refined version.
- 3 Experiments: The synthetic-data pipeline constructs long-multimodal reasoning examples for training models to generate visual subgoals and self-critique visual hypotheses.It uses complex prompts, reflection-reasoning chains, and generated or refined intermediate visual thoughts.
- 3 Experiments: The image-generation process creates first-round images and then refines them using prompts, images, and critique feedback through multimodal inputs.Anole-7b or Flux1-dev generates initial images, while Flux1-Redux performs refinement.
- 3 Experiments: A visual reconstruction loss is combined with cross-entropy, with reconstruction-loss weight 1 yielding optimal GenEval performance in ablations.The loss projects generated-image hidden states into visual feature space and compares them with ground-truth features using MSE.
- 3 Experiments: The models are evaluated on GenEval and DPGBench, including complex prompts requiring at least two objects and comparisons of initial versus self-criticized outputs.TwGI-Anole-7b-Obj. is compared with Anole-7b, while TwGI-Anole-7b-Crit. is evaluated before and after critique.
- 3 Experiments: TwGI-Anole-7b-Obj. consistently outperforms Anole-7b across GenEval and DPGBench, including a Two Obj. score of 0.57 versus 0.38 on GenEval.Reported gains also cover position, color, entity, attribute, and relation categories.
4. Conclusion
The paper concludes that intermediate visual subgoals and self-critique enable multimodal reasoning through generated images. It implements this paradigm with a native long-multimodal thought process and identifies extensions beyond text and vision.
- 4. Conclusion: Breaking visual tasks into intermediate subgoals enables more systematic reasoning and higher-quality generated outputs.The conclusion links this result to the reported benefits of generating intermediate visual thoughts.
- 4. Conclusion: Self-critique improves visual generation, raising overall GenEval from 0.45 to 0.48 and DPG-Bench from 62.83 to 67.14.The model analyzes generated images through textual reasoning and revises them using visual feedback.
- 4. Conclusion: The paper implements Thinking with Generated Images through a native long-multimodal thought process on autoregressive next-token-prediction LMMs.The authors state that the core idea can extend beyond text and vision to diverse modalities.
5 Limitations and Future Directions
The paper identifies limits in current unified LMM development, benchmarking, and visual representation, and outlines stronger architectures, realistic evaluations, and efficient representations as future directions.
- The approach is demonstrated on Anole-7b, while broader capabilities are anticipated from stronger unified LMMs.
- Current vision-generation benchmarks emphasize standard image-generation tasks, motivating more realistic evaluations of emerging real-world capabilities.
- Test-time and post-training scaling on unified LMMs remain early research directions that may become more viable as models improve.
- Efficient visual representations are presented as essential for scalable test-time and post-training scaling in the vision modality.
A.1 Training Loss
The training objective combines unified multimodal language modeling with visual reconstruction, operating over interleaved text and image sequences and adding feature-level supervision for image generation.
- The model uses a unified vocabulary in which image codebook entries and text tokens share one multimodal prediction space.The vocabulary includes 8,192 visual codebook entries, while remaining indices represent text tokens.
- The autoregressive multimodal loss applies cross-entropy across valid text and image token positions, excluding padding and masked labels.
- The visual reconstruction loss compares projected hidden states with original quantized codebook features using mean squared error.
- The total objective sums multimodal language modeling loss and weighted visual reconstruction loss: Ltotal = Lmm + λLrec.When a batch contains no images, Lrec = 0.
- The loss ablation fine-tunes Anole-7b on 50,000 JourneyDB images and evaluates different λ values against a token discrepancy loss on GenEval.Classifier-free guidance was not applied for the comparison.
A.2 Training Stages Details
Training uses a shared stage-1 model followed by separate fine-tuning for intermediate visual subgoals and self-critique, with classifier-free guidance supporting multimodal conditioning.
- Stage-1 training uses approximately 4M text-image pairs, followed by about 5k subgoal samples and 40k self-critique samples.
- The two vision-generation mechanisms share the same stage-1 model but are separately fine-tuned for their respective multimodal thought processes.
- Classifier-free guidance balances conditioning from text thoughts, prior visual hypotheses, visual subgoals, and original prompts.
- Vision generation with intermediate visual subgoals: Intermediate visual subgoals decompose complex generation into components that are generated separately and combined into the requested image.
B.2 Vision generation with self-critique
Self-critique generation uses textual reflection on an initial visual hypothesis to identify shortcomings and produce a refined image.
- The model analyzes an initial visual hypothesis and iteratively improves it through reflection.
- In a yin-yang symbol with a tiger head, self-critique identifies the need for more stylized and integrated tiger faces before refinement.
B.3 Failure Cases
The reported failures expose two weaknesses: intermediate components may not be spatially integrated, and accurate self-critique may not translate into corrected image generation.
- Intermediate visual subgoals: Intermediate subgoals can be individually correct yet fail to combine into a coherent final image.In one example, a microwave and a bench are generated separately but remain in incompatible settings.
- Self-critique: The model may correctly identify an erroneous image and suggest fixes, yet reproduce the same error in its next generation.For a TV-remote request, self-critique recognizes the mistaken TV image, but the second attempt still shows a TV set.
- Self-critique: These cases indicate a gap between analytical reasoning and generative execution in self-critique.The critique can specify appropriate corrections without those corrections appearing in the subsequent image.