PaperlayerView exact revision on arXiv

Source-linked AI summary

CookAnything: A Framework for Flexible and Consistent Multi-Step Recipe Image Generation

Ruoxuan Zhang, Bin Wen, Hongxia Xie, Yi Yao, Songhan Zuo, Jian-Yu Jiang-Lin, Hong-Han Shuai, Wen-Huang Cheng

arXiv:2512.03540v2cs.CVcs.AI

TL;DR

Diffusion models and existing recipe-illustration approaches have difficulty producing semantically distinct multi-step images while adapting to recipes of different lengths. CookAnything combines regional step control, step-aware positional encoding, and cross-step ingredient consistency; the paper reports state-of-the-art results in training-based and training-free settings.

  • Problem

    Single-image diffusion models are limited in structured multi-step generation, and concatenating step prompts can entangle semantics across images.

    Source: §1 · p. 2

  • Method

    CookAnything uses regional control to bind instructions to image regions, step-aware Flexible RoPE, and cross-step consistency control for ingredients.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    CookAnything achieves state-of-the-art results in both training-based and training-free recipe-illustration settings.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The framework produces coherent, semantically diverse step images and scales to variable-length recipes in both settings.

    Source: §5 · p. 8

    The paper identifies original RoPE’s global coordinates as causing repeated layouts, while increasing step counts weaken long-range consistency.

    Source: §3.3 · pp. 4–5

Abstract

from arXiv · show

Cooking is a sequential and visually grounded activity, where each step such as chopping, mixing, or frying carries both procedural logic and visual semantics. While recent diffusion models have shown strong capabilities in text-to-image generation, they struggle to handle structured multi-step scenarios like recipe illustration. Additionally, current recipe illustration methods are unable to adjust to the natural variability in recipe length, generating a fixed number of images regardless of the actual instructions structure. To address these limitations, we present CookAnything, a flexible and consistent diffusion-based framework that generates coherent, semantically distinct image sequences from textual cooking instructions of arbitrary length. The framework introduces three key components: (1) Step-wise Regional Control (SRC), which aligns textual steps with corresponding image regions within a single denoising process; (2) Flexible RoPE, a step-aware positional encoding mechanism that enhances both temporal coherence and spatial diversity; and (3) Cross-Step Consistency Control (CSCC), which maintains fine-grained ingredient consistency across steps. Experimental results on recipe illustration benchmarks show that CookAnything performs better than existing methods in training-based and training-free settings. The proposed framework supports scalable, high-quality visual synthesis of complex multi-step instructions and holds significant potential for broad applications in instructional media, and procedural content creation.

1 Introduction

Recipe illustration must translate sequential instructions into coherent images while preserving temporal progression, ingredient consistency, and distinctions between stages. CookAnything addresses fixed-length and semantic-entanglement limits with arbitrary-length generation and three complementary controls.

1 Introduction

  • Existing recipe illustration assumes a fixed number of steps, while concatenated prompts can entangle step semantics and produce indistinguishable or incoherent images.The task also requires temporal progression, ingredient consistency, and visual distinctions between cooking stages.Source: §1 · p. 2
  • CookAnything combines Step-wise Regional Control, Flexible RoPE, and Cross-Step Consistency Control to generate variable-length recipe sequences with high visual consistency.SRC binds instructions to distinct latent regions; Flexible RoPE resets coordinates per step; CSCC preserves fine-grained ingredient continuity.Source: §1 · p. 2
  • Experiments show state-of-the-art results in both training-based and training-free settings for arbitrary-length, step-wise recipe illustration.The framework is presented as a flexible approach for diverse real-world recipe structures.Source: §1 · p. 2

2 Related Work

Earlier recipe-generation methods largely produced completed-dish images, while step-wise illustration introduced procedural sequences but did not adapt to recipe-length variability. CookAnything is framed as a structured diffusion approach to recipe sequences, modeling grounding, temporal alignment, and ingredient consistency.

2.1 Recipe Analysis Task

  • CookGAN [8], ChefGAN, and ML-CookGAN generated completed-dish images, while StackedDiffusion [18] introduced per-step illustrations but used a fixed image count.CookAnything dynamically adapts to varying recipe lengths while maintaining visual-semantic alignment, procedural coherence, and ingredient consistency.Source: §2.1 · pp. 2–3

2.2 Procedural Sequence Generation Model

  • Most diffusion work focuses on single-image synthesis, and story visualization [15] falls short of recipes’ procedural consistency needs; CookAnything builds on DiT to model grounding, temporal alignment, and ingredient consistency.The framework is presented as a structured, extensible solution for procedural image generation.Source: §2.2 · p. 3

3 Method

CookAnything generates step-specific images in one denoising process, combining regional control with global recipe context. Flexible RoPE and CSCC address positional alignment and ingredient continuity across steps.

3.1 Preliminary

  • Flux.1-dev jointly processes noisy image tokens and text tokens through DiT self-attention, with RoPE encoding latent-token positions.Its text encoding combines T5 and CLIP.Source: §3.1 · p. 3

3.2 Step-wise Regional Control

  • SRC encodes each recipe step separately and restricts attention to its paired visual region and instruction, while weighted interpolation adds whole-recipe context.The interpolation weight controls the trade-off between global context and regional detail.Source: §3.2 · p. 4

3.3 Flexible RoPE

  • Flexible RoPE assigns independent positional encodings to image regions, then concatenates their tokens for joint input with global attention across regions.This is designed to reduce cross-region interference and preserve generation fidelity as step count grows.Source: §3.3 · p. 5
  • Flexible RoPE improves spatial alignment and visual consistency across steps, especially in long recipes.The Lamb Pilaf comparison shows repeated imagery and positional misalignment with original RoPE, while Flexible RoPE maintains step differentiation and sharpness.Source: §3.3 · p. 5, Figure 3 · p. 5

3.4 Cross-Step Consistency Control

  • CSCC addresses small-ingredient continuity by extracting context-aware tokens from the full recipe; a Cooking Agent fills implicit ingredient details consistently across steps.The approach targets small ingredients that may change appearance or disappear during simultaneous denoising.Source: §3.4 · pp. 5–6
  • CSCC averages recipe-wide contextual tokens with independently encoded step tokens to preserve ingredient color and shape while keeping each step’s content independent.Examples show preserved carrot shape and taro presence with CSCC.Source: Figure 4 · p. 5, §3.4 · p. 6

4 Experiment

CookAnything is evaluated on RecipeGen and VGSI-Recipe, with quantitative, ablation, qualitative, and human-study results. The evaluations report strong benchmark performance, ingredient consistency, and user-rated perceptual quality.

4.1 Experiment Settings

  • Experiments use RecipeGen and VGSI-Recipe, assess training-free and training-based settings, and compare against UNet-, DiT-, and layout-aware baselines using defined metrics.The evaluation metrics include faithfulness, consistency, ingredient accuracy, and usability.Source: §4.1 · p. 6, §4.2 · p. 6, Table 1 · p. 6

4.3 Ablation Study

  • Removing Flexible RoPE lowers Goal Faithfulness (1.93) and Step Faithfulness (0.98); λ=0.2 is optimal, and removing the Cooking Agent still leaves strong performance.Removing CSCC produces a notable drop in Cross-Step Consistency.Source: §4.3 · p. 7, Table 4 · p. 8

4.4 Qualitative Evaluation

  • Across Western and Asian dishes, CookAnything better preserves global layout, ingredient fidelity, and recipe-step faithfulness than the compared DiT-based methods.The examples include maintaining soufflé shape and salmon-and-avocado structure; other methods show omissions, hallucinations, and appearance inconsistency.Source: §4.4 · p. 7, Figure 5 · p. 7

4.2 Quantitative Evaluation

  • CookAnything achieves state-of-the-art results across key RecipeGen metrics and the best performance across VGSI-Recipe metrics in both training-free and training-based settings.The reported RecipeGen strengths include goal and step faithfulness, cross-step consistency, ingredient accuracy, and usability.Source: §4.2 · p. 6, Table 2 · p. 8, Table 3 · p. 8

4.5 User Study on Perceptual Quality

  • In a 53-participant study covering 75 questions, CookAnything surpasses Stable Diffusion 3.5 and Flux.1-dev across all metrics, with Aesthetic Quality of 60.38.The study assesses Cross-Step Consistency, Step Faithfulness, Goal Faithfulness, Aesthetic Quality, and Overall Appeal.Source: §4.5 · pp. 7–8, Table 5 · p. 8

5 Conclusion

CookAnything combines step-wise regional control, Flexible RoPE, and cross-step consistency control for flexible, high-fidelity recipe illustration. Evaluations report coherent, diverse step images and scalability to variable-length recipes in both training-based and training-free settings.

5 Conclusion

  • CookAnything integrates Step-wise Regional Control, Flexible RoPE, and Cross-Step Consistency Control to support semantic alignment, step-wise visual disentanglement, and ingredient continuity.The framework is presented as a unified generation process for step-wise textual instructions.Source: §5 · p. 8
  • Extensive evaluations report visually coherent and semantically diverse step images that scale to variable-length recipes in training-based and training-free settings.Source: §5 · p. 8
  • The paper presents CookAnything as a flexible, high-fidelity framework for generating illustrated recipes from step-wise textual instructions.Source: §5 · p. 8

6 Supplementary Material

The supplement documents dataset characteristics and implementation details alongside additional experiments and metric definitions. It describes RecipeGen and filtered VGSI recipe data, plus the training setup used for CookAnything.

6 Supplementary Material

  • The supplement covers dataset and experimental details, contextual-step-token experiments, metric definitions, Cooking Agent prompts, additional ablations and visualizations, human evaluation, and future work.Source: §6 · p. 11

6.1 Datasets Details

  • RecipeGen contains 21,944 recipes spanning 2 to 15 steps; experiments train on 5,000 recipes and evaluate on its 4,389-recipe official test split.Source: §6.1 · p. 11
  • Filtering VGSI by “cook” yields 1,157 recipes and 6,417 images, with 173 recipes randomly selected for testing; its recipe subset has more limited cooking and visual diversity than RecipeGen.Source: §6.1 · p. 11

6.2 Implementation Details

  • Training-based inputs use vertically concatenated 512×512 step images and the full recipe text, prefixed by a GPT-4o summary and step-number markers.Training uses LoRA on Flux.1-dev, with rank 16, Adam, and 20,000 steps on a single A100 GPU.Source: §6.2 · p. 11

6.3 Experiments on Contextual Step Token

Supplementary experiments examine contextual step tokens and detail how step faithfulness, ingredient accuracy, and usability are assessed. The usability criteria also expose a trade-off between flexible step counts and uniform image sizing.

6.3 Experiments on Contextual Step Token

  • Visualizing step tokens only after full decoding yields metrics better than all listed models except CookAnything.The experiment concatenates all step instructions for T5 encoding, then splits the unified token representation by step.Source: §6.3 · p. 11

6.4 More details about Step Faithfulness, Ingredient Accuracy, and Usability

  • Step Faithfulness combines CLIP alignment with GPT-4o scoring of whether depicted ingredient shapes, containers, and states match the recipe step.Source: §6.4 · p. 11
  • Ingredient Accuracy checks ingredient presence and correctness through GPT-based scoring and human verification, addressing omissions, confusion, and hallucinations.Source: §6.4 · p. 11
  • CookAnything performs best on usability’s step clarity, content diversity, process logic, and step-count reasonableness, while fixed-six-image SKD performs well on image-size consistency.The supplement describes usability through five criteria, including consistent sizing, distinct step representation, duplication, process completeness, and reasonable step counts.Source: §6.4 · pp. 11–12
  • Human assessments of ingredient accuracy use 780 recipes and over 5,000 images rated by 13 evaluators; usability review covers 1,300 recipes from four models.Source: §6.4 · p. 12

6.5 Prompt for Cooking Agent

The Cooking Agent uses two specialized prompts to produce structured, detailed recipe-step descriptions while preserving ingredient terminology across steps unless an ingredient’s state changes.

6.5 Prompt for Cooking Agent

  • Two specialized prompts guide GPT-4o to generate structured, detail-rich step descriptions while preserving ingredient terminology unless its physical state changes.This consistency supports the Cross-Step Consistency Control (CSCC) block’s alignment of ingredient representations across images.Source: §6.5 · p. 12

6.6 Specific Recipe for Fig.1 and 10 in Main Paper

  • Examples include a broccoli egg-salad sequence from blanching broccoli to a finished dish and three-step kimchi serving and heating instructions.Source: §6.6 · pp. 12–13
  • The hamburger example spans dough preparation and baking, pan-frying patties and eggs, and assembling the finished burger.Source: §6.6 · p. 12
  • The six-step Mexican chicken wrap example progresses from preparing the tortilla and fillings to rolling and cutting the wrap.Source: §6.6 · p. 12
  • The seven-step tomato-and-scrambled-egg rice example covers ingredient preparation, cooking and combining the components, then serving them over rice.Source: §6.6 · p. 12
  • The dried-fruit pound-cake example gives eight steps, from preparing ingredients and batter to baking, cooling, and cutting.Source: §6.6 · p. 12
  • An emu-oil example gives three steps of application and includes claims and advice about pain, swelling, sunburn, and seeking medical help for deep cuts.Source: §6.6 · p. 13
  • A four-step faux-leather repair example describes color-matching paint, applying it, optionally adding texture paper, and using a heat tool.Source: §6.6 · p. 13

6.7 More Ablation Study

The ablation varies contextual integration with λ and reports its effects on three generation metrics.

6.7 More Ablation Study

  • At λ = 0.2, Goal Faithfulness is 30.12, Step Faithfulness is 29.80, and Cross-Step Consistency is 0.17; performance degrades across all three metrics as λ increases from 0 to 1.Source: §6.7 · p. 13

6.8 More Visualization Result

The section presents additional visual examples across regional cuisines and liquid-based dishes.

6.8 More Visualization Result

  • Additional regional-cuisine examples in Fig. 6 demonstrate the model’s adaptability to diverse cultural food styles.Source: §6.8 · p. 13
  • Fig. 7 shows results for liquid-based dishes, which the passage identifies as challenging because of their fluid textures and fine-grained visual details.Source: §6.8 · p. 13

6.9 User Study

The user-study question is provided in Fig. 12.

6.9 User Study

  • The section points readers to Fig. 12 for the user-study question.Source: §6.9 · p. 13

6.10 Future Work

CookAnything is presented as a foundation for structured visual generation beyond cooking, with future work aimed at video, editing, and real-world data alignment.

6.10 Future Work

  • The authors position CookAnything as a foundation for structured visual generation in instructional manuals, scientific workflows, and educational storytelling.Source: §6.10 · p. 13
  • Future work will explore extending the framework to multimodal video generation and interactive editing.Source: §6.10 · p. 13
  • Future work will also explore alignment with real-world cooking data.Source: §6.10 · p. 13
Loading 2512.03540v2…