ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization

• Complex compositional prompts expose persistent failures in detail capture, including text rendering, attribute binding, unlikely object combinations, and color leakage.Approaches based on larger models and improved data curation typically require training from scratch, limiting their use with existing models.Source: §1 · p. 1
• Reward-based fine-tuning risks reward hacking and may generalize poorly from small prompt sets to unseen compositional prompts.Per-prompt fine-tuning is also computationally expensive.Source: §1 · p. 2
• ReNO optimizes initial noise during inference while keeping model parameters fixed, using one-step models to avoid unstable gradients through multiple denoising steps.It combines reward objectives and uses limited optimization, in-distribution noise regularization, and gradient clipping to mitigate reward hacking.Source: §1 · p. 2
• ReNO improves four one-step models across T2I-CompBench and GenEval, with gains exceeding 20% in some cases and performance surpassing competing multi-step models at equal compute.Generation takes 20–50 seconds; SD2.1-Turbo approaches proprietary DALL-E 3 and SD3.Source: §1 · p. 3
• ReNO uses reward-model feedback to optimize latent noise for a limited number of iterations, rather than fine-tuning the text-to-image model.The approach is demonstrated across four one-step models and extensive user studies.Source: §1 · p. 3

2.1 Background: One-Step Diffusion Models

• One-step generators approximate a full ODE or SDE in a single step, producing a latent that a pre-trained decoder converts into an image.The section covers models trained with adversarial diffusion, diffusion matching, and trajectory segmented consistency distillation.Source: §2.1 · p. 4

2.2 Initial Noise Optimization

• For a differentiable criterion, initial noise can be optimized by gradient ascent; using a one-step generator enables efficient backpropagation and 20–50-second image generation.The passage contrasts this with prior noise optimization times of 10–40 minutes per image.Source: §2.2 · p. 4
• A norm-based regularizer keeps noise near its initial distribution; in the color-channel illustration, optimization changes the car before the background without collapsing to a fully red or blue image.Ten optimization steps provide satisfactory results in this example.Source: §2.2 · p. 5, Figure 3 · p. 5

2.3 Human Preference Reward Models and Our Reward Criterion

• ReNO combines pre-trained reward models in a noise-optimization criterion, and fewer than 50 steps can improve prompt following and visual aesthetics.Generating one image, including optimization, takes 20–50 seconds, depending on model and image size.Source: §2.3 · p. 5
• The reward criterion can combine multiple pre-trained models, whose varied training sets may capture different aspects of image quality and prompt adherence.The passage says combining models can help mitigate their respective weaknesses and reward hacking.Source: §2.3 · p. 5

3 Related Work

• Prior initial-noise optimization methods mainly target specific applications, whereas ReNO aims to improve text-to-image models generally without extra techniques to mitigate exploding or vanishing gradients.Related applications include image control, motion priors, rare-concept generation, image quality, music generation, and inverse problems.Source: §3 · p. 6
• Existing methods on time-dependent generative models take 10 to 40 minutes per image, while a cited bootstrap approach is limited to jointly represented concepts from input images.The passage identifies DOODL [91] and D-Flow [5] as examples of the slower methods, and Samuel et al. [80] for the bootstrap approach.Source: §3 · p. 6
• Reward-based text-to-image work includes fine-tuning with explicit rewards or direct preference optimization, differentiable-reward fine-tuning, classifier guidance, and reward-based distillation.ReNO differs by optimizing initial latent noise during inference.Source: §3 · p. 6

4 Experiments

• Experiments evaluate one-step models on T2I-CompBench, GenEval, and Parti-Prompts using gradient ascent for 50 steps and selecting the highest-reward image.The evaluated models generate at 512 × 512, except HyperSDXL at 1024 × 1024.Source: §4 · p. 6

4.1 Effect of Reward Models

• Combining ImageReward, CLIPScore, PickScore, and HPSv2 improves faithfulness while also increasing image quality, unlike single rewards that trade off between them.ImageReward or CLIPScore alone improves attribute-binding faithfulness but decreases aesthetics; PickScore and HPSv2 improve image quality with modest faithfulness gains.Source: §4.1 · pp. 6–7, Table 1 · p. 6

4.2 Quantitative Results

• Across T2I-CompBench and GenEval, ReNO improves all models in all categories; SD-Turbo’s mean GenEval score rises from 0.49 to 0.62, and HyperSDXL + ReNO beats DALL-E 3 on 4/6 categories.On T2I-CompBench, SD-Turbo’s Color score rises from 55% to 78%; improvements exceed 20% in Color, Shape, and Texture for SD-Turbo and PixArt-α DMD.Source: §4.2 · pp. 7–8, Table 2 · p. 7, Table 3 · p. 7

4.3 User Study Results

• HyperSDXL + ReNO narrowly beats SD3 with a 54% win rate, while SD-Turbo + ReNO exceeds 60% against every benchmarked model, reaching 77% against SD-Turbo.The authors caution that AMT user studies can be noisy and interpret these results alongside quantitative evaluation.Source: §4.3 · p. 8, Figure 4 · p. 8

4.4 Computational Cost of ReNO

• ReNO runs in 20–50 seconds; at SDXL’s roughly 7-second compute budget, SD-Turbo + ReNO significantly outperforms SDXL, while PixArt-α lies just below ReNO’s Pareto frontier.Figure 5 examines attribute-binding performance across optimization steps, and Figure 6 provides qualitative examples of noise optimization.Source: §4.4 · p. 8, Figure 5 · p. 8, Figure 6 · p. 9

4.5 Effect of ReNO on the Diversity of Generated Images

• ReNO maintains and increases one-step-model image diversity, bringing SD-Turbo and SDXL-Turbo toward the diversity levels of their multi-step counterparts.The authors hypothesize that optimization structures noise away from the zero-mean noise distribution, producing more diverse noises.Source: §4.5 · p. 9, Table 4 · p. 9

4.6 Comparison to Multi-Step Noise Optimization

• Against DOODL using the same CLIPScore objective, ReNO achieves four times larger optimized-criterion gains, uses 75% less GPU memory, and runs 100x faster.ReNO’s multi-reward objective yields 21.0–28.9% attribute-binding gains compared with using CLIPScore alone.Source: §4.6 · p. 9, Table 5 · p. 10

4.7 Limitations

• ReNO requires substantially more runtime and GPU VRAM, and current T2I models still struggle with humans, text rendering, and complex compositional relations.VRAM scales with image size: SD-Turbo requires 15GB, while HyperSDXL requires 39GB; the authors also hypothesize that reward-model reasoning limits performance.Source: §4.7 · p. 10, Table 6 · p. 10

5 Conclusion

• ReNO achieves the strongest open-source results on T2I-CompBench and GenEval, with single-step SD-Turbo images preferred over 60% of the time against 50-step SDXL.The conclusion also reports that ReNO is competitive with the 8B-parameter SD3 model in user studies and outperforms SDXL under the same computational budget.Source: §5 · p. 10
• ReNO is a test-time optimization strategy that enhances text-to-image generation without fine-tuning.The conclusion says that ReNO’s gains motivate further study of optimizing initial noise and developing stronger reward models.Source: §5 · p. 10

Appendix

• The appendix covers further qualitative and quantitative results, reward-model analysis, user-study setup, and implementation details.It includes the algorithm among the implementation specifics.Source: Appendix · p. 17

A Further Qualitative Results

• Across one-step models, ReNO fixes artifacts and improves color attribution, spatial reasoning, and generated-face quality, with results broadly on par with proprietary models in these settings.Figures 7–9 compare generations with and without ReNO and against competing models; Figure 8 covers HyperSDXL at 1024 × 1024 resolution.Source: A Further Qualitative Results · p. 17, Figure 7 · p. 17, Figure 8 · p. 18
• Figure 10 reports increased diversity in generated-image content and layout for SD-Turbo with ReNO.The comparison uses two prompts and the first five seeds, without cherry-picking.Source: A Further Qualitative Results · p. 17, Figure 10 · p. 19

B.1 Comparison to Direct Preference Optimization

• Preference-tuned SDXL improves attribute binding and aesthetic score but underperforms SDXL-Turbo with ReNO in the reported comparison.The authors interpret this comparison as evidence of the potential for test-time optimization to generalize better to unseen prompt distributions.Source: §B.1 · p. 20, Table 7 · p. 20

B.2 Compositional Text-to-Image Methods.

• ReNO consistently outperforms specific methods tailored to compositional text-to-image generation.Table 8 reports the quantitative comparison on T2I-CompBench.Source: §B.2 · p. 20, Table 8 · p. 20
• Unlike methods that use tools for planning or correcting generations, ReNO changes only the initial noise and does not alter the generative model.The authors say ReNO could also be incorporated into those tools to further improve performance.Source: §B.2 · p. 20
• For the same objective and model family, ReNO is 120x faster than DOODL.Table 9 compares their computational costs.Source: §B.2 · p. 21, Table 9 · p. 21

C Analysis of Reward Models

• Combining all reward models yields meaningful improvement in both attribute binding and aesthetic score.Table 10 reports combinations over T2I-CompBench attribute-binding categories and the LAION aesthetic score predictor.Source: C Analysis of Reward Models · p. 21, Table 10 · p. 21
• In most cases, at least 80% of images improve on a reward that ReNO did not optimize for.The reported analysis evaluates leave-one-out rewards on Parti-Prompts; Table 11 presents reward change and the fraction of generations that improve.Source: C Analysis of Reward Models · p. 21, Table 11 · p. 22
• ImageReward and CLIPScore appear less correlated with the other rewards, while HPSv2 and PickScore may share a backbone.The authors suggest the similar backbone as a possible explanation for HPSv2 and PickScore’s correlation.Source: C Analysis of Reward Models · p. 21
• Reward weights are chosen so the models’ losses are roughly similar, with greater emphasis on ImageReward.The stated weights are 1.0 for ImageReward, 5.0 for HPSv2, 0.05 for PickScore, and 1.0 for CLIPScore.Source: C Analysis of Reward Models · pp. 21–22

D User Study

• The study uses pairwise comparisons on slightly more than 1,000 Parti-Prompts prompts, with each model pair assessed by at least 339 unique users.Participants could mark ties; the study asks about personal preference, aestheticness, and faithfulness, and uses default generation settings for competing methods.Source: D User Study · pp. 22–23
• Faithfulness means reflecting the prompt’s key concepts, relations, and attributes, while aestheticness concerns depiction style, coloring, and interpretation.Personal preference asks participants to trade off faithfulness against aestheticness.Source: D User Study · p. 22
• Figure 11 reports faithfulness and aestheticness preferences, with aestheticness preferences larger for every model; the authors say this supports ReNO’s general image-quality benefits.The benchmark results discussed in Section 4.2 mainly measure prompt-following improvements.Source: D User Study · p. 23, Figure 11 · p. 23

E.1 Algorithm

• ReNO uses gradient ascent with Nesterov momentum; the authors found it effective within their computational budget, while noting other optimizers may also work.The algorithm initializes noise and iteratively updates it using reward gradients, retaining the best-scoring output.Source: §E.1 · p. 23

E.2 ReNO hyperparameters

• All experiments use λreg = 0.01, with learning rates of 5 for 512 × 512 models and 10 for HyperSDXL at 1024 × 1024.Gradient-norm clipping helps prevent large noise changes; optimized images retain significant details from the initial generation.Source: §E.2 · p. 24

E.3 Models

• The evaluated one-step models include UNet-based SD-Turbo, SDXL-Turbo, and HyperSDXL, plus transformer-based PixArt-α DMD.The HyperSDXL one-step UNet checkpoint is used rather than its LoRA version.Source: §E.3 · p. 24

E.4 Rewards

• ReNO uses four reward signals: HPSv2, PickScore, ImageReward, and CLIPScore, which measures text-image alignment rather than human preference.HPSv2 uses OpenCLIP ViT-H/14; PickScore uses the same backbone and crowd-sourced comparisons, while ImageReward applies an MLP to BLIP features.Source: §E.4 · p. 24, §E.4.1 · p. 24, §E.4.2 · p. 24

E.5 Metrics

• The evaluation section uses three schemes in addition to the user study and reward-model comparisons.These are T2I-CompBench, GenEval, and the improved LAION Aesthetic Predictor.Source: §E.5 · p. 24

E.5.1 T2I-CompBench

• T2I-CompBench evaluates attribute binding, object relationships, and complex compositions using VQA, object detection, and image-text matching measures.Its subcategories include color, shape, texture, spatial and non-spatial relations, and prompts with multiple objects or categories.Source: §E.5.1 · pp. 24–25

E.5.2 GenEval

• GenEval measures object co-occurrence, position, count, and color through object detection and related vision models, averaging results over four seeds.Its instance-level analysis is designed to assess compositional prompt alignment.Source: §E.5.2 · p. 25

E.5.3 LAION Aesthetic Score Predictor

• The LAION Aesthetic Predictor scores images without jointly conditioning on their prompts, and higher scores did not always correspond to higher visual quality.The authors hypothesize that its prompt-independent scoring may be more prone to reward hacking.Source: §E.5.3 · p. 25

E.6 Diversity Analysis

• Diversity is evaluated over 50 seeds for each of 110 PartiPrompts prompts using mean pairwise LPIPS and DINO scores.Higher LPIPS and DINO scores are interpreted as less diversity across generated images.Source: §E.6 · p. 25

E.7 Comparison to DOODL

• The DOODL comparison uses 50 prompts from each T2I-CompBench attribute-binding category and adapts its 50-step implementation to SD2.1.SDXL was excluded because DOODL required more than 40GB of VRAM for a single-GPU run.Source: §E.7 · p. 25

E.8 FLUX-schnell results

• ReNO is applied to one-step FLUX-schnell, and its optimized noise is then used for four-step generation; the appendix compares attribute-binding results and seed-0 examples.The table covers 512 × 512 attribute-binding categories, while the figure compares FLUX-schnell with and without ReNO against FLUX-dev; samples use CPU offloading to fit within 40GB of VRAM.Source: §E.8 · p. 25, Table 12 · p. 26, Figure 12 · p. 26

F Broader Impact

• Because the framework relies on pretrained models, it inherits their biases; the authors suggest safety and fairness objectives as possible mitigation options.The broader-impact discussion says practical deployments, including theirs, must use a safety checker or filter to prevent NSFW content.Source: F Broader Impact · pp. 25–26