PaperlayerView exact revision on arXiv

Source-linked AI summary

GDB-Reward: From Evaluation Metrics to Training Rewards for Graphic Design

Adrienne Deganutti, Purvanshi Mehta, Simon Hadfield, Andrew Gilbert

arXiv:2609.02813v1cs.CV

TL;DR

Graphic design generation requires satisfying precise structured constraints, but prompt optimization for frozen image generators lacks differentiable supervision and informative rewards. The paper introduces GDB-REWARD, which converts heterogeneous design metrics into a unified reinforcement learning reward for optimizing a prompt policy. Experiments show improved design-specification adherence across perceptual quality, rendering fidelity, and spatial accuracy while keeping the image generator frozen.

  • Problem

    Graphic design generation requires precise adherence to typography, layout, color, and communicative constraints, while prompt optimization for frozen generators operates through a non-differentiable pipeline lacking reconstruction gradients.

  • Method

    GDB-REWARD transforms heterogeneous graphic design evaluation metrics into a unified reinforcement learning reward, while GRPO updates only a LoRA adapter on a prompt generator and freezes the image generator and reward models.

  • Results

    GDB-REWARD substantially improves design-specification adherence across perceptual quality, rendering fidelity, and spatial accuracy while leaving the image generator entirely frozen.

  • Takeaways & Limitations

    Heterogeneous, non-differentiable evaluation metrics can function as effective reinforcement learning objectives rather than only passive benchmarks.

Abstract

from arXiv · show

Text-to-image models excel at natural image synthesis but struggle with graphic design, where success depends on satisfying precise constraints on typography, layout, color, and visual communication. While prompt optimization offers an attractive alternative to expensive diffusion model fine-tuning, learning prompts for frozen image generators requires informative reward functions despite the entirely non-differentiable generation process. Reinforcement learning does not require differentiable objectives; it requires only scalar rewards capable of ranking candidate outputs. This raises a simple question: can design evaluation metrics themselves become reinforcement learning rewards? Our central contribution is GDB-Reward, a framework that systematically transforms heterogeneous graphic design evaluation metrics into a unified reinforcement learning reward. Experiments demonstrate that GDB-Reward provides an effective optimization objective, substantially improving adherence to the design specification in perceptual quality, rendering fidelity, and spatial accuracy while keeping the image generator entirely frozen. More broadly, our results demonstrate that heterogeneous, non-differentiable evaluation metrics can move beyond passive benchmarking to become effective optimization objectives for reinforcement learning in domains where differentiable supervision is unavailable.

1 Introduction

Graphic design generation requires satisfying structured constraints that standard text-to-image models often miss, motivating prompt optimization with a unified reward built from design-specific metrics. GDB-REWARD uses reinforcement learning to improve specification adherence while keeping the image generator frozen.

  • Motivation: Graphic design success depends on typography, layout, color consistency, and communicative intent rather than visual plausibility alone.Generated designs may look plausible while failing to reproduce required text, colors, or structure.
  • Motivation: Fine-tuning the image generator would require substantial computation and large collections of paired, high-quality graphic design data.The paper therefore asks whether specification adherence can improve without modifying the generator.
  • Challenge: Prompt optimization lacks gradients and therefore needs scalar feedback that reflects multi-dimensional adherence across typography, color, layout, semantics, and visual quality.The generation pipeline is entirely non-differentiable, so reconstruction loss is unavailable.
  • Prior Evaluation: Design-specific metrics extend evaluation beyond generic perceptual scores but remain heterogeneous, largely non-differentiable, and composed of discrete and continuous signals.GraphicDesignBench spans spatial accuracy, typographic fidelity, structural validity, and human-aligned preference.
  • Core Idea: Reinforcement learning needs only a scalar reward capable of ranking outputs, enabling evaluation metrics to become optimization objectives rather than passive benchmarks.GDB-REWARD transforms a bank of design evaluation metrics into a unified reward for prompt optimization.
  • Approach: GDB-REWARD scores generated designs while GRPO updates only a LoRA adapter on a prompt generator, leaving the image generator and reward models frozen.The framework uses a lightweight language model to rewrite structured layout metadata into natural-language prompts.

2 Related Works

Related work evaluates graphic designs with specialized metrics and optimizes prompts or other pipeline components, but this paper combines heterogeneous design metrics to optimize prompts for frozen image generators. Its approach differs from methods that fine-tune diffusion models or reinforce layout decomposition.

  • Evaluation of Graphic Design: Graphic design evaluation requires multiple metrics because no single measure captures every quality axis, including typography, layout, and structural consistency.Generic perceptual and preference models focus primarily on global image aesthetics and underperform on design quality.
  • Evaluation of Graphic Design: Design-specific frameworks such as PosterReward and GraphicDesignBench assess layout, text rendering, aesthetics, perceptual quality, and other design requirements.GraphicDesignBench combines deterministic and LLM-based metrics across complementary dimensions.
  • Prompt Optimization and Reinforcement Learning: Prompt optimization methods improve outputs from frozen generators through prompt rewriting or reinforcement learning without modifying the underlying diffusion model.Promptist uses an instruction-tuned language model, while PromptLoop uses reinforcement learning.
  • Position of This Work: The proposed methodology maps structured layout metadata into candidate prompts, renders them with a frozen image generator, and ranks them using a composite reward.GRPO updates only the prompt policy's LoRA parameters while the generator and reward function remain fixed.
  • Prompt Optimization and Reinforcement Learning: PosterReward fine-tunes the diffusion model, LaySPA reinforces spatial reasoning, and CreatiParser applies GRPO to graphic design decomposition rather than image generation.These methods optimize different components from the prompt-policy approach described here.

3 Methodology

The method optimizes prompts for a frozen text-to-image generator by converting heterogeneous graphic-design metrics into a unified reward and applying GRPO to a LoRA-parameterized prompt policy. The reward combines perceptual, rendering, and spatial signals on a common scale, while relative candidate ranking guides non-differentiable optimization.

  • Prompt Policy: The prompt policy converts structured layout metadata into natural-language prompts, which a frozen image generator renders into candidate designs.The policy is an instruction-tuned language model with a LoRA adapter; only the adapter is updated during training.
  • GDB-REWARD: GDB-REWARD transforms heterogeneous design metrics into normalized rewards in [0, 1] and combines them into one scalar objective.The metric bank covers perceptual quality, rendering fidelity, and spatial accuracy, with larger normalized values indicating better design quality.
  • Metric Bank: The reward measures complementary design properties, including perceptual similarity, typography and color reproduction, and the number of rendered components.Spatial accuracy uses a YOLO-OBB detector to compare detected text and image regions with reference renderings and layout metadata.
  • Composite Reward: For each sample, the composite reward averages only applicable metrics, avoiding product or minimum aggregation that would over-penalize isolated weaknesses or let one metric dominate.The weighted arithmetic mean preserves contributions from all applicable metrics while providing smooth trade-offs.
  • Policy Optimization with GRPO: GRPO standardizes rewards within sampled prompt groups, so policy updates depend on candidate rankings rather than perfectly calibrated absolute scores.Prompts outperforming the group average are encouraged, while poorer candidates are discouraged; a KL term keeps the policy near the frozen base language model.
  • Evaluation: Table 2 reports mean normalized per-metric scores on the LICA test split, with the composite Full score capped at 0.758 because reference-based perceptual metrics cannot reach 1.The table distinguishes perceptual, rendering, and spatial metrics, including DreamSim, LPIPS, SSIM, OCR, CIEDE2000, and YOLO component counts.

4 Experiments

Experiments evaluate GDB-REWARD across frozen image generators, ablations, supervised fine-tuning, and design-intent preservation. The results show improved specification adherence while preserving the generator and communicative intent.

  • Experimental Setup: The evaluation uses frozen FLUX.1-dev and FLUX.2-dev generators with a Qwen3.5-9B prompt policy trained using GDB-REWARD.Training uses LoRA on the prompt policy while keeping the image generators fixed.
  • Quantitative Analysis: GDB-REWARD increases composite scores under both generators, with gains concentrated in OCR accuracy, color fidelity, and component-count agreement.On FLUX.1, OCR increases from 0.446 to 0.643; the trained FLUX.1 policy reaches 0.559 versus 0.560 for the untrained FLUX.2 baseline.
  • Qualitative Analysis: Qualitative results show more accurate text, closer color palettes, and layouts with more appropriate component counts and preserved composition.These changes correspond to improvements in OCR, color fidelity, and spatial evaluation metrics.
  • Ablations: Removing any reward category degrades overall performance, while combining perceptual quality, rendering fidelity, and spatial accuracy achieves the highest reward.The largest reduction occurs when rendering fidelity is removed, indicating the strongest optimization signal among the groups.
  • Ablations: Within-group reward standardization makes hand-tuned per-metric coefficients largely redundant and sometimes harmful, supporting equal per-concept weighting.The reward-reweighting study compares identical GRPO settings with different training weights.
  • Generator Fine-tuning Comparison: GDB-REWARD prompt optimization is compared with supervised generator fine-tuning, which trains a LoRA adapter on the image generator to reconstruct reference renders.The supplied passages describe the two approaches and note that supervised fine-tuning can trade perceptual and aesthetic quality for exact reconstruction of one target.
  • Design-Intent Evaluation: Upgrading the frozen backbone raises G-Eval from 52.5 to 65.0, while GDB-REWARD predominantly preserves or slightly improves design intent across metrics.Design intent is evaluated from generated descriptions against ground-truth intent using semantic and lexical similarity metrics.

5 Conclusion

The conclusion presents GDB-REWARD as a unified reinforcement learning reward for prompt optimization with frozen text-to-image generators. It frames evaluation metrics as optimization objectives for non-differentiable generation tasks.

  • 5 Conclusion: GDB-REWARD transforms heterogeneous graphic design evaluation metrics into a unified reinforcement learning reward for prompt optimization.The framework optimizes only a lightweight prompt policy while leaving the text-to-image generator frozen.
  • 5 Conclusion: The results suggest that evaluation metrics can serve as effective optimization objectives rather than only passive benchmarks in non-differentiable generation tasks.This conclusion remains within the scope of the reported prompt-optimization setting.

A Additional Methodology Details

The prompt policy is an instruction-tuned language model with a LoRA adapter, conditioned by a fixed system instruction. Figure 4 shows how structured design metadata is converted into a natural-language prompt for the frozen generator.

  • Prompt Policy: The prompt generator is an instruction-tuned LLM equipped with a LoRA adapter and conditioned by a fixed system instruction.The system instruction is shown in Figure 4.
  • System Instruction: Figure 4’s instruction guides the LLM to convert structured graphic design metadata into one natural-language prompt for the frozen text-to-image generator.The figure documents the conditioning instruction used by the prompt policy.

B Experiments Implementation Details

Implementation details specify the prompt policy, frozen image generators, reward configuration, GRPO schedule, and computing infrastructure. The setup optimizes a small LoRA parameter subset while reward models remain in inference mode.

  • Prompt Policy: Qwen3.5-9B is fine-tuned with LoRA, optimizing 29.1M parameters, approximately 0.2% of the model, while its backbone remains frozen.LoRA adapters use r = 16 and α = 32 in attention and MLP projection layers.
  • Image Generator: FLUX.1-dev and FLUX.2-dev serve as frozen image generators, producing 768 × 768 training images and 1024 × 1024 evaluation images.Image generation uses 20 diffusion steps, guidance scale 3.5, and sequence length 512.
  • Reward Configuration: GDB-REWARD uses six dominant palette colors with τ = 20, a fine-tuned YOLO11x-OBB detector for layout analysis, and reward models in inference mode.The detector is taken from the GraphicDesignBench repository.
  • RL Hyperparameters: GRPO runs for 1368 steps on FLUX.1-dev and 912 steps on FLUX.2-dev, using groups of 12 sampled prompts across two layouts per update.The learning rate is 3 × 10^-5, with KL coefficient β = 0.02 and gradient clipping at 1.0.
  • Computing Infrastructure: Experiments run on eight NVIDIA H100 SXM5 GPUs with 80 GB memory each, using PyTorch, Transformers, TRL, PEFT, and Diffusers.The compute node has 168 CPU cores, 907 GB system memory, and 12.2 TB local storage.

C Dataset Examples

Each LICA GraphicDesignBench sample pairs layout metadata, a rendered graphic design, and a natural-language design intent.

  • Each dataset example contains serialized layout metadata, a rendered graphic design, and a natural-language design intent.The metadata describes design elements and their attributes.

D Design-Intent Evaluation Model

The design-intent evaluation model predicts a concise textual description from a rendered design and compares it with the ground-truth intent. Candidate vision–language models are fine-tuned and evaluated using semantic and lexical metrics, with G-Eval providing the primary semantic assessment.

  • Design-intent prediction: A dedicated vision–language model predicts the deliverable, message, target audience, and emotional tone from a rendered graphic design.The model is used exclusively for evaluation and is excluded from the reinforcement-learning reward.
  • Evaluation setting: The dataset examples pair design intent, rendered images, and layout metadata as inputs for the broader design-generation and evaluation setting.
  • Design-intent prediction: The predicted intent ˆy = fϕ(x) is compared with the ground-truth intent yℓ for the generated image x.
  • Evaluation metrics: G-Eval is the primary semantic metric, supplemented by BLEU-1, METEOR, ROUGE-L, and CIDEr.The G-Eval rubric rewards semantic agreement with the reference intent while maintaining comparable brevity.

E Additional Qualitative Examples

Additional qualitative comparisons examine optimized and baseline prompt policies across FLUX.1-dev and FLUX.2-dev on diverse graphic-design tasks. The optimized policy improves alignment with intended structure and aesthetics, but textual descriptions cannot exactly reproduce unavailable photographic assets.

  • Cross-backbone comparisons: Qualitative comparisons span event posters, planners, promotional flyers, awareness campaigns, and advertisements across FLUX.1-dev and FLUX.2-dev.Each backbone is shown with baseline and GDB-REWARD-optimized prompt policies against the ground-truth reference.
  • Qualitative limitations: Missing original photographic assets can produce larger deviations from reference renders in image-centric designs.Portraits of individual DJs are cited as an example that cannot be reproduced exactly from text alone.
  • Qualitative findings: The optimized prompt policy improves alignment with the intended design structure and aesthetics.

F Additional Quantitative Results

Additional quantitative experiments test the GRPO prompt-optimization framework with SDXL while holding the policy, reward, training settings, data split, and random seed fixed. SDXL underperforms FLUX and is constrained by its 77-token text encoding limit, although the qualitative comparison includes optimized SDXL outputs.

  • SDXL evaluation: The full prompt-optimization pipeline is repeated with Stable Diffusion XL as the frozen image generator.The setup is designed to test generalization beyond the FLUX backbone.
  • Quantitative results: SDXL underperforms the FLUX backbones on the composite and constituent evaluation metrics.Table 9 reports the composite GDB-REWARD score and its constituent metrics before and after SDXL prompt optimization.
  • Controlled comparison: The SDXL comparison holds the policy LLM, composite reward, per-concept weighting, GRPO hyperparameters, data split, and random seed fixed.Only the image generator is swapped, with SDXL-specific inference settings.
  • Backbone constraint: SDXL’s 77-token CLIP text encoder cannot ingest the long descriptive prompts learned by the policy.FLUX uses a T5 encoder that admits substantially longer inputs.
  • Qualitative SDXL results: Qualitative SDXL results compare the ground-truth reference, the baseline prompt policy, and the GDB-REWARD-optimized policy using the same generator.
Loading 2609.02813v1…