PaperlayerView exact revision on arXiv

Source-linked AI summary

Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoring

Jisheng Dang, Zhenxuan Wang, Bin Li, Ronghao Lin, Bin Hu, Tat-Seng Chua

arXiv:2609.12830v1cs.CV

TL;DR

Continuous emotion control in image generation must improve affective alignment without changing prompt-specified content, yet existing approaches can exhibit emotion–semantic drift. The paper combines VA-conditioned Flow-GRPO with a prompt-paired neutral feature anchor; its evaluated method lowers emotional errors and improves CLIPScore over EmotiCrafter, with a perceptual-quality trade-off.

  • Problem

    Existing emotion-conditioning approaches may change prompt content as VA conditioning strengthens, while proxy losses and emotion-only image rewards do not jointly ensure final-image alignment and semantic fidelity.

    Source: §I · p. 1

  • Method

    The framework combines a frozen VA-predictor terminal reward optimized with Flow-GRPO and a CLIP-feature anchor comparing emotional and zero-VA images generated from the same prompt and initial latent.

    Source: §I · p. 2

  • Results

    V-Err 1.132 and A-Err 1.492 are the full method's lowest emotional errors, and its CLIPScore 30.179 exceeds EmotiCrafter's 27.742, while IQA is 0.861.

    Source: C. Quantitative Results · p. 10, §V · p. 12

  • Takeaways & Limitations

    Within the evaluated setting, Flow-GRPO reduces continuous VA error and the anchor improves CLIPScore over GRPO-only, supporting complementary objectives for emotional alignment and semantic preservation.

    Source: F. Discussion in Emotional Image… · p. 12

    The experiments do not directly compare against EmoGen, EmoFeedback2, or EmoSENSE because their interfaces, backbones, and evaluation protocols differ.

    Source: F. Discussion in Emotional Image… · p. 12

Abstract

from arXiv · show

Continuous emotion control in text-to-image generation requires a model to improve affective alignment without changing the objects, layout, or scene described by the prompt. Existing supervised emotion-injection methods often optimize feature-space proxies and may therefore exhibit emotion-semantic drift, in which stronger emotional conditioning is accompanied by unintended content changes. We address this problem with a flow-matching image-generation framework that combines continuous valence-arousal (VA) conditioning, Group Relative Policy Optimization (GRPO), and a neutral semantic anchor. The deterministic probability-flow ODE is converted into a marginal-preserving SDE, yielding non-degenerate transition densities for trajectory sampling and policy-ratio estimation. A frozen CLIP-based VA regressor supplies a terminal reward measuring the distance between the predicted and target VA coordinates, while an image generated from the same prompt under zero VA conditioning provides a feature-space reference for semantic preservation. A reduced denoising schedule is used for online RL sampling, whereas the original schedule is retained at inference. Experiments on 3,300 prompt-emotion combinations show substantially lower valence and arousal errors than the VA-conditioned baseline and an improved CLIPScore relative to EmotiCrafter, with a measurable trade-off in reference-free image quality. The results support anchor-regularized Flow-GRPO as a practical approach to balancing emotional alignment and semantic consistency in continuous-affect image synthesis.

I. INTRODUCTION

Emotion-conditioned image generation must change perceived affect without changing prompt-specified content, but stronger VA conditioning can cause emotion–semantic drift. The paper combines output-level emotion optimization with a prompt-paired neutral anchor to balance emotional alignment and semantic preservation.

I. INTRODUCTION

  • The central challenge is changing perceived emotion without changing prompt-specified objects, actions, or scene context.Affect arises from both appearance cues and semantic cues, so changing scene content can alter both meaning and emotion.Source: §I · p. 1
  • Existing supervised methods optimize intermediate proxies, while image-level emotion optimization alone can reward scene changes that reduce semantic fidelity.The paper frames emotion–semantic drift as the resulting failure mode.Source: §I · p. 1
  • The method pairs a frozen VA-predictor terminal reward with a CLIP-feature anchor comparing emotional and zero-VA outputs from the same prompt and initial latent.A marginal-preserving SDE supplies stochastic transitions with tractable likelihoods for Flow-GRPO policy optimization.Source: §I · p. 2
  • The framework is implemented with LoRA and denoising reduction, and experiments report lower VA error and improved CLIPScore versus the VA-conditioned baseline alongside a perceptual-quality trade-off.The experiments separately examine the VA reward, anchor term, and backbone.Source: §I · p. 2

II. RELATED WORK

Prior continuous-affect generators jointly condition on prompts and VA values, while related reinforcement-learning work provides tools for optimizing decoded-image objectives. The paper combines these strands with a task-specific paired semantic reference.

A. Continuous Emotion-Conditioned Generation

  • EmotiCrafter enables continuous VA control with free-form prompts, but coupled prompt and emotion representations can permit scene changes as conditioning strengthens.Other visual-emotion studies primarily improve affect representation rather than jointly optimizing final-image VA error and prompt-conditioned scene preservation.Source: A. Continuous Emotion-Conditioned… · p. 3

B. Reinforcement Learning in Generative Models

  • Prior reward-optimization methods provide mechanisms for improving decoded-image objectives, including policy gradients, denoising-process formulations, and group-relative optimization.Flow-GRPO adapts group-relative optimization to flow matching using a marginal-preserving SDE and fewer online denoising steps.Source: B. Reinforcement Learning in… · p. 3
  • For emotional synthesis, a VA reward may favor semantic shortcuts, while an overly strong semantic penalty may suppress emotional changes.The paper adopts Flow-GRPO and makes its task-specific contribution the pairing of continuous VA conditioning with a semantic reference matched by prompt and initial latent.Source: B. Reinforcement Learning in… · p. 3

C. Reinforcement Learning in Affective Computing

The paper positions its approach against affective-generation systems with different control variables and feedback signals, then formulates generation as joint emotion alignment and semantic preservation. Its framework uses paired emotional and neutral branches to implement those objectives.

C. Reinforcement Learning in Affective Computing

  • EmoFeedback2 uses vision–language-model scores and textual feedback, whereas EmoSENSE uses hierarchical fuzzy reinforcement learning with discrete emotion categories and visual attributes.The paper distinguishes these systems from its continuous VA control and scalar reward based on predicted-versus-target VA distance.Source: C. Reinforcement Learning in Affective… · p. 3

A. Problem Formulation

  • The task is to reduce distance between predicted and target VA coordinates without changing the prompt's scene layout, object identity, or visual intent.The paper describes emotion–semantic drift as stronger emotional alignment achieved at the cost of semantic fidelity.Source: A. Problem Formulation · p. 4

B. Framework Overview

  • The framework treats denoising as a policy, combining a terminal reward from a frozen VA predictor with an anchor regularizer for semantic preservation.The framework overview routes the emotion-conditioned image through the reward path and compares it with a zero-VA neutral branch in CLIP visual-feature space.Source: A. Problem Formulation · p. 4, B. Framework Overview · p. 4

C. VA-Conditioned Generation

  • An Emotion Injection Transformer fuses the continuous VA coordinate with the neutral prompt representation, and the two branches share its parameters while using emotional and zero-VA inputs.LoRA parameters in the image generator and the EIT parameters are optimized while the original backbone and VAE weights remain frozen.Source: C. VA-Conditioned Generation · p. 4

D. Anchor-Based Semantic Constraint

The anchor supplies a prompt-specific neutral reference so the model can preserve scene identity while changing affective appearance. Its feature-space penalty and balancing coefficient control how far the emotional output can move from that reference.

D. Anchor-Based Semantic Constraint

  • A zero-VA reference discourages emotional optimization from improving its score through major content changes such as scene, object, or layout alterations.The reference is generated for each prompt under the neutral condition.Source: D. Anchor-Based Semantic Constraint · p. 5
  • The anchor loss compares frozen CLIP visual features rather than pixels, favoring scene-identity preservation while permitting changes in tone and appearance.The anchor feature is detached from the gradient graph, so the neutral branch serves as a reference rather than being pulled toward the emotional output.Source: D. Anchor-Based Semantic Constraint · p. 5
  • Larger anchor weights emphasize proximity to the neutral reference, while smaller weights permit greater movement toward the target emotion.The paper motivates this balance because an overly strong semantic constraint can limit perceptible emotional variation.Source: D. Anchor-Based Semantic Constraint · p. 5

E. GRPO-based Optimization

The method treats stochastic denoising as a finite-horizon MDP and assigns reward after decoding the final latent. A frozen VA predictor scores how closely the generated image matches the target coordinates.

E. GRPO-based Optimization

  • The finite-horizon MDP treats each denoising step as a state transition, with the next denoised latent as the action.A trajectory runs from zT to z0, and receives nonzero reward only after z0 is decoded.Source: E. GRPO-based Optimization · pp. 5–6

1) VA Terminal Reward:

  • A frozen CLIP-based VA predictor measures the decoded image’s valence and arousal, and larger rewards correspond to smaller Euclidean errors from the target.The predictor uses a CLIP image encoder and released valence and arousal regression heads.Source: 1) VA Terminal Reward · p. 6

2) Marginal-preserving Stochastic Policy:

The deterministic rectified-flow ODE gives a fixed successor from each intermediate latent, so its transition is a Dirac measure rather than an ordinary density. The method uses a marginal-preserving SDE to make transition likelihoods and policy ratios tractable.

2) Marginal-preserving Stochastic Policy:

  • A fixed intermediate latent has a deterministic successor under the rectified-flow ODE, making its transition a Dirac measure without an ordinary density for policy-gradient ratios.Different initial noise samples can still produce different images.Source: 2) Marginal-preserving Stochastic Policy · p. 6
  • The marginal-preserving SDE adds score-dependent drift as well as stochasticity to preserve the original probability path’s marginal density.The schedule uses σt = αt/(1 − t), with α = 0.3.Source: 2) Marginal-preserving Stochastic Policy · p. 6
  • Euler–Maruyama discretization makes each transition an isotropic Gaussian, so transition likelihoods and ratios to old or reference policies are tractable.The covariance is σ²t∆tI.Source: 2) Marginal-preserving Stochastic Policy · p. 6

3) Group-relative Policy Objective:

GRPO normalizes terminal rewards within groups of sampled trajectories and maximizes a clipped policy objective. The final objective combines that policy term with an anchor regularizer to jointly optimize emotional alignment and semantic preservation.

3) Group-relative Policy Objective:

  • For each condition, the old policy samples a group of trajectories whose terminal rewards are normalized within the group.The group reward standard deviation and a small numerical constant are used in the normalization.Source: 3) Group-relative Policy Objective · p. 6
  • The GRPO objective is maximized with clipping and a reference-policy deviation term; reduced denoising is used for online updates, while inference restores the original schedule.The clipping coefficient, reference-policy coefficient, and number of sampled denoising transitions are specified in the objective.Source: 3) Group-relative Policy Objective · p. 6

F. Overall Objective

  • The final minimization combines negative GRPO with the anchor loss, jointly optimizing VA alignment and proximity to the zero-VA reference.The anchor illustration shows that an unanchored output can add prompt-inconsistent elements, while the anchored output stays closer to the requested scene as it moves toward the target VA condition.Source: F. Overall Objective · p. 6, Figure 3 · p. 6

IV. EXPERIMENTS

The experiments evaluate emotion accuracy, prompt consistency, and reference-free image quality on held-out prompt-emotion combinations, using SDXL and SD3.5M backbones. The reported comparisons favor the combined GRPO-and-anchor method, with SD3.5M providing a stronger semantic basis after optimization.

A. Implementation Details

  • Evaluation uses a held-out test set of 3,300 images and measures V-Err, A-Err, CLIPScore, and CLIP-IQA.The set covers 132 prompts across five valence and five arousal values; the metrics assess affective accuracy, content preservation, and perceptual image quality.Source: A. Implementation Details · p. 7
  • The setup compares SDXL and SD3.5M across supervised, anchor-only, GRPO-only, and combined configurations, with both backbones fine-tuned using rank-4 LoRA.The anchor loss is squared Euclidean distance between CLIP embeddings of generated and zero-VA reference images, with λ = 0.5.Source: A. Implementation Details · p. 7
  • After GRPO+Anchor optimization, SD3.5M has substantially lower VA errors and higher CLIPScore, although its initial VA error after EIT injection is slightly higher than SDXL’s.The authors attribute its stronger semantic prior to its MMDiT architecture and T5-XXL text conditioning.Source: A. Implementation Details · p. 7, Figure 4 · p. 7
  • On SD3.5M, combining Anchor and GRPO achieves the best balance of V-Err, A-Err, and CLIPScore; the full method also outperforms SDXL variants, EmotiCrafter, and SDXL All in the reported comparisons.Anchor-only and GRPO-only each improve one aspect but not both.Source: Figure 7 · p. 8

B. Qualitative Results

Qualitative examples show that anchor-constrained GRPO reduces emotion–semantic drift compared with EmotiCrafter, while still allowing residual content variation. It expresses affect mainly through appearance changes while preserving more of the prompt’s scene content.

B. Qualitative Results

  • Across prompts, our method retains more prompt-level content than EmotiCrafter, whose variations often change scene structure, object layout, or other content.The person-centric and landscape examples show more stable key elements across VA conditions, though preservation is not complete.Source: B. Qualitative Results · p. 8, Figure 8 · p. 9, Figure 9 · p. 10
  • As VA coordinates move from neutral, EmotiCrafter can lose key objects or replace the boy with a girl, animal, or cat.Examples include the tree disappearing at (3, −3), the comic book becoming a smartphone at (0, 3), and a cat appearing at (3, 3).Source: B. Qualitative Results · pp. 8–9
  • Anchor-constrained GRPO usually retains the boy, tree, and comic book, expressing emotion through facial expression, color, illumination, and atmosphere.Local details and backgrounds may still vary, so the method reduces rather than eliminates drift.Source: B. Qualitative Results · pp. 8–9, Figure 8 · p. 9, Figure 9 · p. 10

C. Quantitative Results

The full SD3.5M method has the lowest reported emotional errors and improves CLIPScore over EmotiCrafter, with a perceptual-quality trade-off. Ablations show that the anchor and GRPO contribute complementary objectives, while results also depend on the backbone.

C. Quantitative Results

  • V-Err 1.132 and A-Err 1.492 are the full method’s lowest emotional errors; its CLIPScore of 30.179 exceeds EmotiCrafter’s 27.742, while IQA is 0.861.The reported IQA indicates a perceptual-quality trade-off rather than improvement on every metric.Source: C. Quantitative Results · p. 10

D. Ablation Study

  • SDXL All remains close to EmotiCrafter in VA error and CLIPScore, while the SD3.5M configuration improves those metrics; the reported gain is not attributable to GRPO alone.The anchor-versus-GRPO-only comparison shows the anchor raises CLIPScore while slightly increasing VA errors.Source: C. Quantitative Results · p. 10

C. Quantitative Results

  • Relative to the SDXL counterpart, SD3.5M reduces V-Err by 0.359 (24.1%) and A-Err by 0.344 (18.7%), and increases CLIPScore by 2.291.IQA decreases from 0.912 to 0.861, so the comparison does not show an across-the-board quality gain.Source: C. Quantitative Results · p. 11

D. Ablation Study

  • On SD3.5M, combining GRPO and the anchor yields CLIPScore 30.179 and VA errors 1.132/1.492, versus GRPO-only’s lower errors of 1.097/1.375 and CLIPScore 28.648.Anchor-only yields CLIPScore 29.183 versus EmotiCrafter’s 27.742, with VA errors of 1.174/1.652.Source: D. Ablation Study · p. 11
  • All SDXL-based configurations have higher VA errors and lower CLIPScore than the SD3.5M method, but SDXL All has higher IQA than the full method.The reported backbone comparison therefore does not extend to an IQA advantage for SD3.5M.Source: D. Ablation Study · p. 11
  • Adding the anchor to GRPO-only raises CLIPScore by 1.531, while increasing V-Err by 0.035 and A-Err by 0.117.The passage cautions that CLIPScore alone does not fully measure scene-structure preservation.Source: D. Ablation Study · p. 11

E. User Study

The user study compares emotion expressiveness and prompt alignment ratings for generated images across three methods. The proposed method receives higher mean ratings than both baselines on both dimensions in the evaluated samples.

E. User Study

  • The study evaluates 150 images from 10 prompts, five VA conditions, and three methods.The methods are EmotiCrafter, Ours w/o Anchor, and Ours (Full).Source: E. User Study · p. 11
  • Thirty volunteers each rate a random subset of 50 images on emotional expressiveness and text–image alignment using separate 5-point scales.Ratings are made with the original prompt and target VA values shown.Source: E. User Study · p. 11
  • Text–image alignment rises from 2.72 for EmotiCrafter to 4.45 for the proposed method, while emotional expressiveness rises from 3.67 to 4.32.The table caption reports statistically significant differences between Ours (Full) and baselines (paired t-test, p < 0.05).Source: E. User Study · p. 11, Table IV · p. 12
  • Higher mean ratings support improved emotion–semantic balance in the evaluated samples, but do not show that the anchor prevents drift in every image.Source: E. User Study · pp. 11–12

F. Discussion in Emotional Image Generation

Within the evaluated setting, Flow-GRPO lowers continuous VA error and the anchor improves CLIPScore over GRPO-only, while perceptual-quality trade-offs and unavailable cross-framework comparisons bound the findings.

F. Discussion in Emotional Image Generation

  • The current protocol does not support direct quantitative comparisons with EmoGen, EmoFeedback2 [31], or EmoSENSE.Their categorical interfaces or differing backbone and feedback procedure prevent a controlled comparison without a common prompt set, matched backbone, and predefined category-to-VA mapping.Source: F. Discussion in Emotional Image… · p. 12
  • Within the evaluation setting, Flow-GRPO reduces continuous VA error, the anchor improves CLIPScore over GRPO-only, and SD3.5M further contributes to both improvements.The authors describe Flow-GRPO and the anchor as complementary objectives for emotional alignment and semantic preservation.Source: F. Discussion in Emotional Image… · p. 12
  • During training, VA reward steadily increases while CLIPScore mildly declines, indicating a trade-off between emotional optimization and semantic consistency.The authors use this divergence to support explicit semantic constraints.Source: F. Discussion in Emotional Image… · p. 12

V. CONCLUSION

  • The full SD3.5M configuration obtains the lowest valence and arousal errors and improves CLIPScore over EmotiCrafter and the GRPO-only variant.Its IQA score is lower than several baselines.Source: §V · p. 12
Loading 2609.12830v1…