Source-linked AI summary
Position: Universal Aesthetic Alignment Narrows Artistic Expression
Wenqi Marshall Guo, Qingyun Qian, Khalad Hasan, Shan Du
TL;DR
The paper asks whether general aesthetic alignment conflicts with explicit user intent, especially for anti-aesthetic requests. It constructs wide-spectrum prompts and evaluates generation and reward models, finding that aligned systems favor conventional aesthetics over prompt fidelity. The authors conclude that this can narrow users’ expressive control.
Problem
General aesthetic alignment can conflict with instruction following, raising the question of whether image models prioritize a universal preference over explicit user intent.
Source: §1 · p. 1Method
The paper constructs wide-spectrum aesthetics prompts and evaluates image-generation models and reward models on prompt adherence and aesthetic judgments.
Source: §3.3 · p. 6Results
Reward models perform poorly at selecting images that better match wide-spectrum prompts, while generation models often favor conventionally aesthetic outputs over explicit instructions.
Source: §4.1 · p. 6, §4.2 · p. 7Takeaways & Limitations
The authors conclude that universal aesthetic alignment can suppress legitimate creative expression and narrow users’ ability to express dissenting aesthetic choices.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Over-aligning image generation models to a generalized aesthetic preference conflicts with user intent, particularly when "anti-aesthetic" outputs are requested for artistic or critical purposes. This adherence prioritizes developer-centered values, compromising user autonomy and aesthetic pluralism. We test this bias by constructing a wide-spectrum aesthetics dataset and evaluating state-of-the-art generation and reward models. This position paper finds that aesthetic-aligned generation models frequently default to conventionally beautiful outputs, failing to respect instructions for low-quality or negative imagery. Crucially, reward models penalize anti-aesthetic images even when they perfectly match the explicit user prompt. We confirm this systemic bias through image-to-image editing and evaluation against real abstract artworks. Our code, fine-tuned models, and datasets are available on our meta-expression intentionally anti-aesthetics webpage: https://weathon.github.io/icml2026_position/.
1. Introduction
Image-model alignment combines instruction following with general aesthetic preference, but these objectives can conflict when a user requests imagery outside mainstream norms. The authors argue that universal aesthetic alignment can override user intent and limit expressive agency.
1. Introduction
- Alignment fine-tunes image-generation models with human feedback, often using reward models to optimize general aesthetic preferences alongside instruction following.The authors argue that prioritizing a universal aesthetic standard over explicit user instructions can undermine user autonomy and aesthetic pluralism.Source: §1 · p. 1
- The Scream scores 5.23 on HPSv3, while typical high-aesthetic AI images can score around 10–15.The authors use this contrast to illustrate a mismatch between aesthetic scoring and recognized artistic significance.Source: Figure 1 · p. 1
- The authors describe this behavior as reflecting alignment objectives that prioritize generalized aesthetic preferences, rather than a technical failure.They argue that imposing such a preference over user instructions may limit autonomy, expressive agency, personalization, and aesthetic pluralism.Source: §1 · p. 1
- Reward models can score conventionally appealing images above images that better match the user’s intended wide-spectrum aesthetic output.Figure 2 shows this comparison using HPSv3 and the wide-spectrum aesthetics prompt.Source: Figure 2 · p. 2
2. Backgrounds and Related Works
The paper treats deliberate departures from dominant aesthetics as legitimate expression and argues that preference alignment can narrow it. It situates this concern among prior critiques of AI preference alignment and identifies several mechanisms by which universal aesthetic standards can constrain user intent.
2.1. The Role of Wide-Spectrum Aesthetics
- Wide-spectrum aesthetics means intentionally deviating from dominant conventions in response to explicit user instructions, including surrealism, clashing colors, or negative emotions.The term excludes unintended model errors and does not itself imply unsafe content.Source: §2.1 · p. 2
- Aesthetic judgments shift across contexts, and movements such as Fauvism, Expressionism, and Abstract art were initially rejected before gaining recognition.The paper also notes that deliberately ugly art can serve satire and social critique.Source: §2.1 · p. 2
2.2. Previous Concerns with AI Preference Alignment
- Prior critiques connect preference alignment to developer control, value pluralism, ideological bias, and risks to creativity, while AesBiasBench studies personalized image-aesthetic assessment.The related work also discusses AttriCtrl’s explicit attribute control, which does not address intentional ugliness.Source: §2.2 · pp. 2–3
- Earlier image-generation studies raised aesthetic-bias concerns, but the paper says they did not systematically test large-scale generative and reward models together.Examples discussed include the VSF pilot, personalized adapters, and reward models that may penalize valid abstract or emotionally negative imagery.Source: §2.2 · p. 3
2.3. Previous Alignment Works
- Existing benchmarks assess prompt following, general aesthetics, or demographic fairness, but leave unclear how evaluators weigh prompt fidelity against general preference.The paper distinguishes anti-aesthetics from fairness generation: it assumes no distribution-level ground truth, only that systems follow user preferences.Source: §2.3 · p. 3
2.4. Risks of Universal Aesthetic Alignment
- The paper frames universal alignment as a sequence of concerns spanning how preferences are defined, learned, optimized, and expressed in generated content.It identifies value capture, developer-versus-user preference, inherited bias, and individual-versus-collective preference among its concerns.Source: §2.4 · pp. 3–5
- A generalized standard can sanitize explicitly unconventional requests, while its encoded collective preference can align users to the model—a process the authors call reversed alignment.The paper argues that inherited biases can constrain visual preferences beyond demographic categories, including color, lighting, and style.Source: §2.4 · p. 4
- Reducing aesthetics to a reward score can shift the training goal toward maximizing that score and outsource users’ aesthetic choices to a reward model.The paper describes this simplification of a rich value as value capture.Source: §2.4 · p. 5
3. Experiments
The experiments construct prompts that request non-mainstream aesthetic effects, generate comparison images across several model families, and evaluate both instruction adherence and aesthetic scoring. The setup uses multiple reward models and judges to assess image quality, prompt matching, and wide-spectrum effects.
3. Experiments
- The experiments proceed through prompt preparation, image generation, and image evaluation.The investigation tests whether generations follow user-specified wide-spectrum aesthetics prompts or default to conventionally high-quality outputs.Source: §3 · p. 5
3.1. Prompt Generation
- The authors combine COCO base captions with 12 VisionReward aesthetic dimensions and their “bad” descriptions to construct prompts encouraging typically undesirable attributes.For 300 randomly selected base prompts, they sample 2–4 dimensions and use Qwen3-VL to generate the wide-spectrum prompt.Source: §3.1 · p. 5
3.2. Image Generation
- The evaluation spans Flux, SDXL, SD3.5 Medium, and Nano Banana, including base models and variants aligned for aesthetics or prompt following.The model set includes DanceFlux, PrefFlux, Flux Krea, Playground, and two FlowGRPO-aligned SD3.5M variants.Source: §3.2 · p. 5
- Each model generates an image from the original prompt and another from the wide-spectrum prompt; Nano Banana generations are retried if image creation fails.The original-prompt image is denoted Io and the wide-spectrum-prompt image Ia.Source: §3.2 · pp. 5–6
- Table 1 reports statistical comparisons of aesthetics-aligned models with their base models, marking p-values below 10^-5 and 10^-10.The caption identifies the comparison and significance markers but does not provide the table’s individual results.Source: Table 1 · p. 6
3.3. Evaluation and Metrics
- The authors fine-tune Qwen3-VL-4B-Instruct on VisionReward to judge per-dimension aesthetic effects with prompt-independent outputs.The resulting judge is denoted J(I, d), where I is the image and d is the evaluated dimension.Source: §3.3 · p. 6
- Reward-model scores are computed for both images under both the original and wide-spectrum prompts, separating image-quality assessment from guided recognition of requested aesthetics.The authors also compute BLIP scores under the wide-spectrum prompt to check whether images retain the main concept while incorporating requested modifications.Source: §3.3 · p. 6
- The reward-model set includes preference-aligned models and CLIP-L and BLIP-L matching baselines; LLM judgments are checked against human evaluations with quadratic Cohen’s kappa of 0.80.Qwen3-VL-235B provides ground-truth pairwise judgments, and GPT-5-Chat is an external baseline.Source: §3.3 · p. 6
4. Results and Discussion
Across reward models, image generators, controlled emotion tests, and real artworks, the results show that aesthetic alignment can conflict with following prompts for anti-aesthetic or negative-emotion imagery. The tested reward models also favor polished or mainstream-preference images over some prompt-matching alternatives.
4.1. Reward Models
- Most reported p-values were below 1 × 10−5, with tests indicating that alignment toward generalized aesthetic goals conflicts with faithful instruction following, especially on wide-spectrum prompts.The tests compared aligned and base models on scores and success counts.Source: §4.1 · p. 6
4.3. A Pin-Pointed Test for Emotional Bias
- HPSv3 consistently scored positive-emotion images higher despite negative-emotion prompts, and reward models were strongly biased against images matching negative emotions.The controlled test created negative-emotion versions of a happy image while aiming to keep other elements unchanged.Source: Figure 5 · p. 7, §4.3 · p. 7
- DanceFlux usually produced neutral or positive faces for negative-emotion prompts, although it could generate happy faces; across models, DanceFlux handled negative emotions poorly while Flux Krea and SD 3.5 Large followed them better.The broader test used 120 prompts instantiated with happy, sad, fearful, or angry emotions.Source: §4.3 · p. 7, Table 5 · p. 9
4.4. Validation on Real Images
- The real-photo dataset used AVA categories treated as intentional stylistic choices and an LLM workflow to curate 2,928 anti-aesthetic images with paired anti-aesthetic and clean prompts.The workflow retrieved 6.27K images before filtering.Source: §4.4 · pp. 7–8
4.2. Image Generation Models
- Within model families, preference-aligned generators generally performed worst on wide-spectrum prompts, while Flux Krea performed best in the Flux family and instruction alignment gave only a weak benefit.Base models could still handle anti-aesthetic prompts, supporting the authors’ interpretation that the failures reflect alignment-induced bias rather than inability to follow instructions.Source: §4.2 · pp. 6–7, Table 2 · p. 8
4.4. Validation on Real Images
- HPSv3 scored clean generated images 5.90 points higher on average than the original anti-aesthetic photos, with gaps of 13.2 for Analog Degradation and around 8 for Intentional Blur and Exposure Extremes.The comparison conditioned both images on the anti-aesthetic prompt; Figure 6 shows examples of the largest differences.Source: Figure 6 · p. 8, §4.4 · pp. 8–9
4.1. Reward Models
- Reward models often selected the worse-fitting image under wide-spectrum prompts, sometimes performing below random guessing, while unaligned BLIP and CLIP identified the better-fitting image.The authors attribute this pattern to biased alignment rather than a failure of complex prompt understanding.Source: §4.1 · p. 6, Table 3 · p. 8
4.4. Validation on Real Images
- Across ∼10K LAPIS paintings, real art averaged 5.86 on HPSv3, placing 10/12 on the Aug 2025 leaderboard; BLIP caption matching was 0.996 versus 0.086 for shuffled pairs.The authors report that reward models understood the artwork content but penalized works that deviated from mainstream taste.Source: §4.4 · p. 9, Figure 7 · p. 9
5. Alternative Positions and Rebuttal
A default aesthetic that pleases the majority can conflict with user intent and constrain artistic expression. Supporting a wider range of aesthetics need not compromise access to conventionally high-quality images.
5. Alternative Positions and Rebuttal
- Only clarity might be a technical flaw; emotion, realism, and brightness are stylistic or artistic choices, and low clarity can convey emotion, motion, or narrative.The authors argue that constraining these dimensions limits user expression and control.Source: §5 · p. 9
- Overriding explicit prompts with sanitized outputs can marginalize users with niche needs and communicate that their intended aesthetic is wrong.The authors describe this as reinforcing assimilation toward the majority.Source: §5 · p. 9
- Supporting wide-spectrum aesthetics can enlarge expressive range without reducing the quality of conventionally good images.The authors say users can still obtain diverse aesthetics and high-quality traditional outputs.Source: §5 · p. 9
6. Conclusion
The authors conclude that aesthetic alignment suppresses legitimate creative expression by favoring conventionally beautiful outputs and penalizing images that follow wide-spectrum prompts.
6. Conclusion
- Aesthetic alignment suppresses creative expression: generation models override explicit instructions, reward models penalize prompt-faithful images, and historical artworks score below AI-generated images.The authors call optimization toward an imagined average “reversed alignment” and argue that it narrows admissible expression.Source: §6 · p. 9