Source-linked AI summary
ReART: Reference-Guided Retrieval and Refinement for Emotion-Aware Art Generation
Qianqian Tang, Jiayi Gao, Ting Lei, Yang Liu
TL;DR
Emotion-aware artistic generation must coordinate content, style, and emotion despite captions that underspecify the visual details expressing affect. ReART uses structured field-wise retrieval and AAS-driven constrained refinement, achieving 2nd place with AAS 1.00 and overall score 0.78 in AffectiveArt 2026 Track 1.
Problem
Artistic captions conflate content, style, emotion, and fine-grained visual attributes, making affective details such as brushwork, composition, and tonal atmosphere difficult to ground.
Method
ReART decomposes captions and EmoArt annotations into structured visual fields, retrieves role-specific references, and uses AAS-driven diagnosis with constrained repair plans for targeted editing.
Results
2nd place: ReART ranks 2nd in AffectiveArt 2026 Grand Challenge Track 1 with overall score 0.78, perfect AAS 1.00, and FID 77.47.
Takeaways & Limitations
ReART supports fine-grained prompt-image alignment by combining field-specific visual evidence with targeted correction of axis-specific failures.
Takeaways & Limitations
Local200 uses an approximate GPT-5.4 AAS evaluator and its results are not directly comparable to official leaderboard scores.
Abstract
from arXiv · showhide
Emotion-aware artistic image generation requires a model to satisfy semantic content, artistic style, and target emotion simultaneously. The key challenge is that artistic captions conflate these axes into underspecified free-form text, making fine-grained visual attributes such as brushwork, composition, and tonal atmosphere difficult to ground concretely. We present ReART, a reference-guided retrieval and refinement framework. Our method decomposes test captions and each image annotation in the EmoArt database into structured visual fields, and performs field-wise retrieval over subject, layout, brush-line, and tone-mood dimensions to retrieve role-specific visual references that supply the perceptual detail text alone cannot convey; these references are used alongside a structured prompt for initial synthesis. For samples where any Attribute Alignment Score (AAS) axis falls below threshold, an AAS-driven refinement loop diagnoses failures, constructs constrained repair plans specifying elements to keep, errors to fix, and operations to avoid, routes references by correction purpose, and performs controlled editing under structural preservation constraints. Our system ranks 2nd in Track 1 of the AffectiveArt 2026 Grand Challenge, achieving a perfect AAS of 1.00 and an overall score of 0.78. Code is available at https://github.com/oceanflowlab/ReART.git.
1 Introduction
ReART addresses underspecified emotion-aware art captions and the difficulty of satisfying content, style, emotion, and fine-grained attributes simultaneously. Its structured retrieval and AAS-driven refinement framework achieves perfect alignment on the reported challenge evaluation.
- Challenge: Emotion is conveyed through fine-grained visual attributes, but free-form captions compress content, style, emotion, and execution details into underspecified descriptions.Examples include brushwork, color, composition, line, texture, and tonal atmosphere.
- Challenge: Single-pass generation can realize content and layout while missing style-specific execution, and blind resampling or global editing risks damaging correct regions.The paper motivates targeted correction of axis-specific failures while preserving already aligned dimensions.
- Approach: ReART decomposes captions into independently addressable visual fields and retrieves field-aligned artistic references to ground abstract affective descriptions in perceptual evidence.The fields include subject, brushwork, color tone, composition, mood, and material surface.
- Approach: Its AAS-driven refinement loop diagnoses failing axes, creates keep/fix/avoid/priority repair plans, routes purpose-specific references, and performs constrained editing.The loop is designed to correct residual failures without disrupting correctly realized dimensions.
- Results: 1.00 AAS: ReART achieves perfect alignment across all reported sub-axes and ranks 2nd in AffectiveArt 2026 Grand Challenge Track 1.The reported overall score is 0.78, with FID 77.47.
2 Related work
Prior emotion-aware generation and reference-guided methods provide controllable affect or visual conditioning, but generally do not address fine-grained artistic emotion expression with dimension-specific references and repair.
- Affective Artistic Image Generation: Existing emotion-aware generators model discrete or continuous affect and can edit images toward target emotions.Examples include emotion spaces, valence-arousal control, and emotion adapters.
- Affective Artistic Image Generation: These approaches largely overlook brushwork, composition, color, and tonal atmosphere as the visual attributes through which artistic emotion is expressed.ReART addresses this gap by decomposing emotional targets into independently addressable fields.
- Reference-Guided Generation: Reference-guided methods inject visual concepts that text cannot fully specify, but commonly treat references as holistic templates rather than routing them by visual dimension.ReART instead uses field-specific retrieval and purpose-specific references.
- Agentic Refinement: Agentic refinement methods critique images and generate edit signals, but mainly target user-instruction-driven editing with explicitly specified goals.ReART targets emotion-aware generation with structured repair plans and preservation of correctly realized axes.
3 Methodology
ReART uses a two-stage pipeline: structured, field-wise reference retrieval guides initial synthesis, then AAS-driven diagnosis and constrained editing refine axis-specific failures.
- Stage I: Stage I converts captions and EmoArt annotations into structured visual records, retrieves field-aligned references, and supplies them with structured descriptions to guide generation.The prompt directs the generator to use each reference for the visual dimensions it best covers.
- Stage II: Stage II evaluates the generated image, diagnoses alignment failures, and applies targeted constrained refinement to repair failing dimensions.The framework is explicitly organized around diagnosis and controlled editing rather than undirected regeneration.
3.1 Retrieval-Guided Artistic Generation
The retrieval stage represents artistic captions and references as structured fields, searches style-specific pools field by field, and combines selected evidence in a structured multimodal generation prompt.
- Structured Visual Record Construction: GPT-5.4 decomposes captions and reference annotations into eleven independently addressable visual fields for direct field-level comparison.The representation separates entangled semantic, stylistic, compositional, material, brushstroke, texture, and related signals.
- Style-Specific Pool and Field-Wise Retrieval: Style-specific reference pools reduce retrieval noise by preserving consistency in brushwork, surface treatment, and compositional conventions.References are searched within pools matched to the artistic style.
- Style-Specific Pool and Field-Wise Retrieval: Candidates receive a weighted sum of normalized field similarities, with higher weights for semantically critical fields such as subjects and composition.Line quality and mood receive lower weights, while layout-heavy samples increase composition and scene weights; the top 10 references are retained.
- Structured Generation Prompt: The structured prompt gives gpt-image-2 the original caption, target visual fields, structured records, and retrieved references with field-level usage guidance.References provide role-specific visual evidence while the caption remains the primary objective.
3.2 Agentic Constrained Refinement Loop
The refinement loop uses local AAS evaluation to diagnose axis-specific failures, plan repairs, route targeted references, and edit while preserving correct structure.
- Diagnosis: A local GPT-5.4 AAS evaluator selects images for refinement when any sub-score falls below 9.AAS evaluates content, style, and attribute alignment under the official protocol.
- Diagnosis: The diagnosis module identifies satisfied field constraints and dominant error types using the generated image, caption, structured record, and retrieval scores.
- Planner: The planner converts diagnosis into Keep, Fix, Avoid, and priority instructions for interpretable, bounded editing.Keep preserves elements such as subject identity, spatial relations, composition skeleton, and page format; Fix specifies field-level repair targets.
- Reference Routing: Reference routing selects the four highest-scoring candidates for each failing dimension from the initial top-10 references.The routed references provide axis-specific evidence without uniformly reusing all retrieved images.
4 Experiment
Experiments evaluate ReART on official and local settings, showing strong alignment and distributional results, component gains, and second-place official challenge performance.
- Evaluation Protocol: The official AffectiveArt evaluation uses 1,000 test captions and a hidden reference set of 2,000 artworks, while Local200 supports controlled comparisons.Local200 results are not directly comparable to official leaderboard scores because the official backend and hidden references are unavailable.
- Comparison with Baselines: On Local200, ReART obtains FID 147.43 and AAS 9.77, improving over the strongest relevant baselines across distributional similarity and alignment.Compared with the best baseline on each sub-axis, Content, Style, and Attribute increase to 9.88, 9.65, and 9.79, respectively.
- Ablation Study: Structured field decomposition lowers FID from 193.47 to 186.86 and raises AAS from 8.68 to 8.84 without references.The gains are concentrated more on Style and Attribute than on Content.
- Ablation Study: Field-wise reference retrieval produces the largest single improvement, lowering FID from 186.86 to 158.69 and increasing AAS from 8.84 to 9.45.The improvements are especially pronounced on Style and Attribute because references supply perceptual details unavailable in structured text alone.
- Ablation Study: AAS-driven refinement further lowers FID from 158.69 to 147.43 and raises AAS from 9.45 to 9.77, with 71.4% of refined samples converging within one iteration.Among 200 Local200 samples, 42 entered refinement and required 1.38 iterations on average; unconstrained re-editing and removing routing or constraints performed worse.
- Official Challenge Results: ReART achieves an overall score of 0.78, perfect AAS of 1.00, and FID of 77.47 on the official challenge.It ranks 2nd overall in Track 1.
5 Conclusion
ReART addresses emotion-aware artistic generation through structured field-wise retrieval and targeted, constrained refinement, achieving perfect official alignment and second-place challenge ranking.
- Conclusion: ReART decomposes captions into structured visual records and retrieves role-specific references for separate visual dimensions rather than using holistic style templates.
- Conclusion: Its AAS-driven refinement loop diagnoses failures, creates Keep/Fix/Avoid/priority plans, routes references to failing dimensions, and preserves correct structure during editing.
- Conclusion: ReART achieves a perfect AAS of 1.00 on all three official sub-axes and ranks 2nd overall in AffectiveArt 2026 Track 1.