AesCanvas: A Large-Scale Dataset and Benchmark for Aesthetic Critique and Contextual Suitability

I Introduction

• Because symbols, styles, dress, and genre conventions can change whether an image is appropriate, cultural knowledge is integral to aesthetic judgment rather than auxiliary.The introduction distinguishes contextual suitability from generic cultural question answering: context matters because it determines whether an image works as a visual choice.Source: I Introduction · pp. 1–2
• Models can produce ungrounded critiques or apply inappropriate domain priors, while suitability judgments may favor surface appeal or stylistic plausibility over cultural and communicative fit.The paper calls following generic aesthetic priors while neglecting decision-relevant context “aesthetic context bias.”Source: I Introduction · pp. 1–2, Figure 2 · p. 2
• Aesthetic specialists remain competitive on selected critique metrics, but all score below 30% on ContextCanvas versus 91.69% for the strongest evaluated general-purpose model.Reference-based metrics only partially capture critique quality, and aesthetic specialization does not reliably transfer to contextual judgment; correct decisions may also lack grounding in decisive visual evidence.Source: I Introduction · p. 2
• CritiqueCanvas provides 519,136 instruction–response pairs from 54,300 multi-domain images, while ContextCanvas benchmarks contextual suitability across 301 expert-curated cases.The suite evaluates closed-source, open-weight, and aesthetic-specific MLLMs and releases data, prompts, rationales, provenance records, and evaluation code.Source: I Introduction · p. 2

A Image Aesthetic Assessment

• Prior IAA methods predict aesthetic judgments through classification, ranking, or score distributions, but largely use intrinsic or task-fixed criteria rather than contextual suitability.This leaves open whether an image suits a particular purpose, audience, cultural setting, or domain convention.Source: A Image Aesthetic Assessment · p. 2

B Aesthetic Multimodal Models

• Aesthetic multimodal models have progressed from aesthetic perception benchmarks and instruction tuning to language-based scoring, professional critique, multi-attribute analysis, and unified assessment.Prior work includes general-purpose MLLM benchmarks and aesthetic instruction tuning (Huang et al. 2024b,a; Zhou et al. 2024), language-defined rating and joint scoring-and-interpretation (Wu et al. 2024; Zhang et al. 2025), and professional critique and multi-attribute assessment (Qi et al. 2025; Cao et al. 2025b,a).Source: B Aesthetic Multimodal Models · p. 2
• Existing evaluation centers on aesthetic perception, scoring, reference-aligned critique, or domain-specific analysis, whereas AesCanvas tests whether aesthetic articulation transfers to contextual decisions.An attractive or stylistically plausible image may still conflict with its intended use. Recent multimodal foundation models have also explored semantic understanding and language generation over specialized or corrupted visual representations.Source: B Aesthetic Multimodal Models · p. 3

• Contextual multimodal reasoning requires combining visual evidence with task conditions, external knowledge, cultural conventions, and criterion-specific constraints.Source: C Contextual Multimodal Reasoning · p. 3
• Prior benchmarks examine real-world visual reasoning, context-dependent functional inference, and flexible judgment under varying criteria (Yin et al. 2026; Zhang et al. 2026; Xiong et al. 2026; Gao et al. 2024).Source: C Contextual Multimodal Reasoning · p. 3
• Grounding and robustness studies test localized evidence support, failures under illusions or adversarial evidence, and region-specific or multilingual cultural evidence (Li et al. 2026; Wu et al. 2026b; Cai et al. 2024; Hou et al. 2026; Zhou et al. 2026; Moratelli et al. 2026; Satar et al. 2025; Singh et al. 2026).Source: C Contextual Multimodal Reasoning · p. 3
• These works mainly assess factual correctness, grounding, functional inference, or specified-criterion adherence, rather than when contextual evidence should outweigh surface appeal in a concrete visual-use decision.ContextCanvas targets this intersection through use-oriented aesthetic suitability judgments grounded in both visual and contextual evidence.Source: C Contextual Multimodal Reasoning · p. 3

A Dataset Overview

• CritiqueCanvas contains 519,136 instruction–response pairs from 54,300 images, while ContextCanvas evaluates contextual suitability through 301 expert-reviewed cases.The two components branch from a shared multi-domain image pool and address complementary aspects of aesthetic intelligence.Source: A Dataset Overview · p. 3, B Dataset Curation · p. 4

B Dataset Curation

• The shared image pool contains 21,294 photographs, 17,939 paintings, and 15,067 virtual images collected from public web and cultural-heritage sources.The curation process removes low-quality, semantically uninformative, non-compliant, and duplicate samples; source URLs and provenance metadata are retained.Source: B Dataset Curation · p. 4
• CritiqueCanvas annotations combine shared aesthetic dimensions with domain-specific criteria, selecting only the dimensions relevant to each image.Multiple MLLMs produce structured annotations converted into multi-turn conversations covering description, evaluation, interpretation, technical analysis, and constructive improvement; an independent verifier audit found 95.3% acceptance and 97% raw agreement across 1,000 retained pairs.Source: B Dataset Curation · p. 4
• Four PhD-level researchers developed and reviewed 1,060 candidate ContextCanvas cases, each framing an image in a plausible use scenario and asking whether it is an appropriate visual choice.Cases were retained against seven criteria, including context dependence, visually grounded decisive evidence, answerability from perceptible evidence, scenario naturalness, and protection against wording or label shortcuts.Source: B Dataset Curation · p. 4
• The final ContextCanvas benchmark has 301 cases: 300 single-image and one paired-image case, with 291 binary and 10 three-way questions.Each case includes the visual input, scenario, context, options, gold label, source-grounded rationale, and provenance; human reviewers finalized inclusion decisions and labels.Source: B Dataset Curation · p. 4

C Evaluation Protocol

• Models receive identical inputs and instructions without web search, retrieval, or auxiliary recognition tools for both benchmark tasks.Table 1 summarizes the metrics.Source: C Evaluation Protocol · p. 4

Long-form Aesthetic Critique Generation.

• Critique generation uses 6,000 balanced, open-ended samples across photography, painting, and virtual imagery, with image-level splits keeping related conversations together.Samples require aesthetic analysis or image-grounded improvement suggestions.Source: Long-form Aesthetic Critique Generation · p. 4
• Critiques are assessed with lexical overlap metrics BLEU (Papineni et al. 2002), ROUGE-L (Lin 2004), and METEOR (Banerjee and Lavie 2005), alongside semantic-similarity and image–text relevance metrics.BERT-F1 (Zhang et al. 2020) and SBERT-Cos (Reimers and Gurevych 2019) measure token- and sentence-level semantic similarity, while CLIPScore (Hessel et al. 2021) measures image–text relevance; the metrics are interpreted jointly, not as complete measures of aesthetic reasoning.Source: Long-form Aesthetic Critique Generation · pp. 4–5
• Supplementary response measures include average length, aesthetic-dimension coverage, and the three most frequently addressed dimensions; coverage breadth does not establish correctness or grounding.A higher coverage value does not necessarily mean the dimensions identified are relevant, accurate, or well grounded.Source: Long-form Aesthetic Critique Generation · p. 5

Contextual Aesthetic Suitability Judgment.

• For each ContextCanvas case, models choose one fixed option from an image and realistic use scenario; explanations are qualitative only, and missing, conflicting, unparsable, or non-unique answers count as incorrect.A prediction is correct only when a unique option is parsed and matches the gold answer.Source: Contextual Aesthetic Suitability… · p. 5
• ContextCanvas reports exact-match accuracy on 301 cases and MacroF1 and predicted Yes Rate on 291 binary cases, excluding 10 three-way cases from those latter measures.The gold Yes Rate is 38.14%; the predicted rate indicates a tendency toward over-acceptance or over-rejection.Source: Contextual Aesthetic Suitability… · p. 5

A Experimental Setup

• The evaluation compares closed-source frontier, open-weight general, and aesthetic-specific MLLMs; Qwen3-VLFT is adapted on CritiqueCanvas and evaluated only on critique generation.Locally deployed models receive images resized or padded to 448 × 448 pixels, with deterministic decoding when supported and a 256-token maximum for critique generation.Source: A Experimental Setup · p. 5

B Main Results

• Closed-source ContextCanvas accuracy ranges from 44.19% to 91.69%, versus 42.19% for strongest open-weight Qwen3-VL-Instruct; Claude Opus 5 and GPT-5.5 improve over predecessors by 18.93/22.49 and 11.63/15.62 accuracy/Macro-F1 points.The within-family gains are consistent with improved contextual aesthetic judgment in newer frontier models.Source: B Main Results · pp. 5–6
• All evaluated aesthetic-specific models score below 30% ContextCanvas accuracy: ArtQuant-APDD reaches 29.57%, versus 42.19% for Qwen3-VL-Instruct and 91.69% for Claude Opus 5.ArtiMuse, Q-SiT, and AesExpert score 19.60%, 19.60%, and 18.27%, respectively, showing that aesthetic specialization does not reliably transfer to culturally, communicatively, or domain-constrained suitability judgment.Source: B Main Results · p. 6
• Aesthetic-specific models predict Yes in 59.11%–76.98% of binary cases against a 38.14% gold rate, while weaker models can fall below the 61.86% always-No baseline through over-acceptance.The shared marginal tendency may arise from different instance-level failures; frontier models reach 91.69% accuracy on the full benchmark, indicating performance beyond majority-label guessing.Source: B Main Results · p. 6
• CritiqueCanvas has no metric-family winner: Gemini 3.1 Pro leads SBERT-Cos at 0.848, LLaVA-OneVision leads BERT-F1 at 0.724 and CLIPScore at 0.319, and Qwen3-VLFT leads BLEU at 12.23, ROUGE-L at 0.297, and METEOR at 0.296.Relative to base Qwen3-VL, Qwen3-VLFT gains 7.11 BLEU, 0.065 ROUGE-L, and 0.022 METEOR, but SBERT-Cos falls from 0.827 to 0.787 and CLIPScore from 0.313 to 0.285.Source: B Main Results · p. 6
• Aesthetic-specific models remain competitive on selected critique metrics: ArtiMuse scores 0.718 BERT-F1 and UniPercept scores 0.318 CLIPScore, while AesExpert and Q-SiT are weak across most metrics.ArtQuant reaches 10.68 BLEU and 0.280 ROUGE-L; overall critique performance varies across lexical, semantic, and image-grounding criteria.Source: B Main Results · p. 6

Validity of Critique Metrics

• Direct critique-quality ratings do not consistently track reference-similarity metrics, so lexical and semantic overlap capture only part of critique quality.The evaluation uses 100 randomly sampled CritiqueCanvas cases, with human ratings of grounding, specificity, criterion appropriateness, and usefulness alongside an MLLM judge.Source: Validity of Critique Metrics · pp. 6–7, Table 4 · p. 7

Culture-Grounded Counterfactual Sensitivity

• General-purpose MLLMs show positive counterfactual net updating, while both aesthetic specialists update less than every general-purpose model.Across 36 paired variants that change a decisive cue while holding the scenario and overall presentation fixed, general-purpose models’ net correct update is 25.00–30.56; ArtQuant shows almost no measurable response. The audit measures directional updating, not accuracy on an additional split.Source: Culture-Grounded Counterfactual… · p. 7, Table 5 · p. 7

Transfer from Aesthetic Specialization

• Specialization improves ArtQuant and AesExpert over their base models, but yields no measurable gain for ArtiMuse or Q-SiT.Strong alignment with CritiqueCanvas metrics also does not consistently translate into ContextCanvas accuracy.Source: Transfer from Aesthetic Specialization · p. 7, Figure 4 · p. 7

Decisive-Cue Grounding

• ArtiMuse falls from 21% accuracy to 3% Evidence-Grounded Accuracy, which requires correctness, decisive-cue identification, and contextual explanation.On 100 balanced ContextCanvas cases, general-purpose models preserve much of their accuracy under this stricter criterion; fluent rationales may still miss the culture-bearing evidence determining suitability.Source: Decisive-Cue Grounding · p. 7, Table 6 · p. 7

V Conclusion

• Aesthetic specialists can remain competitive on critique metrics yet show low accuracy and over-acceptance on ContextCanvas, with tuning benefits and use of decisive contextual evidence proving inconsistent.These findings span closed-source, open-weight, and aesthetic-specific models.Source: V Conclusion · p. 7
• AesCanvas comprises CritiqueCanvas and ContextCanvas to evaluate aesthetic critique and contextual suitability.Source: V Conclusion · p. 7
• Culturally situated suitability is a distinct training and evaluation objective requiring integration of visual evidence with cultural knowledge, intended use, and domain-specific constraints.Source: V Conclusion · p. 8

S1 Supplement Overview and Positioning

• The supplement’s Section S2 provides dataset, prompt, evaluation, and implementation materials.Source: S1 Supplement Overview and Positioning · p. 12
• Section S3 presents model-output, robustness, and transfer analyses.Source: S1 Supplement Overview and Positioning · p. 12

S1.1 Positioning Relative to Prior Datasets

• Use-context judgment requires an auditable decision about suitability for a stated purpose, audience, or convention, not generic context reasoning or intrinsic aesthetic scoring.Source: S1.1 Positioning Relative to Prior… · p. 12
• Table S1 compares published datasets’ media coverage and aesthetic-supervision affordances, counting support only when explicitly documented and marking limited or indirect support separately.Source: S1.1 Positioning Relative to Prior… · p. 12, Table S1 · p. 12
• The comparison spans photographic-aesthetics datasets (Murray, Marchesotti, and Perronnin, 2012; Kong et al., 2016; Yang et al., 2022), AI-generated-image quality datasets (Li et al., 2024), and multimodal aesthetic-critique and assessment resources (Huang et al., 2024b,a; Qi et al., 2025; Cao et al., 2025; Liu et al., 2026).AVA provides score distributions and semantic/style labels; AGIQA-3K provides perceptual-quality and text–image-alignment scores; PhotoCritique offers broad photographic feedback rather than a fixed multi-dimensional annotation schema.Source: S1.1 Positioning Relative to Prior… · p. 12
• AesCanvas couples long-form critique with a closed-form benchmark where cultural, narrative, functional, or domain-specific evidence can change suitability for a concrete use.Source: S1.1 Positioning Relative to Prior… · p. 12
• Prior resources already provide substantial supervision for intrinsic aesthetic perception and critique.Source: S1.1 Positioning Relative to Prior… · p. 12

S2.1.1 AesCanvas Overview

• The shared pool contains 54,300 images—21,294 photographs, 17,939 paintings, and 15,067 virtual images—with CritiqueCanvas conversations grouped before splitting and ContextCanvas reserved as a fixed diagnostic set.The collection spans publicly accessible web, museum, and cultural-heritage sources; low-quality, duplicate, and other unsuitable candidates were removed.Source: S2.1.1 AesCanvas Overview · p. 13

S2.1.2 CritiqueCanvas

• CritiqueCanvas selects image-relevant dimensions from ten shared aesthetic criteria, supplemented by domain-specific concerns for photography, painting, and virtual imagery.The criteria include content and narrative, composition, color, lighting, lines and brushstrokes, style, emotion, technique, symbolism, and visual appeal.Source: S2.1.2 CritiqueCanvas · p. 13

S2.1.3 ContextCanvas

• ContextCanvas retains 301 cases (300 single-image, one paired-image; 302 unique images); only images, question, and fixed options are model input, while labels, rationales, and development metadata are excluded.Source: S2.1.3 ContextCanvas · p. 14
• Expert review excluded candidates that relied on coarse emotional shortcuts, lacked visual necessity, or had no uniquely compelled gold label; plausible wording or intended answers alone did not ensure retention.The examples are illustrative, not estimates of the distribution of rejection reasons.Source: S2.1.3 ContextCanvas · pp. 14–15

S2.1.4 Provenance and Governance

• The datasets reflect their source pool and contributor expertise, so ContextCanvas is not exhaustive of cultures, audiences, or design practice, and CritiqueCanvas references are not the only valid critiques.ContextCanvas rationales remain open to historically or culturally contestable interpretations; CritiqueCanvas permits multiple valid analyses of an image.Source: S2.1.4 Provenance and Governance · p. 15

S2.2.1 CritiqueCanvas Prompts

• CritiqueCanvas prompts first ask a planner to select visually evidenced dimensions, then an annotator to produce a six-question dialogue with concrete image evidence and an improvement suggestion.The prompt specifies domain-conditioned supplements, including photography concerns such as exposure and depth of field.Source: S2.2.1 CritiqueCanvas Prompts · p. 18
• CritiqueCanvas prompts guide virtual-image critiques with shared aesthetic dimensions and relevant character, scene, narrative, and stylization cues, then request six evidence-based Q&A pairs including improvement advice.Responses should avoid generic comments.Source: S2.2.1 CritiqueCanvas Prompts · p. 19
• Painting prompts add medium-relevant features such as brushwork, surface, genre, symbolism, and emotional expression to shared dimensions, while requiring six evidence-grounded Q&A pairs and improvement advice.The prompt discourages generic comments.Source: S2.2.1 CritiqueCanvas Prompts · p. 20

CritiqueCanvas Verification Prompt

• CritiqueCanvas verification accepts records only when instruction consistency, visual grounding, and response quality pass with no blocking issue; any failed criterion requires rejection and a corresponding issue.Blocking issues include task mismatch or failure, material visual hallucination, inadequate generic or incomplete responses, and observable safety or compliance problems.Source: CritiqueCanvas Verification Prompt · pp. 21–22

ContextCanvas · Evaluation Prompt

• ContextCanvas asks models to judge suitability for a stated real-world aesthetic context from the supplied visual and prompt, considering relevant cultural and narrative meaning.The requested response gives one option ID, a concise reason, and confidence; exact-match grading uses only the predicted option ID.Source: ContextCanvas · Evaluation Prompt · p. 23
• ContextCanvas adds 36 manually checked counterfactual edits that replace decisive incompatible cues while preserving non-target content, plus text-only, neutral-caption, and decisive-visible-cue diagnostic conditions.GPT Image 2 generates the edits, which are regenerated if checks find them invalid.Source: ContextCanvas · Evaluation Prompt · p. 23

S2.2.3 Evaluation and Judge Prompts

• Human critique ratings aggregate four binary, within-batch comparative criteria into a 1–5 score, while Claude Opus 5 assigns four criterion scores and a holistic 1–5 score to anonymized, permuted critiques.The model judge withholds the reference critique; its holistic score is reported as the MLLM score in Table 4 of the main paper.Source: S2.2.3 Evaluation and Judge Prompts · p. 24

Direct Critique-Quality Judge Prompt

• The direct critique judge scores grounding, specificity, criterion appropriateness, analytical usefulness, and overall quality, rewarding concrete, medium-appropriate interpretation rather than length or reference similarity.It evaluates five anonymous critiques of the same image and penalizes generic templates.Source: Direct Critique-Quality Judge Prompt · p. 24
• For 100 ContextCanvas cases, blinded human evaluators require a correct answer, identification of the decisive visible cue or warranted equivalent, and a correct link between that cue and the stated use for Evidence-Grounded Accuracy.Disagreements are discussed, and valid equivalent reasoning need not lexically overlap with the expert rationale.Source: Direct Critique-Quality Judge Prompt · p. 25

S2.3.1 Dataset-Quality Rubrics

• CritiqueCanvas audits consistency, grounding, specificity, coherence, and usefulness, rejecting blocking failures; ContextCanvas retains cases only if all seven suitability, visual-evidence, naturalness, answerability, and anti-shortcut criteria hold.ContextCanvas criteria also require context dependence, visual necessity, and decisive evidence; Figure S3 gives boundary examples for target difficulty, visual necessity, and answerability.Source: S2.3.1 Dataset-Quality Rubrics · pp. 25–26
• 97.0% of audited pairs had no blocking-issue flag from any reviewer, while 95.3% passed the final accepted-set criteria after asset-level validation.The 97.0% figure is unanimous no-blocking-issue agreement, not chance-corrected reliability; 17 otherwise acceptable pairs were later removed for failing the image-clarity threshold.Source: S2.3.1 Dataset-Quality Rubrics · p. 25

S2.3.2 Critique Evaluation

• Critique evaluation combines lexical overlap (BLEU, ROUGE-L, METEOR), semantic similarity (BERT-F1, SBERT cosine), and image relevance (CLIP cosine), interpreting all six metrics jointly.The reported CLIP value is raw mean cosine, without max(·, 0) truncation or 2.5× scaling.Source: S2.3.2 Critique Evaluation · pp. 26–27
• Lexical diagnostics measure verbosity, breadth, and controllability—not correctness, relevance, or grounding; critique ratings assess grounding, specificity, criterion appropriateness, usefulness, and holistic quality.The MLLM judge assigns four criterion-specific scores and a fifth holistic score, each from 1 to 5.Source: S2.3.2 Critique Evaluation · pp. 27–28

S2.3.3 Contextual-Suitability Evaluation

• ContextCanvas computes accuracy over 301 cases, while Macro-F1 and predicted Yes Rate use 291 binary cases; the gold Yes Rate is 38.14%.Evidence-Grounded Accuracy additionally requires a correct label, decisive visible cue, and correct cue-to-use link; counterfactual updates use 36 matched pairs and are reported with directional counts, not as standalone causal scores.Source: S2.3.3 Contextual-Suitability Evaluation · pp. 28–29

S2.4.1 Model and Inference Configuration

• All models receive identical task content and 448 × 448 RGB images, with deterministic decoding where supported and checkpoint-native multimodal serialization.Local runs decode greedily for at most 256 tokens; closed-API runs use temperature 0, four concurrent requests, a 180-second timeout, and up to three transport retries.Source: S2.4.1 Model and Inference Configuration · p. 29

S2.4.2 Fine-Tuning Configuration

• Qwen3-VL-FT is evaluated for critique generation by loading its PEFT adapter onto Qwen3-VL-8B-Instruct without merging.Evaluation uses bfloat16, SDPA, the base processor’s multimodal template, a 448 × 448 image, and greedy decoding capped at 256 new tokens.Source: S2.4.2 Fine-Tuning Configuration · p. 29

S2.4.3 Output Processing and Aggregation

• The ContextCanvas parser validates JSON and option IDs, accepts narrow unambiguous fallbacks, and grades valid answers without using rationales or confidence.Retries recover missing transport or model responses but cannot revise a valid answer; the pipeline computes the reported contextual and critique metrics from prediction files.Source: S2.4.3 Output Processing and Aggregation · p. 30

S3.1.1 Aggregate Behavioral Patterns

• Against a 38.14% gold Yes rate, aesthetic-specific models predicted Yes on 59.11–76.98% of cases, and Q-SiT paired its 76.98% Yes rate with 4.05 No-class F1.The strongest closed-source models predicted Yes on 20.96–30.24% of cases; GPT-5.2 was conservative at 20.96%. These class-specific profiles show how similar overall scores can conceal different error patterns. On the 100-case grounding audit, Gemini retained 79 of 83 correct decisions under EGA, GPT-5.2 55 of 65, and Qwen3.7-Plus 31 of 42; ArtiMuse retained only 3 of 21 correct labels as grounded and correctly linked answers.Source: S3.1.1 Aggregate Behavioral Patterns · pp. 30–31, Table S4 · p. 31

S3.1.2 Cross-Model Case Studies

• The HB208 and HB181 cases show that contextual suitability can hinge on cultural identity or communicative function, not surface appeal alone.For HB208, Gemini 3.1 Pro and GPT-5.2 rejected an image tied to Japanese Touhou fan culture, while Qwen3.7-Plus, ArtiMuse, and ArtQuant-APDD accepted it using surface festival cues. For HB181, roughness, darkness, and tension could serve the painting’s communicative purpose by making the workers’ coordination legible under pressure.Source: S3.1.2 Cross-Model Case Studies · pp. 31–32, Figure S4 · p. 32

S3.1.3 Failure Modes and Representative Outputs

• The examples distinguish four failure levels: ungrounded invention, domain-blind criteria, surface affect over communicative fit, and stylistic plausibility over contextual fit.Reviewed casebook subtypes include cultural-neighbor substitution, aesthetic-halo errors, and salient-composition myopia; these summarize examples, not corpus-wide frequencies. Representative outputs also distinguish grounded responses from unsupported labels, incorrect rationales, and format-sensitive answers.Source: S3.1.3 Failure Modes and… · pp. 32–33

S3.2.1 Extended Main Results

• Critique length, coverage, and prompt adherence vary independently, while generic critiques concentrate on composition, content, and emotion.Qwen3-VL produced the longest critiques at 199.07 words, whereas UniPercept had the highest dimension density at 13.26; Gemini had the broadest coverage at 6.53 dimensions, while Qwen3-VL had 95.2% focused-prompt alignment. Greater length or coverage does not establish relevance, correctness, or visual grounding.Source: S3.2.1 Extended Main Results · pp. 33–34, Table S5 · p. 36

S3.2.2 Critique Metric Validity

• GPT-5.2 and Gemini received higher direct critique-quality ratings than adapted or aesthetic-specific models, although they did not dominate all reference-similarity metrics.Lexical and semantic alignment to one expert answer only partially reflects grounding, image specificity, criterion choice, and usefulness; lower similarity can still accompany grounded, useful analysis, while close stylistic alignment can accompany generic or unsupported claims.Source: S3.2.2 Critique Metric Validity · pp. 34–35

S3.2.3 Counterfactual and Grounding Analyses

• On 36 counterfactual pairs, general-purpose models averaged 27.78 NCU versus 9.72 for specialists, while ArtiMuse’s 97.22% edited accuracy largely reflected acceptance tendency.ArtiMuse changed six decisions correctly and none in reverse, but predicted Yes for 64 of 72 original/edit images. In the separate grounding audit, it fell from 21 correct labels to only 3 grounded, correctly linked answers; its fluent, aesthetically framed rationales often missed or misbound decisive contextual cues.Source: Table S7 · p. 35, S3.2.3 Counterfactual and Grounding… · pp. 36–37, Table S8 · p. 36
• A neutral visual description improved general-purpose models by 28–34 points, while Qwen rose from 52% with images to 78% with neutral descriptions and 88% with the decisive cue explicit.Text-only results did not exceed the 61% majority baseline, and paired McNemar tests for the general-purpose models gave p < 10^-7. ArtiMuse improved partly with explicit evidence but remained far below general-purpose models, while ArtQuant stayed weak; these comparisons diagnose, but do not definitively causally decompose, error sources.Source: Table S9 · p. 37, S3.2.3 Counterfactual and Grounding… · p. 38

S3.2.4 Base-to-Specialist Transfer

• Specialization significantly improved ContextCanvas accuracy for ArtQuant and AesExpert after correction, but produced no measurable gain for ArtiMuse or Q-SiT.The paired comparisons used the same 301 cases for each base–specialist lineage, showing that specialization does not consistently improve contextual judgment.Source: S3.2.4 Base-to-Specialist Transfer · p. 38, Table S10 · p. 38