PaperlayerView exact revision on arXiv

Source-linked AI summary

Presence Is Not Faithfulness: Figurative Vehicle Intrusion in Text-to-Image Generation

Xiaoyu Ma, Chen Yang, Hao Chen

arXiv:2609.32188v1cs.CVcs.AI

TL;DR

Text-to-image models may render a figurative vehicle as a visible object instead of using it to shape the intended tenor, and presence-oriented evaluation can miss this role error. The paper introduces VISTA and V-Score to evaluate figurative faithfulness, and VISTA-Guard as a lightweight mitigation. Evaluations find vehicle intrusion across languages and figurative categories, while the mitigation reduces it.

  • Problem

    Figurative prompts require deciding whether a mentioned concept should appear as a scene entity or serve as guidance for depicting the tenor, a distinction presence-oriented evaluation may miss.

    Source: §1 · pp. 1–2

  • Method

    The paper introduces multilingual VISTA prompts annotated by Figurative Form, Mapping Mechanism, and tenor–vehicle roles, V-Score for role-aware QA evaluation, and VISTA-Guard as a lightweight mitigation.

    Source: §1 · p. 2, §3.2.2 · p. 5, §3.3 · p. 6

  • Results

    Vehicle intrusion persists across languages and figurative categories in contemporary text-to-image models, and presence-oriented evaluation can miss it.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results indicate that figurative faithfulness requires preserving the tenor while avoiding literal vehicle intrusion, and that making roles explicit can reduce this failure.

    Source: §5 · p. 9

Abstract

from arXiv · show

Text-to-image (TTI) models increasingly generate high-quality images from natural-language prompts, yet figurative language exposes a failure: a vehicle that should guide the depiction of a tenor may instead be rendered as a visible object. We call this failure Figurative Vehicle Intrusion: the intruding content is textually licensed, but it is assigned the wrong visual role, showing that visual presence is not always faithfulness and that presence-oriented evaluation can miss such errors. To study it systematically, we introduce Vehicle Intrusion and Semantic Tenor Assessment (VISTA), a multilingual benchmark of figurative prompts organized by Figurative Form and Mapping Mechanism. We further propose V-Score, a diagnostic question-answering metric that evaluates role-aware figurative faithfulness in generated images. Evaluations on recent high-performing TTI models show that vehicle intrusion persists across languages and figurative categories. As a lightweight mitigation, we introduce VISTA-Guard, which partially reduces vehicle intrusion and suggests a practical path toward more figuratively faithful TTI generation. All resources will be released publicly.

1 INTRODUCTION

Figurative prompts can cause text-to-image models to render a vehicle as an unintended scene object rather than use it to characterize the tenor. The paper formalizes this role error and introduces VISTA and V-Score to assess it, then evaluates its prevalence and a lightweight mitigation.

1 INTRODUCTION

  • Figurative Vehicle Intrusion occurs when a textually licensed vehicle that should characterize the tenor is instead rendered as a scene entity.This is a role-assignment error: the model grounds too much prompt content rather than omitting it.Source: §1 · pp. 1–2
  • Presence-oriented checks can reward a visible figurative vehicle even when it should only guide the tenor’s depiction.For the island shaped like a dragon, a TIFA-style check may reward the dragon as visible, although a faithful image should show the island and use the dragon to guide its shape.Source: §1 · p. 2
  • VISTA is a multilingual benchmark annotated by Figurative Form, Mapping Mechanism, and tenor–vehicle roles, while V-Score uses diagnostic QA to assess role-aware faithfulness.V-Score evaluates tenor preservation, vehicle avoidance, scene coherence, and figurative-meaning preservation.Source: §1 · p. 2
  • Vehicle intrusion persists across languages and figurative categories, while presence-oriented evaluation can miss the failure.The authors evaluate contemporary text-to-image models and report this cross-language, cross-category pattern.Source: §1 · p. 2
  • VISTA-Guard is a lightweight skill-based agent that reduces vehicle intrusion in existing text-to-image models without modifying their generators.The paper presents it as a mitigation contribution.Source: §1 · p. 2

2 RELATED WORK

Related work advances image quality, compositional control, and increasingly fine-grained evaluation, but commonly treats prompt-mentioned content as content to render. Work on figurative language and visual generation provides related approaches, while this paper focuses on whether a vehicle is assigned the wrong visual role.

2.1 ADVANCES IN TEXT-TO-IMAGE GENERATION

  • Recent text-to-image advances improve visual fidelity and semantic control, while compositional methods target missing objects, attribute binding, and multi-object relations.The cited examples include Structured Diffusion for missing objects and binding errors and RPG for region-wise generation.Source: §2.1 · p. 3

2.2 TEXT-TO-IMAGE EVALUATION BENCHMARKS AND METRICS

  • Text-to-image evaluation spans broad prompt suites, holistic capability frameworks, compositional benchmarks, scalar metrics, and QA protocols, which mainly reward prompt-derived content presence.VISTA instead asks whether concepts should be visually instantiated according to their semantic roles.Source: §2.2 · p. 3

2.3 FIGURATIVE LANGUAGE IN VISUAL GENERATION

  • Figurative-language work studies visual understanding and richer prompt descriptions or metaphor mappings, while VISTA evaluates vehicle role errors and VISTA-Guard offers a lightweight intervention.The cited generation approaches translate nonliteral text or use explicit metaphor mappings and binding mechanisms.Source: §2.3 · p. 3

3 METHODOLOGY

The methodology defines Figurative Vehicle Intrusion as assigning a figurative vehicle an unintended object-level role, then builds VISTA and V-Score to evaluate role-aware faithfulness. It also introduces VISTA-Guard, which uses an LLM workflow to provide auxiliary context for image generation when a prompt matches its skill.

3.1 FIGURATIVE VEHICLE INTRUSION

  • Figurative Vehicle Intrusion occurs when a vehicle that should characterize the tenor is rendered as an independent scene object.The tenor is the entity intended to exist in the described scene; the vehicle supplies a concept for characterizing it.Source: §3.1 · pp. 3–4

3.2 VISTA: BENCHMARK FOR VEHICLE INTRUSION AND SEMANTIC TENOR ASSESSMENT

  • VISTA addresses the lack of benchmarks that distinguish figurative vehicles from concepts intended to appear as visual objects.Its prompts are organized by source language and annotated for Figurative Form and Mapping Mechanism.Source: §3.2 · p. 4, §3.2.1 · p. 4

3.2.1 DATASET CONSTRUCTION

  • VISTA samples come from five source types and retain prompts with identifiable tenor–vehicle roles and no literal obligation for the vehicle to appear.Each sample records its prompt, language, tenor, vehicle, Figurative Form, and Mapping Mechanism; the benchmark preserves variation in language conventions and source availability.Source: §3.2.1 · pp. 4–5

3.2.2 V-SCORE: AN EVALUATION METRIC USING QUESTION-ANSWERING

  • V-Score evaluates role-aware figurative faithfulness with four questions on tenor preservation, vehicle avoidance, scene coherence, and figurative meaning.Its weights prioritize tenor preservation and vehicle-intrusion avoidance, while retaining the semantic safeguards; it also preserves sub-scores and evaluator rationales for analysis.Source: §3.2.2 · p. 5

3.3 VISTA-GUARD: A SKILL-BASED AGENT FOR MITIGATING VEHICLE INTRUSION

  • VISTA-Guard uses an LLM controller to invoke a role-explicating workflow and condition image generation on the prompt and any auxiliary context.The workflow preserves the tenor as the visual subject, suppresses the vehicle as a literal entity, and conveys vehicle-derived qualities through visual attributes.Source: §3.3 · p. 6

4 EXPERIMENTS AND ANALYSIS

The experiments evaluate five text-to-image models on VISTA, compare role-aware V-Score with presence-oriented evaluation, test VISTA-Guard, and assess V-Score against human judgments. Vehicle intrusion recurs across models and varies with language and figurative structure; VISTA-Guard reduces but does not eliminate it.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses 193 stratified VISTA samples and four source-language questions scored from 3 to 0, covering tenor preservation, vehicle intrusion, scene coherence, and figurative meaning.Five generators produce one image per sample; GPT-5.5 evaluates each image using visible evidence and the conservative option when ambiguous.Source: §4.1 · p. 6

4.2 EVALUATION OF TEXT-TO-IMAGE MODELS

  • Across five models, only 33.6% of outputs receive the highest vehicle-avoidance score, while 57.9% score 0 or 1 on that question.More than 70% receive the highest score on tenor preservation, scene coherence, and figurative meaning; mean V-Scores range from 77.38 for DALL-E-2 to 79.31 for Grok Imagine, despite broad sample-level distributions.Source: §4.2 · p. 6, Figure 4 · p. 7
  • Vehicle intrusion appears in all four evaluated languages, with Arabic and Japanese more difficult for several systems; idioms are consistently difficult, while perceptual resemblance is relatively easier.The results vary with how explicit, conventionalized, and visually recoverable the tenor–vehicle relation is.Source: §4.2 · pp. 6–7

4.3 COMPARISON WITH PRESENCE-ORIENTED EVALUATION

  • Presence-oriented evaluation and V-Score produce markedly different rankings and negligible association, because a visible prompt-mentioned vehicle can satisfy presence checks despite occupying the wrong semantic role.TIFA-like scores concentrate between 92.0 and 98.5; 168 of 200 images receive 100, while Spearman’s ρ between metrics is −0.042.Source: Table 3 · p. 7, §4.3 · p. 8

4.4 EFFECTIVENESS OF VISTA-GUARD

  • VISTA-Guard raises V-Score distributions across five generators, with mean paired gains of 5.2–9.9 points, but remaining overlap and lower-score tails show that difficult failures persist.Against prompt controls, it improves overall score by 7.90 points and Q2 by 1.08 points on average; Generic Anti-Hallucination improves overall score by only 1.3 points and changes Q2 very little.Source: §4.4 · p. 8, Figure 5 · p. 8

4.5 RELIABILITY OF V-SCORE THROUGH HUMAN EVALUATION

  • V-Score aligns strongly with mean human judgment on 30 blind images, with Spearman’s ρ = 0.939 and MAE = 6.3.Twelve participants rated each image; the 12-rater average had an absolute-agreement ICC of 0.978.Source: §4.5 · p. 9

5 CONCLUSION

The conclusion argues that figurative prompts require models to follow visual roles rather than render every mentioned word. It presents VISTA, V-Score, and VISTA-Guard as a benchmark, evaluation protocol, and practical direction for reducing figurative vehicle intrusion.

5 CONCLUSION

  • Figure 7 reports human–MLLM agreement on 30 blind images.The conclusion characterizes V-Score as aligning well with human judgment under the paper’s QA-based protocol.Source: Figure 7 · p. 9, §5 · p. 9
  • The paper concludes that VISTA-Guard can reduce figurative vehicle intrusion by making tenor–vehicle roles explicit before generation.The conclusion also reports that FVI persists in recent high-performing models, is missed by presence-oriented evaluation, and that V-Score aligns well with human judgment under the QA-based protocol.Source: §5 · p. 9

AI USE STATEMENT

The authors describe generative AI assistance across benchmark creation, evaluation, literature work, and manuscript polishing, with manual review of AI-assisted outputs.

AI USE STATEMENT

  • Generative AI assisted with synthetic benchmark generation, data cleaning, MLLM evaluation, literature search and summarization, and language polishing; authors manually reviewed data and evaluation outputs.The authors state that AI did not replace their judgment in defining the problem, validating conclusions, or determining final claims, and that they take responsibility for the submission and artifacts.Source: AI USE STATEMENT · p. 10

A.1 DETAILED VISTA ANNOTATION TAXONOMY

VISTA labels each sample along two complementary annotation views: Figurative Form and Mapping Mechanism.

A.1 DETAILED VISTA ANNOTATION TAXONOMY

  • Figurative Form describes how the tenor–vehicle relation is linguistically expressed, while Mapping Mechanism describes what makes the transfer interpretable.The two views are complementary rather than hierarchical.Source: §A.1 · p. 13

A.1.1 LANGUAGE COVERAGE

The selected languages span distinct language families, writing systems, segmentation conventions, and writing directions while keeping tenor–vehicle role assignment comparable.

A.1.1 LANGUAGE COVERAGE

  • The selected languages introduce typological variation while keeping the semantic problem of tenor–vehicle role assignment comparable across languages.They span distinct families, scripts, segmentation conventions, and writing directions.Source: §A.1.1 · p. 13

A.1.2 FIGURATIVE FORM

VISTA distinguishes figurative forms by how the tenor–vehicle relation is expressed, including explicit comparisons, implicit attributions, referential replacements, and lexicalized expressions.

A.1.2 FIGURATIVE FORM

  • VISTA labels figurative form as simile, metaphor, substitutive metaphor, or idiom, based on how the tenor–vehicle relation is expressed.The categories distinguish overt comparison markers, implicit characterization, vehicle replacement of a recoverable tenor, and conventionalized non-compositional expressions.Source: §A.1.2 · p. 13

A.1.3 MAPPING MECHANISM

The supplementary evaluations show that prompt-coverage scores and role-aware V-Score capture different failure modes, while V-Score’s broad findings persist across weighting schemes and evaluators. The appendix also specifies the benchmark’s mapping labels, generation and evaluation prompts, guard workflow, and evaluation subset.

A.1.3 MAPPING MECHANISM

  • Mapping Mechanism labels distinguish conceptual transfer, perceptual resemblance, cross-modal sensory transfer, and conventionalized idiomatic meaning.The distinctions separate conceptual movement from perceptual similarity, sensory transfer across modalities, and lexicalized non-compositional interpretation.Source: §A.1.3 · p. 14

A.2 PROMPTS

  • Generation prompts use source-language-matched templates, while the evaluator inspects visible evidence and answers four multiple-choice questions using conservative choices when evidence is ambiguous.Generation instructions request a coherent scene and discourage extraneous objects or literalizing figurative language; evaluator instructions say not to assume an entity is present merely because a question mentions it.Source: §A.2.1 · pp. 14–15, §A.2.2 · p. 15

A.3 VISTA-GUARD SKILL SPECIFICATION

  • VISTA-Guard infers tenor and vehicle roles, keeps the tenor central, suppresses literal vehicle realization, and conveys vehicle-derived qualities indirectly in a guarded generation context.It is not intended for fully literal prompts, prompts where both entities should appear, or intentional visual puns.Source: §A.3 · p. 16

A.4 EVALUATION SUBSET COMPOSITION

  • The main experiments use a 193-sample subset selected by 1/10 stratified sampling from the 1,930-sample VISTA pool, preserving language and annotation-view coverage.Table 5 reports the subset composition.Source: §A.4 · p. 16, Table 5 · p. 16

A.5 V-SCORE WEIGHT SENSITIVITY

  • Q3+Q4 scores are consistently higher and resemble conventional criteria, while equal-weight scores remain close to default V-Score and correlate at image level (Spearman’s ρ = 0.988).The compared schemes include role-focused Q1+Q2, semantic-safeguard Q3+Q4, equal weighting, and the default weighting.Source: §A.5 · pp. 16–17

A.6 CROSS-EVALUATOR ROBUSTNESS

  • A second evaluator, qwen3.8-max, yields slightly lower absolute V-Score values but largely preserves model rankings and the finding that FVI persists across generators.The comparison uses the same 40-prompt, 200-image subset as the metric-comparison experiment.Source: §A.6 · p. 17

A.7 SUPPLEMENTARY METRIC COMPARISON

  • Many images receive saturated TIFA-like scores while their V-Scores remain broadly distributed, indicating that prompt-coverage and role-aware evaluation capture different failure modes.Figure 8 compares image-level TIFA-like scores and V-Score on 200 images.Source: §A.7 · p. 17, Figure 8 · p. 17

Related papers

Loading 2609.32188v1…