Source-linked AI summary
Amplify What You Gaze At: Target Saliency Boosting in Text-to-Image Generation
Shengqi Dang, Zhengxi Yu, Feilin Han, Xingyu Lan, Nan Cao
TL;DR
Text-to-image methods have left the distribution of attention among objects largely unexplored, despite its importance for directing viewers’ eyes. GazeME learns saliency markers from object-level saliency data and uses them to guide generation; experiments report stronger target saliency while preserving semantic alignment and image quality.
Problem
How to generate text-described scenes that direct visual attention to a specified object remains underexplored, while existing approaches can require users to provide explicit saliency maps.
Source: §1 · p. 1Method
GazeME learns focus and unfocus marker embeddings from object-level saliency scores and uses relative-saliency-aware training to insert markers into prompts during generation.
Source: §1 · p. 2, §3.1 · p. 4Results
Takeaways & Limitations
The work treats target saliency as relative and uses bidirectional markers as a lightweight interface for controlling attention distribution among objects.
Source: §5 · p. 9The framework does not explicitly model spatial interactions or occlusion, which may limit performance in cluttered scenes.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Text-to-image generation has advanced in controlling what, where, and how objects appear, yet how visual attention is distributed among objects remains largely unexplored. In this paper, we introduce Target Saliency Boosting, a new task aimed at boosting the visual saliency of a specific object during text-to-image generation without requiring any visual priors. Our key insight is that visual saliency is inherently relative: boosting the saliency of a target object also depends on the global saliency distribution across all objects in the scene. Based on this insight, we propose GazeME, a lightweight framework that uses saliency-marked prompts, inserting learnable marker tokens around object descriptions to indicate which objects to visually emphasize or suppress. To learn these markers, we construct a saliency-semantics dataset that associates objects in image--prompt pairs with object-level saliency scores, and propose Saliency Prior Marker Activation (SPMA), a saliency-aware stochastic marker activation strategy that exploits relative saliency relationships for robust training. During inference, GazeME automatically inserts appropriate markers into the prompt, thereby directly enhancing the visual saliency of the target object. Extensive experiments demonstrate that GazeME effectively boosts target saliency while preserving both semantic alignment and image quality.
1 INTRODUCTION
Target Saliency Boosting addresses the underexplored problem of directing attention to a specified object in text-to-image generation without explicit visual priors. GazeME uses saliency-marked prompts and saliency-aware training, and experiments report stronger target saliency while preserving semantic alignment and image quality.
1 INTRODUCTION
- Visual attention matters alongside semantic fidelity because it influences memory retention, cognitive comprehension, and emotional resonance.The introduction frames design and advertising as settings where directing attention to a protagonist or product matters.Source: §1 · p. 1
- Target Saliency Boosting asks generation models to make a described target object highly salient without explicit saliency maps or other visual priors.Prior saliency work mainly predicts fixations in existing images, while map-conditioned generation can require users to specify saliency maps.Source: §1 · pp. 1–2
- The task is challenging because attention depends on contrast, composition, and semantics, while boosting a target can suppress other elements and disrupt scene coherence.The paper also identifies the lack of a generalizable prompt-and-object-saliency dataset as a barrier to supervision.Source: §1 · p. 2
- GazeME uses learnable markers around object descriptions, a saliency-semantics dataset of object-level scores, and SPMA to train marker embeddings from relative saliency relationships.At inference, it automatically inserts markers for the target object and can indicate objects for emphasis or suppression.Source: §1 · p. 2
- Experiments report that GazeME enhances target saliency while avoiding trivial solutions such as background blurring, with user evaluations aligning its advantages with human perceptual preferences.The conclusion also reports preservation of semantic alignment and image quality.Source: §1 · p. 2
2 RELATED WORK
Related work covers visual saliency prediction and applications, then controllable text-to-image methods. The paper identifies a gap in generating images that translate natural-language intent into target-object saliency control.
2.1 VISUAL SALIENCY
- Visual saliency is the property that makes regions stand out and draw attention, commonly represented in prediction as a saliency map of fixation likelihood.Brighter map regions indicate higher visual priority.Source: §2.1 · p. 2
- Saliency prediction estimates where people look, progressing from hand-crafted features to deep models and foundation models, while datasets often lack prompt grounding and low-saliency-object supervision.Existing applications include salient-object detection and segmentation, cropping, retrieval, and ROI coding.Source: §2.1 · pp. 2–3
2.2 CONTROLLABLE TEXT-TO-IMAGE GENERATION
- Controllable text-to-image methods use spatial or structural conditions and diverse intervention mechanisms, but mainly control content, location, or appearance rather than attention distribution.GazeME is presented as enabling target-saliency boosting through natural-language prompting; Figure 1 illustrates focus and unfocus markers.Source: §2.2 · p. 3, Figure 1 · p. 4
3 METHODOLOGY
GazeME uses learnable focus and unfocus markers to encode object-level saliency relationships, then inserts them into prompts to guide target-focused image generation.
3 METHODOLOGY
- GazeME inserts focus and unfocus markers around object descriptions, learns saliency knowledge in their embeddings, and uses the marked prompt to guide image generation.The framework comprises saliency-semantics data annotation, marker-embedding learning, and marker-guided generation.Source: §3 · p. 4
3.1 SALIENCY-SEMANTICS DATA ANNOTATION
- The saliency-semantics dataset contains 1,637 image–prompt pairs with object-level scores measuring average saliency within each object region relative to the whole-image distribution.The annotation pipeline estimates a saliency map and scores each object using an adapted normalized scanpath saliency formulation.Source: §3.1 · pp. 4–5
3.2 MARKER EMBEDDING EFFICIENT LEARNING
- SPMA stochastically activates markers based on objects’ relative saliency and truncates activation patterns to preserve saliency ordering; training freezes the backbone and optimizes only the four marker embeddings.Objects near the grouping threshold are more likely to be deactivated, while similarly scored objects tend to be activated together.Source: §3.2 · pp. 5–6
3.3 MARKER-GUIDED IMAGE GENERATION
- At inference, an LLM identifies the other prompted objects, then GazeME marks the target with focus and the remaining objects with unfocus before generating the image.This prompt construction explicitly represents the target as the focus object and the others as unfocus objects.Source: §3.3 · p. 6
4 EXPERIMENTS
The experiments compare GazeME with prompt-based, training-based, and editing-based baselines on target saliency, semantic alignment, and image quality. Ablations, shortcut analysis, and a user study examine its mechanisms and outcomes.
4.1 EXPERIMENTAL SETUP
- The 432-task testset targets objects the frozen base model places in the bottom half of its saliency scores.Methods are evaluated for saliency alignment, semantic alignment, and image quality.Source: §4.1 · p. 7
4.2 BASELINE COMPARISON
- GazeME achieves the highest Target-NSS, remains competitive on CLIPScore, and matches the best baseline on CLIPIQA.The comparison covers prompt-level, training-based, and editing-based baselines.Source: §4.2 · pp. 7–8, Table 1 · p. 8
- Qualitatively, GazeME makes the target the primary focus, while other methods may amplify other objects or introduce unnatural artifacts.Figure 3 presents generated images with corresponding saliency maps.Source: Figure 3 · p. 7, §4.2 · p. 8
- GazeME keeps low-level factor distributions closest to the base model, while FLUX-Editing mainly blurs backgrounds and prompt-level methods alter target size, position, brightness, and contrast.The analysis considers eight factors, including sharpness and background blur.Source: §4.2 · p. 8, Figure 4 · p. 8
4.3 ABLATION STUDY
- Removing SPMA or marker training lowers Target-NSS from 0.2761 to 0.2276 or 0.2192, respectively; removing <unfocus> lowers it to 0.2521.The ablations indicate contributions from SPMA, optimized marker embeddings, and explicit suppression of non-target objects.Source: §4.3 · pp. 8–9, Table 2 · p. 8
- Target-NSS varies only slightly across γ settings, while CLIPScore and CLIPIQA remain nearly constant; per-sample mean gives the best Target-NSS.This setting also ensures each sample has high- and low-saliency groups for bidirectional supervision.Source: Table 2 · p. 8, §4.3 · p. 9
4.4 USER STUDY
- In the user study, GazeME received 68.8% of saliency-alignment votes, 64.7% of semantic-alignment votes, and 65.6% of visual-quality votes.All three participant-level majorities were significant against a 50% chance level (p < 10^-3).Source: Figure 5 · p. 8, §4.4 · p. 9
5 DISCUSSION
The discussion treats target saliency as relative and presents bidirectional markers and SPMA as ways to control and learn that relativity. It also identifies limits in supervision, spatial modeling, and image scope.
5 DISCUSSION
- Bidirectional markers provide a lightweight interface for relative saliency control, while SPMA encourages marker embeddings to learn robust saliency knowledge.The approach complements control over which objects appear and where they appear by targeting attention distribution among them.Source: §5 · p. 9
- Automatic saliency and segmentation predictors may introduce noise and bias; absent explicit spatial-interaction and occlusion modeling may limit performance in cluttered scenes.The framework targets static images; video would require temporally coherent markers.Source: §5 · p. 9
6 CONCLUSION
The conclusion presents Target Saliency Boosting as a task for controlling visual attention in text-to-image generation without explicit visual priors, and GazeME as its marker-guided framework.
6 CONCLUSION
- GazeME uses bidirectional marker tokens to guide generation and boosts target saliency while preserving semantic alignment and image quality.The authors say they will release the constructed dataset and source code.Source: §6 · p. 9