PaperlayerView exact revision on arXiv

Source-linked AI summary

Dotting the Eye: An Intent-Driven Image Retouching Agent for Visual Focus Enhancement

Chujie Qin, Zilong Zhang, Zewei Chang, Chunle Guo, Ruixing Wang, Tao Hu, Ming-Ming Cheng, Chongyi Li

arXiv:2609.01148v1cs.CV

TL;DR

Existing retouching models improve overall appearance but do not explicitly model user-directed visual focus, which requires coordinated global and local adjustments. EyeControl links weak user intent to focus regions and tonal operations through attention alignment and operation consistency; ablations report the best overall average performance with the latter.

  • Problem

    Current retouching models do not explicitly model shifting visual attention toward an intended region, while effective focus enhancement requires coordinated global and local adjustments.

  • Method

    EyeControl uses pseudo-intention attention alignment from saliency difference maps and an operation-consistency constraint linking user intention, focus, and global-local tonal operations.

  • Results

    Operation-consistency loss achieves the best overall average performance across four scenarios, raising Global PSNR/SSIM from 17.75/0.7657 to 23.34/0.800 and yielding the best Multiturn performance.

  • Takeaways & Limitations

    EyeControl provides intention-consistent and scene-aware focus modulation by concentrating attention within intended regions and constraining plausible editing trajectories.

  • Takeaways & Limitations

    The SC metric does not explicitly measure whether retouching shifts attention toward the specified intent region, so higher SC need not indicate better focus alignment.

Abstract

from arXiv · show

Image retouching is commonly formulated as enhancing overall visual quality through color adjustment, but in practice, it also serves to emphasize visual focus by guiding viewers' attention toward a specific subject or region. Achieving such focus-oriented retouching is inherently challenging, as it requires well-coordinated global and local adjustments to manipulate perceptual saliency while maintaining visual naturalness. This intricate process typically demands substantial professional expertise. In this study, we propose EyeControl, a MLLM-driven agent with a diffusion-based retouching executor that enables visual focus enhancement under weak user intent. With only a few clicks or coarse strokes, EyeControl directs visual attention to the intended region, effectively "dotting the eye" of the image. The core idea is to explicitly link the weak user intention with the target editing region and the corresponding tonal adjustment operations during retouching. To achieve this, the system first interprets the intent and image content to infer the visual focus and generate structured intent guidance for the retouching executor. Second, the retouching executor is encouraged to respond more strongly to the target region, explicitly aligning its attention map with a designed pseudo-intent map. We also introduce an operation-consistency constraint to improve coordination between global and local adjustments, achieving more natural and coherent retouching. Additionally, we contribute ControlArt-Bench, a high-quality evaluation dataset for visual focus enhancement. Extensive evaluations demonstrate that EyeControl yields perceptually appealing results with stronger intent alignment. Code can be found in https://github.com/DragonisCV/EyeControl.

1 Introduction

EyeControl frames image retouching as visual focus enhancement under weak spatial intent, linking user intention to editing operations while coordinating global and local adjustments. It introduces an intent-driven agent, attention alignment, operation consistency, and a dedicated benchmark for evaluating focus enhancement.

  • Motivation: Existing retouching models improve overall quality but do not explicitly model guiding attention toward a visual focus.They support localized edits, yet current approaches do not directly represent purposeful attention guidance.
  • Challenges: Clicks and coarse strokes provide sparse, ambiguous intention signals that require interpreting both user interactions and image context.These signals indicate regions of interest without specifying the desired editing operations.
  • Challenges: Visual focus enhancement requires coordinated adjustments to focal and surrounding regions rather than isolated local modifications.The challenge is to increase a region’s prominence while maintaining coherent relationships with its surroundings.
  • Approach: EyeControl uses Pseudo-Intention Attention Alignment and operation consistency to connect intention with saliency modulation and coordinate global and local edits.PAA aligns internal attention with pseudo-intention labels derived from saliency difference maps, while operation consistency compares joint and sequential global-local applications.
  • Contributions: The paper contributes a visual-focus perspective, the EyeControl framework, and ControlArt-Bench for focus-oriented evaluation.The framework integrates intention reasoning, attention-level modulation, and crossscale coordination while targeting global coherence and perceptual naturalness.
  • Evaluation: Extensive experiments report state-of-the-art visual focus enhancement while remaining competitive on global and local retouching tasks.The evaluation covers focus-oriented performance alongside established global and local retouching settings.

2 Related Work

Prior retouching research includes interpretable adjustment operators, VLM-based instruction following, and saliency retargeting. However, existing datasets and methods provide limited modeling of user intention and region-specific perceptual emphasis.

  • Global Image Retouching: Learning-based retouching methods use tone curves, bilateral grids, or LUTs to approximate traditional editing pipelines.These approaches are supported by large-scale retouching datasets and commonly learn from paired image targets.
  • Global Image Retouching: Widely used paired datasets mainly contain globally adjusted targets without explicit user intention or region-specific emphasis.Their supervision can reflect averaged color preferences rather than intention-driven objectives.
  • VLM-based Retouching: VLM-based retouching systems interpret instructions and image content, sometimes decomposing requests into sequential operations or parameter adjustments.Their primary emphasis is semantic reasoning and instruction following.
  • Saliency Retargeting: Saliency retargeting modifies color, contrast, or luminance to redirect visual attention but remains largely rule-based and separate from modern generative editing.These methods demonstrate that appearance modulation can alter perceptual ordering.

3 Method

EyeControl combines a multi-stage VLM Planner with a diffusion-based Executor to retouch images according to weak spatial intentions. Its training pipeline derives and augments intent supervision, while attention alignment and operation consistency coordinate focus-oriented global and local edits.

  • 3.1 Overview: EyeControl uses a user interface, VLM-based Planner, and diffusion-based retouching Executor to interpret ambiguous intent and produce iterative retouching results.The Planner parses the image, infers intention, selects an execution mode, and generates guidance; the Executor conditions on the image, intention mask, and guidance.
  • 3.2 Data Generation Pipeline: The data-generation pipeline curates retouching pairs, filters for saliency changes, extracts structured intent from saliency differences, and simulates clicks, strokes, and region painting.Filtering removes pairs with negligible perceptual differences or insufficient saliency change, while augmentation preserves intent across varied interaction patterns.
  • 3.3 EyeControl Framework: The Planner decomposes ambiguous interactions into scene-level and region-aware descriptions, infers user intent, generates guidance, and selects Global, Local, or Focus Mode.Focus Mode coordinates global and local adjustments to enhance the intended visual focus.
  • 3.3 EyeControl Framework: The Executor is trained with latent-space Flow Matching reconstruction and decodes the predicted clean latent through a VAE decoder to obtain the retouched output.The output can be returned for subsequent interaction rounds.
  • 3.3 EyeControl Framework: Pseudo-Intent Attention Alignment supervises latent attention with a signed saliency-difference map that marks regions to enhance or suppress.This addresses the limitation that mask concatenation alone does not ensure consistent emphasis during denoising.
  • 3.3 EyeControl Framework: Operation consistency training uses one joint denoising path and two sequential paths with randomly sampled execution orders to coordinate global and local operations.Focus samples contain both global and local editing instructions, while mixed-mode diffusion models can exhibit order-sensitive behavior.

4 Experiments

EyeControl is evaluated on focus enhancement, user preference, ablations, and component behavior. Results support stronger intent alignment, coordinated global–local editing, and stable focus modulation.

  • Comparisons: The evaluation compares EyeControl with agent-based, generative, and commercial retouching systems on ControlArt-Bench.Comparisons use aligned long-form instructions and Intent Masks for fair evaluation.
  • Comparisons: EyeControl outperforms most compared methods across evaluation metrics and performs comparably to NanoBanana 2.The authors note that some competitors obtain high Focus Alignment through large structural changes rather than subtle focus guidance.
  • User preference: EyeControl consistently wins user comparisons across aesthetic quality, intent consistency, and content fidelity.The study includes 50 participants and seven competing approaches.
  • Ablation and discussion: Spatial masks alone yield diffuse, boundary-driven attention and weak perceptual separation, whereas pseudo-intent alignment concentrates attention within the intended region.Dedicated mask projections and explicit attention supervision support clearer separation between focal and non-focal regions.
  • Ablation and discussion: VLM guidance matters most when multiple plausible adjustment directions exist, where its absence causes inconsistent or unstable edits.Its role is semantic disambiguation that constrains plausible editing trajectories in complex scenes.
  • Ablation and discussion: Without operation consistency, execution order changes results: repeated local refinement can override global atmosphere, while global edits can suppress focal emphasis.This indicates interference between heterogeneous editing conditions during diffusion.
  • Ablation and discussion: Operation-consistency regularization makes editing trajectories more consistent across execution orders and preserves focal prominence with coherent global tone.It produces the best average performance across four scenarios and raises Global PSNR/SSIM from 17.75/0.7657 to 23.34/0.800.

5 Conclusion

The paper reframes image retouching as visual-focus enhancement under weak user intent and introduces EyeControl to connect intention, focus, and tonal operations. It also provides a dedicated benchmark for systematic evaluation.

  • EyeControl links weak user intention, visual focus, and tonal adjustment operations during retouching.
  • The framework combines intention understanding, attention-level modulation, and coordinated global–local adjustments while preserving perceptual naturalness.ControlArt-Bench adds paired spatial intent annotations and focus-oriented metrics for evaluation.

Appendices

The appendices document training details, metric definitions and validity, supplementary comparisons, and limitations material.

  • The supplemental materials include sections on training details, metrics, additional experimental results, and limitations.

A.1 Composition of the Training Set

ControlArt combines focus, global, and local retouching data to train corresponding execution modes, with intent guidance organized for executor inputs. Dataset statistics and metric guidance are also documented.

  • Dataset composition: ControlArt contains focus, global, and local samples corresponding to the Focus, Global, and Local execution modes.
  • Dataset composition: Focus samples use generated retouching pairs, extracted intention regions, and final Intent Masks, while global and local datasets strengthen adjustment coordination.
  • Dataset composition: Global training data is filtered from PPR10k to retain global adjustments without explicit visual-focus enhancement.
  • Dataset composition: Local training data contains mask-based local edits with Intent Masks produced through automatic generation and manual annotation.
  • Training inputs: Training derives intent guidance from pre- and post-retouch images and formats it into global and local edit instructions.
  • Training inputs: The executor uses an all-zero mask in Global Mode, whereas Local Mode uses a user mask with local guidance.

B.1 Limitations of Existing Metrics

Existing metrics do not fully capture whether retouching redirects attention toward the user-intended region or preserves content without cross-metric interference.

  • SC does not explicitly measure whether retouching shifts visual attention toward the user-specified intent region.
  • A higher SC score does not necessarily indicate better alignment between the retouched result and intended focus.
  • Jointly predicted PQ can be influenced by other evaluation dimensions when a VLM assesses perceptual quality together with SC.Independent PQ evaluation is intended to reduce this cross-metric interference.
  • Independent PQ evaluation focuses on content preservation and visual artifacts, providing a cleaner estimate of content fidelity than joint prediction.The paper denotes the jointly predicted and independently predicted variants as PQJ and PQI, respectively.

B.2 Metrics for Visual Focus Enhancement

The paper introduces Focus Alignment to measure whether retouched visual focus matches the user’s target region, alongside a reproducible but imperfect VLM-based evaluation protocol.

  • Focus Alignment (FA) directly measures consistency between the retouched visual focus and the user-intended target region.FA is scored on a 0–10 scale.
  • FA evaluates viewer landing point, visual hierarchy, target-region enhancement, attention-guidance mechanisms, and stability.
  • The experiments use Qwen2.5-VL-72B as the evaluator throughout all experiments.The authors report that MLLM-based evaluation is not always fully accurate but remains relatively consistent and reproducible for large-scale assessment.

B.3 Metric Validity

Human-preference validation supports FA for intent consistency and PQI for content fidelity, while showing that VLM metrics should complement rather than replace human studies.

  • FA improves agreement with human Intent Consistency judgments over SC across Spearman correlation, Kendall correlation, and winner judgement accuracy.The reported improvements are 0.260 to 0.370, 0.236 to 0.348, and 67.9% to 87.6%, respectively.
  • PQI consistently outperforms PQJ for human-evaluated Content Fidelity across both correlations and winner judgement accuracy.Spearman correlation rises from 0.331 to 0.390, Kendall correlation from 0.301 to 0.357, and accuracy from 74.6% to 83.42%.
  • VLM metrics predict ties more often than human annotators, with IC-FA tie rates of 74.5% for VLMs and 10.6% for humans.The authors therefore position FA, PQI, and O as complementary to human studies rather than replacements.
  • The overall score O is computed as the geometric mean of FA and PQ.In local retouching, perceptual quality is evaluated on the masked portrait image and denoted PQRC.

C Additional Experimental Results

Additional experiments evaluate EyeControl on global and local retouching benchmarks without training on their training splits, showing strong generalization and content fidelity.

  • C Additional Experimental Results: The additional experiments assess global retouching on MIT-Adobe FiveK and local retouching on MMArt-Bench.
  • C Additional Experimental Results: EyeControl demonstrates strong generalization and high content fidelity despite not being trained on the benchmarks’ training splits.
  • C.1 Comparison on Global Image Retouching: 10.55% and 21.18% are EyeControl’s PSNR improvements over JarvisArt and JarvisEvo, respectively, on MIT-Adobe FiveK.EyeControl also gains 0.21 in PQ over JarvisArt.
  • C.2 Comparison on Local Image Retouching: EyeControl improves PSNRRC and PQRC on MMArt-Bench and is comparable to JarvisArt and JarvisEvo on SSIMRC.
  • C.2 Comparison on Local Image Retouching: EyeControl preserves overall visual realism while enhancing a local portrait region, unlike methods that degrade realism through isolated regional retouching.

C.3 Ablation on Intent Coarseness Level

EyeControl remains effective across click, strike, region, and SAM-generated masks with different spatial granularities. The results indicate that coarse intent cues can be sufficient for visual focus enhancement, rather than precise semantic boundaries being necessary.

  • Robustness across intent coarseness: EyeControl achieves comparable PSNR, SSIM, O, and KL performance across click, strike, region, and SAM-generated mask types.These masks encode the same user intent at different spatial granularities, from sparse cues to precise semantic regions.
  • Robustness across intent coarseness: Semantic masks do not necessarily outperform coarse masks for intent-driven visual focus enhancement.Visual focus may depend more on saliency boundaries and perceptual attention than exact semantic object boundaries.
  • Training implications: Coarse-mask training encourages inference from flexible user cues and can generalize to more precise masks.Training only on precise masks may reduce robustness to sparse or coarse intent inputs.
  • Qualitative evidence: Additional qualitative comparisons show enhanced intended visual focus while preserving overall image naturalness.The supplementary comparisons in Figs. 14–18 illustrate the visual effectiveness of the approach.
Loading 2609.01148v1…