Source-linked AI summary
Agent Banana: High-Fidelity Image Editing with Agentic Thinking and Tooling
Ruijie Ye, Jiayi Zhang, Zhuoxin Liu, Zihao Zhu, Siyuan Yang, Li Li, Tianfu Fu, Franck Dernoncourt, Yue Zhao, Jiacheng Zhu, Ryan Rossi, Wenhao Chai, Zhengzhong Tu
TL;DR
Professional image editing remains constrained by over-editing, weak multi-turn reliability, and evaluation that misses native 4K workflows. Agent Banana combines hierarchical planning, structured context memory, and localized layer editing, and introduces HDD-Bench for verifiable high-definition dialogue evaluation. The framework is reported to improve instruction following, edit locality, and multi-turn stability while preserving non-target regions, with native high-resolution fidelity and multi-turn reliability as its deployment-relevant focus.
Problem
Professional workflows require precise, native high-resolution, multi-turn editing, but existing systems and benchmarks often lack localized fidelity, state tracking, and verifiable stepwise evaluation.
Method
Agent Banana uses Planner-Executor agents with Context Folding for structured long-horizon memory and Image Layer Decomposition for localized high-resolution layer edits.
Results
Agent Banana consistently improves instruction following, edit locality, and multi-turn stability over strong non-agentic baselines on 4K editing workflows and standard single-turn benchmarks.
Takeaways & Limitations
HDD-Bench and object-level state-transition evaluation support stepwise verification of edits and invariants, helping diagnose over-editing, drift, and long-horizon degradation.
Takeaways & Limitations
The authors note that stronger editing capabilities can be misused for misleading visual content or non-consensual manipulation, motivating evaluation and auditing.
Abstract
from arXiv · showhide
We study instruction-based image editing under professional workflows and identify three persistent challenges: (i) editors often over-edit, modifying content beyond the user's intent; (ii) existing models are largely single-turn, while multi-turn edits can alter object faithfulness; and (iii) evaluation at around 1K resolution is misaligned with real workflows that often operate on ultra high-definition images (e.g., 4K). We propose Agent Banana, a hierarchical agentic planner-executor framework for high-fidelity, object-aware, deliberative editing. Agent Banana introduces two key mechanisms: (1) Context Folding, which compresses long interaction histories into structured memory for stable long-horizon control; and (2) Image Layer Decomposition, which performs localized layer-based edits to preserve non-target regions while enabling native-resolution outputs. To support rigorous evaluation, we build HDD-Bench, a high-definition, dialogue-based benchmark featuring verifiable stepwise targets and native 4K images (11.8M pixels) for diagnosing long-horizon failures. On HDD-Bench, Agent Banana achieves the best multi-turn consistency and background fidelity (e.g., IC 0.871, SSIM-OM 0.84, LPIPS-OM 0.12) while remaining competitive on instruction following, and also attains strong performance on standard single-turn editing benchmarks. We hope this work advances reliable, professional-grade agentic image editing and its integration into real workflows.
1 Introduction
Professional image-editing workflows require precise localized changes, native high-resolution processing, and reliable multi-turn state management, but existing systems often over-edit, degrade detail, and struggle with complex requests. Agent Banana addresses these needs with hierarchical planning, layer-aware execution, self-correction, and HDD-Bench for verifiable multi-turn evaluation.
- Motivation: Professional workflows demand localized edits that preserve non-target content while operating directly on native high-resolution assets.Existing models often use reduced resolution or downsampling, which makes fine textures and sharp boundaries difficult to maintain.
- Agent Banana: Agent Banana decomposes complex prompts into discrete editing steps and executes them with layer isolation, masking, and local edits.Its self-reflection mechanism supports retry, rollback, and replanning during inference.
- Core mechanisms: Context Folding compresses long interaction histories into structured memory, while Image Layer Decomposition isolates high-resolution layers to preserve non-target content and prevent drift.Together, these mechanisms target long-horizon control and localized high-resolution editing.
- Evaluation: HDD-Bench evaluates multi-turn editing through dependent instruction chains, stepwise verification, instruction adherence, edit locality, consistency, and native-resolution visual fidelity.Its evaluator analyzes turn-to-turn state transitions with instruction checks and JSON state tracking.
2 Agent Banana Framework
Agent Banana models multi-turn editing as stateful action selection and addresses context growth and iterative image degradation through hierarchical memory and localized layer processing. Its Planner and Executor divide global reasoning from atomic tool-based edits, while Context Folding and Image Layer Decomposition support recovery, fidelity, and compositional operations.
- Problem setup: The task maps instruction sequences and an initial image to an editing trajectory whose actions produce successive image states through environment transitions.Each action includes reasoning and tool invocation, and each resulting state depends on the preceding state.
- Problem setup: Long-horizon editing suffers from context overflow and cumulative detail degradation caused by repeatedly resampling full images, including unchanged regions.These effects introduce irrelevant history noise, exceed context limits, and accumulate artifacts in backgrounds or fixed objects.
- Framework: The Planner interprets intent, decomposes instructions into executable sub-goals, and monitors progress, while the Executor performs localized atomic operations and invokes tools.This separation supports reasoning over long-horizon objectives alongside fine-grained visual execution.
- Context Folding: Context Folding compresses interaction history into Asset, Execution, and Planning schemas for compact state tracking, transient recovery, and persistent retention of verified editing paths.ImageContext records image-state relationships, ToolContext supports current-step recovery, and ActionContext stores the effective path after a round.
- Image Layer Decomposition: Image Layer Decomposition replaces full-image processing with a decompose-edit-fuse pipeline that crops target regions into high-resolution patches and blends edited patches back.Local processing freezes background pixels, reduces non-target degradation, and supports ultra-high-definition editing beyond native model resolution limits.
- Action space: The framework composes five atomic operations—replace, remove, add, adjust, and undo—to express complex image edits.These primitives cover object substitution, removal, addition, attribute changes, and rollback through the state graph.
3 HDD-Bench: High-Definition, Dialogue-based image editing benchmark
HDD-Bench is a high-definition, dialogue-based benchmark that evaluates professional image editing through dependent instruction chains, native-resolution fidelity, and object-level compositionality. It combines symbolic state supervision with semantic and background-preservation metrics for verifiable multi-turn assessment.
- Data construction: A deterministic symbolic data engine updates object attributes through canonical edit commands, producing consistent and checkable intermediate target states without rendering images during construction.A language agent paraphrases commands into natural-language instructions, while human verification corrects the initial scene graph and extracted attributes.
- Benchmark design: HDD-Bench addresses gaps in existing benchmarks by combining multi-turn dependencies, native high-resolution fidelity, and object-level compositionality.Existing benchmarks are often single-turn, low-resolution, or otherwise insufficient for professional editing requirements.
- Data construction: Each sample is a three-turn editing session with natural-language instructions and corresponding symbolic target states for verifiable evaluation.The fixed three-turn format controls difficulty and comparisons while preserving stepwise dependency and error accumulation; longer sessions remain generatable.
- Data construction: HDD-Bench contains 96 curated sessions emphasizing multiple salient objects, non-trivial edit chains, diverse atomic actions, and hybrid multi-action instructions.Covered actions include add, remove, replace, adjust, and undo.
- Evaluation protocol: Evaluation separates semantic correctness from non-target preservation using object-centric state metrics and Otsu-masked pixel/perceptual fidelity metrics.Instruction Following measures targeted-object attributes, Image Consistency measures unchanged non-target objects, and masked PSNR, SSIM, and LPIPS evaluate background regions.
- Evaluation protocol: Qualitative comparisons test whether a color-only instruction changes the intended cooler without reducing resolution, altering structure, or failing to apply the requested color.The comparison highlights preservation of the original image while executing the targeted edit.
4 Experiments
Agent Banana is evaluated on multi-turn, single-turn, native-resolution, and ablation settings, with results emphasizing 4K fidelity, localized preservation, and stability across sequential edits.
- Experimental Setup: Agent Banana is natively capable of processing 4K images while balancing instruction execution with image fidelity on HDD-Bench.The benchmark evaluates PSNROM, SSIMOM, LPIPSOM, instruction adherence, and high-resolution support.
- Multi-turn Editing: Agent Banana is one of only two models capable of maintaining high fidelity at 4K native resolution.The results attribute this capability to Image Layer Decomposition, which prevents detail loss during high-resolution editing.
- Single-turn Editing: Agent Banana achieves leading or comparable results across all metrics on single-turn editing benchmarks.The paper attributes this performance to precise Executor tool control and self-verification by Quality Test modules.
- Ablation Study: Replacing the Planner and Executor’s base model with Qwen-3-8B causes substantial degradation on ambiguous instructions and long-sequence planning.The weaker model frequently generates unparseable tool parameters or erroneous dependencies, interrupting workflows.
- Native-resolution and Locality Analysis: Figure 5 shows that global editing distorts an untargeted sofa cushion, whereas Agent Banana maintains the visual consistency of unedited regions.The native-resolution analysis also reports that Agent Banana preserves high-frequency information that baselines lose through downsampling and upsampling.
- Multi-turn Drift Analysis: Prior-Induced Editing Drift describes non-edited regions gradually shifting toward a generator’s preferred texture and style statistics across turns.Agent Banana keeps PSNROM nearly constant across turns, aligning with reduced accumulated artifacts and better preservation of non-edited details.
5 Related Work
Related work spans instruction-based image editors built on diffusion and autoregressive models and agentic systems that combine language-model reasoning with iterative tool use.
- Instruction-based Image Editing: Instruction-based editing has progressed from systems such as GLIDE and InstructPix2Pix to interactive, context-aware editors including GPT-Image-1 and Nano Banana.Follow-up work also explores attention manipulation and mask-based control.
- Agentic Systems: ReAct established iterative think–act loops for agentic systems, while Model Context Protocol standardizes communication between language models and external tools.These paradigms support interaction and task solving in complex environments.
6 Conclusion
The paper presents Agent Banana and HDD-Bench as a framework and evaluation setup for precise, reliable editing across 4K multi-turn workflows. It also emphasizes auditing and safeguards because powerful editing can facilitate misleading or non-consensual visual manipulation.
- Conclusion: Agent Banana couples LLM planning, VLM perception, and layer-aware tool use for rollback-safe 4K edits that preserve non-target regions.The framework supports explicit object-level state transitions, undo, branching, and long-horizon correction.
- Conclusion: HDD-Bench provides high-resolution, multi-turn evaluation aligned with professional workflows and supports stepwise verification of edits and invariants.The scalable data engine decouples state transitions from pixel rendering to synthesize reasoning traces and edit histories.
- Impact and Scope: The authors recommend provenance, consent, and disclosure mechanisms because stronger editing capabilities can enable misleading content or non-consensual manipulation.The stated contributions emphasize controllability and detecting failure accumulation rather than unconstrained manipulation.