Source-linked AI summary
Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes
Jing Tan, Zhaoyang Zhang, Yantao Shen, Jiarui Cai, Shuo Yang, Jiajun Wu, Wei Xia, Zhuowen Tu, Stefano Soatto
TL;DR
Text-guided object-level geometric editing is difficult for current methods amid scarce paired supervision and limits of pixel-level optimization. TALK2MOVE uses GRPO rollouts and spatial rewards to learn transformations, with step evaluation and active sampling to improve efficiency. Experiments report stronger spatial accuracy and visual coherence than existing open- and closed-source models.
Problem
Method
TALK2MOVE trains a flow-based GRPO editing model with stochastic trajectory exploration, object-centric spatial rewards, and off-policy evaluation to prioritize informative denoising steps.
Source: §1 · p. 2, §5.3 · p. 5Results
Takeaways & Limitations
Abstract
from arXiv · showhide
We introduce Talk2Move, a reinforcement learning (RL) based diffusion framework for text-instructed spatial transformation of objects within scenes. Spatially manipulating objects in a scene through natural language poses a challenge for multimodal generation systems. While existing text-based manipulation methods can adjust appearance or style, they struggle to perform object-level geometric transformations-such as translating, rotating, or resizing objects-due to scarce paired supervision and pixel-level optimization limits. Talk2Move employs Group Relative Policy Optimization (GRPO) to explore geometric actions through diverse rollouts generated from input images and lightweight textual variations, removing the need for costly paired data. A spatial reward guided model aligns geometric transformations with linguistic description, while off-policy step evaluation and active step sampling improve learning efficiency by focusing on informative transformation stages. Furthermore, we design object-centric spatial rewards that evaluate displacement, rotation, and scaling behaviors directly, enabling interpretable and coherent transformations. Experiments on curated benchmarks demonstrate that Talk2Move achieves precise, consistent, and semantically faithful object transformations, outperforming existing text-guided editing approaches in both spatial accuracy and scene coherence.
1. Introduction
TALK2MOVE frames text-guided object geometry editing as an RL problem, using spatial rewards and efficient training to address limits of paired-data and pixel-level approaches.
1. Introduction
- The framework explores transformations through stochastic perturbations to diffusion trajectories and diverse rollouts from simple prompt variations, reducing reliance on costly paired data.Its spatial reward model directly evaluates object displacement, rotation, and scaling beyond pixel-level similarity.Source: §1 · p. 2
- Step-wise active sampling uses off-policy reward evaluation to select informative denoising steps and skip redundant ones, improving training efficiency by 2× while preserving reward robustness.The method quantifies each perturbed step’s influence on task reward.Source: §1 · p. 2
- The paper formulates text-guided geometric object transformation within RL and presents the pipeline as data-efficient relative to SFT-based methods that depend on costly paired annotations.The contribution list also describes a scalable data-collection pipeline for GRPO online training.Source: §1 · p. 2
- TALK2MOVE improves spatial accuracy and scene coherence under limited budgets, with state-of-the-art spatial transformation performance across open- and closed-source editing systems.The introduction attributes these results to experiments on the proposed framework.Source: §1 · p. 2
2. Related Work
Related work spans explicit drag-based controls, text-conditioned editing systems, and RL for visual generation; TALK2MOVE focuses on language-guided object geometry and faster rollout generation.
2. Related Work
- 2D dragging methods use point tracking or feature-space guidance, while 3D-aware approaches [3] [42] [43] transform reconstructed objects but require explicit controls or complex lifting and rendering pipelines.The 2D methods [20] [25] [27] [33] [34] require manual control-point specification and can struggle with high-level instructions.Source: §2 · p. 2
- Text-only diffusion methods [30] [32] and flow models can struggle with fine-grained spatial instructions, while LLM/VLM-connected systems improve prompt following but may compromise visual coherence [28] [7] [35] [4] [19].QwenImageEdit combines VLM semantic and VAE reconstructive features; the paper adopts it as its backbone.Source: §2 · p. 2
- Diffusion RL ranges from one-step reward-weighted regression and multi-step DDPO [2] to stochastic-rollout GRPO methods [21] [40], but sampling remains computationally expensive.TALK2MOVE adds an early-exit mechanism for rollout generation and targets task-dependent object-transformation stages rather than general temporal reweighting [9].Source: §2 · p. 3
3. Geometric Transformation for Object-Level Scene Editing
The paper defines object-level scene editing as text-directed changes to an object’s geometry while keeping the scene static and unedited regions consistent.
3. Geometric Transformation for Object-Level Scene Editing
- The task covers translation, rotation, and resizing of an object according to text, while preserving the static scene and consistency of unedited regions.These three operations are presented as the core dimensions of geometric object transformation.Source: §3 · p. 3
- Standardized transformation templates provide a consistent basis for supervision and evaluation by specifying a target object and spatial relation, plus translation directions and references, rotation axes, directions, angles, or resizing ratios.Source: §3 · p. 3
- These templates constrain open-ended object transformation into a well-defined, verifiable space for reproducible learning.They provide a consistent basis for supervision and evaluation.Source: §3 · p. 3
4. Geometric Transformation Data Curation
The data pipeline combines input-only image–prompt samples for GRPO with a smaller set of synthesized before–after pairs for SFT cold-start training.
4. Geometric Transformation Data Curation
- GRPO-style training requires only a reference image and text prompt, and the paper illustrates its curation process using object translation.This input-only setup differs from the paired supervision used for the SFT cold start.Source: §4 · p. 3
- For translation, LLM-generated scene descriptions guide text-to-image synthesis of reference scenes containing backgrounds and multiple sparsely distributed movable objects.The cited text-to-image models are.Source: §4 · p. 3
- VLM-generated template instructions are parsed into structured geometric attributes, and prompt variations expand 800 translation images into 3,200 samples.The dataset uses ∼800 images and 3,200 samples; the authors leave larger-scale expansion to future work, noting the current dataset is sufficient for training.Source: §4 · p. 3
- For SFT cold start, filtered video-generated motion yields 800 translation pairs and 43 rotation pairs, while image-editing models yield 110 resizing pairs.The paper uses video generation for translation and rotation, but image editing for resizing because current video models perform poorly on that task.Source: §4 · p. 3
5. Method
TALK2MOVE trains a diffusion editor with flow-based GRPO, spatially grounded object rewards, and active selection of informative denoising steps. Its pipeline combines stochastic rollouts with shortcuts that reduce computation and rewards tailored to prompted geometric transformations.
5.1. Preliminary
- Flow-based GRPO casts denoising as a T-step MDP and extends its objective using transition-probability ratios across stochastic rollouts.Noise perturbations turn deterministic ODE sampling steps into stochastic transitions that generate rollout groups.Source: §5.1 · p. 4
5.2. Training Pipeline
- A lightweight LoRA SFT cold start provides coarse spatial priors, followed by specialist-rewarded stochastic rollouts whose group-relative advantages update the policy.The cold-start checkpoint is trained on merged annotated pairs from all three tasks; during RL, specialist models disentangle objects and evaluate their transformations.Source: §5.2 · p. 4
5.3. Efficient GRPO with Early Exit
- Active step sampling identifies informative denoising steps through rollout-reward variance and uses ODE shortcuts to reduce computation while maintaining comparable task performance.Step importance is calibrated by perturbing small sets of steps on 2–4 images; the selected exit step K shortens computation from T steps to approximately K steps.Source: §5.3 · p. 5, Figure 3 · p. 5
5.4. Spatially Grounded Reward Design
- Spatially grounded rewards measure whether translation, rotation, and resizing align with the prompted transformations, using object localization and task-specific signals.Translation rewards compare object-center displacement and depth; rotation rewards assess prompted axis, direction, and angle; resizing rewards compare normalized bounding-box size ratios.Source: §5.4 · p. 5
6. Experiments
Experiments compare TALK2MOVE with image-editing methods on synthetic and real images, assess user preferences, and examine the effects of training choices. The ablations report gains from RL, active step sampling, and spatial rewards while also evaluating scene preservation.
6.1. Implementation Details
- Each subtask used about 160 GPU hours; cold-start training used rank-64 LoRA for 3,000 iterations, while GRPO training used FlowGRPO on 16 H200 GPUs.The cold-start learning rate was 1e-4; GRPO used sample noise level 1.0 and clip range 2e-4.Source: §6.1 · p. 5
6.3. Quantitative Evaluation
- On the synthetic benchmark, TALK2MOVE outperforms open-sourced baselines on most metrics and earns the highest user-study winning rate; real-image tests also support its effectiveness.The synthetic benchmark includes 100 samples per transformation type; real-image evaluation uses 85 OpenImagesV6 samples.Source: Table 1 · p. 6, §6.3 · p. 6, Table 2 · p. 6
6.2. Evaluation Metrics
- Translation evaluation measures distance and accuracy, requiring correct direction, preserved identity, no duplication, and scene consistency; rotation and resizing use angle and scale tolerances.Rotation accuracy allows ±20° from the target; resizing accuracy allows ±10% of the target scaling ratio.Source: §6.2 · pp. 5–6
6.4. Qualitative Evaluation
- Qualitative examples show TALK2MOVE follows instructions more accurately and preserves scene details better than baselines; GPT-Image-1 often changes tone and lighting.Figure 5 includes real and synthetic inputs for translation, rotation, and resizing.Source: §6.4 · p. 6, Figure 5 · p. 8
6.5. Ablation Studies
- With 800 images, RL after SFT further improves translation distance and accuracy on both backbones; with about 80 annotated and 400 input-only pairs, RL reaches full-data-comparable performance where SFT gains are not meaningful.The comparison uses Flux-Kontext and QwenImageEdit backbones, trained on translation data.Source: Table 3 · p. 6, §6.5 · p. 7
- Image-level L1 distance is comparable to other state-of-the-art models, while CLIP similarity is too coarse to detect subtle background changes reliably.Similar CLIP image scores can accompany visibly similar levels of background preservation.Source: §6.5 · p. 7, Table 5 · p. 7
- Active step sampling reduces iteration time by 14% versus sliding-window sampling and 49% versus full sampling, while improving translation distance and accuracy with four sampled steps per rollout.The chosen translation exit step is step 4; focusing on steps with larger transformation space also accelerates convergence.Source: §6.5 · p. 7, Table 4 · p. 7
- The spatial reward yields a rotation error of 0.2861 and accuracy of 29.55%, versus 0.3294 and 11.63% for the VLM-based reward.The comparison evaluates prompt alignment with Qwen2.5-VL-Instruct; the VLM reward is described as overly optimistic and unstable.Source: Figure 4 · p. 7, §6.5 · p. 8
7. Conclusion and Discussion
TALK2MOVE combines GRPO-style training, spatially grounded rewards, and active step sampling for text-guided geometric object transformation. The authors report better spatial accuracy and visual coherence than existing open- and closed-source models.
7. Conclusion and Discussion
- TALK2MOVE outperforms existing open- and closed-source models in spatial accuracy and visual coherence.The authors suggest extending the RL recipe to GANs and autoregressive models as future work.Source: §7 · p. 9
- GRPO-style training and spatially grounded rewards support data-efficient learning with reduced reliance on expensive paired data and finer object transformations.The rewards help disentangle objects from the scene.Source: §7 · p. 9
- Off-policy step evaluation and active sampling prioritize informative steps with higher reward variance and larger transformation space to improve computational efficiency.Source: §7 · p. 9
- The authors propose extending the RL-based recipe to GANs and autoregressive models for verifiable, controllable visual generation as future work.Source: §7 · p. 9