GIFT: Goal-Injected Fine-Tuning for Efficient Manipulation Policy Adaptation

• Existing approaches either do not exploit pretrained VLA backbones or integrate goals in ways that can alter pretrained pathways and require substantial joint retraining.Source: §1 · p. 2
• GIFT injects goal-image representations into observation features through a zero-initialized convolution, which avoids early interference and progressively incorporates goal signals during fine-tuning.Source: §1 · p. 2
• Its refined image-editing method uses in-context edit prompts to generate goal images aligned with task instructions and visually consistent with the observation scene.Source: §1 · p. 2
• GIFT consistently improves VLA performance, achieving 6.0% and 13.4% across two SIMPLER settings and 4.7% on LIBERO, with only a single fine-tuning epoch.Source: §1 · p. 2

• VLA models transfer visual, language, and scene knowledge to manipulation, but scarce and heterogeneous robotic data limits their out-of-distribution performance.Source: §2 · p. 2
• Subgoal-image pipelines guide downstream policies, but integrating goal-conditioned components into existing VLA baselines typically requires full system retraining and substantial computational overhead.Source: §2 · pp. 2–3
• Accelerated diffusion editors can perform editing in 3–4 denoising steps, often at the cost of reduced visual fidelity or limited fine-grained control.Source: §2 · p. 3

3.1 Framework Overview

• GIFT first generates a goal image from the initial observation and instruction, then conditions the pretrained VLA policy on that image when predicting actions.This two-stage pipeline is intended to strengthen goal alignment and manipulation reliability across tasks.Source: §3.1 · p. 3

3.2 Goal-Injected Fine-Tuning

• A shared image encoder extracts goal and current-observation features; their semantic difference is injected through a zero-initialized convolution to preserve early policy stability.The method projects the features into a shared aligned space, and fine-tuning updates only the goal-image projector, zero convolution, and action head while freezing most pretrained parameters.Source: §3.2 · p. 4

3.3 Image Editing for Robotic Goal

• The image-editing pipeline uses latent inversion, combines the observation latent with random noise in a text-to-image diffusion Transformer, and denoises toward an instruction-aligned goal image.In-context editing prompts specify the task, while observation-structure injection during early denoising helps maintain structural alignment with the original scene.Source: §3.3 · pp. 4–5

4.1 Experimental Setup

• Evaluations cover SIMPLER’s Visual Matching and Variant Aggregation settings, four LIBERO task suites, and real-world pick-up and basket tasks, with real-world trials under differing lighting.Simulation models are fine-tuned for one epoch; real-world π0.5 and GIFT variants use LoRA fine-tuning.Source: §4 · p. 5, §4.1 · p. 5

4.2 Experimental Results

• GIFT improves CogACT by 6.0% on SIMPLER Visual Matching and 13.4% on Variant Aggregation; OpenVLA+GIFT improves by 4.7% on LIBERO.OpenVLA+GIFT reaches an 81.2% average success rate across LIBERO task suites.Source: §4.2 · p. 6, Table 3 · p. 6
• On real-world tasks, GIFT raises success rates in both ID and dim-light OOD settings, with larger gains on both OOD tasks.Task1-ood rises from 33.3% to 51.7%, and Task2-ood from 23.3% to 36.7%.Source: Figure 4 · p. 6, §4.2 · p. 6
• Across manipulation tasks, the proposed editor produces instruction-aligned goal images with better object localization, semantic coherence, and structural integrity than SuSIE and Ghil-Glue.For “Open fridge,” it renders the door open while both comparison methods fail to depict that transformation accurately.Source: Figure 5 · p. 7, §4.2 · p. 7
• GIFT strengthens existing policy capabilities rather than creating new skills; OpenVLA remains at 0% success on PutInDrawer after goal injection.The authors bound its gains to the behavioral scope of the original VLA policy.Source: §4.2 · p. 7
• After one epoch of fine-tuning, GIFT completes “put the carrot on the plate,” while the CogACT baseline fails and CoT-VLA shows almost no meaningful behavior.The CoT-VLA reproduction used the same dataset and took 48 hours.Source: §4.2 · pp. 7–8, Figure 6 · p. 8

4.3 Ablation Study

• Semantic Contrast improves overall performance from 57.5% to 67.7% by capturing task-relevant semantic changes; Observation-Structure Injection more reliably infers intended end states while preserving scene-layout consistency across complex scenes.Without injection, edits can show diminished retention of secondary objects and variations in scene brightness.Source: §4.3 · p. 8
• Cross-attention causes optimization instability and near-zero success, while FiLM underperforms ZeroConv; across nine tasks, the proposed goal images outperform generated baselines and nearly match ground truth.Table 4 varies goal-image quality while holding the GIFT injection module fixed.Source: §4.3 · pp. 8–9, Table 4 · p. 8

• GIFT injects goal features through a zero-initialized convolution to support stable adaptation without disrupting the pretrained policy.The framework also introduces refined image editing to create goal images aligned with observations and task instructions.Source: §5 · p. 9
• With minimal fine-tuning, GIFT enhances goal understanding and policy robustness across the reported experiments.Source: §5 · p. 9

• Goal-image editing adds computational overhead, while separating goal generation from action prediction may create efficiency bottlenecks and distributional mismatches.The authors suggest faster image editing or unified end-to-end architectures as possible directions.Source: Limitations · p. 9
• Because GIFT freezes most pretrained VLA parameters, it enhances goal awareness but cannot extend capabilities beyond tasks the baseline already supports.Source: Limitations · p. 9

• Unless specified otherwise, VLA models are fine-tuned for one epoch with batch size 256 and constant learning rate 1e−5.Source: §A.1 · p. 12
• Training uses PyTorch FSDP on four NVIDIA L20 GPUs; WidowX models use BridgeDataset V2, Google Robot models use Fractal, and LIBERO OpenVLA uses LIBERO.Source: §A.1 · p. 12
• Table 5 presents training time in hours for one epoch across datasets using four NVIDIA L20 GPUs.Source: Table 5 · p. 12

A.2 Real-Robot Adaptation and Training Details

• For real-robot adaptation, the authors collected 30 trajectories per task at 30 Hz for pickup and basket-placement tasks, varying object and basket positions.The data includes synchronized visual observations, language prompts, and joint-angle action annotations.Source: §A.2 · p. 12
• The π0.5 policy is first fine-tuned with LoRA, then GIFT is trained for one epoch on the third-view branch while the wrist-view branch remains unchanged.The GIFT stage takes approximately two hours on one NVIDIA L20 GPU, compared with approximately 40 hours for the policy fine-tuning stage.Source: §A.2 · p. 12

A.3 Image Editing Fine-tuning Details.

• The image editor uses initial trajectory frames, final-frame goal images, and task instructions, and is fine-tuned for three epochs with LoRA while its text encoders and VAE remain frozen.Conditional dropout is applied to text and image conditions, each at 0.1.Source: §A.3 · pp. 12–13
• The SIMPLER examples show goal images aligned with task instructions and intermediate execution frames for object-centric manipulation tasks.Examples include precise placement, stacking, and goal-oriented repositioning.Source: Figure 8 · p. 13

B.1 SIMPLER

• SIMPLER evaluations cover Google and WidowX tasks, with Google tested under Visual Matching and Variant Aggregation and WidowX under Visual Matching only.WidowX has 24 original trials per task, repeated five times; SIMPLER is used for efficient benchmarking in high-fidelity simulation.Source: §B.1 · p. 13

B.2 LIBERO

• LIBERO evaluation spans 40 language-conditioned tasks across four suites, with 50 rollouts per task in high-fidelity Robosuite/MuJoCo environments.Agents receive RGB-D images and proprioception at a 20 Hz control frequency.Source: §B.2 · pp. 13–14

B.3 Real World.

• Real-robot tests use a 6-DOF arm and two manipulation tasks under training-matched lighting and dim OOD lighting, with 30 trials per method, setting, and task.Success is defined separately for stable object lifting and placing the target inside a basket.Source: §B.3 · p. 14

B.4 Comparison Algorithms

• The comparison set includes pretrained VLA policies and goal-image approaches, including SUSIE and GHIL-Glue, alongside RT-series models, Octo, Diffusion Policy, and others.The goal-image comparison evaluates SUSIE on BridgeDataset V2 using its released checkpoint.Source: §B.4 · pp. 14–16

C.1 Is Language Still Necessary?

• Without language instructions, the goal-conditioned variant reaches 0% success on both Grasp and Stack, while GIFT gains over CogACT.The comparison indicates that goal images do not replace the pretrained language-conditioned policy structure.Source: §C.1 · p. 16, Table 6 · p. 16

C.2 Robustness to Editing Stochasticity.

• Across three random seeds, success rates vary only slightly: Grasp ranges from 87.0% to 88.3%, Move from 85.7% to 86.9%, and Drawer from 62.9% to 64.7%.Standard deviation is below 1% for all three tasks in the SIMPLER Visual Matching setting.Source: §C.2 · p. 16

C.3 Computational Overhead of GIFT

• GIFT adds 20M–72M parameters across the profiled baselines and 2–4 ms per policy step; goal editing takes about 4.2 seconds once per episode.Episode-level overhead ranges from 22.7% for CogACT to 63.7% for π0.5; OpenVLA average steps decrease from 125.5 to 113.6.Source: §C.3 · pp. 16–17, Table 8 · p. 17

D Editing Example

• On held-out samples, the editor produces visually realistic, semantically consistent goal images aligned with task intent across tabletop and kitchen-like scenes.The test set uses the remaining 10% of BridgeDataset V2 after training on 90% of BridgeDataset V2 and 30% of LIBERO.Source: D Editing Example · p. 17

E Visual Example

• Qualitative examples span SIMPLER and LIBERO simulations and real-robot rollouts under in-domain and OOD lighting, across varied tasks and platforms.The examples illustrate goal editing and policy execution across different task complexities, object categories, and visual conditions.Source: E Visual Example · p. 17

E.1 Qualitative Results on SIMPLER

• In SIMPLER, Google Robot examples under Visual Matching and Variant Aggregation show task-relevant, spatially coherent goals and aligned execution progress across changing scenes.WidowX examples cover fine-grained object manipulation despite variation in geometry, placement, and background appearance.Source: §E.1 · pp. 17–18

E.2 Qualitative Results on LIBERO.

• In LIBERO, edited goals remain visually accurate and instruction-aligned across suites, grounding container placement, drawer interaction, and multi-object manipulation.Examples include simple and compound goals and intermediate states in tasks requiring sequential reasoning and spatial planning.Source: §E.2 · p. 18

E.3 Qualitative Real-Robot Evaluation

• The real-robot evaluation covers bottle pickup and placing a bottle into a basket, with full execution sequences viewed from a third-view camera.The OOD condition changes lighting while keeping the robot, cameras, workspace, objects, and task definitions unchanged.Source: §E.3 · p. 18
• Under in-domain lighting, the robot approaches the target object and completes the manipulation in both tasks.The pickup requires grasping and lifting the bottle; placement requires putting it inside the basket.Source: §E.3 · p. 18
• Under dim OOD lighting, the robot still identifies the task-relevant object, grasps it, and completes goal-directed behavior.The rollouts support robust real-world execution under this visual distribution shift.Source: §E.3 · p. 18