PaperlayerView exact revision on arXiv

Source-linked AI summary

NPU Offloading of a Frozen Visual Encoder for Robot Policy Training

Hyojun Yun, Seungjae Won, Hyungpil Moon

arXiv:2608.15002v1cs.ROcs.ARcs.LG

TL;DR

Frozen visual encoders still consume GPU compute during robot policy training, motivating NPU offloading despite transfer and latency costs. This study evaluates asynchronous GPU–NPU training and finds lower accelerator-board energy but longer training and slightly lower policy success.

  • Problem

    Frozen visual encoders still require repeated GPU forward passes for changing images, leaving the energy impact of offloading insufficiently studied.

  • Method

    An asynchronous pipeline offloads the frozen AR-Actor visual encoder to an INT8 NPU while training the FP32 action expert on a GPU across four boundaries.

  • Results

    Energy per sample fell 17.1–27.9%, while training time rose 15.2–37.7% and policy success was 91.44–92.89% versus 93.33% for GPU-only training.

  • Takeaways & Limitations

    NPU offloading can reduce accelerator-board energy for robot policy training, but the evaluated system trades this reduction for longer training and lower observed success rates.

  • Takeaways & Limitations

    Energy measurements excluded CPU, DRAM, PCIe, and power-supply losses, so total system-energy reductions may differ from the reported results.

Abstract

from arXiv · show

When a robot policy is trained for a new task or dataset, its visual encoder can be frozen and only its action generation module trained, reducing training cost. Freezing removes the encoder's backward pass, but its forward pass must still run at every training step because the input images change, so it keeps consuming GPU compute. We therefore ask whether moving this computation to a low power AI accelerator such as an NPU can reduce total energy despite the added data transfer and longer training time, and how it affects policy performance. We built an asynchronous training pipeline that uses both a GPU and an NPU for the AR-Actor specialist. The frozen visual encoder runs in A8W8 INT8 on a Mobilint Aries2 NPU, while the FP32 action expert is trained on an NVIDIA GeForce RTX 5060 Ti GPU. We compared a GPU-only baseline with four conditions, L1 to L4, which gradually extend NPU offloading from one to four Transformer encoder layers. Each condition was trained for 30,000 steps with three random seeds. We measured GPU board power for the GPU-only condition and combined GPU and NPU board power for the NPU conditions. Energy per sample decreased by 17.1% in L1, which offloaded ResNet18 and the first encoder layer, and by 27.9% in L4, which offloaded ResNet18 and all four encoder layers. In contrast, training time per sample increased by 15.2% in L1 and 37.7% in L4, and peak allocated GPU memory decreased by 19.8 to 20.7%. The 15 resulting policies were each evaluated with the same 300 environment seeds, for a total of 4,500 simulator rollouts. The combined success rate was 93.33% for GPU-only and 91.44 to 92.89% for the NPU conditions. These results show that NPU offloading of a frozen visual encoder can reduce training energy, but it increases training time and lowers policy success rate by 0.44 to 1.89 percentage points compared with GPU-only training.

I. INTRODUCTION

Freezing a visual encoder removes its gradients and optimizer states but not its repeated forward pass, which still consumes GPU resources as camera inputs change. This work studies asynchronous NPU offloading for AR-Actor while retaining the trainable action expert on the GPU and measuring energy, time, memory, and policy success together.

  • Motivation: Growing vision-language models increase training power and GPU-memory demands for visual robot policies.Visual robot policies generate actions from camera observations and must be trained on data from each new task or environment.
  • Motivation: Freezing the visual encoder reduces gradients, optimizer states, power, and GPU memory, but its forward pass still runs at every step because visual inputs change.The frozen encoder must generate new features for the action expert during each training step.
  • Target model: The study targets AR-Actor, whose action expert uses K/V values from all four encoder layers, unlike prior work that passed one intermediate activation.AR-Actor uses ResNet18, and the present design preserves K/V values after splitting computation between the GPU and NPU.
  • Method: The asynchronous pipeline runs the frozen visual encoder on a Mobilint Aries2 NPU and trains the action expert and optimizer on an RTX 5060 Ti GPU.The GPU reconstructs per-layer K/V values from intermediate visual feature tokens sent by the NPU.
  • Experimental conditions: Four offloading boundaries include ResNet18 plus the first, two, three, or four Transformer encoder layers, compared with a GPU-only baseline using three training seeds.The study changes the offloading boundary and jointly measures energy, training time, GPU memory, and policy success rate.

II. METHOD · A. Target Policy and Training Scope · B. Two Training Phases

The study targets the AR-Actor specialist by freezing its visual encoder after full-policy pretraining and retraining a newly initialized action expert. It compares five controlled Phase B conditions on a bimanual ALOHA cube-transfer task using identical training setups.

  • A. Target Policy and Training Scope: The visual encoder comprises ResNet18, an image projection module, and a four-layer Transformer encoder.These components jointly extract visual features, and the implementation contains 51,515,228 parameters.
  • A. Target Policy and Training Scope: The training scope first covers full-policy training, followed by freezing the visual encoder and training only a new action expert.The visual encoder parameters are reused across every comparison condition.
  • B. Two Training Phases: The experiment uses a bimanual ALOHA cube-transfer task with 50 demonstration episodes and 20,000 training frames.The robot transfers a cube from one hand to the other.
  • B. Two Training Phases: Phase A trains the full policy on a GPU for 200,000 steps before evaluating the resulting policy for 10 episodes.The evaluation checkpoint supplies the frozen visual encoder used in Phase B.
  • B. Two Training Phases: Phase B compares GPU-only training with four NPU offloading conditions while keeping the visual encoder frozen.Each condition combines the frozen visual encoder with a newly initialized action expert and runs for 30,000 steps.
  • B. Two Training Phases: The five conditions share initial action-expert weights, data order, random history masks, optimizer settings, and training-step count within each seed.This design compares the training conditions from the same initial state and setup.
  • B. Two Training Phases: AdamW training uses a learning rate of 1 × 10−5, weight decay of 1 × 10−4, and batch size of 8.These settings are applied consistently within each training seed.

C. Offloading Boundaries

The comparison varies how much of the frozen visual encoder is offloaded from the GPU to the NPU, while the GPU reconstructs decoder key/value inputs from exchanged encoder tokens. The NPU compiler could not compile direct key/value output from each layer.

  • Offloading configuration: GPU-only runs the entire frozen visual encoder in FP32 on the GPU, whereas NPU conditions run selected components in A8W8 INT8 on the NPU.NPU offloading includes ResNet18, the image projection module, and the first one to four Transformer encoder layers.
  • Token exchange: The NPU sends Transformer encoder input tokens and output tokens from each offloaded encoder layer to the GPU.The AR decoder uses keys and values from all four encoder layers.
  • Token exchange: The GPU reconstructs key/value values for each encoder layer because direct NPU key/value output could not be compiled.The initial approach of having the NPU output key/value values directly from each layer failed at compilation.

D. Asynchronous Overlap

The pipeline asynchronously overlaps GPU action-expert training for batch N with NPU computation of the frozen visual encoder for batch N+1. Because the NPU path is longer, GPU wait time remains despite overlap.

  • Asynchronous scheduling: A background thread runs the NPU forward pass for batch N+1 while the GPU trains the action expert for batch N.This asynchronous schedule advances the next batch’s frozen visual features during current-batch training.
  • Overlap limitation: The NPU path is longer than the GPU training path, leaving some GPU wait time after the operations overlap.Figure 1 reports the time breakdown of this asynchronous process for each condition.
  • Precision and software: FP32 GPU training is paired with an A8W8 INT8 NPU visual encoder.The implementation uses PyTorch 2.9.1+cu130, LeRobot 0.3.4, and NPU tools qbcompiler 1.2.0 and qbruntime 1.2.0.

III. EXPERIMENTAL SETUP · A. Hardware, Software, and Quantization

The experiments used fixed hardware, software, and precision settings, with INT8 NPU calibration and held-out comparisons against GPU FP32 outputs. Cosine similarities between NPU and GPU visual features remained above 0.9975 for all offloading levels.

  • A. Hardware, Software, and Quantization: All experiments used the hardware, software, and precision settings listed in Table III.
  • A. Hardware, Software, and Quantization: 48 real frames from training episodes 0 to 7 were sampled evenly for NPU quantization calibration.
  • A. Hardware, Software, and Quantization: Calibration evaluated whether NPU visual features remained close to GPU outputs after INT8 quantization and NPU execution.
  • A. Hardware, Software, and Quantization: 40 frames from episodes 40 to 49, excluded from calibration, were used for the held-out feature comparison.
  • A. Hardware, Software, and Quantization: Feature tokens were compared at each NPU–GPU split boundary between the INT8 NPU path and GPU FP32 output.
  • A. Hardware, Software, and Quantization: 0.99752, 0.99763, 0.99782, and 0.99783 were the cosine similarities for L1, L2, L3, and L4, respectively.

B. Data and Evaluation Environment · C. Comparison Conditions and Repetitions · D. Power and Energy

The study used the ALOHA simulated transfer-cube dataset and evaluated 15 policy checkpoints across 4,500 controlled rollouts. Energy per sample was computed by integrating accelerator board power during training and dividing by processed samples, while excluding non-board system components.

  • B. Data and Evaluation Environment: 50 episodes and 20,000 frames comprised the lerobot/aloha_sim_transfer_cube_scripted training dataset.Inputs were one overhead RGB camera and a 14-dimensional robot state; outputs were 14-dimensional actions.
  • B. Data and Evaluation Environment: 300 initial conditions from environment seeds 2000 to 2299 defined the continuous-control policy evaluation.Evaluation used the AlohaTransferCube-v0 simulator environment.
  • C. Comparison Conditions and Repetitions: 15 checkpoints resulted from training GPU-only and L1–L4 under action-expert seeds 2000, 2001, and 2002.GPU and NPU availability were verified before each run, with 20 seconds allowed for system stabilization.
  • C. Comparison Conditions and Repetitions: 4,500 rollouts evaluated every checkpoint using the same 300 environment seeds.GPU-only ran the full visual encoder on the GPU, while each NPU policy preserved its L1–L4 precision and split boundary during evaluation.
  • D. Power and Energy: Once-per-second GPU and NPU board-power measurements were integrated over training with the trapezoidal rule.GPU power came from nvidia-smi, NPU power from mobilint-cli status, and the integrated value was divided by processed samples.
  • D. Power and Energy: GPU-only energy used A = {GPU}, whereas NPU-condition energy used A = {GPU, NPU}.Here, N denotes processed training samples and A the accelerator boards used in a condition.
  • D. Power and Energy: Reported energy was combined GPU-and-NPU board energy rather than total system energy.CPU, DRAM, PCIe, and power-supply energy or loss were excluded.

E. GPU Memory · IV. RESULTS · A. Accelerator Board Energy and Training Time

Peak allocated GPU memory was used to assess training feasibility, while NPU offloading reduced energy per sample across all boundaries. The L4 condition achieved lower energy and board power but required substantially more time per sample.

  • E. GPU Memory: Peak allocated memory, rather than average memory, was measured because the maximum determines training feasibility and batch-size limits.The metric counts the highest amount of memory PyTorch allocated to tensors during training.
  • A. Accelerator Board Energy and Training Time: All four NPU boundaries used less energy per sample than GPU-only.Energy per sample also decreased progressively from L1 to L4 for each training seed.
  • A. Accelerator Board Energy and Training Time: Energy per sample decreased from L1 through L4 for every training seed.Figure 2 connects conditions measured with the same seed and shows means across three seeds.
  • A. Accelerator Board Energy and Training Time: 37.7% longer time per sample in L4 was accompanied by 47.7% lower combined board power than GPU-only.The comparison uses combined GPU and NPU board power for the offloaded condition.
  • IV. RESULTS: 27.9% lower energy per sample in L4 resulted because its board-power reduction exceeded its training-time increase.This is the reported L4 comparison with GPU-only.
  • E. GPU Memory: Figure 3 reports peak allocated GPU memory for each condition as the mean across three training seeds.Each bar represents the condition-level mean.

B. GPU Memory

NPU offloading reduced peak allocated GPU memory primarily when moving from GPU-only to L1, with little further reduction through L4. GPU memory remained necessary because the AR decoder uses K/V values from all four encoder layers.

  • GPU Memory: 234 MiB: Peak allocated GPU memory decreased from GPU-only to L1, but changed little from L1 to L4.Figure 3 reports peak allocated GPU memory for each condition.
  • GPU Memory: All four encoder layers: The AR decoder uses their K/V values, so normalization and K/V projection weights remain on the GPU regardless of the offloading boundary.These weights are needed to reconstruct each layer’s K/V values.
  • GPU Memory: The NPU compiler could not compile a graph that directly outputs K/V values from each layer.This implementation limitation prevented direct layer-wise K/V output from the NPU.

C. Simulator Policy Performance · V. DISCUSSION AND LIMITATIONS · A. Meaning of the Energy Reduction

NPU offloading reduced training energy but slowed training and produced slightly lower simulator success rates than GPU-only training. The energy result reflects the implemented GPU–CPU–NPU system, including changed data movement and scheduling, rather than NPU hardware alone.

  • C. Simulator Policy Performance: 93.33% was the GPU-only success rate, compared with 91.44 to 92.89% for NPU conditions across 900 evaluations per condition.Each condition used three training seeds and the same 300 environment seeds per checkpoint.
  • C. Simulator Policy Performance: 0.44 to 1.89 percentage points lower were the NPU success rates than the GPU-only result.The comparison used the same 300 environment seeds for each checkpoint.
  • A. Meaning of the Energy Reduction: 37.7% slower was L4 than GPU-only, while its combined board power was 47.7% lower.L4 was the deepest offloading boundary evaluated in the experiment.
  • A. Meaning of the Energy Reduction: 27.9% lower was L4’s energy per sample despite its reduced throughput.The result combines the 37.7% slowdown with 47.7% lower combined board power.
  • A. Meaning of the Energy Reduction: Data movement among the GPU, CPU, and NPU differs between GPU-only and NPU conditions.Therefore, the conditions differ in more than which devices run the model.
  • A. Meaning of the Energy Reduction: The scheduling method also changes between GPU-only and NPU conditions.This is another system-level difference alongside device placement and data movement.
  • A. Meaning of the Energy Reduction: The experiments directly show only that the implemented system reduced total accelerator board energy.The result is therefore limited to the tested implementation rather than isolating NPU hardware alone.

B. Limitations

The study’s conclusions are limited by its narrow evaluation scope, confounded hardware and pipeline changes, incomplete energy accounting, few training runs, and partial NPU placement. In particular, compiler limitations left K/V reconstruction on the GPU even in L4.

  • Evaluation scope: The evaluation covered one AR-Actor specialist, one ALOHA simulation task, and one GPU–NPU system, leaving other policies, datasets, batch sizes, and devices untested.More experiments are needed across these configurations.
  • Experimental confounds: Device, numerical precision, data transfer, and scheduling changed together, preventing isolation of GPU-versus-NPU hardware efficiency.The study therefore assesses the combined system configuration rather than a single hardware factor.
  • Energy accounting: Board-power measurements excluded CPU, DRAM, PCIe, and power-supply losses, so total system-energy reductions may differ from the reported results.Power was measured using nvidia-smi and mobilint-cli.
  • Statistical reliability: Each condition used only three training seeds, so small performance differences may not remain consistent in repeated experiments.There was one run for each of the three seeds.
  • NPU coverage: The compiler could not compile the graph directly outputting K/V values, preventing full visual-encoder placement on the NPU.Even in L4, K/V reconstruction computation and tensors remained on the GPU.

VI. CONCLUSION

NPU offloading of a frozen visual encoder reduced accelerator-board energy and GPU memory use, but increased training time and produced slightly lower policy success rates. Future work targets more efficient transfers, improved power measurement, and broader system evaluation.

  • Conclusion: 17.1 to 27.9% lower energy per sample accompanied 15.2 to 37.7% longer training time per sample across three training seeds.Peak allocated GPU memory decreased by 19.8 to 20.7% in the current implementation.
  • Conclusion: 93.33% policy success for GPU-only compared with 91.44 to 92.89% for NPU conditions, although three training seeds prevent determining the exact decrease.The study’s main result is reduced total accelerator-board energy in a working GPU–NPU robot policy training system.
  • Future work: Future work should have the NPU directly output encoder-layer K/V values to reduce intermediate-token transfer and GPU K/V reconstruction.This direction addresses communication and reconstruction overhead introduced by offloading.
  • Future work: Future work should use an external precision power meter and test different GPU–NPU combinations, robot environments, tasks, and policy models.The proposed evaluation would broaden validation beyond the current implementation and hardware setup.
Loading 2608.15002v1…