PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillFM: Generating Skills for LLM Agents via Latent Flow Matching

Zuming Zhang, Jie He, Yizhe Zhang, Jeff Z. Pan

arXiv:2609.39382v1cs.AI

TL;DR

Skill-based agents rely on reusable guidance, but selecting and maintaining growing skill banks brings retrieval and delayed-feedback challenges. SkillFM learns task-conditioned textual skills through a codec and conditional flow matching, then supplies decoded guidance to frozen agents. It reports strong performance across ALFWorld, Search-QA, and WebShop, including an advantage over retrieval under noisy-bank conditions.

  • Problem

    Skill banks require accurate retrieval and maintenance, while the usefulness of updates can be difficult to assess because feedback is delayed and indirect.

    Source: §1 · p. 1

  • Method

    SkillFM encodes and reconstructs textual skills with a codec, then uses improved MeanFlow to generate task-conditioned latent skills for an LLM decoder without test-time retrieval.

    Source: §1 · p. 2

  • Results

    SkillFM achieves the highest reported aggregate ALFWorld and Search-QA scores among compared vectorized-skill approaches, including 39.36% Search-QA micro EM versus 35.62%.

    Source: §4.2 · p. 6

  • Takeaways & Limitations

    The authors conclude SkillFM is a practical alternative to retrieval, producing transferable guidance that remains effective with imperfect skill banks.

    Source: §8 · p. 10

Abstract

from arXiv · show

Textual skills provide reusable guidance for large language model agents, but existing approaches often rely on manually curated skill banks or reinforcement learning with indirect and delayed feedback. We introduce SkillFM (Skill Flow Matching), a generative framework that synthesizes task-conditioned textual skills directly without test-time skill retrieval. Our framework combines a codec for encoding and reconstructing textual skills in a continuous latent space with a conditional flow model trained using improved MeanFlow. At inference time, the learned velocity field enables single-step latent sampling, and an LLM-based decoder converts the sampled representation into textual guidance for a frozen downstream agent. We evaluate the framework on embodied tasks, question answering, and web shopping. On ALFWorld and Search-QA, our method achieves the best overall performance among the compared vector-based skill approaches. Our analyses further demonstrate that latent skill generation is an effective alternative to retrieval-based skill augmentation. Our code and training skill libraries are available at https://github.com/lulushang999/SkillFM.

1 INTRODUCTION

Skill banks can improve LLM agents, but selecting, maintaining, and evaluating their entries creates retrieval and delayed-feedback challenges. SkillFM learns to generate task-conditioned textual skills instead, and the paper reports strong results across embodied tasks, question answering, and web shopping.

1 INTRODUCTION

  • Growing skill banks make relevant procedures harder to distinguish, while delayed and indirect feedback complicates deciding which skills to retain, revise, or discard.Retrieval, utility estimation, and pruning help maintain usability but require the system to manage skills and selection rules.Source: §1 · p. 1
  • SkillFM encodes and reconstructs textual skills with a codec, then uses improved MeanFlow to generate task-conditioned latent skills in one sampling step.An LLM-based decoder renders the sampled representation as textual guidance for the downstream actor.Source: §1 · p. 2
  • The skill bank supplies training examples rather than test-time retrieval, and readable guidance lets the generator guide different actors without changing their parameters.The paper presents this as a route to skill reuse while keeping the downstream actor frozen.Source: §1 · p. 2
  • On ALFWorld, SkillFM achieves 82.14% success on seen tasks and 84.33% on unseen tasks (83.21% overall), improving over reported LatentSkill results by approximately 7.8 and 14.9 percentage points, respectively.Source: §1 · p. 2
  • With 25% irrelevant skills, SkillFM outperforms embedding-based retrieval from the same bank across ALFWorld, Search-QA, and WebShop.The authors identify this as evidence for learned generation when accumulated skills are imperfect.Source: §1 · p. 2

2 RELATED WORK

Prior skill-based systems accumulate reusable procedures but must coordinate their acquisition, selection, and maintenance. Skill internalization reduces repeated context overhead, while transferring learned skills across heterogeneous executors remains a challenge.

Skill-based agent self-evolution

  • Skill-based self-evolution systems use interaction experience as reusable procedures, but require mechanisms for deciding what to retain, retrieve, and revise.Approaches include recursive skill updates, trained curators, and alternating controller optimization with bank refinement.Source: §2 · p. 3
  • Reliable skill-bank evolution can require trajectory attribution, candidate verification, regression checks, and pruning, even with frozen executors.These requirements motivate task-conditioned generation without a separate test-time bank-management loop.Source: §2 · p. 3

Skill internalization

  • Skill internalization reduces repeated context overhead, but approaches tied to policies, adapters, or model weights remain coupled to an execution backbone.SkillFM instead retains a decodable textual interface for reuse across frozen executors.Source: §2 · p. 3

3 METHOD

SkillFM builds a fixed task–skill bank, learns a codec for a continuous skill representation, and trains conditional flow matching over that latent space. At inference, one flow evaluation produces a skill code that is decoded into guidance for a frozen actor.

3 METHOD

  • SkillFM pairs each task condition with a skill, trains a codec, then learns a conditional flow over the codec’s latent space.The generated textual skill is supplied to a frozen executor.Source: §3 · p. 3

3.1 SKILL COLLECTION

  • The fixed training bank is built by iterative execution and summarization, retaining successful skills, removing near-duplicates, and cleaning summaries.Collection stops when success no longer improves; the QA and shopping banks also include authored or revised skills.Source: §3.1 · p. 3

3.2 SKILL CODEC

  • The codec encodes a skill into a latent code and trains a task-conditioned autoregressive decoder to reconstruct readable skill tokens.The decoder receives projected continuous prefix embeddings, and code perturbations expose it to nearby latent points.Source: §3.2 · pp. 3–4

3.3 CONDITIONAL FLOW TRAINING

  • With the codec and query encoder frozen, conditional flow matching trains a Transformer to predict interval-average velocity using the improved MeanFlow objective.At equal interval endpoints the derivative correction vanishes, recovering instantaneous flow matching.Source: §3.3 · pp. 4–5

3.4 SKILL-GUIDED EXECUTION

  • For each task, one flow evaluation predicts and normalizes a skill state, which the frozen decoder turns into guidance held fixed throughout the actor’s rollout.There is no test-time bank lookup; NFE=1 counts only the flow-network call, not query encoding, decoding, or execution.Source: §3.4 · p. 5

4 EXPERIMENTS

SkillFM is evaluated on embodied interaction, web shopping, and question answering, with a frozen Qwen3-8B executor and comparisons to prompting, retrieval, fine-tuning, and adapter baselines. It reports strong benchmark performance and transfer across executor families and scales.

4.1 EXPERIMENTAL SETUP

  • Experiments cover ALFWorld, WebShop, and Search-QA against prompting, retrieval, fine-tuning, and adapter baselines, using Qwen3-8B as the frozen executor.Source: §4.1 · p. 5

4.2 MAIN RESULTS

  • SkillFM has the highest aggregate ALFWorld and Search-QA scores among compared vectorized-skill approaches, with 82.14% seen and 84.33% unseen ALFWorld success and Search-QA micro EM rising from 35.62% to 39.36%.Tables 1 and 2 report ALFWorld and Search-QA results; WebShop results are reported separately.Source: §4.2 · p. 6, Table 1 · p. 6, Table 2 · p. 6
  • On WebShop, SkillFM achieves 20.00% SR and a score of 55.35, exceeding seven of the eight prompting and memory references.Source: §4.2 · p. 6

4.3 GENERALIZATION

  • 21.53–43.43 percentage-point gains across GPT, Llama, and Mistral actors show that the same generator transfers across model families without retraining.The generator was trained on skills derived from Qwen3-8B interactions and guides different frozen executors.Source: §4.3 · p. 7, Figure 3 · p. 7
  • SkillFM improves consistently as Qwen3 executor size increases across all three benchmarks, suggesting larger actors exploit the same generated guidance more effectively.Source: §4.3 · p. 7

5 EXPERIMENTAL ANALYSIS

The analysis tests codec and flow contributions, selected generation hyperparameters, and diagnostics of task conditioning and flow-training dynamics.

5.1 ABLATION STUDIES

  • Removing codec adaptation or bypassing conditional flow inference reduces performance across benchmarks; a task-conditioned decoder alone does not replace conditional code generation.Codec adaptation helps make the latent representation useful for skill generation, while the flow ablation retains the trained decoder and task conditioning but skips flow inference.Source: §5.1 · p. 7, Table 4 · p. 7
  • Decoder-only fine-tuning performs substantially worse than the full method, supporting a division in which the decoder reconstructs skills and flow matching provides task-conditioned generalization.Source: §5.1 · p. 7, Table 5 · p. 7

5.2 HYPERPARAMETER ANALYSIS

  • The selected 2,560-dimensional skill codes and 16 prefix tokens attain or match the highest overall success among tested settings.The analysis varies skill-code dimension and prefix-token count.Source: §5.2 · p. 8, Table 6 · p. 8, Table 7 · p. 8

5.3 FLOW TRAINING ANALYSIS

  • After training, the velocity field produces distinct directions under different task conditions, while the JVP statistic rises early and generally decreases during training.These measurements describe task conditioning and optimization dynamics, not decoded-skill quality or execution success.Source: §5.3 · p. 8, Figure 4 · p. 8

6 ROBUSTNESS ANALYSIS

The robustness analysis compares generated and retrieved guidance under irrelevant-bank contamination, then studies how training-bank size and source-skill quality relate to ALFWorld performance.

6.1 GENERATION VERSUS RETRIEVAL UNDER NOISE

  • With 25% irrelevant skills in the bank, generated guidance outperforms embedding-based retrieval across ALFWorld, Search-QA, and WebShop.The generator learns from the contaminated bank during training; retrieval selects guidance from that bank at test time.Source: §6.1 · p. 8, Table 8 · p. 9

6.2 TRAINING SCALE AND SKILL QUALITY

  • Halving a high-quality bank from 1,000 to 500 skills causes only a small performance drop; with 75% successful source trajectories, performance is lower than at 100% for all bank sizes, but the decrease shrinks as the bank grows.The analysis varies bank size and the fraction of source trajectories that succeeded while keeping evaluation tasks fixed.Source: §6.2 · pp. 8–9, Table 9 · p. 9

7 SKILL GENERATION PROCESS

The paper decodes intermediate latent states along a diagnostic flow trajectory to make changes in generated skill content visible.

7 SKILL GENERATION PROCESS

  • The diagnostic decodes states from noise at t = 1 to the skill endpoint at t = 0, holding task condition and initial noise fixed and using the frozen codec decoder.Source: §7 · p. 9
  • In the heating example, decoded text progresses from malformed repetition to structured guidance specifying the appliance, heating action, placement, action order, and completion conditions.Source: §7 · p. 9
  • Individual trajectories can plateau or regress, and these intermediate decodings are qualitative diagnostics; the main evaluation uses one flow evaluation per skill.Source: §7 · p. 9

8 CONCLUSION

SkillFM generates textual skills for frozen LLM agents through conditional latent flow matching, moving skill-bank information into a generator and removing test-time retrieval. The appendix derives the improved MeanFlow objective and states the assumptions and limits of its population-regression interpretation.

8 CONCLUSION

  • Experiments on ALFWorld, Search-QA, and WebShop demonstrate strong downstream performance, and SkillFM is presented as a practical alternative to retrieval that produces transferable guidance effective with imperfect skill banks.The framework generates skills for frozen LLM agents through conditional latent flow matching and removes test-time skill retrieval.Source: §8 · pp. 9–10

A DERIVATION OF THE IMPROVED MEANFLOW OBJECTIVE

  • The appendix derives the objective using MeanFlow’s average-velocity formulation and improved MeanFlow’s learned boundary-field parameterization.The average-velocity formulation follows MeanFlow, while the boundary-field parameterization follows improved MeanFlow.Source: A DERIVATION OF THE IMPROVED MEANFLOW… · p. 14

A.1 FROM CONDITIONAL FLOW MATCHING TO AVERAGE VELOCITY

  • The marginal velocity is the conditional mean of paired velocity, and its ODE trajectory need not match an individual data–noise pair’s linear interpolation.The endpoint qualification assumes the trajectory limit exists and velocity is integrable, accommodating the codec’s spherical data support at t = 0.Source: §A.1 · p. 14

A.2 BOUNDARY PARAMETERIZATION AND LOSS EQUIVALENCE

  • The average-velocity identity links interval-average and instantaneous velocities; diagonal network evaluation estimates the latter, and equivalent objective forms share gradients under consistent weighting.At r = t, the derivative correction vanishes and the residual reduces to the standard flow-matching residual for the diagonal estimate.Source: §A.1 · p. 14, §A.2 · p. 14

A.3 POPULATION-REGRESSION INTERPRETATION

  • Under the stated assumptions, unweighted or fixed-nonnegative-weight population loss targets marginal velocity up to irreducible conditional variance; the predicted tangent removes sample-pair variability from the derivative input.The population interpretation need not apply to residual-dependent adaptive weights.Source: §A.3 · p. 15
  • These identities concern the exact field and forward regression objective; they do not guarantee that a learned field or single-evaluation sampler generates accurately.The passage also states that the correction does not establish a uniform reduction in complete-residual variance or guarantee downstream improvement.Source: §A.3 · p. 15

B TRAINING RECIPES AND EXECUTION DETAILS

The section specifies domain-specific training and execution settings for SkillFM across ALFWorld, Search-QA, and WebShop, then details codec and flow implementation. It also reports ALFWorld evaluation and identifies the scopes of the WebShop and cross-scale tables.

B TRAINING RECIPES AND EXECUTION DETAILS

  • Training uses the same two-stage architecture across ALFWorld, Search-QA, and WebShop, with domain-specific skill banks and schedules and separate single-hop and multi-hop Search-QA runs.Table 10 summarizes training recipes, and Table 11 specifies actor budgets.Source: B TRAINING RECIPES AND EXECUTION DETAILS · p. 15, Table 10 · p. 15, Table 11 · p. 16
  • Multi-hop uses 2,451/272 training/holdout partitions, while six flow candidates are assessed on fixed-stage holdout criteria rather than task EM.Its codec schedule sets a 24-epoch minimum, 40-epoch maximum, and patience 8; training completes 24 epochs and selects epoch 9. The single-hop run resumes from step-10,000 EMA, rebuilds Adam, and uses global batch 2,304 through step 30,000; token and action limits apply independently.Source: B TRAINING RECIPES AND EXECUTION DETAILS · pp. 15–16
  • SkillFM reaches 228/274 (83.21%) total ALFWorld success, including 115/140 (82.14%) seen and 113/134 (84.33%) unseen tasks, with NFE = 1.Average interaction length is 16.59 steps on seen tasks and 15.79 steps on unseen tasks.Source: B TRAINING RECIPES AND EXECUTION DETAILS · p. 16

B.1 METHOD IMPLEMENTATION DETAILS

  • The codec uses Qwen3-Embedding-4B encoders and a Qwen3-4B-Instruct-2507 decoder (Zhang et al., 2025; Yang et al., 2025), with a projector that outputs 16 decoder-width vectors.Stage I trains the projector and LoRA adapters while freezing pretrained backbone weights.Source: §B.1 · p. 16
  • Latent perturbation rotates unit skill code z toward an orthogonal unit tangent by angle α, applied with probability 0.75 and capped at 0.35 radians.The angular standard deviation is 0.16 radians, and perturbation strength ramps over eight epochs.Source: §B.1 · p. 16
  • The 2,560-dimensional flow network has 12 Transformer layers of width 768 and 12 attention heads, conditioned on query, time, and interval length through adaLN-Zero.Stage II trains only the flow network, with the codec and query encoder frozen and no auxiliary head.Source: §B.1 · p. 16

C.1 WEBSHOP MAIN RESULTS

  • Table 12 compares the frozen Qwen3-8B executor with prompting and memory baselines reported by Xia et al. (2026), using success rate and task score.SR denotes complete success, and Score denotes mean task reward.Source: §C.1 · p. 17, Table 12 · p. 17

C.2 TRANSFER ACROSS EXECUTOR SCALES

  • Figure 3 reports cross-family transfer, while the executor-scale tables report Search-QA EM across seven datasets and WebShop success over 500 tasks and mean reward on a 0–100 scale.Search-QA Avg is the unweighted mean over seven datasets.Source: §C.2 · p. 17, Table 13 · p. 17, Table 14 · p. 17

C.3 TRAINING-BANK SIZE AND QUALITY

The ALFWorld analysis examines training-bank quality and observable execution failures. Target acquisition dominates the recorded failures, while the examples show that correct textual guidance does not ensure the agent preserves progress or grounds object categories correctly.

C.3 TRAINING-BANK SIZE AND QUALITY

  • The source-trajectory success rate is the proportion of successful trajectories used to build the bank; skills are derived from a mixture of successful and unsuccessful trajectories.Table 15 reports success rates on 140 seen and 134 unseen tasks, with overall results pooled across 274 tasks.Source: §C.3 · p. 17, Table 15 · p. 18

D ALFWORLD FAILURE ANALYSIS

  • All 46 failed episodes exhausted the 50-step budget, although all 46 associated skills decoded successfully.The rollouts alone do not establish which model component caused the execution deviations.Source: D ALFWORLD FAILURE ANALYSIS · p. 18

D.1 FAILURE STATISTICS

  • 20/25 seen and 19/21 unseen failures involved failure to acquire the correct target, the dominant observable bottleneck.The analysis describes observable execution behavior and does not identify a causal fault in a particular model component.Source: §D.1 · p. 18

D.2 SEEN: UNDOING A COMPLETED PLACEMENT

  • In the seen two-object example, the agent repeatedly retrieved an already placed instance despite skill guidance to track it, and placed no second instance before timeout.The skill’s scene-specific route also needed adaptation to the observed object identifiers.Source: §D.2 · p. 19

D.3 UNSEEN: SUBSTITUTING THE WRONG OBJECT CATEGORY

  • In the unseen example, the agent acquired and placed a butterknife instead of the required knife, despite the skill specifying the target category.The actions followed the clean-then-place procedure but did not satisfy the target category.Source: §D.3 · p. 19

E SKILL GENERATION PROCESS DETAILS

The section traces decoded guidance along a diagnostic latent trajectory and examines downstream success and reconstruction across latent dimensions and prefix lengths. The results show that better training reconstruction does not necessarily correspond to higher downstream success.

E SKILL GENERATION PROCESS DETAILS

  • The diagnostic decodes six independently sampled states along a fixed five-step Euler trajectory with task condition and initial noise held fixed.It is separate from the one-step inference used in the main evaluation.Source: E SKILL GENERATION PROCESS DETAILS · p. 19
  • In the heating example, guidance progresses from malformed text to a structured protocol that separates the state-changing heat action from final placement.Intermediate states recover the correct appliance before the heat action; the endpoint retains the action order and completion condition.Source: E SKILL GENERATION PROCESS DETAILS · pp. 19–20
  • The four displayed cases include plateaus and regressions, and assess decoded guidance rather than concrete scene routes or environment success.The heating case was selected for display after inspection; the cases use cached noise from a checkpoint-selection holdout that may have been seen by the codec.Source: E SKILL GENERATION PROCESS DETAILS · p. 20

F.1 LATENT DIMENSION AND DOWNSTREAM SUCCESS

  • The 2,560-dimensional setting has the highest seen and overall success among tested dimensions, while 512 dimensions has the highest unseen success.The selected dimension is based on overall success; results do not imply a monotonic benefit from increasing dimension.Source: §F.1 · p. 22

F.2 PREFIX LENGTH AND DOWNSTREAM SUCCESS

  • K = 8 and K = 16 tie for highest pooled success at 83.21%, while increasing the prefix to K = 32 does not improve overall success.K = 32 has the highest unseen success but lower seen and pooled success; the main method uses K = 16.Source: §F.2 · p. 22

F.3 PREFIX LENGTH AND RECONSTRUCTION

  • Training loss drops by more than 99% between epochs 1 and 8 for every prefix length, but validation curves flatten and later rise.K = 32 reaches the lowest training loss at epochs 8, 16, and 24, while downstream success is highest jointly for K = 8 and K = 16.Source: §F.3 · p. 22

F.4 FLOW-TRAINING DIAGNOSTICS

The diagnostics compare flow-training objectives with latent endpoint geometry and track directional-derivative magnitudes during one ALFWorld training run. Lower training loss and declining typical derivative magnitudes do not establish improved endpoint geometry, task success, or a causal stabilization effect.

F.4 FLOW-TRAINING DIAGNOSTICS

  • From 10k to 30k updates, training loss falls 32.50% from 5.670 to 3.827, while holdout Chamfer rises 3.14% from 0.014282 to 0.014731.The 10k checkpoint has the lowest holdout Chamfer among the three evaluated checkpoints, supporting its selection over the final checkpoint.Source: Table 18 · p. 24, §F.4 · p. 24
  • Generated-to-reference mean angles remain between 0.09415 and 0.09498 radians, and pooled P95 stays between 0.28164 and 0.28416 radians, with neither showing sustained improvement after 10k.These latent-geometry measurements do not measure decoded-skill correctness, textual diversity, or task success.Source: §F.4 · p. 24

F.5 DIRECTIONAL-DERIVATIVE DYNAMICS

  • The directional-derivative mean falls from 371.03 before 10k to 186.02 before 30k, but the P90 values are non-monotonic and late spikes remain.The mean is 196.76 before 20k; the largest logged value is 2722.71 at update 24,117.Source: Figure 8 · p. 25, §F.5 · p. 25
  • J_k records weighted directional-derivative magnitudes in one training run; without controlled ablations, these measurements establish neither causal stabilization nor global smoothness.The run uses off-diagonal sampling probability 0.25, and the decline in typical JVP magnitude does not coincide with improved holdout Chamfer.Source: §F.5 · p. 25

F.6 SCOPE OF THE HYPERPARAMETER COMPARISONS

  • The appendix comparisons cover latent dimension, prefix length, and flow-training dynamics; condition-dependent directions and intermediate decoded skills are examined elsewhere.Domain-specific training recipes are in Appendix B.Source: §F.6 · p. 25

Related papers

Loading 2609.39382v1…