Agent Priors-guided Policy Learning

• Few demonstrations leave robots facing new goals, operation orderings, and object locations, while task-level composition and skill-level generalization depend on one another.The task level may lack knowledge of where a skill works, while each skill is trained without the structure used by task-level reasoning.Source: §1 · p. 1
• Existing interfaces can omit the physical conditions under which a skill works: language descriptions say what a tool is for, while symbolic planning uses preconditions and effects.The introduction also notes that learned behavior’s generalization conditions can remain implicit in vision-language-action models.Source: §1 · pp. 1–2
• A structural prior shapes a skill’s generalization during training and tells the runtime agent where that policy is expected to apply.For example, an object-relative grasp prior can support choosing the grasp policy when an object appears at a new absolute location.Source: §1 · p. 2
• APPL’s construction agent segments demonstrations, proposes several priors per skill, and trains and verifies one policy per prior; its runtime agent selects and composes those policies using their descriptions.The stored descriptions include prior, handoff conditions, observed training support, and verification evidence.Source: §1 · p. 2, Figure 1 · p. 2
• Across MetaWorld and ManiSkill, APPL improves out-of-distribution skill generalization and supports unseen compositions; hiding interface information reduces performance.On six MetaWorld tasks, APPL reaches 89.6% OOD success with two demonstrations versus 37.9% for the fixed relational prior; it solves 8 of 16 unseen ManiSkill compositions.Source: §1 · p. 3

• APPL targets both skill and compositional generalization by using the policy’s structural prior as the interface between learning and composition.Source: §2 · p. 3
• Skill-composition research includes symbolic planning and learned abstractions, while SymSkill jointly learns predicates, operators, and skills but fixes its structure to relative frames and dynamical-system controllers.APPL instead permits different priors and alternative policies for each skill, with the prior also describing where a policy applies.Source: §2 · p. 3
• Few-demonstration policy methods build in task structure, but a human typically chooses the prior per task and the prior is used for training rather than later composition.The passage lists object-centric representations, functional correspondence, affordances, equivariance, task-relative frames, and visual cues as examples.Source: §2 · p. 3
• Language models already design parts of robot-learning pipelines, but LGA and KALM each choose one representation prior per task and use it only for training.APPL proposes several kinds of priors per skill, keeps the resulting policies as alternatives, and reuses each prior as a runtime description for composition.Source: §2 · pp. 3–4

• A system consists of a composer and a library that may hold several policies per skill, with each policy exposed to the composer through an interface.Construction uses complete, unsegmented demonstrations and task knowledge to produce a frozen system aimed at maximizing expected success.Source: §3 · p. 4
• Skill generalization requires a policy’s success region to include undemonstrated start states, such as shifted objects or states left by another skill.Object-position changes with the task and skill sequence fixed define motion-level OOD in this formulation.Source: §3 · p. 4
• Compositional generalization covers skill sequences or handoffs absent from demonstrations, while task-level OOD changes a task’s starting progress or requested endpoint without changing demonstrated skill order.Object displacement alone does not count as a new composition.Source: §3 · p. 4
• New compositions couple the two forms of generalization because a changed sequence or handoff can create policy start states that skill generalization must cover.The composer checks an interface applicability region rather than the policy’s generally unobservable success region; a useful interface should be faithful and informative.Source: §3 · p. 4

4 AGENT-GUIDED PRIOR DESIGN AND SKILL COMPOSITION

• APPL uses a skill’s structural prior both to shape policy generalization during training and to describe applicability for runtime selection and composition.The training realization determines coverage, while the interface exposes a construction-time hypothesis about competence for the runtime agent to use when selecting and chaining policies.Source: §4 · p. 4, §4.4 · p. 7

4.1 STRUCTURAL PRIORS IN THE INTERFACE

• A prior shapes policy behavior beyond demonstrations, while its description and demonstrated support define a hypothesized applicability region for runtime selection.An object-relative prior can cover a new absolute object location when the relative configuration remains within demonstrated range; faithfulness is not guaranteed, and overly conservative regions reduce informativeness.Source: §4.1 · p. 5

4.2 SEGMENTING DEMONSTRATIONS INTO SKILLS

• The construction agent groups reusable segments into skill datasets and overlaps adjacent segments so successors train on predecessor-reached handoff states without adding demonstrations.The overlap includes part of the predecessor’s end and successor’s beginning, broadening support around handoffs, including states before nominal completion.Source: §4.2 · p. 5

4.3 PROPOSING, IMPLEMENTING, AND DOCUMENTING PRIORS

• Because the best structural assumption may vary across invocation states, construction retains several prior-specific policies as alternatives for each skill.The design space is open-ended and the deployment objective is unavailable during construction, so an agent proposes and evaluates candidates from demonstrations and task knowledge.Source: §4.1 · p. 5, §4.3 · p. 6
• Each candidate prior receives its own policy implementation and limited verification, after which the library records prior, handoff, support, and verification information in its interface.In the experiments, verification uses demonstrated skill-entry states and checks the demonstrated exit condition after running each policy for 1.5 times its segment length.Source: §4.3 · p. 6

4.4 COMPOSING SKILLS AT RUNTIME

• At runtime, the agent consults frozen policy interfaces to choose a policy and arguments, executes it under stop conditions, then resumes composition from the updated state.It may continue the skill, switch to another policy for that skill, invoke another skill, or terminate; it cannot modify the frozen library or write control code.Source: §4.4 · pp. 6–7

5 EXPERIMENTS

• The setups separate six-task MetaWorld skill tests from five-task ManiSkill tests of motion shifts, task-level variants, and 16 new compositions.In MetaWorld, candidates are selected using development OOD success and frozen before testing.Source: §5.1 · p. 7, §5.2 · p. 8

5.1 AGENT-DESIGNED PRIORS FOR SKILL GENERALIZATION

• 89.58% six-task macro OOD success at N = 2 for q4, versus 28.96% for B0 and 37.92% for B1.Source: §5.1 · p. 8
• Three initial priors and development selection add 10.2–24.0 points, feedback adds 2.9–12.1 more, and q4−q3 does not isolate feedback.Source: §5.1 · p. 8

5.2 LONG-HORIZON COMPOSITION UNDER SHIFTED OBJECTS AND UNSEEN TASK VARIANTS

• 50.0% motion-OOD success for APPL versus 10.0% for both DP and SinglePrior; this system-level comparison combines runtime scheduling with a larger, diverse-prior skill library.Source: Table 2 · p. 9, §5.2 · p. 9
• APPL solves 8/16 new compositions versus 3/16 with prior-derived descriptions hidden; one new handoff returns a full container directly after grasping in two calls.Source: §5.2 · p. 9
• Removing interface information reduces motion-OOD, task-level, and composition success to 20.0%, 65.0%, and 2/16; a fixed rule yields 22.5%, 52.5%, and 1/16.Source: §5.2 · p. 9, Table 2 · p. 9
• Verification scores from demonstrated entry states can misrank policies under shift; the protocol uses no OOD verification and scores fixed-duration final states.Source: §5.2 · p. 9
• Failures commonly follow dropped objects or closed drawers, while demonstrations lack recovery trajectories and the library lacks re-approach skills for lost objects.Source: §5.2 · p. 9

6 CONCLUSION AND LIMITATIONS

• APPL makes each policy’s structural prior part of the runtime interface, so the prior both shapes learning and indicates when the policy is expected to apply.The runtime agent can use the shared prior to choose among alternative skill implementations and compose them for new tasks.Source: §6 · p. 9
• Across MetaWorld and ManiSkill, agent-designed priors improve out-of-distribution skill generalization, while construction-time interface information helps runtime agents select and compose skills under shifted objects and unseen task variants.The experiment reports 144 trained models across six MetaWorld tasks and four demonstration counts.Source: §6 · p. 10, A EXPERIMENT 1: EXPERIMENTAL DETAILS… · p. 18
• A structural prior describes intended generalization but does not guarantee competence; policies may fail beyond demonstrated support, and the library may lack recovery skills.The authors identify verification improvements and targeted data collection or augmentation as possible next steps.Source: §6 · p. 10
• Both experiments evaluate simulated manipulation only, and the studied priors and runtime checks do not by themselves make physical deployment safe.Source: ETHICS STATEMENT · p. 11

A.1 TASKS, DEMONSTRATIONS, AND EVALUATION STATES

• Six adapted MetaWorld tasks each have 20 successful demonstrations, and the tasks and demonstrations had been seen during earlier development.The tasks are pick-place-wall, assembly, drawer-open, door-open, peg-insert-side, and stick-push.Source: §A.1 · p. 18
• IID test states draw from demonstrated factor combinations, C recombines factors, and E places both outside demonstrated intervals; the 20 demonstrations provide design support, but test states are unavailable during design.Source: §A.1 · p. 18, Table 3 · p. 18, Figure 3 · p. 19

A.2 OBSERVATIONS, CONTROL, AND ENVIRONMENT CHANGES

• Policies receive hand and object state, goal and task-specific points, and two consecutive observations; design agents may also inspect up to 16 demonstration images.Policies use no images or API calls; actions use native MetaWorld commands, with 16 predicted actions and the first four executed before replanning.Source: §A.2 · p. 18
• Shared environment changes include rotating and translating the drawer and door cabinets, changing the handle goal, widening assembly’s nut reset range, and rebuilding selected geometry at sampled positions.The modifications were fixed before design sessions and applied to all systems.Source: §A.2 · p. 18

A.3 POLICY, BASELINES, AND TRAINING

• All systems use a conditional 1D U-Net diffusion policy; B0 predicts native actions from state history, while B1 adds three world-frame relation vectors.Agent-designed candidates may add encoders, heads, diffusion channels, or an auxiliary loss within three times B0’s parameter count.Source: §A.3 · p. 19

A.4 METRICS AND INTERVALS

• Each model is tested on 20 IID, 40 C, and 40 E states, with split success rates weighting cells equally.Source: §A.4 · p. 19
• Table 6 reports six-task macro OOD success with 95% paired-state bootstrap intervals, conditional on the trained models and tasks.Intervals do not measure variation across training seeds or independent agent designs.Source: Table 6 · p. 20, §A.4 · p. 20
• OOD is the mean of C and E, and macro results average the six tasks.Source: §A.4 · p. 20

A.5 REVISION CASES

• The selected cases were chosen after seeing results and are illustrative, not ablations; Figure 4 curves describe the design procedure across data sizes, not one architecture’s scaling.Source: §A.5 · p. 20, Figure 4 · p. 21
• 93.75% test OOD success for assembly/N = 5 follows a revision combining metric relations, a transport frame, and lift and gripper cues; the component changes prevent assigning the gain to one change.The comparison is 41.25% for q3 versus 93.75% for q4.Source: §A.5 · p. 20
• 100.00% test OOD success for drawer/N = 2 follows adding an invertible horizontal action residual based on hand–handle error in cabinet coordinates.Success rises from 75.00% to 100.00% (80 of 80 OOD episodes), with the gripper unchanged.Source: §A.5 · p. 20
• At door/N = 20, A4 ties perfect development performance but lowers test OOD from 100.00% to 96.25%; at stick-push/N = 2, A4 is not selected after a development tie and slightly higher latency.Source: §A.5 · p. 20

A.6 TASK HETEROGENEITY

• At N = 20, E success is 25.42% for B0, 36.25% for B1, and 88.33% for q4, while every system reaches 100.00% IID success.Since candidate designs change with N, these curves describe the design procedure across data sizes rather than the scaling of one architecture.Source: §A.6 · p. 21
• Across 24 conditions, q4 exceeds B0 in all 24 and B1 in 23, tying once; stick-push reaches 100% under q3 and q4 at every N.The q4 − q3 comparison combines an additional candidate with development selection and does not isolate feedback alone.Source: §A.6 · p. 21
• Assembly remains the hardest task and is not monotonic in N.Source: §A.6 · p. 21

B.2 METHODS, TRAINING, AND RUNTIME INTERFACE

• For each skill, the construction agent proposes three complementary priors and implements each as its own policy, including object- or mechanism-frame targets and TCP-relative corrections.Across ten skills, eight also received destination-frame priors, and seven priors added a phase input or auxiliary phase objective.Source: §B.2 · p. 22
• Verification tests policies only from demonstrated entry states, checks predefined exit criteria at a fixed final time, and informs an added policy and usage notes.Each policy runs for 1.5 times its segment length; transient passes do not count as final verification passes.Source: §B.2 · p. 22
• The runtime agent receives the goal, current state, policy descriptions, recent turns, and notebook, then calls named policies with typed arguments, duration limits, and numeric stop conditions.Switching policies resets the action queue, while repeating a policy continues it; the executor stops immediately when the task goal holds.Source: §B.2 · pp. 22–23
• The no-interface ablation hides prior, handoff, support, and verification information, while the no-prior ablation restores verification scores and exit criteria.The scripted-planner comparison follows demonstrated skill order and uses geometric subgoals to determine completion.Source: §B.2 · p. 23
• Agent+VLA replaces APPL’s policy library with one fine-tuned vision-language-action policy per task, without prior, handoff, support, or verification descriptions.It is trained on the 204 skill segments and receives an instruction, structured state, and camera image at each query.Source: §B.2 · p. 23

B.3 EVALUATION CASES AND PAIRED COMPARISONS

• APPL completes 18 of 20 re-entry cases and 19 of 20 prefix-goal cases, against 12–14 and 9–17 for the ablations.Source: §B.3 · p. 24
• On composition, APPL solves 2/4 drawer-exchange cases, 1/3 buffer-exchange cases, 3/3 retrieve-and-store cases, 0/3 covered-peg cases, and 2/3 pour-and-return cases.Source: §B.3 · p. 24

B.4 VERIFICATION AND POLICY SELECTION

• Restoring verification scores and exit criteria raises motion-OOD success from 20.0% to 45.0%, task-level success from 65.0% to 72.5%, and composition from 2/16 to 3/16.These paired comparisons hold the trained policy library fixed and vary runtime information.Source: Table 9 · p. 24, §B.4 · p. 25
• Adding prior, handoff, and support descriptions plus usage notes raises task-level success from 72.5% to 92.5% and composition from 3/16 to 8/16; the composition trend is nonsignificant.Source: Table 9 · p. 24, §B.4 · p. 25
• On motion OOD, 49% of calls with verification go to policies passing all twelve checks, compared with 26% without verification.The verification ablation changes both the number of policies and the available scores and usage guidance, so it does not isolate those factors.Source: §B.4 · p. 25

B.5 FAILURE MECHANISMS

• Failures include later actions undoing progress, missed grasps or drops requiring unseen recovery, and compositions that violate demonstrated entry conditions.Two drawer layouts also defeat the scripted planner, so not every failure can be attributed to the learned library.Source: §B.5 · p. 25

B.6 EVALUATION SCOPE AND STOPPING RULE

• The motion-OOD split halved displacement after results on a harder split were seen, while replacement task cases and 12 added compositions were fixed before evaluation and completed by the scripted planner.The added tasks were chosen before any APPL result on them.Source: §B.6 · p. 25
• 173 successful high-level-agent episodes and 22 successful fixed-rule episodes all stopped at first goal attainment, so the intended 300-step hold condition did not establish sustained stability.In composition case C-N3, success was credited while the block was still descending, at an average downward velocity of approximately 1.66 m/s over the final control step.Source: §B.6 · pp. 25–26

C AGENT PROMPTS

• The appendix reproduces verbatim fixed agent instructions; task-specific tool inputs are omitted, and each prompt was fixed before its sessions.The omitted inputs include demonstrations, task goals, check results, and the policy catalogue.Source: C AGENT PROMPTS · p. 27

C.1 EXP. 1 DESIGN AGENT

• Experiment 1 required three substantively different candidates before performance feedback, then allowed one feedback-based A4 revision and selection under a frozen development-success rule.The rule equally weights C and E development success, with ties resolved by latency, parameter count, then candidate ID; IID and hidden-test results do not select.Source: §C.1 · pp. 27–28

C.2 EXP. 2 CONSTRUCTION AGENT

• Experiment 2 segments demonstrations by physical responsibility and requires substantial action-supervised overlap between adjacent skills to support robust handoffs.Boundaries are selected separately for each trajectory, with entry and exit states inspected for each segment.Source: §C.2 · p. 28
• For each skill, the construction agent designs complementary priors targeting different generalization failures, including a displacement-invariant prior; later verification adds a single h04 policy.The prompts require each prior to specify its mechanism, preferred conditions, expected failure, and coverage by another prior; h04 is trained from scratch and may perform worse.Source: §C.2 · pp. 29–30

C.3 EXP. 2 RUNTIME AGENT

• The runtime agent binds intended operations to each policy’s published typed arguments and chooses policies using their contracts, prior descriptions, handoff guidance, and measured support.It should assess progress from observable scene and task evidence rather than treating a closed gripper, reached pose, or predicted phase alone as proof of contact or completion.Source: §C.3 · p. 30
• After a failed attempt, the runtime agent should explain observed changes and revise or continue its plan based on evidence, while treating unsupported recovery as a hypothesis.Final success is exactly the supplied whole-task predicate, with no additional release, clearance, speed, or hold requirement.Source: §C.3 · p. 31
• During evaluation, observation and API deliberation do not advance physics, and agents cannot change weights, schemas, or adapter code or use unpublished capabilities.The runtime agent should maintain a concise notebook of goals, physical state, bindings, attempts, unresolved facts, and next steps, consulting observation and history tools when needed.Source: §C.3 · p. 31

C.4 SINGLEPRIOR BASELINE

• SinglePrior trains one immutable full-task policy without segmentation, runtime policy selection, stage labels, or caller-selected objects; its prior targets manipulated or approached objects’ displacement.Each demonstration has one complete training binding with empty call arguments; the policy uses causal observed state and documents full-task entry, exit, and limitations.Source: §C.4 · p. 31

D EXTENDED RELATED WORK

• Related work is organized around how task structure shapes low-level policies, informs high-level decisions, and compares with APPL’s closest methods.The appendix directs readers to separate discussions of further work in each family and APPL’s closest methods.Source: D EXTENDED RELATED WORK · p. 32

D.1 TWO ROLES OF TASK STRUCTURE

• Most task-structure families address either low-level policy learning or high-level decision-making; shared symbolic abstractions address both through fixed rules or specifications.Table 10 contrasts these two roles of task structure.Source: §D.1 · p. 32, Table 10 · p. 32

D.2 ADDITIONAL RELATED WORK

• Related work finds that useful structural priors depend on task and conditions, while symbolic interfaces, handoff methods, and language specifications address skill planning and composition.Examples include operation-specific priors and condition-dependent perception benefits (Hsu et al., 2022; Ling et al., 2023), competence-aware planning, handoff-aware memory (Xu et al., 2026), and language specifications that describe effects rather than inductive bias (Klissarov et al., 2025; Zabounidis et al., 2026).Source: §D.2 · pp. 32–33
• Prior agentic systems design learning components or compose skills described by names, code, or effects; Agentic Skill Discovery retains policy variants but leaves selection to future work.Examples include systems that shape rewards, tasks, architectures, policy structures, or reinforcement-learning pipelines (Xie et al., 2024; Wang et al., 2024d;c; Ma et al., 2024b; Wang et al., 2024a; Yu et al., 2025; Zhu et al., 2025; Wei et al., 2025), and skill reuse (Wang et al., 2024b; Klissarov et al., 2025; Zabounidis et al., 2026; Nie et al., 2026).Source: §D.2 · p. 33

D.3 COMPARISON WITH THE CLOSEST METHODS

• Table 11 compares method designs across settings from real-robot manipulation to simulated tasks, and does not imply an empirical ranking.Its caption identifies what high-level decision makers read about policies before or during invocation.Source: §D.3 · p. 33, Table 11 · p. 34
• APPL’s distinction from LGA, KALM, and ATK is proposing priors from several families per skill, retaining alternatives, and reusing priors as runtime documentation.Those prior methods automate one family and produce one design per task, which the policy consumes as input.Source: §D.3 · p. 33
• Unlike SymSkill’s single design family and policy per operator, APPL proposes varied priors per skill, retains policy alternatives, and lets runtime choose using natural-language interfaces.Both learn reusable skills from complete demonstrations and use learned structure to organize long-horizon composition.Source: §D.3 · p. 33
• Lorang et al. and DR-LfD use fixed rules or templates and one learning recipe per skill; MaestroMotif and SCALAR share language about outcomes and call conditions.Those language specifications do not describe a policy’s inductive bias or training support.Source: §D.3 · p. 33
• Harness VLA, BATON, and RoboHarness invoke policies with instructions, step budgets, and stop conditions, but learn operating information after policies exist.APPL derives documentation from construction-time priors and training-data overlap; RoboClaw selects data for a fixed training recipe.Source: §D.3 · p. 33
• Agentic Skill Discovery retains policy variants but leaves selection to future work, whereas APPL’s runtime agent chooses among them using their priors.APPL’s overlapping training segments follow prior work widening successor-skill initiation sets; the paper treats overlap as a design choice, not a contribution.Source: §D.3 · pp. 34–35