PaperlayerView exact revision on arXiv

Source-linked AI summary

RLHarness: Co-evolving Procedural Skills with Reinforcement Learning for Long-horizon Multimodal Reasoning

Ziqiao Shang, Zian Xu, Ji-Chen Yan, Weiming Wu, Ziyi Jia, Jie Meng, Tao Huang, Shan Huang, Lan-Zhe Guo

arXiv:2609.32326v1cs.AI

TL;DR

Long-horizon multimodal reasoning requires preserving visual evidence and selecting procedures across dependent decisions, while terminal verification does not specify how to produce a correct answer. RLHARNESS alternates evolution of a versioned Harness with policy learning, reconstructing guidance from current-policy rollouts before further RL. Under the same subsequent RL budget, reconstruction plus adaptation outperforms continuing with the old Harness on all four covered tasks; replacing the Harness alone does not always help.

  • Problem

    Terminal verifiers can score final outputs without specifying how to organize evidence, order steps, or repair reusable rules, leaving the policy to discover a program while learning to execute it.

    Source: §1 · p. 1

  • Method

    RLHARNESS jointly versions Skills, invocation protocols, and demonstrations, builds an initial Harness, then reconstructs those components from current-policy successes and failures before another RL block.

    Source: §1 · p. 2

  • Results

    Under the same subsequent RL budget, the reconstructed Harness outperforms continuing with the old Harness on all four tasks, while replacement without policy adaptation does not always help.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across the covered MetroMap-lite, TravelMap-lite, Fee-VL, and Cancel-VL settings, the results support combining program reconstruction with subsequent policy adaptation.

    Source: §1 · p. 2

    The evaluation covers static multimodal settings with fixed inputs, output contracts, and verifiers; test data are excluded from Harness construction, training, and candidate selection.

    Source: §3.1 · p. 4, §F.2 · p. 41

Abstract

from arXiv · show

Multimodal reasoning requires models to preserve visual evidence through long decision chains while selecting appropriate procedures across diverse scenarios and rules. When learning is guided only by terminal verifiers, reinforcement learning (RL) reveals whether a final answer is correct but not how it should be produced. The policy must therefore discover reusable reasoning procedures while learning to execute them, creating a program cold-start problem. Skills can externalize successful procedures, reduce repeated exploration, and provide inspectable guidance. However, a fixed Skill Bank assumes that this guidance remains compatible with an evolving policy, while updating Skills alone can leave their triggers, execution protocols, and demonstrations stale or mutually inconsistent. We introduce RLHARNESS, which organizes Skills, selection and execution protocols, few-shot demonstrations, and task contracts into a unified, versioned Harness and alternates Harness evolution with policy learning. An Exploration-Distillation Harness builds the initial Harness and version-aligned verified traces for SFT and DAPO I. After the first RL block, a Post-RL Reconstruction Harness rebuilds Skills, protocols, and demonstrations from fresh success-failure rollouts, and DAPO II adapts the policy to the reconstructed program. RLHARNESS improves Accuracy from 16.25%/27.50% to 62.00%/50.00% on MetroMap/TravelMap and raises F1 score from 37.13%/45.50% to 65.81%/65.51% on Fee-VL/Cancel-VL. All four tasks achieve their best results only after reconstruction and DAPO II, showing that an evolving Harness complements RL by continually updating the external program that the policy learns to execute.

1 INTRODUCTION

Long-horizon multimodal tasks require preserving evidence across dependent decisions, but terminal verification does not specify reusable procedures. RLHARNESS jointly evolves a versioned Harness and policy, rebuilding guidance from current-policy rollouts before further RL; experiments support reconstruction plus adaptation across four covered tasks.

1 INTRODUCTION

  • Visual and structured evidence must persist across dependent decisions: one omitted station can invalidate later route and cost calculations, while one misplaced rule can alter related labels.These tasks test long-chain evidence consistency and procedure selection across cases.Source: §1 · p. 1
  • Terminal verifiers score final outputs but need not specify evidence organization, step order, or reusable rules, leaving the policy to discover and execute a program.External Skills can make prior successes explicit and reusable instead of requiring each rollout to rediscover a procedure.Source: §1 · p. 1
  • A Skill-only update can leave its execution order and demonstrations inconsistent, so the update unit is the versioned Harness comprising B/P/E.Skills specify what to do, protocols when and in what order, and examples how rules apply; input assembly, output contract, and verifier remain fixed.Source: §1 · p. 2
  • Reconstruction followed by policy adaptation outperforms continued RL with the old Harness on all four tasks under the same subsequent RL budget.Replacing the Harness without policy adaptation does not always help; the supported conclusion is limited to MetroMap-lite, TravelMap-lite, Fee-VL, and Cancel-VL.Source: §1 · p. 2
  • RLHARNESS jointly reconstructs Skills, invocation protocols, and demonstrations from current-policy trajectories, then trains the policy to execute the revised program.The four stages are initial Harness construction, policy learning, Harness reconstruction from new successes and failures, and further policy learning.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work studies multimodal reasoning under verifiable feedback, external Skills, and broader Harness optimization. RLHARNESS distinguishes itself by reconstructing the combined external program from current-policy traces and adapting the student under fixed task interfaces.

2.1 MULTIMODAL REASONING UNDER VERIFIABLE FEEDBACK

  • Multimodal reasoning work improves visual grounding, trajectory supervision, and feedback granularity; RLHARNESS instead keeps the verifier fixed and revises the external program from success–failure traces.The cited strands include image-grounded reasoning, executable trajectories, evidence-quality distinctions, long-horizon training, and step-level feedback.Source: §2.1 · p. 3

2.2 EXTERNAL SKILLS AND SKILL–POLICY LEARNING

  • Skill–RL methods update Skills or libraries, train a Skill generator, or learn Skill operations in one policy; RLHARNESS jointly versions B/P/E and adapts the student in a separate RL block.Its static-task setting fixes input assembly, output contracts, and verifiers.Source: §2.2 · p. 3

2.3 HARNESS OPTIMIZATION AND MODEL ADAPTATION

  • Harness research broadens editable control beyond Skills, while RLHARNESS reconstructs B/P/E from changed-policy behavior and trains a small model to internalize the revision under fixed inputs, outputs, and verifiers.The cited work includes Harness optimization, policy or weight updates, and safety or compatibility during co-adaptation.Source: §2.3 · p. 3

3 METHOD: RLHARNESS

RLHARNESS alternates discrete Harness commits with policy updates while fixing task interfaces and verifiers. It builds an initial program, learns from version-aligned traces, reconstructs guidance from current behavior, and adapts the policy again.

3.1 PROBLEM FORMULATION

  • Test data are excluded from Harness construction, parameter training, and candidate selection; optimization alternates discrete Harness commits with parameter updates rather than jointly differentiating policy and Harness.Map planning uses complete ordered paths with whole-route verification, while VL adjudication predicts fact labels and, for Cancel-VL, an auxiliary responsibility class.Source: §3.1 · p. 4

3.2 OVERVIEW: BUILD–LEARN–RECONSTRUCT–LEARN AGAIN

  • The four-stage lifecycle builds an initial program, trains the policy to execute it, rebuilds the program around mastered capabilities and remaining errors, then adapts the policy to the revision.Harness I emphasizes task coverage; Harness II targets mismatches remaining or emerging after the first learning block.Source: §3.2 · p. 4

3.3 HARNESS I: BUILDING A LEARNABLE PROGRAM FROM RAW EXPERIENCE

  • Harness I creates and verifies an initial B/P/E package, explores a fixed task-specific subset in batches, and commits a candidate only when its hard accuracy strictly exceeds the incumbent on frozen Val100.After Φ1 is frozen, audited accepted trajectories from complete-training-pool attempts are retained as G1 for SFT; the task interface remains immutable.Source: §3.3 · pp. 5–6

3.4 HARNESS II: RECONSTRUCTING THE LEARNING ENVIRONMENT FROM CURRENT BEHAVIOR

  • Harness II contrasts stratified successes and failures from current rollouts, prunes ineffective Skills, revises triggers and execution order, and rewrites version-matched demonstrations while preserving procedures that work.Input assembly, output contract, source examples, and verifier remain fixed; full parameter adaptation is assigned to DAPO II.Source: §3.4 · p. 6

3.5 POLICY LEARNING: CONVERTING BOTH HARNESSES INTO MODEL CAPABILITY

  • The first learning block distills audited G1 into targets for Skill selection, evidence binding, computation, and answer checking, then runs DAPO I with Φ1 fixed; the second resumes from θ1 under fixed Φ2.Both DAPO blocks share a task reward and Hybrid-DGPO objective; Fee-VL and Cancel-VL additionally apply per-label frequency weights within partial reward.Source: §3.5 · pp. 6–7

4 EXPERIMENTS

The experiments are organized around whether the full build–learn–reconstruct–learn path improves performance, how it compares with other methods, what transfers through learned weights, and how task-specific the reconstructed prompt is. The evaluation spans four long-horizon cross-modal reasoning benchmarks.

4.1 EXPERIMENTAL SETUP

  • The experiments ask whether the complete pipeline improves performance and which components are necessary, how the final system compares, what capabilities transfer through weights, and how task-specific the reconstructed prompt is.Full transfer matrices are provided in Appendices C.5 and C.4.Source: §4.1 · p. 7
  • Four benchmarks cover metro-diagram and travel-map planning plus fare-dispute and cancellation-responsibility judgments from ride-hailing operations.MetroMap-lite and TravelMap-lite are sampled map-planning subsets; Fee-VL and Cancel-VL are multimodal business datasets formed from Didi Chuxing operations.Source: §4.1 · p. 7

Appendix C.1 gives the complete data distributions, label composition, and difficulty stratification.

RLHarness alternates updates to a versioned Harness and policy, evaluating the lifecycle, component ablations, comparisons with SkillRL, and cross-task transfer. Results show that reconstruction followed by policy adaptation matters, while map-task conclusions are limited to retained difficulty strata under stated context and response limits.

4.2 RQ1: DOES THE BUILD–LEARN–RECONSTRUCT–LEARN PATH WORK?

  • Exploration improves on Base by an average of 6.95%, and selective commitment adds 1.30%; version-aligned trajectories improve the average score by 5.51%.Selective commitment retains the incumbent when an edit lacks support, while aligned SFT targets teach the deployed procedure rather than only correct answers.Source: §4.2 · p. 8
  • DAPO I improves map tasks by 13.75% on average but VL tasks by only 1.50%; question-difficulty weighting adds 2.98% over the matched control.Partial- and exact-route rewards help map chains, while difficulty weighting directs training toward unresolved questions but cannot fix a bad Skill boundary in Φ1.Source: §4.2 · p. 8
  • Swapping Φ1 for Φ2 without updating θ1 lowers the average score by 0.68%; stratified evidence is 0.94% better than random evidence, and synchronized demonstrations are 2.44% better than removing them.The immediate swap exposes version mismatch; the rebuilt examples indicate when new rules apply and in what order to execute them.Source: §4.2 · pp. 8–9
  • Relative to the immediate Harness swap, DAPO II raises map and VL averages by 7.00% and 11.40%; the full method exceeds reuse of Φ1 by 2.75% on maps and 9.95% on VL.It is also 1.71% above the random-evidence branch, including 14.75% on Fee-VL; the budget control adds 200 steps while keeping Φ1.Source: §4.2 · p. 9

4.3 RQ2: COMPARISON WITH REPRESENTATIVE METHODS AND MODELS

  • 37.41/13.33 Accuracy points on Metro-Lite/Travel-Lite and 23.00/13.59 Macro-F1 points on Fee-VL/Cancel-VL: RLHarness exceeds SkillRL on all four tasks.The comparison uses SkillRL's original implementation and configuration, changing only datasets and task-interface adapters.Source: §4.3 · p. 9, Table 2 · p. 9

4.4 RQ3 AND RQ4: SEPARATE EVALUATION OF MODEL-WEIGHT AND PROMPT TRANSFER

  • Checkpoint-600 weights improve every tested off-diagonal target over the unadapted baseline, with modest gains strongest within task families; task-matched Post-RL Prompts win all four within-family replacements.The reconstructed Prompt remains strongly task-specific, with smaller cross-task losses between map tasks than between Fee-VL and Cancel-VL.Source: §4.4 · p. 9

5 CONCLUSION

  • Every task peaks only after reconstruction and DAPO II; under the same budget, Φ2 beats continued Φ1, and the 24 primary component comparisons are significant after Holm correction.The lifecycle results support versioned Harness–policy adaptation rather than a one-time Prompt rewrite or longer training.Source: §4.2 · pp. 7–8, §5 · p. 10

C.1.2 VISION–LANGUAGE DATASET STATISTICS

Fee-VL and Cancel-VL evaluate multimodal judgment over ride-hailing orders, pairing images with structured fields and separate fact-label decisions. Their dataset statistics distinguish orders from label decisions and separate training, validation, and test roles.

C.1.2 VISION–LANGUAGE DATASET STATISTICS

  • The VL tasks pair one image with structured order fields; they are not ordinary image-classification tasks.Each order may activate multiple fact labels, so the evaluated decision count can exceed the number of orders.Source: §C.1.2 · pp. 16–17
  • The fixed map splits cross map and query difficulty strata, and the lite construction excludes extreme cases with long contexts or reasoning chains.The exclusion is based on length, not answer correctness or a preference for easy cases.Source: Table 3 · p. 17
  • Training, Harness validation, and held-out test data have distinct roles; validation is frozen before optimization and is not used for parameter training.VL training prompts are fixed subsets of larger pools, while SFT uses a separately audited trace corpus.Source: Table 4 · p. 17, §C.1.2 · p. 17
  • Fee-VL covers three fare-dispute scenarios and nine independent fact labels, while Cancel-VL evaluates three responsibility outcomes plus seven supporting facts.Fee-VL’s scenarios are non-boarding, delayed meter termination, and detours; Cancel-VL’s responsibility classes apply to resolved orders.Source: §C.1.2 · p. 17
  • Cancel-VL’s 55 unclear test orders retain fact annotations but are excluded from three-way responsibility accuracy.The paper reports fact-label Macro F1 as its primary Cancel-VL result, not a responsibility-class score.Source: §C.1.2 · pp. 17–18

C.2 FEE-VL LABEL-LEVEL RESULTS

Fee-VL’s label-level results show broad gains after post-RL reconstruction and DAPO II, alongside a non-monotonic path in which immediate Harness replacement can reduce scores.

C.2 FEE-VL LABEL-LEVEL RESULTS

  • Each Fee-VL table cell is one fact label’s F1, and the final column is their unweighted nine-label mean.Scenario names organize the labels but are not separately averaged into the overall score.Source: §C.2 · p. 18, Table 9 · p. 19
  • DAPO II achieves the highest F1 for all nine Fee-VL facts, and final scores exceed Base by 14.86–47.61 points per label.The largest end-to-end gains include navigation mismatch (+47.61) and meter-end delay at most one minute (+40.82).Source: §C.2 · pp. 18–19, Table 9 · p. 19
  • Immediate PR replacement lowers Macro F1 by 0.92 points and navigation-mismatch F1 by 6.24 points versus DAPO I; DAPO II then improves every label over that replacement.The authors describe this pattern as consistent with a version-switch cost followed by policy adaptation; aggregate tests do not establish significance for each fact label.Source: §C.2 · p. 19

C.3 CANCEL-VL LABEL-LEVEL RESULTS

Cancel-VL’s label-level results favor the final model across fact labels, while cross-task Prompt replacement performs worse than task-matched Prompts in all four tested directions.

C.3 CANCEL-VL LABEL-LEVEL RESULTS

  • The final Cancel-VL model improves every fact label over Base by 13.50–38.55 points and gains 20.01 points in Macro F1.DAPO II adds 5.30 points over immediate PR replacement and produces the best value in every column.Source: Table 10 · p. 19, §C.3 · p. 20

C.4 CROSS-TASK POST-RL PROMPT REPLACEMENT

  • Task-matched Prompts outperform cross-task replacements in all four directions, with losses of 3.75% and 7.75% for maps and 21.95% and 11.79% for VL.The comparison fixes the target checkpoint-400 policy and training setup, replacing only the Prompt source before training to checkpoint 600.Source: Table 11 · p. 20, §C.4 · p. 20

C.3 CANCEL-VL LABEL-LEVEL RESULTS

  • Passenger noncompliance (49.50) and platform cause (36.00) remain the weakest final labels; sparse positive cases warrant caution in label-wise interpretation.The test set has nine positive driver–passenger-disruption cases and 26 positive platform-cause cases; aggregate paired tests do not establish significance for individual labels.Source: Table 10 · p. 19, §C.3 · p. 20

C.4 CROSS-TASK POST-RL PROMPT REPLACEMENT

  • Cross-task replacement losses are smaller between map tasks, which share part of the route-planning procedure, than between VL tasks with task-specific labels and business boundaries.The authors characterize the post-RL Harness as adapted jointly to the current task and policy, rather than as generic Skill text.Source: §C.4 · p. 20

C.5 CROSS-TASK EVALUATION OF TRAINED MODEL WEIGHTS

Cross-task weight evaluation finds some transferable capability but stronger within-task gains, with transfer biased by task family. The accompanying configuration passages describe shared SFT and DAPO settings and a question-difficulty-weighting ablation.

C.5 CROSS-TASK EVALUATION OF TRAINED MODEL WEIGHTS

  • Every off-diagonal weight-transfer gain is positive (2.50–9.88%), but remains below diagonal gains of 37.50% and 26.98%.MetroMap-lite-trained weights transfer more strongly within the map family, while Fee-VL weights transfer more strongly within VL; task-specific Prompts, Skills, and label semantics still constrain generalization.Source: Table 12 · p. 20, §C.5 · pp. 20–21

C.6 SFT CONFIGURATION

  • Reasoning Distillation and its control share the teacher, examples, output contract, token budget, optimizer, and DAPO schedule; only the target construction differs.The control uses the teacher’s direct free-form solution trajectory and answer instead of a Skill-guided trace.Source: §C.6 · p. 21, Table 13 · p. 21

C.7 DAPO CONFIGURATION

  • The question-difficulty ablation removes only question-level weighting while holding the listed reward, data, rollout, optimizer, and decoding settings fixed.The specified ablation uses the unnormalized within-question GRPO advantage; VL label weights remain inside the task reward and do not replace question weights.Source: §C.7 · pp. 21–22

C.8 EVALUATION PROTOCOL AND STAGE MAPPING

The section specifies task-specific evaluation metrics and matched controls for interpreting the Harness and policy ablations. The controls vary one component at a time while holding the other listed conditions fixed.

C.8 EVALUATION PROTOCOL AND STAGE MAPPING

  • The task-specific Hybrid-DGPO definition distinguishes VL label-frequency weighting in the reward from question-difficulty weighting in the advantage.The compact two-Harness labels map to the fixed checkpoint schedule.Source: Table 15 · p. 23, Table 16 · p. 23
  • MetroMap-lite and TravelMap-lite use exact-route accuracy, while Fee-VL and Cancel-VL use nine-label and seven-label overall macro F1, respectively.Metrics are compared only within a dataset; public-map comparisons use common inputs, output contracts, inference budgets, and scorers.Source: §C.8 · pp. 22–23

C.9 MATCHED ABLATION SETTINGS

  • Ablations use matched controls that share splits, examples, models, checkpoints, budgets, decoding, evaluation sets, and scorers, changing only the component under test.The Φ1-reuse branch separately matches the checkpoint-400 starting state and 200-step DAPO-II budget while retaining Φ1.Source: §C.9 · p. 24
  • The selective-context-commit control disables rejection of legal effective candidates while keeping the examples, order, models, verifier, calls, and edit operators fixed.It isolates the commit decision rather than changing visible evidence or search budget.Source: §C.9 · p. 24
  • The skill-guided-trace control keeps the teacher, examples, output contract, token budget, SFT optimizer, and DAPO schedule fixed but removes alignment to the current Skills and protocol.This contrast targets supervision alignment with the deployed Harness execution language.Source: §C.9 · p. 24
  • The question-difficulty control sets wq = 1, retaining the other listed training settings to isolate dynamic question-level difficulty weighting.The stratified-evidence control changes sampling and packing while holding evidence volume and reconstruction components fixed; the few-shot control removes only the demonstration block while retaining Φ2 and the other listed settings.Source: §C.9 · p. 24

C.10 PAIRED STATISTICAL TESTS

The appendix reports paired tests of component and training-stage effects alongside a staged workflow that alternates Harness construction or reconstruction with policy learning, aligning supervision to each Harness version and rebuilding the second from fresh post-DAPO-I rollouts. It also specifies that held-out test examples measure the frozen θ2–Φ2 pair and are excluded from reconstruction, candidate validation, and optimization decisions.

C.10 PAIRED STATISTICAL TESTS

  • All 24 primary component comparisons were significant under the stated criteria, using 10 matched-run paired t-tests with Holm-adjusted p-values and confidence intervals excluding zero.The tests are two-sided, and all confidence intervals had positive lower bounds; pHolm ranged from approximately 0.00007122 to 0.010642.Source: §C.10 · pp. 24–25, Table 17 · p. 25
  • All four cross-task Prompt differences and all eight DAPO-stage gains were significant after separate within-family correction; task-matched Prompt advantages were larger on VL than map tasks.The task-matched advantage was 3.75–7.75 points for map tasks and 11.79–21.95 points for VL tasks; stage tests were exploratory and reported separately from primary component tests.Source: Table 18 · p. 25, §C.10 · pp. 25–26, Table 19 · p. 26

D.1 END-TO-END TRAINING ALGORITHM

  • Training builds Harness Φ1 and aligned traces G1, learns θ1 with SFT and DAPO I, reconstructs Φ2 from fresh θ1 rollouts, then resumes DAPO to produce θ2.The deployed and evaluated system is the frozen θ2–Φ2 pair; test data does not enter the training algorithm.Source: §D.1 · pp. 26–27

D.2 STATE REPRESENTATION AND COMMIT PROTOCOL

  • RLHarness separates temporary trajectory context, persistent execution context Φ, and parameter state θ; Harness updates distill evidence into B/P/E while leaving I, C, and θ unchanged.Only B/P/E receive new committed versions, while control logic validates, repairs or rolls back revisions, and commits complete versions.Source: §D.2 · p. 27

D.3 EXPLORATION–DISTILLATION HARNESS

  • Harness I creates a complete executable program and version-aligned teacher traces, using task-specific initialization rather than imposing the map Skill structure on VL tasks.Maps receive five image, five text/table, and five cross-modal Skills; VL preserves its rule-rich business logic and original output contract.Source: §D.3 · p. 28

D.4 FIRST-STAGE POLICY LEARNING

  • First-stage policy learning distills verified traces and runs DAPO with Φ1 fixed, then collects fresh θ1 rollouts to provide current-policy evidence for Harness II.The checkpoint at step 400 preserves the policy and optimizer state from which DAPO II later resumes.Source: §D.4 · pp. 29–30

D.5 POST-RL HARNESS RECONSTRUCTION

  • Harness II reconstructs Skills, protocols, and demonstrations from fresh post-DAPO-I rollouts, diagnosing success-failure contrasts and validating complete candidates before committing Φ2.It may revise B/P/E but keeps inputs, output contract, source examples, and verifier fixed; it returns a Prompt without updating model weights.Source: §D.5 · pp. 30–31

D.6 SECOND-STAGE POLICY LEARNING

  • DAPO II resumes from θ1 with Φ2 frozen and the same reward, advantage, rollout configuration, and scorer as DAPO I, updating only model parameters to yield θ2.The continued policy optimization runs for steps 400:600.Source: §D.6 · pp. 31–32
  • Held-out test examples measure the frozen θ2–Φ2 pair and are excluded from reconstruction, candidate validation, and optimization decisions.Source: §D.6 · p. 32

D.7 EVIDENCE BUDGETS AND MATCHED CONTROLS

This section describes evidence selection, budget choices, candidate validation, and the stage boundaries of the two Harness algorithms. The map-task budget is selected as a tradeoff among candidate performance, consistency, and analysis cost, not as a theoretical optimum.

D.7 EVIDENCE BUDGETS AND MATCHED CONTROLS

  • For map reconstruction, 128 checkpoint-400 traces comprise 64 successes and 64 failures, stratified by map and query difficulty into eight mixed batches of 16.This arrangement lets the teacher contrast behavior to preserve with failure patterns to repair.Source: §D.7 · p. 32
  • 50.50% is the best single score for the 256-case conservative candidate, while 128 cases yield the highest mean across candidate styles, smallest candidate gap, and half as many teacher-analysis batches.The authors use 128 as an empirical default, not a theoretically optimal constant; the small Budget-Val400 differences are not treated as a significance result.Source: Table 23 · p. 32, §D.7 · p. 32
  • Fee-VL and Cancel-VL build independent evidence pools per fact label, with 64 trajectories per label split between correct and incorrect predictions and balanced gold labels when possible.A rollout may occur in several label pools, so the reported 9 × 64 and 7 × 64 are evidence slots, not unique trajectories.Source: §D.7 · pp. 32–33
  • Each candidate’s How-to block and fixed demonstrations are rewritten, checked, executed on validation cases, and selected from normalized reports without student execution, test data, or human ranking.The minus-Few-shot control removes only the demonstration block while retaining the selected Post-RL Skills, instructions, checkpoint-400 policy, and evaluation settings.Source: §D.7 · p. 33
  • Matched context controls separately alter candidate commitment or evidence composition while holding their specified evidence and compute budgets fixed.The stratified-evidence branch is reported both immediately after reconstruction and after a budget-matched DAPO-II continuation.Source: §D.7 · p. 33

D.8 STAGE-SPECIFIC ALGORITHMS

  • Harness I builds an initial Prompt through task-specific initialization, exploration, verification, candidate acceptance, and audited teacher traces; Harness II prunes by observed usage, diagnoses stratified rollouts, validates candidates, and selects a reconstructed Prompt.The two Harnesses exchange versioned artifacts while label materialization, SFT, and both DAPO calls remain outside their implementation boundaries.Source: §D.8 · p. 34

E PROMPT SPECIFICATION AND CASE STUDIES

The appendix presents a versioned MetroMap-lite prompt and two case studies tracing how Harness and policy updates address route-planning and causal-attribution errors. The discussion also identifies limits involving visual perception, sparse rewards, version coupling, and study scale.

E.1 METROMAP-LITE PROMPT SPECIFICATION

  • The MetroMap-lite prompt grounds adjacency in the image, requires station-by-station route construction, and separates route selection, scoring, demonstrations, and output constraints.The appendix describes the prompt as a versioned program, not a continuous stack of Skill text.Source: §E.1 · p. 35
  • Harness updates must keep task rules, Skill invocation boundaries, protocols, and demonstrations aligned within the same version.The prompt treats task facts and scoring rules as immutable, and requires demonstrations to be rewritten and checked under the revised Skill and protocol version.Source: §E.1 · pp. 35–36

E.2.1 METROMAP-LITE ROUTE-PLANNING CASE STUDY

  • The checkpoint-400 route skipped Heng Fa Chuen, making its first edge illegal before scoring; the reconstructed prompt adds a No-Skip Edge Guard before candidate scoring.The case uses weights 0.409331/0.153055/0.130721/0.306893 for time, price, comfort, and reliability.Source: §E.2.1 · pp. 36–37
  • After DAPO II, the replay inserts Heng Fa Chuen and checks proposed edges for omitted stations before scoring, illustrating the reusable behavior targeted by reconstruction.The verifier accepts the replay, but the appendix explicitly says this is not an additional quantitative result.Source: §E.2.1 · pp. 36–37

E.2.2 FEE-VL CAUSAL-ATTRIBUTION CASE STUDY

  • In the Fee-VL case, the original logic treated trajectory–announcement agreement as evidence of navigation anomaly, producing a false positive despite a competing passenger-direction explanation.The trajectory image can confirm whether spoken directions became observed actions, but does not directly reveal the responsibility label.Source: §E.2.2 · pp. 38–39, Figure 4 · p. 38
  • The revised Fee-VL analysis treats agreement as evidence of execution only and tests causal direction and competing explanations; it returns no hit, matching the gold label.Post-yaw replanning cannot cause an earlier yaw, and the evidence does not establish platform-navigation responsibility in this case.Source: §E.2.2 · pp. 38–39

F.1 DISCUSSION

  • Across the covered tasks, the remaining performance gap reflects perception, long-horizon credit assignment, task semantics, and optimization budget, not just a missing prompt rule.Map inputs require reliable station-and-edge extraction, while business-VL inputs require aligning trajectory, markers, ASR, and OCR on a timeline.Source: §F.1 · p. 40
  • The study uses a 9B student, two policy-learning blocks, and two Harness versions, and does not provide long-term per-example trajectories across further update cycles.Its evidence budgets approximate the current error distribution, and teacher-generated candidates need not contain the globally optimal external program.Source: §F.1 · p. 40

F.2 LIMITATIONS

The limitations qualify the study’s comparison and scope, while the future directions propose richer perception, verification, failure routing, and longer auditable lifecycles.

F.2 LIMITATIONS

  • The statistical tests cover stated aggregate comparisons, not every label-wise difference or candidate-generation path; the 62.00 MetroMap-lite score comes from an earlier reconstructed-Harness branch.The paper says this score supports external revision followed by re-adaptation but does not establish end-to-end reproduction of the complete current candidate-generation procedure.Source: §F.2 · p. 40
  • The external comparison establishes gains over implemented SkillRL, not superiority over unimplemented Harness systems; transfer covers four tasks, and map claims exclude extreme-length examples removed by lite construction.Candidate validation uses Qwen3.5-plus as a task-matched proxy rather than direct execution by checkpoint 400.Source: §F.2 · p. 41
  • The study’s multimodal records expose aggregates, necessary numerical evidence, and de-identified trajectory images, so teacher analysis is operational attribution rather than identification of hidden neural states.A rule update cannot recover evidence that was missing or misread by ASR, OCR, or the static image.Source: §F.2 · p. 41

F.3 FUTURE DIRECTIONS

  • Future work proposes separating visual evidence extraction from rule adjudication, adding stage-level rewards and executable checks, and routing failures to the appropriate repair mechanism.The proposed update interface reserves Harness editing for repeated observable boundary failures and calls for longer lifecycles to measure forgetting, version compatibility, rollback cost, and safety constraints.Source: §F.3 · pp. 41–42

Related papers

Loading 2609.32326v1…