PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents

Bingqing Jiang, Guoxi Zhang, Jasper Wang, Auric Wang, Bingning Wang, Tianyi Lin, Zichao Yu, Yujin Han, Ziye Ma, Difan Zou

arXiv:2609.34557v1cs.AI

TL;DR

Multimodal agents need reusable procedural knowledge for long-horizon tool use, but skill-based policy optimization receives little direct supervision for intermediate decisions and rubrics are difficult to scale and align with execution. SkillRubric pairs actor guidance with evaluator rubrics derived from the same skills, using multimodal evidence for turn-level rewards and alternating validation to evolve both. It consistently outperforms prior skill-based methods across eight benchmarks and two backbones, with paired rollouts showing improved reusable guidance.

  • Problem

    Skill-based multimodal-agent optimization relies mainly on sparse outcome rewards, while scalable rubrics aligned with the actor’s execution procedure remain difficult to construct.

    Source: §1 · p. 2

  • Method

    SkillRubric derives actor-facing guidance and evaluator rubrics from shared skills, uses screenshots and tool outputs to assign turn-level completion and progress rewards, and alternates guidance and rubric validation.

    Source: §1 · pp. 2–3

  • Results

    SkillRubric outperforms the strongest baseline, SkillRL, by 4.76 percentage points on Qwen3-VL-8B and 3.55 percentage points on Qwen3.5-9B on average, and performs best among trained methods across eight benchmarks and both backbones.

    Source: §4.2 · p. 8, §5 · p. 11

  • Takeaways & Limitations

    Controlled paired rollouts show that evolved skills provide more effective reusable guidance for task planning and tool use than their preceding versions.

    Source: §1 · p. 3

Abstract

from arXiv · show

Recent work incorporates reusable skills distilled from past interactions into multimodal agent training, providing procedural guidance for long-horizon planning and tool use. However, policy optimization in these methods remains driven primarily by sparse outcome rewards, providing little supervision for intermediate decisions. Rubric-based rewards address this limitation through explicit intermediate criteria, but reliable rubrics are difficult to construct at scale and often disconnected from the procedure followed by the actor. We observe that a well-structured skill naturally specifies both how to act and what successful execution should achieve. Based on this insight, we introduce SkillRubric, which represents each skill through aligned actor-facing guidance and an evaluator-facing rubric. A multimodal verifier evaluates skill-defined goals using screenshots and tool outputs, assigning completion and progress rewards to the responsible turns. We further introduce an alternating co-evolution scheme that validates guidance revisions through paired rollouts under a frozen policy and rubric revisions offline under fixed guidance. Experiments across diverse multimodal agent benchmarks demonstrate consistent performance gains, while controlled paired rollouts further show that evolved skills provide more effective guidance for planning and tool use than their preceding versions.

1. Introduction

Multimodal agents need reusable procedural knowledge for long-horizon tool use, but skill-guided optimization still relies mainly on sparse outcome rewards, while scalable rubrics remain hard to align with execution. SkillRubric derives actor guidance and evaluator rubrics from a shared skill structure and co-evolves both; experiments report consistent gains across benchmarks.

Problem

  • Multimodal-agent trajectories span interdependent tool-use steps, while current agents have limited mechanisms for turning prior trajectories into reusable procedural knowledge.Skills and experience-based guidance are used to structure planning and tool use on future tasks [12] [16] [36].Source: §1 · p. 1
  • Skill-augmented reinforcement learning lets skill libraries evolve alongside policy optimization, yet policy updates remain driven primarily by sparse outcome rewards that do not directly supervise intermediate decisions [6].Inference-time skill retrieval improves planning and tool use without updating model parameters [14] [47].Source: §1 · p. 2

Motivation

  • Rubric-based rewards offer explicit criteria for intermediate progress, but expert rubrics are costly to scale and automatic rubrics can struggle to identify suitable objectives and evidence.Rubrics that are both scalable and aligned with the agent’s execution procedure remain challenging to obtain.Source: §1 · p. 2

Novelty

  • A skill’s procedural stages, quality criteria, expected evidence, and common mistakes can specify evaluation objectives and failure conditions as well as how to act.Deriving guidance and evaluation criteria from a shared procedural source aligns execution and process evaluation by construction.Source: §1 · p. 2

Contribution

  • SkillRubric pairs actor-facing guidance with a rubric derived from the same strategy, using screenshots and tool outputs to assign evidence-grounded, turn-level rewards.A multimodal verifier assesses goal completion and progress and attributes them to responsible turns.Source: §1 · p. 3
  • Consistent performance improvements across diverse multimodal agent benchmarks are reported, and controlled paired rollouts show evolved skills give more effective planning and tool-use guidance.The paired rollouts compare evolved skills with preceding versions under a frozen policy.Source: §1 · p. 3

2. Related Work

Prior work studies reusable skills, fine-grained agent supervision, and rubric-based rewards, but these lines generally leave skill use and reward specification disconnected. SkillRubric derives evaluator rubrics from the procedural knowledge used to guide the agent.

Multimodal agents and reusable procedural knowledge

  • Prior skill methods use procedural knowledge chiefly for action generation, while SkillRubric also derives evaluator rubrics from that knowledge.The cited approaches include inference-time retrieval and reinforcement-learning methods that integrate skill discovery and refinement [3] [13] [14] [18] [39] [42] [47].Source: §2 · p. 3

Fine-grained supervision for agentic reinforcement learning

  • Fine-grained supervision methods infer local utility from intermediate rewards, rollout structures, learned potentials, or privileged-context teachers, whereas SkillRubric defines verifiable process goals from reusable skills [6] [20] [30] [33] [34] [38].The described approaches include turn-level advantage estimation, progress-based rewards, and self-distillation.Source: §2 · p. 3

Rubric-based rewards and adaptive evaluation

  • Rubric-based methods address scalable construction, fine-grained attribution, and adaptive evaluation, but generally construct rubrics independently of reusable actor skills [5] [11] [15] [17] [19] [32] [41].SkillRubric derives guidance and evaluation criteria from a shared skill representation and validates their updates separately.Source: §2 · p. 3

3. Methodology

SkillRubric builds a dual-view SkillBank from teacher trajectories, uses retrieved guidance and turn-level rubric evaluation in policy optimization, and alternates outcome-grounded revisions to guidance and rubrics. The updates are separated so their effects can be assessed without changing both components at once.

3.1. Dual-View SkillBank Construction from Multi-Teacher Trajectories

  • The SkillBank is distilled hierarchically: successful and failed trajectories inform within-teacher skill candidates, then cross-teacher consolidation reconciles conflicts, preserves complementary skills, and prunes redundancy.Successes additionally warm-start the student through supervised fine-tuning; all trajectories are used for skill distillation.Source: §3.1 · p. 4
  • Each skill links applicability, reusable guidance, and evaluator rubrics; guidance omits task-specific answers and fixed paths, while rubrics avoid action- or tool-specific requirements to support transfer.Similar task-level skills are merged by applicability, guidance, and evaluation criteria, consolidating shared principles and removing residual task-specific details.Source: §3.1 · p. 5

3.2. Skill-Conditioned Policy Optimization with Turn-Level Rubrics

  • The method retrieves a skill by matching its applicability description to the task, then uses its guidance to condition rollouts and its rubric for turn-level process supervision alongside outcome advantage.The selected skill has the highest cosine similarity between task and skill applicability descriptions in a shared embedding space.Source: §3.2 · pp. 5–6
  • A coarse-to-fine verifier selects relevant rubric criteria using accumulated evidence, then evaluates matched turns with context summaries and observations to assign criterion satisfaction and responsibility.Turn-level scoring uses the response and resulting screenshot or tool observation, with preceding evidence available for interpretation.Source: §3.2 · pp. 5–6
  • Turn-level process advantage combines responsibility-weighted rubric satisfaction with incremental progress, which rewards progress or penalizes regression at the turn where it occurs.The process advantage is combined with a group-relative outcome advantage; environment-generated screenshots and tool outputs are excluded from the policy loss.Source: §3.2 · p. 6

3.3. Outcome-Grounded Alternating Skill Co-Evolution

  • The initial SkillBank comes from static teacher demonstrations that cannot anticipate novel strategies or failure modes emerging as the policy learns.The authors alternate updates because guidance shapes trajectories while rubrics determine how those trajectories receive process credit.Source: §3.3 · p. 7
  • Guidance revisions are tested in paired rollouts under a frozen policy, while rubric revisions are validated offline on identical outcome-labeled trajectories under fixed guidance.Rubric validation follows policy updates, and rubric revisions are accepted when they better align with fixed outcome verdicts without uniformly relaxing criteria.Source: §3.3 · p. 7
  • Repeating the alternating cycle progressively refines guidance and supervision while grounding each update in final task outcomes.The design uses the student’s experience as a source for ongoing SkillBank improvement rather than relying only on one-time distillation.Source: §3.3 · p. 7

4. Experiments

Across eight multimodal benchmarks, SkillRubric improves trained-method performance, benefits from evolving both skill components, transfers across models, and reduces interaction turns.

4.1. Experimental Setup

  • The evaluation uses eight multimodal benchmarks, with four in-distribution benchmarks and four out-of-distribution benchmarks reserved exclusively for evaluation.Training tasks are sampled from the in-distribution benchmarks; no out-of-distribution tasks or trajectories enter training.Source: §4.1 · pp. 7–8

4.2. Main Results

  • SkillRubric achieves the best performance among trained methods across eight benchmarks and both backbones, exceeding SkillRL by 4.76 and 3.55 percentage points on average, respectively.The reported comparisons are for Qwen3-VL-8B and Qwen3.5-9B, respectively.Source: §4.2 · p. 8, Table 1 · p. 9

4.3. Ablation Study

  • H=10 yields the best IID and OOD performance on both backbones among tested intervals, and evolving both guidance and rubrics outperforms evolving either alone.All tested finite evolution intervals outperform a fixed SkillBank.Source: §4.3 · pp. 9–10, Table 3 · p. 10

4.4. SkillBank Evolution, Transferability, and Efficiency

  • Successive SkillBank versions improve IID and OOD performance across evaluated model families, including within-family and cross-family transfer settings.All models other than Qwen3-VL-8B and Qwen3.5-9B use versions evolved with Qwen3.5-9B.Source: §4.4 · p. 10, Figure 3 · p. 11
  • 5.86 to 4.49 turns for Qwen3-VL-8B and 9.51 to 4.55 turns for Qwen3.5-9B are the reported average interaction-turn reductions.The reductions are reported across IID and OOD benchmarks.Source: Table 4 · p. 11, §4.4 · p. 11

5. Conclusion

SkillRubric pairs actor guidance with evaluator rubrics derived from shared procedural knowledge, and alternates their co-evolution. Across eight benchmarks and two backbones, it outperforms prior skill-based methods, transfers across model families, and reduces interaction turns.

5. Conclusion

  • SkillRubric pairs actor-facing guidance with evaluator-facing rubrics, turning reusable skills into execution knowledge and evidence-grounded turn-level supervision.Source: §5 · p. 11
  • Its alternating co-evolution updates guidance and rubrics against fixed outcomes while isolating their effects.Source: §5 · p. 11
  • Across eight benchmarks and two backbones, SkillRubric outperforms prior skill-based methods, transfers across model families, and reduces interaction turns.Source: §5 · p. 11

A. Dataset Details

The experiments use four benchmarks for in-distribution training and evaluation and four separate benchmarks for out-of-distribution evaluation. The section describes benchmark task coverage and the splits used for each.

A. Dataset Details

  • The four IID benchmarks supply 100 seed training tasks each, while the four OOD benchmarks are excluded from training and SkillBank construction.IID evaluation uses disjoint tasks; the OOD set includes all tasks from three benchmarks and 200 sampled TIR-Bench tasks.Source: A. Dataset Details · p. 16
  • AgentVista covers realistic visual tasks across 25 subdomains and seven broad categories; 100 tasks train the model and the remaining 109 serve as IID evaluation.Tasks typically require long-horizon interaction with tools such as web search, image search, webpage navigation, and code-based image processing.Source: A. Dataset Details · p. 16
  • BrowseComp-VL tests multimodal multi-hop retrieval and reasoning, while VDR-Bench targets visual retrieval and cross-modal evidence integration.The experiments use 100 BrowseComp-VL training tasks and 299 IID evaluation tasks; VDR-Bench uses 100 training tasks and 300 disjoint evaluation tasks.Source: A. Dataset Details · p. 16
  • VisualToolBench covers visual manipulation and reasoning, but experiments use only its single-turn subset: 100 training tasks and 200 disjoint IID evaluation tasks.The benchmark includes operations such as cropping, enhancement, transformation, measurement, and programmatic image analysis.Source: A. Dataset Details · p. 16
  • OOD evaluation spans visual-native browsing, multimodal browsing and search, and tool-assisted visual reasoning benchmarks.VisBrowse-Bench and MMSearch-Plus are used in full; MM-SearchExam uses all 283 tasks, and TIR-Bench contributes 200 sampled tasks across its 13 categories.Source: A. Dataset Details · p. 17

B.1. Tool Environment

Policies use a shared schema-validated tool environment with five tools and a limit of 20 assistant turns per rollout. Long textual observations are summarized for the policy, while the verifier retains the original evidence.

B.1. Tool Environment

  • All policies use the same five tools—zoom, code interpreter, web search, image search, and visit—and each rollout is limited to 20 assistant turns.The tools and their execution settings are summarized in Table 6; compared methods share the same tools and per-trajectory call limits.Source: §B.1 · p. 17, Table 6 · p. 18
  • ImgBB provides temporary image hosting when an external service requires an image URL.Source: §B.1 · p. 18
  • Web-search and webpage-visit observations longer than 2,000 characters are summarized before entering the policy context.The summary uses Gemini 3.1 Flash-Lite and is capped at 4,096 tokens; original observations and visual evidence remain available to the verifier.Source: §B.1 · p. 18

B.2. Training and Optimization

The training setup combines teacher-generated trajectories, supervised fine-tuning, and reinforcement learning from a shared SFT initialization. Retrieved actor-facing skills guide rollouts, while clipped GRPO optimizes the policy.

B.2. Training and Optimization

  • Three teachers generate three trajectories each for every IID seed task; successful and failed trajectories build the initial SkillBank, while about 2,000 successful trajectories are retained for SFT.Source: §B.2 · p. 18
  • Both backbones receive one SFT epoch with AdamW, while their visual encoders remain frozen.All non-vision parameters are updated; RL methods sharing a backbone start from the same SFT checkpoint.Source: §B.2 · p. 18
  • For each task, the system retrieves the top-1 skill using cosine similarity between normalized applicability-description embeddings.The frozen skill synthesizer constructs the initial SkillBank and proposes co-evolution updates.Source: §B.2 · p. 18
  • Retrieved actor-facing guidance is appended to the system message, but the applicability description, paired rubric, and verifier feedback are hidden from the policy.The guidance is explicitly non-prescriptive and may be replaced by another method that reaches the same evidence state.Source: §B.2 · pp. 18–19
  • Each rollout batch samples eight trajectories for each of eight tasks, and policy optimization uses clipped GRPO.Trajectories are limited to 20 assistant turns; the method uses learning rate 3 × 10^-6, clipping parameter 0.2, and KL coefficient 0.1.Source: §B.2 · p. 19

B.3. Outcome and Rubric Evaluation

A fixed outcome evaluator scores final answers, while a two-stage multimodal verifier evaluates rubric criteria and attributes process credit to turns. The verifier tracks evidence-state changes under causal and binary-judgment rules.

B.3. Outcome and Rubric Evaluation

  • The fixed training-time outcome evaluator assigns binary correctness rewards, averages benchmark-provided criterion judgments for VisualToolBench, and gives zero to trajectories without valid final answers.It is independent of the retrieved rubric; final benchmark evaluation uses a separate judge.Source: §B.3 · p. 19
  • The verifier maps missing or contradicted evidence to 0, hypothesis to 0.25, supported to 0.6, and verified to 1, then scores each turn’s change in that state.Positive, negative, and zero changes indicate progress, regression, and no verified change, respectively.Source: §B.3 · p. 20
  • A coarse selector summarizes each turn using evidence available by its end, checks criterion applicability, and identifies candidate turns for fine-stage review.The fine multimodal verifier examines selected turns and raw evidence, then assigns binary criterion satisfaction and responsibility scores over turns.Source: §B.3 · pp. 19–20
  • Judgments use only evidence available by the end of the evaluated turn; later evidence cannot retroactively reward an earlier guess.Visual evidence must be available by the event turn, and when required visual evidence is absent the verifier uses “unverifiable.”Source: §B.3 · p. 22
  • Checkpoint verdicts are binary, and criteria marked not applicable or unverifiable are excluded from process-advantage computation.Zero-variance satisfaction signals are suppressed while valid progress signals are retained; groups with no outcome or process advantage are removed.Source: §B.3 · pp. 20, 22, 23

B.4. SkillBank Co-Evolution

SkillBank co-evolution diagnoses guidance and rubric defects separately, then validates each revision under controlled conditions. Guidance changes require paired-rollout gains, while rubric changes must pass offline outcome-alignment and pass-rate checks.

B.4. SkillBank Co-Evolution

  • Guidance patches are proposed only for genuine guidance defects, while rubric proposals address diagnosed ambiguity, coverage, applicability, verifiability, weight-calibration, or judge-instability issues.The active rubric remains fixed during proposal generation, and the applicability description and outcome evaluator remain unchanged throughout co-evolution.Source: §B.4 · p. 23
  • A guidance candidate is accepted only if matched rollouts improve mean outcome without a paired regression greater than 0.05.Each comparison uses eight tasks, one rollout per guidance version per task, the same decoding configuration, and at least four valid rollout pairs.Source: §B.4 · p. 25
  • Rubric candidates are evaluated offline on the same 128 outcome-labeled trajectories after another H optimization steps with the selected guidance and current rubric.This delay lets trajectories adapt to updated guidance, while rejudging identical cached trajectories holds policy updates and environment variation fixed.Source: §B.4 · p. 25
  • A rubric candidate must have positive estimated alignment gain, with the one-sided 90% lower confidence bound allowed to extend to −0.02.Alignment compares the rubric's weighted score with the fixed outcome verdict; the lower-bound allowance accommodates finite-sample uncertainty.Source: §B.4 · p. 25
  • Rubric pass-rate change must be at most 0.10; otherwise the current rubric is retained, while validated guidance remains unchanged.Both rubric checks must pass for the candidate rubric to replace the current one.Source: §B.4 · p. 25

B.5. Evaluation Details

This section details evaluation and baseline controls, the staged construction of an initial dual-view SkillBank from teacher trajectories, and the construction and aggregation of aligned guidance and evaluation criteria into transferable SkillBank entries.

B.5. Evaluation Details

  • Evaluation uses all benchmark test tasks with one near-greedy rollout, top-1 retrieved guidance for skill-conditioned models, and judge-based scoring, including criterion-averaged VTB-Bench scores.Final answers must use <answer>...</answer> tags; invalid, aborted, or truncated trajectories score as incorrect. Judges accept semantic equivalence and apply ±5% tolerance only to free-form numerical answers.Source: §B.5 · pp. 25–26

B.6. Baseline Adaptation Details

  • Baselines use the same initial actor-facing skills while retaining their distinct mechanisms; policy-training comparisons share training controls, but baselines lack rubrics, verifier feedback, and turn-level credit.Skill-SD and SKILL0 follow their original inference protocols without skill injection.Source: §B.6 · p. 28

C. Prompts for Initial Dual-View SkillBank Construction

  • Initial SkillBank construction sequentially aggregates trajectories within teachers, consolidates skills across teachers, restructures them into aligned guidance and criteria, then aggregates related tasks.The representation comprises an applicability description, actor-facing guidance, and evaluator-facing rubric criteria.Source: C. Prompts for Initial Dual-View… · pp. 28–29

C.1. Intra-Teacher Trajectory Aggregation

  • For each teacher–task pair, the synthesizer jointly analyzes that teacher’s rollouts to extract recurring strategies, effective decisions, failure patterns, and transferable techniques into one candidate skill.This aggregation reduces stochastic variation across individual rollouts before cross-teacher consolidation.Source: §C.1 · p. 29

C.2. Cross-Teacher Skill Consolidation

  • Cross-teacher consolidation distinguishes shared high-confidence strategies from complementary individual alternatives and disagreements, preserving multiple valid procedures when tasks admit different solution paths.The resulting model-agnostic skill reduces teacher-specific bias while retaining complementary expertise.Source: §C.2 · p. 30
  • Consolidated skills must remain model-agnostic and actionable, preserve valid alternative approaches and intermediate checkpoints, and exclude task-specific answers or ground-truth values.Source: §C.2 · p. 32

C.3. Dual-View Guidance–Rubric Construction

  • Dual-view skills separate reusable action guidance from observable, method-agnostic progress criteria, which must be causally linked to task outcomes and avoid answer leakage.Guidance provides three to five stages, while criteria provide three to five discriminative checkpoints that evaluate progress independently of the final answer.Source: §C.3 · pp. 33–35

C.4. Cross-Task Skill Aggregation

  • Within each benchmark, procedurally similar task-level skills are clustered and merged into capability-level skills, then pooled into a unified SkillBank.Complete-linkage clustering stops when the smallest remaining distance exceeds 1 − τ_cluster; a synthesizer rejects incoherent merges and preserves supported alternatives.Source: §C.4 · pp. 36–37

D.1. Dual-View Skill Representations

The SkillBank presents each skill as actor-facing guidance and an evaluator-facing rubric derived from one procedural skeleton. Across four examples, criteria turn the procedure into verifiable intermediate goals and localized turn-level credit.

D.1. Dual-View Skill Representations

  • Conditional-value reconstruction identifies evidence anchors, binds entities and attributes, resolves governing rules, then computes or verifies the result.The rubric treats wrong bindings, wrong rules, and inconsistent computation as distinct failure points.Source: §D.1 · pp. 37–38
  • Sports-event grounding proceeds from visual anchors through competing contexts and exact event resolution before deriving the requested fact; four rubric criteria track those stages.The cricket example credits anchor extraction and context resolution even when the agent later commits to the wrong league.Source: §D.1 · pp. 38–39
  • Financial-table reasoning localizes visual evidence, binds values to labels, separates comparison scopes, and verifies the final result.The rubric distinguishes reading the correct table but summing the wrong column from never locating the table.Source: §D.1 · pp. 39–40
  • Weak-clue identity resolution tests candidate people against combined visual, biographical, relational, and temporal constraints before extracting a fact for the confirmed person.Its criteria separately track clue isolation, identity resolution, chain alignment, and target extraction.Source: §D.1 · pp. 40–41

D.2. A Representative Case: Image-Anchored Search with Hypothesis Testing

On one cricket task, SkillRubric’s trajectory tests competing event contexts and rechecks image clues, while the base model follows a misleading sponsor cue. The rubric localizes where their paths diverge.

D.2. A Representative Case: Image-Anchored Search with Hypothesis Testing

  • SkillRubric identifies Sri Lanka and gives Ranatunga’s field-first decision and the dew factor; the base model answers with the wrong team and captain.The paired trajectories show the base model drifting toward the Caribbean Premier League and India’s 2011 final, while SkillRubric weighs 1996 against 2014 and rechecks the jersey clues.Source: §D.2 · pp. 41–42
  • The authors attribute the divergence to an early wrong anchor that drives confirmation-seeking, versus a query encoding the visual identity and later verification.The base model’s first query emphasizes the KENT sponsor; the co-evolved policy’s query uses Sri Lanka, Ranatunga, and 1996.Source: §D.2 · p. 42
  • The base model extracts jersey anchors but fails when it commits to the Caribbean league; SkillRubric satisfies all four rubric checkpoints in sequence.Outcome-only supervision gives both trajectories the same terminal zero, whereas the rubric distinguishes correct anchoring followed by wrong resolution from never grounding at all.Source: §D.2 · p. 42
  • The successful run follows the skill’s stages: extract anchors, compare candidate contexts, re-read image evidence, then derive the requested fact.The trajectory compares Sangakkara 2014 with Ranatunga 1996 before committing to the latter.Source: §D.2 · p. 42

D.3. Comparative Trajectory Analysis

Across eight representative tasks, SkillRubric solves cases failed by the base model. The trajectory comparisons attribute the differences to specific choices in query construction, evidence checking, constraint use, and data collection.

D.3. Comparative Trajectory Analysis

  • All eight representative tasks are cases the base model fails and SkillRubric solves, with gains concentrated at recurring critical decision points.The reported recurring nodes include discriminative query construction, source verification, image grounding, constraint adherence, and search-budget management.Source: §D.3 · p. 43
  • Within a seven-search budget, obtaining all four real ratings and correcting “ORAN” to Oban identifies Highland Park as the winner; fabricated missing ratings reverse the top-two ranking.The successful trajectory reports Highland Park’s rating as 88.91 and Value Efficiency Score as 25.87.Source: §D.3 · pp. 43–44
  • A quoted query linking both album titles to one artist, followed by an image check, resolves the musician’s identity where the base model persists with a false candidate.The SkillRubric trajectory identifies Dan Reeder and answers that Smithereens followed Every Which Way; the base model attributes the title to Brian Davison.Source: §D.3 · p. 44
  • Preserving the intent word “disease” in a Chinese query retrieves Menkes disease records, whereas the base model’s generic chemistry search produces a wrong guess.Both models switch to Chinese, but the SkillRubric query retains the disease constraint alongside the reagents.Source: §D.3 · p. 45
  • Checking the requested language against Florence image evidence yields the painting’s Italian title; the base model accepts a misleading French snippet.Both models identify the painting in one step, leaving the language constraint as the decisive difference.Source: §D.3 · p. 45
  • Solving a linear equation from the chart’s endpoint values gives the intersection estimate 67.86 and option 68, while visual interpolation gives option 66.The authors report the interpolation error as 2 units.Source: §D.3 · p. 46
  • A query including the person, group, object, and positional constraint retrieves jersey number 6 immediately; the base model exhausts its search budget without an answer.The base model’s query variants omit “girl to his right.”Source: §D.3 · p. 46
  • Combining chart reading with targeted searches supplies both the gene identity and the external stability criterion needed to answer the reference-gene question.The successful answer names RPS5 and distinguishes low median Cq from stable expression; repeated zooming alone does not answer the conceptual part.Source: §D.3 · p. 47

Related papers

Loading 2609.34557v1…