Pistis Technical Report

1 Introduction

• 82.3 vs. 82.4: Pistis-27B-Thinking is comparable to Qwen3.8-27B on 24 non-grounding benchmarks, while its grounding average is 80.5, 8.6 points higher.Pistis-9B-Thinking achieves the highest averages among compared models at a similar scale on both benchmark groups; both Agentic variants achieve higher overall averages than their corresponding base models.Source: Figure 1 · p. 1, §1 · p. 2
• 8.8, 3.3, 3.2, and 9.7 points: Pistis-27B-Agentic gains over Qwen3.6-27B on BrowseComp-VL, MMSearch, VDR-testmini, and LiveVQA, respectively.Pistis-9B-Agentic gains 8.4, 8.0, 3.2, and 11.4 points over Qwen3.5-9B on those benchmarks, respectively.Source: Figure 1 · p. 1, §1 · p. 2
• Pistis comprises 27B and 9B Thinking models for deep multimodal reasoning and Agentic models for long-horizon interaction, built on large-scale multimodal SFT with structured reasoning data.Agentic variants additionally use trajectories for planning, iterative reasoning, and tool use across textual and multimodal domains.Source: §1 · p. 2
• IDRL alternates on-policy distillation and RL updates within one training loop, while step-level positive-advantage suppression improves credit assignment for long-horizon agentic trajectories.The interleaving improves exploration and optimization stability, and outperforms pure on-policy distillation, pure RL, and their static combination in ablations.Source: §1 · p. 2
• PAH iteratively proposes and validates bounded inference-harness revisions from development trajectories, retaining improvements while keeping the agentic policy and tool interface fixed.The selected harness is frozen for evaluation; the trace-guided procedure is instantiated on multimodal search and applies where an executable harness and measurable development feedback are available.Source: §1 · p. 2

2.1 Supervised Fine-Tuning

• A roughly 3.2M-pair multimodal SFT corpus provides shared reasoning data, with agentic trajectories added for Pistis-Agentic.The corpus spans reasoning domains and mixes tool-integrated reasoning, search, and general agent trajectories; filtering selects intermediate-difficulty successes and checks answer correctness and data quality.Source: §2.1 · pp. 3–4

2.2 Interleaved Distillation and Reinforcement Learning

• IDRL alternates on-policy distillation and RL with flexible phase budgets; 9B variants use IDRL, while 27B variants use pure RL and serve as frozen teachers.Training switches to pure RL after the entropy plateau, when continued distillation adds little diversity and may cap the student at the teacher’s capability.Source: §2.2 · p. 4, Table 1 · p. 5

2.2.1 On-Policy Distillation

• 65.0 (base), 65.4 (off-policy SFT), and 65.7 (on-policy OPD) Geo3K accuracy show marginal direct-distillation gains, while cold-start OPD remains stable and improves.The results point to the initial student–teacher distribution gap as the obstacle; teacher-response SFT before OPD restores effective multimodal distillation.Source: Table 2 · p. 6, §2.2.1 · p. 6, Figure 3 · p. 6

2.2.2 Reinforcement Learning with Verifiable Reward

• SAPO smoothly attenuates off-policy updates, while step-level positive-advantage suppression prevents ineffective intermediate tool calls from being reinforced by successful trajectory rewards.Rewards prioritize automatic verification—format checks, exact-match correctness, or IoU—and use a model judge for open-ended tasks without deterministic checks.Source: §2.2.2 · p. 7

2.2.3 Benefits of Interleaved Distillation and Reinforcement Learning

• Alternating single-objective steps avoids the gradient conflict of a summed RL-plus-distillation loss, while dense, mass-covering OPD signals counter RL-driven entropy collapse.Each active objective receives an uncombined update; experiments report sustained entropy with OPD and smoother gradient norms for interleaving than for the summed variant.Source: §2.2.3 · pp. 8–9

2.3 Closed-Loop Auto-Harnessing for Multimodal Search

• PAH optimizes Pistis-Agentic’s multimodal-search inference harness while keeping the model, tool protocol, and evaluation entry point fixed.PAH names the optimization procedure; its output is a concrete harness selected and frozen for runtime.Source: §2.3 · p. 9

2.3.1 Auto-Harnessing Procedure

• PAH optimizes editable harness components around a frozen Pistis-Agentic model, accepting a change only after canary testing and improved full-development evaluation.It makes one independently switchable change per round, automatically rolls back unsuccessful candidates, and evaluates once on a disjoint test set that does not inform optimization.Source: §2.3.1 · p. 9

2.3.2 Baseline and Resulting Optimized Harness

• The resulting harness adds a Candidate Ledger, conditional Search Skills, evidence-driven checkpoints, and budget-aware workflow while leaving decisions and final answers to the model.The ledger tracks candidates and traceable supporting or refuting evidence; skills encode reusable procedures, and bounded checkpoints help determine when to continue searching or answer.Source: §2.3.2 · pp. 10–11

2.3.3 Relation to Automated Agent Optimization

• PAH differs from ADAS (Hu et al., 2024), AFlow (Zhang et al., 2024a), and GEPA (Agrawal et al., 2025) by optimizing auditable harness changes around a fixed policy and protocol.It can revise deterministic code and prompts sequentially, accepting revisions only when the fixed development metric improves.Source: §2.3.3 · p. 11

2.4.1 Training

• Over 80% end-to-end speedup over a standard Hugging Face pipeline and ∼2× speedup over conventional synchronous RL pipelines are reported for training infrastructure.The SFT setup combines VeOmni, dynamic token-budget batching, FSDP2 sharding, and other optimizations; asynchronous VeRL rollouts with vLLM enable parallel sample generation and parameter updates.Source: §2.4.1 · pp. 11–12

2.4.2 Evaluation

• PistisEvalKit separates model wrappers and inference backends from benchmarks, supports agentic tool-calling evaluation, and automates checkpoint evaluation during training.Public models use their developers’ recommended inference configurations; Pistis models use temperature 0.0 and top-p 1.0.Source: §2.4.2 · p. 12

2.4.3 Inference

• Over 100% inference-speed improvement in some prefill scenarios and up to 99% GPU utilization with multistream are reported for system-level optimizations.The inference setup also pads affected ViT parameter dimensions to multiples of 128 to enable high-performance TMA operators on H20 GPUs.Source: §2.4.3 · p. 13

3.1.1 Perception and Reasoning Tasks

• Pistis-27B averages 80.5 on grounding versus 76.4 for Qwen3.6-27B and 71.9 for Qwen3.8-27B; Pistis-9B reaches 80.1 versus 73.0 for Qwen3.5-9B.Outside grounding, Pistis-27B averages 82.3 on non-grounding benchmarks versus 82.0 for Qwen3.6-27B and 82.4 for Qwen3.8-27B; Pistis-9B averages 80.6, 0.6 points above Qwen3.5-9B. Gains are broad but nonuniform, including weaker results on MMVP and, for Pistis-27B, a 7.1-point deficit to Qwen3.8-27B on BLINK.Source: §3.1.1 · pp. 13–14

3.1.2 Agentic Tasks

• Across 18 agentic benchmarks, Pistis-27B-Agentic averages 78.3 versus 75.7 for Qwen3.6-27B, while Pistis-9B-Agentic averages 74.7 versus 71.5 for Qwen3.5-9B.Multimodal search is a major source of improvement over the corresponding base models, with gains on BrowseComp-VL, MMSearch, VDR-testmini, and LiveVQA; Pistis-27B-Agentic and Qwen3.8-27B have similar overall scores (78.3 vs. 78.0) but different task strengths.Source: §3.1.2 · pp. 14–15, Table 5 · p. 14

3.2 Held-Out Evaluation of the PAH-Optimized Harness

• On VDR-testmini, the frozen Optimized Harness reaches 28.6% accuracy versus 26.8% for the Baseline Harness, a gain of 1.8 percentage points at the same 15-interaction limit.The harness used 8.60 average interactions versus 8.68 for the baseline; it also transfers without target-benchmark tuning, improving MMSearch, BrowseComp-VL, and LiveVQA by 0.4, 1.7, and 1.0 points, respectively.Source: §3.2 · p. 15, Table 6 · p. 15, Table 7 · p. 15

3.3 Ablation Study

• On 18 benchmarks, IDRL achieves the highest reported average, 74.7, versus 74.1 for sequential OPD→RL, approximately 74.0 for pure RL, 74.1 for joint RL+OPD, and 73.4 for pure OPD.IDRL alternates RL and OPD, showing phase-wise entropy variation and more stable gradient norms than pure RL; its average is highest, though it does not lead every category against sequential OPD→RL.Source: §3.3 · p. 16, Table 8 · p. 17, Figure 7 · p. 17
• Positive-advantage suppression raises the 18-benchmark average to 74.7 from 74.3 without PAS, with the largest drops without PAS on Search-Oriented tasks (1.2 points), Multimodal Reasoning (0.9), and PinchBench (0.8).PAS sets positive token advantages to zero for rejected or ineffective assistant steps while retaining zero or negative advantages, selectively limiting positive credit for ineffective intermediate actions.Source: §3.3 · p. 17, Table 9 · p. 18

3.4 Training Observations and Design Lessons

• Training uses task-specific interaction limits—5 rounds for TIR and 15 for search—because unnecessarily long TIR horizons can prompt uninformative actions such as blank-image generation.Qwen3.6-27B-generated filtered SFT data yielded stronger downstream performance than data from a higher-raw-pass-rate generator; high-resolution crops aid TIR, while reliable page acquisition and fine-grained extraction support search.Source: §3.4 · pp. 17–18

3.5 Comparison with Business Benchmarks

• On the Pistis Benchmark, business-data-fine-tuned †Pistis-9B scores 89.7, versus 86.2 for Qwen3.5-9B and 84.3 for Qwen3-VL-8B-Thinking.The benchmark contains approximately 10,000 high-quality samples for business content-safety and integrity scenarios, testing policy-grounded, cross-modal tasks across video and image inputs.Source: §3.5 · pp. 18–19, Table 10 · p. 19

4 Failure Analysis and Future Directions

• Pistis-Agentic may misinterpret salient visual cues, use tools to confirm a preselected hypothesis, or fail to connect retrieved evidence and uncertainty to its final answer.These failures often occur despite adequate low-level perception, so stronger benchmark performance alone does not ensure reliable evidence use.Source: §4 · p. 19
• PAH uses candidate–evidence bindings, evidence-driven checkpoints, and conditional Search Skills, but cannot recover correct entities missing from the upstream candidate set.Future work should combine inference-time orchestration with process-level training signals for intent verification, decision-relevant tool use, candidate recall, and consistency between retrieved evidence and final answers.Source: §4 · p. 19

• On PinchBench, Pistis-9B-Agentic exceeds Qwen3.5-9B by 2.8 points, and Pistis-27B-Agentic exceeds Qwen3.6-27B by 0.8 points.Pistis-Agentic also shows its strongest gains in multimodal search and consistently improves Claw-Style interaction at both scales.Source: §5 · p. 20
• Large-scale reasoning-data SFT followed by IDRL yields Pistis-Thinking for deep multimodal reasoning and Pistis-Agentic, whose SFT additionally includes agentic trajectories.IDRL achieves the strongest aggregate Claw-Style score in ablations, removing PAS reduces that score, and interleaving distillation with reinforcement learning is more stable and effective than either alone or static-weight joint optimization.Source: §5 · p. 20
• Under a matched interaction budget, PAH raises VDR-testmini accuracy from 26.8% to 28.6% with a frozen model.Its closed optimization loop revises the harness, which combines a Candidate Ledger, conditionally loaded Search Skills, and budget-aware adaptive workflows. The same harness improves MMSearch, BrowseComp-VL, and LiveVQA without target-set tuning, transfers positively to Seed-2.1-turbo and GPT-5.5, but remains unvalidated in coding and Claw-Style domains.Source: §5 · p. 20
• Pistis-Agentic can still mis-bind question intent, use tools for confirmation rather than measurement, and leave retrieval and reasoning loosely coupled.The conclusion calls for process-level supervision of intent verification, decision-relevant tool use, and consistency between intermediate and final answers, potentially integrated into IDRL’s RL phases.Source: §5 · p. 20

A Additional Analysis of the PAH-Optimized Harness

• All reported statistics are computed after the Optimized Harness is frozen.Source: A Additional Analysis of the… · p. 23
• The statistics do not feed back into proposal generation or version selection.Source: A Additional Analysis of the… · p. 23
• The analysis characterizes the resulting system rather than isolating any component’s causal effect.Source: A Additional Analysis of the… · p. 23

A.1 Runtime Audit of the Candidate Ledger and Search Skills

• 497 of 500 trajectories attempted a Candidate Ledger record, 490 recorded one successfully, and Search Skills activated on 135 trajectories, usually after difficulty.The first ledger attempt occurred after 2.61 environment interactions on average.Source: Table 11 · p. 23, §A.1 · p. 23
• The Candidate Ledger serves as the routine evidence-state mechanism, while Search Skills provide sparse query recovery after deterministic search failures.The repair-search-query skill diagnoses an unproductive query and formulates a materially revised one; no trajectory reached the ledger context-truncation or updatecount cap.Source: §A.1 · p. 23
• The audit does not establish that skills provide little benefit: limited activation may reflect learned procedures or limitations in triggering and routing.Because difficult trajectories are more likely to trigger recovery, lower accuracy among skill-using trajectories would not show that skills cause failure; controlled ablations are needed to quantify their marginal contribution.Source: §A.1 · p. 23
• A verified ledger candidate signals multi-source support, not proof that visual identity, relation direction, all premises, or the requested terminal field are correct.Record frequency or source count alone is therefore not an appropriate optimization objective.Source: §A.1 · p. 23

A.2 A Candidate Ledger Trajectory

• In a successful VDR-testmini trajectory, the ledger retains a useful style hypothesis after a film hypothesis fails, exposes a missing material constraint, and lets later evidence repair the entity.The query asks how a dress pattern influenced a 1980s horror subgenre and which materials were used for robotic antagonists in a representative film.Source: §A.2 · p. 23, Table 12 · p. 24
• The ledger makes hypothesis revision and constraint coverage explicit while the same frozen reasoning model carries out every evidence action and gives the final answer.Evidence counts in the candidate-evolution table denote source-bound items recorded by the ledger.Source: §A.2 · p. 23, Table 12 · p. 24

A.3 Transfer across Frozen Policy Models

• All three policies improved over their corresponding Baseline Harnesses with average interactions close to matched baselines, using the same frozen Optimized Harness without revision or model-specific version selection.The comparison transfers the harness to Seed-2.1-turbo and GPT-5.5 while retaining the Pistis-27B-Agentic matched-budget comparison.Source: Table 13 · p. 24, §A.3 · p. 24
• For GPT-5.5, 47 positive and 27 negative flips occurred among 74 discordant examples; stronger policies may use structured state more effectively, but causality was not isolated.The passage presents this as a possible explanation for GPT-5.5's larger gain, not a causal finding.Source: §A.3 · p. 24

A.4 Design Lessons and Limitations

• A revision should be accepted only when the complete development-set task metric improves; mechanism activation alone is diagnostic.This guards against treating activation of a mechanism as evidence of overall task improvement.Source: §A.4 · p. 24
• Guidance should be exposed conditionally from replayable states, since globally persistent prompts can perturb already-correct trajectories.Each candidate should also be independently derived from the current best harness and remain reversible.Source: §A.4 · p. 24
• Address upstream identity and relation errors before strengthening field extraction, or stricter completion may reinforce an answer about the wrong entity.The lesson is to fix upstream errors before adding downstream constraints.Source: §A.4 · p. 24
• The frozen Optimized Harness may over-commit to an incumbent without an evidence-backed challenger test, and apparent source diversity may mask shared underlying material.These limitations concern the current Optimized Harness.Source: §A.4 · p. 24
• Relation endpoints and image-to-text entity binding remain incompletely certified; budget-aware termination can also prevent decisive verification, and most skills beyond query repair are not reliably activated.Whether PAH finds stronger mechanisms for coding or Claw-Style interaction requires separate development/test studies.Source: §A.4 · p. 24

B Detailed Failure Cases

• Representative failures show adequate perception can still lead to misread task intent or evidence: a chart gap is read as an age gap, and an arrow overrides cross-frame evidence that a circle never moves.The broader failure analysis identifies task interpretation, decision-relevant evidence gathering, and binding evidence to the answer as central weaknesses.Source: Figure 8 · p. 25, B Detailed Failure Cases · p. 25

B.1 Multimodal Reasoning: Salient Cues Are Read, Intent Is Not

• In two visual reasoning cases, Pistis-Agentic misreads salient cues: it treats an age difference as a gap between group ages and follows an arrow instead of comparing a circle’s position across frames.The model identifies the age-group labels correctly but answers a different question; in the motion task, it never compares the circle’s position across frames.Source: §B.1 · pp. 25–26
• In both tool-integrated failure cases, the code interpreter confirms a preselected hypothesis rather than measuring the visual evidence.One rollout crops only the hypothesized region without comparing candidates; the other prints a hard-coded count that is then cited as confirmation.Source: Figure 9 · p. 26

B.2 Tool-Integrated Reasoning: Tools Confirm Instead of Measure

• In agentic search failures, the model either contradicts a correct judgment surfaced by retrieval or skips the available search tool and hallucinates a date with fabricated attribution.The figure marks faulty steps in red and correct judgment in green.Source: Figure 10 · p. 27
• In two code-interpreter rollouts, Pistis-Agentic crops only prior-selected regions or hard-codes a count, treating tool outputs as confirmation rather than decision-relevant measurements.In one rollout it never examines the waist, head, or back; in the other, it eyeballs bars as just over 90 and scripts a hard-coded count. An executed cell that adds no information is treated as independent evidence.Source: §B.2 · pp. 26–27

B.3 Agentic Search: Retrieval and Reasoning Are Loosely Coupled

• Visual search can surface the correct judgment, yet the final answer contradicts it because the model over-differentiates related concepts and fails to bind retrieved evidence.The reasoning trace contains the correct intermediate conclusion, but the model overrides it.Source: §B.3 · p. 28
• For time-sensitive questions, the model may avoid an available search tool, verbalize uncertainty, guess a date from memory, and fabricate attribution.The uncertainty is not operationalized into a tool call.Source: §B.3 · p. 28
• PAH addresses these failures with a Candidate Ledger for sourced candidate facts, checkpoints to update state before answering, and Search Skills for blocked retrieval.The audit also identifies a limit: downstream evidence organization cannot recover a correct entity missing from the upstream candidate set.Source: §B.3 · p. 28

B.4 Implications for Future Work

• Intent-grounding training should teach models to state and verify interpretations of underspecified questions against visual evidence before computation.This is one of three proposed complementary training directions.Source: §B.4 · p. 28
• Measurement-grounded tool-use training should reward decision-relevant evidence generation, such as comparing candidate regions or calibrating chart axes, and penalize confirmatory no-op calls.These examples target tool calls that inform a judgment rather than merely confirm it.Source: §B.4 · p. 28
• Retrieval–reasoning training should reward candidate recall, consistency between intermediate conclusions and final answers, and targeted retrieval prompted by expressed uncertainty.These signals address failures in coupling retrieved candidates, reasoning, and answers.Source: §B.4 · p. 28
• Rule-checkable trajectory signals can enter IDRL’s RL phases, while its distillation phases transfer the corresponding behaviors from a stronger teacher.The paper leaves systematic quantitative evaluation of these failure modes to future work.Source: §B.4 · p. 28