IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems

Motivation

• Stronger CLI capabilities are needed because CLI environments let language models use tools, access files, and execute code through external systems.The paper frames CLI capability as an important target for building more capable agentic models.Source: §1 · p. 2

Results

• IQuest-Q1 demonstrates capabilities across coding, general agentic and reasoning benchmarks, alongside user-facing CLI interaction on IQuest-CLIBench.The results include DeepSWE v1.1, CyberGym, NL2Repo, JobBench, Agents’ Last Exam, and Humanity’s Last Exam.Source: §1 · p. 2

Contributions

• The open-weight sparse MoE model has 320B total parameters, with 15B activated per token, and combines general-purpose, coding-agent, and CLI interaction capabilities.Its model weights are released to support community research and development.Source: §1 · p. 2

Methods

• The CLI-centric pipeline develops capabilities through staged data mixtures, then specializes them with task and environment synthesis, multi-scaffold RL, and MOPD.General pre-training increases the coding-data mixture in its second stage; mid-training expands domains and formats while extending context.Source: §1 · pp. 2–3

Contributions

• IQuest-Q1 participates in a human-supervised iterative model-development process that evaluates its capabilities and suggests improvements for successor models.The process is described as shortening the overall development cycle.Source: §1 · pp. 2–3

2.1. Overall Architecture

• IQuest-Q1 combines sparse MoE layers and hybrid sliding-window attention with Stable SplitNorm, Muon, and μP parameterization.Its attention stack interleaves GQA and SWA; the first Transformer layer uses a dense MLP rather than sparse MoE routing.Source: §2.1 · p. 3
• SplitNorm normalizes the residual stream entering attention but leaves the stream entering the FFN unnormalized, combining Pre-Norm stability with stronger depth utilization.The design adds attention-output RMSNorm and first-layer SandwichNorm to address instability when scaling the formulation to large models.Source: §2.1 · pp. 3–4

2.2. Scaling Efficiency and Stability

• 1.93× higher scaling efficiency than the Qwen3-MoE-style baseline, using identical training data; IQuest-Q1 needs about 52% of the compute to reach the same training loss.The comparison follows the Qwen3-30B-A3B configuration.Source: §2.2 · p. 5
• Across measured scales, IQuest-Q1’s safe learning rate remains substantially above its optimal rate, indicating stability headroom beyond the studied model sizes.The paper defines Critical Model Size as the scale where optimal and safe learning rates coincide.Source: §2.2 · p. 5

2.3. Infrastructure

• Best-fit sequence packing limits padding overhead, while reordering packed sequences across data-parallel replicas balances long-context computation.The infrastructure also overlaps sliding-window attention halo exchange with computation and fuses learnable sink contributions into attention.Source: §2.3 · pp. 5–6

3.1. Pre-Training Data

• The pre-training corpus combines web pages, code, books, papers, news, and domain-specific materials, with cleaning, quality filtering, and exact and fuzzy deduplication.The code data preserves diversity across programming languages, modalities, and task granularities; the four-phase pipeline raises its share from 40% to 70%.Source: §3.1 · pp. 6–7

3.2. Hyper-Parameters

• Muon optimizes most parameters, while Adam handles embeddings, the language-model head, normalization parameters, and MoE routers; learning rates follow a warmup-stable-decay schedule.The schedule reaches 6 × 10^-4, then decays to 2 × 10^-4 by Stage 2’s end and 6 × 10^-5 by mid-training’s end.Source: §3.2 · pp. 7–8

3.3.1. Training Curriculum

• Mid-training combines natural corpora, QA pairs, and agentic trajectories, with token-level loss masks and context extension through 32K, 128K, and 512K tokens.A single cosine learning-rate decay spans the three context phases without additional warmup; optimizer state is reinitialized at transitions while schedule progress is preserved.Source: §3.3.1 · p. 8

3.3.2. Agentic Data Construction Guided by Trajectories

• Because suitable agentic trajectories are scarce and task coverage is limited, selected trajectories and pre-training documents guide construction of more than one million executable instances.Agents are rolled out on the instances, and valid, complete trajectories are retained for mid-training.Source: §3.3.2 · p. 8

3.4. Evaluation

• Pre-training and mid-training are evaluated with held-out trajectory BPB and atomic-ability assessments of long-horizon agent behavior.Lower BPB indicates higher likelihood assigned to behaviors in competent trajectories; the two evaluation paradigms provide complementary signals for tracking and improving capabilities.Source: §3.4 · pp. 8–9

4.1. Large-Scale and Diverse Task Synthesis

• Task synthesis spans general, reasoning, and coding settings, using distinct data sources and domain-specific pipelines to generate tasks at scale.Coding instances use executable environments, multiple harness versions, varied queries, layered verifiers, agent-judge validation, and difficulty adjustment; the library also includes re-tasked verified environments.Source: §4.1 · pp. 9–10

4.2. Environment Integrity and Reward Hacking Mitigation

• Three safeguards address reward hacking: environment preparation, execution-time checks, and trajectory auditing.Confirmed hacking trajectories are excluded from training, and detected cases inform refinements to restrictions and blocking rules.Source: §4.2 · pp. 10–11

4.3. Supervised Fine-Tuning

• SFT starts from the mid-trained checkpoint and trains on assistant responses, including tool calls, while masking user instructions and tool observations from the loss.It proceeds in two stages, with 240B+ tokens in Stage 1 and 50B+ in Stage 2.Source: §4.3 · p. 11

4.4. Environment-Centered Reinforcement Learning

• RL trains a shared policy on task trajectories from multiple harnesses and versions, using independent verifiers for terminal correctness and attributing failures before optimization.Policy-caused errors provide learning signals; unrelated environment or harness faults are masked, and evaluator faults block sample consumption.Source: §4.4 · p. 11

4.5. Multi-Teacher On-Policy Distillation

• MOPD initializes a student from SFT Stage 2 and distills from four frozen experts covering terminal interaction, multi-harness use, long-horizon tasks, and general agentic work.A domain-selected teacher evaluates student-generated tokens to obtain a clipped distillation advantage; instructions and tool observations are excluded from the loss.Source: §4.5 · pp. 11–12

4.6. Model Merging

• Model merging combines parameter deltas from expert and MOPD checkpoints relative to a designated reference, then adds the weighted, norm-calibrated update back to it.Merging is interleaved with training across stages and within expert branches.Source: §4.6 · p. 12

4.7. Post-Training Infrastructure

• The post-training runtime maps different harness protocols to a unified policy interface and supports asynchronous distillation with controlled sample staleness.Teacher weights are loaded on demand, with only active-teacher shards in GPU memory and the next teacher prefetched during the current forward pass.Source: §4.7 · pp. 12–13

5.1. Benchmarks

• The suite covers coding benchmarks including DeepSWE v1.1, SWE-Bench Pro, CyberGym, Terminal-Bench versions 2.1–4.0, and PaperBench, plus general benchmarks including GPQA Diamond, JobBench, and HLE.The evaluation is framed around coding and general agentic capabilities.Source: §5.1 · p. 13
• IQuest-CLIBench evaluates CLI agents in human-in-the-loop settings using script-driven user simulation, execution traces, and environment-state invariants.It is included alongside conventional task performance to assess user experience and interaction quality.Source: §5.1 · p. 13

5.2. Baselines

• Comparisons include proprietary models Claude Opus 5 and GPT-5.6 Sol, and open-source models GLM-5.3-Flash, DeepSeek-V4-Flash-0731, Hy3, and MiniMax M3.Public scores are reported where available; otherwise models are evaluated using corresponding benchmark setups, harnesses, and specified runtime limits.Source: §5.2 · p. 13, §5.3 · p. 13

5.4. Results

• IQuest-Q1 scores 57.5 on SWE-Bench Pro, 64.6 on DeepSWE v1.1, 63.0 on NL2Repo-Bench, and 86.0 on PaperBench.The reported coding results cover repository bug fixing and feature addition, repository generation from natural language, and research-code reproduction.Source: §5.4 · pp. 13–14
• IQuest-Q1 scores 75.0 on Toolathlon-Verified, 52.7 on AutomationBench, and 77.1 on Claw-Eval, while HLE scores 39.2 without tools and 55.7 with tools.Other reported results include 85.5 on BrowseComp, 81.8 on WideSearch, 55.7 on JobBench, 50.9 on APEX-Agents 1.1, 93.3 on GPQA Diamond, and 93.0 on IMOAnswerBench-Verified.Source: §5.4 · p. 14

6.1. Interactive Frontend Generation

• IQuest-Q1 generates browser-runnable HTML, CSS, and JavaScript applications combining visual design, scene construction, and interaction logic.Examples span games, creative tools, and scientific visualization; the garden editor connects planting actions and flower parameters to a 3D scene, while games coordinate controls and feedback.Source: §6.1 · p. 15, Figure 7 · p. 15

6.2. Iterative Development under Human Supervision

• In a human-supervised iteration, an agent traced 35 tool-call errors in one long-horizon task to unfamiliar schemas, mismatched tool names, and parameters.Analysis across 6,054 tool calls found training-data bias toward one schema and preprocessing mismatches; the agent proposed cleaning and augmentation, with human experts reviewing the intervention.Source: §6.2 · p. 15
• The agent implemented mixed-schema data augmentation and prepared a training run using 2.35B tokens and 938.7 GPU hours, then evaluated the checkpoint on held-out and familiar schemas.Human experts reviewed key decisions and controlled costly experiments and changes carried forward to subsequent iterations.Source: §6.2 · p. 16

6.3. Cyber Security

• IQuest-Q1 scored 84.5 on CyberGym, which evaluates reproducing and validating real-world vulnerabilities in large open-source codebases.The result provides evidence supporting vulnerability diagnosis and remediation.Source: §6.3 · p. 16
• During a three-day deployment, experts assisted by IQuest-Q1 produced 30 vulnerability reports, all independently validated by fellow security experts.The reports affecting open-source projects were responsibly disclosed to maintainers, and participating experts reported substantial support for understanding and validating security issues.Source: §6.3 · p. 16

Input modality

• IQuest-Q1 has no native image, audio, or video input capability.Its input is text-only.Source: Limitations · p. 16

Output reliability

• Generated explanations and code may be incorrect, so code changes should be reviewed or verified with task-appropriate tests.Source: Limitations · p. 16

Tool integration

• Structured tool execution and reasoning extraction require compatible serving parsers and an agent harness.Tool calls use the IQuest-specific format in the chat template.Source: Limitations · p. 16

Challenges in Real-World CLI Tasks

• In real-world CLI tasks, IQuest-Q1 may overlook constraints, repeat failed attempts, or leave issues unresolved, necessitating human review and supervision.These tasks often require iterative debugging and verification.Source: Limitations · p. 16

Ongoing development

• The authors state that IQuest-Q1 still has limitations in capability and reliability and that development will continue.They release model weights, deployment configurations, and agent-integration instructions to facilitate further community research and development.Source: Limitations · p. 16

Conclusion

• IQuest-Q1 is presented as an open-source agentic foundation model with broad CLI capabilities.Source: Conclusion · p. 16
• Its multi-stage training pipeline progressively develops general knowledge, long-context capabilities, and agentic CLI capabilities.Source: Conclusion · p. 16
• The conclusion reports capabilities on coding, general agentic tasks, and reasoning-heavy tasks, alongside participation in an internal iterative model-development loop.The loop is described as providing a foundation for continuing model development.Source: Conclusion · p. 16

• The author roster spans the names listed across the two author-list passages.Source: §7 · p. 17
• The author list distinguishes core contributors, senior advisory committee members, and the corresponding author.Asterisks mark core contributors, daggers mark the senior advisory committee, and a double dagger marks the corresponding author.Source: §7 · p. 17
• The marked roles identify contributors separately from the senior advisory committee and corresponding author.Source: §7 · p. 17

• The safe learning rate is the largest tested learning rate whose training-loss trajectory is classified as stable.Stable runs show consistently decreasing loss; warning runs rebound transiently before recovering, while collapsed runs remain at high loss after rebounding.Source: A. Pre-Training · p. 28
• The training rules apply an L^-1/2 output scaling factor to both attention and FFN branches, with branch-specific implementation.The attention factor is incorporated into output RMSNorm; the FFN factor is applied directly to branch output.Source: A. Pre-Training · p. 28
• The optimal learning rate transfers consistently across models differing by width and depth under the stated scaling rules.Figure 9 presents optimal learning-rate transfer across model widths and depths under μP scaling.Source: A. Pre-Training · p. 28, Figure 9 · p. 29
• Partial halo attention lets each query access preceding tokens across a shard boundary while overlapping halo exchange with partial attention computation.For the described setting, valid key positions satisfy i-H ≤ j ≤ i; execution dependencies are shown schematically.Source: A. Pre-Training · p. 28, Figure 10 · p. 30
• Recursive multi-token prediction feeds each draft step’s hidden representation and predicted-token embedding back into a shared MTP block.The first step uses the backbone’s final-layer representation; later steps reuse the preceding step’s output representation.Source: A. Pre-Training · p. 29, Figure 11 · p. 30