Source-linked AI summary
Dr. Claw: An AI Scientist Workspace for Vibe Research
Dingjie Song, Hanrong Zhang, Dawei Liu, Yixin Liu, Zongxia Li, Zhengqing Yuan, Siqi Zhang, Henry Peng Zou, Zhiling Yan, Yuxuan Zhang, Yanfang Ye, Philip S. Yu, Lichao Sun
TL;DR
Dr. Claw addresses fragmented and weakly auditable end-to-end research workflows by wrapping existing coding-agent executors in a controllable human-in-the-loop workspace. It combines persistent state, reusable skills, and orchestration across planning, execution, and writing, then evaluates the wrapper against the same bare executor. Holding the executor fixed, Dr. Claw scores higher on research completeness while preserving an auditable, recoverable process trail.
Problem
Research decisions, intermediate artifacts, and cross-stage workflow state are often scattered or lost, while researchers switch among tools for orchestration and writing.
Method
Dr. Claw wraps an existing coding agent with task-graph orchestration, persistent research state, a reusable skill library, and human checkpoints across a traceable workflow.
Results
Holding the executor fixed, Dr. Claw scores higher on research completeness and uniquely persists an auditable, recoverable process trail.
Takeaways & Limitations
The workspace links human decisions to AI execution in one controllable, traceable, and recoverable research loop.
Takeaways & Limitations
The exploratory pilot uses limited tasks and participants, so differences are directional evidence rather than causal or statistically powered effects.
Abstract
from arXiv · showhide
Command-line coding agents (e.g., Claude Code, Gemini CLI) can already read and write files and sustain long sessions, yet end-to-end research still fragments across chat tools, IDEs, terminals, and writing environments, and the decisions that make it auditable are rarely preserved. We present Dr. Claw, an open-source workspace that wraps existing coding-agent executors in a controllable and auditable human-in-the-loop workflow rather than introducing another autonomous agent. Persistent state objects, a reusable skill library, and multi-executor coordination link human decisions to AI execution, turning planning, execution, and writing into one traceable, recoverable loop. We demonstrate Dr. Claw through an interactive three-view scenario and a failure-recovery walkthrough, and evaluate it against a bare command-line agent sharing the same backend executor, so the comparison contrasts the whole orchestration layer (task graph, state objects, and skill library) with the agent it wraps. Holding the executor fixed, Dr. Claw scores higher on research completeness while persisting an auditable, recoverable process trail. Demo access: repository https://github.com/OpenLAIR/dr-claw, released under AGPL-3.0 with GPL-3.0 upstream components.
1 Introduction
Dr. Claw addresses fragmented, weakly auditable research workflows by combining human control with AI execution in a traceable workspace. It wraps existing coding agents with orchestration, persistent state, skills, and evaluation evidence showing higher completeness and an auditable recovery trail.
- Research workflows remain fragmented across decomposition, scheduling, tracking, validation, and writing, weakening reproducibility and delivery reliability.The cited motivation identifies verification and context maintenance as major collaboration costs.
- Vibe Research assigns high-throughput execution to AI while humans retain research direction, evaluation criteria, trade-offs, and final acceptance.The paradigm emphasizes sustained human takeover rather than full autonomy.
- Dr. Claw unifies planning, execution, and writing into a controllable, traceable, recoverable, and auditable research loop around an existing executor.Users provide goals, constraints, and acceptance criteria while the system decomposes tasks, executes actions, records artifacts, and supports revise, retry, and handoff.
- Dr. Claw implements task-graph orchestration, a modular skill library with 58 stage-mapped skills across five stages, and multi-agent execution compatible with coding agents.The deployed catalogue contains 171 skills.
- Holding the executor fixed, Dr. Claw scores higher on completeness across tasks and uniquely preserves an auditable, recoverable process trail.The pilot was not statistically powered at one run per task.
2 Related Work
Related work increasingly emphasizes controllability, verification, and intervention, while Dr. Claw combines these concerns with built-in research state around an existing coding agent. Its distinct focus is long-horizon, auditable orchestration rather than autonomous agent construction alone.
- 2.1 Research Agents and End-to-End Automation: End-to-end research systems automate ideas-to-papers with minimal intervention but rarely prioritize sustained controllability in local engineering environments.Prior systems provide checkpointing, interrupts, replay, or intervention, but their emphases differ.
- 2.1 Research Agents and End-to-End Automation: Dr. Claw wraps an existing command-line coding agent rather than introducing a new executor.This distinguishes its orchestration contribution from systems centered on agent construction.
- 2.1 Research Agents and End-to-End Automation: Dr. Claw makes the research process a first-class object through a Task Graph, Artifact Store, Decision Log, and Execution Trace.These four persistent state objects support long-horizon workflow management.
- 2.2 Human–AI Collaboration and Context-Switching Costs: Dr. Claw differs by combining built-in research state, mid-run takeover, and sustained human control across ideation, experimentation, and publication.Table 1 defines built-in state as shipped research-artifact objects and mid-run takeover as intervention at arbitrary points.
- 2.2 Human–AI Collaboration and Context-Switching Costs: HCI evidence identifies interpretation, validation, correction, reading, confirmation, and revision as process burdens requiring clearer supervision.Interactive decomposition and process visualization improve monitoring and intervention in the cited work.
3 System Overview
Dr. Claw represents research as a continuous, traceable workflow that coordinates human decisions with AI execution. Four persistent state objects support iterative orchestration, inspection, and takeover rather than one-shot generation.
- Dr. Claw orchestrates a Task Graph, Artifact Store, Decision Log, and Execution Trace.Together, these objects convert interaction history into a reviewable process.
- The system maps high-level goals and constraints to executable tasks with continuous inspection and human takeover.Each task interaction is treated as a state transition.
Appendix B.2, and Appendix B.2.
Dr. Claw represents research as an iterative, stateful workflow that maps human goals and constraints into executable tasks. Its layered architecture, reusable skills, executor abstraction, and policy controls support traceable and recoverable orchestration.
- Stateful orchestration: Four persistent state objects—Task Graph, Artifact Store, Decision Log, and Execution Trace—turn interaction history into a reviewable process with continuous inspection and takeover.The workflow updates state iteratively rather than producing one-shot responses.
- Layered architecture: The system uses Interaction, Orchestration, and Execution layers to connect unified workspace interactions, lifecycle management, and heterogeneous backend execution.Backend switching preserves workflow semantics and unified state and audit views.
- Workflow loop: Plan–Execute–Verify–Write-back iteratively converts a research idea into a dependency-aware task plan, writes outputs to the Artifact Store, and records interventions in the Decision Log and Execution Trace.Task states are updated in the Task Graph, with explicit human checkpoints supporting long-horizon iteration.
- Skill-based capabilities: The skill library contains 58 stage-mapped skills across five research stages and 171 skills in the deployed catalogue, with versioning and schema checks before activation.Skills can be suggested by stage and task type, auto-loaded from keywords, or invoked manually.
- Safety and controllability: Fine-grained tool and command policies restrict execution to actions in the executable action space defined by the current policy.This aligns execution capability with safety boundaries.
4 Demo Scenario
The demo presents Dr. Claw as a human-in-the-loop workflow for transforming high-level research intent into executable, trackable steps. Three coordinated views support capability discovery, orchestration and approval, and synchronized execution monitoring.
- Scenario goal: The scenario examines whether high-level research intent can become executable workflows while preserving controllability, recoverability, and auditability.It focuses on workflow orchestration quality rather than one-shot model output.
- Orchestration view: The center orchestration view combines goals, constraints, and acceptance criteria with decomposition, dispatch, state feedback, and human approval.The user acts as research lead while Dr. Claw handles decomposition, dispatch, and state feedback.
- Three-view loop: The three-view loop links capability discovery, goal and intent processing, explicit human decisions, execution progress, and synchronized task cards.The Skills Dashboard is on the left, the orchestration interface in the center, and the Task List on the right.
- Capability discovery: Users browse skills by research stage, filter them by theme, and manually add skills for explicit invocation in later tasks.The sub-scenario tests whether selected skills can be converted into executable steps in context.
- Execution control: The Task List shows total, done, in-progress, and pending counts, a progress bar, stage-grouped task cards, linked skill tags, and next-step actions.Each task card exposes a task ID, objective, and linked skill tags.
5 Evaluation
Using the same backend executor, Dr. Claw evaluates whether orchestration improves open-ended research completeness while adding an auditable, recoverable process trail. The evaluation combines automated task comparisons, a failure-recovery demonstration, and a retrospective human study.
- Evaluation design: The comparison holds the backend executor fixed, measuring Dr. Claw’s task graph, artifact store, decision log, execution trace, and skill library as one orchestration layer.The automated comparison uses open-ended, unenumerated research instructions and scores 21 best-practice elements deterministically from produced files.
- Research completeness: 0.952 versus 0.873 pooled completion: Dr. Claw wins two of three tasks and ties the third against the bare agent.Both conditions achieved 1.00 on modeling elements including multiple models, cross-validation, calibration, ablation, and statistical rigor.
- Research completeness: Research-hygiene pass rates increased for limitations sections from 0.33→1.00, subgroup analysis from 0.33→1.00, and real literature citations from 0.00→0.67.The advantage is attributed to reference-audit, analysis, and paper-writing skills; the reference audit did not fire on the tied clinical-note task.
- Auditability and recovery: Every Dr. Claw run persisted a queryable process model, while the bare agent persisted none by design.Dr. Claw recorded a task graph, execution trace, decision-log brief, named stages, and claim→evidence maps; the paper treats this as an architectural affordance rather than a performance score.
- Caveats: The pooled completion difference was +0.079 with a 95% bootstrap CI of [-0.00, +0.14], so the exploratory one-replicate-per-task result was not yet significant.Dr. Claw was at least as complete as the bare agent on all three tasks but remained slower because orchestration records state.
- Auditability and recovery: The failure walkthrough recovered accuracy 0.892 after an induced wrong-path error while retaining all 5 pre-existing files across 23 tool events.The workspace halted on the error, preserved prior pipeline state, and added corrected artifacts without wiping the failed attempt; no matched bare-agent recovery run was provided.
- Human study: In a retrospective study of seven AI PhD researchers, Dr. Claw was associated with shorter completion-time bands, highest output-quality ratings, fewer tool switches, and higher experience scores.The effect was strongest and fully pairwise-significant for experience across Ideation, Experiment, and Publication stages.
Limitations
The evaluation is a small-scale exploratory demonstration with several scope and measurement boundaries. It provides directional workflow evidence rather than causal or statistically powered conclusions.
- The pilot uses limited tasks and participants, so its differences are directional evidence rather than causal or statistically powered effects.
- Dr. Claw’s task graph, persistent state objects, skill library, and workflow instructions are evaluated as one bundle, preventing attribution to any single component.
- Completeness counts the presence of 21 expected research components, not their correctness; scientific soundness requires expert per-artifact review.
- The control is the wrapped bare executor rather than a state-of-the-art orchestration framework, and transfer beyond medical-domain tasks remains unshown.
- For high-risk domains such as healthcare, system outputs must not be used directly for real-world clinical decisions.
A Human-Study Evaluation (Retrospective Three-Condition Study)
The retrospective study compares Dr. Claw with no AI tools and general-purpose AI assistants across three research stages. Dr. Claw is associated with shorter completion times, higher output scores, fewer switches, and higher experience scores.
- Seven AI PhD researchers were compared across Ideation, Experiment, and Publication using completion time, output quality, switching count, and experience measures.
- Across all stages, Dr. Claw is associated with shorter completion-time bands and the highest stage output scores.
- Across all stages, Dr. Claw is associated with lower switching bands and higher experience scores than the comparison conditions.
- The omnibus signal is strongest and fully pairwise-significant for Experience, while Time Band is significant in Stages 2–3.
A.1 Participants and Data Collection
The study recruited seven experienced AI PhD researchers and collected hybrid live-log and retrospective data across three defined research stages. Stage outputs were blind-rated with stage-specific rubrics, and exploratory nonparametric tests were applied.
- Participants and Data Collection: Seven AI PhD participants with experience in high-performance AI, medical AI, or large language models took part after a three-week Dr. Claw familiarization period.
- Data Collection: Dr. Claw used live logs, whereas No-AI and Web/Desktop-AI controls supplied retrospective normalized time, switching, and experience ranges.
- Research Stages: The study covered Ideation, Experiment, and Publication, including problem framing, experiment analysis, drafting, and final polishing.
- Outcome Measurement: Stage outputs were blind-rated from 1–5 using rubrics tailored to novelty, analysis, technical accuracy, readability, and reproducibility.
- Statistical Procedure: With retrospective data included, the analysis used Friedman tests and Holm-corrected pairwise Wilcoxon tests as exploratory statistics.
A.3 Detailed Statistical Results
Stage-wise tests used seven matched participants and reported omnibus and Holm-corrected pairwise comparisons. Time, switching, performance, and experience showed varying stage-level statistical signals, but the study has important validity threats.
- Time Band: Time Band: χ2(2) = 8.96, 13.56, 14.00 with p = 0.0114, 0.0011, <0.001 across Stages 1–3.
- Switching Band: Switching Band: χ2(2) = 1.50, 11.57, 10.75 with p = 0.4724, 0.0031, 0.0046 across Stages 1–3.
- Validity Threats: Validity threats include limited sample size, familiarity differences, retrospective recall bias, exclusion of model runtime in Stage 2, and subjective experience scores.
- Process State: The write-back equation unions newly added or revised artifacts with the prior artifact state while retaining dependency-status histories in the Execution Trace.
B.2 Implementation, Recovery, and Reproducibility
Dr. Claw implements a staged research workspace with persistent pipeline state, task-specific skills, backend adapters, and policy-controlled execution. It also provides layered failure recovery and a reproducibility procedure that archives state and artifacts under fixed environment conditions.
- Implementation: Project initialization creates fixed Ideation, Experiment, and Publication folders alongside persistent state for configuration, the research brief, and tasks.
- Implementation: Task nodes use normalized fields, while a resolver combines stage-base and task-type skills, reaching 58 skills across five stages.The passage lists 11 survey, 14 ideation, 18 experiment, 22 publication, and 3 promotion skills.
- Implementation: Backend adapters support Claude and Codex SDKs with Cursor hooks, while explicit policies constrain tools, permissions, sandboxing, and approvals.
- Implementation: Pipeline mutations are persisted before WebSocket broadcasts, so the interface synchronizes against one shared source of truth.
- Recovery: Recovery covers pipeline/file errors, permission denials and timeouts, and abortable sessions with structured errors and consistent lifecycle states.Initialization can recreate pipeline-state defaults, and recovery uses non-destructive task mutation APIs.
- Reproducibility: Reproducibility requires fixed repository, lockfile, runtime, backend, model, and permission settings, with metadata, state files, and stage artifacts archived together.Setup uses Node.js LTS, with v22 recommended, and a standard install-and-run sequence.