PaperlayerView exact revision on arXiv

Source-linked AI summary

K-Dense BYOK: An Open-Source AI Research Assistant That Runs Locally and Keeps a Hash-Chained Lab Notebook

Aubrey M. Brueckner, Darshil Patel, Yuhuan He, Timothy Kassis

arXiv:2610.00074v1cs.AI

TL;DR

Scientists choosing between managed research platforms and general coding agents face a trade-off between scientific scaffolding and control over models, data, and records. K-Dense BYOK combines local project work and scientific tools with an append-only notebook and an application-observed action log. In a twenty-prompt comparison, it led two managed platforms on scientific quality and research execution, while its provenance log still omits the software environment.

  • Problem

    Managed scientific platforms limit researcher control over models, data, work location, and records, while local coding agents lack scientific structure and research records.

    Source: §1 · p. 2

  • Method

    K-Dense BYOK combines scientific scaffolding and a linked, append-only notebook with an application-written provenance log that observes agent actions.

    Source: §1 · p. 3, §3.3 · p. 9

  • Results

    In a twenty-prompt comparison, K-Dense BYOK led two managed platforms on scientific quality and research execution, with execution advantages over Claude Science more than three times as large as its quality lead.

    Source: §5 · p. 13

  • Takeaways & Limitations

    The system leaves researchers a local folder of files and records, suited to analysis work where collaborators, reviewers, or regulators may later ask about the work.

    Source: §8 · p. 18

    The provenance log records actions and file contents but not software-library or Python versions or random seeds, so it does not yet capture the environment needed to reproduce results.

    Source: §6.1 · p. 14

Abstract

from arXiv · show

K-Dense BYOK (bring your own keys) is a free, open-source AI research assistant for scientists in any field that runs on the researcher's own computer. The researcher supplies access to a model of their choice, hosted or running locally, and the application supplies everything else: a place for the work to run, a layer of scientific scaffolding, and a complete record. Each project is an ordinary folder, so the data, the code, the results, and the record stay on a machine the researcher administers and can be read years later without the application. Three things separate it from a chat assistant or a general-purpose coding agent. It ships a library of written scientific procedures, guided workflow templates, catalogs of where research data can be found, and reviewer and writer roles the agent can hand work to. It keeps a Living Lab Notebook whose entries link into an argument and are added to but never erased. And it records what happened by watching what the agent does rather than by taking the agent's word for it, in a log the agent has no tool that can write to. That choice targets the most common failure, model overclaiming, in our earlier benchmark of nine frontier models, by making claims checkable rather than preventing them. On twenty interdisciplinary research prompts, scored under a rubric fixed in advance, K-Dense BYOK led two managed platforms on both scientific quality and research execution. Its deliverables were the only ones that recorded the software they ran in, and the only ones that usually arrived with a command that regenerates the results. One of the managed platforms ran the same frontier model and supplied neither. Those environment records were files the agent wrote, not part of the observed log, which does not yet capture the software environment itself. The code is available under the MIT license at https://github.com/K-Dense-AI/k-dense-byok.

1 Introduction

Managed scientific platforms offer scaffolding but limit researchers’ control, while local coding agents offer control without scientific structure. K-Dense BYOK adds scientific scaffolding and records both the agent’s notebook and its actions, then evaluates the system against two managed platforms.

1 Introduction

  • Managed platforms constrain control over models, data, work location, and records, while local coding agents lack scientific procedures, data catalogs, reviewer roles, and research records.Researchers face a trade-off between platform scaffolding and control over their work.Source: §1 · p. 2
  • 31.4% of assessments in K-Bench 01 identified overclaiming as the most common single failure, while scientific accuracy and delivered-file quality lagged writing in all nine models.The authors argue that claims need grounds, checks, and documentation beyond the model’s own account.Source: §1 · p. 2
  • K-Dense BYOK adds scientific procedures, workflow templates, research-data catalogs, specialist roles, an append-only linked notebook, and an application-observed log of agent actions.The log is separate from the notebook the agent writes, because an agent-authored record would not independently verify its actions.Source: §1 · p. 3
  • The paper describes the system and compares it with two managed scientific platforms on twenty research prompts, while noting that the evaluation was run by the authors.The paper also describes intended uses, limitations, and the evaluation protocol.Source: §1 · p. 3

2 Design commitments and architecture

K-Dense BYOK separates model access from the local workspace and record: researchers choose a hosted or local model, while the application runs work in ordinary project folders and observes agent actions. Its architecture supports concurrent work and remote computation, while labeling uncertainty in the record.

2.1 Four commitments

  • Researchers supply model access and choose a model for each chat; local models support tasks whose data must stay on the machine, while panel chats cannot read files or run code.Hosted access options include OpenRouter, NVIDIA, and existing subscriptions; local models can be served through Ollama or LM Studio.Source: §2.1 · pp. 3–4
  • Each project is an ordinary local folder containing work artifacts, so it can be opened, moved, archived, or read later without the application.The application hosts nothing, and the project folder includes results, conversation history, notebook entries, costs, and provenance.Source: §2.1 · p. 4
  • The application, not the agent, writes the provenance log by observing actions; where the recorder cannot establish a fact, it labels uncertainty or marks an exceeded limit.Notebook entries and the provenance log refer to shared files, whose status can be reported as current, stale, or unverified.Source: Figure 1 · p. 4, §2.1 · p. 5

2.2 Architecture

  • A local interface and background program divide presentation from execution; the agent can use file and command tools, specialists, and external tools via the Model Context Protocol.Model requests go directly from the agent to the researcher’s chosen provider under that provider’s privacy terms.Source: §2.2 · p. 5
  • Independent chat tabs share a project folder and support up to ten concurrent lines of work; remote Modal jobs can be tracked, cancelled, retried, and recovered after interruption.Requested result files are copied back only after all arrive, avoiding partial results after interrupted transfers.Source: §2.2 · p. 5
  • Python environments are set up inside working folders, while model-provider credentials stay in a protected file outside project folders.The scientific skills catalog is copied into a project on first start.Source: §2.2 · p. 5

3 Capabilities

K-Dense BYOK combines a scientific workspace and reusable research outputs with scientific scaffolding, observed provenance, and a linked notebook. It also supports local or hosted models, project cost controls, and use cases with different fit and limitations.

3 Capabilities

  • Kady asks clarifying questions, performs analyses, and leaves scripts, figures, tables, reports, notebook entries, and action records as ordinary files that researchers can inspect and rerun.No programming is required, and structured result cards link to underlying files but are not an independent check of the agent’s reports.Source: §3 · p. 6

3.1 Scientific scaffolding

  • Written scientific procedures, guided workflow templates, data-resource catalogs, and configurable specialist reviewers and writers provide reusable research scaffolding.Skills can be browsed, edited, disabled, or installed at a chosen version; specialists can work sequentially or in parallel.Source: §3.1 · p. 6

3.2 Provenance from observation

  • Each action is logged with its tool, inputs, time, run, model, and fingerprints of files read or written; uncertain file links are labeled rather than guessed.Links from opaque commands may be inferred when neighboring steps could account for a change, while model-asserted links remain declared; specialist files are fingerprinted when collected. Citations are flagged as stale when files have newer versions, and recorder limits mark affected steps.Source: §3.2 · pp. 7–8, Table 2 · p. 8

3.3 The Living Lab Notebook

  • The Living Lab Notebook links hypotheses, methods, observations, and decisions into an argument, while preserving corrections and the original entries.Notebook entries are written during work and can be linked to files tracked by provenance fingerprints, allowing stale citations to be caught; entries can be exported in several formats.Source: §3.3 · pp. 8–9

3.4 Cost tracking and control

  • Token usage and list-price value are recorded per run and project, while optional firm spending limits can reserve estimated remote-computation costs before execution.Researchers can inspect or edit code, redirect a run, or stop it.Source: §3.4 · p. 9

3.5 Getting started

  • Installation requires no programming and supports macOS, Linux, and Windows 10/11, with hosted model access or local models available.The application installs scientific packages and skills on first start; local models can keep work from being sent to outside companies, but small models may struggle with repeated tool use, and some named models have not been evaluated here.Source: §3.5 · pp. 9–10

3.6 Where it fits today

  • The beta best fits analysis with data in hand, repeatable reanalysis, literature-backed reviews, or manuscript drafting where a durable record matters.Very small local models are not yet capable of multi-step tool use, and built-in literature search and Methods export are not yet available.Source: §3.6 · p. 10

4 Evaluation against two managed platforms

Across twenty interdisciplinary prompts, K-Dense BYOK led on judged scientific quality and research execution, while its deliverables more often included reproducibility artifacts.

4 Evaluation against two managed platforms

  • All three platforms completed twenty prompts; K-Dense BYOK and Claude Science used Claude Opus 4.8, while Biomni Lab's model was undisclosed and may vary by prompt.The evaluation used one submission per prompt, without follow-up messages.Source: §4 · p. 10

4.1 What the deliverables contained

  • K-Dense BYOK supplied software-environment records on every prompt and regeneration commands on most; Biomni Lab did so rarely, and Claude Science supplied neither.The automated yes-or-no inventory checked seven standard items; Figure 2 reports five and does not establish scientific correctness.Source: §4.1 · pp. 10–11, Figure 2 · p. 11
  • The inventory and rubric assessed delivered bundles, not K-Dense BYOK's provenance log or notebook, so the evaluation does not measure that record.Environment records credited in the evaluation were files the agent produced in the bundle.Source: §4.1 · p. 11

4.2 Judged outcomes

  • The rubric allocated 75 points across nine scientific-quality criteria and 25 across four research-execution criteria covering regeneration, traceability, and auditability.The two outcomes were reported as percentages of available points.Source: §4.2 · p. 11
  • K-Dense BYOK led on both judged outcomes and ranked first on research execution in all twenty prompts, while neither comparator ranked first there.On scientific quality, each comparison platform ranked first on at least one prompt.Source: §4.2 · pp. 11–12, Figure 4 · p. 13

5 Discussion

The discussion argues that scientific-agent comparisons should consider deliverable usability and reproducibility, not only scientific quality, while distinguishing researcher control and checkability from scientific correctness.

5 Discussion

  • The execution gap centered on regeneration commands and software-environment records, and only the execution lead over the third platform held on every prompt.The authors argue that researchers should assess what arrives at the end of a session, not only how the answer reads.Source: §5 · p. 13
  • Claude Science used the same frontier model, so the observed deliverable gap concerns what the surrounding software handed researchers, not proof that no record was built.Anthropic documents that Claude Science keeps code and figure environments inside its application.Source: §5 · p. 13
  • Changing the file-and-command interface raised software-repair solve rate from 11.0% to 18.0%, a relative gain of 64%.The discussion notes that the effect of changing the software layer is not automatic.Source: §5 · p. 14
  • Existing scientific benchmarks test reproducing others’ work, whereas this study measures whether an agent’s own work can be reproduced.The authors frame this as a complementary property to existing reproduction benchmarks.Source: §5 · p. 14
  • The observation-built record does not yet capture the software environment or independently tell readers what is needed to rerun work; closing that gap is the key remaining task.The authors call the environment gap the most important remaining work.Source: §5 · p. 14
  • Researcher-selected models, locally administered files, readable records, and inspectable open layers make work checkable, but do not make its science correct.A managed platform may offer some of these properties while keeping the model, requests, and intervening software under vendor control.Source: §5 · p. 14

6 Limitations

The system’s own record is bounded by incomplete environment capture and limits of observing file effects, while supplied-model dependence and security boundaries constrain use. The evaluation is limited by confounding, judging and sampling choices, and the authors’ conflict of interest.

6.1 Limitations of the system

  • The provenance log omits software, interpreter, seed, application, and skill-library versions; observation can miss fast writes, remote work, nested handoffs, and command internals.Concurrent changes may be marked inferred, and rewrites preserving file size and modification time can go undetected.Source: §6.1 · pp. 14–15
  • Scaffolding depends on supplied-model reliability, and the study neither shows that individual skills improve scientific work nor isolates their contribution.Models may skip or misapply procedures; these effects are strongest with below-frontier and small local models.Source: §6.1 · p. 15
  • Running under the researcher’s account exposes saved credentials to agent actions, while third-party skills extend trust and several automated capabilities remain unavailable.Third-party skills are not automatically updated or audited, and the application cannot override a provider’s safety rules.Source: §6.1 · p. 15

6.2 Limitations of the evaluation

  • Platform bundles prevent clean model/software separation; no common-model comparison or same-model coding-agent control isolates platform effects or the scaffolding’s contribution.Holding the model roughly fixed against Claude Science strengthens attribution to surrounding software but does not make the study a controlled experiment.Source: §6.2 · p. 16
  • The judge was unblinded, ran once per bundle, and scored exported bundles rather than interfaces; twenty prompts and model-written prompt bias further bound interpretation.A record retained inside an application could therefore be absent from its exported deliverables.Source: §6.2 · p. 16
  • The evaluation was not independent because the authors develop K-Dense BYOK and are employed by K-Dense Inc.The results should be read as an internal evaluation under the tested setup, not a universal measure of platform performance.Source: §6.2 · p. 17

7 Methods

The methods evaluate three platforms on twenty interdisciplinary research prompts using a frozen rubric, fixed run conditions, and paired prompt-level comparisons with bootstrap confidence intervals.

7.1 Task set

  • Twenty loosely specified prompts span eleven scientific areas and require data acquisition, analysis, and decision-relevant outputs.The prompts were written with GPT-5.6 Sol and drawn from a broader internal benchmark.Source: §7.1 · p. 17

7.2 Rubric and judging protocol

  • A frozen rubric scores scientific quality from 0–75 and research execution from 0–25, with a secondary combined score out of 100.Evidence must appear in the submitted bundle or be verifiable from cited sources; described but undocumented work earns no credit.Source: §7.2 · p. 17

7.3 Platforms and required deliverables

  • K-Dense BYOK and Claude Science used Claude Opus 4.8, while Biomni Lab used its Max profile; each platform submitted one bundle per prompt without retries.Platforms were set to their highest reasoning option; preflight checks assessed required reports, scripts, fingerprints, environments, results, source receipts, and logs.Source: §7.3 · pp. 17–18

7.4 Statistical analysis

  • Platform means and paired mean differences were estimated for both co-primary outcomes, with 95% bootstrap confidence intervals resampling prompts.Intervals used 10,000 resamples of twenty prompts and a fixed seed of 2026; platform comparison considered both co-primary outcomes together.Source: §7.4 · p. 18

8 Conclusion

K-Dense BYOK leaves research files and records in an ordinary folder on the researcher’s computer, while its usefulness depends on the supplied model and its provenance log still lacks environment capture.

8 Conclusion

  • The assistant works on the researcher’s files and computer, leaving an openable folder containing data, code, figures, notebook conclusions, and an action record.The project folder can be read without the application.Source: §8 · p. 18
  • The scaffolding follows the model the researcher supplies, so a capable model matters; the record does not yet capture the software environment.The paper identifies environment capture as the largest reproducibility gap and the next one to close.Source: §8 · p. 18
  • The system is free, MIT-licensed, and in beta, and researchers are invited to try it, report problems, and contribute field-specific procedures.Skills and specialists are written in plain language and can be added by describing a procedure.Source: §8 · p. 18

Availability

The paper describes release 0.9.14 of K-Dense BYOK as MIT-licensed open-source software, with an automated test suite and a contribution path through its repository.

Availability

  • K-Dense BYOK is MIT-licensed open-source software; the version described is release 0.9.14, published on 2 September 2026.Questions, bug reports, and contributions are invited through the repository’s issue tracker.Source: Availability · pp. 18–19
  • The agent is built on the Pi open-source toolkit, with delegation and web access supplied by separate community packages.Its scientific skills come from the openly licensed Scientific Agent Skills library.Source: Availability · p. 19
  • An automated test suite runs on every change, and the repository issue tracker accepts questions, bug reports, and contributions.Source: Availability · p. 19

Data and code availability

The repository contains K-Dense BYOK’s open-source code, but not the study’s benchmark data, result bundles, or judge outputs. The prompts, rubric, and per-prompt scores appear in the manuscript appendices; frozen bundles and judge outputs are available from the authors on request.

Data and code availability

Use of K-Dense BYOK in preparing this manuscript

The authors used K-Dense BYOK to help search literature, work with evaluation results, and prepare the manuscript. They state that scientific claims, evaluation design, and interpretation remain their responsibility, and disclose that the paper was written with help from the system it discusses.

Use of K-Dense BYOK in preparing this manuscript

Competing interests

All authors work for K-Dense Inc., which develops the evaluated platform and sells a managed research platform. The authors disclose a commercial interest in both tool families and say independent replication is warranted.

Competing interests

A Additional evaluation figures

The appendix provides additional evaluation context, including prompt- and criterion-level figures, the benchmark prompts and frozen scoring instrument, and definitions of reproducibility and traceability criteria. It also specifies the controlled, paired-prompt judging protocol used to assess research execution.

A Additional evaluation figures

  • Table 4 lists scores for all 60 bundles; prompt-level figures show research-execution separation on nearly every prompt, while platforms trade places on scientific quality.The appendix says the figures and table use the same sixty bundles and judging pass as the Section 4 summaries.Source: A Additional evaluation figures · p. 26, Table 4 · p. 26, Figure 7 · p. 28
  • K-Dense BYOK separates from both comparison platforms mainly on research execution, leads all nine scientific-quality criteria modestly, and has its clearest criterion-level gaps on R1 and R3.The spread between platforms is wider on research execution than scientific quality.Source: Figure 5 · p. 27, Figure 6 · p. 27, Figure 8 · p. 29

B Benchmark prompts

  • The benchmark comprises 20 prompts presented verbatim to all platforms and deliberately underspecified to resemble real scientific requests.Source: B Benchmark prompts · p. 30

C.1 Structure and score construction

  • The frozen rubric scores each prompt–platform bundle on co-primary scientific quality (0–75) and research execution (0–25), with their sum reported as a secondary composite.It cautions against calling a platform generally superior from the composite alone when the two dimensions materially disagree.Source: §C.1 · p. 35

C.2 Criterion weights

  • Task fulfillment and methods each carry 13 points, while executable regeneration carries 8 and source lineage 7; these weights are prespecified priorities, not estimated constants.Source: §C.2 · pp. 35–36

C.3 Universal 0–4 anchors

  • The integer 0–4 anchors require affirmative evidence for a 4, and scoring relies only on frozen-bundle evidence or claims directly verifiable from cited sources.The prompt, rubric, checklists, bundles, manifests, judge settings, and evaluation order are frozen before scoring.Source: §C.3 · p. 36, §C.4 · p. 36

C.6 Scientific-quality criteria (A–I)

  • Scientific-quality criteria assess task fulfillment, data integrity, methods and validation, result reasoning, robustness, and decision quality and follow-up.Proposed-but-unexecuted methods do not earn execution credit unless the prompt asks for a plan.Source: §C.6 · p. 37
  • The remaining scientific-quality criteria assess narrative and structure, precision and epistemic honesty, and the usability of figures, tables, and deliverables.Communication is judged against evidence and presentation quality, not polish alone.Source: §C.6 · p. 37

C.7 Research-execution criteria (R1–R4)

  • Research execution is assessed for reproducible environments and traceable, reusable outputs.R3’s score of 4 requires a lockfile or container, runtime specification, parameters, seeds, and resource requirements supporting one-command setup; R4’s score of 4 requires a complete requirement-to-source chain, consistent structured outputs, and failure records.Source: §C.7 · p. 38

C.8 Judging protocol

  • The v3.0 protocol uses one fresh-context judge run per prompt and platform bundle, with equal prompt weighting and paired prompt-level comparisons.Platforms are evaluated separately under shared prompts, judge settings, evidence rules, access policies, environments, and resource limits; deviations are recorded as evaluator access failures.Source: §C.8 · p. 38

C.5 Standard result-bundle contract

The standard result-bundle contract requires functionally equivalent submissions with documented, verifiable artifacts for evaluating decision-critical workflows. A preflight validator inventories the artifacts, but their presence does not establish scientific correctness.

C.5 Standard result-bundle contract

  • Each platform must submit a functionally equivalent bundle with a README, manifest, executable workflow entry point, and environment specification.Exact filenames may differ; the README must name the main report and give a documented regeneration command, and the manifest must include cryptographic hashes.Source: §C.5 · p. 36
  • Bundles must include machine-readable decision-critical results, source receipts and transformation records for inputs, and execution logs of failures and interventions.Logs disclose failures, retries, substitutions, and manual interventions; results use stable identifiers and units.Source: §C.5 · p. 36
  • A preflight validator inventories required items, while missing items are scored under relevant criteria rather than imposing an automatic score cap.The validator must not infer scientific correctness from artifact presence.Source: §C.5 · p. 36
Loading 2610.00074v1…