PaperlayerView exact revision on arXiv

Source-linked AI summary

AIPOCH Open-Science: An Open-Source AI Research Workbench with Reproducible .science Packages

Wen Ouyang, Bocheng Huang, Qianyu Yao, Fei Sun, Minying Deng, Haoyue Cheng, Wenwen Zhou, Lei Mou, Yusi Wei, Jiang Li, Kun Zhu, Lei Wang, Yi Feng, Jiaheng Dai, Ke Qian, Dan Wang, Yahui Wang, Chenyi Wang, Mingsheng Lyu, Yaqiang Zhang, Ping Zheng, Chunhua Tang, Xinge Ma, Xu Zhang, Youjia Fan, Can Lu, Zhijun Dai, Man Hu, Qiqi Xie, Beinan Liu, Yuanning Guo, Yunze Liu, Jinhong Jiang, Xianjun Fu, Mingwei Zhang, Yuanji Xu, Shilin Xia, Bowen Yang, Mingjun Yu, Jingdun Xie, Xuwei Chen, Xia Zou, Yuzhou Xue, Jianguo Zhou, Chunhao Cao, Donghao Tian, Lei Huang, Jinxiao Li, Shengrong Long, Zefeng Song, Honglin Li, Jiang Luo, Wenying Jiang, Shanshan Li, Lei Dai, Ning Jiang, Ying Cai, Chaohui Li, Chunyan Peng, Hetong Zhao, Angqing Li, Chunyan Yang, Chongyang Sun, Yong Li, Xinyu Shi, Wei Dai, Ming Peng, Chen Sang, Yuanyuan Han, Desong Qiu, Haixin Miao, Jian Xu, Yi Chen, Xiaoliang Shi, Tian Shen, Teng Hu, Shuhao Wang, Yingyong Hou, Huimei Wang

arXiv:2610.17110v1cs.SEcs.AI

TL;DR

Researchers need to know how instructions, computations, artifact revisions, and checks relate when AI-assisted work is shared. AIPOCH Open-Science implements a .science exchange contract and tests its preservation rules; the controlled exchange retained tested associations, while a fixed-input registry rerun reproduced two saved tables.

  • Problem

    Recipients need to distinguish unexecuted plans, assessments of earlier artifact versions, and retained identities whose payloads may be withheld.

    Source: §1 · pp. 5–6

  • Method

    AIPOCH Open-Science packages persistent session records and selected contents, preserving attributable version and assessment targets while allowing mapped identifiers and explicit eligible-content omissions.

    Source: §1 · p. 6

  • Results

    138 specified native record-field comparisons matched the independent source reference in both complete and partial exports, and each receiver matched on 136 mapped record-field comparisons and ten snapshot-semantic comparisons.

    Source: §4.4 · p. 17

  • Takeaways & Limitations

    The tested operations provide design rules and a concrete validation procedure for exchanging recorded research work, within the scope of the tested records and operations.

    Source: §6 · p. 20

    The study used one controlled trajectory and two same-installation receiving projects; source capture completeness remains unmeasured, and the registry rerun did not replay the complete agent process.

    Source: §5.3 · p. 20

Abstract

from arXiv · show

AI-assisted research leaves instructions, computations, artifact revisions and checks whose relationships matter when another researcher receives the work. We present AIPOCH Open-Science, an open-source research workbench with portable .science session packages, and specify an exchange contract for these recorded relationships. Version and assessment targets remain attributable while receiving-workspace identifiers can be remapped, eligible contents can be omitted explicitly, and imported history remains separate from a writable fork. A controlled session tested an incorrect first version, a reviewed correction, an unreviewed revision and a planned but unexecuted step. We compared complete and partial exports with source records extracted independently of the exporter, then inspected imports into separate projects within the same installation. Both conditions retained the tested version and review associations, kept the unreviewed revision distinct from earlier assessments and represented excluded content as unavailable. A documentation-rich ordinary directory also supplied evidence for all seven static inspection questions. A separate registry case connected a table to its recorded task, execution and review; offline execution of its retained script and fixed input reproduced two saved CSV tables byte for byte. The study contributes an implemented exchange contract and a controlled examination of its preservation rules, content-selection constraints and interpretation boundaries.

1 Introduction

AI-assisted research produces instructions, computations, revisions, and checks whose relationships a recipient needs to interpret. AIPOCH Open-Science addresses this exchange problem with a contract for preserving recorded relationships while allowing specified changes during transfer.

1 Introduction

  • A session recipient needs to identify which task, activity, artifact version, and check belong together, and which underlying materials remain available.The paper frames the research session as an important object of sharing because results emerge through instructions, computations, and decisions.Source: §1 · p. 5
  • Prior agent and reproducibility studies motivate retaining production and interpretation context, including material availability, prompt framing, runtime conditions, and methodological choices [1] [2] [4].The many-analysts study found that changed verdicts did not necessarily accompany comparable changes in numerical estimates [2].Source: §1 · p. 5
  • Three exchange conflicts concern unexecuted plans, assessments tied to earlier versions, and artifact identities whose payloads may be withheld.The paper distinguishes these interpretation questions from the task score of the model that generated the work.Source: §1 · p. 6
  • The exchange contract preserves attributable version and assessment targets across local identifier mapping, makes eligible payload omissions explicit, and separates imported history from writable continuation.Relationship-dependent capture and content-selection guards implement these rules.Source: §1 · p. 6
  • A controlled trajectory and a separate registry case examine preservation, interpretation limits, and fixed-input re-execution of recorded research work.The trajectory includes an incorrect output, a reviewed correction, an unreviewed revision, an unexecuted plan, and selective sharing.Source: §1 · p. 6

2 Related work

Related work spans scientific coding agents, provenance systems, and agent-record frameworks. This paper situates its contribution at the exchange boundary for versioned workbench records and selected contents.

2.1 Scientific execution with coding agents and research workbenches

  • Scientific agents and workbenches support computational tasks and retain materials or execution history, while this study examines attribution across revisions and content selection [1] [2] [18].OpenAI4S is a close implemented precedent for persistent scientific execution, versioned artifacts, environment records, and read-only session import [11]; the systems are not experimentally compared here.Source: §2.1 · pp. 6–7

2.2 Provenance representations and partial disclosure

  • Provenance systems and RO-Crate establish foundations for plans, executions, graph consistency, and packaging [7] [8]; .science pairs native workbench records with RO-Crate metadata [7] [8].Selected records stay with required evidence, while eligible immutable payloads can be omitted explicitly; some exclusions are refused when bytes also occur in evidence that cannot safely be separated.Source: §2.2 · p. 7

2.3 Agent records, artifacts and evaluation evidence

  • Agent-provenance frameworks bind artifacts to interactions, claims, evaluation, and verification [12] [13] [14] [15], while .science retains a workbench session without requiring a complete publication argument [21].Inspection depends on locatable relationships; computational reuse depends on retained inputs and runtime conditions.Source: §2.3 · p. 7

3 Methods and implementation

The methods map researcher-directed sessions to typed records and relations, then implement capture and selection rules for exchanging those records and eligible contents.

3.1 Research setting and evaluation questions

  • The evaluation separates intended events, captured source records, selected export records, and receiver representations, assessing source capture, exchange preservation, and re-execution separately.Its questions concern how to retain task context, activities, revisions, and checks, and how their attribution survives exchange and withheld contents.Source: §3.1 · p. 8

3.2 The source workbench and why it maintains these records

  • The workbench records versioned artifacts with producer, task, and execution context, while review findings and scope snapshots preserve which version was assessed.Input associations retain their reported strength: turn attachment, resolver access, and validated or inferred producer links are distinguished.Source: §3.2 · p. 8

3.3 Relationship-dependent record capture

  • An explicit native-table allowlist and relationship traversal capture selected records with required upstream versions, findings, owning reviews, dispositions, and scoped artifacts.The bounded traversal fails when required source evidence is missing and does not automatically copy unrelated conversations.Source: §3.3 · p. 9

3.4 Selection of contents and explicit availability

  • Eligible payloads may be omitted while metadata and integrity information remain, but exclusions must match a known storage key, filename, and size.The manifest records excluded content explicitly; conflicting retained copies can make an omission unsafe and cause the system to refuse a smaller package.Source: §3.4 · p. 9

3.5 Package representation and RO-Crate mapping

The package combines native session records and retained contents with RO-Crate metadata, then imports history through mapped receiving identities. Supporting evidence covers implementation correspondence, function probes, and a registry-case rerun.

3.5 Package representation and RO-Crate mapping

  • The .science package retains native session and record structures alongside RO-Crate descriptions of retained entities, files, and recorded activities.The format includes a manifest, session and record files, RO-Crate metadata, a README, and retained objects with integrity checks.Source: §3.5 · p. 10

3.6 Export consistency and receiving-workspace control

  • Implementation checks revalidate source state before export and inspect package paths, compatibility declarations, retained contents, and native-record constraints.The freeze compares examined modules with the installed backend source map without attesting the entire application build.Source: §3.6 · p. 10, §3.7 · p. 12
  • Import remaps receiving-workspace identities while preserving reference targets; imported history remains read-only, with later work in a separate writable context.Figure 1 schematizes these distinctions and is a design schematic, not an additional experiment.Source: §3.6 · p. 10, Figure 1 · p. 11

3.7 Implementation freeze and supporting evidence

  • Nine production-function assertions passed for review projection, file-selection validation, retained-copy conflicts, and selection preview.The probes used synthetic records at a separately pinned source revision.Source: §3.7 · p. 12
  • A retained registry script run offline with fixed input reproduced two saved CSV tables byte for byte.The calculation used saved registry data; it did not import the package into a fresh application or replay the original agent session.Source: §3.7 · p. 12

3.8 Controlled paired validation and the exchange contract

The controlled validation compared complete and partial exports against independently extracted source records, then checked imports and their preserved relationships. It also specifies which identities and assessments must remain attributable and how omissions and writable continuations are represented.

3.8 Controlled paired validation and the exchange contract

  • The tested trajectory included an incorrect first output, a reviewed correction, an unreviewed revision and a planned but unexecuted permutation test.Independent source snapshots captured key states; the partial export excluded an unused note, while an attempted exclusion of the computational CSV was blocked by its required-evidence control.Source: §3.8 · p. 12
  • Source-to-export and source-to-receiver checks compared selected recorded fields and relations, rejecting missing referenced objects or mismatched expected targets.Review checks allowed rewritten paths and serialization differences; the auditors were also tested against deliberately altered copies.Source: §3.8 · p. 13
  • Application observations found that imported history remained read-only and a fork created a distinct editable session, but no independent recipient participated.The retained audit materials lacked a complete UI recording and separate post-fork copies of both source and imported files.Source: §3.8 · p. 13
  • The exchange contract preserves version identity, revision direction and assessment targets across local identifier mapping, while keeping eligible omissions explicit and imported history separate from writable continuation.A historical failed finding can remain associated with its assessed version while a newer version remains unreviewed.Source: §3.8 · p. 13

4 Results

The controlled results preserve version-specific assessment relationships across exchange, while keeping explicit which contents are unavailable and which history can be continued in a separate context. A registry case also links a table to its recorded task, computation and review, and reproduces two CSV outputs from retained inputs.

4.1 The implemented package carries relational records as well as files

  • Source inspection identified native fields for artifact derivation, input association, task and run references, finding-to-version binding, dispositions and review-scope snapshots.The controlled trajectory exercised populated version, review and snapshot relations; file selection acted on eligible payloads under record-retention rules.Source: §4.1 · pp. 14–15

4.2 The controlled source separated revision and assessment events

  • The failed finding remained bound to the initial summary version, its disposition linked correction to the reviewed version, and no matching scope assessed the final revision.The permutation test remained a recorded plan, with no implementation in the retained Notebook cells.Source: §4.2 · p. 15, Figure 2 · p. 16

4.3 A public registry case connects a table with task, execution and review

  • The registry package contained ten artifact versions, two retained message snapshots, two reviews, eight findings and two review-scope snapshots.The case had zero native ArtifactVersionInput rows, so observed input use did not establish an exhaustive dependency graph.Source: §4.3 · p. 15
  • The imported table’s version record linked it to a task message, completed Notebook run and scope-validated review finding.Offline execution of the retained calculation regenerated trial_calculations.csv and availability_summary.csv byte for byte from the saved script and normalized source.Source: §4.3 · p. 15

4.4 Paired source-to-import validation

  • Both receiving projects retained tested-version and review associations; the partial import omitted the unused note’s bytes and recorded that exclusion while retaining its version record.The computational CSV’s required-evidence safeguard was an interface observation.Source: Figure 2 · p. 16, §4.4 · p. 17
  • 138 specified native record-field comparisons matched the independent source reference in both export conditions; each receiver also matched 136 mapped fields and ten snapshot-semantic checks.The exports passed 52 and 50 inventory size-and-digest checks, respectively; these are fixed comparisons within one source trajectory, not estimates of capture completeness.Source: §4.4 · p. 17, Table 2 · p. 17
  • Imported history remained read-only, while “Fork to continue” created an editable session with a distinct saved identity and left source and imported history unchanged.No new analysis was requested in the fork.Source: Figure 2 · p. 16, §4.4 · p. 17

4.5 Static evidence availability across two delivery representations

  • All seven static inspection questions had locatable evidence in both the .science export and the documentation-rich ordinary directory.The ordinary directory did not undergo a corresponding application import or fork, and the audit had no independent recipient answers or recovery-time measurement.Source: §4.5 · p. 18

5 Discussion

The discussion frames exchange as preserving version-specific judgments and making selective omissions explicit, while distinguishing what the records establish from what they leave unresolved. It locates the implemented contract within existing provenance infrastructure and identifies boundaries for capture, interpretation, and future handoff.

5.1 Design lessons from versioned research-session exchange

  • An earlier failed assessment, a corrective assessment, and no assessment of a new revision can coexist when historical judgments retain their targets and each requested version is evaluated separately.The implementation uses native version references, review scopes, and finding dispositions to preserve that separation.Source: §5.1 · p. 19
  • Eligible omissions should be represented explicitly, but selections that conflict with inseparable retained evidence should be refused; downstream behavior with missing computational inputs remains untested.An unused note retained its version record without its bytes, whereas a required-evidence control protected the computational CSV.Source: §5.1 · p. 19

5.2 Contribution within existing provenance and agent infrastructure

  • The contribution is an implemented contract for preserving workbench-managed version and review relations across local identity mapping and selected content omissions, alongside a paired source, export, and receiver examination.The discussion distinguishes this contribution from existing provenance and agent infrastructure.Source: §5.2 · p. 19
  • Adequate documentation in an ordinary directory supplied evidence for all seven static inspection questions, while exchange behavior was supported only in the tested installation and user understanding or efficiency was not measured.The directory control and application exchange test address different questions.Source: §5.2 · p. 19

5.3 Applicability, capture limits and observation boundaries

  • A completed review’s current scope is recomputed against frame or branch identities and block sequence, with source identifiers and content hashes used to mark differences stale.If recomputation is unavailable, verification is unavailable rather than proof that content changed; the cause of observed outdated labels remains undetermined.Source: §5.3 · pp. 19–20
  • The exchange test covered one controlled trajectory and two projects in the same installation; input-capture completeness was unmeasured, and the fixed-input rerun did not replay the full agent process.The fork established a writable context, but scientific continuation was not tested; several identified safeguards also lacked failure-injection tests.Source: §5.3 · p. 20

5.4 Future research handoff

  • A planned human-readable account would link explanations of context, steps, inputs, revisions, review scope, omissions, and continuation conditions to supporting records while distinguishing facts from interpretation.The separately prepared public READMEs are outside the export workflow and do not constitute this planned account.Source: §5.4 · p. 20

6 Conclusion

AIPOCH Open-Science implements an exchange contract that preserves recorded version and assessment relationships while allowing local identifier mapping and explicit omission of eligible contents. The tested operations retained those associations and reproduced two fixed-input tables byte for byte, within the scope of the examined records and operations.

6 Conclusion

  • The controlled exchange retained tested version and review associations, distinguished an unreviewed revision from earlier assessments, and represented omitted eligible contents as unavailable.A separate writable fork was produced; the case16 fixed-input run reproduced two saved tables byte for byte.Source: §6 · p. 20

Data and code availability

The paper identifies public access points for the software, research cases, and validation materials. The validation release documents its contents, transformations, and evidence boundaries.

Data and code availability

  • AIPOCH Open-Science is available under the Apache License 2.0, with an official website and a public research-case collection.The evaluated case16 snapshot is identified by separate integrity records for the original export and its de-identified publication copy.Source: Data and code availability · p. 21
  • A versioned, de-identified validation release includes protocols, source snapshots, packages, import receipts, comparisons, audits, probes, and the case16 publication package and rerun.Its README maps files to manuscript results and documents runtimes, commands, access, reuse conditions, and evidence boundaries; transformation records distinguish public copies from preserved original evidence.Source: Data and code availability · p. 21

Declaration of AI assistance

Codex assisted with literature retrieval, code inspection, application operation, evidence organization, manuscript revision, validation code, and offline re-execution. The authors report responsibility for checking sources, interpreting findings, and reviewing the submitted materials.

Declaration of AI assistance

  • Codex assisted with literature retrieval, source-code inspection, application operation, evidence organization, manuscript revision, validation code, and offline re-execution.The authors state that they checked sources, interpreted findings, and reviewed the submitted text and materials.Source: Declaration of AI assistance · p. 21

Supplementary Information

Figure S1 links a registry table to retained task, execution and review evidence, and reports byte-for-byte reproduction of two saved CSV tables from fixed inputs.

Supplementary Information

  • Offline execution of analysis.py against registry_source.json under Python 3.12.14 reproduced the two saved CSV tables byte for byte.The retained calculation uses fixed inputs rather than a fresh registry query or model-performance evaluation.Source: Figure S1 · p. 23
  • trial_calculations.csv is associated with a task message, completed producing Notebook activity, message snapshot and scope-validated finding whose parent review includes the selected version.RO-Crate metadata also maps a file object to that version.Source: Figure S1 · p. 23
  • The package contains no native ArtifactVersionInput rows, and Figure S1 arrows gather associations without asserting an exhaustive causal graph.Source: Figure S1 · p. 23
  • The retained calculation considers 93 candidates, excludes 13 and includes 80.All 100 inventory entries passed size and checksum checks.Source: Figure S1 · p. 23
Loading 2610.17110v1…