PaperlayerView exact revision on arXiv

Source-linked AI summary

Discover, Falsify, Revise: Auditing Input-Use Claims from Source Code to Predictive Contribution in Agent-Discovered Cell Models

Mengran Li, Bo Li, Chengyang Zhang, Yang Yan, Jinfeng Xu, Zhenchao Tang

arXiv:2609.27234v1cs.LGq-bio.QM

TL;DR

Predictive performance alone does not establish whether a model uses its supplied perturbation information, a gap that matters in agentic model discovery. CELLAUDIT audits input-use claims from source code through fitted dependence to predictive contribution. Its evaluations find that prediction and input-use support can diverge, including across independent-acquisition refits.

  • Problem

    Held-out predictive performance alone does not establish whether a model uses supplied perturbation information, a gap relevant to score-driven agentic model discovery.

    Source: §1 · p. 1

  • Method

    CELLAUDIT checks whether registered inputs enter cited computations, affect fitted predictions, and improve prediction under matched replacement tests.

    Source: §1 · p. 2, §4.2 · p. 4

  • Results

    Audits distinguish predictive gains from input-use support: sci-Plex differences were suggestive with paired intervals spanning zero, and independent-acquisition refits retained dose but not compound-identity support.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Predictive success and claimed input use should be evaluated separately, since source consumption, fitted dependence, and target-relevant contribution can diverge.

    Source: §7 · p. 9

    Conclusions depend on the fitted model, evaluation setting, and registered replacement distribution; observed-support replacements do not establish biological exchangeability, mechanism, or causal effects.

    Source: §7 · p. 9

Abstract

from arXiv · show

AI virtual cells aim to predict cellular responses to specified interventions, yet held-out predictive performance alone does not establish use of the supplied perturbation information. This prediction-claim gap matters in agentic model discovery, where language-model agents generate and revise predictors using score-based feedback. We introduce CELLAUDIT, which audits input-use claims by asking whether an input can enter the cited computation, whether fitted predictions depend on it, and whether that dependence improves prediction of observed response. On a paired morphology-transcriptomics perturbation benchmark (BBBC047), an agent-selected predictor attains a mean held-out Global Pearson correlation coefficient (PCC) of 0.3153 but remains invariant to compound replacement; a control-profile-only predictor reaches 0.3142. Source inspection identifies a compound-query pathway blocked by singleton key-value attention, and the invariance persists after refitting with disjoint control wells. In a stratified audit of 48 candidates across two linked tasks, 47 change predictions under compound replacement on both held-out folds, but only 20 show target-loss gains with intervals above zero on both folds. On BBBC047, falsification-guided revisions recover positive mean compound contributions while retaining gains over the control-profile-only baseline. In matched sci-Plex searches, audit-enriched feedback yields higher held-out performance and larger mean compound and dose contributions across five trajectories, although paired intervals span zero. Refitting fixed designs on an independently acquired cohort shows predictive generalization need not imply generalization of input-use claims: dose contribution persists, whereas support for compound identity does not. CELLAUDIT adds a falsification layer to agentic model discovery, moving from generate-score-revise toward discover-falsify-revise.

1 INTRODUCTION

Predictive performance alone does not establish whether a model uses its registered perturbation inputs, a concern for agentic discovery driven by score-based feedback. CELLAUDIT tests input use from source code through fitted dependence to predictive contribution, and the paper shows these can diverge across benchmarks and refits.

1 INTRODUCTION

  • CELLAUDIT links registered input-use claims to source inspection, matched input replacement on frozen checkpoints, and tests of target-relevant predictive contribution.The three questions are whether an input can enter the cited computation, whether fitted predictions depend on it, and whether that dependence improves prediction.Source: §1 · p. 2
  • On BBBC047, the agent-selected predictor reached mean held-out Global PCC 0.3153 versus 0.3142 for a control-profile-only predictor, yet remained compound-invariant.A singleton key–value attention pathway blocked compound information through its query, and invariance persisted after refitting with disjoint control wells.Source: §1 · p. 2
  • Across 48 candidates, 47 changed predictions under compound replacement on both held-out folds, but only 20 had target-loss scaffold intervals above zero on both folds.This separates fitted dependence from evidence that dependence improves prediction.Source: §1 · p. 2
  • On BBBC047, falsification-guided revisions recovered positive compound contributions with predictive gains; audit-enriched sci-Plex feedback improved mean performance and contributions, but paired intervals spanned zero.The sci-Plex comparison covered five trajectories, so evidence for improved discovery remained suggestive rather than conclusive.Source: §1 · p. 2
  • On independently acquired LKCP, predictive gains and dose contribution persisted, whereas support for compound-identity contribution changed substantially.A separate CRISPRa evaluation applied the same auditing principle to unseen genetic combinations.Source: §1 · p. 2

2 RELATED WORK

Prior work studies perturbation prediction and evaluation, scientific model discovery, and tests of fitted-model behavior. CELLAUDIT starts from a registered input-use claim and traces it through computation and model behavior.

2 RELATED WORK

  • Perturbation-prediction research evaluates how well chemical or genetic responses can be predicted, while CELLAUDIT asks whether registered input-use claims are supported by source code and fitted behavior.The benchmark literature also examines linear references, shared response variation, calibrated metrics, and distribution shifts.Source: §2 · p. 2
  • Reliance and behavioral tests probe fitted-model behavior, whereas CELLAUDIT begins with a registered claim and its cited computation, then tests source consumption, fitted dependence, and predictive contribution.The resulting evidence can also guide model revision.Source: §2 · pp. 2–3
  • Figure 1 presents a workflow from registered context, compound, and dose inputs through auditing, development-guided revision, and held-out evaluation.Its panels also identify a compound-invariant BBBC047 predictor and a sci-Plex revision retaining dose-conditioned FiLM and rank-128 fusion.Source: Figure 1 · p. 3

3 PROBLEM SETUP: PREDICTIVE PERFORMANCE AND REGISTERED INPUT-USE

The setup registers the inputs and interventions for each prediction task, then fixes data processing, budgets, evaluation, and selection rules before search.

3 PROBLEM SETUP: PREDICTIVE PERFORMANCE AND REGISTERED INPUT-USE

  • Each example specifies biological context, perturbation, optional attributes such as dose, and a response; registered input coordinates and admissible interventions define the task.A prediction–claim gap occurs when cited source contradicts a registered claim or fitted predictions are invariant to input replacement; predictive performance is separate from the definition.Source: §3 · p. 3
  • Folds 1–2 fit candidates, Fold 3 supplies feedback and selects checkpoints, and Folds 4–5 evaluate selected checkpoints under the stated source-selection exceptions.Data processing, budgets, evaluation, and selection rules are fixed before search.Source: §3 · p. 3

4 METHOD

The method fixes agentic search and evaluation, checks registered claims in source code, measures dependence and predictive contribution, and uses falsification evidence to guide revisions.

4.1 AGENTIC MODEL SEARCH WITH FIXED DATA AND EVALUATION

  • The CellScientist policy proposes executable revisions from the current model, development diagnostics, execution status, and history, while a fixed Fold-3 rule selects the endpoint.Preflight and training code allow up to three repairs per failed proposal; input-use tests are deterministic.Source: §4.1 · p. 3

4.2 CHECKING REGISTERED INPUT-USE CLAIMS IN SOURCE CODE

  • CELLAUDIT’s syntax-tree checker classifies cited computation as implemented, implementation-contradicted, or unresolved; singleton key–value attention cannot transmit compound information through its query.Whole-model replacement then tests fitted dependence, while target-loss contrasts test whether that dependence helps prediction.Source: §4.2 · p. 4

4.3 TESTING FITTED DEPENDENCE AND PREDICTIVE CONTRIBUTION

  • With checkpoints and targets fixed, CELLAUDIT replaces one registered input while retaining others; prediction distance tests dependence, and score and loss contrasts test predictive benefit.Positive effects indicate benefit relative to registered replacements; replacement distributions and observed-support coverage are task-specific, and continuous contributions use paired intervals.Source: §4.3 · p. 4

4.4 USING FALSIFICATION EVIDENCE TO GUIDE THE NEXT SEARCH

  • Path-constrained discovery compiles design cards with an explicit compound route, while BBBC falsification-guided discovery uses residual modeling and input-use feedback.The context predictor is fitted on Folds 1–2 and fixed; BBBC047 selects by predictive score among qualifying candidates, whereas BBBC036 prioritizes compound loss gain among predictively noninferior candidates.Source: §4.4 · p. 4
  • In sci-Plex searches, Audit feedback adds grouped errors, source checks, and compound/dose replacements to score metrics and learning curves while keeping training and selection rules fixed.Paired outcomes evaluate the complete feedback package, with selected sources and checkpoints frozen before held-out evaluation.Source: §4.4 · p. 4

5 EXPERIMENTAL DESIGN

The experiments develop predictors on linked morphology–transcription tasks and test input-use claims across separated control references, matched search conditions, independently acquired cohorts, and genetic perturbations.

5.1 DEVELOPMENT ON LINKED MULTIMODAL PERTURBATION TASKS

  • BBBC036 and BBBC047 predict joint morphology–transcription responses from same-plate control profiles, compound representations, and dose, using response-blind scaffold folds.Global PCC selects models; MSE and modality-specific metrics diagnose gains.Source: §5.1 · p. 5
  • Discovery policies are compared under matched initialization, executor, feedback, and ten-candidate budgets, with ten trajectories per task and five paired refits per selected model.Follow-up formulations retain the folds and budget and carry all ten selected designs through Folds 4 and 5.Source: §5.1 · p. 5
  • Shared-versus-disjoint control-bank refits hold target references fixed, isolating physical control-well overlap across two orientations, five seeds, and fixed checkpoints.The evaluation includes the score-selected source, a control-only source, and a fixed guided design.Source: §5.1 · p. 5
  • Matched sci-Plex searches hold out compound–cell-line combinations while keeping both marginals in training and evaluate compound or dose replacements in matched contexts.Both conditions use five paired seeds, a shared anchor, and five new candidate slots per trajectory.Source: §5.1 · p. 5

5.2 TESTING INPUT-USE CLAIMS ON NEW COHORTS AND PERTURBATION TYPES

  • Frozen designs are refit on independently acquired LKCP, while a separate Norman CRISPRa task evaluates unseen double-perturbation combinations against GEARS.The LKCP refit uses no source revision, model reselection, or audit-rule changes; the Norman input bundle includes pair identity and its matching single-perturbation anchor.Source: §5.2 · p. 5

6 RESULTS

Across the linked perturbation tasks and follow-up evaluations, CELLAUDIT separates predictive performance, fitted input dependence, and target-relevant contribution. Results expose compound-invariant high-scoring models, while guided revisions and cross-cohort tests show that predictive and input-use evidence can diverge.

6.1 CELLAUDIT DETECTS COMPOUND INVARIANCE IN A HIGH-SCORING PREDICTOR

  • 0.3153 mean held-out Global PCC accompanies exact compound invariance in the BBBC047 score-selected model, versus 0.3142 for control-only; singleton key–value attention blocks its compound query.Compound replacement changes PCC by 0.0000 in every refit; control replacement lowers PCC by 0.1932 on Fold 4 and 0.2004 on Fold 5.Source: §6.1 · p. 5, Table 1 · p. 6
  • 47 of 48 candidates change predictions under compound replacement on both folds, but only 20 have positive target-loss scaffold intervals on both.Figure 2 distinguishes fitted dependence from contribution; its green outlines mark the 20 candidates meeting the two-fold interval criterion.Source: §6.1 · p. 6, Figure 2 · p. 7

6.2 THE MISMATCH PERSISTS WITH DISJOINT CONTROL REFERENCES

  • With disjoint control wells, the score-selected source remains compound-invariant, while all six shared–disjoint PCC comparisons have five-seed intervals crossing zero.The fixed guided design retains PCC gains over disjoint control-only of +0.0028 on Fold 4 and +0.0025 on Fold 5, with positive compound-replacement loss effects.Source: §6.2 · p. 6

6.3 AN EXPLICIT COMPOUND ROUTE AND ITS FITTED CONTRIBUTION

  • The explicit compound route yields mean factorial compound effects of 0.0083 and 0.0074 on Folds 4 and 5, but several refits remain below or overlap the reference threshold.All 200 candidate evaluations pass source and interface checks; control-profile effects exceed the threshold throughout.Source: §6.3 · p. 6

6.4 GUIDED RESIDUAL DISCOVERY COMBINES PREDICTIVE GAINS WITH TARGET-RELEVANT COMPOUND CONTRIBUTION

  • Guided residual models exceed control-only by +0.0037 on Fold 4 and +0.0028 on Fold 5, with positive compound effects, yet none of ten models exceeds the reference-relative criterion.On observed-support replacements, guided checkpoints retain positive compound contributions; fixed residual learners show a sensitivity–score trade-off, and BBBC036 Fold 5 has a −0.0020 gain over the context anchor.Source: §6.4 · pp. 6–7

6.5 AUDIT-ENRICHED FEEDBACK YIELDS HIGHER MEAN HELD-OUT PREDICTION

  • Audit feedback yields higher mean PCC and lower MSE than Score on both sci-Plex partitions; MSE is 9.58% lower on Fold 4 and 6.10% lower on Fold 5.Compound target-loss gains rise from 0.0186 to 0.0220 on Fold 4 and from 0.0064 to 0.0079 on Fold 5, but paired intervals for Audit–Score contributions include zero.Source: §6.5 · pp. 7–8, Figure 3 · p. 8

6.6 PREDICTIVE GAINS AND INPUT-USE SUPPORT GENERALIZE DIFFERENTLY UNDER INDEPENDENT ACQUISITION

  • Frozen LINCS-selected designs improve over control-only on LKCP, and dose contribution persists, but no LKCP refit passes the compound-identity criterion.The dose-replacement allocation is positive on both cohorts and folds; LINCS compound-identity criterion passes fall from 35/50 refits on Fold 4 to 10/50 on Fold 5.Source: §6.6 · pp. 8–9, Figure 4 · p. 9

6.7 THE AUDIT DETECTS DEPENDENCE ON THE PERTURBATION-PAIR INPUT BUNDLE

  • All five Norman CELLSCIENTIST refits pass the pair-input-bundle criterion on both folds; shuffling pair identity with its single-perturbation anchor lowers PCC by 0.5578 and 0.6280.Mean PCC exceeds h0 by +0.0060 and +0.0045 and the observed-single additive baseline by +0.0209 and +0.0114.Source: §6.7 · p. 9

7 DISCUSSION AND CONCLUSION

CELLAUDIT separates predictive success from support for claimed input use, while the paper’s cohort documentation specifies how the paired tasks and evaluations were constructed. Its conclusions remain conditional on the fitted model, evaluation setting, and registered replacement distribution.

7 DISCUSSION AND CONCLUSION

  • CELLAUDIT shows that predictive success and claimed input use should be evaluated separately; independent-acquisition refits show predictive generalization need not imply claim generalization.In sci-Plex, audit-enriched feedback improves mean prediction and input contributions, but paired intervals over five trajectories span zero.Source: §7 · p. 9
  • Conclusions are conditional on the fitted model, evaluation setting, and registered replacement distribution; observed-support replacements do not establish biological exchangeability, mechanism, or causal effects.The sci-Plex study evaluates the complete audit-feedback package rather than individual diagnostics.Source: §7 · p. 9

A DATA PROVENANCE, TRANSFORMATIONS, AND TASK STATISTICS

  • BBBC036 and BBBC047 are linked cohorts from one paired cellular-profile release, with 1,916 and 20,081 matched compound–dose conditions, respectively.The matched subsets span 47 CP/22 L1000 treatment plates for BBBC036 and 273 CP/360 L1000 treatment plates for BBBC047.Source: A DATA PROVENANCE, TRANSFORMATIONS,… · p. 14
  • Each condition uses a same-plate control-only CP median, a 2,048-bit radius-2 Morgan fingerprint for compound identity, and log-transformed observed dose.Responses are aggregated by condition median; joint targets contain 1,568 dimensions for BBBC036 and 1,678 for BBBC047.Source: A DATA PROVENANCE, TRANSFORMATIONS,… · pp. 14–15
  • The plate-family transfer split is constructed without response values and assigns connected components to fit, selection, audit, and replication roles.BBBC036 has six components and BBBC047 has 64; Folds 1–2 fit models, Fold 3 supplies feedback, and later-fold roles differ by search panel.Source: A DATA PROVENANCE, TRANSFORMATIONS,… · p. 15, Table 3 · p. 15
  • Ten trajectory-selected LINCS–Pilot1 designs are transferred to independently acquired Cell Painting LKCP Batch 2 with the same compound, dose, and control-profile inputs.The fixed-design refit tests predictive and input-use claims across acquisition cohorts.Source: A DATA PROVENANCE, TRANSFORMATIONS,… · p. 16

A.1 PHYSICAL SEPARATION OF CONTROL INPUTS AND TARGET REFERENCES

Physical control-well separation preserves the guided model’s predictive and compound-use increments, while the score-selected and control-only predictors remain invariant. The reconstructed comparison separates physical wells at the CP reference layer, but does not establish removal of all upstream preprocessing or biological dependence.

A.1 PHYSICAL SEPARATION OF CONTROL INPUTS AND TARGET REFERENCES

  • The comparison allocates physical control wells to balanced, disjoint A/B banks and uses direction-specific reference banks while keeping shared and disjoint targets byte-identical.This separates wells at the reconstructed CP reference layer, not all upstream preprocessing or biological dependence.Source: §A.1 · p. 17, Table 6 · p. 18
  • +0.0028 and +0.0025 are the guided-minus-control-only disjoint-arm PCC gains on Folds 4 and 5, respectively.The five-seed paired 95% t intervals are [+0.0017, +0.0038] and [+0.0012, +0.0037].Source: §A.1 · p. 18, Table 10 · p. 20
  • Only row-uniform maps were evaluated; equality of row-uniform and compound-balanced donor probabilities does not establish biological exchangeability.All 6,268 retained source pools had equal donor probabilities under the two map constructions.Source: §A.1 · p. 18
  • The reconstructed full-bank CP mid-CDF check has 99.9799% agreement at 10^-6, but retains a maximum absolute discrepancy of 0.19140625.The discrepancy was retained rather than repaired by choosing a favorable target transform.Source: §A.1 · p. 18
  • The guided model has positive compound RMS, target-loss gain, and PCC drop in all 40 evaluations, while Score and control-only are exactly invariant.The guided disjoint-arm PCC advantage remains positive under both reported interval methods, but uncertainty scopes do not establish an unchanged shared-to-disjoint effect.Source: §A.1 · p. 18, Table 11 · p. 21, Table 12 · p. 21

B EXACT PROTOCOL, DECISION RULES, AND STATISTICAL UNITS

The protocol fixes data, candidate selection, evaluation, and replacement rules before held-out testing, with uncertainty computed at the appropriate trajectory, model, or scaffold level. Search comparisons use matched ten-slot trajectories, while BBBC047 scaffold-aware checks find positive intervals for all three reported quantities on both held-out folds.

B EXACT PROTOCOL, DECISION RULES, AND STATISTICAL UNITS

  • Candidate proposals are limited to specified model changes, selected without held-out responses, and evaluated under registered replacement and threshold rules.Fold 3 supplies feedback and checkpoint selection; source selection avoids Folds 4 and 5 except the disclosed across-policy Fold-4 choice for the prediction-score study.Source: B EXACT PROTOCOL, DECISION RULES, AND… · p. 22
  • Compound replacements preserve control profiles and dose, while dose tests use observed same-compound values and require minimum coverage and contrast.BBBC047 has insufficient eligible held-out dose coverage on both folds, so dose is not identifiable in those panels.Source: B EXACT PROTOCOL, DECISION RULES, AND… · pp. 22–23
  • Positive hierarchical intervals on both BBBC047 held-out folds cover Full−control-only PCC, compound-shuffle PCC drop, and compound target-loss gain.For BBBC036, compound and target-loss intervals remain positive on Fold 4, while Fold-5 intervals include zero.Source: B EXACT PROTOCOL, DECISION RULES, AND… · pp. 24–25
  • Uncertainty treats trajectories, fitted models, or scaffold groups as statistical units; repeated input permutations are not independent biological replicates.The central falsification-guided analysis reports both conditional scaffold-resampling intervals and hierarchical intervals resampling models and scaffolds.Source: B EXACT PROTOCOL, DECISION RULES, AND… · pp. 23–25

C SEARCH TRAJECTORIES, RELIABILITY, AND COST

  • Matched discovery comparisons use ten-slot trajectories and report predictive quality alongside execution reliability, resource costs, and selected-source refit performance.Tables 19–23 describe trajectory-level metrics, candidate validity, provider usage, training time, and wall time; the supplied passages do not state their comparative values.Source: Table 17 · p. 26, Table 19 · p. 26, C SEARCH TRAJECTORIES, RELIABILITY,… · p. 26

D INPUT-USE CLAIMS IN COMBINATORIAL GENETIC-PERTURBATION

The genetic-perturbation audit replaces complete pair-input bundles on unseen double combinations and tests whether fitted predictions use them. The independent-acquisition and selected-model panels separately assess transfer of input-use claims and calibrate source and behavioral checks.

D INPUT-USE CLAIMS IN COMBINATORIAL GENETIC-PERTURBATION

  • The Norman held-out test replaces both pair identity and its matching single-perturbation anchor, preserving the source response target and testing dependence on the complete bundle.Each held-out boundary has 26 unseen double-perturbation conditions and uses 32 complete derangements; the intervention prevents the anchor from retaining pair information.Source: D INPUT-USE CLAIMS IN COMBINATORIAL… · p. 31
  • The genetic transfer uses a common 5,045-gene response task while GEARS retains its native graph inputs and model formulation.The comparison holds singles and controls fit-only and evaluates unseen double combinations whose constituent single-gene responses were available during fitting.Source: D INPUT-USE CLAIMS IN COMBINATORIAL… · p. 31
  • All four selected Norman models pass the pair-input-bundle test in 5/5 refits, while Global-PCC ordering varies across folds.CellScientist has the highest mean Best@10 and frontier AUC, and leads condition-, DEG-20-, and DEG-50-PCC at Fold 5.Source: Table 26 · p. 31, D INPUT-USE CLAIMS IN COMBINATORIAL… · p. 32, Table 33 · p. 34

E INDEPENDENT-ACQUISITION TEST OF SMALL-MOLECULE CLAIM

  • On independent LKCP acquisition, dose status exceeds the reference-relative criterion consistently, whereas compound status varies by cohort and fold.The ten LINCS-selected design instances are fixed and refit without source reselection; compound permutations hold dose fixed, and dose replacements use observed same-compound doses.Source: E INDEPENDENT-ACQUISITION TEST OF… · p. 35, Table 35 · p. 35, Table 36 · p. 36

F FULL SELECTED-MODEL AUDIT PANEL AND CALIBRATION

  • The selected-model audit panel compares predictive metrics with compound-replacement tests, and none of four score-selected sources meets the aggregate compound-use criterion on either task at Fold 5.The panel separates fitted predictions, input permutations, and registered threshold statuses; control-adjusted residual PCC is also reported separately from MSE.Source: Table 37 · p. 37, Table 38 · p. 38, Table 40 · p. 39
  • Controlled code fixtures and synthetic known-dependence fixtures independently check source-inspection and held-out permutation adjudication.The fixed audit rule is calibrated separately from biological-model results.Source: Figure 5 · p. 36, Table 41 · p. 39

F.1 KNOWN-TRUTH MEASUREMENT AND UNCERTAINTY CHECKS

Known-truth simulations distinguish fixed-dataset conditional effects from population effects across four input-dependence cases. They show that map-only intervals can severely undercover population effects, while group-aware intervals improve coverage but retain limitations in some scenarios.

F.1 KNOWN-TRUTH MEASUREMENT AND UNCERTAINTY CHECKS

  • Across four simulated cases, the loss contrast distinguishes invariant, sensitive-without-benefit, beneficial, and harmful inputs, while prediction distance detects sensitivity rather than target benefit.The loss truths are 0, 0, .08, and −.08; all non-invariant cases have distance truth .08.Source: §F.1 · p. 40
  • The 24 simulation scenarios cross group count, group-size balance, within-group correlation, and noise level, with 400 independent datasets per scenario.The two donor-map budgets are nested on the same datasets rather than treated as independent cohorts.Source: §F.1 · p. 40
  • Map-only intervals attain near-nominal coverage for fixed-dataset conditional effects but severely undercover population effects in the sensitive-without-benefit null.Source-group intervals improve population coverage, although finite-group undercoverage remains; changing reference-context variation can also change qualification despite fixed compound benefit.Source: §F.1 · p. 40
  • Simulation results do not establish biological conditional exchangeability or support, and group-aware methods still undercover in some scenarios.Bootstrap performance is particularly weak with few groups, and positive null declarations can exceed the nominal one-tail rate.Source: §F.1 · pp. 40–41

G COMPLETE INPUT-INTERVENTION RESULTS

The intervention results compare predictive performance with compound and control effects, while source checks, prediction replacements, discovery traces, and factorial summaries separate pathway implementation, fitted dependence, and target-relevant contribution.

G COMPLETE INPUT-INTERVENTION RESULTS

H ALL-CANDIDATE SOURCE CHECKING

  • An implemented multi-tower source is exactly invariant, while checker findings based on source locations are code-level evidence and complete-model replacement tests provide the corresponding behavioral evidence.Source: H ALL-CANDIDATE SOURCE CHECKING · pp. 45–46

I PREDICTION-SPACE DEPENDENCE

  • Normalized RMS prediction changes measure fitted-output sensitivity to replacement, while separate target-loss contrasts test whether that sensitivity improves predictive utility.The prediction-space analysis requires inference over fixed checkpoints.Source: I PREDICTION-SPACE DEPENDENCE · p. 47, Table 54 · p. 47

J.1 REPRESENTATIVE MULTIMODAL DISCOVERY TRACE

  • The representative multimodal trace moved from Fold-3 Global PCC 0.2668 to 0.2928 with a multi-head cross-modal-attention revision.The selected candidate used separate control, compound, and dose encoders, cross-modal attention, and a shared response program.Source: §J.1 · pp. 47–48, Table 55 · p. 48

J.2 REPAIR-CONTAINING DISCOVERY TRACE

  • The repair-containing trace reached Fold-3 Global PCC 0.3230 after resolving an output-shape mismatch, while a later slot exhausted all three repair attempts.The repaired output was returned as a [batch, target] tensor.Source: §J.2 · p. 48, Table 56 · p. 48

K EXTENDED RELATED WORK

  • CELLAUDIT complements work on predictive benchmarks and scientific agents by testing whether registered input-use claims hold from cited code through fitted behavior and target-loss changes.The related-work discussion positions this as distinct from predictive evaluation and reliance tests alone.Source: K EXTENDED RELATED WORK · pp. 48–49

L FACTORIAL CONTRASTS AND STABILITY SUMMARIES

  • In the synthetic check, all 96 target-relevant instances had positive mean effects, but only 32 exceeded the reference-relative criterion, unchanged across 32, 128, or 512 maps.The criterion compares the effect with reference variation; a positive effect can remain below its operational threshold.Source: L FACTORIAL CONTRASTS AND STABILITY… · pp. 49–50

M.1 SINGLE-COORDINATE REPLAY ON OBSERVED SUPPORT

Observed-support replays show that input sensitivity varies by coordinate, policy, and cohort; matched references, candidate audits, and feedback comparisons further qualify those patterns.

M.1 SINGLE-COORDINATE REPLAY ON OBSERVED SUPPORT

  • On BBBC047, guided compound PCC drops were 0.00534 [0.00336, 0.00770] on Fold 4 and 0.00404 [0.00179, 0.00634] on Fold 5; score-selected drops were zero.Path-constrained compound effects were also positive on both boundaries.Source: §M.1 · p. 50
  • On LKCP, guided dose effects had positive intervals on both folds, while compound intervals included zero on both folds.For example, Fold 4 dose was 0.02407 [0.01154, 0.04369] and compound was 0.00043 [−0.00149, 0.00239].Source: §M.1 · p. 50, Table 57 · p. 50

M.2 COMPLETE MATCHED INPUT-REFERENCE FAMILY

  • Dose improved the LINCS reference, with smaller increments on BBBC and LKCP; on LKCP Fold 5 the full reference had a positive PCC increment but an MSE interval including zero.These comparisons use the same masked-MLP family and five paired training seeds.Source: §M.2 · p. 51, Table 58 · p. 51, Table 59 · p. 51

M.3 LINKED SOURCE AND BEHAVIOR IN REAL CANDIDATES

  • In 48 linked real candidates, 47 changed predictions and 20 had positive target-loss intervals on both boundaries; one MultiTowerFusion source was exactly invariant.The counts describe the structurally sampled set, not all generated candidates.Source: §M.3 · pp. 51–52, Table 60 · p. 52

M.4 CONTROL-REFERENCE REUSE ON SCI-PLEX

  • All six shared-minus-disjoint PCC intervals included zero in the sci-Plex control-reference reuse comparison.The response target was held fixed while the context input reused the reference bank or used the other bank.Source: §M.4 · p. 52, Table 61 · p. 52

N MATCHED FEEDBACK: PREDICTIVE OUTCOMES AND INPUT CONTRIBUTION

N.1 SCORE–AUDIT FEEDBACK COMPARISON: COMPLETE RECORDS

  • The descriptive paired comparison used five trajectories (df=4), 4,000 shared trajectory–compound bootstrap draws, and retained all five pairs in both uncertainty summaries; wins favor higher PCC or lower MSE.F4 and F5 evaluate the same selected checkpoints on held-out combinations.Source: F5 PCC +1.460 [-0.496, +3.416]… · p. 53

N.2 ROBUSTNESS ACROSS THE COMPLETE FIVE TRAJECTORY PAIRS

The five-trajectory robustness analysis shows that uncertainty summaries depend on their sampling assumptions and scope, and that additional reconstruction does not constitute an independent full-array counterfactual check.

N.2 ROBUSTNESS ACROSS THE COMPLETE FIVE TRAJECTORY PAIRS

  • The feedback comparison’s exact sign-flip analysis has a minimum attainable two-sided tail probability of 2/32 = .0625, and no multiplicity-adjusted confirmatory claim is made.The analysis enumerates all 32 sign assignments under a pair-exchangeability/sign-symmetry null.Source: §N.2 · p. 55
  • With five trajectory pairs, bootstrap and paired-t inference target the same conditional mean contrast but can differ because of their sampling assumptions; more bootstrap draws do not establish broad generalization.All predictive bootstrap intervals exclude zero, while paired-t intervals and exact sign-flip tails do not provide the same evidence.Source: §N.2 · pp. 55–56, Table 72 · p. 56
  • Coordinate reconstruction verifies stored sufficient statistics but is not an independent full-array counterfactual check.The retrospective descriptions do not reselect checkpoints, retrain models, or revise historical qualification labels.Source: §N.2 · p. 56

O RECORDED FEEDBACK, SOURCE REVISIONS, AND FROZEN ENDPOINTS

The recorded audit-feedback cases trace how agent feedback changed candidate source code and how selected checkpoints were evaluated on frozen sci-Plex partitions. The records also show that selection performance and perturbation contribution can diverge, while some revisions fail to improve their parent designs.

Recorded feedback and source revisions

  • Before slot 4, feedback included slot 2’s best epoch, learning curve, group-error diagnostics, source checks, and positive compound and dose replacement effects.The emitted revision reduced trainable parameters from 4,056,400 to 2,754,896 by narrowing the chemical/context encoders and adding dropout; dose-conditioned FiLM and the rank-128 bilinear interaction remained.Source: O RECORDED FEEDBACK, SOURCE REVISIONS,… · pp. 56–57

Frozen endpoints

Recorded feedback and source revisions

  • In the second trajectory, slot 4 returned to slot 1 as its design parent and was selected by PCC, while slot 5’s explicit pairwise projections lowered PCC and were not selected.Both trajectory cases retain all five audit-feedback proposals; every new candidate was trained from scratch while the shared context-plus-dose predictor was frozen.Source: O RECORDED FEEDBACK, SOURCE REVISIONS,… · pp. 56–57

Frozen endpoints

  • The recorded figure links candidate ancestry and verified source edits to matched anchor, Score, and Audit checkpoints on Folds 4 and 5, then shows the selected checkpoint’s compound and dose loss gains.Table 74 records selection-partition PCC and MSE, with bold PCC marking each trajectory’s selected endpoint.Source: Figure 7 · p. 58, Table 74 · p. 59
Loading 2609.27234v1…