PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy Foundations for Multi-Institutional Scientific Artificial Intelligence

Olivera Kotevska, Sumit Jha, Aurélien Bellet, Rui Hu, Nathaniel D. Bastian, Rafael Ferreira da Silva, Ravi Madduri, Kibaek Kim

arXiv:2609.39787v1cs.CR

TL;DR

Scientific AI collaborations create privacy risks around institutional participation, strategy, and capability, while existing mechanisms protect different assets and do not automatically compose. The paper frames assurance through six claim elements and uses a claim register to examine lifecycle gaps. It identifies composition risks and facility-specific channels, while bounding assurance by the observers and channels that can be enumerated.

Abstract

from arXiv · show

Scientific artificial intelligence (AI), spanning foundation models (FMs) to federated data-analysis pipelines, is becoming shared infrastructure across national laboratories, universities, hospitals, and industrial partners. This collaboration creates privacy risks whose natural unit is often an institution's participation, research strategy, or technical capability rather than a single record. Differential privacy (DP), federated learning (FL), secure computation, trusted execution, and provenance each protect parts of the stack, but their guarantees rarely compose across mixed-trust institutions, access tiers, and autonomous agents. This perspective recasts privacy for scientific AI as an assurance problem defined by six elements: protected asset, observer, channel, permitted disclosure, guarantee, and evidence. We demonstrate the framing through a claim register for a composite cross-institutional scenario and use it to assess the model lifecycle. Two of the resulting gaps are specific to leadership-class facilities: scheduler, allocation, and telemetry metadata expose an institution's resource posture, and instrument-attached control loops leak research strategy through timing and contention on shared accelerators. We identify six research priorities: institution-level guarantees, agent-communication privacy, cross-tier information flow, privacy-compatible reproducibility, leadership-scale accounting, and instrument side channels. The contribution is a common form for stating, comparing, and auditing claims whose guarantees otherwise remain fragmented across the scientific AI stack.

I. PRIVACY BECOMES INFRASTRUCTURE

Scientific AI is becoming shared infrastructure across institutions with different obligations and overlapping roles, making enterprise cross-silo learning an inadequate privacy model. The paper frames privacy assurance around institutional assets and claims that must align across mechanisms, channels, and lifecycle stages.

I. PRIVACY BECOMES INFRASTRUCTURE

  • Institutions share scientific data and facilities under incompatible obligations, while one organization may be collaborator, operator, consumer, and auditor.Scientific claims also outlive training runs, so controls must persist across organizational and temporal boundaries.Source: §I · pp. 1–2
  • FL, DP, secure aggregation, MPC, and trusted execution protect different assets or stages, but their guarantees do not automatically compose.FL keeps raw data at its source, yet gradients may leak examples; record-, user-, or client-level DP does not hide institutional participation, strategy, or capability.Source: §I · p. 1
  • A privacy assurance stack aligns governance, statistical guarantees, cryptographic controls, information-flow policies, trusted hardware, and auditable evidence to an explicit threat model.The paper applies a six-element claim register and lifecycle analysis to expose gaps across training, release, agents, provenance, HPC infrastructure, and instruments.Source: §I · p. 2
  • The protected unit often extends beyond records to institutional participation, research strategy, technical capability, and tier-restricted knowledge.The framework applies to scientific AI broadly, including federated analytics and autonomous data-science pipelines.Source: §I · p. 2

II. THE UNIT OF HARM IS NOT ALWAYS A RECORD

Scientific collaborations can expose institutional assets that record-level privacy does not capture, including participation, strategy, capability, and tier-restricted knowledge. The paper proposes institution-matched neighboring worlds and observer views, while noting that the scientific-consortium model and cross-channel guarantees remain unresolved.

II. THE UNIT OF HARM IS NOT ALWAYS A RECORD

  • Privacy, confidentiality, access control, and information-flow security name different properties in the consortium, not interchangeable guarantees.The hospital seeks statistical privacy, the industrial partner confidentiality, and the shared model information-flow control against recovery of restricted inputs.Source: §II · p. 2
  • Institutional privacy assets include participation, strategy, capability, tier-restricted knowledge, resource posture, and collaboration structure, with harms dependent on the deployment.Correlated observations can make institution-level group privacy too weak, and aggregate behavior may reveal institutional assets even without exposing individual records.Source: §II · p. 2, Table I · p. 2
  • Adversaries may be collaborators, curious peers, external attackers, or compromised agents, so record-level attacks target the wrong unit for some institutional harms.The paper scopes its claims to confidentiality and controlled disclosure, distinguishing them from integrity, availability, poisoning, safety, fairness, and research ethics.Source: §II · pp. 2–3

A. Making the neighboring world precise

  • Institution-level participation privacy models the secret as an institution's contribution during a round set and the observer's view across released weights, timing, and orchestration metadata.The distribution class must capture correlations between institutional data and scheduling behavior; choosing it for a scientific consortium remains open, and no deployment in Table III meets the stated target.Source: A. Making the neighboring world precise · p. 3
  • An ε-guarantee for each observer view does not imply one for their union unless the constituent channels are conditionally independent given θ.This formalizes the paper's cross-channel composition failure.Source: A. Making the neighboring world precise · p. 3

III. A SIX-ELEMENT PRIVACY ASSURANCE CLAIM

The paper represents privacy assurance as a six-element claim spanning assets, observers, channels, permitted disclosures, guarantees, and evidence. It uses this common, lifecycle-spanning form to compare heterogeneous mechanisms and make institution-level claims auditable.

III. A SIX-ELEMENT PRIVACY ASSURANCE CLAIM

  • The claim form adapts assurance-case and threat-modeling practices to institutions and observation surfaces that cross mechanisms and scientific-AI lifecycle stages.Its contribution is the institution-level, lifecycle-spanning form and its use as a common register across heterogeneous privacy mechanisms.Source: §III · p. 3
  • LINDDUN and STRIDE do not represent institutional participation or instrument control-loop gaps, while assurance-case notations leave the privacy claim itself unspecified.The six-element claim combines evidence-based assurance with explicit observation-surface enumeration, applied at the institution level.Source: §III · p. 3
  • A privacy claim specifies the protected asset, observer, observation channel, permitted disclosure, claimed guarantee, and evidence.The elements distinguish useful collaboration from leakage and make a claim testable.Source: §III · p. 3

A. Design principles for institution-level assurance

Institution-level assurance requires claims tied to specific assets, observers, channels, and permitted disclosures. A claim register makes those claims testable by examining evidence and reviewing how guarantees compose across mechanisms and workflow stages.

A. Design principles for institution-level assurance

  • Name a specific institutional asset, bind each guarantee to an observer and observation set, and count model, agent, infrastructure, metadata, and instrument paths as channels.State what may be declassified and keep evidence proportional to the claim.Source: A. Design principles for… · pp. 3–4
  • Table II compares controls against the six claim elements in isolation, while the paper proposes an assurance framework rather than a new DP algorithm, FL protocol, or replacement for existing controls.No individual control row answers all six questions for the multi-institutional setting.Source: Table II · p. 4, A. Design principles for… · p. 4
  • Review composed views across mechanisms, stages, and institutions, binding each claim to a versioned workflow and validity period.These practices make the register an assurance artifact rather than a static inventory of controls.Source: A. Design principles for… · p. 4
  • Omitting any claim element makes a claim hard to test and easy to overstate; secure aggregation and audit logs can each leave disclosure risks.A colluding subset can subtract its inputs from an aggregate, while an audit log copied into general analytics can become a disclosure channel.Source: A. Design principles for… · p. 4

IV. A WORKED EXAMPLE: AN ASSURANCE CLAIM REGISTER

The worked register shows how individually sound controls can leave institution-level assets exposed when claims compose, and why evidence must be matched to the guarantee it supports. Its review checks assets, all six claim elements, and explicit residual risks.

IV. A WORKED EXAMPLE: AN ASSURANCE CLAIM REGISTER

  • The register is an illustrative deployment artifact whose review checks that every institutional asset has a claim, each claim names six elements, and every unprotected item is recorded as residual risk.Table III marks C3 and C6 as gaps that remain despite a per-mechanism control review.Source: §IV · p. 4, Table III · p. 5

A. Where individually sound claims fail to compose

  • C3 permits inference of hospital participation when public membership and C2 timing metadata are combined with C4 model access, despite no individual mechanism being violated.The inference becomes visible when participation is stated as its own claim with an observer set spanning C2 peers and C4 users.Source: A. Where individually sound claims… · p. 4
  • Per-round communication time rises from roughly 33 s to 275 s as clients increase to 96; unchanged payload sizes indicate synchronization and barrier waiting, not transfer.This establishes a leadership-scale timing channel, not a demonstrated participation-inference attack.Source: A. Where individually sound claims… · p. 4
  • C2, C4, and C5 compose under conjectured conditions including explicit combined-view analysis, bounded or tested residual timing metadata, monotone declassification, and no shared randomness or identifier.The conditions remain unproved, and the Table III deployment violates the residual-metadata condition.Source: A. Where individually sound claims… · p. 4

B. Where evidence is weaker than it looks

  • TEE attestation supports that specified code ran, not that it enforced C4 or C5 disclosure policy, so using it for those claims upgrades the evidence beyond its guarantee.C5’s agent-leakage benchmark bounds observed leakage on a test distribution, not worst-case adversaries; audits can falsify a DP claim but cannot certify it.Source: B. Where evidence is weaker than it… · pp. 4–5

C. The channel that model-level analysis misses

Instrument-attached control loops and beamline scheduling can expose research strategy through timing and contention, even when data and model channels are sealed. This instrument-to-compute path can be missed by data- and model-centric reviews.

C. The channel that model-level analysis misses

  • Beamline job cadence and instrument-control timing can leak research strategy through timing, power, and cache contention even when data and model channels are sealed.Shared accelerators make the risk concrete: covert and side channels have been demonstrated across co-located tenants on multi-GPU machines used for scientific AI.Source: C. The channel that model-level… · p. 5
  • The instrument-to-compute channel crosses a boundary that a system-level review can easily omit.The paper places hardware-security methods alongside institution-level assets in addressing this gap.Source: C. The channel that model-level… · p. 5
  • A purely data- and model-centric review omits claim C6, which concerns the instrument-side channel.The paper identifies C6 as an omitted claim in such reviews.Source: C. The channel that model-level… · p. 5

V. USING THE CLAIM REGISTER

The claim-register procedure enumerates assets, roles, and lifecycle channels, then records permitted disclosure, guarantees, evidence, and workflow versions. The supplied examples also identify agent communication, selective disclosure, and audit artifacts as parts of the review.

V. USING THE CLAIM REGISTER

  • A consortium review enumerates assets, observers, authorized roles, and channels across data, training, adaptation, release, autonomous operation, HPC services, instruments, and provenance.It states permitted disclosure, attaches a guarantee and evidence artifact to each channel, then reviews cross-channel and cross-stage composition and records residual risks.Source: §V · p. 5
  • The listed abstraction target is a feasible region rather than strategic rationale, alongside an aggregate structure–property trend.Source: §V · p. 5, B. Where evidence is weaker than it… · p. 5
  • The listed methods include task-conditioned abstraction with an empirical policy filter and information-flow control with adversarial release testing.The accompanying labels describe an agent-leakage benchmark as partial.Source: §V · p. 5, B. Where evidence is weaker than it… · p. 5
  • The listed audit options include filtered replay for an auditor, proof for a reviewer, selective disclosure, and a zero-knowledge compliance proof.The associated implementation labels include a signed manifest and audit record.Source: §V · p. 5

A. Completeness beyond individual claims

Completeness is necessarily relative to enumerated observers and channels, while system assurance also requires coverage of combined views across claims, stages, institutions, or time. The register records residual risks when channels remain unnamed and tracks dependencies among claims.

A. Completeness beyond individual claims

  • An open-ended channel set makes completeness undecidable; the register can establish coverage only relative to enumerated observers and channels, with unnamed surfaces retained as residual risk.C6 illustrates the boundary: the instrument channel was missing from model-centric reviews because it had not been listed.Source: A. Completeness beyond individual claims · pp. 5–6
  • System assurance requires element, asset, and observer–channel coverage plus analysis of views combined across claims, stages, institutions, or time.A deployment can satisfy claims individually yet fail under an observer’s combined view, so the register records cross-claim dependencies and residual composition risks.Source: A. Completeness beyond individual claims · p. 6
  • Composition analysis is exponential in the number of claims visible to an observer, although role-based review and sparse overlap in channels or observers help keep it tractable.The register grows roughly with assets times observers; choosing combinations to analyze in larger federations and supporting that work with tools remain open problems.Source: A. Completeness beyond individual claims · p. 6

VI. EVIDENCE AND ASSURANCE MAINTENANCE

Privacy assurance depends on claim-relative evidence and must be maintained as scientific-AI workflows and configurations change. The register organizes evidence adequacy and links lifecycle stages so reviewers can assess whether claims held over time.

VI. EVIDENCE AND ASSURANCE MAINTENANCE

  • Evidence classes are complementary, not ordered, and a claim’s evidence is meaningful only relative to its asset, observer, channel, disclosure, and guarantee.Formal, operational, adversarial, and provenance evidence support different parts of an assurance case; operational evidence checks that deployment matches assumptions, while provenance binds results to the workflow run.Source: §VI · p. 6, Table IV · p. 6

A. Evidence adequacy and confidence

  • For each claim, record evidence relevance, coverage, strength, assumptions, scope, date, responsibility, and blind spots rather than collapsing confidence into one score.Partial evidence is useful only when residual uncertainty stays explicit.Source: A. Evidence adequacy and confidence · p. 6

B. Assurance is a maintained property

  • Assurance is a lifecycle: define claims at design, bind them to configurations before deployment, monitor assumption-sensitive evidence in operation, and reassess affected claims after change.At retirement, retain minimum evidence for later verification while revoking access and deleting disclosure-prone traces; a versioned register links the stages.Source: B. Assurance is a maintained property · p. 6

VII. APPLYING THE FRAMEWORK ACROSS THE LIFECYCLE

  • The claim register applies the six assurance questions across the lifecycle to locate where assurance breaks.Source: §VII · p. 6

A. Data and distributed training

Across distributed training, release, and autonomous operation, privacy controls must address institution-level exposure and the information visible through agent and workflow channels. The framework calls for workflow-aware accounting, controlled release, and measurement of channel-specific leakage.

A. Data and distributed training

  • Federated learning orchestrates training but does not guarantee privacy, and record-level DP leaves institution-level participation open.Decentralization can make guarantees depend on an observer’s network position; long adaptive workflows need accountants coupled to workflow and scheduler state.Source: A. Data and distributed training · pp. 6–7

B. Adaptation, evaluation, and release

  • Tiered release requires labeled datasets, adapters, and checkpoints; explicit declassification; mediated tool outputs; adversarial tests; and a versioned certificate, while label preservation through distillation remains open.Watermarking and provenance bind the model, policy, and deployment context to the certificate.Source: B. Adaptation, evaluation, and release · p. 7

C. Autonomous operation

  • Autonomous operation must treat plans, tool arguments, retrieved context, and persistent memory as privacy channels, replacing raw content with recipient-specific abstractions and measuring leakage per channel and observer.The proposed acceptance criterion bounds what an observer can infer about the protected hypothesis.Source: C. Autonomous operation · p. 7

VIII. RESEARCH PRIORITIES AND OUTLOOK

The section turns the six-element assurance claim into a research agenda, framing success as bounded, falsifiable privacy statements tied to specific evidence artifacts and validity periods. It identifies gaps spanning institution-level guarantees, agents, access tiers, reproducibility, accounting, and instrument side channels.

Institution-level guarantees

  • A consortium-level guarantee should bound participation-inference advantage for a stated observer and observation window, tested through leave-one-institution-out, timing-only, and combined attacks.The combined attack considers released weights, public membership, and timing.Source: §VIII · p. 7

Agent-communication privacy

  • Agent-communication privacy lacks a scientific-domain task set and agreed acceptance thresholds for protecting unpublished hypotheses, target regions, and facility capabilities.Source: §VIII · p. 7

Cross-tier information flow

  • Cross-tier information-flow work still lacks a guarantee that tier-restricted labels survive distillation.Source: §VIII · p. 7

Privacy-compatible reproducibility

  • Privacy-compatible reproducibility needs audience-specific verification and replay, while integrated FM and agent-workflow designs still need evaluation for scientific fidelity and secrecy.The proposed forms include protected traces for auditors, commitments or proofs for reviewers, and policy-filtered replay packages for collaborators.Source: §VIII · p. 7

Leadership-scale accounting

  • Leadership-scale accounting must span workflow and scheduler state, allocate privacy loss across institutions, and report utility, time, communication, energy, and privacy together.Source: §VIII · p. 7

Instrument side channels

  • Instrument-side-channel research lacks representative threat models and trace data; defenses should be judged against control-loop latency and calibration accuracy.Source: §VIII · p. 7

Limitations

  • The framework specifies claims but does not itself provide privacy or establish implementation compliance; the scenario is illustrative and formal composition remains unresolved.Source: §VIII · p. 7
Loading 2609.39787v1…