PaperlayerView exact revision on arXiv

Source-linked AI summary

Modelstamp: Pre-Deserialization Verification of Machine-Learning Artifacts and Runtime Environment State

Anagha Dhekne

arXiv:2609.01781v1cs.SEcs.CR

TL;DR

Persisted ML artifacts can remain byte-identical while their loading environments change, and artifact integrity checks alone cannot expose that drift. Modelstamp records artifact and selected runtime evidence in a sidecar manifest and verifies both before deserialization; controlled evaluations showed the intended drift and trust-boundary behavior, with verification time rising from 0.032 s to 3.334 s across 10 MiB to 1 GiB artifacts.

  • Problem

    Byte-identical persisted models may encounter changed model-relevant runtimes, a discrepancy that artifact-integrity checks alone cannot reveal.

  • Method

    Modelstamp binds a serialized artifact to a manifest containing a digest, runtime metadata, tracked package versions, and a model-relevant comparison subset verified before deserialization.

  • Results

    Controlled evaluations distinguished tested relevant from unrelated environmental changes, confirmed stated trust-boundary behavior, and measured verification time increasing from 0.032 s at 10 MiB to 3.334 s at 1 GiB.

  • Takeaways & Limitations

    Modelstamp provides a complementary pre-deserialization reference-state verification layer for artifact identity and selected runtime context.

  • Takeaways & Limitations

    The selected environment representation is incomplete, and Modelstamp does not establish behavioral compatibility, public provenance, freshness, or safe deserialization.

Abstract

from arXiv · show

Persisted machine-learning models can remain byte-identical while the software environments in which they are loaded evolve, creating a verification problem that artifact integrity checks alone cannot expose. This paper presents Modelstamp, a lightweight Python persistence library for verifying artifact integrity and represented runtime-environment state before deserialization. At persistence time, Modelstamp associates a serialized artifact with a sidecar JSON manifest containing a SHA-256 digest, runtime metadata, and installed versions from a bounded tracked-package set; a separately recorded model-relevant subset determines which package versions participate in drift comparison. Optional HMAC authentication supports workflows in which the producer and verifier share a secret key. At verification time, the artifact and represented current environment are checked against this recorded evidence before the model is deserialized. Modelstamp is evaluated using 14 controlled environment-drift scenarios, eight controlled trust-boundary scenarios, and an artifact-size scaling benchmark from 10 MiB to 1 GiB. The controlled drift experiments behaved as specified across relevant dependency changes, unchanged environments, and unrelated environmental changes, including broader noise controls. The trust-boundary experiments similarly confirmed both intended detections and expected limitations, including shared-key forgery and replay. Median verification time increased from 0.032 s at 10 MiB to 3.334 s at 1 GiB, with measured throughput of approximately 307-312 MiB/s in the benchmark environment. These results characterize Modelstamp as a complementary pre-deserialization reference-state verification control rather than as a replacement for dependency-management systems, malicious-model detection, safe deserialization, or public publisher authentication.

I. INTRODUCTION

Modelstamp addresses the distinct problem of verifying both serialized artifact integrity and relevant runtime-environment state before deserialization. It records bounded environment evidence, filters package comparisons by model relevance, and evaluates these controls under controlled drift, trust-boundary, and scaling scenarios.

  • A byte-identical model artifact can depend on an evolving Python runtime, framework, and supporting libraries at deserialization time.
  • Artifact integrity and environment consistency are separate properties: hashing detects byte modification, but an unchanged digest cannot reveal relevant dependency changes.
  • Modelstamp records a SHA-256 digest, runtime metadata, and tracked installed-package versions, then compares a model-relevant subset during pre-deserialization verification.
  • The evaluation covers 14 controlled environment-drift scenarios, eight trust-boundary scenarios, and artifact-size scaling from 10 MiB to 1 GiB.
  • Median verification time increased from 0.032 s at 10 MiB to 3.334 s at 1 GiB, while throughput remained approximately 307–312 MiB/s.
  • Modelstamp complements, rather than replaces, malicious-artifact detection, safe deserialization, public publisher authentication, and related controls.

C. Runtime Environments and Dependency Drift

The paper treats dependency drift as a difference between recorded and current model-relevant runtime state, while separating captured package evidence from the subset compared for a particular artifact. This provides narrower reference-state verification, not behavioral compatibility or complete reproducibility.

  • C. Runtime Environments and Dependency Drift: An artifact digest can remain valid after runtime packages change, so artifact integrity alone cannot expose software-state drift.
  • C. Runtime Environments and Dependency Drift: Installed package versions are recorded directly at persistence time rather than reconstructed later from declarations or lockfiles.
  • C. Runtime Environments and Dependency Drift: Detected environment drift is evidence of recorded-state difference, not proof of harmfulness, incompatibility, behavioral change, or deserialization failure.
  • C. Runtime Environments and Dependency Drift: Modelstamp compares a bounded captured package set through a model-relevance projection, while runtime-level fields are compared independently.
  • C. Runtime Environments and Dependency Drift: Package drift is the set of relevant dependencies whose versions differ or are absent, combined with runtime drift when recorded runtime tuples differ.
  • C. Runtime Environments and Dependency Drift: The formalization separates artifact consistency from recorded-environment consistency, yielding four conceptual verification states.

III. THREAT MODEL AND SCOPE

Modelstamp assumes a trustworthy established reference state and a trusted verification process, then checks the artifact, manifest, and represented environment within that boundary. Its threat model explicitly excludes host or verifier compromise and recognizes shared-key capabilities.

  • The reference artifact and associated metadata are assumed trustworthy when the reference state is established.
  • The sidecar manifest stores the artifact SHA-256 digest, runtime metadata, selected package versions, and optional HMAC evidence.
  • Authentication, artifact-integrity, and environment checks occur before the serialization library reconstructs the model object.
  • The threat model includes artifact, manifest, and evidence manipulation after reference establishment, including replacement of artifacts and manifests together.
  • A legitimate shared-secret holder can generate cryptographically valid new evidence, so shared-secret authentication does not provide independent publisher identity.
  • Modelstamp does not protect against compromise of the operating system, interpreter, filesystem, verification code, trusted secrets, or verification results.

C. Integrity and Authentication Properties

Modelstamp separates byte-level artifact integrity, shared-secret authentication, and represented environment drift, applying these checks before deserialization. The design deliberately limits its claims to consistency with recorded evidence and a selective environment representation.

  • C. Integrity and Authentication Properties: SHA-256 recomputation detects whether candidate artifact bytes differ from the artifact represented by the recorded digest.
  • C. Integrity and Authentication Properties: HMAC can authenticate protected evidence within a shared-secret relationship, but every legitimate secret holder can generate valid evidence.
  • C. Integrity and Authentication Properties: Neither SHA-256 nor HMAC establishes public provenance or freshness, so valid artifact-and-manifest pairs may be replayed later.
  • C. Integrity and Authentication Properties: Environment verification reports changes in selected runtime state but does not predict behavioral incompatibility or deserialization failure.
  • C. Integrity and Authentication Properties: The represented environment omits or incompletely captures operating-system libraries, hardware, GPU drivers, CUDA versions, external services, and other native dependencies.
  • C. Integrity and Authentication Properties: Modelstamp is a pre-deserialization verification layer rather than a mechanism for malicious-model detection, safe deserialization, complete reproducibility, or public signing.

A. Design Goals

Modelstamp is designed to verify artifact integrity and relevant runtime state before deserialization while keeping artifact integrity and environment drift as separate properties. Its bounded capture and relevance filtering aim to reduce unrelated package warnings within an explicit shared-secret trust model.

  • Design goals: Verification precedes pickle or joblib deserialization, with artifact-integrity and applicable authentication checks performed first.The design also specifies environment comparison before deserialization.
  • Design goals: Artifact integrity and environment drift remain separate properties because byte-consistent artifacts can coexist with changed runtime environments.A modified artifact can also occur without a relevant dependency change.
  • Manifest evidence: The manifest records artifact filename, SHA-256 digest, size, serialization backend, model information, environment data, selected packages, and optional authentication information.Environment data includes Python version, implementation, platform, package versions, and creation timestamp; Git state may also be recorded.
  • Design goals: Modelstamp captures a bounded tracked-package state, then filters it to model-relevant packages for drift comparison.The relevance set is rule-based, uses recognized model and component modules, and is broader at capture time than at comparison time.
  • Integrity verification: Artifact hashing is incremental in 1 MiB chunks, and verification checks size before recomputing SHA-256.The resulting digest and size are stored in the manifest, establishing consistency relative to recorded evidence subject to trust assumptions.
  • Authentication: Optional HMAC-SHA256 authenticates deterministic manifest evidence, but shared-key authentication does not provide freshness, public publisher identity, or non-repudiation.Any legitimate holder of the shared secret can generate new valid evidence.

G. Verification and Deserialization Ordering

Modelstamp separates non-deserializing inspection and verification from loading, and its load path authenticates and verifies the artifact before comparing environment state and deserializing. The evaluation uses controlled matrices to test relevant-drift detection, trust-boundary behavior, and verification cost.

  • Verification operations: verify() authenticates the manifest when applicable and checks artifact existence, size, and SHA-256 without deserializing the artifact.check() instead combines integrity and runtime diagnostics into a mismatch report without reconstructing the object.
  • Load ordering: load() reads and validates the manifest, authenticates it, verifies the artifact, compares the environment, and only then deserializes.Environment mismatches occur before deserialization unless the caller explicitly configures them to be ignored.
  • Failure handling: Environment mismatches support warn, raise, and ignore policies, while integrity or authentication failures prevent the normal load path from reaching deserialization.The default mismatch policy is warn.
  • RQ1 evaluation: The RQ1 matrix evaluates relevant package changes against unchanged, unrelated-package, and broader-noise controls, requiring observed changed-package sets to match expected sets.The operational package-drift set contains relevant packages whose versions differ or are absent from the current relevant state.
  • RQ1 evaluation: The primary RQ1 outcome is the set of relevant packages reported as changed, with an empty expected set for unchanged and unrelated-package controls.The design tests sensitivity to selected relevant changes and suppression of selected irrelevant changes.

C. RQ2: Trust-Boundary Evaluation

The trust-boundary evaluation isolates Modelstamp’s integrity and HMAC behavior using controlled acceptance and rejection scenarios, while the scaling benchmark measures non-deserializing verification across artifact sizes. Results are interpreted as conformance to specified guarantees, not statistical attack-detection accuracy.

  • Trust-boundary design: The eight-scenario RQ2 matrix isolates integrity and authentication mechanisms from framework-specific model behavior using simple Python objects.Scenarios cover artifact modification, manifest changes, unsigned replacement, untrusted keys, trusted-key replacement, and replay.
  • Integrity checks: Artifact or digest modification is expected to fail when recalculated SHA-256 does not match the recorded digest claim.This applies both to modified artifacts and to edited digest evidence with the artifact left unchanged.
  • Authentication boundaries: Trusted shared-key possession permits new valid HMAC evidence, while replay of an older valid signed pair remains possible because HMAC does not establish freshness.Unsigned replacement under required HMAC verification and replacement with an untrusted key are rejected.
  • Interpretation: The RQ2 outcomes test conformance to documented guarantees and non-guarantees rather than a generic attack-detection percentage.Expected acceptance of shared-key forgery or replay represents a documented boundary, not a failed security test.
  • Scaling benchmark: RQ3 benchmarks verify() on 10 MiB, 100 MiB, and 1024 MiB artifacts using streaming SHA-256 without deserialization as a confounding operation.Reported timings use three measured runs after a filesystem-cache warmup.

F. Reproducibility

The reproducibility materials connect the reported validation results to executable repository artifacts and show that Modelstamp behaved as predicted across drift, trust-boundary, and scaling experiments.

  • Reproducibility: The immutable repository commit anchors the evaluated code, while workflow, matrix, and benchmark definitions trace reported results to executable artifacts.
  • Environment-drift validation: All 14 controlled environment-drift scenarios matched their predicted outcomes.Modelstamp surfaced tested relevant dependency changes while suppressing unrelated pandas, requests, and six-package noise changes.
  • Trust-boundary validation: All eight controlled trust-boundary scenarios matched the behavior predicted by the trust model.The experiments rejected tested artifact and authentication violations while exposing defined limits involving unsigned replacement, shared-key replacement, metadata changes, and replay.
  • Verification scaling: Median verification time increased from 0.032 s at 10 MiB to 3.334 s at 1 GiB, while throughput remained approximately 307–312 MiB/s.The benchmark measured 10 MiB, 100 MiB, and 1 GiB artifacts; the nearconstant throughput was consistent with streaming SHA-256 verification.

D. Summary of Findings

Modelstamp’s controlled evaluations matched its intended distinctions between relevant dependency drift, unrelated environmental changes, artifact integrity, and authentication boundaries. The results also clarify that Modelstamp complements rather than replaces existing artifact-security and dependency-management mechanisms.

  • Evaluation results: The controlled matrices confirmed expected behavior for relevant drift, unrelated changes, artifact integrity, and optional HMAC authentication.The evaluation also exercised conditions intentionally outside Modelstamp’s guarantees.
  • Positioning: Modelstamp extends existing dependency tooling by combining bounded package capture, relevance-filtered comparison, integrity evidence, authentication, and pre-deserialization checks.Existing systems address related but different questions about dependencies, contents, execution, provenance, and reproducibility.
  • Security boundary: A verified artifact is not necessarily benign or safe to deserialize because Modelstamp does not classify malicious contents or constrain deserialization.Malicious-artifact detection and constrained loading remain complementary controls.
  • Authentication boundary: HMAC authenticates recorded evidence under a shared-secret model, but a key holder can authenticate replacement content and replay previously valid evidence.It therefore does not provide public publisher identity or freshness.

D. Dependency Metadata and Broader Environment Capture

Modelstamp records concrete dependency and runtime evidence at persistence time, then compares only the artifact-specific subset considered relevant. This narrower state comparison avoids treating complete-environment capture or reproducibility as its objective.

  • Dependency metadata: Modelstamp records concrete package state rather than reconstructing dependencies from declarations, lockfiles, or other metadata sources.This addresses the incompleteness of reconstructing Python dependency state from heterogeneous ecosystem artifacts.
  • Environment capture: Broader reproducibility systems capture execution context and package it for later execution, whereas Modelstamp compares selected recorded state.It does not package a complete execution environment or reconstruct one.
  • Scope of comparison: A clean represented-state comparison means only that the defined package/runtime discrepancy was not detected, not that the complete environments are equivalent.The tracked snapshot is not intended to reproduce training or guarantee identical model outputs.
  • Positioning: Modelstamp’s distinctive position combines artifact integrity, bounded runtime capture, relevance projection, optional shared-secret authentication, and pre-deserialization verification.These properties are presented as complementary to adjacent dependency, security, authentication, supply-chain, and reproducibility approaches.
  • Evaluation implication: The controlled results surfaced selected model-relevant dependency changes while suppressing tested unrelated environmental changes.Environment verification remains an early warning for compatibility testing or review, not a determination of behavioral change.

B. Integrity, Authentication, and the Trust Boundary

Modelstamp separates artifact integrity and authentication from broader claims about security, provenance, freshness, and execution safety. Its verification cost scales with artifact size, while the benchmark’s throughput remains approximately stable in the tested environment.

  • Integrity: SHA-256 detects artifact-byte mismatch but cannot detect an attacker replacing both an artifact and its unsigned digest.A self-consistent replacement pair remains outside hashing alone’s guarantee.
  • Authentication: HMAC detects authenticated-evidence modification when the attacker lacks the trusted secret, but shared-key holders can generate valid evidence.This limitation is demonstrated directly by the trust-boundary scenarios.
  • Freshness: Replay remains possible because hashing and HMAC do not establish whether an artifact/manifest pair is the latest authorized state.Freshness would require an additional trusted version, timestamp, append-only history, or comparable mechanism.
  • Scaling: Verification cost grew with artifact size: median time reached 3.334 s at 1 GiB while throughput remained approximately 307–312 MiB/s across tested sizes.The measurements isolate verification cost and reflect the tested benchmark environment rather than universally negligible overhead.
  • Interpretation: Modelstamp verification should be interpreted narrowly as satisfying configured integrity, authentication, and represented-environment checks, not as proving safety, benignity, correctness, or reproducibility.Additional controls may be composed for publisher authentication, malicious-artifact detection, constrained deserialization, and compatibility testing.

B. Internal Validity

The evaluation demonstrates conformance for deliberately constructed scenarios and selected benchmark conditions, but its evidence does not generalize automatically across environments, ecosystems, versions, or deployment settings. The main limitations concern internal validity, dependency coverage, and measurement scope.

  • Internal validity: The 14/14 and 8/8 agreements demonstrate conformance for exercised scenarios, not real-world detection accuracy, attack coverage, or failure probability.The scenarios were deliberately constructed rather than randomly sampled.
  • Internal validity: Because the scenarios and implementation were developed within the same project, embedded implementation assumptions may influence results despite negative and boundary controls.Independent evaluation and adversarial test construction would provide stronger evidence.
  • Dependency coverage: Pinned save/check pairs improve control and reproducibility but sample only specific dependency-version transitions.Other version combinations may exhibit behavior absent from the current matrix.
  • Ecosystem scope: The relevance mechanism is concentrated on classical and tabular Python ML frameworks, excluding ecosystems such as PyTorch, TensorFlow, Transformers, ONNX, and safetensors.Results should therefore be generalized beyond the evaluated ecosystems only with caution.
  • Environment scope: Native libraries, GPU runtimes, system packages, dynamically imported components, and external services may influence execution without appearing in recorded Python package state.The evaluated artifacts and changes also do not cover every architecture, serialization mechanism, operating system, hardware configuration, or package interaction.
  • Measurement validity: The benchmark reports three warmed-cache artifact sizes and should not be treated as a universal performance rate or fitted performance model.Verification is isolated from deserialization, and storage, hardware, caching, concurrency, and artifact location can change observed cost.

B. Authentication, Freshness, and Key Management

Modelstamp’s authentication and environment-state checks have explicit boundaries: shared-secret HMAC does not provide publisher identity or freshness, and selected package-state drift does not establish behavioral impact. Broader environment coverage, compatibility validation, and independent evaluation remain complementary needs.

  • Authentication and Freshness: Shared-secret HMAC authenticates evidence but does not establish public publisher identity, non-repudiation, or freshness.A trusted-secret holder can authenticate replacement evidence, and previously valid authenticated evidence can be replayed.
  • Environment Coverage: Python package versions capture only part of the state that can influence model execution.Native libraries, operating-system components, hardware, accelerator runtimes, environment variables, and external resources may also matter.
  • Compatibility Validation: Modelstamp identifies changes in selected recorded state but does not determine whether those changes materially affect model behavior.External compatibility validation can use detected drift to trigger model tests, prediction-regression checks, or organization-specific validation.
  • Evaluation Scope: The current evaluation provides controlled evidence for specified drift, trust-boundary, and performance properties, but broader deployment evaluation would strengthen confidence.Future studies should vary model families, frameworks, serialization mechanisms, platforms, hardware, storage, and independently designed adversarial scenarios.
  • Scope of the Control: Modelstamp is a pre-deserialization verification layer rather than a guarantee of benignness, behavioral correctness, reproducibility, or safe deserialization.Additional authentication, environment, dependency-relevance, and compatibility mechanisms can extend coverage while preserving separate responsibilities.
Loading 2609.01781v1…