PaperlayerView exact revision on arXiv

Source-linked AI summary

DITTO: A Context-aware Pickle-based Pre-Trained Model Scanner for Effective Security Audits

Qiaolin Qin, Wanpeng Li, Benoit Baudry, Lorenzo De Carli, Heng Li, Ettore Merlo

arXiv:2610.10735v1cs.CRcs.SE

TL;DR

Malicious behavior can be triggered while loading serialized PTMs, creating a software supply-chain risk. DITTO combines Pickle state tracking with context-aware semantic analysis and is evaluated on PickleBench. The authors report full scanning coverage and low false-positive and false-negative rates on that benchmark.

  • Problem

    Deserializing an untrusted PTM can trigger attacker-controlled behavior, including arbitrary code execution, data exfiltration, or file manipulation.

    Source: §I · p. 1

  • Method

    DITTO emulates Pickle virtual-machine state transitions and applies context-aware semantic analysis to retained security-critical states to assess model intentions.

    Source: §I · p. 2, B. DITTO Design · p. 5, 2) Intention Analyzer · p. 7

  • Results

    On PickleBench, DITTO achieved 100% scanning coverage, 0% FNR, 0.7% FPR, and a 0.966 F1 score, outperforming baselines.

    Source: §I · p. 2, §IX · p. 13

  • Takeaways & Limitations

    DITTO provides practical reports and auditable evidence intended to support human-in-the-loop security audits of untrusted PTMs.

    Source: §I · p. 2, §IX · p. 13

    Highly sophisticated, model-specific adversarial attacks may still challenge reliable semantic analysis.

    Source: B. Limitations and Mitigations · p. 12

Abstract

from arXiv · show

Pre-trained models (PTMs) are widely distributed as serialized binaries, but their reuse often exposes software supply chains to deserialization attacks. Despite the emergence of safer serialization formats, the unsafe Pickle format remains prevalent: our analysis of over 10,000 popular Hugging Face repositories reveals that 9.3% rely on Pickle. While many defense mechanisms have been proposed, state-of-the-art model scanners suffer from a coverage-precision gap, missing security-sensitive behaviors and generating excessive false alerts. In this paper, we introduce DITTO, the first stack-based, context-aware scanner for Pickle-based PTMs. DITTO faithfully tracks Pickle virtual machine state transitions and performs context-aware semantic analysis to infer model intentions. We also present PickleBench, a benchmark of 959 benign and 92 malicious real-world models, including extension registry attacks previously missed by existing tools. Across multiple evaluations, DITTO achieves 100% scanning coverage, a 0% false-negative rate, and a 0.7% false-positive rate, yielding an F1 score of 0.966, significantly outperforming state-of-the-art scanners. By minimizing false alerts while preserving detection accuracy, DITTO generates actionable security reports with contextual evidence, enabling safe PTM reuse and strengthening software supply chain integrity.

I. INTRODUCTION

DITTO addresses the risk of malicious behavior during Pickle-based model loading with a context-aware scanner and reports strong PickleBench results. The section also describes Pickle’s prevalence, loading process, and security-sensitive operations.

I. INTRODUCTION

  • Pickle remains in use despite safer formats: 9.3% of 10,023 popular Hugging Face repositories contained 9,544 Pickle-based PTMs across 929 repositories.The identified models include highly downloaded PTMs and models published by major organizations.Source: §I · p. 1
  • DITTO combines framework-consistent Pickle tracing without executing security-sensitive operations and context-aware semantic analysis of retained states to judge whether behavior is legitimate.Its trace generator emulates the stack-based Pickle virtual machine; its intention analyzer examines security-critical states and their usage context.Source: §I · p. 2
  • On PickleBench, DITTO achieved 100% scanning coverage, 0% false-negative rate, 0.7% average false-positive rate, and 0.966 F1 versus the best existing scanner’s 0.776 F1.It successfully scanned all models and detected all malicious instances across three independent runs.Source: §I · p. 2
  • PickleBench contains 959 benign and 92 malicious models, including extension registry attacks that the authors found existing scanners had not examined.The benchmark supports systematic comparisons of DITTO with state-of-the-art scanners.Source: §I · p. 2

A. PTM Serialization and Loading

B. Pickle Deserialization and Security Risks

  • Pickle attacks can invoke an imported callable such as posix.system or alter copyreg registry state so a later extension opcode resolves to os.system.The security implication depends on preceding virtual-machine state changes and how the imported global is subsequently used.Source: B. Pickle Deserialization and Security… · p. 3

III. MOTIVATION

The authors examine recent Pickle use and existing defenses, identifying continued use among popular models and limitations in both restricted loaders and scanners.

A. Recent Pickle-based PTM Usage Trends

  • The study identifies 9,544 Pickle-based PTMs across 929 of 10,023 popular Hugging Face repositories, or 9.3%.The authors verified suspected files lacking valid scan results using format-appropriate checks.Source: A. Recent Pickle-based PTM Usage Trends · pp. 3–4

1) Existing Pickle Defenses:

  • Restricted loaders can reject benign models that require globals outside their allowlists, while scanners flag globals based largely on identity.PickleBall failed to load 51 (20.2%) of 252 benign models in its evaluation, including 36 (70.6%) of 51 YOLO models.Source: 1) Existing Pickle Defenses · p. 4
  • The authors identify three scanner limitations: incomplete behavior capture, context-insensitive risk assessment, and impractical analysis context.They focus on scanners because they allow pre-loading assessment while preserving compatibility with standard framework loading.Source: 1) Existing Pickle Defenses · p. 4

2) Incomplete Deserialization Behavior Capture:

The authors identify failures in behavior capture, context-insensitive risk assessment, and the amount of context scanners provide. These findings motivate faithful state tracking and focused contextual reasoning.

2) Incomplete Deserialization Behavior Capture:

  • All scanners missed the loading-path bypass examples, and Fickling missed both extension-registry attacks; ModelScan scanned those attacks but missed their malicious behavior.PickleScan flagged an unknown copyreg import rather than resolving the eventual security-sensitive os.system use.Source: 2) Incomplete Deserialization Behavior… · pp. 4–5

3) Context-Insensitive Risk Assessment:

  • PickleScan falsely flagged 177 of 697 benign models, while global identity alone could not distinguish legitimate reconstruction from attack behavior.The same callable, builtins.getattr, can be used legitimately in model reconstruction or participate in attacks.Source: 3) Context-Insensitive Risk Assessment · p. 5

4) Impractical Analysis Context:

  • Context-free alerts omit usage details, while full traces bury security-critical information among routine reconstruction operations.The authors argue for retaining security-critical context while suppressing redundant reconstruction information.Source: 4) Impractical Analysis Context · p. 5

IV. APPROACH

DITTO scans Pickle-based PTMs through the intended framework loading path, safely emulates security-sensitive Pickle operations, and analyzes their state and usage context. Its threat model assumes users know the intended loader and that their system and trusted framework are uncompromised before loading.

A. Threat Model

  • DITTO assumes an unverified Pickle-based PTM may contain attacker-controlled behavior, while the user knows its intended loading function and the system and trusted framework are initially uncompromised.The attacker may publish a malicious model on a public hub but has no prior access to the user's system or DITTO.Source: A. Threat Model · p. 5

B. DITTO Design

  • DITTO pairs a trace generator with an intention analyzer to assess Pickle-based PTMs without directly executing security-sensitive Pickle operations.The trace generator captures security-sensitive PVM transitions; the analyzer extracts relevant context and reasons about its usage.Source: B. DITTO Design · p. 5, Figure 2 · p. 6

1) Trace Generator:

  • The trace generator follows the intended framework loader, safely emulates routine PVM reconstruction, and represents sensitive operations symbolically rather than executing them.Symbolic states preserve the operations, operands, and dependencies that explain how security-sensitive values are imported, transformed, invoked, or altered.Source: 1) Trace Generator · p. 6
  • For EXT1, EXT2, and EXT4, DITTO tracks registry-altering states and retains the resulting import as unresolved to preserve dependencies on mutable registry state.This avoids resolving an extension import against an unaltered registry.Source: 1) Trace Generator · p. 6

2) Intention Analyzer:

DITTO extracts security-critical state dependencies from deserialization traces, then reasons about how retained objects are used. In a running example, it identifies an eval import used on attacker-controlled code as likely malicious.

2) Intention Analyzer:

  • DITTO extracts the dependency closure of security-sensitive and unresolved states, suppressing routine reconstruction states rooted exclusively in allowlisted objects.The resulting focused context retains predecessor states needed to explain provenance and subsequent usage while omitting unrelated states.Source: 2) Intention Analyzer · pp. 6–7
  • Rather than judging a global by identity alone, the analyzer considers its arguments, targets, invocations, alterations, derived states, and dependencies to distinguish reconstruction from malicious behavior.A common interface lets different semantic analyzers use the extracted context, and assessments include concise reasoning grounded in retained states.Source: 2) Intention Analyzer · p. 7

C. Running Example

  • In the running example, DITTO reduces a 258-state, 42 KB trace to two security-critical states totaling 8 KB, then classifies eval invoked on attacker-controlled code as likely malicious.The analyzer preserves the import's dependencies and subsequent usage as evidence for its assessment.Source: C. Running Example · p. 7, Figure 3 · p. 8

V. IMPLEMENTATION

  • The implementation supports loading functions from six trusted ML-related modules and uses a configurable intended loader when suffix-based mapping may be unreliable.Its default semantic analyzer is DeepSeek-V4-Flash-0731, invoked through the DeepInfra API with temperature set to zero.Source: §V · p. 7

VI. EVALUATION

The evaluation compares DITTO with three Pickle scanners on PickleBench, a curated benchmark of benign and malicious real-world models. DITTO scans the full benchmark and reports strong detection with few false alerts, at higher per-model execution time.

VI. EVALUATION

  • The evaluation asks about end-to-end effectiveness, context-aware analysis, and the effects of state extraction and semantic-analyzer selection.It compares DITTO with ModelScan, PickleScan, and Fickling, using scanning coverage, FPR, FNR, F1, and per-model execution time.Source: §VI · p. 8, A. Evaluation Setting · p. 9

A. Evaluation Setting

  • PickleBench contains 1,051 Pickle-based PTMs: 959 benign and 92 malicious models curated from two sources and validated before inclusion.The malicious samples cover attacks including command execution, network connection, file manipulation, and data exfiltration.Source: A. Evaluation Setting · p. 8

1) Scanning and Detection Effectiveness:

  • 100% scanning coverage, 0% FNR, 0.7% average FPR, and F1 score 0.966 are reported for DITTO, versus at most 0.776 F1 for evaluated baselines.DITTO detects all 92 malicious PTMs; it averages seven false positives across three runs. The comparison is reported in Table V.Source: Table V · p. 9, 1) Scanning and Detection Effectiveness · p. 9
  • DITTO averages 4.211 seconds per PTM, compared with 0.162–0.247 seconds for baseline scanners, trading longer analysis for broader coverage and more accurate assessments.The reported time includes loading, tracing, semantic analysis when applicable, and result generation.Source: 1) Scanning and Detection Effectiveness · p. 9, A. Evaluation Setting · p. 9

2) Behavioral Coverage:

DITTO uses state dependencies and subsequent usage to distinguish benign reconstruction from attacks involving security-sensitive globals. Its remaining false positives stem from analyzer inconsistency or insufficient context.

2) Behavioral Coverage:

  • DITTO detects extension-registry attacks by tracking registry state dependencies and interpreting the later indirect import, while PickleScan flags an unknown copyreg import.The attacks establish registry state before triggering a later import of os.system.Source: 2) Behavioral Coverage · p. 9
  • 100% scanning coverage, detection of all 92 malicious PTMs, and 0.7% average FPR are reported for DITTO on PickleBench.The RQ1 summary reports these outcomes alongside the cost of higher analysis time.Source: 2) Behavioral Coverage · p. 10

1) Context-Aware Decisions:

  • Benign uses of getattr for retrieving a YOLO11 forward method or enum members are classified as likely benign because their contexts show ordinary reconstruction, not subsequent system-level behavior.These are representative cases of baseline scanners flagging models that DITTO correctly classifies.Source: 1) Context-Aware Decisions · p. 10, Table VI · p. 10

2) Residual False-Positive Analysis:

  • Of 20 false-positive reports across three runs, 7 (35%) involve analyzer inconsistency and 13 (65%) involve insufficient context for a confident benign assessment.The manual analysis identifies these as distinct sources of DITTO's residual false positives.Source: 2) Residual False-Positive Analysis · p. 10, Table VII · p. 11

D. RQ3: Ablation and Sensitivity

  • The sensitivity study compares DITTO without state extraction and with GPT or Gemini analyzers; Table VIII frames context size before and after extraction as a reduction in the average value.The supplied passages identify the study variants and table scope but provide no outcome values.Source: D. RQ3: Ablation and Sensitivity · p. 10, Table VIII · p. 11

1) State-Extraction Ablation:

State extraction sharply reduces analyzer context while improving scanning effectiveness and efficiency; changing the semantic analyzer can also materially change classification outcomes. The timing comparison among analyzers reflects serving infrastructure as well as inference.

1) State-Extraction Ablation:

  • 88.9%: state extraction cuts average analyzer input from 255,003 to 28,414 tokens, while reducing average state count by 86.5% and average size by 86.4%.For the largest traces, token count falls from approximately 11.1 million to 195,755.Source: 1) State-Extraction Ablation · p. 11
  • Without extraction, false positives rise from 7 to 50, F1 falls from 0.966 to 0.785, and scanning coverage drops from 100% to 99.2%.The variant retains zero false negatives, but analysis cannot be completed for eight PTMs.Source: Table IX · p. 11, 1) State-Extraction Ablation · p. 11
  • 4.211 s to 6.750 s: removing extraction increases total scanning time, while average semantic-analysis time rises from 4.038 s to 6.582 s.The execution-time table reports trace-generation and semantic-analysis breakdowns.Source: Table X · p. 11, 1) State-Extraction Ablation · p. 11

2) Semantic-Analyzer Sensitivity:

  • Analyzer choice changes classification: GPT reduces false positives from 7 to 3 but adds 24 false negatives and lowers F1 to 0.830, while Gemini achieves F1 1.000.The reported timing differences also reflect model-serving infrastructure, not just inference.Source: 2) Semantic-Analyzer Sensitivity · pp. 11–12

VII. DISCUSSION

DITTO combines deterministic security-critical state traces with semantic assessments to support human-in-the-loop audits, while acknowledging analyzer and context-window limits. The discussion situates the scanner among prior defenses, reports its evaluation, and describes ethical safeguards and privacy considerations.

A. Security Guarantees:

  • DITTO tracks security-sensitive operations without executing them, retains relevant variables through state extraction, and supplies compact transitions alongside semantic reasoning for human audits.The authors note that no exploit chaining predefined safe globals from PyTorch’s weights_only list had been discovered.Source: A. Security Guarantees · p. 12

B. Limitations and Mitigations:

  • Semantic assessments inherit analyzer stochasticity, may be inconsistent, and could be vulnerable to sophisticated model-specific prompt-injection attacks despite a successful preliminary test.DITTO provides deterministic state transitions to help users verify uncertain verdicts.Source: B. Limitations and Mitigations · p. 12
  • Exceptionally complex models can still produce extracted contexts of approximately 195,000 tokens, approaching standard LLM context-window limits despite no observed overflow in the evaluation.The authors suggest hierarchical trace summarization or more aggressive pruning as future work.Source: B. Limitations and Mitigations · p. 12

VIII. RELATED WORK:

  • Prior scanners face utility or coverage gaps: restricted loaders can block benign models, while Fickling’s AST emulation ignores state transitions and crashes on extension-registry manipulations.DITTO instead analyzes Pickle security through critical state transitions.Source: §VIII · p. 13

IX. CONCLUSION:

  • On PickleBench, DITTO reports 100% scanning coverage, 0% FNR, 0.7% FPR, and F1 0.966, and provides contextual reports for security audits.The conclusion also describes balancing effectiveness, efficiency, analysis cost, and privacy preservation.Source: §IX · p. 13

ETHICAL CONSIDERATIONS:

  • The study used public Hugging Face data, avoided collecting private user data, disclosed identified vulnerabilities, and tested only non-destructive payloads in isolated environments.Remote semantic-model use can raise confidentiality concerns for proprietary or sensitive PTMs, so users should consider provider data-handling policies.Source: ETHICAL CONSIDERATIONS · p. 13

APPENDIX

The appendix documents the repository-study formats and supplementary procedures for generating malicious loading-path examples. Its examples cover a PyTorch TAR payload and compressed files handled through joblib.

APPENDIX

  • The appendix includes investigated formats, generation procedures for additional malicious examples, and implementation details for loader mappings and the allowlist.Source: APPENDIX · p. 15

A. INVESTIGATED FORMATS

  • The repository study began with 13 suffixes matching Pickle-based PTM formats identified by Liu et al. [23], but excluded .keras because Keras no longer supports the stated allow_pickle=True loading configuration.Source: A. INVESTIGATED FORMATS · p. 15

B.1 Model Loading Path

  • Seven malicious files package an os.system payload invoked with REDUCE into PyTorch TAR and six compression formats loadable by joblib; all deserialize and print “Yippie!”.The appendix shows the exploit class and TAR member construction used to build the examples.Source: §B.1 · p. 15

B.2 Extension Registry Exploits

Pickle extension opcodes resolve globals through the manipulable `_inverted_registry`, creating direct and indirect paths to inject attacker-chosen globals. The paper demonstrates both paths by importing and invoking `os.system` during deserialization.

B.2 Extension Registry Exploits

  • EXT1, EXT2, and EXT4 use extension codes to resolve globals through `copyreg._inverted_registry`, which is empty by default but can be manipulated during deserialization.The opcodes then import the global identified by the registry’s module-name pair.Source: §B.2 · p. 15
  • Direct injection obtains `_inverted_registry` with GLOBAL and inserts code 42 mapped to the module-name pair (“os”, “system”).The example’s pickle instructions encode the registry lookup and the values inserted.Source: §B.2 · p. 15
  • Indirect injection obtains `copyreg.add_extension` with GLOBAL and invokes it with the tuple (“os”, “system”, 42).The example’s instructions encode the function lookup and tuple values.Source: §B.2 · p. 15
  • Both examples inject an `(1, os.system)` mapping, import `os.system` with EXT1 using key 1, and invoke it with REDUCE in the native Python PVM.Both examples print a “Yippie!” message when executed.Source: §B.2 · p. 15

7 # Exploit _inverted_registry

The appendix describes malicious-file generation for extension-registry and malformed-Pickle exploitation, and documents loading-function mappings and DITTO’s safe-global allowlist.

12 # Exploit malformed Pickle

  • The listing generates malicious files for extension-registry and malformed-Pickle exploitation by loading byte values and writing them as exploitation{k}.pkl files.The code iterates over exp, calls pickle.load on each value wrapped in BytesIO, and writes each value to a file.Source: §12 · p. 15

C.1 Loading Function Mappings

  • Six default loading functions map to PTM extensions, and users may specify a loading function based on their knowledge.The mappings are predefined based on PTM extensions.Source: §C.1 · p. 16, Table XII · p. 16

C.2 Allowlisted Globals

  • DITTO’s allowlist filters benign state changes using safe globals collected with PyTorch’s _get_allowed_globals() function.Table XIII enumerates the safe callables in the allowlist.Source: §C.2 · p. 16, Table XIII · p. 17
Loading 2610.10735v1…