PaperlayerView exact revision on arXiv

Source-linked AI summary

Vulnerable Models in the AI Supply Chain: An Empirical Study of Adoption, Propagation, and Usage Patterns

The Anh Nguyen, Triet Huynh Minh Le, M. Ali Babar

arXiv:2610.12893v1cs.CRcs.SE

TL;DR

The paper addresses the limited evidence about whether and how developers integrate vulnerable models downstream of model registries. It studies 16,977 flagged models across software and deployment contexts, finding widespread references and recurring usage motifs without observed sanitization or provenance verification.

  • Problem

    Prior research has focused on model-file vulnerabilities and hosting-platform prevalence, leaving downstream adoption, propagation, and integration patterns in developer codebases largely unknown.

    Source: §1 · p. 1

  • Method

    The authors analyze 16,977 vulnerable models across code, packages, fine-tuning chains, and deployed applications, and mine CPGs for recurring usage patterns.

    Source: §1 · p. 1, §4.1 · p. 4, §4.2 · p. 6

  • Results

    Over 21,000 code occurrences across more than 5,000 repositories accompany propagation through 419 fine-tuned derivatives, 185 packages, and over 1,200 deployed applications.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    Across the four usage motifs, the authors find no evidence of model sanitization or cryptographic provenance verification, indicating that artifact-level scanning alone is insufficient.

    Source: §1 · p. 2

    Code-level references approximate potential exposure but do not establish runtime loading or execution, and dynamically constructed identifiers may be missed.

    Source: §6.3 · p. 13

Abstract

from arXiv · show

The rapid integration of Artificial Intelligence (AI) models has established a complex supply chain, but many of these shared artifacts harbor severe vulnerabilities such as insecure serialization and architectural backdoors. While prior efforts have extensively scanned model registries to flag malicious artifacts, a critical gap remains in understanding how developers actually adopt and integrate these unsafe models into downstream software systems. We bridge this gap by conducting a large-scale empirical study on the usage and propagation of 16,977 known vulnerable models on HuggingFace flagged by InsightDB. We track their prevalence across GitHub repositories, software packages, fine-tuning chains, and deployed applications. Our analysis reveals that, despite being publicly flagged, these vulnerable models appear in over 21,000 code locations across more than 5,000 open-source repositories, with 3,757 distinct flagged models (22.1\%) actively referenced in at least one project. To understand how these models are integrated in code and identify recurring usage patterns, we categorize and analyze the recurring patterns surrounding them via a customized pipeline that combines Code Property Graphs extraction with a tailored Frequent Subgraph Mining approach. Across all patterns, we find no evidence of provenance verification or model sanitization logic, with developers frequently bypassing security parameters or delegating trust to permissive framework defaults. These findings underscore that scanning model files alone is insufficient, highlighting the urgent need for context-aware security analysis and stricter integration guardrails at the application layer of the AI supply chain.

1. Introduction

The paper shifts attention from identifying vulnerable model artifacts to how developers adopt and integrate them downstream. It finds widespread use, recurring integration motifs, and no evidence of sanitization or provenance verification across those patterns.

1. Introduction

  • The study examines 16,977 known vulnerable Hugging Face models across GitHub code, packages, fine-tuning chains, and deployed Spaces.It combines Code Property Graph extraction with custom normalization and constrained Frequent Subgraph Mining using the Top-K Graph algorithm.Source: §1 · p. 1
  • Over 21,000 code occurrences across more than 5,000 repositories show widespread adoption of publicly flagged vulnerable models.The study also reports propagation through 419 fine-tuned derivatives, 185 software packages, and over 1,200 deployed applications.Source: §1 · pp. 1–2
  • A data-driven mining approach surfaces recurring propagation structures and candidate sinks directly from code, reducing manual source-sink specification.The authors present it as a foundation for future vulnerability-query design.Source: §1 · p. 2
  • The observed integrations fall into four recurring motifs: Direct Usage, Local Propagation, Method Propagation, and Condition-Guarded Propagation.Source: §1 · p. 2
  • Across all extracted patterns, the authors find no evidence of model sanitization or cryptographic provenance verification.Developers delegate security to permissive framework defaults or explicitly bypass available safeguards.Source: §1 · p. 2

2. Related Work and Motivation

Prior work examines vulnerabilities in model artifacts and hosting platforms, model supply-chain risks, and code-analysis techniques. The paper identifies downstream integration and the discovery of unknown usage patterns as remaining research gaps.

2.1. Security Threats in AI models

  • Prior work studies artifact risks including unsafe deserialization, architectural backdoors, and malicious configurations, while platform studies measure threats but provide limited insight into downstream integration.The motivating example, lxyuan/distilbert-student, appears in code, fine-tuning, deployment, and production-container contexts despite being flagged for pickle-related arbitrary code execution.Source: §2.1 · p. 2

2.2. The AI Model Supply Chain Ecosystem

  • Supply-chain studies examine model reuse, provenance, dependency management, and lifecycle risks, but the paper reports no prior systematic study using security advisories to analyze vulnerable-model adoption and usage patterns in developer codebases.HF security signals, including ProtectAI and JFrog advisories, provide a basis for studying this propagation.Source: §2.2 · p. 3

2.3. AI Vulnerabilities Analysis Techniques

  • Code Property Graphs combine syntax with control- and data-flow structure, while Frequent Subgraph Mining can discover recurring code patterns without predefined sinks.The paper adapts TKG with custom CPG normalization and a constraint anchoring patterns to vulnerable model identifiers.Source: §2.3 · p. 3

3. Research Questions

The study asks how widespread vulnerable-model use is and what recurring code patterns accompany it. The questions address ecosystem exposure and the source-code integrations relevant to mitigation.

3. Research Questions

  • RQ1 measures flagged-model exposure through direct code references, software packages, fine-tuning lineage, and deployed applications.Source: §3 · pp. 3–4
  • RQ2 identifies recurring motifs through which vulnerable models are introduced, propagated, and consumed by sink APIs, then examines their security-relevant risks.Source: §3 · p. 4

4. Methodology

The methodology collects flagged-model and ecosystem data, quantifies exposure across four supply-chain dimensions, and mines normalized code graphs for recurring integration patterns. The resulting patterns are simplified and grouped into an interpretable motif taxonomy.

4. Methodology

  • The study combines vulnerability-report collection and ecosystem tracing with separate analyses of exposure (RQ1) and code usage patterns (RQ2).The authors state that processing logic, dataset, and source code are available in a replication package.Source: §4 · p. 4

4.1. Data Collection

  • The curated dataset contains 16,977 unique model identifiers from ProtectAI Insights DB, spanning deserialization, backdoor, and runtime threats.GitHub Code Search locates repositories with exact identifier matches; HF APIs provide fine-tuning metadata and identify Spaces referencing the models.Source: §4.1 · p. 4

4.2. Data Analysis and Pattern Extraction

  • Exposure analysis covers code references, package distribution, fine-tuning chains reconstructed from base-model tags, and publicly accessible HF Spaces.These dimensions capture direct integration, redistribution, model lineage, and application-level exposure, respectively.Source: §4.2 · p. 4
  • For pattern mining, the authors restrict analysis to non-test Python files, replace vulnerable identifiers with a uniform source literal, and generate and normalize CPGs.The corpus contains 1,156 candidate files across 817 repositories; normalization removes noise while retaining program semantics for analysis.Source: §4.2 · pp. 4–5
  • The tailored TKG mining process anchors every discovered subgraph to an unsafe-model-name node and uses k = 50 and a 50-edge maximum for subsequent analysis.The authors report that this configuration captured the pattern search space, with no patterns exceeding 48 edges.Source: §4.2 · p. 6
  • Post-processing removes patterns isomorphic to subgraphs of larger patterns, abstracts leaf labels into wildcards, and groups structures by their dominant semantic features.The authors then classify recurring structures into a motif taxonomy, excluding patterns containing only CFG or AST edges.Source: §4.2 · pp. 6–7

5. Results

Vulnerable models appear across repositories, packages, fine-tuning chains, and deployed applications. The study identifies four recurring integration motifs and finds that vulnerable load paths often lack safeguards or provenance checks.

5.1. Vulnerable Model Adoption and Propagation

  • 21,313 code occurrences span 5,040 repositories, with 3,757 (22.1%) of 16,977 flagged models referenced on GitHub.Exposure also reaches 185 packages, 419 fine-tuned derivatives, and 1,239 Spaces; lxyuan/distilbert-student spans all four supply-chain dimensions.Source: §5.1 · pp. 7–8

5.2. Integration Motifs and Security Implications

  • Mining yielded 809 patterns and 195 maximal patterns, grouped into four motifs: Direct Usage, Local Propagation, Method Propagation, and Condition-Guarded Propagation.The motifs distinguish direct argument binding, local data flow, interprocedural flow, and branch-governed use.Source: §5.2 · p. 8

5.2.1. Motif 1 (M1): Direct Usage.

  • M1 passes vulnerable model identifiers directly to sink APIs; it appears across 11 frameworks, with highest support in Transformers and HF Hub.The two reported highest supports are 121 occurrences in Transformers and 66 in HF Hub.Source: §5.2.1 · p. 8

5.2.2. Motif 2 (M2): Local propagation.

  • M2 spans 12 frameworks; local staging and aliasing obscure unsafe identifiers, requiring intra-procedural data-flow reconstruction to trace them to sinks.Its highest support is in Transformers (115), Sherpa (76), and HF Hub (60).Source: §5.2.2 · pp. 9–10

5.2.3. Motif 3 (M3): Method propagation.

  • M3 routes identifiers through application-defined methods, distributing flows across call sites and requiring reconstruction of interprocedural source-to-sink paths.It appears mainly in Ultralytics (5 occurrences) and Diffusers (3).Source: §5.2.3 · p. 10

5.2.4. Motif 4 (M4): Condition-Guarded Propagation.

  • M4 occurs across five frameworks, but its branches select or trigger loading rather than validate model identity or integrity.A checkpoint example uses torch.load with weights_only=False, explicitly disabling the primary available safeguard; exploitability depends on runtime execution and the artifact loaded.Source: §5.2.4 · pp. 10–11

5.2.5. Cross-framework dependency.

  • Framework-specific integrations often converge on shared retrieval and loading APIs, creating potential defensive leverage in hardening those sinks.The cited frameworks include Keras, TensorFlow, Ultralytics, Diffusers, ONNX, and Joblib.Source: §5.2.5 · p. 11

5.2.6. Vulnerability-relevant Usage Analysis.

  • 553 of 554 vulnerability-relevant load paths (99%) matched the referenced models’ vulnerability conditions; 533 were deserialization paths and 20 were runtime paths.The analysis considered a usage vulnerable when a flagged model was loaded through a compatible susceptible path without corresponding safeguards, or with unsafe behavior enabled.Source: §5.2.6 · p. 11

6. Discussion

The discussion finds that fragmented provenance limits visibility into vulnerable models’ cross-layer propagation, while integration patterns commonly lack local trust enforcement. It recommends downstream tracing and integration-layer safeguards, while noting that the findings are bounded by detection and ecosystem scope.

6.1.1. Ecosystem-level provenance fragmentation.

  • Vulnerable models can span code, packages, derivatives, and deployed applications, but disconnected metadata and tooling leave measured exposure a likely lower bound.The authors recommend machine-readable provenance and vulnerability records that track model relationships across reuse surfaces.Source: §6.1.1 · pp. 11–12

6.1.2. Lack of security and provenance tooling during integration.

  • Across motifs, models reach loading APIs without explicit local identity or integrity checks, reflecting enforcement displaced to frameworks and often-optional safeguards.The authors characterize this as provenance-blind integration; the motifs point to different mitigation needs, including stronger sink defaults, data-flow analysis, interprocedural tracing, and not treating branch logic as trust enforcement.Source: §6.1.2 · pp. 12–13

6.2. Recommendations

  • FSM can complement CPG workflows by surfacing recurring propagation structures and candidate sinks without exhaustive sink enumeration.In this study, FSM and VF2 account for 30% of the 20-hour end-to-end runtime.Source: §6.2 · p. 13
  • The authors recommend moving security decisions closer to model loading through explicit settings, surfaced advisory signals, hardened shared APIs, and machine-readable vulnerability records.They also identify motifs as a basis for static-analysis rules, CI checks, IDE warnings, and automated remediation.Source: §6.2 · p. 13

6.3. Limitations

  • Prevalence estimates use code references rather than runtime analysis, so they neither confirm execution nor capture indirect usages such as dynamically constructed identifiers.The authors therefore interpret these measurements as potential exposure, not confirmed exploitation.Source: §6.3 · p. 13
  • The study covers models flagged by HuggingFace scanners across GitHub and HuggingFace, excluding other platforms and threats outside scanner coverage.Patterns may also differ in proprietary frameworks, private registries, or languages other than Python.Source: §6.3 · p. 13

7. Conclusion

The conclusion reports widespread vulnerable-model use and propagation, identifies four integration motifs, and argues that artifact scanning alone is insufficient without application-layer defenses.

7. Conclusion

  • The study found 21,000 references in more than 5,000 repositories, propagation through packages, fine-tuning chains, and deployed applications, and four recurring integration motifs.The motifs are Direct Usage, Local Propagation, Method Propagation, and Condition-Guarded Propagation.Source: §7 · p. 14

Appendix A. Ethical Considerations

The ethics appendix describes the use of publicly accessible data and the study’s intended benefits, while acknowledging possible reputational and dual-use harms.

Appendix A. Ethical Considerations

  • The study analyzes publicly accessible sources and uses named instances to illustrate patterns rather than assign fault to developers.The authors state that the study involved no human subjects.Source: Appendix A · p. 17
  • The authors identify possible reputational harm and misuse of findings, and say they mitigate these risks by using already-disclosed vulnerabilities and omitting exploit instructions.They state that the analysis provides generalized structural patterns rather than functional exploit payloads.Source: Appendix A · p. 17
  • The authors decided to publish because they judged the benefits to outweigh the harms and expected the work to support improved provenance tooling, auditing, and safer integration.Source: Appendix A · p. 17

Appendix B. Meta-Review

The passage identifies a program-committee meta-review prepared for the 2027 IEEE Symposium on Security and Privacy as part of its review process.

Appendix B. Meta-Review

  • A program-committee meta-review was prepared for the 2027 IEEE Symposium on Security and Privacy.Source: Appendix B · p. 18
  • The passage says the meta-review forms part of the symposium’s review process.Source: Appendix B · p. 18
  • The passage points to the call for papers for details of that review process.Source: Appendix B · p. 18

B.1. Summary

The paper studies adoption, propagation, and integration patterns of vulnerable HuggingFace models across software ecosystems, using CPG analysis and FSM to identify recurring usage patterns.

B.1. Summary

  • The study analyzes 16,977 vulnerable HuggingFace models identified by InsightDB.It traces their usage across GitHub repositories, software packages, fine-tuning workflows, and deployed applications.Source: §B.1 · p. 18
  • The analysis combines Code Property Graph analysis with Frequent Subgraph Mining to identify recurring integration patterns.Source: §B.1 · p. 18
  • The study examines vulnerable-model adoption, propagation, and usage patterns in the AI supply chain.Source: §B.1 · p. 18

B.2. Scientific Contributions •

The listed scientific contributions are framed around confirming important results, providing a dataset and tool, addressing a longstanding issue, identifying an impactful vulnerability, and advancing an established field.

B.2. Scientific Contributions •

  • The contribution list claims independent confirmation of important results despite limited prior research.Source: §B.2 · p. 18
  • It lists providing a new dataset for public use as a contribution.Source: §B.2 · p. 18
  • It lists creating a new tool to enable future science as a contribution.Source: §B.2 · p. 18
  • It frames the work as addressing a long-known issue and identifying an impactful vulnerability.Source: §B.2 · p. 18
  • It characterizes the work as a valuable step forward in an established field.Source: §B.2 · p. 18

B.3. Reasons for Acceptance

The paper is presented as an empirical complement to registry-focused research, showing how flagged models are used and propagated in downstream software.

B.3. Reasons for Acceptance

  • The study empirically examines adoption, propagation, and integration of models already flagged by security scanners.Source: §B.3 · p. 18
  • Its downstream software perspective complements prior work focused primarily on model registries.Source: §B.3 · p. 18
  • The acceptance rationale presents this perspective as a valuable step forward in AI supply-chain security.Source: §B.3 · p. 18

B.4. Noteworthy Concerns

The concerns center on limited evaluation of the FSM methodology, similarity between usage-analysis rules and detection patterns, and absent exploitability evidence.

B.4. Noteworthy Concerns

  • The FSM methodology is not sufficiently evaluated against simpler alternatives or its claimed reduction in manual effort.Source: §B.4 · p. 18
  • The rule-based vulnerability-usage analysis appears to use rules similar to those in vulnerable-model detection.Source: §B.4 · p. 18
  • The paper provides no evidence that the identified vulnerable model usages are exploitable, so results require this limitation's context.Source: §B.4 · p. 18
Loading 2610.12893v1…