Source-linked AI summary
CASHEWS: Source Preprocessor for LLM-based Malicious Package Detection
Jean-Charles Noirot Ferrand, David Adei, Anders Møller, Alexandros Kapravelos
TL;DR
LLM-based malicious-package detectors can struggle with large, obfuscated, or bundled JavaScript because context limits impede analysis and benign obfuscation can mislead verdicts. CASHEWS preprocesses source to uncover behavior and reduce the code sent to detectors. Across its evaluation, it improved coverage, reduced false negatives, and lowered net analysis cost, while adaptive attacks may still evade its processing.
Problem
Method
CASHEWS iteratively decodes JavaScript, extracts embedded code and bundled modules, slices code reaching security-sensitive sinks, and abbreviates large literals and identifiers.
Source: §1 · p. 2Results
98.8–100% coverage, up to 18.6 percentage points lower false-negative rate, and 34.6% net cost saving were reported with CASHEWS.
Source: §1 · p. 2Takeaways & Limitations
CASHEWS improves LLM-based detector coverage and scalability for npm packages when adversaries exploit context-window limits.
Source: §9 · p. 13Adaptive adversaries may craft payloads that evade preprocessing, including malicious code that avoids supported sinks while benign code uses them.
Source: §7.1 · p. 12
Abstract
from arXiv · showhide
Malicious npm package detection tools now leverage LLMs' semantic understanding of source code to detect malicious intent at scale. This capability has proven invaluable in identifying packages involved in recent supply-chain attacks such as Shai-Hulud. However, threat actors exploit the limited context windows of LLMs through JavaScript techniques such as code obfuscation that yields high token density and bundling malicious code with benign packages, causing detectors to skip large files or miss malicious behavior. This creates an attack surface for evading detection. In this paper, we present CASHEWS, a JavaScript preprocessor that reduces file size by rewriting source code to remove code that is irrelevant to analysis or likely to mislead the model. Given a package source file, CASHEWS deobfuscates it through iterative decoding, extracts bundled modules and dynamically executed code, identifies malicious sinks and computes backward slices that reach them, and abbreviates long literals and identifiers to produce a compact representation for the detector. Across 512 large package files, two scanner types, and three LLMs, CASHEWS increases analysis coverage from 69.1--85.7% to 98.8--100% and reduces the false-negative rate by up to 18.6 percentage points. CASHEWS also has a median preprocessing time of 30 seconds while reducing net analysis cost by 34.6%, making registry-wide LLM-based analysis more practical. By preprocessing source code before analysis, CASHEWS enables researchers and industry practitioners to use more powerful models for malicious package detection at the same or lower analysis cost as less powerful models.
1 Introduction
LLM-based malicious-package detectors can miss large or obfuscated files because of context limits, while benign obfuscation can also mislead them. CASHEWS preprocesses JavaScript to expose relevant behavior and reduce tokens; its evaluation improves coverage, false-negative rates, and analysis cost.
1 Introduction
- The introduction situates the problem amid 7,197 published malware advisories in 2025, a 69% increase over 2024, most of which it attributes to Shai-Hulud.The passage reports that the campaign’s first wave reached more than 1,150 packages and a second wave reached 796 more.Source: §1 · p. 1
- CASHEWS iteratively decodes code, extracts embedded code and modules, slices code flowing to sinks, and abbreviates large literals and identifiers.Its decoding aims to uncover behavior rather than reconstruct the exact pre-obfuscation source.Source: §1 · p. 2
- On 512 files exceeding 25,000 tokens, existing workflows analyze only 69.1–85.7% of inputs.Large files can exceed context windows, and obfuscation can raise token counts or hide behavior.Source: §1 · pp. 1–2, Figure 1 · p. 1
- 98.8–100% coverage, up to 18.6 percentage points lower false-negative rate, and 34.6% net cost saving were reported with CASHEWS.The evaluation also reports a 30 percentage point coverage improvement and median preprocessing time of 30.2 seconds.Source: §1 · p. 2
2 Background
Tokenization and context-window limits shape which source code LLM-based detectors can analyze. Obfuscation and bundling can make code harder or costlier to analyze by increasing token density or surrounding package-specific logic with dependencies.
2.1 Tokenization and Context Windows
- A context window must fit instructions, chat history, source code, and auxiliary prompt information, so oversized files may be split, truncated, or skipped.Similar-length strings can consume different numbers of tokens; high-entropy strings typically use more tokens than recognizable text.Source: §2.1 · p. 2
2.2 JavaScript Code Transformations
- JavaScript’s eval, Function constructor, computed module names, and dynamic property accesses can defer code and targets until runtime.These features complicate static reasoning because call targets, data dependencies, and executed code can depend on program state.Source: §2.2 · p. 2
- Obfuscation can encode literals, rename identifiers, rewrite control flow, or generate code dynamically, hiding behavior and increasing token density.The paper’s example transforms a six-token “Hello World” program into 790 tokens, about 132× more.Source: §2.2 · p. 3
- Bundling combines source files and dependencies into one or a few files, where package-specific logic may occupy only a small fraction.A detector may need to process the whole bundle, increasing token use and potentially pushing relevant code beyond the context window.Source: §2.2 · p. 3
3 Problem Statement
Large or obfuscated files can be skipped, while benign code containing obfuscation signals can be flagged as malicious. The paper assumes attackers control malicious package source and sets goals for preserving classification, bounding preprocessing, reducing token cost, and supporting different detectors.
3 Problem Statement
- A 4.5 MB package file containing an encoded array of 1,338,787 elements may be skipped because it exceeds many LLMs’ context windows.A separate benign package example contains Base64-encoded bundled code that causes the model to flag it as malicious.Source: §3 · p. 3
- Large or obfuscated files may be skipped or costly to analyze, while benign obfuscation patterns can produce false positives.The paper frames source presentation as affecting both detection accuracy and analysis cost.Source: §3 · p. 4
3.1 Threat Model
- The threat model gives attackers control of malicious npm package source but excludes attacks that compromise legitimate packages.It includes obfuscation, dynamic code generation, bundling, large literals, and prompt-like content; the scope is JavaScript, not behavior implemented entirely in native binaries or other non-JavaScript components.Source: §3.1 · p. 4
3.2 Design Goals
- CASHEWS should preserve malicious signals and enough context for classification, while bounding each preprocessing step’s time, memory, and output.When preprocessing cannot reduce an input, it should avoid increasing token count; its output should work across downstream scanners without scanner-specific preprocessing.Source: §3.2 · p. 4
4 CASHEWS Preprocessor
CASHEWS preprocesses JavaScript before it reaches an LLM detector, decoding and partitioning source, retaining code relevant to security-sensitive operations, and abbreviating long text to create a compact representation.
4.1 Overview
- CASHEWS parses source, decodes obfuscation, extracts bundled code, slices statements that may affect security-sensitive sinks, and abbreviates long identifiers and literals.The resulting compact representation is intended for a downstream detector.Source: §4.1 · p. 4
4.2 Iterative Decoding
- The decoding loop repeatedly applies transforms until no supported pattern can be decoded or a bound is reached, using a shared AST and rewrite phase.Transforms cover AST simplifications and techniques such as Base64, ROT arrays, and AES; untrusted initialization code needed to resolve values runs in a sandbox.Source: §4.2 · p. 5
4.3 Unit Extraction
- For recognized bundlers, format-specific fail-closed detectors use bundle markers to extract modules; oversized remaining files can be divided at AST boundaries.Extracted units receive identifiers that replace their original code locations, with a map linking each identifier to its code.Source: §4.3 · p. 6
4.4 Code Slicing
- A forward pass removes unreachable definitions, then a backward slice retains dependencies that may affect security-sensitive sinks or top-level exports.The backward pass preserves enclosing functions, conditions, and blocks needed for syntactic validity.Source: §4.4 · p. 6, Figure 6 · p. 6
4.4.1 Preserving Classification
- To preserve classification when analysis is uncertain or costly, CASHEWS conservatively retains possible call targets and keeps original code when slicing validation rejects a replacement.Slicing is skipped above limits of 256 units, 750,000 AST nodes, or AST depth 2,000.Source: §4.4.1 · pp. 6–7
4.5 Source Abbreviation
- CASHEWS abbreviates identifiers and literals above configured budgets, using distinct replacement forms for variable, function, property, and private-field names.Long URLs retain their first 48 and final 8 characters because hosts may be security relevant.Source: §4.5 · p. 7
5 CASHEWS Outputs
CASHEWS supports zero-shot and SocketAI detector workflows with either a combined processed-source representation or independently analyzed units.
5 CASHEWS Outputs
- The two workflows are a single-prompt zero-shot classifier and SocketAI, which produces an initial verdict, critiques and refines it, then selects a report.Both are presented as representative LLM-based detector workflows.Source: §5 · p. 7
5.1 Output per Processed Source File
- The combined representation contains decoded and sliced source with long identifiers and literals abbreviated, while extracted units replace their original locations and are appended under identifying comments.The comment markers separate untrusted content using the Spotlighting strategy.Source: §5.1 · pp. 7–8
5.2 Independent Units
- In the independent-unit representation, the lean source and extracted units are stored separately, and selected units are analyzed individually before the lean file is assessed with their verdicts.Separating units also allows reuse of verdicts for bundled code already analyzed in another package.Source: §5.2 · p. 8
- CASHEWS ranks units by sink and source counts, favors payloads on ties, and retains the most suspicious units for analysis.Zero-shot analysis takes the maximum malicious score across units; the multi-stage workflow uses selected units for its initial verdict stage.Source: §5.2 · p. 8
6 Evaluation
The evaluation tests CASHEWS's effects on detection, scalability, and the contribution of its steps. On large files, preprocessing improves coverage and false-negative rates, while its preprocessing cost is offset by token savings.
6.1 Experimental Setup
- The evaluation uses two LLM-based detector workflows and three LLMs on a 512-file subset of packages published from January 2025 to July 2026.The subset comprises files exceeding 25,000 tokens, including 166 malicious and 346 benign files; the full dataset contains 4,884 files.Source: §6.1 · p. 8
6.2 RQ1: Preprocessing and Classification
- Coverage reaches 98.8–100% and the false-negative rate falls by up to 18.6 percentage points with CASHEWS on large files.Preprocessing increases coverage by nearly 10 percentage points and reduces false-negative rates by a similar amount across settings, with at most a 0.3 percentage-point false-positive increase; the Top-3 fallback can raise false positives and introduce false negatives.Source: §6.2 · p. 9, Table 1 · p. 9
6.3 RQ2: Scalability
- 30 seconds is the median preprocessing time for files larger than 25,000 tokens, rising to about 3 minutes for files above 250,000 tokens.The largest 20% of files account for 91.7% of preprocessing time and 99.1% of token savings; Figure 8 relates median processing time and tokens saved to original token count.Source: §6.3 · p. 9, Figure 8 · p. 9
- 34.6% net modeled cost savings accompany a 36.6% reduction in input-token cost, with token savings outweighing preprocessing cost for files above 1,000 tokens.Figure 9 plots token-cost savings against CASHEWS CPU cost at $0.045 per core-hour; files below 1,000 tokens are the exception where preprocessing cost slightly outweighs token savings.Source: §6.3 · p. 10, Figure 9 · p. 10
6.4 RQ3: Ablation Study
- Decoding contributes less token reduction but unlocks later extraction and slicing, while source abbreviation is the cheapest step and delivers the second-highest reduction.The four steps activate on 89.1%, 47.7%, 27.3%, and 91.8% of files for decoding, extraction, slicing, and source abbreviation, respectively.Source: §6.4 · pp. 10–11, Table 2 · p. 10
- Top-three unit selection cuts tokens passed to the scanner by 98.9% at P90 and 43.3% at the median; extraction yields the largest coverage gain for files with bundles or embedded payloads.Most units are below 1 MB, with a median size of about 1 KB, and 36.2% contain none of the supported sinks.Source: Table 2 · p. 10, §6.4 · p. 11, Figure 10 · p. 11
- Among large files, complexity bounds trigger most slice guards; across the full dataset, missing modeled sinks are 4.9× more frequent in benign files and reachable dynamic evaluation 7.3× more frequent in malicious files.The slice step is skipped for about 72.7% of large files; missing sinks and incomplete module resolution are common reasons, while complexity bounds dominate guard activations above 25,000 tokens.Source: §6.4 · p. 11, Table 3 · p. 11
7 Discussion
The discussion examines adaptive attacks on CASHEWS and deployment boundaries as JavaScript ecosystems evolve. It also distinguishes preprocessing for malicious-file classification from preprocessing for behavior-grounded review.
7.1 Adaptive Attacks on CASHEWS
- Adaptive attackers may exhaust preprocessing resources or cause CASHEWS to remove malicious code, threatening bounded processing and detection goals.The discussion considers resource-exhaustion attacks and processing-evasion attacks as separate classes.Source: §7.1 · p. 12
- Explicit time, memory, and output bounds limit resource attacks, but an adversary can still consume the full allowed budget.When bounds are exceeded, a step may degrade its output or be skipped rather than failing the entire preprocessing run.Source: §7.1 · p. 12
- Slicing can select benign code when malicious payloads avoid supported sinks while benign code uses one.More conservative reachability could improve selection but would require more analysis time.Source: §7.1 · p. 12
- Abbreviation may remove a malicious payload embedded in a dynamically extracted literal; detecting such literals could increase token cost and prompt-injection exposure.The paper illustrates this with a literal whose middle portion contains the executed payload.Source: §7.1 · p. 12
7.2 Deployment
- New bundling frameworks, JavaScript features, or obfuscation strategies mean current implementation results may not generalize to future packages.The authors describe CASHEWS as flexible and extensible, with future work potentially adding package-directory ingestion.Source: §7.2 · p. 12
- For classification, long-literal abbreviation may suffice in the discussed example, whereas grounding a verdict in code behavior requires the decoding loop.The paper says CASHEWS should be tuned to its output consumers, including detectors and human reviewers.Source: §7.2 · p. 12
8 Related Work
Prior work reduces code for security analysis, deobfuscation, debundling, or LLM consumption, while CASHEWS combines these goals in a detector-agnostic preprocessor. Its distinguishing focus is exposing embedded programs and bundle modules as selectable LLM inputs while bounding processing costs.
8 Related Work
- CASHEWS combines iterative deobfuscation, payload and module extraction, conservative slicing, and code compaction while retaining input when safe reduction cannot be established.Prior static pre-screening and slicing methods are often embedded in particular detectors and assume a useful slice can be constructed.Source: §8 · pp. 12–13
- Prior slicing systems report high token reduction, including 93.7% median input-token reduction [24], but do not address obfuscated samples and can require long processing times.CASHEWS applies slicing after decoding to handle obfuscation while targeting npm-scale processing.Source: §8 · p. 13
- Unlike systems aimed at readable code or conventional analysis, CASHEWS exposes embedded programs and bundle modules as independently selectable LLM inputs and bounds expensive processing.Table 4 compares capabilities across deobfuscation, debundling, and LLM-oriented code-reduction systems.Source: §8 · p. 13, Table 4 · p. 13
- CASHEWS compacts identifiers and abbreviates literals to address excessive token counts and prompt-injection risk in LLM-oriented code reduction.The cited prior work includes JavaScript normalization and lexical reduction for LLM consumption.Source: §8 · p. 13
9 Conclusion
CASHEWS preprocesses JavaScript package files through four stages to support LLM-based detection. The conclusion reports improved detector coverage and scalability to npm-registry analysis under context-window constraints.
9 Conclusion
- CASHEWS reduces package-file size through decoding, bundled-unit extraction, code slicing, and source abbreviation.Source: §9 · p. 13
- The paper reports that CASHEWS significantly improves LLM-detector coverage and scalability to npm-registry analysis.The conclusion frames the result in the setting of adversaries exploiting detector context-window limits.Source: §9 · p. 13
- The conclusion presents source preprocessing as a way to support LLM-based malicious-package detection when files challenge context-window limits.Source: §9 · p. 13
A Experimental Setup Details
The appendix identifies model-cost information, the zero-shot detector prompt, and additional full-dataset results. The supplied passages describe these materials but do not report their values or findings.
A.1 LLM Costs
- Table 5 lists input and output token costs for evaluated models at evaluation-time provider pricing.The table does not account for higher-context pricing for GPT-5.6 Luna.Source: §A.1 · p. 14, Table 5 · p. 14
A.2 Zero-Shot Prompt
- Figure 11 presents the prompt used for the zero-shot LLM-based detector.Source: §A.2 · p. 14
B Additional Results
- Additional results cover the full dataset, including files under 25,000 tokens.Source: B Additional Results · p. 14
B.1 Full Dataset Results
On the full 4,884-file dataset, CASHEWS is primarily designed for large files; deploying it across all package files negatively affects cost and classification. The sensitivity analysis identifies τ = 0.5 as the best combined net-change threshold among those evaluated for both workflows.
B.1 Full Dataset Results
- For GPT-5.6 Luna, the Zero-Shot workflow’s false-negative rate increases by 0.7 percentage points on the full dataset.The passage also notes an increase for the Multi-Stage workflow but does not provide its complete value here.Source: §B.1 · p. 14
- Across all 4,884 package files, deploying CASHEWS on every file negatively affects both cost and classification.The full-dataset results use τ = 0.5; Table 6 distinguishes the preprocessed representation from a top-3-unit fallback.Source: §B.1 · p. 14, Table 6 · p. 14
B.2 Sensitivity Analysis
- τ = 0.5 yields the largest combined net improvement among evaluated thresholds: +8 model–file pairs for Zero-Shot and +2 for Multi-Stage.Combined net change subtracts verdict regressions from corrected verdicts.Source: §B.2 · p. 14
C CASHEWS Implementation Details
CASHEWS implementation details cover enforced bounds, decoding transformations, and source and sink taxonomies. When a stage exceeds its bound, it degrades that stage and leaves the corresponding source unchanged rather than failing the file.
C.2 Decoding Transformations
- The decoding step uses JavaScript transformations listed with their priority.Source: §C.2 · p. 14
C.1 Enforced Bounds
- When a CASHEWS stage exceeds its enforced bound, that stage degrades and the corresponding source remains unchanged rather than the file failing.Table 8 lists the bounds enforced by each stage.Source: §C.1 · p. 14, Table 8 · p. 15
C.3 Taxonomy of Sinks
- The sink taxonomy supports both backward slicing and unit selection.The backward slice is detailed in Figure 6.Source: §C.3 · p. 15
C.4 Taxonomy of Sources
- The source taxonomy supplies sources for slice safeguards and unit selection.The safeguards are described in Section 4.4.1 and evaluated in Table 3.Source: §C.4 · p. 15