PaperlayerView exact revision on arXiv

Source-linked AI summary

A Function-level Dataset of Vulnerable and Fixed Source Code in JavaScript and TypeScript

Tamás Viszkok, Péter Hegedűs

arXiv:2609.38012v1cs.CRcs.SE

TL;DR

JavaScript and TypeScript vulnerability detection needs high-quality training data, but existing datasets can contain duplication, temporal bias, and ecosystem-specific noise. JsVul curates fixing commits from seven sources into a function-level dataset through language-specific processing and labeling. Validation reports a consistent, learnable signal across three model architectures.

Abstract

from arXiv · show

JavaScript and TypeScript are widely used in modern web development, making their security critical; however, automated vulnerability detection is often constrained by the availability of high-quality training data. Here we present JsVul, a dataset curated from seven major sources. Unlike generic multi-language datasets that may retain noise -- such as minified code and cosmetic edits -- JsVul utilizes a language-specific pipeline. We collected pre-fix and post-fix versions of files around security fixes and, by filtering irrelevant artifacts and applying automated syntax normalization, isolated security-related changes. We ensured data integrity through multi-stage deduplication and heuristic-based labeling. Provided in a time-ordered JSONL format, JsVul supports robust model training in the JavaScript and TypeScript ecosystem and demonstrates the importance of language-aware preprocessing in building vulnerability datasets.

Background & Summary

JsVul addresses the need for clean, time-aware JavaScript and TypeScript vulnerability data with a language-specific pipeline that turns fixing commits into labeled function-level examples.

Background & Summary

  • JavaScript vulnerability datasets face risks from duplication, temporal bias, and language-specific noise, while the ecosystem has lacked a rigorously cleaned benchmark.Automated mining can introduce noise, and generic filtering may retain minified files, tests, configuration files, or cosmetic edits.Source: Background & Summary · pp. 1–2

Methods

  • JsVul combines seven-source commit aggregation, JavaScript-specific cleaning, and targeted manual verification to isolate security-relevant function changes.The pipeline is organized into Merge, Process, and Unify stages.Source: Background & Summary · p. 2, Methods · p. 2

Data Acquisition (Merge)

  • The Merge stage treats source data as seed commit lists, verifies references, standardizes repository and commit metadata through GitHub, and retrieves pre- and post-fix files.It identifies the vulnerable parent commit, including for merge commits, and retains only fixing commits that modify JavaScript or TypeScript files.Source: Data Acquisition (Merge) · pp. 2–3

Data Processing (Process)

  • The Process stage filters irrelevant files and minified code, normalizes syntax and formatting, and extracts function bodies to focus diffs on executable logic.It uses custom minification heuristics, ESLint, Prettier, Babel ASTs, and recast.Source: Data Processing (Process) · p. 4
  • Content-based intra-commit deduplication relabels unchanged pre-fix functions as non-vulnerable, while labeling refinement applies onefunc and nvdcheck filters.The approach uses normalized function identity to distinguish cosmetic edits or moved functions from semantic changes.Source: Data Processing (Process) · p. 4

Dataset Consolidation (Unify)

  • Unify structures records in JSONL and chronologically orders them by publish_time into 80% training, 10% validation, and 10% test splits.The chronological split is intended to mitigate temporal bias.Source: Dataset Consolidation (Unify) · p. 5

Data Records

JsVul provides the curated dataset in JSONL archives, with function bodies, labels, identifiers, and metadata organized for use and traceability.

Data Records

Archives js_vul and js_vul_pairs_only

  • Each JSON object includes a unique id formatted from repository, commit, file path, and start coordinates; an optional paired_id for a corresponding vulnerable or fixed counterpart; project, sha, and file; function location coordinates; and the function’s source-code body.Source: Archives js_vul and js_vul_pairs_only · p. 5

Archive merged_data

  • The merged_data archive contains pre- and post-commit source files organized by repository and commit SHA, plus project metadata keyed by fixing commit hash.Metadata fields include vulnerability identifiers, commit message, change counts, file information, vulnerable SHA, and sources.Source: Archive merged_data · p. 6

Technical Validation

Validation examines noise reduction, deduplication, and whether models can detect a learnable signal in JsVul.

Noise Reduction via Syntax Normalization

  • Syntax normalization reduced the average number of lines modified per commit, indicating that the original data included non-semantic formatting noise.The authors manually inspected 25% of randomly selected changes to check that code was not semantically modified.Source: Noise Reduction via Syntax Normalization · p. 6

Deduplication and Integrity Checks

  • Commit-wise function deduplication removed 12 vulnerable false positives and recovered 17 instances; duplicate commit pairs were manually verified and 87 of 97 were merged.Dataset-wide function conflicts were resolved by retaining the vulnerable label to minimize false negatives.Source: Deduplication and Integrity Checks · p. 6

Technical Validation

  • Table 1 summarizes pipeline-stage effects using metrics for noise reduction and preservation of relevant code units.The validation is framed around data purity, redundancy removal, and semantic relevance.Source: Technical Validation · p. 6, Table 1 · p. 7

Semantic Signal Verification

  • Scores across CodeBERT, Qwen2.5-Coder-7B-Instruct, and Gemini 3 Pro show a consistent, learnable signal in the js_vul_pairs_only subset.The comparison uses an encoder-only model, an open-weight coding LLM, and a closed-source zero-shot LLM.Source: Semantic Signal Verification · p. 7, Table 2 · p. 7

Usage Notes

The usage notes describe the dataset’s record content and recommend chronological splitting for vulnerability forecasting.

Usage Notes

  • JsVul is distributed in JSONL format for efficient line-by-line processing.Source: Usage Notes · p. 7

Data Structure

  • Each record contains a function body and a binary label, with 1 denoting vulnerable and 0 denoting non-vulnerable.Source: Data Structure · p. 7

Temporal Splitting

  • Records are ordered by publish_time, and the authors recommend chronological 80%/10%/10% training, validation, and test splits to simulate vulnerability forecasting.Source: Temporal Splitting · p. 7

Funding

The work was supported by Hungary’s Ministry of Culture and Innovation through Project K 147225; the University of Szeged Open Access Fund covered the publication charge.

  • Project K 147225 received support through Hungary’s Ministry of Culture and Innovation and the National Research, Development and Innovation Fund’s K_23 OTKA Funding Scheme.Source: Funding · p. 7
  • The University of Szeged Open Access Fund covered the publication charge under Grant Nr. 8358.Source: Funding · p. 7

Data Availability

The study’s data are available in Zenodo as three compressed archives containing the curated dataset, matched function pairs, and pre-processed source data and metadata.

  • The Zenodo record contains the complete curated dataset in JSONL format as the js_vul archive.Source: Data Availability · p. 8
  • The js_vul_pairs_only archive contains matched vulnerable and fixed function pairs.Source: Data Availability · p. 8
  • The merged_data archive contains pre-processed source data and metadata, including pre- and post-commit files and per-project JSON metadata.Source: Data Availability · p. 8

Code Availability

The data-processing code is publicly available on GitHub, with instructions for reproducing the dataset or generating custom variants.

  • The GitHub repository provides the filtering, normalization, and deduplication pipeline, plus instructions for reproducing the dataset or generating custom variants.Source: Code Availability · p. 8

S1 Supplementary Material

The supplement identifies the model configurations and hyperparameters and provides prompt templates for Qwen and Gemini. The templates instruct the model to classify source code as vulnerable or safe using a one-word response.

Model Configurations

  • Table S1 presents model configurations and hyperparameters; Listings S1 and S2 provide the exact prompt templates used for Qwen and Gemini.Source: Table S1 · p. 12

Listing S1: System Prompt used for inference.

Listing S2: User Message Construction Template.

A. Raw Diff (Before Normalization)

The supplementary examples show how normalization separates security fixes from cosmetic edits and illustrate a paired vulnerability-and-patch dataset entry.

A. Raw Diff (Before Normalization)

  • Figure S1 shows whitespace and keyword changes obscuring a security fix before normalization, then absent after normalization.Source: Figure S1 · p. 14

Raw Diff (Before Normalization)

  • Figure S2 illustrates Prettier formatting effects in index.js from chaijs/pathval.Source: Figure S2 · p. 15

Example Dataset Entries

  • Figure S3 presents a paired Prototype Pollution entry and patch, with each record containing an id, function body, vulnerability label, and metadata.Source: Figure S3 · p. 16
Loading 2609.38012v1…