The Hidden DNA of LLM-Generated JavaScript: Structural Patterns Enable High-Accuracy Authorship Attribution

1 Introduction

• Existing studies often represent AI-generated code with one model or a small set, limiting the robustness of attribution methods.The introduction also notes concerns that overlapping training corpora and possible style convergence could make model-level attribution harder.Source: §1 · p. 1
• The study extends JavaScript attribution to within-family model distinctions and larger 10–20-class tasks, beyond prior C-code stylometry among five LLM families [6].It focuses on JavaScript, described as flexible and dynamically typed, and examines models within the same family.Source: §1 · p. 1
• The proposed pipeline generates Node.js code from 20 LLMs, checks and transforms the outputs, trains classifiers, then evaluates them on an independent dataset.The introduction points readers to Figure 1 for the seven-step methodology.Source: §1 · p. 2, Figure 1 · p. 2
• LLM-NodeJS contains 50,000 programs expanded into 250,000 JavaScript samples, and the project releases its dataset, notebooks, and training scripts.The stated goal is to support attribution research that treats AI-generated code as diverse rather than uniform.Source: §1 · p. 2
• 95%+ accuracy is reported across five GPT-family models, with nearly 90% for 20-class attribution; results remain strong on several transformed variants.Minified, terser-mangled, and deobfuscated code reach 91%, 93%, and 85%, respectively; obfuscation lowers accuracy due to token inflation.Source: §1 · p. 2

2 Related Work

• The review is organized by technique family, followed by a discussion of benchmark datasets and tasks.This is the section’s stated structure.Source: §2 · p. 2

2.1 Authorship attribution architectures

• Traditional classifiers use engineered code features and are inexpensive and interpretable, but reportedly trail deep learning in accuracy [6].Examples include logistic regression, naïve Bayes, SVMs, and tree ensembles.Source: §2.1 · p. 3

2.1.1 Traditional Machine Learning.

• Neural models learn stylistic hierarchies directly from code, scaling to many authors but requiring large labeled datasets.An RNN-plus-Random-Forest hybrid achieved 92.3% accuracy on 8,903 Google Code Jam authors.Source: §2.1.1 · p. 3

2.1.2 Deep Learning.

• Pre-trained code models provide contextual embeddings and strong attribution with few manual features; fine-tuned CodeT5 variants also work across closely related authors [13] [16] [41] [21].The cited work covers BERT, CodeBERT, and RoBERTa [13] [16] [41].Source: §2.1.2 · p. 3

2.1.3 Pre-trained and Generative Models.

• Generative LLMs support zero- or few-shot attribution, while prior AI-vs-AI work reported over 95% accuracy across five LLM families using modified CodeT5 [6].Most AI-vs-human studies cited here instead treat AI as a single-source class.Source: §2.1.3 · p. 3

2.2.1 Feature Engineering Approaches.

• Early code stylometry used lexical cues, formatting habits, and software metrics; the Code Stylometry Feature Set was introduced for human authorship [38].Hand-crafted features can degrade under obfuscation, adversarial edits, or IDE suggestions [32].Source: §2.2 · p. 3

2.2.2 Graph and Structural Representations.

• AST paths, Program Dependence Graphs, and Control Flow Graphs represent syntax, dependencies, and control flow; graph features are more robust to formatting changes.AST path classifiers reached 98% on curated sets but dropped sharply on realistic corpora [7].Source: §2.2.1 · p. 3
• Static stylistic features are fragile under obfuscation, while syntax- and AST-based methods remain effective; one deep-learning study reported about 93% accuracy on obfuscated code [32].These findings distinguish structural representations from surface-sensitive features.Source: §2.2.2 · p. 3

2.2.3 Obfuscated Code Attribution.

• Dynamic attribution uses runtime traces to expose algorithmic signatures resistant to superficial edits, but sandboxing is costly and mostly used in malware forensics [37].Binary attribution is another line of work; stylistic markers can survive compilation, and reported accuracy remains above chance despite compiler variability.Source: §2.2.3 · p. 3

2.3 Datasets, Benchmarks and Tasks

• Human-attribution research uses labeled corpora such as Google Code Jam, Codeforces, GitHub OSS, POJ-104, and BigCloneBench [7].Classifiers designed for natural language reportedly perform poorly on source code [31].Source: §2.3 · p. 3
• AI-vs-human studies report strong binary results but are limited to GPT-4 as the source of AI-generated code.The cited results are F1 ≈0.91 and accuracy ≈92%.Source: §2.3 · p. 3
• Bisztray et al. [6] introduced LLM-AuthorBench, a 32k-program C dataset, and reported 95% accuracy distinguishing five LLMs.Their study is identified as the only large-scale AI-vs-AI study in this review.Source: §2.3 · p. 3

3.1 Dataset Creation- The LLM-NodeJS dataset

• The 250 tasks are decomposed into meaningful subtasks to promote structural and stylistic diversity, and each task is posed ten times to each model.Only syntactically correct programs that passed node --check were included.Source: §3.1 · p. 3
• LLM-NodeJS contains 50,000 validated programs from 20 models, expanded into 250,000 samples with four code variants and JSIR and AST representations.The dataset is intended to support JavaScript analysis beyond authorship attribution.Source: §3.1 · pp. 3–4

3.2 Selected Models for Dataset Evaluation

• The evaluation compares traditional ML, encoder-only Transformers, and a custom architecture on the new dataset.The traditional ML baseline uses TF–IDF token vectors with a vocabulary cap of 400 across five algorithms.Source: §3.2 · p. 4, §3.2.1 · p. 4

3.2.2 Transformer-based Models .

• BERT, CodeBERT, and Longformer provide encoder-only Transformer comparisons, with training settings adjusted to available GPU memory and model context windows.Experiments used bf16 precision, fused AdamW, and cosine restarts.Source: §3.2.1 · p. 4, §3.2.2 · p. 4

3.2.3 Custom CodeT5-JSA Architecture.

• CodeT5-JSA removes CodeT5+'s decoder and revises the classification head; bfloat16 cuts 20-class training from 958 to 111 minutes with a 1–3% accuracy drop.Replacing GELU with Tanh and adding initial dropout improved stability; on deobfuscated twenty-class data, accuracy rose from 72% to 79.94%.Source: §3.2.2 · p. 4, §3.2.3 · p. 4

• The first experimental stage compares ML and Transformer architectures for attributing original and obfuscated JavaScript to its generating model.Source: §4 · p. 4
• The second stage tests whether attribution performance holds as the number of classes increases.Source: §4 · p. 4
• The experiments also test whether some models are too similar to distinguish accurately.Source: §4 · p. 4

4.1 Dataset diversity analysis

• The analysis compares median N-gram, syntax, and dataflow similarities for GPT-4o, GPT-5, and Gemini, summarizing intra–inter separation with an average gap Δ.Larger Δ values indicate stronger model discrimination.Source: §4.1 · p. 4, Table 1 · p. 4

4.1.1 Key Results.

• Dataflow similarity most clearly separates same-model from cross-family code, including median values of 0.46 for GPT-4o pairs and 0.11 for GPT-4o–GPT-5 pairs.N-gram similarity shows a comparable gap but is sensitive to surface-level edits.Source: §4.1 · p. 4
• Syntax has the smallest intra–inter gap because structural patterns remain relatively stable across tasks.The authors caution that this does not mean syntax lacks discriminative power.Source: §4.1.1 · p. 5

4.1.2 Implications for attribution.

• Dataflow and syntax provide deeper, more transformation-resilient signals than surface cues, while n-gram discrimination degrades under lexical variation.The analysis identifies dataflow as the most obfuscation-resilient indicator of model identity.Source: §4.1.1 · p. 5

4.1.3 Token length distribution.

• Figure 2 presents tokenized sequence-length distributions, with PX marking the percentage of samples below each token count.The distribution informs architecture choice because short-to-medium contexts fit encoder-only models.Source: Figure 2 · p. 5, §4.1.2 · p. 5

4.2.1 Five-Class Attribution for Different Model Families.

• BERT𝐵 reaches 97.9% accuracy, F1, and precision/recall when distinguishing five models from different families.The setup replicates the C-code attribution scenario in [6] using JavaScript.Source: §4.2.1 · p. 5, §4.2 · p. 5

4.2.2 Five-Class Attribution for Same Model Families.

• BERT𝐵 attains 90.2% accuracy among five GPT variants, while errors cluster among closely related releases and variants.Architecturally distinct versions are more readily distinguished, as shown in Figure 3.Source: §4.2.1 · p. 5, §4.2.2 · p. 5, Figure 3 · p. 5

DEOBFUSCATED JAVASCRIPT RESULTS

• As attribution expands from five to ten and twenty classes, accuracy falls across models, but Transformer encoders consistently outperform classical ML baselines.Source: DEOBFUSCATED JAVASCRIPT RESULTS · p. 6

4.2.3 Ten and Twenty-Class Attribution.

• CodeT5-JSA achieves 94% accuracy in ten-class attribution and above 88% in twenty-class attribution on original JavaScript, leading overall while Longformer requires substantially more computation.Class expansion reduces accuracy across models; XGBoost remains competitive but degrades more rapidly than CodeT5-JSA.Source: §4.2.3 · p. 6
• XGBoost remains efficient with a favorable accuracy-runtime balance and sometimes outperforms BERT𝐵 on mangled and deobfuscated code.Linear methods fall below 60% accuracy at twenty classes.Source: §4.2.3 · p. 6

4.2.4 Different JavaScript Variants.

• Attribution stays accurate on mangled and deobfuscated code; restoring syntactic structure substantially recovers performance after transformations remove naming cues.The results support reliance on structural and semantic regularities rather than surface-level naming or token frequencies.Source: §4.2.3 · p. 6, §4.2.4 · p. 7

4.3 Cross-check Validation

• The 500 cross-check samples were generated through the original OpenAI API, independently of OpenRouter.ai, to check that OpenRouter did not watermark its outputs.The samples covered Node.js-specific programs and completely unseen tasks.Source: §4.3 · p. 7
• CodeBERT’s five-class attribution performance on original JavaScript dropped only 1–2% on the independent cross-check dataset.The samples came from completely unseen tasks and were generated using the original OpenAI API, without overlap with the 50,000 samples used for training or evaluation.Source: §4.3 · p. 7
• The authors publicly released the validation dataset as CROSS_CHECK_DATASET.json on GitHub.Source: §4.3 · p. 7

5 Limitations and Future Work

• Experiments used moderate-sized Transformer encoders and did not test larger or extended-context architectures or AST and JSIR inputs.Computational constraints limited model exploration; AST and JSIR representations were released for future research.Source: §5 · p. 7
• The dataset covers JavaScript web-development tasks, leaving other languages, cross-language attribution, coding domains, and prompt styles for future study.The authors propose studying languages including C++, Rust, Python, and Java to assess generality.Source: §5 · p. 7
• The multi-class classifier assumes samples come from the 20 studied LLMs; open-set recognition, continual learning, and temporal fingerprint stability remain unexamined.Source: §5 · p. 7
• Robustness testing has not yet covered adversarial modifications such as transpilation or control-flow rewriting.Source: §5 · p. 7
• Attribution systems may be vulnerable to deliberate style spoofing or adversarial evasion, risks the authors say need investigation and safeguards.Source: §5 · p. 7

6 Conclusion

• The 250,000-sample LLM-NodeJS corpus contains 50,000 validated Node.js programs, four code variants per program, and AST and JSIR representations.The study compared classical machine-learning algorithms and fine-tuned Transformer encoders with a modified architecture based on 770M-parameter CodeT5.Source: §6 · p. 7
• 95.8% accuracy for CodeT5-JSA on five-way GPT-only attribution, 94.6% on ten classes, and 88.5% on twenty classes.On terser-mangled, minified, and deobfuscated code, performance remained close, typically with single-digit drops.Source: §6 · p. 7
• Dataflow provides the clearest same-versus-cross-model separation, while accurate attribution remains possible without n-gram signals.The authors report that classifiers exploit persistent logic, dependency structure, and control/dataflow rather than relying primarily on superficial lexical features.Source: §6 · p. 7
• Model-specific fingerprints remain detectable across families, ten- and twenty-class tasks, and common code transformations, although accuracy declines as stylistic boundaries narrow.Semantically grounded signals continue to support reliable attribution.Source: §6 · p. 7