Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distances

1 Introduction

• Models can exploit recurring superficial cues in familiar test sets, and their performance may degrade on unfamiliar projects, later vulnerability instances, or unseen code structures.The paper frames model familiarity as important because security predictions are actionable only when there is an indication that the input lies in a familiar region.Source: §1 · p. 1
• Vulnerable and patched code can remain close in embedding space, while local overlap may pull safe patches toward vulnerable neighbourhoods and trigger false alarms.Aggregate metrics do not diagnose either geometric overlap or this local failure mode.Source: §1 · p. 2, Figure 2 · p. 3
• The study tests whether calibrated embedding distances expose failure boundaries across controlled shifts using CodeBERT and Qwen2.5-Coder.It evaluates an encoder and a generative LLM against pattern, project, and temporal shift scenarios.Source: §1 · p. 2
• The paper contributes a paired hard-negative benchmark and reports that degradation tracks representation distance, with accuracy dropping by over 35 percentage points toward chance under shift.It also identifies vulnerable-neighbourhood entrapment as a driver of false alarms on repaired code and tests proximity-based triage.Source: §1 · p. 3

2 Dataset and Experimental Design

• The dataset design requires reliable vulnerability–patch relations, project metadata, and controlled shifts; curation addresses repository mining artifacts that can encourage shortcut learning.The study combines curated sources with explicit pair information and uses a three-stage construction pipeline.Source: §2 · p. 3, Figure 3 · p. 4

2.1 Data Foundation and Hard-Negative Design

• The dataset combines PrimeVul, TitanVul, and VulGate, and uses 12,190 matched vulnerable–patched pairs while excluding arbitrary benign code to focus evaluation on localized security edits.It restricts the data to memory- and resource-safety weakness classes and enriches incomplete project metadata using the National Vulnerability Database.Source: §2.1 · pp. 3–4

2.2 Data Cleaning and Normalization

• Long pairs are reduced with pair-aligned multi-hunk extraction to retain changed lines and context; cleaning and pair-level deduplication yield 8,065 verified pairs.Comments are stripped, empty or identical pairs are removed, and both model families use the same normalized inputs.Source: §2.2 · p. 4

2.3 Controlled Evaluation Groups

• Five evaluation groups separate familiar data from pattern, project, and temporal shifts, with strict pair integrity and a priority cascade intended to prevent overlapping holdout conditions.The groups include same-CWE new-pattern OOD, project OOD, and temporal OOD based on 2024 CVEs.Source: §2.3 · p. 5

2.4 Evaluation Metrics

• Accuracy and F1 measure classification, Patch-FPR tracks repaired code flagged as vulnerable, and AUROC evaluates how well distance scores rank unfamiliar or incorrect samples.FNR measures missed genuine vulnerabilities; AUROC is independent of an operational threshold.Source: §2.4 · p. 5

3 Methodology

• The study compares CodeBERT embeddings with Qwen2.5-Coder predictions to test whether encoder-derived distances transfer as warnings of failures in a separate language model.CodeBERT is evaluated both pretrained and after joint cross-entropy and supervised contrastive fine-tuning.Source: §3 · p. 6, §3.1 · p. 6

3.1 Model and Training Setup

• The encoder combines cross-entropy with supervised contrastive learning, which pulls same-label representations together and pushes differently labeled representations apart within each mini-batch.Pair-complete batches ensure vulnerable functions and their patched counterparts appear together during training.Source: §3.1 · p. 6

3.2 Embedding Extraction & Calibration

• Embeddings are extracted from multiple CodeBERT layers, then whitened to center and decorrelate the training distribution so localized repair semantics can influence latent distances.Layer choice is treated empirically because lower and higher layers may encode different features; the whitening transformation equalizes variance across dimensions.Source: §3.2 · p. 7

3.3 Distance-Based OOD Scoring

• OOD scoring uses non-parametric k-nearest-neighbour cosine distance, with a diagonal class-conditional Mahalanobis distance as a global parametric baseline.Test samples are grouped into ten equal-frequency distance bins to assess accuracy and error rates across shift types, both intrinsically and across models.Source: §3.3 · pp. 7–8

3.4 Vulnerable-Neighbourhood Proximity

• For patched test samples, the analysis compares false positives with true negatives using four calibrated-embedding measures of proximity to the vulnerable class.The measures include centroid and nearest-neighbour margins, a k-neighbourhood margin ratio, and the vulnerable-neighbour share among the top 10 neighbours; diagnostic value is assessed with group means, AUROC, and proximity deciles.Source: §3.4 · p. 8

4 Results

• All models perform above chance but none achieves high absolute performance; Qwen leads modestly, while SCL helps local diagnostics more than global OOD ranking.SCL structures the latent space for local class-conditional diagnosis of patched false positives, but does not noticeably improve global OOD ranking over the raw encoder.Source: §4 · p. 8

4.1 RQ1: Embedding Space Calibration

• Raw pretrained CodeBERT at layer 3 with whitening and k=1 cosine scoring achieves the strongest mean OOD-AUROC, 0.739.Fine-tuned variants peak at 0.735, while the best geometry comes from the raw encoder.Source: §4.1 · p. 9
• Whitening raises best mean OOD-AUROC from 0.710 to 0.739; layer 3 peaks at 0.739, and k=1 is strongest overall.The diagonal Mahalanobis baseline reaches only 0.547; project OOD is an exception, peaking at k=3 with 0.776.Source: §4.1 · p. 9

4.2 RQ2: Generalization Boundaries

• Accuracy falls from 0.834 to 0.479 across distance bins, but temporal accuracy (0.574) exceeds cross-project accuracy (0.517), where Patch-FPR reaches 0.507.Temporal samples are geometrically farthest, yet cross-project shifts cause the sharpest degradation.Source: §4.2 · pp. 10, 12, Figure 6 · p. 12
• Pair-exact correctness falls below the 25% random baseline under every OOD shift, while CWE-level failure varies independently of embedding distance.Pair-exact correctness is 22.5% for unseen patterns, 19.8% for temporal shifts, and 18.9% across projects; CWE-787 accuracy is 0.722 versus 0.542 for CWE-416.Source: §4.2 · pp. 12–13, Table 5 · p. 13

4.3 RQ3: Vulnerable-Neighbourhood Proximity and Patched False Positives

• Cross-project patched samples alone have a positive centroid vulnerability margin (+0.016), connecting this shift to over-warning of repaired code.These samples are closer to the vulnerable centroid than to the patched centroid.Source: §4.2 · p. 12, §4.3 · p. 14
• Patched false-positive rates rise from 9.4% in the lowest proximity-risk decile to 58.8% in the highest.The k=10 distance ratio ranks patched false positives with an AUROC of 0.636.Source: §4.3 · pp. 14–15, Figure 8 · p. 15

5.1 Interpretation of the Main Findings

• Fine-tuning collapses upper-layer features, motivating a dual-representation design that pairs LLM triage with a frozen, whitened CodeBERT Layer 3 guardrail.The paper reports that the 125M-parameter encoder is over 50 times smaller than a 7B LLM and scores distances in under 5 ms per function.Source: §5.1 · p. 15
• LLM confidence remains comparatively high as accuracy declines with distance, so confidence alone would miss high-distance inputs flagged by encoder-derived distance.The discussion frames this as non-redundant reliability information from an externally calibrated encoder.Source: §5.1 · p. 16
• Cross-project patched samples are the only evaluation group closer geometrically to the vulnerable-class centroid than to the patched-class centroid.The discussion connects this proximity pattern to cross-project failures and measurable shortcut learning.Source: §5.1 · p. 16
• CWE-level results indicate that vulnerability semantics shape generalization beyond distance and shift type, with lifetime- and control-flow-dependent weaknesses less locally observable.CWE-416 and CWE-415 show weaker results, while CWE-787 and CWE-120 appear more robust; the discussion suggests token-level context may be insufficient for some weaknesses.Source: §5.1 · p. 16

5.2 Threats to Validity

• Vulnerable–patched pairs omit arbitrary benign code, and hunk extraction assumes the root cause fits within context rather than requiring whole-function localization.Patch-diff clusters are also operational proxies for fix categories, not manually validated semantic taxonomies.Source: §5.2 · p. 16
• Label noise or non-security refactoring can affect classification and distance, while hunk extraction may remove broader function context relevant to some vulnerabilities.RQ2 and RQ3 are conditioned on the strongest configuration selected in RQ1.Source: §5.2 · p. 16
• External validity is limited to seven memory- and resource-safety CWE classes in C/C++, specific held-out projects, a 2024 cutoff, and two evaluated models.Results may differ for encoders with explicit data-flow inputs or substantially larger LLMs.Source: §5.2 · p. 16
• Fixed training runs, dataset-dependent distance bins, and observational associations constrain conclusions; vulnerable-neighbourhood proximity interacts with thresholds, truncation, and project context.The authors explicitly note that statistical association does not establish strict causality.Source: §5.2 · p. 16

Distribution Shift in Vulnerability Detection

• Prior shift studies identify shortcut features, temporal leakage, project identity, and OOD generalization gaps; this work isolates pattern, project, and temporal shifts and finds project degradation disproportionate to displacement.The cited work includes Chakraborty et al., Jimenez et al., Steenhoek et al., and Li et al..Source: §6 · p. 17

Hard-Negative and Patch-Based Detection

• Prior work uses hard-negatives and neighbourhoods for training or retrieval, while this paper uses them diagnostically through pair-exact evaluation and a geometric signature of shortcut behaviour.MVP combines vulnerability and patch signatures, while Shimmi et al. and Papernot and McDaniel [35] show diagnostic value in local neighbourhood structure.Source: §6 · p. 17

Contextual Embedding Geometry

• The RQ1 findings jointly support whitening for code-embedding anisotropy and extend layer specialization to OOD analysis, where last-layer specialization costs broader distributional structure.The related work cites Ethayarajh, Su et al. [44], Huang et al., and Tenney et al..Source: §6 · p. 17

OOD Detection and Model Calibration

• Calibration results confirm local kNN scoring for code, while cross-model evaluation finds encoder distance exposes reliability information that code-specific LLM confidence misses.The comparison builds on maximum-softmax OOD scoring, non-parametric kNN distance, and confidence-calibration findings.Source: §6 · p. 17

• Calibrated distances expose gradual, shift-specific boundaries: cross-project shifts cause the sharpest accuracy drops, while temporal shifts drift furthest and inflate false negatives.Source: §7 · p. 18
• Whitened intermediate CodeBERT layers provide the strongest separation of familiar and unfamiliar code, while generative LLMs remain vulnerable to localized syntax traps.Scaling model capacity alone does not resolve hard-negative confusion.Source: §7 · p. 18
• Patched-code false positives systematically correlate with geometric proximity to vulnerable training neighbourhoods.Source: §7 · p. 18
• Post-hoc whitening can retrofit scanners with OOD scores without gradient updates or separate shadow models, supporting selective prediction and shift-specific review planning.Unfamiliar repositories call for reviewer capacity for inflated false alarms; temporally shifted code calls for recall-focused monitoring against missed defects.Source: §7 · p. 18
• Future work should extend the framework to graph- and data-flow-based models, explore OOD-density fine-tuning objectives, and validate boundary stability across multi-year code evolution.Source: §7 · p. 18