Keep Evaluation Fair: Detecting Data Leakage in Code Generation Benchmarks via Membership Inference Attacks

Dongdong Zhao; Jian Chen; Guancheng Lin; Jianwen Xiang; Jacky Wai Keung; Xiao Yu

Source-linked AI summary

Keep Evaluation Fair: Detecting Data Leakage in Code Generation Benchmarks via Membership Inference Attacks

Dongdong Zhao, Jian Chen, Guancheng Lin, Jianwen Xiang, Jacky Wai Keung, Xiao Yu

arXiv:2609.09865v1Sep 9, 2026cs.SE

TL;DR

Benchmark leakage can inflate code-generation performance and undermine evaluation validity, while existing detection methods leave functional-correctness signals underused. CGMIA combines shadow-model membership inference with heterogeneous expert and semantic features, and experiments show it outperforms existing MIA methods while demonstrating practical effectiveness.

Problem
Benchmark samples can enter LLM pre-training data, inflating performance and undermining the validity of benchmark-based evaluations; prior methods underuse functional correctness for full-program benchmarks.
Method
CGMIA uses shadow modeling to generate labeled member and non-member data, then combines expert and semantic features in a classifier.
Results
CGMIA outperforms existing membership inference methods and demonstrates strong practical effectiveness in detecting code-generation benchmark leakage.
Takeaways & Limitations
Integrating heterogeneous features advances leakage detection for code-generation benchmarks and supports future research on other benchmark types.
Takeaways & Limitations
CGMIA may fail when an adversary deliberately modifies or rewrites reference code before training or fine-tuning the target LLM.

Abstract

from arXiv · show

Code generation benchmarks are widely used to evaluate Large Language Models (LLMs), but benchmark data leakage into training sets can inflate performance and undermine evaluation validity. DetectLeak, a method specifically designed for code generation benchmark leakage detection, relies on perplexity scores to identify likely leaked samples. However, perplexity mainly reflects general familiarity with code patterns and may perform poorly on complex or rare samples. It also overlooks other useful signals, such as code similarity, functional correctness, and semantic representations. To address these limitations, we propose CGMIA (Code-Generation-specific Membership Inference Attack), a method for detecting leakage in code generation benchmarks. CGMIA fine-tunes a shadow model on a subset of benchmark samples to construct labeled member and non-member data. For each sample, it collects the input prompt, generated code, and reference solution, and extracts expert features, including CodeBLEU, edit distance, test pass rate, and perplexity, together with semantic features from CodeBERT embeddings. An integrated learning module combines these features to capture both surface-level memorization signals and deeper behavioral patterns, enabling a classifier to predict whether a sample was included in the target model's training set. Experiments on eight code generation benchmarks show that CGMIA outperforms eight existing membership inference methods in most cases. It also effectively detects known leaked APPS samples in StarCoder-7B's training data.

1 Introduction

Code-generation benchmarks can be contaminated by training-data leakage, making strong model performance difficult to interpret as genuine generalization. CGMIA applies membership inference to benchmark samples by combining code-generation-specific signals and semantic features.

Motivation: Benchmark leakage can inflate performance and undermine whether benchmark results reflect generalization or memorization.StarCoder-7B achieved a pass@1 score 4.9 times higher on 108 leaked APPS samples than on unconfirmed samples.
Membership inference: Existing membership inference methods distinguish members from non-members using model behavior, often through shadow-model-generated labeled data.Shadow models provide known member and non-member samples for training an inference classifier.
Research gap: Code-domain methods largely target copyright protection through code completion or masked prediction, whereas full-program benchmarks require functional-correctness signals.DetectLeak is identified as the only existing method specifically designed for leakage detection in full-program code-generation benchmarks.
Contribution: CGMIA combines code-generation-specific expert features with semantic embeddings to capture surface memorization signals and deeper behavioral patterns.The method predicts whether a benchmark sample was included in the target model’s training set.
Evaluation: Experiments cover eight benchmarks and four open-source LLMs, comparing CGMIA with eight baseline methods and analyzing leakage detection under multiple conditions.The study includes feature contributions, shadow-model mismatch, class imbalance, and known StarCoder-7B/APPS leakage.

2 Related Work

Prior code-domain membership inference methods use either learned classifiers or output-based metrics, but they often miss functional-correctness evidence available in code-generation benchmarks. CGMIA addresses this gap by combining correctness, similarity, perplexity, and semantic information.

Classifier-based methods: Classifier-based methods train an attack model from model outputs, often using completion, masked prediction, or embedding-based features.Gotcha, CodeMI, and TraWiC exemplify this family of approaches.
Metric-based methods: Metric-based methods infer membership directly from output statistics without training an explicit attack model.Examples include perturbation sensitivity in Buzzer and perplexity-based detection in DetectLeak.
Existing limitations: DetectLeak identifies likely leaked samples through lower generated-code perplexity, but metric-based approaches can be limited by model robustness to minor perturbations.Buzzer compares [CLS] embedding changes after input perturbations, while DetectLeak uses perplexity as a memorization signal.
Research gap: Existing methods often overlook test cases, although leaked samples may produce abnormally high pass rates that provide a strong leakage signal.The proposed approach supplements this correctness signal with code similarity and perplexity.

3 Methodology

CGMIA operates in a query-based, score-access black-box setting and uses shadow-model data to train a membership classifier. It extracts expert and semantic features, aligns their dimensions, fuses them, and predicts member versus non-member status.

Task formulation: The attacker can query the target model for generated code and token-level log-probabilities, without access to its architecture, parameters, gradients, or training data.The log-probabilities support computation of the perplexity feature.
Membership inference classifier training: CGMIA fine-tunes a shadow model on benchmark members, queries it on members and non-members, and trains a classifier on the resulting labeled examples.Equal member and non-member counts are used to avoid training-data imbalance.
Expert features: For each sample, CGMIA extracts CodeBLEU, edit distance, perplexity, and test pass rate from five generated outputs.These four features produce a 20-dimensional expert vector, reducing variance from stochastic generation.
Semantic features: CodeBERT encodes the generated outputs and reference solution, while a CNN compresses the five generated-code embeddings into a dimension compatible with the reference representation.The semantic features capture contextual and structural code semantics.
Membership classifier: A three-layer feed-forward classifier maps the fused representation to non-member and member probabilities.The classifier uses membership labels and cross-entropy training, while the initialized CNN and expert-feature projection remain frozen.

4 Experimental Setup

The experiments simulate benchmark leakage with separated target and shadow-model splits, evaluate eight code-generation benchmarks, and compare CGMIA with established membership-inference baselines. Performance is assessed using classification metrics and paired statistical tests across benchmarks, while the authors note that real-world detections cannot be validated without training-data access.

Models and leakage simulation: Four open-source models serve as target or shadow models, with benchmark datasets reportedly removed from their released pre-training and post-training data.The selected models are CodeGemma, DeepSeek-Coder, Qwen2.5-Coder, and Phi-2.
Models and leakage simulation: The benchmark data are randomly divided into four mutually exclusive subsets to prevent overlap between target-model and shadow-model member or non-member data.The setup assigns separate subsets to target training, target non-members, shadow-model training, and shadow-model non-members.
Models and leakage simulation: The shadow model produces prompts, generated code, and reference solutions that train a membership classifier, which then predicts membership for corresponding target-model outputs.Shadow-model outputs from member and non-member subsets form positive and negative classifier examples before testing on target-model data.
Benchmarks: The evaluation covers eight benchmarks spanning algorithmic, practical, repository-level, and object-oriented programming tasks, with Python and Java selected for multilingual benchmarks.HumanEval-X and MBXP provide multilingual versions, while NaturalCodeBench and EvoCodeBench represent practical and repository-level settings.
Benchmarks: Pass@1 scores range from 1.4% to 59.7% across the eight benchmarks, indicating substantial variation in task difficulty before fine-tuning.The reported scores cover Qwen2.5-Coder, CodeGemma, DeepSeek-Coder, and Phi-2, with fine-tuning results also tabulated.
Baselines and evaluation: CGMIA is compared with DetectLeak, Gotcha, and CodeMI, while Precision, Recall, MCC, and AUC are compared across benchmarks using Wilcoxon tests, Cliff’s δ, and Benjamini-Hochberg adjustment.DetectLeak is tailored to code-generation benchmark leakage, whereas Gotcha and CodeMI are adapted from code-completion membership-inference settings.

5 Experimental Results

Across eight benchmarks and four target models, CGMIA generally outperforms existing membership-inference methods, while remaining effective across training settings and shadow-model choices. Feature ablations and robustness analyses show that expert features contribute complementary evidence, although architectural mismatch can reduce performance.

RQ1: CGMIA improves average MCC and AUC over the strongest baseline for every target model.MCC gains are 0.17, 0.26, 0.18, and 0.21, while AUC gains are 0.10, 0.09, 0.09, and 0.13 across the four models.
RQ1: 99 of 128 reported comparisons have p-values below 0.05, supporting statistically significant improvements over baselines in most cases.The remaining 29 comparisons, or 23%, have p-values greater than or equal to 0.05.
RQ2: Adding more expert features generally improves all four metrics, with the full feature set producing the highest scores except for DeepSeek-Coder Recall.Level-4 remains close to Level-3a when Test Pass Rate is unavailable, with differences no greater than 0.05 across metrics.
RQ3: CGMIA remains effective across target-model fine-tuning epochs, but different shadow architectures cause moderate performance changes.Precision, Recall, and AUC remain strong under shadow-model mismatch, while MCC declines significantly; performance ranges are 0.73–0.81 Precision, 0.68–0.77 Recall, 0.81–0.87 AUC, and 0.41–0.57 MCC.
RQ4: Ensembling shadow models reduces worst-case architectural mismatch and exceeds the best single shadow model in 10 of 16 model-metric cases.This corresponds to 62.5% of cases, although ensemble Recall is slightly lower than the best single shadow model for CodeGemma.

6 Threats of validity

The evaluation is limited by model scale, feature coverage, leakage simulation, interface requirements, long-code representation, and uncertainty in negative labels. Additional experiments partly address unknown leakage prevalence, but broader real-world validation remains necessary.

Model scale and feature scope: Evaluation uses smaller models because of hardware constraints, leaving CGMIA’s effectiveness on substantially larger models uncertain.The experiments use two NVIDIA 4090 GPUs, limiting fine-tuning of target and shadow models.
Model scale and feature scope: CGMIA uses only four expert-designed features alongside semantic embeddings, so additional or alternative features may improve detection accuracy.Feature expansion is identified as an important direction for future research.
Leakage assumptions and simulation: The method assumes leaked samples remain closely aligned with benchmark reference solutions and may miss deliberate code rewriting before training.CGMIA targets unintentional leakage involving complete code segments rather than deliberate evasion tactics.
Leakage assumptions and simulation: Main experiments simulate leakage through LoRA fine-tuning on benchmark subsets, which may create stronger memorization signals than incidental pre-training contamination.The StarCoder/APPS experiment offers preliminary evidence under a known real-leakage scenario, while broader real-world validation remains future work.
Deployment and representation boundaries: CGMIA maintains strong detection performance at 20%, 30%, and 40% member proportions and avoids manually specified thresholds under unknown prevalence.This contrasts with threshold-dependent methods such as DetectLeak, whose threshold selection depends on prevalence.
Deployment and representation boundaries: CGMIA requires token-level output scores and is therefore not directly applicable to strict text-only APIs.Its current applicability is limited to score-access black-box settings.
Deployment and representation boundaries: Sliding-window CodeBERT encoding may lose long-range dependencies or dilute localized memorization signals in very long programs.Long-context or structure-aware code representations are suggested for future work.
Deployment and representation boundaries: The StarCoder-7B evaluation has uncertain negative labels because APPS samples not confirmed as leaked cannot be guaranteed non-leaked.These negatives are more accurately treated as samples with unconfirmed leakage status.

7 Conclusion

The paper proposes CGMIA for detecting potential leakage in code generation benchmarks by combining shadow-model labeling with expert and semantic features. It reports stronger performance than existing membership inference methods and emphasizes heterogeneous feature integration as a direction for broader leakage detection.

Conclusion: CGMIA detects potential benchmark leakage by training a membership inference classifier with shadow-model-generated member and non-member data.The method extracts expert and semantic features before building the classifier.
Conclusion: CGMIA outperforms existing membership inference methods in experiments and demonstrates practical effectiveness for code-generation benchmark leakage detection.The conclusion presents this as the paper’s principal empirical outcome.
Conclusion: Integrating heterogeneous features supports future research on detecting data leakage in other benchmark types.The paper frames this as a broader research direction rather than a demonstrated result beyond code-generation benchmarks.

8 Data Availability

The study publicly provides its benchmark configurations, experimental code, and replication materials through an online repository.

Data Availability: The online repository contains benchmark data, prompt templates, LoRA fine-tuning and generation scripts, feature-extraction tools, and membership-inference implementations.It also includes CGMIA and baseline attack implementations.

A Supplementary Machine Learning Baseline Results

The appendix supplements the main traditional machine-learning baseline results with additional metrics and baseline methods.

Supplementary Results: Supplementary results report Precision, Recall, MCC, and AUC for additional traditional machine-learning baselines.These results complement the Logistic Regression and Random Forest results shown in the main text.
Supplementary Results: Tables 10–13 present supplementary Precision, Recall, MCC, and AUC results, respectively.The table captions identify the metric reported in each table.