PaperlayerView exact revision on arXiv

Source-linked AI summary

LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks

Xin Zhou, Martin Weyssow, Ratnadira Widyasari, Ting Zhang, Junda He, Yunbo Lyu, Jianming Chang, Beiqi Zhang, Dan Huang, David Lo

arXiv:2502.06215v1cs.SEcs.AIcs.CL

TL;DR

Undisclosed pre-training data leaves unclear whether SE benchmark samples have been seen by LLMs, a concern for evaluation validity. The study analyzes leakage across 83 benchmarks using near-duplicate detection and manual labeling. Leakage is generally low but varies across benchmarks, and leaked samples can inflate evaluation scores.

  • Problem

    The extent to which SE benchmark data appears in LLM pre-training data has not been comprehensively studied, while such exposure can compromise evaluation validity and fairness.

    Source: §1 · p. 2

  • Method

    The study analyzes 83 SE benchmarks using near-duplicate detection and manual labeling, and quantifies leakage with sample counts and ratios.

    Source: §3.3 · p. 8, §8 · p. 24

  • Results

    Average leakage ratios were 4.8% for Python, 2.8% for Java, and 0.7% for C/C++ benchmarks, but leaked samples also yielded higher evaluation scores.

    Source: §8 · pp. 24–25

  • Takeaways & Limitations

    LessLeak-Bench provides cleaned versions of the 83 studied benchmarks with identified leaked samples removed for future LLM evaluations.

    Source: §1 · p. 3

    Findings are limited to the studied SE benchmarks and LLMs and may not generalize to all benchmarks or models.

    Source: §6.3 · p. 23

Abstract

from arXiv · show

Large Language Models (LLMs) are widely utilized in software engineering (SE) tasks, such as code generation and automated program repair. However, their reliance on extensive and often undisclosed pre-training datasets raises significant concerns about data leakage, where the evaluation benchmark data is unintentionally ``seen'' by LLMs during the model's construction phase. The data leakage issue could largely undermine the validity of LLM-based research and evaluations. Despite the increasing use of LLMs in the SE community, there is no comprehensive study that assesses the extent of data leakage in SE benchmarks for LLMs yet. To address this gap, this paper presents the first large-scale analysis of data leakage in 83 SE benchmarks concerning LLMs. Our results show that in general, data leakage in SE benchmarks is minimal, with average leakage ratios of only 4.8\%, 2.8\%, and 0.7\% for Python, Java, and C/C++ benchmarks, respectively. However, some benchmarks exhibit relatively higher leakage ratios, which raises concerns about their bias in evaluation. For instance, QuixBugs and BigCloneBench have leakage ratios of 100.0\% and 55.7\%, respectively. Furthermore, we observe that data leakage has a substantial impact on LLM evaluation. We also identify key causes of high data leakage, such as the direct inclusion of benchmark data in pre-training datasets and the use of coding platforms like LeetCode for benchmark construction. To address the data leakage, we introduce \textbf{LessLeak-Bench}, a new benchmark that removes leaked samples from the 83 SE benchmarks, enabling more reliable LLM evaluations in future research. Our study enhances the understanding of data leakage in SE benchmarks and provides valuable insights for future research involving LLMs in SE.

1 INTRODUCTION

Because pre-training data is often undisclosed, it is difficult to determine how much SE benchmark data LLMs have encountered, despite the risk this poses to evaluation validity and fairness. The study examines leakage across 83 SE benchmarks, reports generally low but uneven leakage and its impact on evaluation, and introduces a cleaned benchmark set.

1 INTRODUCTION

  • The study investigates leakage across 83 SE benchmarks in Java, C/C++, and Python, focusing on StarCoder because its pre-training data is publicly available.The benchmarks span tasks including code generation, repair, fault localization, clone detection, test generation, and vulnerability detection.Source: §1 · p. 2
  • DetectLeak combines automated near-duplicate detection with developer review to identify overlaps between benchmark samples and LLM pre-training data.The study reports comparing approximately 1.7 trillion pairs.Source: §1 · p. 2
  • Average leakage ratios were 4.8% for Python, 2.8% for Java, and 0.7% for C/C++ benchmarks, although QuixBugs reached 100.0% and BigCloneBench 55.7%.On APPS, StarCoder-7b's Pass@1 score was 4.9 times higher on leaked than non-leaked samples.Source: §1 · pp. 3–4
  • LessLeak-Bench provides cleaned versions of the 83 studied benchmarks with identified leaked samples removed.The authors state that it is intended to enable more reliable future LLM evaluations.Source: §1 · p. 3

2 BACKGROUND

The background defines leakage as evaluation data appearing in model-construction data, including exact matches and semantically equivalent samples. It distinguishes pre-training leakage as harder to detect than training/evaluation overlap, and separates leakage detection from code clone detection by purpose and data scope.

2.1 Data Leakage Definition

  • Leakage includes an exact match or a semantically equivalent pair between an evaluation sample and a construction-data sample.The semantic-equivalence function returns 1 when samples are semantically equivalent and 0 otherwise.Source: §2.1 · p. 4
  • Pre-training leakage is difficult to inspect because datasets may be undisclosed and large; this study focuses on overlap between LLM pre-training data and SE benchmarks.By contrast, training/evaluation overlap is easier to check when training data is available.Source: §2.1 · pp. 4–5

2.2 The Relationship Between Data Leakage Detection and Code Clone Detection

  • Unlike code clone detection, data leakage detection checks whether benchmark data overlaps an LLM's pre-training corpus to assess possible test exposure, across code, text, and structured data.Clone detection instead targets duplicate or semantically equivalent code to reduce redundancy; leakage detection aims to preserve evaluation validity by removing overlaps.Source: §2.2 · p. 5

3 METHODOLOGY

DetectLeak screens benchmark samples against LLM pre-training data, manually verifies candidate duplicates, and quantifies leakage. The resulting cleaned collection, LessLeak-Bench, removes all identified leaked samples from the studied benchmarks.

3 METHODOLOGY

  • DetectLeak first screens for potential duplicates, then manually verifies candidates and quantifies leaked benchmark samples and their leakage ratios.The pipeline identifies benchmark samples with confirmed duplicates in pre-training data as leaked.Source: §3 · p. 6, §3.3 · p. 8

3.1 Automated Data Leakage Detection

  • MinHash+LSH screens for potential duplicates across code and text without training data; candidate pairs exceed a Jaccard similarity threshold of 0.7.The study uses 2-gram tokens; exact matching misses semantic duplicates, while many clone-detection tools require training data or focus on code snippets.Source: §3.1 · pp. 6–7

3.2 Manual Labeling of Potential Duplicates

  • Two annotators independently classified candidate pairs into four categories, achieving Cohen’s kappa of 0.9424 for four-class labels and 0.9591 for binary labels.A third annotator resolved disagreements.Source: §3.2 · p. 7
  • 1,950 duplicate pairs and 4,741 non-duplicate pairs were identified among 6,691 pairs flagged by the automated tool.These counts are based on the selected LLM and SE benchmarks.Source: §3.2 · p. 8

3.3 Quantifying the Extent of Data Leakage

  • Leakage count is the number of leaked benchmark samples, while leakage ratio is N_leak/N_total.N_leak is the number of identified leaked samples and N_total is the benchmark’s total sample count.Source: §3.3 · p. 8

3.4 LessLeak-Bench

  • LessLeak-Bench removes all identified leaked samples from each studied benchmark to provide cleaned versions for LLM evaluation.The authors describe the collection as a more reliable and comprehensive benchmark across diverse SE tasks.Source: §3.4 · p. 8

4 EXPERIMENTAL SETUP

The study selects StarCoder and a diverse set of SE benchmarks for leakage analysis, using three programming languages. It defines four research questions spanning leakage extent, causes, evaluation effects, and behavioral inference.

4.1 LLM Selection

  • StarCoder was selected as the research LLM because it met the study’s openness, effectiveness, and influence criteria and served as a foundation for later models.The authors also expect findings to extend to models that incorporate StarCoder’s data or adopt similar curation practices.Source: §4.1 · pp. 8–10
  • StarCoder’s studied pre-training subsets contain 12M Python files, 20M Java files, and 14M C/C++ files.The pre-training corpus is The Stack, collected from public GitHub repositories between 2015 and 2022.Source: §4.1 · p. 10

4.2 Benchmark Data Selection

  • Benchmark selection drew on a survey of 395 LLM-for-SE papers, prioritizing varied tasks and excluding datasets without replication packages or accessible data.The final collection contains 83 datasets spanning a broad range of SE tasks.Source: §4.2 · pp. 10–11
  • Variants of multi-language or multi-scenario benchmarks received distinct names combining the original benchmark, scenario, and language when applicable.For example, a Python Code Debug variant is named CodeEditorBench-debug-py.Source: §4.2 · pp. 10–11

4.3 Implementation Details

  • Benchmark data came from replication packages or official websites, and the study used the BigCode Team’s MinHash+LSH implementation.Computations ran on an NVIDIA GeForce A5000 GPU with 24 GB of memory.Source: §4.3 · p. 11

4.4 Research Questions

  • The four research questions address leakage extent, factors behind high leakage, evaluation effects, and whether LLM behavior can indicate leakage without pre-training data access.The behavioral question considers signals such as model perplexity.Source: §4.4 · p. 11

5 EXPERIMENTAL RESULTS

Across the studied benchmarks, 2% of samples were flagged as potential duplicates with StarCoder’s pre-training data. Manual labeling confirmed 28% of flagged pairs as duplicates.

Overall Results

  • The study compared 46k Python, 42k Java, and 21k C/C++ benchmark samples against 12M, 20M, and 14M corresponding pre-training samples, respectively.This amounted to over 1.7 trillion comparisons.Source: §5 · p. 11
  • 2% of studied benchmark samples were flagged as potentially matching StarCoder’s pre-training data.Figure 3 presents the overall data leakage results.Source: §5 · p. 11, Figure 3 · p. 12
  • 28% of pairs flagged by MinHash+LSH were confirmed as duplicates after manual labeling, while 72% were not duplicates..Source: §5 · p. 12

5.1 RQ1: Data Leakage Status in SE Benchmarks

Leakage is low on average across languages, but varies sharply by benchmark and task. The section also reports benchmark-specific risks and introduces cleaned versions through LessLeak-Bench.

5.1.1 Data Leakage in Python Benchmarks.

  • Python benchmarks average 4.8% leakage, but rates reach 100% for QuixBugs and 10.8% for APPS; several other listed Python benchmarks have 0% leakage.Elevated rates may compromise evaluation fairness and validity.Source: §5.1.1 · pp. 12–13

5.1.2 Data Leakage in Java Benchmarks.

  • Java benchmarks average 2.8% leakage; BigCloneBench reaches 55.7%, while CodeEditorBench-switch-java reaches 9.9%.The authors recommend removing identified leaked samples before using those two benchmarks in future research.Source: §5.1.2 · pp. 13–14

5.1.3 Data Leakage in C/C++ Benchmarks.

  • C/C++ benchmarks average 0.7% leakage, the lowest language average, though CodeEditorBench-debug-c and CodeEditorBench-switch-c reach 4.5% and 8.3%.The authors emphasize removing identified duplicate data when using those two benchmarks.Source: §5.1.3 · p. 14

5.1.4 Data Leakage Status in Specific SE Tasks.

  • Code Generation benchmarks average 0.62% leakage, whereas Program Repair averages 12.5%; Program Repair, Code Editing, and Issue Fix also account for more top-10 high-leakage benchmarks.Twenty-six of 33 Code Generation benchmarks have 0% leakage, compared with one of nine Program Repair benchmarks.Source: §5.1.4 · pp. 14–15

5.1.5 LLM Evaluation with LessLeak-Bench.

  • LessLeak-Bench provides cleaned versions of the 83 studied benchmarks with identified leaked samples removed, for more reliable future LLM evaluations.The authors recommend it because many leaked samples are no longer suitable for evaluation.Source: §5.1.5 · p. 15

5.1 RQ1: Data Leakage Status in SE Benchmarks

  • Average leakage ratios are 4.8% for Python, 2.8% for Java, and 0.7% for C/C++, while QuixBugs and BigCloneBench reach 100.0% and 55.7%.Program Repair benchmarks average 12.5% leakage; LessLeak-Bench provides cleaned versions of the studied benchmarks with identified leaked samples removed.Source: §5.1.5 · p. 15

5.2 RQ2: Understanding High Data Leakage in SE Benchmarks

The analysis traces high leakage to overlap between benchmark and pre-training data, including cloned repositories and shared sources. It also identifies coding-platform dependence and offers risk-reduction recommendations.

5.2.1 Which Types of Software Repositories Are Likely Contributing to Data Leakage?

  • The PatrickShaw/QuixBugs repository contributed 92 duplicate pairs, while “leetcode” appeared in eight of the 20 most frequent repository names.The authors link the cloned QuixBugs repository in pre-training data to QuixBugs’ high leakage ratio.Source: §5.2.1 · p. 16
  • The authors recommend scanning and filtering repository names for benchmark names, coding-platform names, and algorithm-related keywords, and avoiding benchmarks sourced primarily from coding websites or tutorials.They also advise comparing chosen benchmarks against pre-training datasets to assess leakage risk.Source: §5.2.1 · p. 16, §5.2.2 · p. 17

5.2.2 Why Do Certain SE Benchmarks Exhibit High Data Leakage?

  • Four potential causes of high leakage are direct benchmark inclusion, overlap with source repositories, reliance on coding platforms such as LeetCode, and shared sources such as GitHub issues.Examples include QuixBugs, BigCloneBench and BugsInPy, LeetCode-derived benchmarks, and SWE-Bench.Source: §5.2.2 · pp. 16–17

5.3 RQ3: Impact of Data Leakage for LLM Evaluation

Using APPS, the authors compare LLM performance on leaked and non-leaked samples in zero-shot experiments. Models perform substantially better on leaked samples, indicating that leakage can inflate evaluation metrics.

  • The experiments use APPS because it has comprehensive correctness tests and enough leaked samples for comparison, unlike many other code-generation benchmarks.APPS contains 10,000 problems split into 5,000 training and 5,000 test samples; each problem averages 21.2 test cases.Source: §5.3 · pp. 18–19
  • The authors divide the APPS test set into leaked and non-leaked subsets, then evaluate StarCoder models of three sizes in zero-shot prompting using Pass@k.Pass@k measures the percentage of problems for which at least one of the top-k generated code samples passes all test cases.Source: §5.3 · p. 19
  • StarCoder-7b’s Pass@1, Pass@2, and Pass@3 scores are 4.9, 5.6, and 6.0 times higher on leaked than non-leaked APPS samples.The authors report that StarCoder-7b, StarCoder-3b, and StarCoder-1b consistently perform better on leaked samples.Source: §5.3 · p. 19

5.4 RQ4: Automatic Data Leakage Detection Without Knowledge of Pre-training Data

The authors investigate whether model perplexity can identify leaked benchmark data without access to pre-training data. Perplexity-based detection performs poorly, and leaked samples are not consistently less perplexing.

  • AutoDetectLeak-Bench contains 1,300 deduplicated samples balanced by random under-sampling of manually labeled leaked and non-leaked data.It serves as a test set for automated leakage-detection metrics.Source: §5.4 · pp. 19–20
  • Leaked samples are not necessarily lower in perplexity than non-leaked samples; above perplexity 10, leaked samples significantly outnumber non-leaked samples.The figure excludes the top 2% of high-perplexity outliers from both groups for clarity.Source: §5.4 · p. 20, Figure 4 · p. 20
  • The detection method ranks labeled samples by ascending perplexity and classifies the lowest-perplexity top-k as leaked, with k varied from 100 to 1,000.The approach assumes lower perplexity indicates higher model confidence and greater likelihood of prior exposure.Source: §5.4 · p. 20
  • Perplexity-based detection accuracy is only 40%–50% in most cases, making automated leakage detection with this metric challenging.The authors point to AutoDetectLeak-Bench as a test set for evaluating future methods when pre-training data is unavailable.Source: §5.4 · pp. 20–21

6 DISCUSSION

The discussion links benchmark leakage to evaluation bias and recommends practical checks and cleaned data for researchers and providers. Its findings are bounded by the studied benchmarks and models, and automated screening may miss leaked samples.

6.1 Implications

  • Leakage can inflate LLM performance metrics and undermine fair comparisons; QuixBugs is entirely leaked to StarCoder, making it unsuitable for evaluating StarCoder-trained models.The paper extends this concern to related models built on StarCoder or its pre-training data.Source: §6.1 · p. 21
  • LessLeak-Bench removes identified leaked samples, while AutoDetectLeak-Bench provides manually verified leaked and non-leaked labels for developing detection methods when pre-training data is unavailable.The authors describe the latter dataset as a foundation for future leakage-detection research.Source: §6.1 · p. 22, §6.3 · p. 24

6.2.1 Suggestions for LLM Providers.

  • Providers should check a wider range of benchmark overlaps, inspect repositories that identify or resemble benchmarks, and scrutinize LeetCode-related sources.The recommendations also call for checking instruction-tuning data for overlap with SE benchmarks.Source: §6.2.1 · pp. 22–23

6.2.2 Suggestions for LLM Users.

  • For analyzed benchmarks, researchers are advised to use LessLeak-Bench or apply DetectLeak to identify and remove potentially leaked samples.For new benchmarks, the paper cautions against relying on LeetCode or algorithm-focused tutorial repositories and suggests considering tasks introduced after model release.Source: §6.2.2 · p. 23

6.3 Threats to Validity

  • Findings may not generalize beyond the studied benchmarks and LLMs, and automated detection may miss leaked samples.The study used 83 diverse benchmarks and StarCoder, selected in part for its open pre-training data; manual verification of all pre-training data was impractical.Source: §6.3 · p. 23

7 RELATED WORK

The paper distinguishes its work from prior leakage and memorization studies through its focus on SE benchmarks, broader benchmark coverage, and large-scale manual labeling. It contrasts this scope with studies of smaller models, fewer benchmarks, or gray-literature methods.

7 RELATED WORK

  • Unlike Yang et al. [99], this study examines leakage in SE benchmarks rather than general memorization and adds large-scale manual labeling to duplicate detection.Yang et al. [99] studied CodeParrot and CodeParrotsmall using clone detection on model-generated outputs.Source: §7 · p. 24
  • Compared with Lopez et al., the study covers 83 rather than three SE benchmarks and analyzes StarCoder, models approximately 125 times larger than those examined by Lopez et al..The paper also uses large-scale manual labeling to check identified leakage.Source: §7 · p. 24
  • Against gray-literature studies focused on two or three benchmarks, the study evaluates 83 benchmarks across three languages and incorporates manual labeling.The authors also provide recommendations for LLM developers and users to mitigate leakage.Source: §7 · p. 24

8 CONCLUSION AND FUTURE WORK

The paper combines near-duplicate detection with manual labeling to analyze leakage across 83 SE benchmarks. Leakage is generally low but varies across benchmarks, and leaked samples can substantially raise evaluation scores; the authors plan to expand the analysis and investigate prevention methods.

8 CONCLUSION AND FUTURE WORK

  • The analysis combines an efficient near-duplicate detection algorithm with extensive manual labeling across 83 benchmarks and three programming languages.Source: §8 · p. 24
  • 4.8%, 2.8%, and 0.7% are the average leakage ratios for Python, Java, and C/C++ benchmarks, respectively, while leakage inflates evaluation metrics on leaked samples.StarCoder-7b achieved a Pass@1 score 4.9 times higher on leaked samples.Source: §8 · pp. 24–25
  • The study identifies direct benchmark inclusion, source-repository overlap, coding platforms such as LeetCode, and shared data sources such as GitHub issues as leakage causes.Source: §8 · p. 24
  • The authors plan to analyze additional benchmarks and explore methods to prevent or further reduce leakage.Source: §8 · p. 25
Loading 2502.06215v1…