PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Trustworthy LLMs for Code: A Data-Centric Synergistic Auditing Framework

Chong Wang, Zhenpeng Chen, Tianlin Li, Yilun Zhao, Yang Liu

arXiv:2410.09048v2cs.SE

TL;DR

Research on code-LLM trustworthiness has often focused on training or evaluation separately, leaving questions about alignment between stakeholders. DataTrust proposes a unified, data-centric framework that links training-data quality with model evaluation through inspection, synthesized evaluation data, and root-cause attribution. It also proposes a trustworthiness arena and identifies unresolved implementation challenges.

  • Problem

    Research has often addressed code-LLM trustworthiness in either training or evaluation, raising questions about whether stakeholders align in their understanding of trustworthiness and how to guide them toward a unified direction.

    Source: §I · p. 1

  • Method

    DataTrust links training-data quality indicators with model-evaluation trustworthiness indicators through training-data inspection, evaluation-data synthesis, and root-cause attribution.

    Source: §IV · p. 4

  • Results

    The paper presents DataTrust as a vision for unified, data-centric trustworthiness auditing that connects training and evaluation data and refines their indicator relationships.

    Source: §IV · p. 4

  • Takeaways & Limitations

    The framework aims to align understanding, standardize auditing, and improve transparency about code LLMs and their data for stakeholders.

    Source: §III · p. 4

    Precisely connecting model-trustworthiness indicators to training-data quality indicators is difficult because LLMs have complex working mechanisms.

    Source: A. Trustworthiness Indicators and… · p. 2

Abstract

from arXiv · show

LLM-powered coding and development assistants have become prevalent to programmers' workflows. However, concerns about the trustworthiness of LLMs for code persist despite their widespread use. Much of the existing research focused on either training or evaluation, raising questions about whether stakeholders in training and evaluation align in their understanding of model trustworthiness and whether they can move toward a unified direction. In this paper, we propose a vision for a unified trustworthiness auditing framework, DataTrust, which adopts a data-centric approach that synergistically emphasizes both training and evaluation data and their correlations. DataTrust aims to connect model trustworthiness indicators in evaluation with data quality indicators in training. It autonomously inspects training data and evaluates model trustworthiness using synthesized data, attributing potential causes from specific evaluation data to corresponding training data and refining indicator connections. Additionally, a trustworthiness arena powered by DataTrust will engage crowdsourced input and deliver quantitative outcomes. We outline the benefits that various stakeholders can gain from DataTrust and discuss the challenges and opportunities it presents.

I. TRUSTWORTHINESS AUDITING OF LLMS FOR CODE

LLMs for code are widely used, but concerns about their trustworthiness persist while research often treats training and evaluation separately. DataTrust proposes a unified, data-centric audit linking training-data quality with model evaluation and serving multiple stakeholders.

I. TRUSTWORTHINESS AUDITING OF LLMS FOR CODE

  • Trustworthiness concerns for code LLMs include robustness, security, timeliness, privacy, and fairness, as generated code may contain vulnerabilities that create security risks.The section also notes risks common to general-purpose LLMs, such as jailbreaking threats.Source: §I · p. 1
  • DataTrust links evaluation trustworthiness indicators to training-data quality indicators, inspecting training data, evaluating models with synthesized data, and attributing evaluation issues to corresponding training data.It also refines the indicator connections and proposes a crowdsourced arena with quantitative and comparative outcomes.Source: §I · p. 1
  • Vendors can use the audit cycle for training-data inspection, filtering, model training, evaluation, and root-cause analysis, while publishing reports can improve transparency for downstream stakeholders.Source: §I · p. 1
  • Evaluators can update trustworthiness reports for commercial and open-source models and training corpora, giving downstream stakeholders information for selecting models or data.Identified issues can also help upstream vendors investigate training data.Source: §I · p. 2
  • Users can review audit reports and contribute encountered issues to a crowdsourced arena, supporting evaluator refinement and vendor root-cause analysis.Source: §I · p. 2

II. DATATRUST: METHODOLOGY AND CHALLENGES

DataTrust organizes its methodology around linked training-data quality and model-trustworthiness indicators. Three iterative processes support inspection, evaluation-data synthesis, and root-cause attribution, alongside a data-driven arena.

II. DATATRUST: METHODOLOGY AND CHALLENGES

  • DataTrust compiles model-trustworthiness indicators across dimensions and links them to corresponding training-data quality indicators.The indicators guide training-data assessment and construction of evaluation data.Source: §II · p. 2
  • Its three data-centric processes are Training Data Inspection, Evaluation Data Synthesis, and Root Cause Attribution, operating iteratively during model training and evaluation.The processes continuously refine indicator connections while engaging multiple stakeholders.Source: §II · p. 2
  • A Data-Driven Trustworthiness Arena engages users and benchmarks mainstream code LLMs and their training datasets, drawing on existing tools, crowdsourced resources, analysis methods, and LLM-based agents.Source: §II · p. 2

A. Trustworthiness Indicators and Connections

DataTrust pairs training-data quality indicators with model-evaluation trustworthiness indicators. The proposed connections cover several code-related risks, but precise, fine-grained links remain difficult to establish.

A. Trustworthiness Indicators and Connections

  • Each T ◦ E entry pairs a training-data quality indicator, T, with a model-evaluation trustworthiness indicator, E.Source: A. Trustworthiness Indicators and… · p. 2
  • Insufficient training-data diversity across domains, functionalities, and identifiers may relate to robustness concerns, including inconsistent domain performance and vulnerability to adversarial attacks.Source: A. Trustworthiness Indicators and… · p. 2
  • Insecure repository code may be reflected in insecure model outputs, while outdated patterns, libraries, and APIs in training data may lead to outdated outputs.Source: A. Trustworthiness Indicators and… · p. 2
  • Training-data licensing and copied source files raise concerns about license-term violations and generated outputs, while sensitive credentials and personal information may be reproduced.Source: A. Trustworthiness Indicators and… · p. 2
  • Historical code data can perpetuate demographic biases in generated code or algorithms, and complex model mechanisms make precise, fine-grained indicator connections challenging.The proposed initial relationships draw on existing studies and are expected to be refined through community collaboration.Source: A. Trustworthiness Indicators and… · p. 2

B. Training Data Inspection

DataTrust proposes automated inspection of code-LLM training data using diverse scanning tools and LLM-based agents. Scaling scans and integrating tools with differing requirements and formats remain challenges.

B. Training Data Inspection

  • DataTrust automatically inspects code-LLM training data for potential quality issues that could affect model trustworthiness.Source: B. Training Data Inspection · p. 2
  • Inspection can combine code extraction and semantic annotation, vulnerability scanners, software composition analysis, clone detection, and privacy or fairness analysis.Source: B. Training Data Inspection · p. 2
  • Scanning corpora containing millions of files creates resource and time constraints; sampling part of a corpus can provide an approximate quality assessment.DeepSeek-Coder’s corpus is cited as containing 603 million files.Source: B. Training Data Inspection · pp. 2–3
  • LLM-based agents can refine and aggregate scanning outputs into measurable indicators, including profiling data diversity and counting deprecated libraries or APIs using version-tracing results and online documentation.Source: B. Training Data Inspection · p. 3
  • Integrating scanners is difficult because tools differ in prerequisites, input/output formats, and configuration options; proposed directions include unified formats, agent-selected configurations, and result aggregation.For example, Infer requires a compilable code project and allows customizable detection settings.Source: B. Training Data Inspection · p. 3

C. Evaluation Data Synthesis

DATATRUST synthesizes evaluation data from evolving, diverse resources and uses LLM-based agents to turn them into executable tests. Integrating heterogeneous crowd resources remains a challenge.

C. Evaluation Data Synthesis

  • DATATRUST automatically synthesizes evaluation data and benchmarks by referencing evolving resources to audit trustworthiness across indicators.Source: C. Evaluation Data Synthesis · p. 3
  • DATATRUST dynamically generates up-to-date evaluation data from diverse real-time resources, rather than relying only on manually curated benchmarks [33],.Sources include software platforms, API documentation, package tools, developer forums, vulnerability databases, search engines, and social media.Source: C. Evaluation Data Synthesis · p. 3
  • LLM-based agents fetch resource data, generate test cases and oracles, and inspect results for specific evaluation dimensions.For deprecated API testing, an agent identifies deprecated APIs from release notes, retrieves related code, and prompts the model under test.Source: C. Evaluation Data Synthesis · p. 3
  • Heterogeneous crowd resources such as informal Stack Overflow text complicate integration and motivate unified representations, adaptive parsing, and conflict resolution.The paper identifies information fusion and conflict resolution as additional research opportunities.Source: C. Evaluation Data Synthesis · p. 3

D. Root Cause Attribution

DATATRUST links evaluation-time trustworthiness issues to potentially problematic training instances, refining indicator connections and identifying gaps in data inspection. It combines attribution methods with LLM-based analysis, while code-specific performance and pattern accuracy remain challenges.

D. Root Cause Attribution

  • Root cause attribution links evaluation-time trustworthiness issues to problematic training instances, refining indicator connections and revealing instances missed by direct inspection.The resulting links can clarify which training instances significantly affect model trustworthiness when considered with direct inspection results.Source: D. Root Cause Attribution · p. 3
  • Instance-level attribution can use influence functions, similarity search [44], gradient-based analysis, or attention analysis, while pattern-level analysis can yield lightweight inspection rules.The paper gives CodeQL queries for vulnerability types as an example of such rules, which can support incremental inspection.Source: D. Root Cause Attribution · p. 3
  • LLM-based attribution agents combine outputs from attribution methods with code and natural-language analysis to select instances related to an evaluation issue.For a recurring vulnerability, an agent first identifies potentially contributing training instances, then compares their code patterns with the vulnerability patterns.Source: D. Root Cause Attribution · p. 3
  • Code-related performance of instance-level attribution methods remains untested, and accurate patterns are needed to guide reliable pattern-level attribution.Combining instance-level and pattern-level analysis for incremental inspection is also a challenge.Source: D. Root Cause Attribution · pp. 3–4

E. Data-Engined Trustworthiness Arena

DATATRUST proposes an arena with separate leaderboards for code LLMs and training data, maintained through its auditing processes. Community contributions could support test creation and validation, but complex tasks raise participation barriers.

E. Data-Engined Trustworthiness Arena

  • The trustworthiness arena maintains separate leaderboards for code LLMs and training data by applying DATATRUST’s three data-centric processes.LLMs with non-open-sourced training data can still be ranked on the code-model leaderboard using evaluation data synthesis alone.Source: E. Data-Engined Trustworthiness Arena · p. 4
  • Community contributors can create test cases, validate oracles, and confirm results through interfaces designed to reduce participation effort.LLM-based agents can guide contributors and present relevant vulnerability information as a readable checklist.Source: E. Data-Engined Trustworthiness Arena · p. 4
  • Code-security tasks create a higher participation threshold because even experienced developers may struggle to identify or verify vulnerabilities.The paper proposes simplifying tasks, such as asking users to verify checklist items like index-range validation instead of relying only on yes/no annotations.Source: E. Data-Engined Trustworthiness Arena · p. 4

III. RELATED WORKS

Prior work addresses general and code-specific trustworthiness, but the paper positions DATATRUST as a unified auditing framework that aims to align understanding, standardize processes, and improve transparency.

III. RELATED WORKS

  • General LLM trustworthiness research benchmarks dimensions including safety, fairness, robustness, privacy, and accountability, [28], [50], concerns that code LLMs can inherit through their training.The paper notes that fine-tuning general LLMs or training on text and code corpora carries broader trustworthiness concerns into code models.Source: §III · p. 4
  • Code-focused work covers trustworthiness dimensions and principles [32], correctness calibration [52], and specific areas such as synthesis [54], backdoor triggers [55], and code summarization [56].The section also cites a call for trustworthy and synergistic AI for Software Engineering.Source: §III · p. 4
  • DATATRUST aims to align understanding, standardize auditing processes, and enhance transparency about LLM trustworthiness and their data.The paper presents earlier works as a foundation for this unified auditing framework.Source: §III · p. 4

IV. SUMMARY

DATATRUST is a data-centric vision for unified trustworthiness auditing that links training and evaluation data. It connects evaluation trustworthiness indicators with training data quality indicators while supporting extensibility and evolvability, though several implementation challenges remain.

  • DATATRUST links model trustworthiness indicators in evaluation with data quality indicators in training through a unified, data-centric auditing framework.Source: §IV · p. 4
  • It inspects training data, evaluates model trustworthiness with synthesized data, and attributes potential causes from evaluation data to corresponding training data while refining indicator connections.Source: §IV · p. 4
  • Integration of diverse tools and resources supports extensibility, while real-world information and knowledge support evolvability; challenges include tool integration, resource aggregation, method adoption, and interaction paradigms.Source: §IV · p. 4

V. FUTURE PLANS

The future plan begins with a prototype using selected indicators, open-source models and training data, and code generation. It then develops the auditing processes, launches an arena, and refines DATATRUST through community collaboration and feedback.

  • The initial prototype will focus on security and copyright indicators for open-source code models such as StarCoder, open-source training data, and code generation.Source: §V · pp. 4–5
  • The next stage will implement three data-centric auditing processes using the selected indicators, models, and application task, with approaches designed to address identified challenges.Source: §V · p. 5
  • The plan then launches an arena with continuously updated leaderboards and refines DATATRUST through collaboration and broader-community feedback.Source: §V · p. 5
Loading 2410.09048v2…