Source-linked AI summary
CC2Vec: Distributed Representations of Code Changes
Thong Hoang, Hong Jin Kang, Julia Lawall, David Lo
TL;DR
Existing patch representations often depend on manually crafted, task-specific features. CC2Vec learns distributed code-change representations from log-message semantics using hierarchical attention, and models using these representations outperform state-of-the-art approaches across three software-engineering tasks.
Problem
Prior software-patch studies commonly rely on manually crafted features for individual tasks, motivating a more general code-change representation.
Method
CC2Vec learns distributed code-change vectors from accompanying log messages, modeling code-change structure with hierarchical attention and comparing removed with added code.
Results
Models using CC2Vec outperform existing state-of-the-art approaches on log-message generation, bug-fixing patch identification, and just-in-time defect prediction.
Takeaways & Limitations
CC2Vec provides code-change representations that can be integrated into existing models for multiple tasks involving code changes.
Takeaways & Limitations
The evaluation covers only three tasks, so CC2Vec may not generalize beyond the tasks studied.
Abstract
from arXiv · showhide
Existing work on software patches often use features specific to a single task. These works often rely on manually identified features, and human effort is required to identify these features for each task. In this work, we propose CC2Vec, a neural network model that learns a representation of code changes guided by their accompanying log messages, which represent the semantic intent of the code changes. CC2Vec models the hierarchical structure of a code change with the help of the attention mechanism and uses multiple comparison functions to identify the differences between the removed and added code. To evaluate if CC2Vec can produce a distributed representation of code changes that is general and useful for multiple tasks on software patches, we use the vectors produced by CC2Vec for three tasks: log message generation, bug fixing patch identification, and just-in-time defect prediction. In all tasks, the models using CC2Vec outperform the state-of-the-art techniques.
1 INTRODUCTION
CC2Vec addresses the limits of manually crafted, task-specific patch features by learning code-change representations guided by log-message semantics. Its representations support multiple software-engineering tasks and improve performance over existing approaches.
- Motivation: Prior patch-based approaches commonly rely on manually crafted features specific to patch properties, history, authors, and reviewers.These features are then used with machine-learning classifiers for software-engineering tasks.
- Approach: CC2Vec learns distributed code-change representations supervised by the semantic meaning of accompanying log messages.It optimizes code-change vectors to predict words from the first line of each log message.
- Approach: CC2Vec models removed and added code across files using hierarchical structure and attention.The architecture constructs representations for code organized as words, lines, and hunks.
- Motivation: The learned representations can leverage large collections of unlabeled patches to improve supervised tasks, especially when labeled data are limited.This supports a semi-supervised learning use of code-change representations.
- Evaluation: CC2Vec improves over the best baselines across log-message generation, bug-fixing patch identification, and just-in-time defect prediction.Reported gains include 24.73% BLEU improvement, 5.22%–9.18% across bug-fixing metrics, and 7.03%–7.72% AUC improvement for defect prediction datasets.
2 APPROACH
CC2Vec learns patch code-change representations from log-message semantics, preserving patch hierarchy while comparing removed and added code. Its framework extracts per-file embeddings, combines them into a patch-level vector, and trains that vector through word prediction.
- Framework overview: CC2Vec takes a patch’s code changes and first separates them into affected files containing removed and added code lines.The preprocessing stage extracts file-level code documents before further processing.
- Learning objective: The first line of a patch’s log message supervises learning by treating its words as semantic labels and minimizing differences between predicted and actual words.The learned code-change vector is obtained from the intermediate output between feature extraction and feature fusion layers.
- Input layer: Each changed file is encoded as a three-dimensional matrix organized by hunks, lines, and words, with padding or truncation enabling parallelized inputs.The representation uses dimensions F, H, L, and W for files, hunks, lines, and words.
- Feature extraction layers: Embeddings from affected files are concatenated into a single code-change vector representing the patch.The per-file vectors are combined after feature extraction to form the patch-level representation.
- Feature extraction layers: Hierarchical attention networks separately embed removed and added code, then comparison layers construct vectors capturing their differences.The HAN processes word, line, and hunk structure, while comparison layers operate on the resulting removed-code and added-code embeddings.
- Hierarchical attention network: Within the HAN, bidirectional GRUs summarize contextual word information, while learned word-level attention weights highlight words informative for predicting log-message content.Word annotations concatenate forward and backward hidden states before attention scores are computed against a learned context vector.
eNT
CC2Vec compares removed and added code embeddings through multiple functions, concatenates the resulting file representations, and predicts log-message words from the patch representation.
- Comparison layers: Five comparison functions capture different aspects of the difference between removed and added code embeddings.They include a neural tensor network, a feed-forward neural network, euclidean distance, cosine similarity, element-wise subtraction, and element-wise multiplication.
- Comparison layers: The comparison-layer outputs are concatenated into an embedding vector for each affected file.The resulting file vector is denoted efi, where fi is the i-th file in the patch.
- Feature fusion: CC2Vec concatenates affected-file embedding vectors to form the patch-level code-change representation ep.The affected-file vectors are produced from feature extraction layers and then combined into one patch representation.
- Feature fusion and word prediction: The patch representation passes through a fully connected hidden layer and a word-prediction layer to estimate probabilities for words in log messages.VM contains words extracted from the first line of log messages, and the output assigns each word a probability score.
- Parameter learning: Training learns code-change embeddings, encoder states, comparison parameters, and prediction-layer parameters while using regularization, dropout, Adam, and backpropagation.The objective includes an L2 regularization term to prevent overfitting.
3 EXPERIMENTS
The experiments evaluate whether CC2Vec representations support three software-engineering tasks and compare models with and without the learned code-change vectors.
- 3 EXPERIMENTS: The evaluation covers log message generation, bug fixing patch identification, and just-in-time defect prediction.These tasks test whether one code-change representation can be applied across multiple software-engineering problems.
- 3 EXPERIMENTS: For log message generation, CC2Vec vectors identify similar patches whose log messages can be reused.For the other two tasks, CC2Vec supplies additional features to the task models.
- 3 EXPERIMENTS: The study compares performance with and without the CC2Vec code-change vector.The experiments describe the tasks, baselines, and results for these comparisons.
3.1 Task 1: Log Message Generation
Log message generation addresses the need for concise patch summaries by retrieving messages for code changes using CC2Vec-based similarity rather than bag-of-words similarity. LogGen outperforms NNGen and NMT on both cleaned and original datasets under BLEU-4, with the reported improvements indicating stronger alignment with patch meaning.
- Motivation: Log message generation aims to produce a brief summary of a patch’s code change because developer log messages can be empty or low quality.Log messages support program comprehension and understanding software evolution.
- State-of-the-art approaches: NNGen represents code changes as bags of words, computes cosine similarity, and reuses messages from the nearest training patches.NMT instead models the task as neural machine translation from code changes to log messages.
- LogGen: LogGen trains CC2Vec, retrieves training patches with the closest code-change vectors, and reuses their known log messages.This replaces NNGen’s bag-of-words representation with CC2Vec vectors.
- Evaluation: BLEU-4 evaluates LogGen, NNGen, and NMT on original and cleaned datasets containing 30K/3K and 22K/2.5K training/testing patches, respectively.The cleaned dataset filters bot and trivial log messages, while BLEU-4 measures closeness to a human translation.
- Results: 24.75%: LogGen improves over NNGen on the cleaned dataset in BLEU-4.On that dataset, LogGen exceeds NNGen and NMT by BLEU-4 scores of 4.06 and 6.29, respectively.
- Results: 12.06% and 2.07%: LogGen improves over NNGen and NMT, respectively, on the original dataset in BLEU-4.The corresponding BLEU-4 differences are 4.65 over NNGen and 11.28 over NMT.
- Results: The authors conclude that LogGen’s retrieved messages are closer to human translations and that CC2Vec correlates with patch meaning more strongly than bag-of-words.This conclusion is based on LogGen outperforming NNGen and NMT.
3.2 Task 2: Bug Fixing Patch Identification
CC2Vec augments bug-fixing patch classifiers with learned code-change representations, improving both PatchNet and LPU-SVM over their original versions.
- Existing Approaches: PatchNet represents removed and added code as a three-dimensional matrix and uses a 3D-CNN, but lacks attention to important words, lines, and hunks.
- Our Approach: CC2Vec representations are concatenated with existing embeddings for PatchNet and supplied as features to the LPU-SVM classifier.
- Experimental Setting: 42K bug-fixing and 40K non-bug-fixing Linux kernel patches comprise the evaluation dataset, with changes limited to 100 lines.
- Results: PatchNet augmented with CC2Vec improves over PatchNet by 5.22% accuracy, 9.18% precision, 4.37% F1, and 6.51% AUC.
- Results: LPU-SVM augmented with CC2Vec improves over LPU-SVM by 5.47% accuracy, 2.80% precision, 11.45% recall, 7.09% F1, and 4.24% AUC.
3.3 Task 3: Just-in-Time Defect Prediction
The JIT defect-prediction evaluation tests whether CC2Vec can augment DeepJIT for classifying defective patches, using datasets from QT and OPENSTACK.
- Task: JIT defect prediction is modeled as binary classification of patches labeled as containing a defect or not.
- Existing Approach: DeepJIT uses a CNN to extract features from a patch’s log message and code change but ignores the structure of removed and added code.
- Our Approach: CC2Vec code-change vectors are concatenated with DeepJIT’s embedding vectors for each patch.
- Experimental Setting: The evaluation uses 25K QT patches and 12K OPENSTACK patches, with 8% and 13% labeled defective, respectively.
- Results: CC2Vec improves DeepJIT’s AUC by 7.03% on QT and 7.72% on OPENSTACK.
4 DISCUSSION
The discussion reports that comparison functions materially support CC2Vec’s performance, while validity concerns remain about experimental error and generalization beyond three tasks.
- Ablation Study: CC2Vec and six reduced variants are compared across log message generation, bug-fixing patch identification, and JIT defect prediction.
- Ablation Results: CC2Vec outperforms every reduced variant across all three tasks, while removing all comparison functions performs worst.
- Threats to Validity: The authors reused existing baseline implementations and double-checked code and data, but errors may remain.
- Threats to Validity: Only three tasks were studied, limiting evidence about generalization beyond those tasks despite coverage of different projects and programming languages.
5 RELATED WORK
Related work learns representations of source code and code changes using varied structures and neural methods; CC2Vec instead targets general, non-task-specific representations of multi-file changes without requiring parseable code.
- Source-Code Representations: Prior studies learn representations for identifiers, APIs, libraries, control-flow graphs, symbolic traces, and runtime execution traces.
- Source-Code and Language: Other work jointly learns source-code and natural-language embeddings or predicts natural-language outputs such as code summaries.
- Structural Methods: Structural representation methods differ from CC2Vec, including AST-based sequences and code2vec representations.
- Design Boundary: CC2Vec does not require parseable code, allowing use on patches with compilation errors and avoiding retrieval and parsing of entire changed files.
- Scope of Representation: Unlike most prior representations focused on a single function, CC2Vec represents changes spanning multiple functions and files.
- Generality: CC2Vec is task-general and orthogonal to PatchNet and DeepJIT, supplying representations that can augment both generative and classification models.
6 CONCLUSION
CC2Vec represents code changes with hierarchical attention and supports existing models across code-change tasks. Evaluation finds improvements over state-of-the-art approaches, while future work targets broader external validity.
- CC2Vec: CC2Vec produces distributed code-change representations through a hierarchical attention network.The model captures structural information and identifies important code-change aspects relative to accompanying log messages.
- Evaluation: Approaches using or augmenting CC2Vec embeddings outperform existing state-of-the-art approaches without those embeddings across three tasks.The evaluated tasks are log message generation, bug fixing patch identification, and just-in-time defect prediction.
- Future work: Future work will integrate CC2Vec into other tools and evaluate it on additional software-patch tasks to reduce the threat to external validity.This identifies a scope boundary for the current evaluation.