Low-Resources Project-Specific Code Summarization

1 INTRODUCTION

• Project-specific code summarization (PCS) targets summaries for an individual project, unlike general code summarization (GCS), which uses independent code-summary pairs from many projects.Developers are described as more concerned with high-quality, consistent summaries for their own working projects.Source: §1 · p. 1
• GCS models may miss a target project’s domain knowledge and writing style, producing summaries that differ from its historical conventions.The Flink example contrasts a common “return true if” phrasing with the project’s “checks whether” style.Source: §1 · p. 1, Table 1 · p. 2
• Many projects lack enough historical summaries for fine-tuning: nearly one-third of surveyed projects have fewer than 100.The investigation sampled 300 projects from Apache, Google, and Spring.Source: §1 · p. 2, Table 2 · p. 2
• The method treats projects as independent domains and uses meta-learning to transfer shared cross-project knowledge through model weight initialization.Its project-specific prefixes keep the pretrained model parameters frozen while optimizing a small number of extra parameters.Source: §1 · p. 2
• Evaluation on nine real-world projects verifies the method’s overall effectiveness and the necessity of its individual components.The results also suggest research opportunities in project-specific code summarization.Source: §1 · p. 2
• MPCos integrates project-specific prefix-based fine-tuning into a meta-learning framework to capture project knowledge with limited training samples.The paper presents it as a baseline for future low-resource project-specific code summarization research.Source: §1 · p. 2

2 PROBLEM FORMULATION

• The task uses general knowledge from a large multi-project corpus and cross-domain knowledge from accompanying projects to generate summaries for a target project.The nine-target-project setting gives each target eight accompanying projects; a target may also accompany other projects.Source: §2 · p. 3
• The study focuses on low-resource target-project settings with N_tgt = 10 and N_tgt = 100 code-summary pairs.N_tgt denotes the number of code-summary pairs in the target project.Source: §2 · p. 3

3 METHODOLOGIES

• MPCos comprises a Transformer code-summarization module, a meta-transfer module for prefix initialization, and separate project prefixes intended to avoid cross-contamination.The method uses the Transformer as its backbone.Source: §3 · p. 3

3.1 Code Summarization

• The code summarizer uses a Transformer encoder-decoder to encode input code and generate the summary autoregressively, trained by minimizing negative log-likelihood.The model is first trained on a general code-summarization corpus.Source: §3.1 · p. 3

3.2 Prefix Tuning

• MPCos inserts project-specific prefix vectors into Transformer layers, updates prefix-related parameters, and retains only the prefixes after training.The framework first meta-trains prefix initialization, then uses it to initialize the target-project model for prefix tuning.Source: §3.2 · p. 3, Figure 1 · p. 4

3.3 Meta Transfer Learning

• Meta-transfer learning adapts project-specific prefix parameters on support sets and evaluates them on query sets before updating the shared initialization.The base model parameters are frozen during this process; the authors describe the learned initialization as enabling faster adaptation across projects.Source: §3.3 · p. 4

3.4 Meta Datasets Selection

• For each target project, meta-datasets are drawn from other projects, with top projects selected by average ranking on ROUGE-2 recall, code representation similarity, and code length.The selection discussion lists additional possible metrics, but the stated selection criteria are these three.Source: §3.4 · pp. 4–5, §3.3 · p. 4

4.1 Project-Specific Code Summarization Dataset

• The authors curate more than 3.7k Java method-summary pairs from nine active projects across Apache, Google, and Spring.They extract the first sentence or, if no period is present, the first newline; they remove tool-generated, very short, and non-English summaries.Source: §4.1 · p. 5

4.2 Principle behind the Dataset Construction

• Projects are selected across diverse domains and organizations to expose models to varied transfer-learning scenarios.The stated rationale is to support generalization of domain adaptation capabilities.Source: §4.2 · p. 5
• The evaluation uses well-maintained projects to preserve golden summaries for testing, so it simulates rather than directly samples poorly documented projects.The authors call this setting not perfect, noting that using only ten summaries for training would leave no extra golden summaries for testing.Source: §4.2 · p. 5

• Each project's code-summary pairs are randomly divided into five equally sized folds, with one fold held out for testing at each step.Source: §4.3 · p. 5
• The other four folds form the candidate training set from which fine-tuning samples are drawn.Source: §4.3 · p. 5
• The two low-resource settings use randomly selected samples of 10 or 100 pairs for fine-tuning.Source: §4.3 · p. 5

4.4.1 Hyperparameters.

• Transformer parameters are frozen except for the last linear layer to reduce overfitting in the low-resource setting.The model is trained for up to 30 epochs with early stopping after five consecutive validation iterations without improvement.Source: §4.4.1 · pp. 5–6

4.4.2 Evaluation details.

• For each target project, the remaining projects provide meta-data, and the meta-learned model is fine-tuned with 10 or 100 labeled samples.Source: §4.4.2 · p. 6
• Target-project fine-tuning is repeated 10 times to reduce randomness, and the average is used for each cross-validation step.Source: §4.4.2 · p. 6

• BLEU and ROUGE-L are used to evaluate GCS and PCS model performance.Source: §4.5 · p. 6
• BLEU measures candidate-reference similarity by counting shared n-grams.Source: §4.5 · p. 6
• ROUGE-L captures sentence-level structural similarity using automatically identified longest co-occurring sequence n-grams.Source: §4.5 · p. 6

4.6 Baselines

• The comparison set covers prior source-code and text-summarization work, grouped into GCS and transfer-learning-based models.Source: §4.6 · p. 6

4.6.1 GCS models. •

• The GCS baselines include LSTM, dual-input GRU, contextual, Transformer, and retrieval-based approaches, including EditSum's prototype-editing method.Source: §4.6.1 · p. 6
• Table 4 emphasizes the opportunity to add project-specific knowledge to GCS models, not an unfair cross-scenario GCS-versus-PCS comparison.Source: Table 4 · p. 6

4.6.2 Transfer-learning-based models. •

• Transfer-learning baselines include adapter methods and MTL-ABS, which leverages multiple corpora for abstractive summarization.Source: §4.6.2 · p. 6

5.1 RQ1: MPCos v.s. GCS models

• MPCos(10) and MPCos(100) improve over GCS models on PCS, including nearly 100% relative BLEU enhancement over EditSum.The authors caution that the PCS and GCS scenarios use different datasets and hyperparameters, so their scores are not directly comparable.Source: §5.1 · p. 7

5.2 RQ2: MPCos vs. Transfer-learning-based models

• MPCos outperforms the transfer-learning baselines, and moving from 10 to 100 examples yields an average 30% relative BLEU enhancement.Adaptor and MAM-Adaptor underperform in PCS; MPCos(10) also substantially improves over Transformer.Source: §5.2 · p. 7

5.3 RQ3: Effect of meta learning

• Removing meta-training lowers BLEU and Rouge-L in all projects; overall scores fall from 7.03/30.63 to 6.38/29.92 and from 9.10/33.21 to 7.78/31.70 in the 10- and 100-sample settings, respectively.The authors conclude that source projects provide valuable information, particularly when a target project has few historical summaries.Source: §5.3 · p. 8

5.4 RQ4: Effect of prefix-tuning

• At 100 samples, removing prefix-tuning lowers BLEU and Rouge-L from 9.10/33.21 to 8.02/31.65, while performance is similar at 10 samples.The authors report that MPCos degrades less than MTL-ABS when initialized from the pretrained model: 28% versus 61% BLEU decrement on Guava.Source: §5.4 · p. 8

5.5 RQ5: How project-specific knowledge be learned

• With 10 examples, MPCos mainly learns shallow textual style; with more examples, it captures deeper source-code semantics and can correct a factual error in the Transformer and MPCos(10) example.The reported word-frequency changes shift from mostly high-frequency words in the preliminary stage to a more balanced pattern in the refining stage.Source: Figure 2 · p. 8, §5.5 · p. 9

5.6 Human Evaluation

• MPCos outperforms all baselines on naturalness, informativeness, and relevance in the nine-project human evaluation.With 100 samples, it improves informativeness and relevance over the 10-sample setting while achieving similar naturalness.Source: §5.6 · p. 9, Table 7 · p. 9
• MPCos outperforms all baselines on naturalness, informativeness, and relevance; compared with GCS models, its gains are larger for informativeness and relevance than naturalness.The authors interpret the 100-sample gains in informativeness and relevance, alongside similar naturalness, as evidence of progressively learned project-specific knowledge.Source: §5.6 · p. 9, Table 7 · p. 9

6.1 Time cost of Meta-Prefix tuning

• Inference averages 102.4 samples/second for MPCos and GCS, while MPCos adds meta-tuning and project-specific tuning.Across nine projects, average tuning costs were 148 seconds for meta-tuning and 99 seconds for project-specific tuning; the authors consider the extra training short relative to code revision periods.Source: §6.1 · pp. 9–10

6.2 Directions of Future Research

• Future work includes extracting explicit project knowledge and studying zero-shot PCS, where no historical code summaries are available.The authors also raise which methods should be manually summarized first when converting zero-shot PCS into a low-resource setting.Source: §6.2 · p. 10

6.3 Threats to Validity

• Evaluation results may apply only to the nine selected projects, which cannot represent all practical multi-project scenarios.The authors identify project selection as a nontrivial topic for future research.Source: §6.3 · p. 10
• Using each code document’s first sentence as ground truth can introduce noise, including mismatches between methods and their one-sentence summaries.The authors note this is also common practice in prior code-summarization research [13].Source: §6.3 · p. 10
• The EditSum result is based on a reimplementation whose details may differ from the original work.The authors state that they tried to follow the original paper carefully.Source: §6.3 · p. 10

7.1 General Code Summarization

• Neural code summarization adapts text-generation techniques, while project context has also been used to augment models.The cited project-level encoder uses selected code files as features, but the authors say it offers limited information utilization and makes style or domain inference difficult.Source: §7.1 · p. 10

7.2 Parameter Efficient Learning

• Parameter-efficient methods freeze most model parameters and train smaller components; this paper follows prefix tuning [17] to capture project-specific knowledge.The cited approaches include Adapters [11], Projected Attention Layers [22], and a unified model [9].Source: §7.2 · p. 10

7.3 Meta-Transfer Learning

• Meta-transfer learning addresses limited-resource adaptation, and this work builds on Chen [5] while replacing adapters with prefix tuning for code summarization.Chen [5] studied source-corpus selection for low-resource text summarization; the authors report that adapter performance is poor for code summarization.Source: §7.3 · p. 10

8 CONCLUSION

• Low-resource project-specific code summarization targets working projects with limited historical code summaries, a setting not examined by prior code-summarization efforts on specific projects.The authors frame the task around developers’ interest in high-quality summaries for their own projects.Source: §8 · p. 11
• The method integrates project-specific prefix-based fine-tuning into a meta-transfer learning framework to capture project knowledge from limited training samples.Source: §8 · p. 11
• The authors present the method as a baseline for future low-resource project-specific code-summarization studies.They also discuss open questions and research opportunities.Source: §8 · p. 11