CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context

1 Introduction

• Modular programming requires knowledge from both the current file and the project, while in-file-only models can miss imported class members and function arguments.The paper identifies finding relevant cross-file context and combining it with in-file context under input-length limits as challenges.Source: §1 · pp. 1–2
• CCFINDER retrieves relevant project entities, and COCOMIC jointly learns in-file and retrieved cross-file context for code completion.CCFINDER analyzes imports and retrieves neighboring entities from a project context graph; COCOMIC compresses cross-file context and attends to it alongside concrete in-file context.Source: §1 · p. 2
• 33.94% relative improvement in exact match and 28.69% relative improvement in identifier matches were reported for COCOMIC with relevant cross-file context over the CodeGen backbone.The evaluation used a statement-level code-completion dataset built from open-source Python projects on PyPI.Source: §1 · p. 2
• The authors report that COCOMIC with CCFINDER context outperforms baselines by up to +33.94% in exact match.They also conduct ablation studies of different components.Source: §1 · p. 2
• The paper releases a diverse, high-quality dataset for statement-level completion intended to test models’ use of cross-file context.Source: §1 · p. 2

2 Preliminaries

• Project entities are files, functions, classes, and global variables, each represented by selected code details such as signatures, docstrings, bodies, or values.Source: §2 · p. 2
• Entity relations capture intra-file code hierarchies and inter-file dependencies between files.The passage notes that each category has further relation types.Source: §2 · pp. 2–3
• Locale assigns each entity a unique name based on its relative project location, preserving a one-to-one mapping between entity and location.It helps map code snippets to project entities and indicates hierarchical relations.Source: §2 · p. 3
• In-file context is code before the prediction position in the current file; cross-file context is relevant project code outside that file and imported by it.Source: §2 · p. 3

3 Cross-file Context Finder: CCFINDER

• CCFINDER addresses the need to navigate project dependencies and extract concrete code details, which the described off-the-shelf dependency tools do not provide together.Source: §3 · p. 3

3.1 Project Context Graph

• The project context graph represents code components as nodes and their relations as directed, typed edges, organized from a project root through files to component-level structures.The graph also connects files to imported files and links components within a file according to dependencies or scope.Source: §3.1 · pp. 3–4

3.2 Cross-file Context Retrieval

• CCFINDER locates nodes for local imports and retrieves their neighbors within a configurable number of graph hops as cross-file context.The experiments set k = 2, then reorder retrieved nodes to follow original code order within each source file.Source: §3.2 · p. 4

4 Proposed Framework: COCOMIC

• CoCoMIC encodes the in-file snippet and retrieved cross-file context with an autoregressive language model, predicting each next token from both.The framework builds on a pretrained code language model.Source: §4 · p. 4

4.1 Input Representation

• Each cross-file entity carries a locale as a relational hint and ends with a [SUM] token used to represent that entity.The [SUM] token can attend to the preceding entity tokens; it performs better than mean pooling.Source: §4.1 · pp. 4–5

4.2 Encoding Cross-file Context

• CoCoMIC encodes each retrieved entity into a single representation, balancing input-length limits against the need to retain useful cross-file information.The representation is taken from the entity’s final [SUM] token.Source: §4.2 · p. 5

4.3 Modeling In-file and Cross-file Context for Code Completion

• At each Transformer layer, CoCoMIC combines keys and values from the in-file and cross-file contexts so the next-token state can attend to both.The in-file context at time step t consists of the preceding source-code tokens.Source: §4.3 · pp. 5–6

5.1 Data

• The dataset contains 60,891 permissively licensed Python projects and 6,888 held-out prompts for statement-level completion involving local APIs.Projects with at most five Python files or at least 5,000 graph nodes were filtered out; test projects were excluded if used as training-project dependencies.Source: §5.1 · p. 6

5.2 Implementation Details

• CCFINDER uses Tree-sitter to parse code, and retrieves up to 128 entities within two graph hops, with up to 128 tokens per entity.The paper states these thresholds were data-driven to cover most relevant cross-file context.Source: §5.2 · p. 6
• Experiments fine-tune CodeGen-350M-Mono for five epochs and compare zero-shot and fine-tuned CodeGen with a fine-tuned text-prompt baseline.The text-prompt baseline prepends cross-file context and reserves the first 128 input tokens for it.Source: §5.2 · p. 6, §5.3 · p. 6

5.3 Baselines & Evaluation Metrics

• Evaluation measures code completion with exact match and BLEU-4, API prediction with identifier-match exact match, precision, and recall, and token perplexity.Perplexity assesses performance on test tokens for which cross-file context is not always required.Source: §5.3 · p. 6
• +33.94% Code Match EM and +28.69% ID Match EM over the in-file-only baseline are reported for CoCoMIC with cross-file context.The table caption also reports no degradation in perplexity when evaluating all test-set tokens, where cross-file context is not always required.Source: Table 1 · p. 7

6.1 COCOMIC Outperforms the Baselines

• COCOMIC outperforms all baselines on every metric; unlike plain-text prompting constrained by input length, it encodes each entity as one token to fit more context.The authors also report the lowest perplexity, including tokens not requiring cross-file context.Source: §6.1 · p. 7

6.2 CCFINDER Retrieves Relevant Cross-file Context

• Retrieved identifiers raise recall from 75.19% in-file to 95.55% (a 27.07% increase), while Table 1 shows only a 7.08% identifier-recall improvement.The authors suggest that intelligent prompting or training can better use retrieved context.Source: §6.2 · p. 7, Table 2 · p. 7
• Random entities from the same project do not provide useful information because they are not necessarily related to the input code; 2-hop retrieval outperforms 1-hop retrieval.Source: §6.2 · p. 7

6.3 Entity Representation with [SUM] Token

• The [SUM] token significantly outperforms mean pooling for representing cross-file context.The comparison kept the rest of the settings the same.Source: §6.3 · p. 8, Table 4 · p. 8

6.4 Locales Help Learning Cross-file Context

• Locales consistently improve code-completion metrics, whereas auxiliary edge prediction achieves 97.2% accuracy but hardly improves completion.The authors hypothesize that locales provide direct textual signals such as class_name.method_name.Source: §6.4 · p. 8

• Most code-generation models were prompted with snippets from the current file; Zhou et al. (2022) retrieved API documentation from natural-language intent, whereas this work retrieves project code using import statements.Source: §7 · p. 8
• Earlier software-engineering tools extracted repository information for code-fragment completion, while later neural work modeled intra- and inter-class context using UML class diagrams (Wang et al., 2021a).Source: §7 · p. 8
• Recent approaches include repository-level prompt generation (Shrivastava et al., 2022) and iterative retrieval-generation with cross-file context (Zhang et al., 2023).Source: §7 · p. 9

• The work targets the absence of cross-file context, which limits code LMs' practicality in modern software development.Source: §8 · p. 9
• CCFINDER builds a project context graph and uses import statements to retrieve relevant cross-file context.Source: §8 · p. 9
• CCFINDER retrieves 27.07% more relevant context, and the best CoCoMIC model achieves a 33.94% relative improvement over the in-file-context-only baseline.Source: §8 · p. 9

• The study focuses on Python projects; extending the approach to other languages and incorporating third-party packages are left for future work.Source: Limitations · p. 9
• CoCoMIC assumes access to other project files; without cross-file context, it performs 5–7% lower than fine-tuned CodeGen.Source: Limitations · p. 9
• Experiments use CodeGen-350-mono, and the authors acknowledge they have not established that the technique improves models of any size.Source: Limitations · p. 9

Ethics Statement

• The authors do not expect negative broader impact, while noting code LMs can generate biased, toxic, or insecure code; they also report computational-resource usage in Appendix D.Source: Ethics Statement · p. 9

A Edge Types of Project Context Graph

• CCFINDER uses directed project-graph edges, including reverse edges that let it reach sibling entities through a shared parent file.For example, reverse edges can connect an imported function through its file to global variables it depends on.Source: A Edge Types of Project Context Graph · p. 12, Table 6 · p. 12

B Statistics of Retrieved Entities

• CoCoMIC caps retrieved context at 128 entities and 128 tokens per entity, aiming to include most cross-file context without excessive computational overhead.Tables 7 and 8 report threshold distributions for retrieved entities per file and tokens per entity.Source: B Statistics of Retrieved Entities · p. 12, Table 7 · p. 12, Table 8 · p. 12

C Additional Details on Data Preprocessing

• Long code files are split into model-length chunks paired with the same retrieved context; at test time, prompts are cut where completion needs cross-file context, whose long entity sequence is compressed with [SUM].The test-set prompt and cross-file-context length statistics are summarized in Table 9.Source: C Additional Details on Data… · p. 12, Table 9 · p. 12

D Additional Details on Experiments

• Experiments used eight Nvidia A100s, took about 400 GPU hours per job, and were run once because repetition was expensive; selected hyperparameters had relatively stable performance in an initial small-scale search.Source: D Additional Details on Experiments · pp. 12–13

E.1 k-hop Retrieval

• Two-hop retrieval outperformed one-hop retrieval because it reaches class members through imported files, while retrieval beyond two hops brought too many unrelated entities.The authors therefore used k = 2 throughout the work.Source: §E.1 · p. 13

E.2 Additional Baseline Variants

• Simplified cross-file context only marginally improves the baseline, which still underperforms CoCoMIC; CoCoMIC compresses up to 16,384 tokens into 128 vectors.Source: §E.2 · p. 13

F Case Study

• The case study compares CoCoMIC with finetuned CodeGen models both with and without cross-file context.Source: F Case Study · p. 13

F.1 COCOMIC vs. CodeGen finetuned w/o cross-file context

• In Figures 4–5, the baseline predicts methods described as nonexistent, while CoCoMIC predicts the correct functions in the examples.The paper characterizes inaccurate or unfaithful generation given the input as hallucination.Source: §F.1 · p. 13, Figure 4 · p. 14, Figure 5 · p. 14

F.2 COCOMIC vs. CodeGen finetuned with cross-file context

• In Figures 6–7, the baseline’s cross-file context is truncated, while CoCoMIC encodes entities individually and uses their embeddings in self-attention to predict correctly.Selecting the most useful retrieved entities to improve the baseline is left for future work.Source: §F.2 · p. 13, Figure 6 · p. 15, Figure 7 · p. 16