PaperlayerView exact revision on arXiv

Source-linked AI summary

LongCodeZip: Compress Long Context for Code Language Models

Yuling Shi, Yichun Qian, Hongyu Zhang, Beijun Shen, Xiaodong Gu

arXiv:2510.00446v1cs.CLcs.SE

TL;DR

Long-context code processing is costly and difficult because existing compression methods can miss code structure and dependencies. LongCodeZip introduces training-free hierarchical compression that selects relevant functions and then prunes their blocks under adaptive budgets. Across code completion, summarization, and question answering, it achieves up to a 5.6× compression ratio without sacrificing task performance.

  • Problem

    Existing long-context compression methods overlook code-specific structure and dependencies, while large code contexts create substantial computational and API-cost bottlenecks.

  • Method

    LongCodeZip uses training-free hierarchical compression: conditional-perplexity function selection followed by perplexity-based block pruning with adaptive token allocation.

  • Results

    Across code completion, summarization, and question answering, LongCodeZip achieves up to a 5.6× compression ratio without sacrificing task performance and consistently outperforms baselines.

  • Takeaways & Limitations

    LongCodeZip reduces context size and computational costs while preserving competitive performance across tasks, models, and a lightweight 0.5B compression model.

  • Takeaways & Limitations

    Summarization evaluation relies on LLM-generated scores, which may differ from human assessments and suffer from ordering effects.

Abstract

from arXiv · show

Code generation under long contexts is becoming increasingly critical as Large Language Models (LLMs) are required to reason over extensive information in the codebase. While recent advances enable code LLMs to process long inputs, high API costs and generation latency remain substantial bottlenecks. Existing context pruning techniques, such as LLMLingua, achieve promising results for general text but overlook code-specific structures and dependencies, leading to suboptimal performance in programming tasks. In this paper, we propose LongCodeZip, a novel plug-and-play code compression framework designed specifically for code LLMs. LongCodeZip employs a dual-stage strategy: (1) coarse-grained compression, which identifies and ranks function-level chunks using conditional perplexity with respect to the instruction, retaining only the most relevant functions; and (2) fine-grained compression, which segments retained functions into blocks based on perplexity and selects an optimal subset under an adaptive token budget to maximize relevance. Evaluations across multiple tasks, including code completion, summarization, and question answering, show that LongCodeZip consistently outperforms baseline methods, achieving up to a 5.6x compression ratio without degrading task performance. By effectively reducing context size while preserving essential information, LongCodeZip enables LLMs to better scale to real-world, large-scale code scenarios, advancing the efficiency and capability of code intelligence applications.

I. INTRODUCTION

Long-context code tasks are constrained by rising computational and API costs, relevance failures, and code dependencies that exceed context windows. LongCodeZip addresses these issues with code-aware hierarchical compression and reports strong multi-task results without sacrificing performance.

  • Motivation: Long-context code processing faces rising costs, relevance-selection failures, context truncation, and dependencies spanning functions, classes, and files.These constraints can produce uncompilable code, violated patterns, or ignored requirements when relevant context exceeds the model window.
  • Limitations of Existing Approaches: General compression and similarity-based retrieval methods overlook code structure or implicit dependencies, limiting their effectiveness for code tasks.RAG may retrieve lexically similar snippets while missing non-lexical dependencies such as configuration values used by another function.
  • LongCodeZip: LongCodeZip combines function-level relevance ranking with perplexity-based block pruning to preserve code semantics while reducing token consumption.The framework is training-free, model-agnostic, and plug-and-play, with adaptive token budgeting in the fine-grained stage.
  • Evaluation: Across code completion, summarization, and question answering, LongCodeZip achieves up to a 5.6× compression ratio without sacrificing performance.The evaluation also reports broad generalization across tasks and models, including use of a 0.5B compressor, with reduced generation time and token costs.

III. METHODOLOGY

The problem is to compress a long code context into a token budget while preserving information useful for the task instruction. LongCodeZip ranks candidate context by how much it improves instruction prediction rather than relying only on embedding similarity.

  • A. Problem Formulation: Context compression produces c′ ⊆ c under the token constraint |c′| ≤ B while aiming to maximize task performance.The context may include unfinished code and retrieved code snippets, with the instruction specifying the task such as code completion.
  • A. Problem Formulation: Approximated mutual information ranks context by the reduction in instruction perplexity when the context is provided.Lower conditional perplexity indicates a more likely instruction, so larger perplexity reduction signals greater relevance.
  • A. Problem Formulation: Retaining snippets with the highest mutual information captures both surface-level and dependency-based relevance for code generation.The method uses the model’s next-token prediction probabilities and treats higher AMI as evidence that context improves instruction prediction.

B. Overview

LongCodeZip uses a coarse-to-fine pipeline that first selects relevant functions and then prunes their internal semantic blocks. Adaptive budgets and knapsack selection preserve the most relevant content within the available token budget.

  • Coarse-Grained Compression: The pipeline ranks function-level chunks by instruction-conditioned perplexity and selects the top functions under a coarse budget.Function or class boundaries provide modular, syntactically valid chunks before fine-grained processing.
  • Fine-Grained Compression: Retained functions are segmented into semantic blocks, assigned adaptive retention ratios, and optimized with 0/1 knapsack selection.The selection maximizes block relevance while fitting each function’s allocated token budget.
  • Overview: Combining coarse filtering with fine-grained pruning balances aggressive compression, semantic preservation, efficiency, and task performance.The framework allocates more tokens to important functions while globally rescaling retention rates to match the target budget.
  • Budgeted Compression: Unselected chunks can be replaced with placeholders, preserving global structure while reducing context length.The coarse budget is determined from the final budget and configurable fine-grained compression ratio.

D. Fine-Grained Compression: Intra-Function Pruning

LongCodeZip compresses retained functions at block level while preserving semantic coherence and allocating more budget to important functions. It then selects blocks by relevance under per-function token budgets.

  • D. Fine-Grained Compression: Intra-Function Pruning: LongCodeZip segments each retained function into semantically coherent blocks before pruning to reduce context without breaking internal logic.Whitespace-only splitting is described as inadequate for preserving intra-function structure.
  • D. Fine-Grained Compression: Intra-Function Pruning: Higher-importance functions receive larger token budgets through adaptive allocation, while functions shorter than five lines are retained in full.Importance is estimated from AMI scores.
  • D. Fine-Grained Compression: Intra-Function Pruning: Retention ratios are normalized, importance-biased, clamped to [0, 1], and globally rescaled to match the large-function token budget.The parameter β controls how strongly allocation favors important functions.
  • D. Fine-Grained Compression: Intra-Function Pruning: For each function, 0/1 knapsack selection retains the block subset with maximum relevance within its allocated token budget.Each block is treated as an item with a relevance value and token cost.

IV. EXPERIMENTAL SETUP

The evaluation tests whether LongCodeZip preserves downstream performance, how its components contribute, whether it generalizes across models, and what efficiency benefits it provides. Experiments cover completion, summarization, and question answering on long-context code benchmarks.

  • IV. EXPERIMENTAL SETUP: The experiments ask whether LongCodeZip preserves downstream performance while compressing code context.This is the primary research question, alongside component, generalization, and efficiency questions.
  • IV. EXPERIMENTAL SETUP: Evaluation spans code completion, code summarization, and code question answering using long-context benchmarks.The tasks are intended to assess whether compressed code retains sufficient information for downstream performance.
  • IV. EXPERIMENTAL SETUP: The completion set contains 500 Python examples with contexts longer than 5,000 tokens, while the summarization set contains 139 examples exceeding 2,000 context tokens.RepoQA adds 600 multilingual tests across 60 repositories and 6 programming languages.
  • IV. EXPERIMENTAL SETUP: RepoQA evaluates whether models retrieve the correct function from long context using a natural-language instruction.The benchmark contains 600 tests across 60 repositories and 6 programming languages.

C. Baselines and Models

The experimental setup compares LongCodeZip with uncompressed, context-free, random, retrieval-based, and code-compression baselines across multiple open and closed-source code models. Evaluation measures compression and downstream task performance.

  • C. Baselines and Models: Baselines include full-context and instruction-only conditions, representing performance upper and lower bounds.The full-context condition uses no compression, whereas the instruction-only condition provides no code context.
  • C. Baselines and Models: Random baselines remove individual tokens or whole code lines, while retrieval baselines use sliding-window or function-level chunks with UniXCoderbase embeddings.These methods provide token-, line-, and retrieval-based comparisons.
  • C. Baselines and Models: The comparison includes compression components from DietCode and SlimCode alongside LongCodeZip.These are evaluated as code-compression baselines.
  • C. Baselines and Models: Models include Deepseek-Coder-6.7B, Qwen2.5-Coder-7B, SeedCoder-8B, GPT-4o, and Claude-3.7-Sonnet.The setup covers both open-source and closed-source models.
  • C. Baselines and Models: Compression efficiency is measured by the ratio of original-context tokens to compressed-context tokens.The evaluation also reports downstream generation performance with compressed context.
  • C. Baselines and Models: Completion uses Exact Match and Edit Similarity, summarization uses CompScore, and question answering uses retrieval accuracy above a BLEU threshold of 0.8.CompScore ranges from 0 to 100, with 50 indicating equal preference.

E. Implementation Details

Across code completion, summarization, and RepoQA, LongCodeZip consistently outperforms compression baselines, often at stricter compression ratios. Results also show strong performance on closed-source models and against advanced retrieval methods.

  • LongCodeZip consistently outperforms compression baselines across code completion, summarization, and RepoQA, including under comparable or stricter compression ratios.The reported margins are statistically significant across 10 repeated experiments.
  • 57.55 ES and 32.40 EM at a 4.3× ratio surpass RAG Function Chunking's 52.79 ES and 26.00 EM at 3.1× on Qwen2.5-Coder-7B.The compressed context is reported as 28% shorter than the RAG baseline.
  • 28.01 CompScore at a 2.5× compression ratio makes LongCodeZip the most competitive method on Long Module Summarization.RAG-based methods do not show clear advantages over other baselines on this task.
  • 16% higher overall score than LongLLMLingua while compressing context to half the length demonstrates LongCodeZip's RepoQA advantage on Deepseek-Coder-6.7B.The method is reported to achieve the best performance across all evaluated models.
  • On GPT-4o, LongCodeZip reaches 64.72 ES versus 65.13 without compression at 4.3× compression, while Claude-3.7-Sonnet reaches 66.27 versus 66.24.It also surpasses the no-compression baseline on RepoQA for both closed-source models.
  • LongCodeZip consistently outperforms advanced RAG methods and achieves higher information density under the same token budget.The comparison covers SeedCoder and Claude-3.7-Sonnet.

B. RQ2: Ablation Study

The ablation study finds conditional-perplexity function ranking to be the most influential component, while fine-grained allocation and selection mechanisms provide additional gains.

  • The ablations remove or replace coarse ranking, fine-grained compression, adaptive allocation, chunking, and selection while holding the token budget and other hyperparameters fixed.The study uses Qwen2.5-Coder-7B on Long Code Completion.
  • Conditional-perplexity ranking outperforms similarity-based ranking by 7.89% and random selection by 17.79% in ES, making it the most critical component.This supports semantic relevance over lexical similarity for coarse-grained selection.
  • Adaptive budget allocation improves ES by 2.34% by assigning more fine-grained tokens to higher-importance functions.Very small functions shorter than five lines are kept in full.
  • Perplexity-based chunking improves ES by 1.57% over simple line chunking and is more computationally efficient.Line-by-line compression ranking would incur higher overhead than block-based analysis.
  • Knapsack-based selection outperforms random line selection by 2.48% in ES, confirming the value of relevance-guided block selection.

C. RQ3: Transferability

LongCodeZip transfers across compression and generation models while retaining strong long-code-completion performance. Its efficiency gains come with modest compression overhead but substantially lower token use and generation latency.

  • LongCodeZip generalizes across model architectures and sizes in cross-model evaluation, including promising performance with a 0.5B compression model.
  • 77% lower input token costs and generation latency reduced from 15.70s to 6.59s demonstrate practical efficiency gains despite 2.58s compression overhead.The comparison is against no compression on Qwen2.5-Coder-7B.
  • Compression overhead can be mitigated with a lightweight 0.5B model, while quantization may further enhance efficiency.
  • 4.3× compression with 2.6s overhead reduces generation time from 15.7s to 6.6s while maintaining high downstream performance.

VI. DISCUSSION

LongCodeZip maintains strong performance across compression ratios by selecting relevant code early, while its fine-grained stage preserves semantic blocks under deployment-specific cost constraints. Its main limitations arise when instructions are ambiguous or irrelevant context provides little guidance, and evaluation validity is constrained by LLM-based summarization scores and possible dataset or model specificity.

  • Compression-performance trade-off: LongCodeZip achieves the highest ES scores across all tested compression ratios on Long Code Completion, outperforming representative methods.The comparison uses Qwen2.5-Code-7B results and shows stronger performance than RAG-based and other baseline methods.
  • Fine-grained compression: Perplexity-based boundaries separate independent functional modules while grouping smaller related segments during fine-grained compression.The case study illustrates semantic block detection based on sharp relative increases in line perplexity.
  • Failure modes: The method may struggle when context lacks instruction-relevant information or when ambiguous instructions cannot be aligned with context segments.These conditions make it difficult to identify and preserve useful blocks.
  • Deployment trade-offs: Fine-grained compression can be disabled for faster, cheaper compression, but becomes critical for reducing costs with powerful, expensive APIs.The coarse-grained stage provides the largest compression gains, whereas fine-grained pruning balances compression overhead against task-model cost.
  • Threats to validity: Summarization evaluation relies on LLM-generated scores, while findings may remain specific to the evaluated datasets, languages, or models.The study uses prompt-order averaging, an independent referee, and diverse evaluation settings to mitigate these threats.

VIII. RELATED WORK

Prior code LLMs perform strongly on downstream tasks, but long-context use remains difficult when relevant information is buried in prompts or spans structural dependencies. Existing compression and retrieval methods often require fine-tuning, rely on text similarity or model-specific heuristics, or target narrower settings, motivating LongCodeZip as a training-free, model-agnostic solution for long-context code compression.

  • Long-context code modeling: Long-context code tasks remain challenging because relevant information may occur mid-prompt or depend on cross-function and structural relationships.These limitations persist despite increasingly large context windows.
  • General compression methods: Soft prompt compression is memory-efficient but requires target-model fine-tuning, limiting practicality in closed-source settings.Hard prompt methods instead manipulate the input by removing or rephrasing less informative content.
  • Code compression methods: Natural-language compression methods often miss source-code structure and semantics, while code-specific methods such as DietCode and SlimCode have adaptability or generalization constraints.DietCode relies on model-specific attention, whereas SlimCode uses rule-based pruning tied to token types and dependency graphs.
  • Code-specific compression: LongCodeZip is presented as the first training-free, model-agnostic framework explicitly targeting long-context compression for code LLMs.It is designed to preserve task-relevant content under tight token budgets.
  • LongCodeZip: LongCodeZip combines function-level selection with block-level pruning and achieves up to a 5.6x compression ratio without sacrificing task performance.The framework is evaluated across completion, summarization, and question answering, with cross-model generalization and a lightweight 0.5B compressor.
Loading 2510.00446v1…