Source-linked AI summary
TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories
Daniel Agyei Asante, Yang Li
TL;DR
Long-context compression must reduce inference cost while preserving evidence that remains coherent and connected across long inputs. TopoCompress selects semantic spans using query relevance and semantic acceleration, then propagates relevance over a hybrid graph. Across five tasks and multiple target models, it consistently outperforms evaluated baselines, matching the strongest baseline with a compression budget up to 4× smaller and reducing compression time relative to the fastest baseline.
Problem
Existing compression methods can fragment evidence, rely on target-model-specific alignment, and score tokens without modeling relationships among supporting evidence.
Method
TopoCompress is a training-free, model-agnostic framework that scores coherent spans with query relevance and semantic acceleration, then propagates scores over semantic and sequential graph connections.
Results
Across five long-context tasks and multiple target models, TopoCompress consistently outperforms evaluated baselines; at K = 500, it achieves performance comparable to LongLLMLingua at K = 2000.
Takeaways & Limitations
Graph-based evidence selection provides a practical long-context compression paradigm that preserves connected evidence while remaining fast for long-context pipelines.
Takeaways & Limitations
Perplexity-based approaches may discard intermediate information needed to preserve evidence chains in multi-document and multi-hop tasks.
Abstract
from arXiv · showhide
Long-context compression is essential for reducing the cost and latency of large language model inference. However, existing methods can fragment important evidence, require additional training or alignment, and often depend on the target model for effective compression. We introduce TopoCompress, a training-free and model-agnostic framework that compresses long contexts by selecting coherent semantic spans. TopoCompress first scores each span using dense and lexical query relevance together with semantic acceleration. It then constructs a hybrid graph that connects spans based on semantic similarity and sequential adjacency, and propagates the query-guided relevance scores over the graph. Across five long-context tasks-HotpotQA, 2WikiMQA, MuSiQue, Qasper, and MultiFieldQA-en-TopoCompress consistently outperforms strong compression baselines. Notably, TopoCompress achieves performance comparable to the strongest baseline while using a 4x smaller compression budget, and provides a 1.41x smaller compression time over the fastest baseline.
1 Introduction
Long-context compression must reduce inference costs while preserving coherent, connected evidence that models can use across long inputs. TopoCompress addresses these challenges with training-free graph-based span selection and reports strong quality and efficiency across tasks and models.
- Motivation: Finite context windows make long inputs costly and slow to process.Longer contexts also do not necessarily improve performance when relevant evidence is difficult to use, especially in the middle of the context.
- Limitations of Existing Methods: Token-level pruning can fragment word boundaries, phrases, named entities, and syntactic units.The resulting distortion may require post-hoc recovery, adding computation.
- Limitations of Existing Methods: Compressor alignment ties existing methods to specific target models and introduces additional computation.The compressor must learn to imitate what the target model considers informative.
- Limitations of Existing Methods: Perplexity-based scores treat tokens independently and may discard intermediate information needed to connect evidence across documents.This makes perplexity insufficient for determining downstream task importance in multi-document and multi-hop settings.
- TopoCompress: TopoCompress formulates compression as query-guided relevance propagation over connected semantic spans in a hybrid graph.It combines semantic span scoring, semantic acceleration, semantic similarity, and sequential adjacency.
- Results: TopoCompress consistently outperforms evaluated compression baselines across compression budgets and target models while keeping compression lightweight and fast.The framework is evaluated on five long-context question-answering tasks.
- Results: TopoCompress preserves complete evidence where token-level methods can remove, corrupt, or fragment answer-bearing phrases.Figure 1 contrasts LLMLingua, LongLLMLingua, and TopoCompress.
2 Related Work
Prior work uses graph representations to capture relationships among textual units for ranking and summarization. TopoCompress extends this perspective to long-context compression by connecting semantic spans and preserving supporting information without target-model-specific distillation.
- Graph-Based Modeling of Text: TextRank and LexRank apply graph-based ranking to words, sentences, or sentence-similarity structures for extraction and summarization.These methods identify keywords or salient sentences from textual relationships.
- Graph-Based Modeling of Text: GraphLSS combines structural and semantic relations in a heterogeneous graph to identify salient sentences for long-document summarization.
- TopoCompress: TopoCompress brings graph-based modeling to long-context compression by representing contexts as connected semantic spans.This preserves intermediate supporting information and avoids reliance on a separate compressor model or target-model-specific distillation.
3 TopoCompress
TopoCompress compresses contexts by scoring coherent semantic spans for query relevance and semantic transitions, propagating scores through a hybrid topology, and selecting diverse evidence within a global budget.
- Semantic Span Construction: TopoCompress represents each document as contiguous semantic spans and seeks a compact evidence set sufficient to answer the query under token budget K.This avoids treating individual tokens as independent compression units.
- Dense and Lexical Query Relevance: Each span receives dense and lexical query-relevance scores, which are normalized and combined through a balance parameter µ.Dense relevance uses cosine similarity, while lexical relevance captures exact signals such as entities, dates, and relation terms.
- Semantic Acceleration: Semantic acceleration boosts query-aligned spans near strong semantic transitions while preventing unrelated transitions from dominating selection.Acceleration is computed from neighboring span embeddings along the original sequence and incorporated through λ.
- Span Topology and Relevance Propagation: Personalized relevance propagation allows intermediate spans that do not directly match the query to gain importance through connections to relevant evidence.The update repeats until convergence, producing propagated scores for subsequent selection.
- Span Topology and Relevance Propagation: The hybrid span graph combines semantic-neighbor edges with sequential adjacency, then propagates query-gated relevance so connected supporting spans can gain importance.The graph uses W = αWsem + βWseq, and propagation restarts toward the original query-gated scores.
- Redundancy-Aware Evidence Selection: TopoCompress greedily selects high-propagated-relevance spans while penalizing semantic redundancy, stops at budget K, and restores original document order.The resulting evidence set is designed to remain diverse, readable, and discourse-coherent for the target model.
4 Experiments
TopoCompress is evaluated across five LongBench QA tasks, three target models, multiple compression baselines, and budgets down to K = 500. It consistently delivers stronger performance and faster compression, while controller and ablation analyses examine efficiency and component contributions.
- Setup: The evaluation covers HotpotQA, 2WikiMQA, MuSiQue, Qasper, and MultiFieldQA-en across GPT-5-mini, Llama-3.1-8B, and Qwen3-8B.The tasks span multi-document, scientific-document, and long-article question answering, with distributed evidence requirements.
- Performance: TopoCompress consistently achieves the best performance across compression budgets and target models.At K = 500, it outperforms LongLLMLingua by 8.15 points on GPT-5-mini, 8.44 points on Llama-3.1-8B, and 7.21 points on Qwen3-8B.
- Performance: At K = 500, TopoCompress reaches performance comparable to LongLLMLingua at K = 2000, using a compression budget up to 4× smaller.On GPT-5-mini, the reported values are 51.37 versus 51.44.
- Efficiency: TopoCompress completes compression across all five tasks in less than 7 minutes and achieves a 1.41× speedup over LLMLingua-2.LLMLingua and LongLLMLingua require approximately 44 and 52 minutes, respectively.
- Controller Analysis: The controller reduces average compressed context length from 1000 tokens to roughly 681–857 tokens, while providing only modest performance gains.The controller checks answerability after accumulating half the evidence budget and can generate a focused follow-up question.
- Ablation Study: Removing graph propagation reduces performance by up to 14.1% on MuSiQue, 7.4% on HotpotQA, and 5.9% on 2WikiMQA.Removing query relevance causes the largest performance drop, whereas removing semantic acceleration changes average F1 more modestly.
5 Conclusion
The paper concludes that TopoCompress performs long-context compression as graph-structured selection over coherent semantic spans. It reports consistent gains and faster compression across target models, positioning graph-based evidence selection as a practical alternative to token-level, target-model-dependent designs.
- 5 Conclusion: TopoCompress combines dense–lexical query relevance, semantic acceleration, and graph propagation over semantic and sequential span connections.The framework selects compact, coherent, and non-redundant evidence for the target model.
- 5 Conclusion: TopoCompress consistently outperforms strong context compression baselines across closed-source and open-weight target models.The conclusion also reports that it is faster than existing baselines for long-context pipelines.
A Graph-Based Evidence Recovery
Multi-hop questions may require evidence chains whose answer-bearing spans are only indirectly related to the query. The graph propagation analysis shows how topology can promote such supporting spans into the compressed context.
- A Graph-Based Evidence Recovery: Multi-hop questions often require connecting evidence across several documents through intermediate evidence before identifying the answer-bearing document.Direct query relevance alone may therefore miss the evidence needed for the answer.
- A Graph-Based Evidence Recovery: In the MuSiQue example, graph propagation promotes the answer-bearing span p_19:span_3 from rank 14 to rank 7.The analysis uses gold supporting-paragraph annotations to inspect whether ranked spans follow the true evidence chain.
- A Graph-Based Evidence Recovery: Graph propagation lowers the rank of a non-supporting span from third, helping prioritize connected supporting evidence that is weakly matched to the question.This illustrates its role beyond direct query matching.
B Additional Results for TopoCompress
Additional results are reported in task- and target-model-specific tables for GPT-5-mini, Llama-3.1-8B, and Qwen3-8B.
- B Additional Results for TopoCompress: Tables 4, 5, and 6 provide additional TopoCompress and baseline results for GPT-5-mini, Llama-3.1-8B, and Qwen3-8B, respectively.
C Controller-Guided TopoCompress
The controller-guided variant generates intermediate reasoning queries during compression to select additional evidence when the current compressed context appears insufficient.
- The controller generates intermediate reasoning queries to guide additional evidence selection when the current compressed context appears insufficient.
D Ablation Study Results
The ablation study reports results for graph propagation and other TopoCompress components across long-context tasks and target models, with supporting comparisons in Tables 3–9.
- The reported ablation results are presented in Tables 8 and 9.
- Table 3 qualitatively analyzes graph-based multi-hop evidence recovery on MuSiQue using gold supporting paragraphs.
- The study includes comparisons across GPT-5-mini, Llama 3.1-8B, and Qwen3-8B target models.
- The controller comparison evaluates TopoCompress with and without the controller across several tasks, budgets, and target models.
- Graph propagation ablation results are reported on multi-hop long-context tasks, with parenthetical values indicating the relative F1 drop after removal.