PaperlayerView exact revision on arXiv

Source-linked AI summary

Text summarization via global structure awareness

Jiaquan Zhang, Chaoning Zhang, Shuxu Chen, Yibei Liu, Chenghao Li, Qigan Sun, Shuai Yuan, Fachrina Dewi Puspitasari, Dongshen Han, Guoqing Wang, Sung-Ho Bae, Yang Yang

arXiv:2602.09821v2cs.CLcs.AI

TL;DR

Long-document summarization must reduce redundant context without losing semantic cores and logical dependencies, while existing approaches can overlook global structure or incur high resource costs. GloSA-sum uses TDA, a Protected Pool, topology-guided iteration, and hierarchical processing to preserve structure efficiently; experiments report consistent gains over strong baselines and shorter LLM contexts retaining essential reasoning chains.

  • Problem

    Long-document summarization needs to remove redundant information without losing key content and logical chains, but existing methods often overlook global structure or incur substantial resource and time costs.

  • Method

    GloSA-sum constructs a semantic-weighted graph, uses persistent homology and a Protected Pool to preserve structural backbones, and combines proxy-guided iteration with hierarchical summarization.

  • Results

    GloSA-sum consistently improves ROUGE and human evaluation over strong baselines while reducing context length and retaining essential reasoning chains for downstream LLM tasks.

  • Takeaways & Limitations

    The framework balances summarization accuracy, structural fidelity, scalability, and efficiency for long-text processing while supporting downstream LLM tasks.

Abstract

from arXiv · show

Text summarization is a fundamental task in natural language processing (NLP), and the information explosion has made long-document processing increasingly demanding, making summarization essential. Existing research mainly focuses on model improvements and sentence-level pruning, but often overlooks global structure, leading to disrupted coherence and weakened downstream performance. Some studies employ large language models (LLMs), which achieve higher accuracy but incur substantial resource and time costs. To address these issues, we introduce GloSA-sum, the first summarization approach that achieves global structure awareness via topological data analysis (TDA). GloSA-sum summarizes text efficiently while preserving semantic cores and logical dependencies. Specifically, we construct a semantic-weighted graph from sentence embeddings, where persistent homology identifies core semantics and logical structures, preserved in a ``protection pool'' as the backbone for summarization. We design a topology-guided iterative strategy, where lightweight proxy metrics approximate sentence importance to avoid repeated high-cost computations, thus preserving structural integrity while improving efficiency. To further enhance long-text processing, we propose a hierarchical strategy that integrates segment-level and global summarization. Experiments on multiple datasets demonstrate that GloSA-sum reduces redundancy while preserving semantic and logical integrity, striking a balance between accuracy and efficiency, and further benefits LLM downstream tasks by shortening contexts while retaining essential reasoning chains.

1 INTRODUCTION

The introduction identifies a gap in sentence-level and model-focused summarization: these approaches can miss global semantic structure and logical dependencies in long documents. GloSA-sum addresses this gap with TDA-based structure preservation, efficient topology-guided compression, and hierarchical processing.

  • Motivation: Long documents create efficiency and accuracy bottlenecks because redundant information exhausts LLM context windows and distracts models from core content.The resulting challenge is to shorten texts without losing key information or logical chains.
  • Limitations of Existing Methods: Sentence-level pruning methods are efficient but often rely on local similarity or shallow features that miss global semantics and long-range logical dependencies.Prior work includes graph ranking and optimization approaches for balancing coverage and redundancy.
  • Proposed Framework: GloSA-sum introduces TDA into summarization to model and preserve semantic clusters and cross-paragraph logical dependencies.The framework encodes sentences as semantic embeddings and uses global topological structure to preserve key reasoning chains.
  • Proposed Framework: A one-time topological analysis and proxy-based iterative strategy avoids repeated persistent homology computations while balancing efficiency and semantic integrity.The Protected Pool retains the identified structural backbone during summarization.
  • Proposed Framework: A hierarchical framework coordinates local summarization with global integration to improve scalability and robustness for long texts.The approach is designed for coordinated segment-level and document-level processing.
  • Reported Outcomes: Experiments report stronger summarization than strong baselines and improved LLM downstream performance through shorter contexts that retain essential reasoning chains.The claimed benefits span summarization quality, context reduction, and preservation of reasoning structure.

2 RELATED WORK

Related work follows two main directions: model-level improvements and sentence-level pruning. TDA research shows that topological features can capture structure in high-dimensional text representations and support discourse-level analysis.

  • Summarization Methods: Text summarization methods broadly divide into model-level improvements and sentence-level pruning based on estimated sentence importance.Early compression methods treated sentences as atomic units selected or discarded according to importance.
  • Sentence-Level Pruning: Graph-based methods such as TextRank and LexRank construct sentence-similarity graphs and use centrality ranking to identify core sentences.These methods demonstrate the usefulness of graph structure for sentence selection.
  • Topological Data Analysis: TDA provides a systematic framework for capturing global structure in high-dimensional and complex data.Prior applications integrate persistent-homology features with embeddings for textual analysis.
  • Topological Data Analysis: Existing TDA applications report benefits for contradiction detection, linguistic acceptability judgments, interpretability, and discourse-level analysis.These studies extend topological features beyond conventional summarization settings.

3 METHODOLOGY

GloSA-sum applies topological data analysis to sentence-embedding graphs, preserving semantic themes and logical dependencies in a Protected Pool before efficient iterative compression. Its proxy-based scoring removes redundancy while retaining structurally important and query-relevant sentences.

  • 3 METHODOLOGY: TDA identifies persistent semantic clusters and logical cycles, which form the document backbone preserved in the Protected Pool.The pool combines core H0 themes with critical H1 discourse cycles.
  • 3 METHODOLOGY: GloSA-sum constructs a weighted sentence graph from normalized embeddings, combining semantic distance with sequential proximity.Mutual k-nearest-neighbor connectivity preserves global relations while maintaining graph sparsity.
  • 3 METHODOLOGY: Persistent homology is computed once, after which lightweight proxy metrics guide compression without repeated high-cost TDA computations.The one-time analysis fixes the global structural backbone for subsequent sentence removal.
  • 3 METHODOLOGY: The Protected Pool selects the top-K longest-lived H0 features and top-M persistent H1 cycles to retain semantic clusters and logical dependencies.These complementary components keep the protected set controllable while preserving essential discourse structure.
  • 3 METHODOLOGY: Topology-guided deletion combines structural connectivity and task relevance, deleting lower-scoring sentences earlier while retaining important content.TopoScore uses shortest paths to protected nodes, whereas TaskScore combines embedding similarity and BM25 when a query is available.

4 EXPERIMENT

GloSA-sum is evaluated across long-text datasets for quality, efficiency, coherence, downstream utility, and component effectiveness. Results report stronger summarization performance, favorable efficiency, best human-evaluation results, statistically significant gains, and substantial contributions from its topological components.

  • Performance Evaluation: GloSA-sum improves ROUGE scores across long-text datasets, including +2.14 ROUGE-L over BART on ArXiv and a 44.5 ROUGE-L score on PubMed.It also reports +1.19 ROUGE-2 over BigBird on GovReport and +2.17 ROUGE-L over BigBird on PubMed.
  • Efficiency: GloSA-sum balances summarization accuracy and efficiency through one-time topological analysis and protected-pool construction instead of repeated high-cost computations.The comparison covers time, memory complexity, and parallelizability across extractive, abstractive, long-document, and candidate-selection methods.
  • Human Evaluation: GloSA-sum achieves the best overall human-evaluation performance, with gains in coherence and informativeness while maintaining strong conciseness.The evaluation measures coherence, informativeness, and conciseness.
  • Statistical Significance: ROUGE-L gains over DANCER are statistically significant across CNN/DM, GovReport, ArXiv, and PubMed, with all paired-bootstrap p-values below 0.01.The test uses 1,000 bootstrap resamples per dataset.
  • Ablation Study: Removing the Protected Pool drops ROUGE by more than 5 points, while random replacement of TopoScore causes a decline of around 3 points.The ablation results identify the protected backbone and topology-guided selection as important components.
  • Robustness: The sentence encoder choice is intended to isolate gains from TDA rather than from global reasoning embedded in larger contextual encoders.The selected all-mpnet-base-v2 encoder provides lightweight, context-local representations.

5 CONCLUSION

GloSA-sum combines TDA, protected semantic backbones, topology-guided iteration, and hierarchical compression for long-text summarization. Across multiple datasets, it improves quality and efficiency while shortening downstream LLM contexts and retaining essential reasoning chains.

  • Conclusion: GloSA-sum integrates TDA to preserve semantic clusters and logical dependencies in long texts.A one-time persistent-homology analysis and Protected Pool preserve the semantic backbone while avoiding repeated high-cost computations.
  • Conclusion: Experiments show consistent improvements over strong baselines in ROUGE and human evaluation, with notable efficiency advantages.The framework is designed to combine scalability with structural fidelity.
  • Conclusion: Downstream LLM evaluations show that GloSA-sum reduces context length while retaining essential reasoning chains.The reported benefit extends beyond summarization.

6 REPRODUCIBILITY STATEMENT

The paper documents its evaluation metrics, experimental environment, model settings, baselines, and datasets to support reproducibility. These details cover fixed hardware and seeds, graph and topology configurations, comparison systems, and long-text evaluation corpora.

  • Evaluation Metrics: Evaluation combines ROUGE metrics with human ratings of coherence, informativeness, and conciseness.ROUGE-1 measures unigram overlap, ROUGE-2 bigram overlap, and ROUGE-L longest-common-subsequence similarity.
  • Human Evaluation: Human evaluation uses a 5-point Likert scale, three NLP-trained graduate annotators, and 50 randomly sampled documents per dataset.The evaluated dimensions are coherence, informativeness, and conciseness.
  • Implementation Environment: Experiments run on one NVIDIA RTX 4090 GPU, a 16-core Intel Xeon Gold 6330 CPU, and 256GB RAM with batch size 1 and an 8,192-token maximum.The setup uses single-GPU document-level processing.
  • Model and Method Details: Reproducibility settings include all-mpnet-base-v2 embeddings, mutual adaptive k-nearest-neighbor graphs, Lazy Witness Complexes, H0/H1 homology, TopoScore, and fixed random seed 42.The graph neighborhood grows logarithmically and is bounded between 5 and 20.
  • Baseline Models: The baseline suite includes graph-based, extractive, abstractive, long-document, and reinforcement-learning summarization methods.Examples include TextRank, LexRank, BERTSum, MatchSum, MemSum, BART, PEGASUS, BIGBIRD, and DANCER.
  • Datasets: The evaluation uses GovReport, ArXiv, PubMed, CNN/DailyMail, and additional listed long-text or argumentative datasets with distinct structural challenges.GovReport contains nearly 9,500 long documents, while ArXiv contains about 5,000 scientific papers.

A.5 PERFORMANCE EVALUATION

GloSA-sum is evaluated with semantic and factuality metrics, showing stronger preservation of semantic equivalence and factual accuracy than the reported baselines across domains and document lengths.

  • Evaluation Metrics: BERTScore and QAFactEval evaluate semantic similarity and factual consistency, respectively, providing complementary measures of summary quality.BERTScore uses contextual embeddings, while QAFactEval tests whether summaries preserve essential facts.
  • Results: GloSA-sum consistently preserves semantic equivalence and factual accuracy across domains and document lengths.The evaluation reports this pattern across BERTScore and QAFactEval.
  • Results: 0.88/0.78 are GloSA-sum’s BERTScore results on CNN/DM, compared with TextRank’s 0.73/0.58.These results indicate stronger semantic matching than the cited extractive baseline.
  • Results: 0.83/0.75 on ArXiv and 0.86/0.76 on PubMed are GloSA-sum’s QAFactEval results against abstractive baselines scoring around 0.79–0.83.The reported PubMed results surpass all cited baselines in that biomedical domain.

A.6 HUMAN EVALUATION COMPARED TO LLMS-BASED BASELINE MODELS

Human and benchmark evaluations compare GloSA-sum with LLM-based summarizers and assess its downstream utility. The reported results show competitive human scores, stronger metric performance, and stable behavior on long documents.

  • Human Evaluation: 4.30 is GloSA-sum’s highest average human-evaluation score, matching GPT-4.GloSA-sum scores 4.4 versus GPT-4’s 4.3 on coherence.
  • Human Evaluation: 4.4 versus 4.3 is GloSA-sum’s coherence score compared with GPT-4 in human evaluation.The paper attributes this advantage to explicit modeling of global semantic and logical structures.
  • Benchmark Comparison: GloSA-sum consistently outperforms strong LLM summarization baselines across all four datasets on the reported metrics.The comparison includes ROUGE-1/2/L, BERTScore, and QAFactEval.
  • Benchmark Comparison: GloSA-sum achieves the highest QAFactEval scores, indicating stronger factual consistency than the compared modern LLMs.The paper notes that LLM performance degrades significantly on ultra-long GovReport documents.
  • Downstream Evaluation: The downstream evaluation uses SQuAD 2.0 to test whether summaries remain effective for machine reading comprehension.The task requires extracting answer spans or predicting that no answer is available.

A.8 HYPERPARAMETER EXPERIMENT

The hyperparameter experiment studies the fusion coefficient α on GovReport using ROUGE scores. Performance is best at α=0.5, where semantic and temporal signals are balanced.

  • Hyperparameter Results: 57.3/26.8/52.4 are the best ROUGE-1/2/L scores at α=0.5 on GovReport.The fusion coefficient balances semantic and temporal signals.
  • Hyperparameter Results: Increasing α from 0.0 to 0.5 steadily improves performance, with α=0.0 representing temporal-only information.The reported trend supports combining semantic and temporal signals rather than relying solely on one component.
  • Experiment Setup: Figure 2 presents the hyperparameter experiment.The supplied figure label identifies the experiment but does not provide additional plotted values.

A.9 ENCODER ROBUSTNESS ANALYSIS

The encoder robustness analysis tests GloSA-sum with sentence encoders of different capacity. Results improve monotonically with encoder capacity while the topological backbone remains consistent across encoders.

  • Experimental Design: A compact all-mpnet-base-v2 encoder avoids leaking global discourse information into representations, isolating TDA’s contribution.This design is intended to test whether TDA itself constructs long-range semantic and logical structures.
  • Robustness Results: Encoder capacity produces clear and monotonic improvements across all four datasets.The tested encoders include all-roberta-large-v1 and text-embedding-3-small/large.
  • Robustness Results: The topological backbone operates consistently across the underlying embedding models, with smooth rather than volatile gains.The results are summarized in Table 10.
  • Practical Implications: GloSA-sum is described as encoder-agnostic and plug-and-play for different computational budgets, accuracy requirements, and latency constraints.Practitioners can select encoders according to deployment needs.

A.10 POSITION DISTRIBUTION ANALYSIS OF THE PROTECTED POOL

The Protected Pool is selected from semantic and topological importance rather than sentence position. On GovReport, it therefore captures substantial content from the middle and end of documents while preserving core semantic and logical structures.

  • Position distribution: GloSA-sum selects Protected Pool sentences using high-dimensional semantic structure rather than positional heuristics.The analysis contrasts TDA-based selection with Lead-3’s fixed preference for the first three sentences.
  • Position distribution: Table 11 compares the relative sentence-position distribution of the Protected Pool with the Lead-3 baseline.
  • Position distribution: More than 70% of Protected Pool sentences come from the middle and end sections of GovReport documents.This distribution sharply contrasts with the front-loaded Lead-3 baseline.
  • Qualitative evidence: Qualitative cases show one-time TDA analysis and the Protected Pool preserving core semantic clusters and logical loops while compressing redundant content.

B.1 CASE 1: GOVREPORT DATASET (POLICY OVERSIGHT AND HEALTH PROTECTION)

The GovReport case shows GloSA-sum condensing policy and oversight reports while retaining their main themes, logical dependencies, and accountability chains. Its Protected Pool preserves structural integrity while removing redundant background and detail.

  • Report content: The report covers DOD deployment health protection, monitoring policies, and implementation problems involving data tracking and health monitoring.
  • Semantic themes: The protected themes include the Emergency-Essential Program, deployment health policies, and gaps in centralized health monitoring data.
  • Logical structure: Missing centralized data, incomplete assessments, and ineffective monitoring form a recurring health-protection challenge cycle.The report links data gaps to policy gaps that weaken effective enforcement.
  • Summarization effect: GloSA-sum retains essential themes and reasoning chains while pruning redundant background information and detailed statistics.The resulting summary is described as concise yet structurally faithful.
  • Structural preservation: The method preserves global structure by fixing semantic clusters and logical relations in the Protected Pool before summarization.This supports concise summaries with maintained logical integrity and consistency.
Loading 2602.09821v2…