Source-linked AI summary
Automatic summarising: factors and directions
Karen Sparck Jones
TL;DR
Automatic summarising needs a better research methodology and a focused strategy because effective summaries depend on context factors and current systems face intrinsic and technological limits. The paper analyzes those factors and argues for shallow, indicative summarising as a nearer-term direction, while recognizing that evaluation and discourse-level understanding remain constrained.
Problem
Method
Results
The paper concludes that shallow summarising is a feasible nearer-term research programme with potential practical value and a route toward scalable summarising.
Source: §11 · pp. 18–19Takeaways & Limitations
Context-sensitive shallow summarising can provide a practical starting point while offering insight into discourse structure and the use of limited domain information.
Source: §11 · pp. 18–19Large-scale discourse understanding remains a long-term research topic, and evaluation without a task context is limited to reader acceptability and extraction comparisons.
Source: §3 · p. 5, §10 · p. 18
Abstract
from arXiv · showhide
This position paper suggests that progress with automatic summarising demands a better research methodology and a carefully focussed research strategy. In order to develop effective procedures it is necessary to identify and respond to the context factors, i.e. input, purpose, and output factors, that bear on summarising and its evaluation. The paper analyses and illustrates these factors and their implications for evaluation. It then argues that this analysis, together with the state of the art and the intrinsic difficulty of summarising, imply a nearer-term strategy concentrating on shallow, but not surface, text analysis and on indicative summarising. This is illustrated with current work, from which a potentially productive research programme can be developed.
1 Introduction
The paper argues that progress in automatic summarising requires better methodology focused on context factors and a research strategy centered on shallow techniques. It presents these directions as responses to current technology limits and the difficulty of emulating human summarising.
- The paper asks which methodological and strategic directions can advance automatic summarising given the current state of the art.Source: §1 · p. 1
- It argues that useful systems require attention to input, purpose, and output context factors, especially purpose factors shaping summaries.Source: §1 · pp. 1–2
- Because human-level summarising is not expected in the medium to long term, generally applicable procedures should use relatively shallow techniques within current NLP capabilities.Source: §1 · pp. 1–2
- Technology limitations make it necessary to identify summary tasks and application conditions where limited technology could add value.Source: §1 · p. 2
- The paper is programmatic: its state-of-the-art review is selective, and its proposed direction is broadly assertive with indicative illustrations.Source: §1 · p. 2
2 Background
The paper frames summarising as a reductive transformation through interpretation, transformation, and generation, while emphasizing that capturing important content makes the task intrinsically hard.
- A summary reduces source content through selection and/or generalisation of what is important in the source.Source: §2 · p. 2
- The basic process model separates source interpretation, representation transformation, and summary generation into three stages.Source: §2 · p. 2
- Distinct processing phases and data types clarify system assumptions, enable comparisons, and expose cases where source and summary representations are conflated.Source: §2 · p. 2
- Summarising is hard because it requires characterising a source as a whole, capturing important information and expression, and distinguishing essential from salient content.Source: §2 · p. 2
3 Current state
Current summarising research is dominated by text extraction and fact extraction, which offer complementary strengths but leave important gaps in generality, quality, and source interpretation. More serious summarising requires discourse structure, although that remains a long-term research topic and nearer-term progress is needed.
3 Current state
- Generating summaries from non-linguistic source data is excluded because the paper treats interpretation of long source text as the critical general summarising issue.Source: §3 · p. 3
- Text extraction transfers source sentences using statistical, locational, and cue-word criteria, while fact extraction locates expressions matching predefined generic concepts or relations.Source: §3 · pp. 3–4
- Short sources can make single-sentence extraction or near-complete factual processing reasonable, while multi-source fact extraction may perform more transformation of combined representations.Source: §3 · p. 4
- Text extraction offers generality but usually produces lower-quality, weakly coherent summaries because importance identification and organisation are indirect.Source: §3 · pp. 3–4
- Fact extraction can improve substance and presentation for selected material, but requires explicit specification of information types that may not matter for the source.Source: §3 · p. 4
- Automatic summarising therefore needs more power than text extraction and more flexibility than fact extraction offers.Source: §3 · p. 4
- Large-scale discourse structure is necessary for human-quality summarising but requires unresolved decisions about its information types, forms, indicators, roles, and uses.Source: §3 · pp. 4–5
4 Methodology
The paper proposes analyzing context factors systematically rather than choosing a summarising strategy by trial and error. This analysis should support more rational strategy selection and better-understood development for particular applications.
4 Methodology
- Effective general summarising strategies require identifying the operational context factors of each application.Source: §4 · p. 5
- Context analysis replaces crude try-and-see system development with a more rational choice of strategy and better-understood strategy development.Source: §4 · p. 5
- A general-purpose summary is unrealistic because different input sources and output requirements impose different constraints.Source: §4 · p. 5
- What a summary should be like depends on its intended purpose as well as the properties of its source, including assumptions about readers’ subject knowledge.Source: §4 · p. 5
5 Context factors
The paper frames summarising as a context-dependent function of input, purpose, and output factors, arguing that these factors must guide strategy selection and evaluation. It distinguishes source characteristics, intended use and audience, and desired content, format, and style.
5 Context factors
- Purpose factors are especially important because they guide strategy selection and provide the basis for evaluation.The paper argues that their implications are often overlooked or treated as obvious, despite their ramifying effects.Source: §5 · p. 6, §5.2 · p. 8
5.1 Input factors
- Input factors include source form, subject type, and unit, covering structure, scale, medium, genre, domain knowledge, and single versus multiple sources.These properties may be preserved or transformed depending on purpose, but they must be identified so summarising transformations respond to the source.Source: §5.1 · pp. 6–7
- Source properties need not be replicated in the summary, but they must be recognised because purpose-driven output requirements determine how source content is transformed.Even passive strategies carry assumptions about the context of use.Source: §5.1 · pp. 6–7
5.2 Purpose factors
- Purpose factors comprise situation, audience, and use, which determine the context, readers, and intended function of a summary.Situations may be tied to a known application or floating; audiences range from untargeted to targeted; uses include retrieval, preview, substitution, memory refresh, and action prompting.Source: §5.2 · p. 8
- Purpose analysis must be developed in detail because a specific audience, use, and situation can determine which claims or background information a summary should include.The summary may import presumed domain or situational knowledge when the application requires it.Source: §5.2 · pp. 8–9
5.3 Output factors
- Output factors are material, format, and style: summaries may be covering or partial, running text or headed, and informative, indicative, critical, or aggregative.These are not independent properties; they follow from the input source and the purpose the summary must satisfy.Source: §5.3 · p. 9
- The paper expresses summarising as satisfying purpose requirements through output devices given input data, while leaving final output choices partly to judgement.Future or indeterminate use prevents input and purpose analysis from mandating one output design, although good analysis narrows the choices.Source: §5.3 · pp. 9–10
6 An illustration
The paper illustrates context analysis with book-review summaries for public-library purchasers. The application yields specific input, purpose, and output requirements before production details are addressed.
6 An illustration
- For library purchasing, review sources are variable-length literary prose in single running-text units, with floating circumstances and an untargeted librarian audience.The summaries substitute for reviews that librarians may not readily access.Source: §6 · p. 10
- The resulting summaries should cover the reviews, use running text with bibliographic and source details, and indicate both the book aspects discussed and the reviewer’s attitude.The paper treats these as rational output choices derived from the input and purpose analysis.Source: §6 · p. 10
- Only after this context-based starting position is established can the detailed production of the required summaries be addressed.Source: §6 · p. 10
7 Evaluation
The paper argues that summary evaluation must be grounded in the context and purpose for which summaries are intended. Source comparison and human-reference methods can help development, but neither provides a sufficient basis for rigorous evaluation.
7 Evaluation
- Proper evaluation requires knowing the summary’s purpose and accounting for the relevant context conditions.Context analysis is therefore needed not only for strategy selection but also for evaluation design.Source: §7 · pp. 10, 12
- Comparing a summary with its source can help development or rapid vetting, but it cannot by itself provide rigorous evaluation.Identifying important concepts and relations is difficult, and summarising may add information or introduce new perspectives rather than merely replicate source content.Source: §7 · p. 11
- Human-reference comparison is also only a rough preliminary tool because shared key ideas are difficult to determine and a human summary is not necessarily the best standard.Different indexing descriptions can be equally effective for retrieval despite differing from manual descriptions [Salton 1972, 1986; Cleverdon 1977].Source: §7 · p. 11
- Even source-summary and human-summary comparisons must be interpreted modulo the pertinent context factors.The paper recommends direct tests of purpose because there is enough latitude in summarising that similarity between summaries is not decisive.Source: §7 · pp. 11–12
- Purpose-based evaluation still requires explicit goals, measures, data, procedures, and sampling designs suited to the application.The library example ranges from checking administrative information to testing whether librarians make the same purchasing decisions from summaries as from source reviews.Source: §7 · p. 12
8 Research strategy
Nearer-term summarising research should pursue general methods that require little tailoring and target indicative summaries in interactive environments where users can consult other information sources.
8 Research strategy
- General methods should require at most limited tailoring or be assembled from standard components rather than built from scratch.This provides a practical parallel to the longer-term pursuit of deeper source interpretation.Source: §8 · p. 13
- Indicative, skeletal summaries suit environments where users have loosely defined tasks, other aids, and enough expertise to check or extend interpretations interactively.The summaries are helpful but not essential, allowing users to compensate for their limited detail.Source: §8 · p. 13
- The near-term goal is ‘sufficient for the day’ summaries that support interactive browsing and document notification beyond keyword lists or repeated titles.These generic activities can occur across many task environments where a short readable indication of source substance is valuable.Source: §8 · p. 13
9 Forward direction
The proposed forward direction is intermediate, shallow linguistic processing rather than surface extraction or domain-specific deep fact extraction. It derives neutral source representations that can support multiple summary-selection strategies, while acknowledging incomplete interpretation and difficult evaluation.
9 Forward direction
- Nearer-term methods should go beyond surface extraction but avoid the deep, domain-dependent information required by fact extraction.The strategy is constrained by current NLP capabilities and the need for general applicability.Source: §9 · p. 14
- Sentence parsing into logical form with local anaphor resolution offers intermediate processing that exposes discourse entities and relations without a reference-domain model.It preserves more of the source than fact extraction while remaining linguistic rather than domain-driven.Source: §9 · p. 14
- Linking sentence representations through common elements yields a shallow but integrated and neutral source representation that is not committed to a predefined summary form.Without world knowledge, full discourse-entity identification remains impossible, but the representation leaves scope for alternative summarising methods.Source: §9 · p. 14
- Summary selection can combine entity-frequency statistics with markedness cues such as text location, discourse focus, and cue expressions.Because these cues remain connected to propositional referents rather than only surface strings, they can inform derived summary representations.Source: §9 · p. 14
- The strategy assumes robust parsing can capture enough important content because extended text is redundant and important information is emphasized or reiterated.This remains an assumption covering both systematic incompleteness and uneven analysis failures.Source: §9 · p. 15
- Evaluation is difficult because tolerant users with loosely defined tasks and varied tools make comparative judgments about alternative summaries harder.The proposal also faces missing detail from incomplete logical forms and unresolved coreference or quantification.Source: §9 · p. 15
10 An illustration
The illustration implements shallow summarising with quasi-logical forms, cohesion graphs, graph-based content selection, and constrained generation. It demonstrates the approach’s potential while exposing unresolved issues in analysis, evaluation, presentation, and computational trade-offs.
10 An illustration
- The illustrative system parses unrestricted news stories into quasi-logical forms, decomposes them into atomic predications, and links them in a cohesion graph.Graph links encode shared predicates, shared arguments, or similar arguments across sentences.Source: §10 · pp. 15–16
- Graph-based transformation selects predications using weighted edge types and a greedy score balancing centrality, representativeness, and coherence.Different weightings and scoring definitions can produce different summary node sets.Source: §10 · p. 16
- Generation synthesizes selected predications into a restricted ‘semi-text’ indicative summary, potentially consisting of sentences or fragmentary phrases.The experimental procedure groups predications, constructs logical forms, and uses the CLE to generate output while broadly retaining source order.Source: §10 · p. 16
- The approach extends network summarising from surface or lightly normalized words to more thorough logical-form interpretation implemented with modern NLP techniques.This is presented as the major step beyond earlier network-based approaches.Source: §10 · pp. 16–17
- Its assumptions are that source content has substantive meaning relations, summaries should cover important source content, and structured content must be conveyed as text.These assumptions motivate meaning representations, graph aggregation, clustering, generalisation, and entity-frequency marking.Source: §10 · p. 17
- Evaluation currently uses human acceptability, comparisons with statistical extraction, and many within-system parameter experiments, but functional usefulness remains unresolved.The absence of a task context limits evaluation, and broader testing must address analysis failures, topic organisation, output presentation, and computational effort.Source: §10 · p. 18
11 Conclusion
The paper concludes that shallow summarising is feasible, potentially useful, informative for discourse research, and adaptable to longer texts. Its value depends on producing more useful summaries than extraction-based methods.
11 Conclusion
- Shallow summarising is worthwhile if it produces more useful summaries than extraction-based methods such as Brandow, Mitze and Rau (1995).The programme can begin with existing NLP technology while pursuing practical value.Source: §11 · p. 18
- Predication networks may provide insight into discourse structure and ways to exploit limited domain information.These benefits are presented as learning opportunities from pursuing shallow strategies, not as established empirical results.Source: §11 · p. 19
- The approach is naturally extensible to longer texts, where stronger summary needs coincide with greater reduction challenges.Statistics and markedness clues should become more evident and exploitable, offering a route toward scalable summarising.Source: §11 · p. 19