Cited Text Spans for Citation Text Generation

1 Introduction

• Existing abstractive systems condition on cited-paper abstracts, although gold citations may refer to body content unavailable in those abstracts.The authors hypothesize that this mismatch helps explain why judges rate gold citations as less relevant when given only the abstract.Source: §1 · p. 1
• Manually annotated CTS datasets are small and low-agreement, while trained retrieval systems perform poorly and require the gold citation as input.The reported inter-annotator agreement is Cohen’s κ of 0.16∼0.52, and retrieval systems have F1<0.2.Source: §1 · p. 1
• Citation texts generated using CTS are significantly more accurate and faithful than those generated using cited-paper abstracts.Source: §1 · p. 1
• Distant CTS labeling can substitute for expensive human annotations, and keyword-based retrieval offers a human-in-the-loop route to grounding generation in cited-paper full text.Source: §1 · p. 2

2 Background and Related Work

• CL-SciSumm and AbuRa’ed manually annotated CTS, but both datasets have low inter-annotator agreement and support poorly performing retrieval systems.CL-SciSumm covers 364 citation texts citing 20 papers; AbuRa’ed annotated sentence-level CTS for 20 related-work sections.Source: §2 · p. 2
• ScisummNet used ROUGE-based retrieval for community evidence, while later work used CTS for extractive related-work generation and additional summarization context.Source: §2 · p. 2
• These prior approaches use the target citation text as the retrieval query, which is unavailable in citation generation.Source: §2 · p. 2

3 A Large-Scale CTS Dataset: Distant vs. Human Labeling

• The authors treat low agreement as evidence that a citation can have multiple reasonable CTS, as with extractive summaries.Source: §3 · p. 2

3.1 Approach

• The approach ranks cited-paper sentences by average ROUGE-1 and ROUGE-2 recall against the gold citation text, using stop-word removal and no GPU.The authors prefer unsupervised retrieval because human-labeled CTS datasets are small; BERTScore slightly underperforms ROUGE in their experiments.Source: §3.1 · p. 3, §3.3 · p. 3

3.2 Evaluation Metrics

• Evaluation measures token coverage with ROUGE recall and faithfulness with QuestEval and ANLI non-contradiction scores.Source: §3.2 · p. 3

3.3 Findings

• Top-40 distant CTS cover 80% of human-labeled CTS in CL-SciSumm and 95% in AbuRa’ed; cited papers average 200 sentences, so this is roughly the top 20%.Source: §3.3 · p. 3, Figure 2 · p. 3, Figure 3 · p. 3
• Distant labels have higher token overlap with target citations than human labels, while both are comparably faithful; strict intersection scoring shows annotators select different but reasonable sentences.Source: §3.3 · pp. 3–4, Figure 4 · p. 4, Table 1 · p. 4
• Distantly labeled CTS performs comparably to human labels on AbuRa’ed and outperforms them on CL-SciSumm in downstream generation.The authors conclude that expensive human CTS labels are not necessary for training a retriever for this task.Source: §3.3 · p. 4

4.1 Problem Formulation

• The task generates citations one at a time using up to two preceding sentences as context and retrieved CTS from the cited paper.The study uses dominant-type CORWA citations grounded in specific CTS.Source: §4.1 · p. 4

4.2.1 CTS Retrieval Strategies

• CTS retrieval has higher token overlap and faithfulness to target citations than abstracts; the study compares context, oracle, and keyword retrieval strategies.Context retrieval uses citation context, oracle retrieval uses the gold citation as an upper-bound query, and keyword retrieval simulates researcher-provided guidance.Source: §4.2.1 · pp. 4–5, Table 3 · p. 5

4.4 Results and Discussion

• Oracle and Keyword CTS generation massively outperform Abstract in Table 4, while Abstract receives the lowest human Overall rating.All settings score above 4 on Relevance when judges see the retrieved CTS.Source: Table 4 · p. 5, §4.4 · p. 6

4.2.2 Citation Text Generation Strategies

• Generation uses RAG with FiD and BART-base or an LED model, conditioning on citation context and retrieved CTS.The LED input also includes the citation mark.Source: §4.2.2 · p. 5

4.2.3 Experimental Settings

• Models are trained on the distantly labeled CORWA set, tuned on its smaller human-labeled training set, and evaluated on the human-labeled test set.The human evaluation compares Abstract and each CTS strategy, with judges rating fluency, relevance, coherence, and overall quality in two rounds.Source: §4.2.3 · pp. 5–6, §4.3 · p. 6

4.4 Results and Discussion

• CTS-based generations have higher coverage but lower density than Abstract, extracting and reorganizing shorter phrases; the authors note this does not itself address plagiarism risk.They suggest post-processing, such as rewriting, to further reduce that risk.Source: §4.4 · pp. 6–7, Figure 5 · p. 7
• Context-only retrieval is challenging because citation context and CTS come from different papers; Oracle and Keyword tend to retrieve more specific sections, which may add unnecessary detail.The reported section-header pattern includes “Model or Architecture” and “Results.”Source: §4.4 · p. 7, Table 7 · p. 7

4.3 Human Evaluation

• In blind evaluation, human judges cannot distinguish gold from generated citation texts; revealing the gold leads them to raise its scores and lower candidate scores.The evaluations use two rounds, with gold citations revealed only in the second.Source: Table 5 · p. 6, §4.3 · p. 6, §4.4 · p. 7

5 Conclusion

• CTS-based citation generation is more accurate and faithful than prior abstract-based approaches.Distant labeling can substitute for expensive manual CTS annotations and supports a dataset large enough to train a supervised retriever.Source: §5 · p. 8
• Human-in-the-loop, keyword-based CTS retrieval achieves strong performance in a practical setting.The approach avoids using the unavailable gold citation text as the retrieval query.Source: §5 · p. 8
• Context-only citation-context retrieval is challenging and does not achieve the best performance on automated metrics.Despite this, human judges prefer it, making it a direction for further investigation.Source: §5 · p. 8

Limitations

• Context-only CTS retrieval is unsatisfactory, so support-document retrieval and document-based citation generation should be treated as separate tasks.The automated retrieval component requires further investigation.Source: Limitations · p. 8
• User-provided keywords can make retrieval perform well, but the human-in-the-loop setting may be unfriendly to less-experienced researchers.Future work could automatically predict gold-citation keywords using paragraph- or document-level topic analysis with LLMs.Source: Limitations · pp. 8–9
• ROUGE-based distant labels may miss semantically relevant CTS with lower lexical similarity, and the resulting retriever is not claimed to be state of the art.The authors also note that stronger retrieval methods and larger, more extensively tuned generators could improve performance.Source: Limitations · p. 9
• The authors could not perform large-scale quality assurance on distantly retrieved CTS because manual collection or evaluation is costly.They suggest a future fact-checking module to filter CTS sentences that do not entail the citation text.Source: Limitations · p. 9

Ethics Statement

• CTS-based generators are more extractive and directly copy parts of the input CTS into citation outputs.The authors say this copying could result in plagiarism if outputs are used directly without further modification.Source: Ethics Statement · p. 9
• A rewriting module may be needed to reduce the plagiarism risk, but post-processing is outside this work’s scope.The authors note that abstract-based systems also extract longer subsequences, so the issue is not unique to CTS.Source: Ethics Statement · p. 9
• The CTS-based approach extracts shorter subsequences and rearranges their order, unlike the longer subsequences extracted by the abstract-based approach.Source: Ethics Statement · p. 9

A.1 Opting Out of FAISS

• CTS retrieval searches only cited-paper sentences and is performed separately for each citation and its cited paper.These constraints differ from the original RAG setup, where every Wikipedia article could be a retrieval candidate.Source: §A.1 · p. 12
• FAISS does not support the partitioned search required by this CTS retrieval setup.The authors therefore replace it with simpler CTS retrieval strategies.Source: §A.1 · p. 12
• FAISS was used in the original RAG implementation to speed up retrieval across candidate Wikipedia articles.Source: §A.1 · p. 12

A.2 Freezing Query Encoder Weights

• Sentence-level CTS retrieval makes on-the-fly query-encoder updates intractable, so experiments cache retrieved sentences in advance without fine-tuning the encoder.Preliminary paragraph-level experiments found no significant improvement from fine-tuning over freezing the query encoder.Source: §A.2 · p. 12

A.3 Citation Text Generation Hyper-parameter Settings

• Table 8 reports the tuned hyper-parameter settings for CTS-based citation text generation experiments.Source: §A.3 · p. 12

B.1 Retrieval and Generation

• Table 9 covers citation-text similarity and faithfulness across input settings using ROUGE-recall, QuestEval, and ANLI.Source: Table 9 · p. 12
• DPR retrieves documents using query and document encoders, while RAG generates from retrieved documents and FiD jointly decodes their encodings.RAG-sequence uses a fixed distribution over retrieved documents for the full output; RAG-token uses a different distribution for each token.Source: §B.1 · p. 12

• QuestEval combines precision from citation-generated questions answered by the input with recall from input-generated questions answered by the citation.Source: C QuestEval and CTS-based Citation… · p. 12
• QuestEval struggles with short CTS-based citations: detailed CTS inputs generate questions the citation text cannot answer, yielding extremely low recall.CTS-based citations have a much higher compression ratio than abstract-only citations.Source: C QuestEval and CTS-based Citation… · pp. 12–13
• Longer Abstract-only citations can receive higher QuestEval recall scores than shorter Oracle citations, despite the Abstract input being shorter and less entity-rich than the Oracle input.The authors report many similar test-set examples.Source: C QuestEval and CTS-based Citation… · p. 13
• Table 10 illustrates an Oracle CTS citation that receives a low QuestEval score despite being factually supported by its cited CTS sentence.Its claim about parse-tree validity is supported by the cited work’s Introduction.Source: C QuestEval and CTS-based Citation… · p. 13, Table 10 · p. 14