PaperlayerView exact revision on arXiv

Source-linked AI summary

Controllable Citation Sentence Generation with Language Models

Nianlong Gu, Richard H. R. Hahnloser

arXiv:2211.07066v2cs.CL

TL;DR

Citation generation often gives authors little control over intent and keywords, although those attributes may be specified in advance. The paper fine-tunes a language model on context and user attributes, then uses controllability metrics with Proximal Policy Optimization. The conclusion reports advantages over GPT-3.5-turbo in automatic metrics and human evaluations.

  • Problem

    Citation-generation systems leave authors little room to specify the intent and keywords they want represented in a citation sentence.

    Source: §1 · p. 1

  • Method

    The framework fine-tunes a language model on manuscript and cited-paper context with user-specified intent and keywords, then uses controllability metrics as a Proximal Policy Optimization reward.

    Source: §1 · pp. 1–2

  • Results

    The conclusion reports that scientific-corpus-pretrained language models are essential for citation generation and that the fine-tuned model outperformed GPT-3.5-turbo on automatic metrics and human evaluations.

    Source: §7 · p. 8

  • Takeaways & Limitations

    The framework combines attribute suggestion and conditional citation generation in one language model, allowing users to control the generation process.

    Source: §1 · p. 2

    The study focuses primarily on citation intent and keywords, leaving other possible controls, such as specific sentences from the cited paper, for future work.

    Source: §8 · p. 9

Abstract

from arXiv · show

Citation generation aims to generate a citation sentence that refers to a chosen paper in the context of a manuscript. However, a rigid citation generation process is at odds with an author's desire to control specific attributes, such as 1) the citation intent, e.g., either introducing background information or comparing results, and 2) keywords that should appear in the citation text. To provide these degrees of controllability during citation generation, we propose to integrate the manuscript context, the context of the referenced paper, and the desired control attributes into a structured template and use it to fine-tune a language model (LM) via next-token prediction. We then utilize Proximal Policy Optimization to directly optimize the LM in favor of a high score of our proposed controllability metric. The proposed workflow harmoniously combines citation attribute suggestion and conditional citation generation into one LM, allowing for better user control.

1 Introduction

Citation generation has largely been automated with little room for author control, despite authors having specific intents and keywords in mind. The paper combines attribute-conditioned citation generation and controllability optimization in one language model, alongside a curated dataset.

1 Introduction

  • Prior neural systems generate citation sentences from manuscript and cited-paper context, but sequence-to-sequence pipelines leave little room for user control.Source: §1 · p. 1
  • Authors may want background or result-comparison intent and specific keywords, and may revise generation when it misses those motivations.Source: §1 · p. 1
  • A structured template orders manuscript and cited-paper context, citation attributes, and the target sentence for supervised next-token fine-tuning of an attribute-conditioned language model.Source: §1 · p. 1
  • The framework unifies citation-attribute suggestion and citation-sentence generation within one language model to give users control.Source: §1 · p. 2
  • Intent matching, keyword recall, fluency, and ROUGE-F1 metrics are used to form a Proximal Policy Optimization reward after supervised training.Source: §1 · pp. 1–2
  • The authors curate a dataset by parsing contextual text and citation attributes for controllable citation-generation research.Source: §1 · p. 2

2 Related Work

Prior citation-generation work often left control limited, and controlled-generation studies frequently focused on a single attribute. This work broadens the attributes considered and investigates reinforcement learning as a way to enhance controllability.

2 Related Work

  • Earlier sequence-to-sequence approaches used recurrent networks and knowledge-graph enhancements but did not fully address citation-generation complexity (Xing et al. (2020); Ge et al. (2021); Wang et al. (2022)).Source: §2 · p. 2
  • Controlled citation-generation studies often focused on one attribute, such as intent, while this work extends control to a broader range of attributes (Jung et al. (2022); Wu et al. (2021); Yu et al. (2022)).Source: §2 · p. 2
  • The work introduces methods to suggest potential attributes and balance automation with controllability.Source: §2 · p. 2
  • Beyond prompt-based approaches (Yang et al. (2022)), the study investigates reinforcement learning for controllability enhancement through experiments with cutting-edge language models.Source: §2 · p. 2

3 Method

The method combines manuscript and cited-paper context with citation intent and keywords in one language model, then evaluates and optimizes controllability using metric-based rewards.

3.1 LM Supervised Fine-tuning

  • The structured template provides manuscript and cited-paper context alongside intent and keyword controls, enabling one LM to generate attributes and citation sentences.The context includes up to five preceding sentences, manuscript title and abstract, and cited-paper title and abstract; intents are background, method, or result, with one or two noun-phrase keywords.Source: §3.1 · pp. 2–3

3 Method

  • The method trains one LM to infer citation attributes from context and to generate a citation sentence conditioned on context and attributes.This supports controlled generation from user-provided attributes and an uncontrolled mode in which the LM infers attributes before generating.Source: §3.1 · pp. 2–3

3.2 Controllability Evaluation Metrics

  • Four automatic metrics assess whether generated sentences align with the specified intent, include keywords, remain fluent, and match the reference text.IAS uses a SciBERT-based intent scorer, KR uses ROUGE-L recall, FS uses normalized SLOR, and ROUGE-F1 measures alignment with the ground-truth sentence.Source: §3.2 · p. 4

3.3 Controllability Enhancement via PPO

  • PPO optimizes the supervised fine-tuned model using controllability metrics as rewards because cross-entropy alone may not reliably satisfy specific criteria.The reward combines IAS, KR, FS, and ROUGE scores; the model generates and evaluates batches of citation sentences during PPO training.Source: §3.3 · pp. 4–5

4 Dataset

The dataset is built from citation triplets and split chronologically to reduce overlap with papers the evaluated models may have seen during pretraining.

4 Dataset

  • Citation triplets pair a citing paper, its citation sentence, and the cited paper, from which manuscript and cited-paper context is extracted.Citation intents are predicted with a SciBERT-based scorer, while up to two keywords are selected from sentence noun phrases by embedding similarity to manuscript and cited-paper context.Source: §4 · p. 5
  • A chronological split places training citing papers before March 1, 2023, and validation and test papers after that date to avoid prior exposure during pretraining.The authors state this strategy is intended to prevent tested language models from having encountered the same papers and associated citation sentences in the validation and test sets.Source: §4 · p. 5

5 Experiment

The experiments compare several pretrained model families, apply supervised fine-tuning and PPO to citation generation, and evaluate GPT-3.5-turbo using prompts.

5 Experiment

  • The study experiments with BART, GPT-Neo, Galactica, and LLaMa, spanning model sizes from 125M to 7B parameters.The models were pretrained on corpora dated before March 1, 2023.Source: §5 · p. 5
  • Larger Galactica and LLaMa models use INT4 quantization and LoRA to reduce GPU memory use during supervised fine-tuning.The larger models use a learning rate of 1e-4; all models are trained for 5k steps with batch size 128.Source: §5 · p. 6
  • Galactica and LLaMa receive additional PPO fine-tuning, with generated citation sentences used as episodes for mini-batch backpropagation.All PPO fine-tuning uses a learning rate of 1.41e-5.Source: §5 · p. 6
  • GPT-3.5-turbo is evaluated through prompts in both controlled and uncontrolled modes against the fine-tuned models.Controlled prompts provide context and citation attributes; uncontrolled prompts provide only context.Source: §5 · p. 6

6 Results and Discussion

The results examine whether citation attributes and PPO improve generation, how model scale and pretraining relate to performance, and how human preferences compare with automatic evaluations.

6 Results and Discussion

  • The evaluation investigates controllability, PPO, model size and pretraining, and human judgments beyond automatic metrics.The study compares generation modes, models, and a human evaluation against automatic metrics.Source: §6 · p. 6

6.1 Controllability Is Necessary

  • With context alone, Galactica-1.3B reaches only 0.66 intent precision and 0.26 keyword ROUGE-1 F1; adding gold intent improves ROUGE, with keywords amplifying the gains.The case study likewise shows intent shifting a mismatched background citation toward the target results, while a control keyword improves keyword inclusion.Source: §6.1 · pp. 6–7

6.2 PPO Enhances LM Controllability

  • PPO consistently improves IAS, KR, fluency, and ROUGE F1 across the compared Galactica and LLaMa models; Galactica-6.7B-PPO leads in ROUGE-F1.The comparison covers Galactica-125M, Galactica-6.7B, and LLaMa-7B models, each with and without PPO.Source: §6.2 · p. 7

6.3 Model Size and Pretraining Matter

  • Larger language models tend to perform better, and Galactica outperforms same-sized counterparts; its scientific-text pretraining is contrasted with GPT-Neo’s corpus containing little scientific literature.The authors attribute the performance gap to distinct pretraining datasets.Source: §6.3 · p. 7

6.4 Human Evaluation

  • In the human study, GPT-3.5-turbo sentences are preferred for fluency but often diverge from the ground-truth citations.Four computer-science researchers compared 105 randomly sampled examples using four criteria aligned with the automatic metrics.Source: §6.4 · p. 8

7 Conclusion

The conclusion presents a controllable citation-generation framework centered on user-specified attributes and PPO. It reports advantages over GPT-3.5-turbo in automatic metrics and human evaluations.

7 Conclusion

  • The framework integrates user-specified citation attributes into language-model citation generation.The conclusion highlights the role of user-specified attributes in the generation process.Source: §7 · p. 8
  • The study explores PPO as a means of improving citation-generation controllability.The authors describe the work as a foundation for future controllable citation-generation research.Source: §7 · pp. 8–9
  • The fine-tuned model shows advantages over GPT-3.5-turbo in automatic metrics and human evaluations.Source: §7 · p. 8

8 Limitations

The authors identify constraints on model scale, the citation attributes considered, and the prompting used for GPT-3.5-turbo.

8 Limitations

  • GPU constraints limit experiments to models of at most 7B; single-node fine-tuning of Galactica-120B or LLaMa-65B remains challenging.The authors identify the performance potential of larger fine-tuned models as future work.Source: §8 · p. 9
  • The study focuses on citation intent and keywords, leaving other attributes such as specific cited-paper sentences for future exploration.Source: §8 · p. 9
  • GPT-3.5-turbo may be underused; example-rich prompting could improve its performance but requires a long-input model such as GPT-3.5-turbo-16k.Source: §8 · p. 9

A Training and Evaluation of SciBERT-based Intent Scorer

The SciBERT-based intent scorer is trained primarily for citation intent classification, with two auxiliary tasks included to enhance the main task. Its loss weights the three cross-entropy objectives differently, and Table 6 reports intent-category and macro F1 scores on SciCite.

A Training and Evaluation of SciBERT-based Intent Scorer

B Hardware Requirements and PPO Training Hyperparameters

The experiments favor beam size 1 for citation generation, while examples show that intent and keyword controls can improve alignment and ROUGE F1. The section also reports hardware and GPT-3.5-turbo prompting settings.

B Hardware Requirements and PPO Training Hyperparameters

  • Training used four NVIDIA A100 80 GPUs, while inference used one NVIDIA RTX A6000.PPO batch and mini-batch sizes varied by model, and training continued until reward stopped improving.Source: B Hardware Requirements and PPO… · p. 12

C Prompt Templates for Querying GPT-3.5-turbo API

  • GPT-3.5-turbo prompts include cited-paper, manuscript, and local context, with citation attributes added for controlled generation.The API used a 2k maximum token limit and temperature 0.1.Source: C Prompt Templates for Querying… · p. 12

D Influence of Beam Size on Citation Generation Performance

  • Beam size 1 yielded the most consistent validation results; larger beam sizes did not improve performance and negatively affected ROUGE scores.The authors suggest that larger beams may favor keyword synonyms over the specified keywords, and used beam size 1 in their experiments.Source: D Influence of Beam Size on Citation… · p. 12

E Citation Generation Examples

  • With accurate citation intent alone, generated sentences aligned well with that intent and improved ROUGE F1; adding keyword attributes further boosted scores.The examples are generated by Galactica-6.7B-PPO in uncontrolled, intent-controlled, and intent-and-keywords-controlled modes.Source: E Citation Generation Examples · p. 12

F Web Interface for Human Evaluation

The web interface presents citation context, attributes, and generated sentences for human evaluation. Participants can rate outputs, submit evaluations, or skip examples outside their expertise.

F Web Interface for Human Evaluation

Loading 2211.07066v2…