Source-linked AI summary
Controlled Text Generation with Natural Language Instructions
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell, Mrinmaya Sachan
TL;DR
Controlled generation must meet task-specific constraints without sacrificing fluent text, yet existing decoding-time approaches can affect quality and speed and adapt poorly to new constraints. INSTRUCTCTG trains a model on weakly supervised examples expressed as natural-language instructions, with demonstrations available for in-context learning. It reports comparable or better constraint satisfaction than decoding-based methods, faster generation in some cases, and few-shot generalization to unseen constraints.
Problem
Applications need generated text to satisfy task-specific constraints, while decoding-time methods can reduce quality or speed and may not handle novel constraints or combinations well.
Source: §1 · p. 1Method
INSTRUCTCTG imputes constraints in natural text with NLP tools or simple rules, verbalizes them as natural-language instructions, and trains an instruction-tuned model with optional demonstrations.
Source: §1 · pp. 1–2, §2 · p. 2Results
INSTRUCTCTG achieves comparable or better constraint satisfaction than decoding-based methods, is faster in some cases, and shows few-shot generalization to unseen constraints.
Source: §1 · p. 2Takeaways & Limitations
The framework uses instruction following for controlled generation without modifying the model’s decoding procedure.
Source: §1 · p. 2, §5 · p. 8The experiments cover only a few constraint types and use T0 as the sole backbone model.
Source: §A.1 · p. 12
Abstract
from arXiv · showhide
Large language models generate fluent texts and can follow natural language instructions to solve a wide range of tasks without task-specific training. Nevertheless, it is notoriously difficult to control their generation to satisfy the various constraints required by different applications. In this work, we present InstructCTG, a controlled text generation framework that incorporates different constraints by conditioning on natural language descriptions and demonstrations of the constraints. In particular, we first extract the underlying constraints of natural texts through a combination of off-the-shelf NLP tools and simple heuristics. We then verbalize the constraints into natural language instructions to form weakly supervised training data. By prepending natural language descriptions of the constraints and a few demonstrations, we fine-tune a pre-trained language model to incorporate various types of constraints. Compared to existing search-based or score-based methods, InstructCTG is more flexible to different constraint types and has a much smaller impact on the generation quality and speed because it does not modify the decoding procedure. Additionally, InstructCTG allows the model to adapt to new constraints without re-training through the use of few-shot task generalization and in-context learning abilities of instruction-tuned language models.
1. Introduction
LLMs can generate fluent text, but applications also demand task-specific constraints. INSTRUCTCTG shifts control into instruction-conditioned training and reports competitive constraint satisfaction and generalization to unseen constraints.
1. Introduction
- Decoding-time methods can lower text quality, slow generation, and require bespoke features or classifiers for novel constraints.They may also struggle with new constraint types or novel combinations of existing constraints.Source: §1 · p. 1
- INSTRUCTCTG treats naturally occurring sentences as weakly supervised examples, imputing latent constraints with off-the-shelf NLP tools or simple rules and verbalizing them as instructions.The model is instruction-tuned on the augmented corpus.Source: §1 · p. 1
- For some constraints, a few constraint–text demonstrations support in-context learning; the framework leaves the decoding procedure unchanged.The authors report improved generation quality and efficiency compared with methods that directly incorporate constraints into decoding.Source: §1 · p. 2
- Across lexical, syntax, semantic, style, and length constraints, INSTRUCTCTG achieves comparable or better constraint satisfaction, is faster in some cases, and generalizes to unseen constraints.The evaluation fine-tunes T0 across these constraint types.Source: §1 · p. 2
2. INSTRUCTCTG
INSTRUCTCTG synthesizes weakly supervised constraint–text pairs and verbalizes constraints as natural-language instructions for an instruction-tuned model. The framework covers multiple constraint types, their compositions, and conditional generation tasks.
2.1. Data Collection
- Lexical constraints require keywords; augmentation adds disjunctive and exclusion requirements, while syntactic constraints specify POS sequences or parse-tree structures.The data pipeline extracts and lemmatizes keywords, and synthesizes lexical and syntactic pairs from C4 sentences.Source: §2.1 · pp. 2–3
- Semantic and style controls use classifiers and task datasets, while length is constrained to a five-word interval; the pipeline synthesizes one million pairs per category and 50,000 each for development and test.The considered semantic controls include topic, sentiment, and political slant; style includes formality, politeness, biasedness, toxicity, and simplicity.Source: §2.1 · p. 3
2.2. Constraint Verbalization
- Natural-language templates verbalize constraints, and diverse phrasings are used to support prompt-based generation and generalization to unseen constraints.Instructions can describe new constraints without retraining.Source: §2.2 · pp. 3–4
2.3. Composition of Constraints
- For composition, individual constraint instructions are joined with “and,” with training examples combining between two and five constraints.A further million examples with multiple constraints are sampled from C4.Source: §2.3 · p. 4
2.4. Extension to Conditional Text Generation
- For conditional generation, task prompts are combined with control instructions and examples are drawn from conditional-task datasets rather than general unlabeled text.The experiments include paraphrase and question generation.Source: §2.4 · pp. 4–5
2.5. Instruction-based Meta In-context Learning
- Training prepends constraint instructions with five demonstrations, omitting demonstrations on half of examples so the model can also generate without them.The language model is fine-tuned using maximum likelihood estimation and teacher forcing.Source: §2.5 · p. 5
2.6. Decoding
- At inference, users describe target constraints and may add examples, then use an out-of-box generation method without modifying decoding.Seen constraints can use existing templates; unseen constraints can be described anew.Source: §2.6 · p. 5
3. Experiments
Across constraints seen during training, INSTRUCTCTG achieves comparable or better constraint satisfaction while generating more fluent text, and it generalizes to unseen constraints with few demonstrations. The analyses also report results on conditional generation and compositions of constraints.
3.1. Experimental Setup
- Experiments fine-tuned T0-11B on mixed constraints, holding out political slant, humor, and politeness; constraint satisfaction used constraint-specific metrics, while OPT-30B perplexity measured fluency.Source: §3.1 · p. 5
3.2. Results on Seen Constraints
- On five seen constraint types, INSTRUCTCTG matched or beat many methods on constraint satisfaction, generated more fluent text, and was over 4× faster than NeuroLogic and over 40× faster than Diffusion-LM.Source: Table 2 · p. 6, §3.2 · p. 6
3.3. Results on Unseen Constraints
- With a few demonstrations and no model update, INSTRUCTCTG achieved higher constraint satisfaction and output quality than baselines and performed competitively with oracle CFT.Source: Table 3 · p. 6, §3.3 · p. 6
- INSTRUCTCTG beat score-based unseen-constraint baselines and achieved 97% accuracy at 100–105 words, versus CFT’s 91% after both trained only below 60 words.CFT is a skyline in this setting because it was trained on constraints unseen by the other methods.Source: §3.3 · p. 6
- For unseen length control, INSTRUCTCTG satisfied 82% of constraints without demonstrations and 88% with five, versus 65% for T0-11B.Source: §3.3 · p. 6
3.4. Analysis
- Ablations found that natural-language constraint verbalization and meta in-context learning improve performance, especially on unseen tasks; multiple templates marginally help, and five demonstrations balance performance and efficiency.Training with multiple constraints at once also improved performance.Source: §3.4 · p. 7
- On conditional generation tasks, INSTRUCTCTG achieved constraint satisfaction comparable to prior methods while substantially improving task-specific metrics.Source: §3.4 · p. 7
- INSTRUCTCTG effectively handled multiple constraints at the same time.Source: §3.4 · p. 7
4. Related Work
Related work reviews standard text-generation stages and controlled-generation approaches, emphasizing how constraint handling affects output quality, flexibility, and speed.
4.1. Neural text generation
- Neural text generation separates probability-model training from decoding, which uses algorithms such as beam search and sampling.Source: §4.1 · p. 7
4.2. Controlled text generation
- Table 4 reports better task-specific performance with comparable SUCC for INSTRUCTCTG versus baselines on conditional generation tasks.Source: Table 4 · p. 7
- Constrained search modifies the search space to enforce lexical constraints, but can slow generation, reduce text quality, and exclude non-lexical controls.Source: §4.2 · p. 7
- Score-based sampling supports diverse and composed constraints, but offers no satisfaction guarantee and can slow generation or degrade quality through distribution changes and score-weight tuning.Source: §4.2 · pp. 7–8
- Other approaches train with lexical constraints appended to inputs, while CTRL uses pretraining structures associated with text to support some style and domain controls.Source: §4.2 · p. 8
5. Conclusion
INSTRUCTCTG uses large language models’ instruction-following ability as an alternative to modifying their decoding procedure.
5. Conclusion
- INSTRUCTCTG exploits large language models’ instruction-following ability for controlled text generation without modifying the decoding procedure.Source: §5 · p. 8
A. Appendix
Table 5 presents generated samples from INSTRUCTCTG and comparison models across control tasks.
A. Appendix
- Table 5 provides qualitative samples from INSTRUCTCTG and compared models for different control tasks.Source: Table 5 · p. 12
A.1. Limitations and Potential Social Impacts
The paper’s evaluation covers only a few constraint types and the T0 backbone; it also identifies possible misuse to generate toxic or biased text.
A.1. Limitations and Potential Social Impacts
- Experiments cover only a few constraint types and use T0 as the sole backbone, leaving utility across broader constraint sets and model sizes untested.Source: §A.1 · p. 12
- INSTRUCTCTG could be misused through instructions that encourage generation of toxic or biased text, a stated risk shared by controlled text generation methods.Source: §A.1 · p. 12
A.2. Implementation Details
The implementation uses one million single-constraint and one million multi-constraint examples for fine-tuning, with separate procedures for baseline tuning and constraint-composition evaluation.
A.2. Implementation Details
- Fine-tuning uses 1 million single-constraint examples and 1 million examples with 2–5 constraints.For conditional generation, training also uses all original dataset examples and synthesized lexical-constraint data; the learning rate is 5e-5.Source: §A.2 · p. 12
- The CFT baseline uses InstructCTG’s fine-tuning hyperparameters, while other baselines use paper-recommended settings followed by development-set grid search.The authors report that the shared CFT settings work well for both conditional fine-tuning and their approach.Source: §A.2 · p. 12
- Constraint composition is tested on 1,000 examples with at least one valid constraint per category, composing 2–5 randomly sampled constraints.Source: §A.2 · p. 12
A.3. Examples of Generated Text
The section presents generated-text samples for InstructCTG and the compared baselines for qualitative analysis.
A.3. Examples of Generated Text
- Table 5 presents generated-text samples from InstructCTG and compared baselines for qualitative analysis.Source: §A.3 · p. 12