Source-linked AI summary
CTRL: A Conditional Transformer Language Model for Controllable Generation
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, Richard Socher
TL;DR
Large language models generate text effectively but provide limited user control over specific attributes. CTRL addresses this by conditioning a 1.63 billion-parameter Transformer on naturally derived control codes, enabling controlled generation, task-specific behavior, and source-attribution analysis. The model makes generation more predictable under content or domain control and links sequences to likely training-data subsets, while its evidence remains descriptive rather than normative.
Problem
Users cannot easily control particular aspects of text generated by large-scale language models, especially when generation is not restricted to a specific task.
Method
CTRL trains a conditional language model on raw text prepended with naturally occurring control codes, learning p(x|c) for controllable generation.
Results
Control codes make generation more predictable under fixed prompts and support task-specific behavior, while code–data links allow prediction of likely training-data subsets.
Takeaways & Limitations
CTRL provides explicit controls for domains, content, entities, relationships, dates, and tasks, alongside a model-based way to analyze correlations in large text collections.
Takeaways & Limitations
CTRL reflects cultural associations in its non-universally covering training data and should provide descriptive evidence about domain correlations, not normative or prescriptive claims.
Abstract
from arXiv · showhide
Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text. We release CTRL, a 1.63 billion-parameter conditional transformer language model, trained to condition on control codes that govern style, content, and task-specific behavior. Control codes were derived from structure that naturally co-occurs with raw text, preserving the advantages of unsupervised learning while providing more explicit control over text generation. These codes also allow CTRL to predict which parts of the training data are most likely given a sequence. This provides a potential method for analyzing large amounts of data via model-based source attribution. We have released multiple full-sized, pretrained versions of CTRL at https://github.com/salesforce/ctrl.
1 INTRODUCTION
CTRL addresses limited control over unconstrained text generation by conditioning a large language model on codes derived from naturally occurring text structure. These codes specify generation features, support task-specific behavior, and link outputs to likely training-data subsets.
- Motivation and contribution: CTRL is a 1.63 billion-parameter conditional Transformer language model conditioned on codes for domain, style, topics, dates, entities, relationships, plot points, and task behavior.The codes are derived from structure that naturally co-occurs with raw text, preserving unsupervised training.
- Motivation and contribution: Control codes make desired features of generated text more explicit while retaining a general language-modeling setting.Codes can represent broad domains, subdomains, and other structure found in training resources.
- Motivation and contribution: CTRL can make text generation more predictable by controlling content or domain even when the initial prompt remains fixed.The codes also let users trigger text from different linguistic communities without mastering particular prompting patterns.
- Source attribution: CTRL can predict which training-data subset is most likely given a sequence, enabling model-based analysis of correlations learned from different domains.This relationship provides a way to study large amounts of text through the language model.
- Task-specific control: Task-specific control codes include question answering and machine translation, and can be combined with domain- or content-related codes for crossover behavior.The paper states that this inclusion improves important skills without harming model generality.
- Release: The authors released multiple full-sized, pretrained versions of CTRL for further research on controllable, general natural-language processing models.The release is provided through the Salesforce CTRL repository.
2 LANGUAGE MODELING
Language modeling learns a probability distribution over symbol sequences and factorizes it into next-word predictions. New text is then generated by sequentially sampling symbols conditioned on the preceding generated context.
- Sequence modeling: Language modeling aims to learn p(x) for sequences x=(x_1,...,x_n) whose elements come from a fixed symbol set.The sequence distribution is factorized using the chain rule of probability.
- Training objective: The chain-rule factorization decomposes language modeling into next-word prediction.Neural models are trained to minimize negative log-likelihood over a dataset.
- Generation: A language model generates a new sequence by sequentially sampling each symbol from p_θ(x_i|x_<i), conditioned on previously generated symbols.The process begins with p_θ(x_0) and continues through the generated prefix.
3 LANGUAGE MODELING WITH CTRL
CTRL changes standard language modeling by conditioning every sequence on a control code and learning p(x|c). It prepends codes to raw-text sequences, processes them with a causal Transformer, and samples the next token from the resulting distribution.
- Conditional modeling: CTRL learns the conditional distribution p(x|c), decomposing and training it with the control code included.The control code provides a point of control even when sampling the first token.
- Input representation: CTRL learns p_θ(x_i|x_<i,c) from raw-text sequences prepended with control codes.Each token combines a learned token embedding with a sinusoidal positional embedding before Transformer processing.
- Transformer architecture: The Transformer processes embedded sequences through attention layers whose causal mask prevents attending to future tokens.Each layer uses multi-head attention and a feedforward network with ReLU activation, layer normalization, and residual connections.
- Output and generation: Training applies cross-entropy to vocabulary scores, while generation softmax-normalizes the final-token scores to sample the next token.The same output scores therefore support both optimization and autoregressive generation.
- Training data: CTRL is trained on 140 GB of text spanning encyclopedic, literary, social-media, news, review, translation, and question-answer data.The corpus draws from multiple domains and task-oriented collections.
- Training data: The data is tokenized with BPE using a roughly 250K-token vocabulary, filtered for excessive unknown tokens, and stored as a contiguous token stream with prepended domain codes.These preprocessing choices reduce the collected corpus from 180 GB to 140 GB.
- Model configuration: The model uses dimension d=1280, inner dimension f=8192, 48 layers, and 16 attention heads per layer.Token embeddings are tied to the final output embedding layer, with dropout probability 0.1 after residual connections.
- Optimization: CTRL is trained for 800k iterations on 256 TPU-v3 cores with global batch size 1024, taking approximately two weeks.Training uses Adagrad with warmup and gradient clipping.
4 CONTROLLABLE GENERATION
CTRL uses control codes and sampling choices to make generation more predictable across domains, styles, tasks, and content. The paper demonstrates fine-grained control, task triggering, link-based conditioning, and novel combinations of codes.
- Sampling: Greedy sampling with θ ≈1.2 balances truthful generation and reduced repetition, provided the model has learned a sufficiently reliable distribution.The penalty discounts scores of previously generated tokens, while θ = 1 is equivalent to the unmodified distribution.
- Control codes: CTRL uses control codes to produce predictable variation for identical prompts and can generate domain-specific text without a prompt.Codes can specify domains and be combined for finer-grained control.
- Control codes: Combining domain, rating, and title-related codes increasingly constrains the generated text.The paper illustrates finer-grained control by combining Reviews, Rating:, and VALUE codes.
- Control codes: Link-shaped control codes specify domain, subdomain, entities, entity relations, and dates, enabling novel content from URL structures.The model interprets entities differently depending on dates, domains, and topics.
- Control codes: Task-specific codes trigger question-answering and translation generation through complex templates mixed with natural-language prompts.These codes constrain generation more strongly than general domain controls.
- Control codes: Mixed control codes generate novel cross-over behavior, including coherent English-German diet text and politics-related French prompts unseen in training.The diet example preserves the domain while alternating translated lines; the politics example combines an unseen domain-language pairing.
5 SOURCE ATTRIBUTION
CTRL uses domain control codes to rank which training-data subsets are most likely given a sequence, providing descriptive evidence about learned correlations rather than truth or value judgments.
- Source attribution: Domain control codes partition the training data into mutually exclusive sets and support ranking domains by likelihood for a sequence.The ranking is computed from pθ(x|c) together with a prior over domain control codes.
- Source attribution: A uniform prior is used because the empirical training-data prior overweights domains with larger datasets.
- Limitations: Source attribution depends on the original training sources’ cultural associations and lacks universal coverage.
- Interpretation: CTRL’s attribution is descriptive evidence of domain-language correlations, not a measure of veracity or a basis for normative claims.Contradictory statements may be attributed to the same sources because the model learns correlations rather than whether claims are true or false.
6 RELATED WORK
Related work positions language models as transferable representations and increasingly general task learners, while text-generation research also addresses repetition and the novelty–consistency trade-off.
- Language modeling: Language models support transferable word, contextualized-word, and document representations, and can themselves transfer to new tasks through fine-tuning.
- Language modeling: Recent language-modeling research uses memory mechanisms and large Transformer architectures to improve benchmark perplexity and generation capabilities.
- Multi-task learning: Language models can learn multiple tasks and adapt quickly to patterns in input prompts, offering an alternative to supervised multitask learning.
- Sampling methods and coverage mechanisms: Text-generation research targets repetition through sampling methods, coverage mechanisms, and context-based losses, while balancing novelty against consistency.
7 FUTURE DIRECTIONS
Future directions extend CTRL’s control codes, task coverage, training-data analysis, and human interface while preserving the model’s general language capabilities and recognizing current scope boundaries.
- More control codes and finer-grained control: CTRL’s domain, topic, entity, relation, and date codes could be refined through URL structure or manually extracted document structure.
- Extensions to other areas in NLP: Including supervised or task-specific data may improve important skills without necessarily harming the general nature of unsupervised learning.
- Extensions to other areas in NLP: Whether CTRL can learn extractive question answering or state-of-the-art multilingual machine translation while preserving general functionality remains open.The experiments covered trivia-style question answering without context documents and small amounts of machine translation data.
- Analyzing the relationships between language models and training data: CTRL’s explicit relationship with training data offers a way to study cultural associations learned from different sources, although its data is not representative of many linguistic communities.
- Making the interface between humans and language models more explicit and intuitive: Future work aims to make the interface between humans and language models more intuitive by providing users with more control over generation.
8 CTRL-ALT-DEL: THE ETHICS OF LARGE LANGUAGE MODELS
CTRL’s release emphasizes openness alongside governance measures intended to anticipate misuse, encourage responsible use, and monitor deployment after release.
- Openness and replicability: The authors released all trained CTRL models as an expression of openness and replicability.
- Responsible release: The team used prerelease expert review and a technology-foresight exercise to broaden governance inputs and anticipate plausible malicious uses.
- Responsible release: A README code of conduct modeled on emerging licenses aims to encourage reflection and reduce unintended uses, but has no legal force.
- Responsible release: The README includes deliberation questions to encourage users to reflect on norms and responsibilities associated with generated artificial content.
- Responsible release: Users are asked to document applications and report usage to support post-release monitoring of how CTRL is used in the wild.
9 CONCLUSION
CTRL is a publicly released 1.63-billion-parameter language model designed to make text generation more controllable through explicit codes.
- CTRL is the largest publicly released language model to date, with 1.63 billion parameters.
- Its control codes specify domain, subdomain, entities, relationships, dates, and task-specific behavior.
- The release aims to support more controllable, general natural-language-processing models and invites discussion of artificial generation.
A DATA SOURCES AND BREAKDOWN
Table 7 maps data sources to the control codes available for conditioning generation, with secondary codes varying by source type.
- Wikipedia, Books, News, and multilingual data have no secondary control code.
- Reviews can use a Rating: secondary code with values from 1.0 through 5.0.
- Links accept full or partial URLs, while Reddit data can use Title: or Text: secondary codes.