PaperlayerView exact revision on arXiv

Source-linked AI summary

TITAN: T Cell Receptor Specificity Prediction with Bimodal Attention Networks

Anna Weber, Jannis Born, María Rodríguez Martínez

arXiv:2105.03323v1q-bio.BMcs.LG

TL;DR

Predicting TCR–epitope binding is difficult because sequence space is vast while available data are sparse and concentrated on a small number of epitopes. The paper introduces TITAN, a bimodal model that explicitly encodes TCRs and epitopes, using SMILES-based transfer learning and augmentation. TITAN reaches ROC-AUC 0.87 for epitopes seen during training and improves unseen-epitope performance, although sparse data can cause the model to treat epitopes as classes.

  • Problem

    TCR specificity prediction is challenging because TCR sequence diversity is large, while datasets are sparse and contain many TCRs for relatively few epitopes.

  • Method

    TITAN explicitly encodes TCR and epitope sequences with a bimodal attention network, using SMILES representations for transfer learning and data augmentation.

  • Results

    TITAN’s best pretrained, semifrozen, augmented model achieves ROC-AUC 0.87 on epitopes included in training data and outperforms ImRex by a large margin.

  • Takeaways & Limitations

    TITAN supports strong generalization to unseen TCRs and improves prediction on unseen epitopes, while attention analysis identifies chemically meaningful molecular structures.

  • Takeaways & Limitations

    Generalization to unseen epitopes remains challenging because sparse data can lead TITAN to treat epitopes internally as classes.

Abstract

from arXiv · show

Motivation: The activity of the adaptive immune system is governed by T-cells and their specific T-cell receptors (TCR), which selectively recognize foreign antigens. Recent advances in experimental techniques have enabled sequencing of TCRs and their antigenic targets (epitopes), allowing to research the missing link between TCR sequence and epitope binding specificity. Scarcity of data and a large sequence space make this task challenging, and to date only models limited to a small set of epitopes have achieved good performance. Here, we establish a k-nearest-neighbor (K-NN) classifier as a strong baseline and then propose TITAN (Tcr epITope bimodal Attention Networks), a bimodal neural network that explicitly encodes both TCR sequences and epitopes to enable the independent study of generalization capabilities to unseen TCRs and/or epitopes. Results: By encoding epitopes at the atomic level with SMILES sequences, we leverage transfer learning and data augmentation to enrich the input data space and boost performance. TITAN achieves high performance in the prediction of specificity of unseen TCRs (ROC-AUC 0.87 in 10-fold CV) and surpasses the results of the current state-of-the-art (ImRex) by a large margin. Notably, our Levenshtein-distance-based K-NN classifier also exhibits competitive performance on unseen TCRs. While the generalization to unseen epitopes remains challenging, we report two major breakthroughs. First, by dissecting the attention heatmaps, we demonstrate that the sparsity of available epitope data favors an implicit treatment of epitopes as classes. This may be a general problem that limits unseen epitope performance for sufficiently complex models. Second, we show that TITAN nevertheless exhibits significantly improved performance on unseen epitopes and is capable of focusing attention on chemically meaningful molecular structures.

1 Introduction

TCR specificity prediction is important but difficult because receptor diversity and sparse, epitope-skewed datasets make the sequence-to-binding relationship challenging to learn. TITAN addresses limitations of categorical models by explicitly encoding both TCRs and epitopes, while using attention and SMILES representations for interpretability and molecular detail.

  • Motivation: TCR repertoires can theoretically contain 10^15–10^20 receptors, making reliable sequence-based specificity prediction highly challenging.Such predictions could inform infectious-disease history, vaccine effectiveness, autoimmune reactions, and therapeutic T-cell applications.
  • Data constraints: Available datasets remain sparse relative to TCR sequence space and typically contain tens of thousands of TCRs but only a few hundred epitopes.This imbalance reflects experimental settings that link many TCRs to a single epitope.
  • Prior approaches: Categorical epitope models learn patterns of TCRs binding the same epitope but cannot predict specificity for unseen epitopes.This limitation follows from treating epitopes as fixed categories represented in training data.
  • TITAN: TITAN explicitly encodes TCR and epitope sequences with a bimodal neural network that combines convolutions, context attention, and binding-probability prediction.The architecture is designed to study generalization to unseen TCRs and epitopes.
  • Interpretability: Context attention supports interpretation by highlighting amino acids or atoms associated with the model’s binding prediction, while SMILES provides an atom-level epitope representation.SMILES is a string-based molecular representation used here to expose chemically meaningful structures.

2.1 Data

The study combines VDJ and COVID-19 TCRβ datasets, applies filtering and balancing procedures, and generates negative examples by shuffling TCR–epitope pairings. A separate ImRex-derived training set and McPAS-based independent test set support final state-of-the-art comparison.

  • Dataset construction: The combined data comprise VDJ and COVID-19 samples, while the study restricts analysis to TCRβ sequences because paired-chain data remain rare.The combined dataset is called VDJ+COVID-19.
  • VDJ dataset: After excluding rare epitopes and downsampling to 400 TCRs per epitope, the VDJ dataset contains 10,599 examples across 87 epitopes.The source VDJ collection contained 40,438 TCR sequences assigned to 191 peptides.
  • COVID-19 dataset: The COVID-19 dataset is filtered to unique-epitope, productive samples and processed identically, yielding 12,996 examples.The original dataset contained 154,320 examples associated with 269 epitopes or epitope groups.
  • Negative sampling: Negative examples are generated by shuffling TCR and epitope sequences to create pairings not shown to bind.The procedure is used because randomly selected TCR–epitope pairs have a low probability of binding and avoids adding naive TCRs from other sources.
  • Evaluation data: For comparison with ImRex, models are trained on its public dataset and evaluated on an independent McPAS-derived test set after removing overlapping TCRs.The ImRex training data contain 13,404 samples for 118 epitopes with 50% negatives.

2.2 Models

The paper formulates binding prediction as learning a mapping from TCR–epitope pairs to binary affinity labels and establishes a Levenshtein-distance K-NN classifier as a non-parametric baseline. The baseline retrieves nearby sequence pairs and predicts from their labels.

  • Problem formulation: The prediction problem learns a mapping Φ: E×T → A, where each TCR–epitope pair receives a binary binding label.The training set consists of epitope sequences, TCR sequences, and labels indicating whether binding occurred.
  • K-NN baseline: The K-NN baseline measures similarity using the sum of length-normalized Levenshtein distances between epitope and TCR sequences.For the baseline, the TCR sequence is the CDR3 region.
  • TITAN model: TITAN’s architecture accepts TCR and epitope sequences as separate input streams before applying convolutions, context attention, and dense layers to output binding probability.The figure presents the model as a bimodal architecture with attention over each sequence conditioned on its binding partner.
  • Prediction procedure: For an unseen test pair, K-NN retrieves the k closest training samples and predicts the unknown label from their observed labels.Odd k values from 1 through 25 are evaluated to avoid ties, and the best ROC-AUC determines k.

2.3 Model Architecture

TITAN separately encodes TCRs and epitopes, supports amino-acid or SMILES representations, and combines multi-scale convolutions with bimodal context attention. Attention weights use one modality as context to identify relevant tokens in the other before dense layers predict binding.

  • Input encoding: TITAN encodes TCRs with BLOSUM62 and represents epitopes either as amino-acid sequences or atom-level SMILES strings.The TCR input can use the full variable sequence or only the CDR3 region, and sequences are padded to 500 tokens.
  • Convolutional processing: Three convolutional channels with kernel sizes 3, 5, and 11 aggregate local sequence information, alongside a residual no-convolution channel.The architecture is adapted from compound-protein-interaction models.
  • Context attention: Context attention uses the binding partner to compute token importance over the reference sequence.This allows the model to condition the relevance of one input on information from the other input.
  • Attention computation: The attention mechanism projects reference and context representations into a shared space, combines them, and applies softmax to produce attention weights.The resulting weights filter the inputs before prediction.
  • Prediction: After attention, TCR and peptide information passes through dense layers with 368 and 184 nodes to produce the binding probability.The two modalities are therefore fused before the final prediction stage.

2.4 Pretraining

The model is pretrained on BindingDB by treating SMILES-encoded epitope binding as a binary CPI prediction task.

  • Pretraining: BindingDB data are used to pretrain the model on SMILES-encoded epitope receptor-binding affinity.The database is binarized to reduce complexity and potential measurement-platform biases.

2.5 Data Splitting

Two 10-fold cross-validation schemes separately test generalization to unseen TCRs and to unseen TCRs and epitopes.

  • Data Splitting: 10-fold cross-validation evaluates generalization under two splitting strategies.The TCR split withholds TCRs while usually retaining seen epitopes; the strict split withholds both TCRs and epitopes.
  • TCR split: The TCR split assigns each TCR to one fold but distributes epitopes randomly across folds.Validation data therefore contain unseen TCRs but mostly previously observed epitopes.
  • Strict split: The strict split assigns each TCR and epitope to a single fold, preventing either from appearing in training and validation.Negative examples are generated by shuffling within each fold to preserve separation.

2.6 Model Training

Training uses binary cross-entropy with Adam, dropout, ReLU activations, and large-batch optimization; UMap visualizations illustrate how fold assignments structure the data.

  • Model Training: Models optimize binary cross-entropy with Adam at a learning rate of 0.0001.Convolutional and dense layers use dropout with p = 0.5 and ReLU activation.
  • Model Training: Training uses batches of 512 on POWER8 processors with a single NVIDIA Tesla P100.The learning rate is tuned on the VDJ dataset, while remaining hyperparameters are selected from prior experience.
  • Data Visualization: UMap projections show clear sample clusters whose fold coloring differs between the TCR and strict splits.In the strict split, samples within a cluster belong to the same fold, suggesting clusters correspond to distinct epitopes.

3 Results

On the TCR split, full TCR sequences and SMILES epitope encoding improve TITAN, although K-NN remains slightly stronger. Performance drops sharply on unseen epitopes, where attention analyses expose limited epitope adaptation but some moderate generalization.

  • 3.1 Performance on TCR Split: Full TCR sequence encoding boosts performance over using only the CDR3 loop.This indicates that regions outside CDR3 contain useful specificity information.
  • 3.1 Performance on TCR Split: SMILES epitope encoding further improves performance compared with amino acid encoding.SMILES represents short epitopes at atom-level resolution.
  • 3.1 Performance on TCR Split: 0.774 ± 0.006 mean ROC-AUC and 0.716 ± 0.005 mean balanced accuracy were achieved by the SMILES-plus-full-sequence model.The k = 13 K-NN baseline reached ROC-AUC 0.779 ± 0.007 and balanced accuracy 0.709 ± 0.008, slightly outperforming this TITAN setting on the TCR split.
  • 3.1 Performance on TCR Split: Pretraining on BindingDB and related augmentation improved TITAN performance, while the model retained high performance for TCRs most distant from training examples.The supplied results identify transfer learning and augmentation as useful for enriching the sparse interaction setting.
  • 3.2 Analysis of Attention Layers: 4.9 · 10^-10 inter-epitope and 2.5 · 10^-9 intra-epitope attention variance indicate nearly fixed attention positions across epitopes.By contrast, attention adapts across TCR sequences and contexts, suggesting that the amino-acid epitope model fails to learn meaningful epitope patterns.
  • 3.3 Performance on Strict Split: 0.62 ± 0.05 mean ROC-AUC was the best strict-split score, while all settings showed severe performance degradation on unseen epitopes.The pretrained semifrozen augmented model achieved this best score, but pretraining did not strongly improve unseen-epitope performance.
  • 3.3 Performance on Strict Split: Validation performance stagnated beyond 140 epochs on the strict split, unlike the steadily converging TCR split.The authors link this behavior to training and validation data being too distinct for proper generalization.
  • 3.4 Comparison to ImRex Model on Independent Test Set: All TITAN settings outperformed ImRex by a large margin on both independent test sets.The unseen-epitope test set produced ROC-AUC values clearly above chance, including 0.78 for the pretrained semifrozen augmented model, but contained only 1500 samples.

4 Discussion

TITAN is a generic bimodal neural network for TCR–epitope binding prediction, with performance gains from SMILES representations, augmentation, and pretraining. Its strongest generalization is to unseen TCRs, while unseen-epitope prediction remains challenging and may be limited by epitope sparsity.

  • TITAN is a generic bimodal, sequence-based neural network for predicting TCR–epitope binding probability.
  • SMILES-based epitope encoding, augmentation, and pretraining improve TITAN performance, with the best configuration reaching mean ROC-AUC 0.868 ± 0.005 on the TCR split.
  • The Levenshtein-distance K-NN baseline reaches mean ROC-AUC 0.779 ± 0.007 on the TCR split and outperforms complex neural networks from previous publications.
  • Pretrained and base TITAN models, along with K-NN, outperform ImRex by a large margin; the best TITAN model reaches ROC-AUC 0.87 on seen epitopes.
  • Unseen-epitope generalization remains difficult: TITAN reaches ROC-AUC 0.62 ± 0.05 on a strict split and 0.78 on an independent unseen-epitope test set.
  • Attention analysis suggests that sparse epitope sampling can lead TITAN to treat epitopes as classes rather than learn their molecular properties.
Loading 2105.03323v1…