Source-linked AI summary
Technical Note on Transcription Factor Motif Discovery from Importance Scores (TF-MoDISco) version 0.5.6.5
Avanti Shrikumar, Katherine Tian, Žiga Avsec, Anna Shcherbina, Abhimanyu Banerjee, Mahfuza Sharmin, Surag Nair, Anshul Kundaje
TL;DR
Interpreting genomic deep-learning models is difficult because existing pattern-learning approaches do not leverage base-level importance scores. TF-MoDISco clusters influential sequence segments, including hypothetical-base contributions, into task-patterned seqlets and metaclusters, with the authors reporting clean and distinct metaclusters relative to alternative strategies.
Problem
Existing genomic deep-learning interpretation methods assign base-level importance, while pattern-learning methods generally do not exploit these scores; distributed CNN representations make individual filters difficult to interpret.
Method
TF-MoDISco clusters high-contribution sequence segments into seqlets and metaclusters using per-base importance scores, hypothetical importance, and cross-task contribution patterns.
Results
The authors found that TF-MoDISco metaclustering produced relatively clean and distinct patterns compared with k-means clustering or community detection.
Takeaways & Limitations
Importance scores computed from all network neurons can support identification of consolidated motifs learned by genomic deep-learning models.
Takeaways & Limitations
The continuous-Jaccard approach may be unreliable when results are highly sensitive to score magnitudes or when outliers disproportionately affect mean normalization.
Abstract
from arXiv · showhide
TF-MoDISco (Transcription Factor Motif Discovery from Importance Scores) is an algorithm for identifying motifs from basepair-level importance scores computed on genomic sequence data. This technical note focuses on version v0.5.6.5. The implementation is available at https://github.com/kundajelab/tfmodisco/tree/v0.5.6.5
1 Introduction
TF-MoDISco addresses the difficulty of interpreting genomic deep-learning models by clustering base-level importance-score segments into consolidated motifs. Its importance scores integrate information from all network neurons, rather than relying on individual filters.
- The biological roles of flanking nucleotides and combinatorial interactions among transcription factors remain largely uncharted.These factors influence in vivo transcription-factor binding at non-coding regulatory elements.
- TF-MoDISco identifies high-quality, consolidated motifs from deep-learning importance scores.It clusters segments with high input importance to recover motifs learned by the model.
- Deep-learning interpretation methods assign importance scores to individual bases, but pattern-learning methods generally do not exploit those scores.Existing approaches are largely limited to visualizing individual CNN-filter representations.
- Individual convolutional neurons may be uninformative because CNNs learn highly distributed representations.
2 Input to TF-MoDISco
TF-MoDISco uses per-base importance scores for each prediction task and can augment them with hypothetical scores for unobserved bases. These scores represent how bases influence model outputs and support motif discovery from sequence inputs.
- TF-MoDISco takes per-base importance scores for every prediction task as input.Positive, negative, and near-zero values indicate support for positive predictions, support for negative predictions, and little task relevance, respectively.
- Hypothetical importance asks how the importance of a newly introduced base would change after mutating the sequence at a position.For gradient × input, gradients approximate contributions of absent bases when mutations do not greatly change the gradient.
- DeepLIFT hypothetical contributions reuse multipliers from the current input while substituting the hypothetical input’s difference-from-reference.When the reference is zero and the multiplier is the gradient, this reduces to gradient × input.
- With one-hot DNA inputs and a nonzero reference, contributions from all bases at a position must be projected onto the actually present base.
- GkmExplain scores should be normalized so hypothetical importance across ACGT bases sums to zero at each position before TF-MoDISco use.
3 TF-MoDISco, Phase 1: Metaclusters
Phase 1 identifies influential sequence segments, called seqlets, and groups them into metaclusters with distinct task-contribution patterns. It uses thresholded importance scores, cross-task normalization, and activity-pattern compatibility to form these groups.
- 3 TF-MoDISco, Phase 1: Metaclusters: Phase 1 identifies seqlets with substantial contribution to at least one task and clusters them into metaclusters by task-contribution pattern.Multi-task processing is optional; poor multi-task results motivate running TF-MoDISco one task at a time.
- 3.1 Seqlet identification: Overlapping task-specific seqlets are unionized, retaining the seqlet with the higher task-specific score when overlap exceeds the default 50%.Flank expansion later in the algorithm is intended to reduce information loss from discarding overlapping seqlets.
- 3.2 Metaclustering: Cross-task contribution scores are converted to signed percentiles so task scores on different scales can be compared during metaclustering.Extreme positive and negative scores map to +1.0 and -1.0, respectively, under the default absolute-value percentile transformation.
- 3.2 Metaclustering: Seqlets are assigned to the most specific compatible activity pattern, with ties resolved using the higher absolute transformed score.Strong and weak compatibility use separate thresholds, while zeros impose no constraint.
- 3.2 Metaclustering: The authors found that this metaclustering strategy produced relatively clean and distinct patterns compared with k-means clustering or community detection.
4 TF-MoDISco, Phase 2: Clustering Within a Metacluster
Phase 2 clusters seqlets within each metacluster using task-relevant importance information and a two-stage affinity computation. Fine-grained matching uses Continuous Jaccard similarity, which is presented as preferable to correlation for noisy seqlet scores.
- Affinity computation: Phase 2 subclusters seqlets using importance scores from tasks relevant to each metacluster.A task is relevant when its activity pattern is nonzero for that metacluster.
- Affinity computation: O(N^2) pairwise affinity computation is accelerated by first finding coarse-grained nearest neighbors, then applying a slower metric to the default 500 neighbors per seqlet.The coarse stage determines which seqlet pairs receive fine-grained affinity calculations.
- Coarse-grained affinities: Coarse affinities use gapped k-mer embeddings and cosine similarity, taking the maximum over forward and reverse-complement seqlets unless revcomp is disabled.This sequence-based affinity procedure is specific to DNA clustering.
- Fine-grained affinities: Continuous Jaccard similarity is preferred to correlation because it matches the pattern agreeing at more positions, whereas correlation can favor magnitude-dominated matches and be distorted by outliers.In the toy example, correlation gives 0.98 for a vector agreeing at one element versus 0.87 for one agreeing at six elements; Continuous Jaccard gives 4/11 and 7/11, respectively.
- Fine-grained affinities: Fine-grained affinities combine normalized actual and hypothetical importance tracks across relevant tasks and compare aligned overlaps with Continuous Jaccard similarity.The combined matrices include both importance types for every relevant task and are compared across valid alignments.
4.2 Clustering the Affinity Matrix
TF-MoDISco adapts affinity-based clustering to local data density by converting similarities into conditional probabilities and distances, then applying community detection after filtering noisy seqlets.
- Adapting distance to local density: Affinity clustering first adapts distance to the local density of seqlets, because similarity thresholds suitable for strong motifs may not suit weak motifs.Dense neighborhoods permit stricter notions of closeness, whereas sparse neighborhoods require more lenient ones.
- Noise filtering and clustering: Seqlets with coarse- versus fine-grained affinity Spearman correlation below 0.15 are discarded as noise before clustering.The remaining seqlets are clustered into an initial motif set.
- Adapting distance to local density: Conditional probabilities p_j|i represent the likelihood of selecting seqlet s_j given that seqlet s_i was selected.The distribution is tuned to a target perplexity, which approximates the neighborhood size receiving substantial probability.
- Adapting distance to local density: Perplexity 10 was used because it proved effective at extracting small clusters.A neighborhood where k nearest seqlets each have probability 1/k has perplexity k.
- Similarity-to-distance mapping: The similarity-to-distance mapping outperformed y = 1 − x because similarities near zero map to distances tending toward infinity.In practice, the lowest similarity between seqlets tends to be around zero rather than −1.
- Community detection: TF-MoDISco clusters directly in the higher-dimensional similarity space rather than computing a lower-dimensional t-SNE embedding.The t-SNE-inspired procedure is used to transform affinities for clustering, not to generate an embedding.
- Community detection: The conditional-probability transformation symmetrizes affinities into joint probabilities before community detection.TF-MoDISco supports Louvain and Leiden clustering through the pipeline’s community-detection stage.
Clustering using Louvain
The Louvain workflow addresses instability by consensus clustering, while version 0.5.6 adds Leiden as an alternative selected through repeated seeded runs and modularity.
- Louvain: Louvain scales well and does not require pre-specifying the number of clusters, but its community assignments are nondeterministic.Direct application to the joint probabilities produced somewhat unstable results.
- Louvain: Consensus clustering runs Louvain 200 times by default and replaces affinities with the fraction of runs placing seqlet pairs in the same top-level cluster.Louvain is then rerun on this consensus affinity matrix, using the lowest-level communities as clusters.
- Leiden: Version 0.5.6 adds Leiden because Louvain can produce arbitrarily badly-connected communities.The authors observed that the Louvain consensus procedure did not work well with Leiden.
- Leiden: Leiden clustering uses 50 random seeds and selects the clustering with the best modularity, iterating each seed until modularity no longer improves.Unlike the Louvain workflow, this Leiden pipeline does not include consensus clustering.
4.3 Seqlet Aggregation
TF-MoDISco aggregates each seqlet cluster into a motif by greedily aligning high-importance seqlets and averaging their aligned data tracks.
- Seqlet aggregation: Seqlets within each cluster are sorted by descending total importance magnitude across relevant tasks before aggregation.This ordering prioritizes seqlets with the largest combined importance contributions.
- Seqlet aggregation: Seqlets are merged greedily at the alignment maximizing Continuous Jaccard similarity among permitted overlaps.An alignment is considered where at least min_overlap_while_sliding of the seqlet overlaps the current motif.
- Seqlet aggregation: The aggregated motif is updated by averaging each data track at every aligned position over the merged seqlets.The procedure therefore combines aligned contribution information into a cluster-level motif representation.
4.4 Motif boundary editing
Motif boundaries are edited by retaining well-supported positions, expanding aligned seqlets to fill the motif, and selecting a uniform high-importance window.
- Boundary editing: Motif boundaries are edited in three steps after seqlet aggregation.The steps first trim by support, then expand seqlet ends, and finally enforce a uniform window size.
- Support-based trimming: Positions are initially retained when their aligned seqlet-center count reaches 20% of the maximum by default.This uses frac_support_to_trim_to, whose default value is 0.2.
- Boundary expansion: After trimming, the remaining seqlet ends are expanded to fill the motif boundaries, giving every position information from all aligned seqlets.The expansion is intended to improve the quality of the aggregated motif.
- Uniform window sizing: Each motif is standardized to the 30 bp window with the highest combined real and hypothetical importance by default.Seqlets within that window are expanded to match the resulting motif boundaries.
4.5 Discard motifs that disagree with metacluster activity pattern
After motif boundary editing, TF-MoDISco filters motifs whose aggregated contribution patterns disagree with their metacluster activity patterns, often because seqlet centers fall in noisy flanks.
- TF-MoDISco filters motifs whose aggregated contribution score patterns disagree with their metacluster activity patterns.Such disagreement can arise when a seqlet center lies in a noisy flank whose importance-score sign differs from the motif’s.
4.6 Repeat Seqlet Clustering
TF-MoDISco performs a second clustering round after editing and expanding seqlet boundaries, because newly included informative positions can improve clustering quality and stability.
- TF-MoDISco subjects seqlets from processed motifs to a second clustering round after editing and expanding their boundaries.The expanded seqlets can incorporate additional informative positions.
- The default workflow performs one additional clustering round, although more rounds are possible.
5 TF-MoDISco Phase 3: Post-processing of clusters
Phase 3 removes spurious motif merges, collapses genuinely similar clusters, reallocates seqlets from undersized motifs, and expands retained motifs to expose flanking patterns.
- 5.1 Identifying spurious merging: Louvain diclustering recursively checks motifs for spurious merges by splitting them into at most two clusters and selecting the highest-modularity result across random seeds.The process continues until no modularity improvement is observed over 20 consecutive random seeds.
- 5.1 Identifying spurious merging: Motifs with more than 30 seqlets undergo spurious-merging checks using Continuous Jaccard similarity, recursive diclustering, and a Pearson-correlation threshold of 0.8.Sub-motifs below the correlation threshold are recursively examined; otherwise, recursion terminates.
- 5.2 Merging similar motifs: Final motif collapsing combines cross-correlation over overlapping alignments with density-sensitive similarity to merge highly similar but non-transitively related motifs.Cross-correlation is evaluated for overlaps of at least 0.7, while density-sensitive similarity accounts for cluster packing.
- 5.3 Disbanding small motifs: Motifs with fewer than 30 seqlets are disbanded, and their seqlets are reassigned to remaining motifs only when Continuous Jaccard similarity exceeds 0.2.
- 5.4 Expanding motifs: Retained motifs are expanded by 10 bp on each side by default, while uninformative flanks may be trimmed.
6 (New in v0.5.6) Leveraging MEME to initialize the TF-MoDISco clusters
Version 0.5.6 adds MEME-based cluster initialization, using PWM matches among seqlet sequences to seed clustering and improve motif-focused similarity computations.
- Version 0.5.6 can initialize TF-MoDISco clusters by running MEME on seqlet sequences and assigning matching seqlets to their strongest PWM cluster.Seqlets without PWM matches form their own clusters, and matches are determined by maximum alignment log-odds exceeding MEME’s Bayes-optimal threshold.
- MEME initialization expands fine-grained similarity searches to nearest neighbors within each initialized cluster, in addition to globally nearest neighbors.
- MEME-based clustering initializes Leiden community detection, but this workflow is not supported with Louvain.
- Applying motif discovery to seqlet sequences can improve sensitivity by focusing on regions likely to contain motifs rather than full input sequences.