PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning in Bioinformatics

Seonwoo Min, Byunghan Lee, Sungroh Yoon

arXiv:1603.06430v5cs.LGq-bio.GN

TL;DR

Transforming large quantities of biomedical data into valuable knowledge remains a major bioinformatics challenge. This review surveys deep-learning research across bioinformatics domains and architectures, discusses limitations and future directions, and reports improvements including a greater-than-25% F-measure increase in one comparison.

  • Problem

    Transforming large quantities of data into valuable knowledge is a major challenge in bioinformatics.

  • Method

    The review organizes bioinformatics research by input data, research objectives, bioinformatics domain, and deep-learning architecture while discussing limitations and promising directions.

  • Results

    >25% increase in F-measure compared to existing alternatives was achieved in one reviewed study.

  • Takeaways & Limitations

    Prudent preparation regarding the discussed issues is presented as important for applying deep learning in bioinformatics.

  • Takeaways & Limitations

    Deep learning is not a silver bullet and cannot provide great results in ad hoc bioinformatics applications.

Abstract

from arXiv · show

In the era of big data, transformation of biomedical big data into valuable knowledge has been one of the most important challenges in bioinformatics. Deep learning has advanced rapidly since the early 2000s and now demonstrates state-of-the-art performance in various fields. Accordingly, application of deep learning in bioinformatics to gain insight from data has been emphasized in both academia and industry. Here, we review deep learning in bioinformatics, presenting examples of current research. To provide a useful and comprehensive perspective, we categorize research both by the bioinformatics domain (i.e., omics, biomedical imaging, biomedical signal processing) and deep learning architecture (i.e., deep neural networks, convolutional neural networks, recurrent neural networks, emergent architectures) and present brief descriptions of each study. Additionally, we discuss theoretical and practical issues of deep learning in bioinformatics and suggest future research directions. We believe that this review will provide valuable insights and serve as a starting point for researchers to apply deep learning approaches in their bioinformatics studies.

Key Points

Deep learning is increasingly applied to bioinformatics to extract knowledge from biomedical big data. This review organizes research by bioinformatics domain and deep learning architecture, discusses practical and theoretical issues, and identifies future directions.

  • The review categorizes bioinformatics research by omics, biomedical imaging, biomedical signal processing, and four deep learning architecture groups.
  • The paper discusses challenges involving imbalanced data, interpretation, hyperparameter optimization, multimodal learning, and training acceleration.
  • The review aims to provide insight and serve as a starting point for applying deep learning approaches in bioinformatics studies.
  • Biomedical big data create a central challenge: transforming large quantities of data into valuable knowledge.
  • Deep learning has become increasingly important in bioinformatics as advances in data, computing, and training algorithms overcome earlier limitations.
  • It surveys applications including splice-junction discovery from DNA sequences, finger-joint recognition in X-ray images, and lapse detection from EEG signals.

Deep learning: a brief overview

Deep learning addresses the limitations of hand-designed features by learning hierarchical representations from data. The review organizes architectures, training principles, implementation considerations, and bioinformatics applications.

  • Hand-designed features make raw-data processing highly dependent on human expertise, motivating data-driven representation learning.Explicitly programming all task details is difficult for complex real-world problems.
  • Deep learning uses multiple nonlinear layers to discover increasingly abstract hierarchical representations by combining simpler learned features.Representation learning can jointly discover effective features and their mappings for given tasks.
  • The review categorizes architectures into DNNs, CNNs, RNNs, and emergent architectures, using DNNs specifically for multilayer networks.Examples of emergent architectures include DST-NNs, MD-RNNs, and CAEs.
  • Training optimizes layer weights through forward computation, loss evaluation, backpropagated gradients, and stochastic-gradient updates.Regularization methods such as weight decay and dropout are used to avoid overfitting and improve generalization.
  • Deep learning libraries differ in speed, pretrained models, extensibility, and distributed-computing support, with no clear overall front-runner.The comparison covers Caffe, TensorFlow, and Torch, while the text also discusses Neon and Theano.

Deep neural networks

DNNs transform inputs through successive nonlinear layers to learn hierarchical representations. Their training can use labeled data alone or combine unlabeled pre-training with supervised fine-tuning when labels are limited.

  • DNNs consist of an input layer, multiple hidden layers, and an output layer that progressively transform representations into more abstract forms.Each layer applies weighted sums and nonlinear activations such as sigmoid, tanh, or ReLU.
  • MLPs use purely supervised training and are typically applied when many labeled examples are available.Their optimization occurs in a high-dimensional parameter space.
  • SAEs and DBNs first perform layer-wise unsupervised pre-training on unlabeled data, then supervised fine-tuning with an output classifier.This two-phase procedure differs from MLP training.
  • Using unlabeled data, SAEs and DBNs can produce fairly regularized results when labeled data are insufficient.The passage identifies label scarcity as common in real-world settings.
  • DNNs are promising for bioinformatics because its data are complex and high-dimensional, and hierarchical learning may reveal abstract patterns and correlations.The review states that DNN capabilities have not yet been fully exploited.
  • Future DNN progress is expected to depend on encoding raw data appropriately and learning suitable features directly from them.

Convolutional neural networks

CNNs process spatially organized data through convolution, nonlinear transformation, and pooling. Their shared filters and subsampling support efficient extraction of increasingly complex local patterns across imaging and other bioinformatics data.

  • CNNs use convolution, nonlinear, and pooling layers to process especially two-dimensional images and build feature maps.
  • Convolution filters are repeatedly applied across data, reducing parameters while detecting patterns regardless of location.
  • Pooling subsamples non-overlapping regions, helping CNNs aggregate locally varying but semantically similar features into more complex representations.
  • CNNs are among the most successful deep learning architectures because of their capacity to analyze spatial information.
  • The review identifies potential CNN applications in genomic motifs, omics interactions, and time-frequency matrices of biomedical signals.
  • The authors expect CNN popularity and promise in bioinformatics applications to continue in the years ahead.

Recurrent neural networks

RNNs process sequential inputs while retaining past information through recurrent hidden states, enabling sequence-to-sequence and fixed-size predictions. They are relevant to omics and biomedical signals, although training historically faced long-term dependency problems.

  • RNNs can map variable-length input sequences to another sequence or to a fixed-size prediction.
  • RNNs use cyclic connections so hidden-state vectors retain past information when computing outputs for current inputs.
  • Bidirectional RNNs incorporate both past and future inputs when both affect the current output.
  • Vanishing gradients made learning long-term dependencies difficult, while LSTM and GRU memory cells significantly help prevent this problem.
  • RNNs are promising for bioinformatics because omics data and biomedical signals are typically sequential.
  • The review anticipates RNN use in biomedical text analysis and dynamic CT or MRI, with attention mechanisms expected to improve performance and information extraction.

Emergent architectures

The review introduces emergent architectures beyond DNNs, CNNs, and RNNs, focusing on DST-NNs, MD-RNNs, and CAEs and their bioinformatics applications.

  • Emergent architectures extend deep learning beyond DNNs, CNNs, and RNNs.The review identifies them as a distinct architecture group.
  • The review introduces DST-NNs, MD-RNNs, and CAEs as three emergent architectures.These architectures are presented together with applications in bioinformatics.
  • DST-NNs: DST-NNs progressively refine multi-dimensional outputs by combining persistent spatial features with gradually altered temporal features.Their hidden layers use local correlations through feature compositions.
  • MD-RNNs: MD-RNNs process non-sequential multi-dimensional data as groups of sequences and incorporate contexts from all possible directions.For two-dimensional data, processing order produces four contexts for each position.
  • CAEs: CAEs combine CNN-style convolution and pooling encoders with deconvolution and unpooling decoders for unsupervised reconstruction-based representation learning.This design aims to capture spatial information while retaining autoencoder regularization.
  • Applications: The review links DST-NNs to protein structure prediction, MD-RNNs to biomedical-image segmentation, and CAEs to recurring-pattern discovery in limited, imbalanced data.These examples match the architectures’ progressive refinement, contextual, and spatial representation capabilities.

Omics

The review surveys deep learning across omics tasks including protein structure prediction, gene-expression regulation, protein classification, and cancer anomaly classification. It describes DNN, CNN, RNN, and emergent-architecture applications to biological sequences and expression data.

  • Omics studies use genome, transcriptome, proteome, raw biological sequences, and gene-expression data as deep-learning inputs.Common sequence inputs include DNA, RNA, and amino acid sequences.
  • Protein structure prediction, gene-expression regulation, protein classification, and cancer anomaly classification are major omics research areas.Examples include secondary structure, splice junction, subcellular localization, and cancer classification tasks.
  • DNNs: DNNs have been applied to protein secondary structure, torsion-angle, accessible-surface-area, splice-junction, and expression-prediction problems.Studies used stacked autoencoders, deep belief networks, and multilayer perceptrons with sequence or expression inputs.
  • DNNs: Boosted contrastive divergence and sparsity regularization improved splice-junction prediction and detected subtle non-canonical splicing signals.The method targeted imbalanced data and sparse DNA sequences.
  • CNNs: CNNs learn local sequence motifs from data, deeper patterns and regulatory codes, and shared features for related multitask predictions.Applications included transcription-factor binding, DNA accessibility, chromatin-factor prediction, and disease-associated variant prioritization.
  • CNNs: The DeepSEA framework jointly predicts chromatin factors and prioritizes expression quantitative trait loci and disease-associated genetic variants.Its tasks include transcription-factor binding, DNase I sensitivity, and histone-mark profiles.
  • RNNs: LSTM-based RNNs achieved significantly improved accuracy relative to state-of-the-art approaches in microRNA identification and target prediction.RNNs were also applied to protein structure prediction and classification tasks.

Biomedical imaging

Biomedical imaging is reviewed across anomaly classification, segmentation, recognition, and brain decoding, with CNNs predominating because these tasks resemble general image analysis. DNNs and multidimensional RNNs are also applied to clinical and cellular images.

  • Biomedical imaging research covers anomaly classification, segmentation, recognition, and brain decoding.Examples address cancer, schizophrenia, cellular structures, brain tumors, and behavioral or emotional interpretation.
  • DNNs: DNNs have classified schizophrenia from brain MRIs, detected cell nuclei in histopathology images, and decoded viewed digits indirectly through functional MRI.These applications use deep belief networks or stacked autoencoders.
  • CNNs: CNNs have been applied to CT, histopathology, PET, and MRI data for anomaly classification, segmentation, and recognition.Reported tasks include metastasis, lymph-node and polyp classification, mitosis detection, chemotherapy-response prediction, and anatomical recognition.
  • CNNs: Cascaded CNNs combine local and global contextual features for brain-tumor segmentation from MRI.Other segmentation work targets cellular structures in microscopic images.
  • RNNs: MD-RNNs extend recurrent processing to three-dimensional images and have segmented neuronal structures in electron microscopy images and MRIs.This applies multidirectional context to volumetric biomedical data.

Biomedical signal processing

Biomedical signal-processing studies primarily address EEG-based brain decoding and disease-related anomaly classification. The review describes DNN, CNN, RNN, and CAE applications while emphasizing preprocessing for noisy, artifact-rich signals.

  • Biomedical signals include EEG, ECoG, ECG, EMG, and EOG recordings, with EEG receiving most study attention.Both raw signals and extracted features are used as inputs.
  • The review categorizes biomedical signal processing into EEG-based brain decoding and anomaly classification for disease diagnosis.The source describes these as two research groups.
  • Preprocessing: Because biomedical signals contain noise and artifacts, studies often decompose raw signals into wavelet or frequency components before deep learning.Some studies also use hand-designed features such as normalized decay and peak variation.
  • DNNs: DNNs have classified motor imagery, emotions, ECG beats, EEG waveform abnormalities, and Alzheimer’s disease from decomposed or raw signals.Reported models include deep belief networks and stacked autoencoders.
  • CNNs: CNNs analyze raw EEG with one-dimensional convolutions and can also process signal-derived two-dimensional patterns to predict seizures.Derived features such as phase-locking synchrony and wavelet coherence were encoded as pixel colors in one approach.
  • RNNs: RNNs have been used with raw EEG, wavelet features, and EEG log-power spectra to predict seizures and detect lapses.Applications include perceptron RNNs and LSTM RNNs.
  • CAEs: CAEs have supported brain-decoding tasks including finger flex-and-extend classification and musical-rhythm classification.These studies used raw ECoG or EEG signals.

Discussion

The discussion identifies limited and imbalanced data, interpretability, and architecture selection as major practical challenges for deep learning in bioinformatics. It reviews preprocessing, cost-sensitive and algorithmic strategies, visualization methods, and multimodal architectures as responses to these challenges.

  • Limited and imbalanced data: Bioinformatics datasets are often limited and imbalanced because acquisition is expensive, class distributions are unequal, and sensitive treatment-group data may be unavailable.These conditions can make standard accuracy misleading and complicate model training.
  • Limited and imbalanced data: Solutions include data preprocessing, cost-sensitive learning, and algorithmic modification through sampling, loss adjustments, pre-training, and transfer learning.Examples include undersampling, synthetic minority oversampling, spatial image deformations, class-sensitive objectives, and fine-tuning models pretrained on related data.
  • Limited and imbalanced data: >25% increase in F-measure was achieved by a microRNA target prediction method using unsupervised pre-training with an RNN-based autoencoder compared with existing alternatives.The example illustrates how pre-training can address limited-data settings.
  • Interpretability: Interpretability is important in biomedical applications because clinicians need logical reasoning, but converting deep learning from a black box into a white box remains in its early stages.Visualization, motif extraction, mutation maps, deconvolutional networks, attention mechanisms, and mathematical analyses are discussed as interpretability approaches.
  • Architecture selection: Architecture selection remains challenging because the appropriate model depends on input characteristics and research objectives, while the advantages of DNNs, CNNs, and RNNs are only roughly understood.The review associates DNNs with high-dimensional correlations, CNNs with spatial information, and RNNs with sequential information.
  • Future directions: Multimodal deep learning is presented as a promising direction because it combines information from multiple input sources relevant to bioinformatics.The review also identifies specialized hardware and incorporation of traditional architectures as future-oriented developments.

Conclusion

The conclusion characterizes deep learning as promising for bioinformatics while emphasizing that it is not a universal solution. The review synthesizes applications, discusses unresolved challenges, and proposes preparation around data, interpretation, architecture, multimodality, and acceleration for future work.

  • Review scope: The review examines bioinformatics research using deep learning according to input data, research objectives, and established deep learning architectures.It is intended to provide insight and support future applications.
  • Scope and challenges: Deep learning is promising but cannot provide great results in ad hoc bioinformatics applications.The conclusion frames this as a boundary on the approach rather than a universal solution.
  • Future directions: Future applications must address limited or imbalanced data, interpretation of deep learning results, and selection of an appropriate architecture and hyperparameters.The conclusion also identifies multimodality and acceleration as requiring further study.
  • Contribution: The review is intended to provide valuable insight and serve as a starting point for applying deep learning to advance bioinformatics research.The authors present prudent preparation around the discussed issues as important for future approaches.

Figure captions

The figures introduce deep-learning architectures and show how they are applied across bioinformatics domains. They also depict representation learning, recurrent computation, spatial processing, and unsupervised layer-wise pre-training.

  • Bioinformatics applications: Deep learning applications span omics, biomedical imaging, and biomedical signal processing, including splice-junction, finger-joint, and EEG lapse detection examples.The examples use deep neural, convolutional, and recurrent neural networks, respectively.
  • Deep neural networks: DNNs comprise an input layer, multiple hidden layers, and an output layer that computes values through weighted sums and nonlinear functions.The hidden layers produce hierarchical representations from input data.
  • Unsupervised pre-training: SAE and DBN pre-training stacks layers sequentially, training each weight matrix with an RBM or AE before repeating for the desired depth.The process begins with W1 between input units and the first hidden layer, then uses learned representations to train subsequent layers.
  • Convolutional architectures: CNNs apply learned filters to obtain filter maps and use pooling to combine them, while CAEs encode and decode data while integrating spatial information.The CAE structure uses convolution and pooling in the encoder and deconvolution and unpooling in the decoder.
  • Recurrent architectures: RNNs propagate information across time by combining current inputs with previous hidden states, whereas BRNNs incorporate both past and future information.The recurrent computation can be represented by unrolling the network in time.
  • Emergent architectures: DST-NNs use multiple contextual hidden layers and progressive refinement to incorporate local correlations in multidimensional data.Four contextual groups propagate information to compute each output-layer hidden unit.
Loading 1603.06430v5…