PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning in Mining Biological Data

Mufti Mahmud, M Shamim Kaiser, Amir Hussain

arXiv:2003.00108v1q-bio.QMcs.LGstat.ML

TL;DR

Biological big data are multimodal, multidimensional, and complex, making pattern analysis challenging and motivating sophisticated machine-learning tools. The review surveys deep-learning concepts and applications, open-access data sources, open-source tools, and tool performance, while identifying research challenges and future perspectives. It concludes that such learning techniques have been used to decipher complex patterns and variable interactions in biological data.

  • Problem

    Biological big data are multimodal, multidimensional, and complex, creating a major challenge for analyzing patterns and interactions.

  • Method

    The review surveys DL, RL, and deep RL concepts, biological applications, open-access data sources, open-source tools, and comparative tool performance.

  • Results

    The review reports that learning techniques have trained machines to understand and decipher complex patterns and interactions in biological data.

  • Takeaways & Limitations

    The paper provides a consolidated overview of biological-data applications, accessible datasets, open-source tools, benchmark findings, and future research directions.

  • Takeaways & Limitations

    Biological data reuse is often hampered by non-unified reporting standards, non-uniform information, and challenges in provenance, curation, and annotation.

Abstract

from arXiv · show

Recent technological advancements in data acquisition tools allowed life scientists to acquire multimodal data from different biological application domains. Broadly categorized in three types (i.e., sequences, images, and signals), these data are huge in amount and complex in nature. Mining such an enormous amount of data for pattern recognition is a big challenge and requires sophisticated data-intensive machine learning techniques. Artificial neural network-based learning systems are well known for their pattern recognition capabilities and lately their deep architectures - known as deep learning (DL) - have been successfully applied to solve many complex pattern recognition problems. Highlighting the role of DL in recognizing patterns in biological data, this article provides - applications of DL to biological sequences, images, and signals data; overview of open access sources of these data; description of open source DL tools applicable on these data; and comparison of these tools from qualitative and quantitative perspectives. At the end, it outlines some open research challenges in mining biological data and puts forward a number of possible future perspectives.

Introduction

Biological data are increasingly large, heterogeneous, dynamic, and complex, creating major challenges for pattern recognition and meaningful analysis. The review frames deep learning and related machine-learning methods across biological application domains and surveys available tools and data sources.

  • Technological advances generate biological data across DNA sequencing, gene expression, bioimaging, neuroimaging, and brain-machine interfaces.
  • Biological big data are hierarchical, heterogeneous, dynamic, and complex, making pattern identification and meaningful inference difficult.
  • Supervised learning classifies data using known annotations, whereas unsupervised learning forms groups by identifying similarity.
  • Reinforcement learning learns through interaction with an environment but is outside this review’s scope.
  • The review organizes biological data into omics, bioimaging, medical imaging, and brain/body-machine-interface domains addressed by diverse machine-learning methods.
  • Its stated contributions include coverage of DL, RL, and deep RL concepts, biological applications, open-source libraries and frameworks, open-access datasets, and open research issues.

1 Overview of Deep Learning

Deep learning learns increasingly abstract representations from raw biological data and uses several architectures to compress, model, classify, or detect patterns in complex inputs.

  • Deep learning learns hierarchical representations by defining increasingly abstract features from less abstract lower-level representations.
  • Autoencoder: Autoencoders compress high-dimensional inputs into lower-dimensional hidden representations and minimize reconstruction error during training.Deep autoencoders stack multiple autoencoders, using one layer’s hidden activations as the next layer’s inputs and greedy layer-wise pretraining to address poor local minima.
  • Restricted Boltzmann Machine and Deep Belief Network: Restricted Boltzmann Machines model probability distributions with symmetric visible–hidden connections, while Deep Belief Networks stack RBMs into hybrid generative architectures.RBMs are trained by maximizing observation log-likelihood with contrastive divergence; DBN layers are greedily pretrained and may receive generative fine tuning.
  • Convolutional Neural Network: Convolutional neural networks use convolutional and pooling layers before fully connected layers to learn from locally correlated multidimensional inputs.Weight sharing reduces parameters and pooling reduces feature maps, computation, and sensitivity to small spatial shifts.
  • Recurrent Neural Network: Recurrent neural networks detect sequences by computing each state from the current input and previous state outputs.Their memory-like property supports streaming data applications, although backpropagation through time can cause vanishing or exploding gradients.

2 Deep Learning and Biological Data

The review surveys deep-learning applications to biological data, emphasizing open-access studies across sequences, images, and signals. It also organizes these applications in a summary table.

  • 2 Deep Learning and Biological Data: Open-access studies apply diverse deep-learning architectures and parameter settings to pattern analysis in biological data.The review reports these studies in Table 1.
  • Sequences: For biological sequences, stacked denoising autoencoders support cancer diagnosis, classification, and related-gene identification from gene-expression data.
  • Sequences: Deep belief networks and active learning are used for DNA splicing-junction prediction, while LSTM models predict miRNA precursors and their targets.
  • Images: CNNs support mitosis detection, neuronal-structure segmentation, brain-tumor segmentation, and Alzheimer’s diagnosis from histology and MRI data.
  • Signals: EEG applications include emotion detection, motor-imagery decoding, and classification using PCA, covariate-shift adaptation, stacked denoising autoencoders, DBNs, and CNNs.

3.1 Omics

The review catalogs open-access omics resources spanning genome, sequence, expression, protein, pathway, and disease-related data. These databases support biological interpretation, drug design, and genetic-disease research.

  • Omics Databases/Datasets: SGD provides comprehensive Saccharomyces cerevisiae information and tools for studying genome expression, transcriptomes, and computational biology.
  • Omics Databases/Datasets: PubChem contains millions of compound structures and activity datasets, with web and FTP access plus plotting and clustering services for omics and drug design.
  • Omics Databases/Datasets: ENCODE supplies human and mouse whole-genome datasets with associated metadata.
  • Omics Databases/Datasets: Additional resources cover protein structures, gene expression, cancer datasets, RNA-binding proteins, transcription-factor binding, miRNAs, and mutations associated with genetic diseases.Examples include PDB, GEMS, CPD, iONMF, JASPAR, and IGDD.
  • Omics Databases/Datasets: INSDC integrates DNA sequence resources from Japan, Europe, and GenBank, covering raw reads through functional annotation with contextual sample and experiment information.

3.2 Imaging

The imaging resources cover cellular, biomedical, and neuroimaging data, with applications ranging from image analysis and anomaly detection to segmentation and disease diagnosis. The catalog includes both general imaging repositories and specialized brain datasets.

  • Imaging Databases/Datasets: The listed imaging resources include cellular, biomedical, medical, and neuroimaging datasets spanning microscopy, MRI, CT, PET, SPECT, MEG, EEG, and optical imaging.
  • Imaging Databases/Datasets: Cell-focused resources provide image datasets, libraries, viewers, APIs, and analysis tools for studying cell biology and high-resolution 2D, 3D, and 4D microscopy data.
  • Imaging Databases/Datasets: MITOS supports breast-cancer mitosis detection and nuclear-atypia evaluation, while IBSR provides MRI segmentation results for developing segmentation methods.
  • Imaging Databases/Datasets: Neuroimaging resources support brain-structure and function studies, disease identification, diagnostic classification, surgical planning, and Alzheimer’s research.
  • Imaging Databases/Datasets: TCIA contains CT, MRI, and nuclear-medicine images for clinical diagnosis, biomarker research, and cross-disciplinary investigation.

3.3 [Brain/Body]-Machine Interfaces (BMI)

The BMI resources provide multimodal neural, muscular, cardiac, and physiological recordings for brain–machine-interface, affective-state, anomaly-detection, and biomedical signal research. They range from benchmark collections to specialized datasets with annotations and algorithms.

  • [Brain/Body]-Machine Interfaces (BMI): BCI Competition datasets span EEG, ECoG, and MEG recordings for signal processing, classification, and brain–machine-interface research.
  • [Brain/Body]-Machine Interfaces (BMI): Physiological-signal resources support affective-state analysis using EEG, sEMG, ECG, MEG, eye movements, facial video, and other multimodal recordings.
  • [Brain/Body]-Machine Interfaces (BMI): NinaPro provides kinematic and sEMG data from 27 subjects performing finger, hand, and wrist movements for biorobotics research.
  • [Brain/Body]-Machine Interfaces (BMI): PhysioNet includes neuroelectric, myoelectric, waveform, and multiparameter databases for epilepsy, arrhythmia, seizure detection, and BMI instrumentation.
  • [Brain/Body]-Machine Interfaces (BMI): Other repositories provide ERP, ERD, seizure, brain-mapping, EEG, EOG, and ECoG datasets for BMI-device modeling, anomaly detection, and related analyses.

4 Open Source Deep Learning Tools

The review catalogs open-source deep learning libraries, frameworks, and platforms, comparing their architectures, languages, interfaces, and deployment characteristics. These tools support diverse model types and computing environments, with interoperability highlighted as an important practical feature.

  • Tool overview: The reviewed open-source tools provide maintained implementations of multiple deep learning algorithms and are intended to facilitate community development.The review presents a comprehensive list while omitting individual tool publication references in favor of their websites.
  • Representative tools: Caffe targets computer vision and is described as the fastest CNN implementation, while Chainer supports dynamic graph creation and CNN, RNN, and DA architectures.Caffe provides C++ with Python and Matlab bindings; Chainer is a Python library with dynamic graph creation.
  • Representative tools: DL4J targets distributed quick prototyping on JVM and processing frameworks, whereas DyNet dynamically creates graphs for each training example and specializes in NLP-oriented CNN, RNN, and LSTM models.DL4J also supports Keras model imports through a Python API; DyNet provides C++ with Python bindings.
  • Representative tools: Keras is a user-friendly Python library for rapid experimentation built on Theano or TensorFlow, while TensorFlow supports large-scale networks, dynamic graphs, visualization, and multiple device types.Keras supports CNN, RNN, and DBN; TensorFlow supports CNN, RNN, LSTM, and RBMs.
  • Representative tools: Other reviewed platforms extend deployment or programming flexibility, including MXNet across cloud-to-embedded devices, Veles across heterogeneous platforms, and H2O across several frameworks and languages.The review also covers Lasagne, MCT, Neon, PyTorch, TF.Learn, Theano, and Torch.

5 Relative Comparison of DL Tools

The review compares deep learning tools by usage trends, community participation, interoperability, and hardware scalability. TensorFlow and Keras lead several popularity and community indicators, while Keras and DL4J provide notable interoperability options.

  • Comparison criteria: The comparison evaluates tool usage trends, community participation, interoperability, and scalability.These four measures define the relative comparison among available open-source deep learning tools.
  • Usage trend: TensorFlow searches increased progressively after release, followed by Keras; arXiv mentions identified TensorFlow, PyTorch, Caffe, Keras, Torch, and Theano as the top six tools.The search analysis covered July 2015 to June 2017, while the arXiv analysis examined around 2,000 papers submitted in March 2017.
  • Community participation: Community effort was concentrated on TensorFlow, followed by Keras and Caffe, using normalized GitHub forks and contributor counts alongside stars.Bubble size represents the normalized combined effect of forks and contributors, with release year also indicated.
  • Interoperability: Keras is described as the most flexible tool for interoperability because it runs on TensorFlow or Theano, while DL4J imports models configured and trained using Keras.DL4J provides abstraction layers over TensorFlow, Theano, Caffe, and CNTK backends.
  • Scalability: Hardware acceleration, energy efficiency, and higher performance motivate the development of chipset-based deep learning systems beyond software-only use.The review identifies ASICs and FPGAs as chipsets expected to gain an expanded role alongside GPUs and CPUs.

6 Performance of Tools and Benchmark

The benchmark compares training and forward performance across deep learning architectures, tools, CPUs, and GPUs. Results show architecture- and platform-dependent differences: Theano and Torch lead stacked autoencoder timings, MCT leads LSTM, and GPUs substantially accelerate CNN training.

  • Benchmark design: The benchmark measures training time for FCN, CNN, RNN, and DA architectures across popular tools on multicore CPU and GPU platforms.The compared tools include Caffe, CNTK, MXNet, Theano, TensorFlow, and Torch.
  • Benchmark design: The benchmark uses three Intel Xeon CPU setups with GPUs including GTX Titan X, GTX 980, GTX 1080, Tesla K80, M40, and P100.The experimental setups differ primarily in GPU hardware despite using similar CPU configurations.
  • DA benchmark: Stacked autoencoders were evaluated with three tied-weight hidden layers of 400, 200, and 100 units using sigmoid activations and two-step MNIST training.The evaluation used experimental setup 1 and assessed forward runtime and training time.
  • DA benchmark: Theano and Torch outperform TensorFlow in both gradient and forward times for stacked autoencoders, regardless of CPU thread count or GPU.Forward runtime measures information evaluation through the network, whereas gradient computation time measures training.
  • LSTM benchmark: MCT outperforms other tools for the evaluated LSTM on CPU and all three GPU platforms, while TensorFlow and MXNet perform similarly on GPUs.The LSTM used 10,000 input and output units, two layers, approximately 13 million parameters, and input length 32.
  • CNN benchmark: GPU training substantially accelerates CNNs over CPUs, but speedup varies by GPU, tool, and network; Tesla P100 performs best among the tested accelerators.Across the CNN comparison, TensorFlow is fastest on CPUs, while Caffe often has the best GPU-over-CPU speedup and Torch slightly outperforms TensorFlow in most networks.

7 Open Issues and Future Perspectives

The review identifies methodological, data, infrastructure, and benchmarking gaps that constrain deep learning for biological data and proposes directions to address them. It also highlights deep reinforcement learning as an unexplored opportunity for biological pattern recognition.

  • Methodological limitations: DL methods require large datasets and can fail when biological relationships are causal or too complex for geometric transformations.The review also notes difficulties interpreting high-level outcomes and susceptibility to misclassification and over-classification.
  • Data limitations: Open biological repositories are difficult to reuse because reporting standards and information are non-uniform, while provenance, curation, and annotation remain challenging.
  • Computational infrastructure: Distributed and cloud-based DL remains underused, with repeated retraining, privacy and security concerns, and underdeveloped real-time processing limiting deployment.
  • Future DL directions: Future DL foundations should improve generalization, quantitative assessment, computational-complexity analysis, data-driven self-organization, and visualization.
  • Deep reinforcement learning: Deep RL has not yet been applied to biological pattern recognition, although it could address dynamically changing, multilevel data and reduce reliance on large labeled datasets.
  • Future perspectives: The field needs standardized biological data practices, secure distributed infrastructure, and biological benchmarks covering additional tools and architectures.Existing benchmarks are mostly based on non-biological data, and their scalability to biological data is not well established.

8 Conclusion

The article surveys deep learning applications to complex biological big data and provides resources for applying and evaluating these methods. It covers applications, open-access repositories, open-source tools, and tool comparisons.

  • Biological big data are multimodal, multidimensional, and complex, creating a need for sophisticated machine-learning tools for pattern analysis.
  • The article surveys deep-learning applications, relevant open-access repositories, and open-source tools and frameworks for biological data.
  • The review compares deep-learning tools for popularity and performance and concludes by outlining open challenges and future perspectives.
Loading 2003.00108v1…