Source-linked AI summary
Efficient Machine Learning for Big Data: A Review
O. Y. Al-Jarrah, P. D. Yoo, S Muhaidat, G. K. Karagiannidis, K. Taha
TL;DR
The paper addresses the mismatch between rapidly growing data and machine-learning systems whose computational demands can rise sharply, creating sustainability concerns. It reviews sustainable data-modeling research and discusses algorithmic strategies for reducing complexity while maintaining or improving predictive performance and stability. The review concludes that sustainable modeling offers a way to handle large quantities of e-science data and may extend across e-science areas.
Problem
Large datasets expose efficiency and scalability limits in current machine-learning systems, while nonparametric methods can require high computational cost and produce exponentially increasing complexity.
Method
The paper reviews theoretical, empirical, and experimental sustainable machine-learning literature and examines data-intensive structures, model efficiency, and lower-cost algorithmic approaches.
Results
The review identifies local and semiparametric strategies as reducing model complexity, while sustainable data modeling is presented as a way to handle large quantities of e-science data.
Takeaways & Limitations
Sustainable data modeling provides a research direction for combining computational efficiency with predictive performance across large-scale data-intensive fields.
Abstract
from arXiv · showhide
With the emerging technologies and all associated devices, it is predicted that massive amount of data will be created in the next few years, in fact, as much as 90% of current data were created in the last couple of years,a trend that will continue for the foreseeable future. Sustainable computing studies the process by which computer engineer/scientist designs computers and associated subsystems efficiently and effectively with minimal impact on the environment. However, current intelligent machine-learning systems are performance driven, the focus is on the predictive/classification accuracy, based on known properties learned from the training samples. For instance, most machine-learning-based nonparametric models are known to require high computational cost in order to find the global optima. With the learning task in a large dataset, the number of hidden nodes within the network will therefore increase significantly, which eventually leads to an exponential rise in computational complexity. This paper thus reviews the theoretical and experimental data-modeling literature, in large-scale data-intensive fields, relating to: (1) model efficiency, including computational requirements in learning, and data-intensive areas structure and design, and introduces (2) new algorithmic approaches with the least memory requirements and processing to minimize computational cost, while maintaining/improving its predictive/classification accuracy and stability.
1. Introduction
The paper frames sustainable machine learning as a response to rapidly growing data and computational demands. It reviews energy-efficient modeling research and proposes a roadmap for reducing computational requirements while preserving predictive performance.
- 1. Introduction: ICT infrastructure is expanding rapidly, increasing electricity use and environmental pressure from data-center computing.The number of servers in data centers increased sixfold to 30 million over the last decade, while aggregate server electricity use doubled between 2000 and 2005.
- 1. Introduction: Machine-learning systems often lack the efficiency and scalability needed for large datasets, especially when nonparametric methods incur high optimization costs.High-dimensional data can produce an exponential rise in computational complexity and increase susceptibility to generalization problems.
- 1. Introduction: Sustainable data modeling seeks to maximize performance while minimizing energy use, challenging the traditional performance-versus-energy tradeoff.The paper notes that only a few studies have addressed efficient and sustainable modeling for large, complex data-intensive fields.
- 1. Introduction: The review covers theoretical, empirical, and experimental literature on sustainable and energy-efficient machine learning.It aims to provide a new perspective and roadmap for computer science and green ICT researchers.
- 1. Introduction: The paper examines large-scale data-intensive areas, sustainability issues in intelligent data modeling, and recommendations for future approaches.Its structure covers data-intensive-area characteristics, current modeling issues, recommendations, and concluding remarks.
2. Big data challenge
Large-scale scientific and societal datasets are growing across environmental, biological, health, and astronomical domains. Existing intelligent systems often cannot efficiently exploit this data, motivating new approaches in intelligent information science.
- 2. Big data challenge: E-science results depend on increasing data quantity and quality, but current machine-learning systems leave some of this information unexplored and underexploited.The review limits its scope to analytical aspects of science areas using immense datasets.
- 2. Big data challenge: Environmental monitoring generates about a gigabyte of data per wireless sensor per day from measurements across spatial and temporal dimensions.Thousands of wireless sensors are currently deployed, producing continuously expanding environmental datasets.
- 2. Big data challenge: Climate research must integrate vast datasets from satellites, radars, and sensor networks in many formats.The World Data Centre for Climate archives hundreds of terabytes of earth-system, observational, and simulation data.
- 2. Big data challenge: Biological repositories are expanding rapidly as genome, protein, gene-expression, and structural data accumulate.As of December 2014, GenBank contained above 178 million nucleic-acid entries and SWISS-PROT about 18 million protein-sequence entries; both databases were doubling every 15 months.
- 2. Big data challenge: Healthcare systems increasingly generate and archive large volumes of electronic records and medical images.InSiteOne’s U.S. archives included almost 4 billion medical images and 60 million clinical studies across about 800 clinical sites.
- 2. Big data challenge: Astronomical surveys produce demanding data rates and volumes from increasingly capable observatories.VISTA reached 55 megabytes/second and up to 1.4 terabytes per night, while DES planned to image 5000 square degrees of the southern sky over about five years.
- 2. Big data challenge: The growth of scientific data is expected to reshape conventional science and motivate new approaches in intelligent information science.The paper presents this expansion as both a research opportunity and a challenge requiring further investigation.
3. Sustainable data modeling and efficient learning
Sustainable data modeling seeks to handle large data volumes efficiently by reducing computational cost while maintaining or improving predictive performance. The section presents ensemble, local, semiparametric, and deep-learning approaches as routes toward more scalable learning.
- Sustainable data modeling targets maximum learning accuracy with minimum computational cost and rapid processing of large data volumes.
- Ensemble models: Ensemble methods combine multiple models for better performance, but their computational requirements can grow exponentially on large-scale data.
- Local learning strategy: Local learning partitions training samples into clusters and builds a separate local model for each cluster to reduce computational cost.
- Local learning strategy: Semiparametric local models reduce complexity and computational requirements while balancing the lower bias of nonparametric models against the lower variance of parametric models.
- Local learning strategy: Local learning is reported to be more efficient than global learning, especially for large-volume data problems.
- Deep learning: Deep neural networks achieve strong recognition and classification performance but have high computational cost and scalability difficulties.
4. Concluding Remarks
The review surveys sustainable data modeling for large-scale, data-intensive fields and presents it as a way to handle growing e-science data while supporting its use across applications.
- The review covers theoretical and experimental research on model energy efficiency, computational requirements, and data-intensive area structure and design.
- Sustainable data modeling is presented as a way to handle large quantities of e-science data and extend data-modeling approaches across e-science areas.