Source-linked AI summary
The Role of Big Data Analytics in Industrial Internet of Things
Muhammad Habib ur Rehman, Ibrar Yaqoob, Khaled Salah, Muhammad Imran, Prem Prakash Jayaraman, Charith Perera
TL;DR
IIoT generates substantial big data, but processing is constrained by limited device-end computational, networking, and storage resources, while research on IIoT–BDA convergence remains limited. This paper surveys BDA technologies, algorithms, frameworks, and case studies, develops a taxonomy, and concludes that BDA adoption in IIoT remains at an early stage.
Problem
IIoT produces large-scale data, yet device-end resource constraints complicate processing and only a few studies examine IIoT and BDA together.
Method
The paper surveys BDA technologies, algorithms, techniques, frameworks, and case studies, organizing the literature through a taxonomy of key parameters.
Results
BDA adoption in IIoT is still in its early stage, despite identified frameworks, case studies, opportunities, technologies, and research challenges.
Takeaways & Limitations
BDA provides distributed processing across sensors, endpoints, edge servers, and cloud systems for developing intelligent IIoT systems.
Takeaways & Limitations
Fully interoperable IIoT systems still require well-defined protocols addressing interoperability parameters, cross-industry BDA execution, and heterogeneous data and technologies.
Abstract
from arXiv · showhide
Big data production in industrial Internet of Things (IIoT) is evident due to the massive deployment of sensors and Internet of Things (IoT) devices. However, big data processing is challenging due to limited computational, networking and storage resources at IoT device-end. Big data analytics (BDA) is expected to provide operational- and customer-level intelligence in IIoT systems. Although numerous studies on IIoT and BDA exist, only a few studies have explored the convergence of the two paradigms. In this study, we investigate the recent BDA technologies, algorithms and techniques that can lead to the development of intelligent IIoT systems. We devise a taxonomy by classifying and categorising the literature on the basis of important parameters (e.g. data sources, analytics tools, analytics techniques, requirements, industrial analytics applications and analytics types). We present the frameworks and case studies of the various enterprises that have benefited from BDA. We also enumerate the considerable opportunities introduced by BDA in IIoT.We identify and discuss the indispensable challenges that remain to be addressed as future research directions as well.
1. Introduction
IIoT and BDA have been widely studied separately, but their convergence remains underexplored despite growing industrial data production and incomplete integration. This study surveys the area, proposes a taxonomy, presents frameworks and case studies, and identifies opportunities and challenges.
- Motivation: Only a few studies have explored the convergence of IIoT and BDA, although both domains have been widely studied independently.
- Motivation: IIoT generates extensive big data through large-scale sensing, but complete BDA integration and implementation in IIoT systems remain unavailable.Existing surveys have generally focused on adjacent topics, including adoption, edge-cloud integration, marketplaces, virtualisation, CPS, smart manufacturing, and business operations.
- Contributions: The study builds a theoretical case for BDA in IIoT and discusses its role and complete process for enriching system intelligence.
- Contributions: The authors classify existing IIoT research in terms of BDA by devising a taxonomy of the literature.
- Contributions: The paper presents frameworks and case studies in which BDA processes are adopted to improve overall IIoT system performance.
- Contributions: The study identifies research opportunities, challenges, and future technologies addressing gaps between state-of-the-art research and industrial practice.
2. BDA in IIoT Systems
BDA supports IIoT value creation by processing data across integrated industrial technologies and distributed computing resources. IIoT design therefore combines interoperability, virtualisation, decentralisation, real-time capability, service orientation, modularity, and security.
- Design Principles: Seven principles guide IIoT design: interoperability, virtualisation, decentralisation, real-time capability, service orientation, modularity, and security.
- Data Production: CPS and IoT devices generate massive raw data streams, while real-time analysis can improve machine health and support defect-free product manufacturing.
- Industrial Devices: IoT devices remotely sense and actuate in industrial environments, either independently or attached to existing CPS.
2.3. Concentric Computing Model for BDA in IIoT
The concentric computing model distributes BDA across sensors, endpoints, edge servers, and cloud systems. This enables local data reduction and distributed computational load sharing across heterogeneous resources.
- Computing Model: Concentric computing is a highly distributed model spanning sensors, IIoT endpoints, edge servers, and centralised or decentralised cloud systems.
- Resource Distribution: Sensors and IoT devices can filter and reduce raw data streams locally despite limited computational power.
- Resource Distribution: Edge servers and centralised computing clusters distribute the computing load for BDA applications.
- BDA Execution: Concentric environments require multistage execution, automation, and management of data engineering, data preparation, and data analytics.
2.4. Big Data Analytics for Delivering Intelligence in IIoT Systems
BDA in IIoT requires a multistage pipeline that transforms raw data into models and knowledge while adapting to evolving streams. Data preparation is labor-intensive, and automated lifecycle management remains incomplete.
- Data Engineering: BDA pipelines sequence data engineering, preparation, and analytics, with data engineers ingesting, cleaning, conforming, shaping, and transforming IIoT data.
- Data Preparation: Data scientists spend 70% −80% of their time on data preparation activities.Preparation includes refinement, data blending, cleaning, noise removal, and outlier or anomaly detection.
- Data Analytics: Analytic processes generate learning models from prepared data, score sample datasets, rank attributes, and deploy tuned models to discover patterns in future data.
- Pipeline Management: Existing literature lacks automated data pipelines for IIoT, motivating holistic lifecycle management from raw acquisition through knowledge visualisation and actuation.
- Pipeline Management: Evolving data streams cause knowledge shift, requiring monitoring, change detection, adaptive reconfiguration, and re-execution of BDA processes.
3. Technologies and Algorithms for BDA in IIoT systems
BDA technologies and algorithms are applied across IIoT use cases including smart production, zero-defect manufacturing, shop-floor monitoring, energy planning, machine-health monitoring and predictive maintenance.
- Mass Product Customization: Self-organising maps support massively customised manufacturing by optimising big data for production systems.The approach uses feedback across smart design, manufacturing, production and services to improve subsequent operations.
- Industrial Time Series Modeling: Neo-fuzzy neuron time-series modelling connects multiple input streams with final outputs, reducing data streams and learning-model iterations.It addresses high-dimensional streams produced by monitoring multiple manufacturing components for zero-defect production.
- Intelligent Shop Floor Monitoring: Trajectory clustering, Petri nets and decision trees support smart-object tracking, performance analysis and exception diagnosis on shop floors.The literature identifies a need for a component-based BDA architecture, while the tested model shows feasibility.
- Energy Management: BDA methods use manufacturing, environmental and energy-consumption data for industrial microgrid planning and improved energy utilisation.The passage notes that quantifiable studies remain incomplete.
- Monitoring Machine Health: PHM analysis supports early fault diagnosis, remaining-useful-life assessment and maintenance planning for machine components.Studies also examine component dependencies, failure modes, faults, errors and failures.
- Predictive and Preventive Maintenance: Hadoop and Storm processing combined with neural-network prediction supports offline prediction and online maintenance without shutting down manufacturing units.Predictive and preventive maintenance are identified as key requirements of large-scale IIoT systems.
4. Taxonomy
The taxonomy classifies BDA in IIoT by data sources, analytics tools, techniques, requirements, industrial applications and analytics types.
- Data Sources: Industrial data sources include sensors, ERP, MES, SCADA, CRM and machine or IoT devices.
- Analytics Tools: The taxonomy includes analytics software, algorithm repositories, visualisation tools, modelling tools and online analytics packages.These tools support prediction, algorithm sharing, advanced data presentation, requirements analysis and web-traffic analysis.
- Analytics Techniques: Analytics techniques include text analytics, machine learning, data mining, statistical methods and natural language processing.These techniques are presented as ways to obtain value from industrial big data and support faster, better decisions.
- Requirements: IIoT analytics systems require maturity models, functional architecture, infrastructure architecture and integrated analysis.Maturity models monitor analytics capabilities, development effort and the health of organisational big-data programs.
- Industrial Analytics Applications: Industrial analytics applications span manufacturing and operations, logistics and supply chain, marketing and sales, and research and development.Predictive analytics can support maintenance rescheduling and decision-support systems in manufacturing.
- Analytics Types: Analytics types are descriptive, real-time, predictive and prescriptive, covering historical insight, current situations, potential issues and recommended actions.
5. Frameworks and Case Studies
The reviewed frameworks and case studies apply BDA to value creation, integrated analytics, self-configuration, fault detection, and cleaner production across industrial settings.
- BDA Value Creation: BDA integrates customer and enterprise data with historical and real-time data to support customised production and zero defects.
- BDA Frameworks: SnappyData unifies Spark processing and GemFire in-memory storage for transaction processing, analytical processing and streaming analytics.
- IIoT Case Studies: Ipanera continuously monitors water level and fertilizer quality through IIoT layers and generates insights for self-configuration.
- Fault Detection Classification: A fault-detection and classification framework combines IoT devices, CPS, cloud computing and deep belief networks to classify faulty products.Deployment in a car-headlight manufacturing unit produced reliable results.
- Cleaner Production: A four-stage cleaner-production architecture covers lifecycle objectives, IoT-based data acquisition, Hadoop and Storm processing, and BDA algorithms.The architecture was evaluated on an axial-compressor manufacturing unit.
- Railway Maintenance: Japan’s railway system is adopting IIoT, BDA and automation for condition-based maintenance, AI work support and railway-asset management.
6. Opportunities, Research Challenges, and Future Technologies
BDA is presented as a route to operational and value-chain improvements in IIoT, while the review finds that existing literature lags behind this vision.
- Opportunities: BDA opportunities include maximising operational efficiency, reducing product-development cost, enabling mass customisation and streamlining supply-chain management.
- Research Challenges: The review identifies a substantial gap between the envisioned adoption of BDA in IIoT and the existing literature, motivating research challenges and solutions.Table 2 summarises the challenges and perceived solutions for fuller adoption.
6.1. Opportunities
BDA creates opportunities for IIoT through intelligent automation, richer human-machine interaction, cybersecurity analytics, interoperable data practices, and end-to-end industrial analytics.
- Automation and AI: AI methods can optimise and analyse high-dimensional, multimillion-variable datasets formed by integrating customer and enterprise data.Future IIoT systems are expected to ingest data from online, offline, inbound, and outbound operations.
- Human Machine Interaction: BDA can combine real-time knowledge patterns with wearable, augmented-reality, and robotic systems to support autonomous, self-sustaining IIoT environments.The stated opportunity is richer human-machine interaction and highly productive user interfaces.
- Cybersecurity, Privacy, and Ethics: BDA can provide real-time cyber threat intelligence by analysing attacks, privacy leaks, unauthorised access, unethical data collection, anomalies, and vulnerabilities.The analysis spans security-related network and enterprise data across IIoT systems.
- Universal Standards: Universal standards could specify data collection, security, preservation, sharing, stakeholder benefits, and customer value across industries.The paper links such standards to addressing ethical issues and enabling personalised products and services.
- Protocols for Interoperability: Interoperability protocols could support cross-industry BDA despite heterogeneity in data, computing technologies, and industrial production systems.The protocols must define interoperability parameters and how BDA is executed across industries.
- End-to-end Industrial Analytics: An end-to-end analytics pipeline could process customer, operational, IoT, CPS, and manufacturing data in parallel to identify cross-system knowledge patterns.Existing systems manage these sources separately, leaving an opportunity for integrated analysis.
6.2. Research Challenges and Future Technologies
Future IIoT research must integrate real-time big data processes across heterogeneous devices, platforms, and industries while developing technologies that improve scalability, efficiency, and interoperability.
- Research Challenges: Research efforts are needed to improve the entire IIoT technology ecosystem and address its massive heterogeneity without compromising operational efficiency.The paper frames these needs as future research opportunities associated with increasing BDA adoption.
- Big Data Process Integration into IIoT Systems: Developing end-to-end, real-time big data processes remains a major challenge because each industry requires planning, deployment, maintenance, and continuous improvement.The paper envisions single-dashboard access to industry-wide intelligence and vertically aligned data sources.
- Heterogeneity and Efficiency: IIoT analytics must accommodate heterogeneous processing capabilities, storage systems, device power sources, and communication channels with different bandwidths.Concentric computing distributes computational and storage support across different devices and systems.
- Emerging Technologies: Virtualisation and microservices are relevant technologies for executing scalable BDA processes across mobile, distributed, and loosely coupled IIoT platforms.IoT mobility requires continuous virtual-machine migration, while microservices support orchestration across platforms and devices.
- Emerging Technologies: Graph structures and big graph analytics can reduce the complexity of multipoint, multisite, and high-dimensional IIoT datasets.These methods separate, map, and analyse data in different graph formats for more efficient BDA execution.
- Research Challenges: Table 2 summarises research challenges together with their perceived solutions.The supplied caption identifies the table’s organising relationship but provides no individual rows or comparisons.
- Emerging Technologies: Fog computing and blockchain are identified as emerging technologies for localised processing, storage, reduced cloud delays, and decentralised data management.The passages identify both technologies as potentially pivotal to BDA for IIoT.
7. Conclusions
The paper surveys BDA technologies, algorithms, frameworks, and case studies in IIoT, develops a taxonomy, and identifies opportunities and challenges for future research. It concludes that BDA adoption remains early and that interoperability standards and IIoT-specific process capabilities are still needed.
- Conclusions: The paper surveys related technologies, algorithms, frameworks, and case studies and provides a taxonomy of key IIoT-BDA concepts.It also discusses future opportunities, technologies, and research challenges.
- Conclusions: BDA adoption in IIoT systems remains at an early stage, while complementary components such as IoT devices, augmented reality, and CPS are also in their infancy.The conclusion characterises current BDA systems as generic frameworks for data engineering, preparation, and analysis.
- Conclusions: Existing BDA processes require considerable alteration to meet IIoT demands, including new standards for interoperability among cross-Industry 4.0 BDA platforms.The conclusion also calls for end-to-end capabilities in future IIoT systems.