Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things

IoT Architecture

• IoT connects sensors, actuators, and devices through networks to collect data, exchange information, and deliver smart services.Its three-layer architecture comprises perception, network, and application layers.Source: §I · p. 1
• 50 billion devices are projected to connect to IoT by 2025, generating 79.4 zettabytes of data and creating bandwidth and latency challenges for centralized cloud processing.Fog computing addresses these constraints by placing storage, computation, and networking closer to devices.Source: §I · p. 1

AI for IoT

• Heterogeneous sensor data and handcrafted-feature pipelines are increasingly unsuitable for perceiving and understanding dynamic, complex IoT environments.The introduction contrasts classical methods based on handcrafted features with deep learning approaches that learn representations end to end.Source: §I · p. 2
• Deep neural networks can automatically learn discriminative representations from labeled or unlabeled data and support low-latency processing across cloud, fog, and edge platforms.The paper links this capability to improved perception in IoT systems with many heterogeneous sensors.Source: §I · p. 2
• IoT systems must address unseen edge cases and changing operating conditions because models trained on initial data may lose performance as environments or machine modes vary.The introduction connects these issues to few-shot, zero-shot, meta-, unsupervised, semi-supervised, transfer, and domain-adaptation learning.Source: §I · p. 2

Survey Scope

• The survey covers AIoT architecture, AI progress organized around perceiving, learning, reasoning, and behaving, promising applications, and challenges with research opportunities.It aims to show how AI technologies empower IoT applications.Source: §I · p. 2, A. Contributions of this Survey · p. 3

Survey Scope

• Existing IoT surveys often focus on selected topics such as architectures, networking, applications, security, or privacy rather than AIoT as a whole.The paper positions its review as a focused treatment of AIoT.Source: B. Relationship to Related Surveys · p. 3, 1) Architecture · p. 3, 2) Networking Technologies · p. 3
• This survey applies a taxonomy based on how deep learning improves AIoT systems’ perceiving, learning, reasoning, and behaving abilities.It focuses on deep learning for IoT data processing and covers recent progress across application domains.Source: 3) Data Processing · p. 3

Security and Privacy

• Security and privacy remain related IoT concerns because connected sensors may collect biometric data vulnerable to cyberattacks.The survey reviews AI-based approaches such as federated learning alongside access control and trust management.Source: 4) Security and Privacy · p. 3

Applications

• The review also summarizes promising AIoT applications across domains including smart cities, homes, healthcare, agriculture, industry, vehicles, and video things.It discusses how AI enables these applications to become faster, smarter, greener, and safer.Source: 5) Applications · p. 3

• The paper first discusses AIoT computing architecture, then surveys enabling AI technologies, summarizes applications, and addresses challenges, opportunities, and conclusions.Source: C. Organization · p. 3
• Cloud computing provides scalable computation, storage, and networking for integrating and processing data from distributed sensors.Source: 1) Cloud Computing Layer · p. 4
• The architecture discussion includes cloud computing, fog computing, and edge computing, with hybrid deployment across these tiers.Source: 2) Fog Computing Layer · p. 4
• Fog computing places storage, computation, and networking near devices to provide lower latency than remote cloud processing.Source: 2) Fog Computing Layer · p. 4

Edge Computing Layer:

• Fog computing operates at the network side, whereas edge computing operates at the thing side near sensors and actuators.The paper treats the concepts as distinct for clarity despite overlapping usage in some literature.Source: 3) Edge Computing Layer · p. 5
• Edge computing can reduce latency and network bandwidth compared with fog and cloud computing.Source: 3) Edge Computing Layer · p. 5
• AIoT systems can deploy different models across cloud platforms, fog nodes, and edge devices to form hybrid computing architectures.Offloading computation from edge devices to fog nodes and cloud is expected to combine low latency with deep-learning capacity for massive data.Source: 3) Edge Computing Layer · p. 5
• GPU parallelism accelerates neural-network operations such as convolution, reducing training and inference time.Source: 1) Hardware · p. 5
• TensorFlow and PyTorch support mobile deployment through model compression, quantization, and hardware acceleration, while ONNX enables model porting across frameworks.Source: 2) Software · p. 5
• The survey organizes AI technologies for AIoT by their contributions to perceiving, learning, reasoning, and behaving.It selects topics closely related to and useful for AIoT applications rather than surveying deep learning broadly.Source: §III · p. 5

2) Object Detection:

• Generic object detection recognizes both object categories and locations and supports downstream tasks including face recognition, re-identification, pose estimation, and behavior analysis.Source: 2) Object Detection · p. 6
• Two-stage and one-stage methods form the main categories of contemporary deep-learning object detectors.Source: 2) Object Detection · p. 6
• Object detection applications include pedestrian, car, traffic-sign, and license-plate detection for traffic surveillance, public safety, autonomous driving, and video structuring.Source: 2) Object Detection · p. 6
• Object tracking has progressed through multi-resolution deep features, end-to-end representation learning, and Siamese-network approaches.Source: 3) Object Tracking · p. 7
• Fully convolutional encoder–decoder networks are the dominant paradigm for semantic segmentation because they learn multi-resolution features while preserving spatial correspondence.Source: 4) Semantic Segmentation · p. 7
• Text spotting jointly performs text detection and recognition despite variable text shapes, lengths, fonts, styles, and backgrounds.Sequence-to-sequence modeling and connectionist temporal classification have also been applied to the task.Source: 5) Text Spotting · pp. 7–8
• Human-centric perceiving covers biometric recognition, person re-identification, pose, gesture, and action estimation, and crowd-density estimation.Source: 5) Text Spotting · p. 8

Biometric Recognition

• Face recognition systems typically combine detection, alignment, representation, and classification or verification, with deep learning improving shared feature representations.Jointly modeling auxiliary tasks such as alignment, pose estimation, and gender classification can provide additional supervisory signals.Source: 6) Biometric Recognition · p. 8
• Biometric solutions span face, fingerprint, iris, palmprint, and person re-identification, with method choice depending on sensors, usage scenarios, latency, and power consumption.Applications include access control, aviation security, e-banking, and mobile payment.Source: 6) Biometric Recognition · pp. 8–9, 7) Person Re-Identification · p. 9

Human-Centric Perceiving

• Human pose estimation detects body joints using top-down or bottom-up methods, while current advances target feature representation, training strategies, sub-pixel processing, and crowded-scene occlusion.Top-down methods separate person detection from keypoint detection; bottom-up methods detect and associate keypoints directly.Source: 8) Human Pose Estimation and… · p. 9
• AIoT human-centric perception also includes hand gesture recognition and crowd counting using visual, radar, Wi-Fi, or Bluetooth sensing.Practical crowd-counting research seeks lightweight models, joint crowd-flow estimation, and multimodal sensor integration.Source: 8) Human Pose Estimation and… · p. 9, 9) Crowd Counting · pp. 9–10

Perception Overview

• Figure 6 illustrates generic scene, human-centric, and 3D perception through segmentation, detection, text spotting, parsing, pose, facial analysis, and depth estimation.The figure processes a frame from the video “Walking Next to People” using these complementary techniques.Source: Figure 6 · p. 10

Depth Estimation

• Depth estimation in AIoT supports monocular, stereo, and multi-view camera configurations, with recent work focusing on unsupervised monocular video and camera-pose learning.Photometric reprojection losses derived from multi-view geometry provide an optimization signal.Source: 10) Depth Estimation/Localization/SLAM · pp. 10–11
• Depth and pose estimation must address occlusions, dynamic objects, per-frame scale ambiguity, and temporal inconsistency.These conditions complicate reliable estimation even when convolutional networks provide strong representation capacity.Source: 10) Depth Estimation/Localization/SLAM · p. 11

Localization and SLAM

• Visual odometry and visual-inertial odometry calculate sequential camera poses from camera and IMU data and serve as front ends in SLAM systems.SLAM back ends optimize pose graphs to obtain globally consistent and drift-free pose estimates.Source: 10) Depth Estimation/Localization/SLAM · p. 11

Applications

• Depth estimation, pose estimation, VO/VIO, and SLAM provide 3D perception for smart transportation, industry, agriculture, cities, and homes.Multiple viewpoints can support depth estimation and 3D reconstruction of objects or scenes.Source: 10) Depth Estimation/Localization/SLAM · p. 11

Perception Preprocessing

• AIoT visual perception often requires preprocessing such as illumination enhancement, contrast improvement, and distortion rectification before downstream tasks.Low-light and hazy images are ill-posed enhancement problems addressed by prior-based and learning-based methods.Source: 10) Depth Estimation/Localization/SLAM · p. 11, 11) Image Enhancement · p. 11

Image Rectification and Stitching

• Wide-FOV cameras support surveillance and autonomous driving but introduce distortion because their imagery violates the perspective-transformation assumption.Rectification is therefore performed before downstream perception tasks.Source: 12) Image Rectification and Stitching · p. 11
• Rectification methods include camera-calibration approaches and direct distortion-parameter estimation followed by mapping to a rectified image.Optimization-based and learning-based methods exploit geometric cues such as lines to formulate constraints or loss functions.Source: 12) Image Rectification and Stitching · pp. 11–12

Auditory Perception

• Deep neural networks revolutionized speech recognition by enabling end-to-end modeling without handcrafted feature engineering, explicit HMM design, or domain knowledge.Real-time ASR can run on devices or through cloud-assisted computing for interaction, transcription, and voice-assistant applications.Source: 13) Speech Recognition · p. 12

Natural Language Processing

• Neural machine translation uses encoder-decoder architectures and attention mechanisms to model source context during decoding.Attention supports joint alignment and translation, with reported benefits for long sentences.Source: 15) Machine Translation · pp. 12–13

Multimedia Analysis

• Cross-media matching and retrieval seek semantic alignment across modalities, while disentangled representation learning separates modality-exclusive information from shared embeddings.Related generative tasks include image or video captioning and text-to-image generation.Source: 16) Multimedia and Multi-modal Analysis · p. 13

Multimodal Perceiving

• RGB data can be combined with depth, LiDAR, or thermal infrared inputs for scene parsing, low-light detection and tracking, and action recognition.Fusion occurs at input, feature, or output level, with feature-level fusion described as most prevalent.Source: 16) Multimedia and Multi-modal Analysis · p. 13

Applications

• Multimedia generation and cross-modal analysis support retrieval, personalized item descriptions, education, chatbots, smart security, and rehabilitation monitoring.These applications span speech-, image-, video-, and text-based interaction and understanding.Source: 16) Multimedia and Multi-modal Analysis · p. 13

Efficient Models

• Neural architecture search automatically explores architectures within a predefined search space using evolutionary, reinforcement-learning, or gradient-based methods.NAS replaces manual architecture design with an automated search procedure.Source: 16) Multimedia and Multi-modal Analysis · p. 13, 17) Network Compression and NAS · p. 14
• Network compression improves DNN efficiency under AIoT computational limits through pruning, quantization, low-rank factorization, and knowledge distillation.Pruning commonly trains a large network, removes components using a criterion, and retrains the reduced model.Source: 17) Network Compression and NAS · pp. 13–14

Learning

• AIoT learning research addresses changing environments through unsupervised, semi-supervised, transfer, domain-adaptation, few-shot, and zero-shot learning.Unlabeled and multimodal sensor data can support continual model improvement and cross-modal pretext tasks.Source: B. Learning · p. 14, 1) Unsupervised/Semi-supervised Learning · p. 14

Transfer Learning

• Transfer learning addresses target tasks with insufficient training data by transferring knowledge from a related source task.Unlike semi-supervised learning, source and target data distributions need not be identical.Source: 2) Transfer Learning and Domain… · p. 14
• Transfer dependencies across related AIoT tasks can reduce labeled-data demands while maintaining performance and improving generalization.A computational taxonomic map across twenty-six visual tasks provides empirical guidance on transfer directions and supervision reuse.Source: 2) Transfer Learning and Domain… · p. 15

Domain Adaptation

• Domain adaptation learns models for a target domain from one or more source domains for the same task, including unsupervised settings without target annotations.Its core strategy is learning domain-invariant representations by matching source and target distributions.Source: 2) Transfer Learning and Domain… · p. 15
• Domain adaptation and transfer learning are practically useful when AIoT models move to new scenarios or machine operating modes.These settings include related tasks such as object detection, tracking, and semantic segmentation in video surveillance.Source: 2) Transfer Learning and Domain… · p. 15

Few-shot Learning

• Few-shot learning uses prior knowledge to learn from only a few annotated samples, supporting rare-case recognition when large-scale collection and annotation are difficult.Examples include car collisions, cyber attacks, and machine faults.Source: 3) Few-/Zero-shot Learning · p. 15

Zero-shot Learning

• Zero-shot learning maps data to semantic representations so models can recognize unseen classes, including future fault types or newly emerging categories.Auxiliary attributes or text descriptions represent both seen and unseen classes.Source: 3) Few-/Zero-shot Learning · p. 15

Federated Learning

• Federated learning enables multiple data owners to collaboratively train a model without leaking their datasets to one another.Its architecture typically includes a central server and distributed client devices that compute gradients locally.Source: 5) Federated Learning · p. 16

Reasoning

• Reasoning abilities are important for AIoT systems that must make smart and explainable decisions from knowledge such as web information, medical records, and financial transactions.The paper reviews knowledge graphs and reasoning as mechanisms for using this information.Source: C. Reasoning · p. 16
• Knowledge graphs represent entities and relations as triples, enabling knowledge reasoning that infers new knowledge from existing facts.Knowledge reasoning supports graph refinement, classification, entity resolution, relation extraction, question answering, and recommendation.Source: 1) Knowledge Graph and Reasoning · p. 16
• Causal reasoning includes causal inference for estimating effects and causal discovery for finding causal relations.It is relevant to recommendations, fault analysis, driving safety, and interpretation of DNN decisions.Source: 2) Causal Reasoning · pp. 16–17

Behaving

• AIoT behaving systems control sensors and actuators to transform a measured or inferred current state into a target state determined by rules or decision models.Control algorithms are task-oriented and designed for the physical structure and sensors of each system.Source: 1) Control · p. 17

Interaction

• AIoT systems interact through input/output devices, mechanical arms, or gearing for communication, manipulation, and transportation.Examples include touchscreens and microphones, industrial robot arms, and mobile-robot wheels.Source: 2) Interaction · p. 17
• Human-machine interaction can accept images, video, keywords, text, sketches, lyrics, and scores to generate descriptions, poems, images, or music.Speech synthesis provides another communication channel between people and AIoT systems.Source: 2) Interaction · p. 17
• Human-robot interaction includes service robots gripping and filling cups and industrial robots assembling products, with research addressing planning, safety, and imitation learning.These examples connect physical assistance and industrial production within smart-home and IIoT settings.Source: 2) Interaction · p. 17

AIoT Applications

• AIoT applications combine perceiving, learning, reasoning, and behaving abilities across security, transportation, healthcare, education, industry, energy, agriculture, homes, and cities.The survey frames these capabilities as enabling faster, smarter, greener, and safer systems.Source: §IV · p. 17

Smart Transportation

• Self-driving systems integrate environmental perception, driving-policy decisions, and interaction with traffic participants, while domain adaptation and transfer learning address deployment challenges.Deep reinforcement learning can learn driving policies from visual input, but real-world training is unaffordable.Source: B. Smart Transportation · p. 18

Transportation and Healthcare

• AIoT supports driver monitoring, human-activity recognition, explainable healthcare assistance, and multimodal medical services using cameras, wearable sensors, and robots.Applications span dangerous-driving detection, rehabilitation, information queries, and auxiliary diagnosis.Source: B. Smart Transportation · p. 18, C. Smart Healthcare · pp. 18–19

Smart Education

• AIoT can help learners recognize species, learn languages, select personalized resources, and support visually impaired users through interaction.Few-shot learning enables edge devices to recognize new bird and animal species from only a few labeled samples.Source: D. Smart Education · p. 19
• Offline speech-to-speech translation on mobile devices combines speech recognition, language translation, and text-to-speech synthesis for language learning.The system can translate multiple languages and provide grammatical information.Source: D. Smart Education · p. 19

Smart Industry

• A digital twin is a mirror digital representation of a physical system that supports smart-factory monitoring, fault diagnosis, and downtime prevention.Connected sensors and actuators send real-time production-line data to the digital twin in the cloud.Source: E. Smart Industry · pp. 19–20
• PCB manufacturing illustrates AIoT use in production, visual defect inspection, and machine fault diagnosis.Deep learning enables automatic real-time visual defect inspection, while AI improves industrial robot functions such as grasping.Source: E. Smart Industry · p. 20

Smart Grids

• Smart grids apply AIoT to fault diagnosis, load monitoring and scheduling, cyber-attack detection, and UAV-based damage assessment.CNN processing of UAV images predicts damage extent, while edge algorithms support real-time sound-source localization.Source: F. Smart Grids · p. 20

Smart Agriculture

• Precision agriculture uses sensors, autonomous machines, and geographic information systems to observe, measure, and respond to crop variability.AIoT supports crop counting, yield estimation, continuous monitoring, path planning, navigation, and greenhouse plant-growth control.Source: G. Smart Agriculture · pp. 20–21

Smart Cities/Homes/Buildings

• Smart homes and cities reuse AIoT techniques from other sectors for speaker authentication, gesture-based control, and related transportation and security functions.These applications connect home interaction with methods developed for broader smart-sector systems.Source: H. Smart Cities/Homes/Buildings · p. 21

Multi-modal Heterogeneous Data Processing, Transmission, and Storage

• Heterogeneous AIoT data streams challenge processing, transmission, and storage, motivating efficient coding and edge extraction of compact representations.Task-oriented representations should be computed at the network edge to reduce bandwidth and latency.Source: 1) Multi-modal Heterogeneous Data… · p. 21

Deep Learning on Edge Devices

• Edge deployment requires lightweight, computationally efficient, and hardware-friendly deep neural networks because edge devices have limited computational and storage resources.Network pruning, compression, and quantization remain open directions.Source: 2) Deep Learning on Edge Devices · p. 21

Computational Scheduling in the AIoT Architecture

• Computation scheduling across cloud, fog, and edge resources must jointly consider data, bandwidth, latency, accuracy, energy, security, privacy, and changing demand.Dynamic adaptive scheduling is needed to handle unbalanced data flow and user demands over time.Source: 3) Computational Scheduling in the… · p. 21

Big and Small Data for Deep Learning in AIoT

• Most AIoT data are unlabeled, while labeling is expensive; unsupervised, transfer, semi-supervised, and few-shot learning are proposed for multimodal data, new classes, rare cases, and state drift.The cited opportunities address both abundant unlabeled data and small labeled datasets.Source: 4) Big and Small Data for Deep… · p. 21

Data Monopoly

• Data collection and exploitation can create data monopolies that form de facto barriers to market entry and threaten free-market competition.The paper frames proprietary data protected by established interests as the basis of this risk.Source: 5) Data Monopoly · p. 21

Data Security and Privacy

• Ubiquitous AIoT sensing raises questions about biometric-data ownership, retention, use, security, and privacy, requiring appropriate protective measures and legislation.The paper cites GDPR as an example giving individuals control over personal data.Source: 6) Data Security and Privacy · pp. 21–22

Energy Efficiency

• Communication technologies are projected to use 21% of global electricity, with data centers contributing more than one-third of that usage, underscoring energy-efficiency needs.Cooling, workload analysis, and task scheduling are discussed as sustainability-related responses.Source: 7) Growing Energy Consumption in Data… · p. 22

Built-in Neural Processing Capacity for Edge Devices

• Built-in neural processing on edge devices can reduce processing latency and network bandwidth consumption by processing sensing data locally.Local biometric storage and verification can also reduce data-leakage risk, while federated learning supports distributed training.Source: 1) Built-in Neural Processing Capacity… · p. 22

Event-Based and Neuromorphic Processing

• Event-based cameras and neuromorphic processors reduce redundant data and dense computation by operating on changing pixels and sparse asynchronous event streams.The paper connects these designs with low-power, low-latency gesture and action recognition.Source: 2) Event-based Sensors and… · p. 22

Virtual Training Environments

• Virtual 3D platforms can train embodied-AI models cost-effectively, especially for deep reinforcement learning, but domain shift between virtual and physical environments remains critical.Examples include Voyage Deepdrive, OpenAI Gym, and Habitat.Source: 3) Deep Learning from the Virtual to… · p. 22

Data and Knowledge Integration for Perceiving, Learning, Reasoning, and Behaving

• Combining prior knowledge with deep learning supports data-efficient learning through zero-shot attribute transfer, knowledge graphs, and graph neural networks for reasoning tasks.The paper identifies question answering and fault or disease diagnosis as example application areas.Source: 4) Data and Knowledge Integration for… · p. 22

Privacy-Preserving Learning

• Federated learning offers a privacy-preserving approach for training on distributed local sensor data without requiring all data to be transmitted to and stored in the cloud.The survey presents federated learning as a representative solution to AIoT privacy concerns.Source: 5) Privacy-preserving Deep Learning · p. 22, 1) Built-in Neural Processing Capacity… · p. 22

Survey Scope and Research Opportunities

• The survey covers AIoT architectures, AI-enabled perceiving, learning, reasoning, and behaving, applications, and the field’s challenges and opportunities.Its conclusion emphasizes lightweight models, edge intelligence, dynamic-environment learning, knowledge and causal reasoning, control, and multimodal interaction.Source: §VI · pp. 22–23