Performance Analysis and Comparison of Machine and Deep Learning Algorithms for IoT Data Classification

1. Introduction

• IoT data are often raw and nearly meaningless until processing extracts information, while machine learning can identify patterns and support predictions for smarter IoT systems.The introduction frames AI and machine learning as approaches to analyzing data and informing future decisions.Source: §1 · p. 1
• Prior IoT machine-learning studies are described as nascent and often limited to one problem type, leaving a need for broader performance analysis.The authors note that extensive research remains necessary.Source: §1 · p. 1
• The study compares common machine and deep learning algorithms for classification across IoT datasets from varied domains, dimensions, and features.It uses multiple evaluation metrics and presents comparative tables and diagrams.Source: §1 · p. 2

2. Related Work

• The authors identify few thorough studies comparing machine and deep learning performance across different types of IoT problems.They characterize much prior work as addressing a particular problem rather than providing broad performance analysis.Source: §2 · p. 2
• On five IoT datasets, Decision Trees achieved the highest accuracy, 99% for all datasets, whereas Logistic Regression and Naïve Bayesian showed the weakest performances, according to.For shopping-mall air-conditioning load prediction, found hybrid Chaos-SVR and WD-SVR outperformed single models, though hybrid-model complexity could increase considerably.Source: §2 · p. 2
• In activity-recognition experiments, deep-learning models were not competitive; compared algorithms in offline and real-time settings and reported ANN achieved a 97% recognition rate, better than all previous experiments without parameter search.Source: §2 · p. 3
• Other reported winners depended on the task: found C4.5 and C5.0 stronger, while reported 92% accuracy for CBR and better KNN performance on small datasets.[18] also reports that Naïve Bayesian performance remained almost constant as dataset size and feature count increased.Source: §2 · p. 3

3. Research Methodology

• A separate experiment measures how quickly the models learn, alongside the evaluation of their performance metrics.The methodology frames the algorithms, datasets, and evaluation measures as inputs to the later performance comparison.Source: §3 · p. 3, §3.3 · p. 5

3.1. Machine and Deep Learning Algorithms

• The study selects eight machine-learning methods and three deep-learning models, including ANN, CNN, and LSTM, for evaluation on IoT datasets.The machine-learning set includes LR, KNN, GNB, DT, RF, SVM, SGDC, and Adaboost.Source: §3.1 · pp. 3–4

3.2. Datasets

• Six smart-environment datasets span transportation mode, occupancy, activity recognition, daily living and fall detection, weather, and pump maintenance.The datasets have distinct features and target classes, and are labeled DS1 through DS6.Source: §3.2 · pp. 4–5

3.3. Performance Evaluation Metrics

• The comparison uses precision, recall, f1-score, accuracy, confusion matrices, and ROC-AUC to assess classification performance.The text notes that accuracy may be inappropriate when target classes are unbalanced.Source: §3.3 · p. 5

4.1. Experiment 1: Performance Comparison using Evaluation Metrics

• RF led most DS1 metrics, averaging 85% test accuracy (87% highest); on DS2, KNN and RF reached 98% precision and F1, while KNN, RF, and SVM reached 0.98 ROC-AUC.GNB was fastest on both datasets; DS2's potentially biasing “light” feature was removed because it alone yielded 99% accuracy.Source: Table 3 · p. 6, §4.1 · pp. 6–7, Table 4 · p. 7
• On DS3, KNN led recall and F1 while RF slightly led precision; RF and KNN had the highest training accuracy at 92%, and deep models’ test accuracy was around 75%.GNB processed the data in under a second; LR was the slowest machine-learning model at 17.11 seconds, and ANN was faster than CNN and LSTM.Source: Table 5 · p. 7, §4.1 · p. 7
• On DS4, RF led most metrics with 79% test accuracy and 1.2-second execution, while CNN led deep models at 33.5 seconds; RF and DT had 100% training accuracy, indicating overfitting.NB’s accuracy never exceeded 16%.Source: §4.1 · p. 7, Table 6 · p. 8
• On DS5, ANN alone reached 86% test accuracy and led ROC-AUC at 0.74; GNB was fastest at 0.19 seconds and LSTM slowest at 188.7 seconds.LR, RF, SGDC, Adaboost, and CNN had test accuracies close to ANN; categorical-feature conversion nearly tripled the dataset’s feature count.Source: Table 7 · p. 8, §4.1 · p. 8
• On imbalanced DS6, RF was the final winner: RF and Adaboost each made just one false positive among models that predicted all 303 pump failures, and RF took 1.8 seconds versus Adaboost’s 34 seconds.The study used confusion matrices because its other metrics were not meaningful for this dataset; the matrices’ rows are true labels and columns predicted labels.Source: §4.1 · pp. 8–9, Table 8 · p. 9

4.2. Experiment 2: Convergence Speed

• On DS1, RF reached 77% accuracy with 5% of the data and 88% with 50%; on DS2, KNN, DT, RF, and SVM reached at least 93% using only 5%.RF led convergence on DS1; several DS2 models reached 97–99% accuracy by the end of training.Source: §4.2 · pp. 9–10, Figure 1 · p. 10, Figure 2 · p. 10

4.3. Discussion

• RF and DT showed greater overfitting risk, while LSTM often took longest to execute; CNN often outperformed ANN and LSTM, and convergence comparisons favored KNN, DT, RF, and SVM among machine models and ANN and CNN among deep models.The discussion cautions that complex deep architectures can sharply increase execution time and suggests distributed or federated learning to spread workloads across constrained IoT devices.Source: §4.3 · p. 11

• Random Forest outperformed other machine-learning algorithms on most metrics, while ANN and CNN achieved the best results among deep-learning models.Source: §5 · p. 12
• Random Forest learned faster than the other models when the input training-data volume was remarkably small.Source: §5 · p. 12
• GNB had lower execution time than the other evaluated algorithms.Source: §5 · p. 12
• The study evaluated 11 machine- and deep-learning algorithms in separate experiments for performance metrics and learning speed.Source: §5 · p. 12