A Comparative Analysis of Supervised Machine Learning Algorithms for IoT Attack Detection and Classification
DOI:
https://doi.org/10.62411/jcta.11901Keywords:
Deep Learning, Internet of Things, Intrusion Detection, Machine Learning, Network Security, Supervised LearningAbstract
The proliferation of Internet of Things (IoT) devices has introduced significant security challenges, necessitating robust attack detection mechanisms. This study presents a comprehensive comparative analysis of ten supervised learning algorithms for IoT attack detection and classification, addressing the critical challenge of balancing detection accuracy with practical deployment constraints. Using the CICIoT2023 dataset, encompassing data from 105 IoT devices and 33 attack types, we evaluate Naive Bayes, Artificial Neural Networks (ANN), Logistic Regression (LR), k-NN, XGBoost, Random Forest (RF), LightGBM, GRU, LSTM, and CNN algorithms based on some performance metrics. The comparative test results show superior performance to the traditional ensemble approach, with RF achieving 99.29% accuracy and leading precision (82.30%), followed closely by XGBoost with 99.26% accuracy and 79.60% precision. Deep learning approaches also demonstrate strong capabilities, with CNN achieving 98.33% accuracy and 71.18% precision, though these metrics indicate ongoing challenges with class imbalance. The analysis of confusion matrices reveals varying success across different attack types, with some algorithms showing perfect detection rates for certain attacks while struggling with others. The study highlights a crucial distinction in IoT security: while high precision remains important, the potentially catastrophic impact of missed attacks necessitates equal attention to recall metrics, as evidenced by the varying recall rates across algorithms (RF: 72.19%, XGBoost: 71.69%, CNN: 64.72%). These findings provide vital insights for developing balanced, context-aware intrusion detection systems for IoT environments, emphasizing the need to consider performance metrics and practical deployment constraints.References
F. Meneghello, M. Calore, D. Zucchetto, M. Polese, and A. Zanella, “IoT: Internet of Threats? A Survey of Practical Security Vulnerabilities in Real IoT Devices,” IEEE Internet Things J., vol. 6, no. 5, pp. 8182–8201, Oct. 2019, doi: 10.1109/JIOT.2019.2935189.
A. K. Goel, A. Rose, J. Gaur, and B. Bhushan, “Attacks, Countermeasures and Security Paradigms in IoT,” in 2019 2nd International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), Jul. 2019, vol. 1, pp. 875–880. doi: 10.1109/ICICICT46008.2019.8993338.
J. P. Ntayagabiri, Y. Bentaleb, J. Ndikumagenge, and H. EL Makhtoum, “A Comprehensive Approach to Protocols and Security in Internet of Things Technology,” J. Comput. Theor. Appl., vol. 2, no. 3, pp. 324–341, Dec. 2024, doi: 10.62411/jcta.11660.
I. Ahmad, M. Basheri, M. J. Iqbal, and A. Rahim, “Performance Comparison of Support Vector Machine, Random Forest, and Extreme Learning Machine for Intrusion Detection,” IEEE Access, vol. 6, pp. 33789–33795, Jan. 2018, doi: 10.1109/ACCESS.2018.2841987.
A. Çetin and S. Öztürk, “Comprehensive Exploration of Ensemble Machine Learning Techniques for IoT Cybersecurity Across Multi-Class and Binary Classification Tasks,” J. Futur. Artif. Intell. Technol., vol. 1, no. 4, pp. 371–384, Feb. 2025, doi: 10.62411/faith.3048-3719-51.
J. A. B. Angelin and C. Priyadharsini, “Deep Learning based Network based Intrusion Detection System in Industrial Internet of Things,” in 2024 2nd International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT), Jan. 2024, pp. 426–432. doi: 10.1109/IDCIoT59759.2024.10467510.
P. K. Mall et al., “A comprehensive review of deep neural networks for medical image processing: Recent developments and future opportunities,” Healthc. Anal., vol. 4, p. 100216, Dec. 2023, doi: 10.1016/j.health.2023.100216.
L. Ding, W. Fang, H. Luo, P. E. D. Love, B. Zhong, and X. Ouyang, “A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory,” Autom. Constr., vol. 86, pp. 118–124, Jan. 2018, doi: 10.1016/j.autcon.2017.11.002.
A. Halbouni, T. S. Gunawan, M. H. Habaebi, M. Halbouni, M. Kartiwi, and R. Ahmad, “CNN-LSTM: Hybrid Deep Neural Network for Network Intrusion Detection System,” IEEE Access, vol. 10, pp. 99837–99849, Jan. 2022, doi: 10.1109/ACCESS.2022.3206425.
M. Fatima, O. Rehman, I. M. H. Rahman, A. Ajmal, and S. J. Park, “Towards Ensemble Feature Selection for Lightweight Intrusion Detection in Resource-Constrained IoT Devices,” Futur. Internet, vol. 16, no. 10, p. 368, Oct. 2024, doi: 10.3390/fi16100368.
Z. A. Al Waisi, “Optimized Monitoring and Detection of Internet of Things resources-constraints Cyber Attacks,” IMT School for Advanced Studies Lucca eprints, Jan. 21, 2023. http://e-theses.imtlucca.it/392/
S. Mishra, T. Anithakumari, R. Sahay, R. K. Shrivastava, S. N. Mohanty, and A. H. Shahid, “LIRAD: lightweight tree-based approaches on resource constrained IoT devices for attack detection,” Cluster Comput., vol. 28, no. 2, p. 140, Jan. 2024, doi: 10.1007/s10586-024-04792-x.
M. Fatima, O. Rehman, S. Ali, and M. F. Niazi, “ELIDS: Ensemble Feature Selection for Lightweight IDS against DDoS Attacks in Resource-Constrained IoT Environment,” Futur. Gener. Comput. Syst., vol. 159, pp. 172–187, Oct. 2024, doi: 10.1016/j.future.2024.05.013.
A. R. Khan, A. Yasin, S. M. Usman, S. Hussain, S. Khalid, and S. S. Ullah, “Exploring Lightweight Deep Learning Solution for Malware Detection in IoT Constraint Environment,” Electronics, vol. 11, no. 24, p. 4147, Jan. 2022, doi: 10.3390/electronics11244147.
Z. Alwaisi, T. Kumar, E. Harjula, and S. Soderi, “Securing constrained IoT systems: A lightweight machine learning approach for anomaly detection and prevention,” Internet of Things, vol. 28, p. 101398, Jan. 2024, doi: 10.1016/j.iot.2024.101398.
U. J. Otokwala, “Lightweight intrusion detection of attacks on the Internet of Things (IoT) in critical infrastructures.,” Jan. 2024, doi: 10.48526/rgu-wt-2571244.
T. Dias, N. Oliveira, N. Sousa, I. Praça, and O. Sousa, “A Hybrid Approach for an Interpretable and Explainable Intrusion Detection System,” in Intelligent Systems Design and Applications, vol. 418, A. Abraham, N. Gandhi, T. Hanne, T.-P. Hong, T. Nogueira Rios, and W. Ding, Eds. Cham: Springer International Publishing, 2022, pp. 1035–1045. [Online]. Available: https://link.springer.com/10.1007/978-3-030-96308-8_96
T. Zhong and J. Li, “Ransomware Detection with Machine Learning by Applying the Lapranove Function on Bytecode.” May 30, 2024. doi: 10.31219/osf.io/zk3sw.
I. Zakariyya, H. Kalutarage, and M. O. Al-Kadri, “Towards a robust, effective and resource efficient machine learning technique for IoT security monitoring,” Comput. Secur., vol. 133, p. 103388, Jan. 2023, doi: 10.1016/j.cose.2023.103388.
H. Taherdoost, “Deep Learning and Neural Networks: Decision-Making Implications,” Symmetry (Basel)., vol. 15, no. 9, p. 1723, Jan. 2023, doi: 10.3390/sym15091723.
O. A. Alimi, K. Ouahada, and A. M. Abu-Mahfouz, “A Review of Machine Learning Approaches to Power System Security and Stability,” IEEE Access, vol. 8, pp. 113512–113531, Jan. 2020, doi: 10.1109/ACCESS.2020.3003568.
S. Muneer, U. Farooq, A. Athar, M. Ahsan Raza, T. M. Ghazal, and S. Sakib, “A Critical Review of Artificial Intelligence Based Approaches in Intrusion Detection: A Comprehensive Analysis,” J. Eng., vol. 2024, no. 1, p. 3909173, Jan. 2024, doi: 10.1155/2024/3909173.
A. H. Abdi et al., “Security Control and Data Planes of SDN: A Comprehensive Review of Traditional, AI, and MTD Approaches to Security Solutions,” IEEE Access, vol. 12, pp. 69941–69980, Jan. 2024, doi: 10.1109/ACCESS.2024.3393548.
Olakunle Abayomi Ajala, Chinwe Chinazo Okoye, Onyeka Chrisanctus Ofodile, Chuka Anthony Arinze, and Obinna Donald Daraojimba, “Review of AI and machine learning applications to predict and Thwart cyber-attacks in real-time,” Magna Sci. Adv. Res. Rev., vol. 10, no. 1, pp. 312–320, Feb. 2024, doi: 10.30574/msarr.2024.10.1.0037.
S. O. Ooko and S. M. Karume, “Application of Tiny Machine Learning in Predicative Maintenance in Industries,” J. Comput. Theor. Appl., vol. 2, no. 1, pp. 131–150, Aug. 2024, doi: 10.62411/jcta.10929.
A. Kumar, A. K. Singh, S. S. Ali, and B. J. Choi, “Expand and Shrink: Federated Learning with Unlabeled Data Using Clustering,” Sensors, vol. 23, no. 23, p. 9404, Jan. 2023, doi: 10.3390/s23239404.
S. P. Anh and Y. Nakamura, “A Baseline Investigation into the Evolution and Prevalence of Mirai and Hajime Utilizing a Network Telescope,” IEEE Access, Jan. 2024, [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10613408/
M. Kulbacki et al., “A Review of the Weaponization of IoT: Security Threats and Countermeasures,” Jan. 2024, pp. 279–284. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10619778/
H. Jin, G. Jeon, H. W. A. Choi, S. Jeon, and J. T. Seo, “A threat modeling framework for IoT-Based botnet attacks,” Heliyon, vol. 10, no. 20, Jan. 2024, [Online]. Available: https://www.cell.com/heliyon/fulltext/S2405-8440(24)15223-1
H. Almazarqi, “Profiling IoT botnet activity,” University of Glasgow, 2024. [Online]. Available: https://theses.gla.ac.uk/id/eprint/84102
M. Gelgi, Y. Guan, S. Arunachala, M. Samba Siva Rao, and N. Dragoni, “Systematic Literature Review of IoT Botnet DDOS Attacks and Evaluation of Detection Techniques,” Sensors, vol. 24, no. 11, p. 3571, Jun. 2024, doi: 10.3390/s24113571.
E.-M. Călin, “IoT and Critical Infrastructures: Technological Transformation And Security Implications,” in Proceedings-The 20th International Scientific Conference “Strategies XXI” Technologies–Military Applications, Simulation And Resources, Jan. 2024, pp. 386–394. [Online]. Available: https://www.ceeol.com/search/chapter-detail?id=1251750
P. Kumari and A. K. Jain, “A comprehensive study of DDoS attacks over IoT network and their countermeasures,” Comput. Secur., vol. 127, p. 103096, Apr. 2023, doi: 10.1016/j.cose.2023.103096.
K. B. Adedeji, A. M. Abu-Mahfouz, and A. M. Kurien, “DDoS Attack and Detection Methods in Internet-Enabled Networks: Concept, Research Perspectives, and Challenges,” J. Sens. Actuator Networks, vol. 12, no. 4, p. 51, Jul. 2023, doi: 10.3390/jsan12040051.
A. Pakmehr, A. Aßmuth, N. Taheri, and A. Ghaffari, “DDoS attack detection techniques in IoT networks: a survey,” Cluster Comput., vol. 27, no. 10, pp. 14637–14668, Jan. 2024, doi: 10.1007/s10586-024-04662-6.
R. Hallman, J. Bryan, G. Palavicini, J. Divita, and J. Romero-Mariona, “IoDDoS — The Internet of Distributed Denial of Sevice Attacks - A Case Study of the Mirai Malware and IoT-Based Botnets,” in Proceedings of the 2nd International Conference on Internet of Things, Big Data and Security, Jan. 2017, pp. 47–58. doi: 10.5220/0006246600470058.
N. Singh, R. Buyya, and H. Kim, “Securing Cloud-Based Internet of Things: Challenges and Mitigations,” Sensors, vol. 25, no. 1, p. 79, Jan. 2025, doi: 10.3390/s25010079.
E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, and A. A. Ghorbani, “CICIoT2023: A Real-Time Dataset and Benchmark for Large-Scale Attacks in IoT Environment,” Sensors, vol. 23, no. 13, p. 5941, Jun. 2023, doi: 10.3390/s23135941.
N. Koroniotis, N. Moustafa, E. Sitnikova, and J. Slay, “Towards Developing Network Forensic Mechanism for Botnet Activities in the IoT Based on Machine Learning Techniques,” in Mobile Networks and Management, vol. 235, J. Hu, I. Khalil, Z. Tari, and S. Wen, Eds. Cham: Springer International Publishing, 2018, pp. 30–44. doi: 10.1007/978-3-319-90775-8_3.
E. Hodo et al., “Threat analysis of IoT networks using artificial neural network intrusion detection system,” in 2016 International Symposium on Networks, Computers and Communications (ISNCC), May 2016, pp. 1–6. doi: 10.1109/ISNCC.2016.7746067.
J. K. Samriya, R. Tiwari, X. Cheng, R. K. Singh, A. Shankar, and M. Kumar, “Network intrusion detection using ACO-DNN model with DVFS based energy optimization in cloud framework,” Sustain. Comput. Informatics Syst., vol. 35, p. 100746, Sep. 2022, doi: 10.1016/j.suscom.2022.100746.
A. Deshmukh and K. Ravulakollu, “An Efficient CNN-Based Intrusion Detection System for IoT: Use Case Towards Cybersecurity,” Technologies, vol. 12, no. 10, p. 203, Jan. 2024, doi: 10.3390/technologies12100203.
A. Javad et al., “Leveraging Convolutional Neural Network (CNN)-based Auto Encoders for Enhanced Anomaly Detection in High-Dimensional Datasets,” Eng. Technol. Appl. Sci. Res., vol. 14, no. 6, p. 17894, Jan. 2024, [Online]. Available: https://engagedscholarship.csuohio.edu/bus_facpub/356/
R. A. Disha and S. Waheed, “A Comparative study of machine learning models for Network Intrusion Detection System using UNSW-NB 15 dataset,” in 2021 International Conference on Electronics, Communications and Information Technology (ICECIT), Jan. 2021, pp. 1–5. doi: 10.1109/ICECIT54077.2021.9641471.
A. Shehadeh, H. ALTaweel, and A. Qusef, “Analysis of Data Mining Techniques on KDD-Cup’99, NSL-KDD and UNSW-NB15 Datasets for Intrusion Detection,” in 2023 24th International Arab Conference on Information Technology (ACIT), Jan. 2023, pp. 1–6. doi: 10.1109/ACIT58888.2023.10453884.
F. Türk, “Analysis of Intrusion Detection Systems in UNSW-NB15 and NSL-KDD Datasets with Machine Learning Algorithms,” Bitlis Eren Üniversitesi Fen Bilim. Derg., vol. 12, no. 2, pp. 465–477, Jan. 2023, doi: 10.17798/bitlisfen.1240469.
S. Choudhary and N. Kesswani, “Analysis of KDD-Cup’99, NSL-KDD and UNSW-NB15 datasets using deep learning in IoT,” Procedia Comput. Sci., vol. 167, pp. 1561–1573, Jan. 2020, [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1877050920308334
G. Kocher and G. Kumar, “Analysis of Machine Learning Algorithms with Feature Selection for Intrusion Detection Using UNSW-NB15 Dataset.” Social Science Research Network, Rochester, NY, Jan. 21, 2021. doi: 10.2139/ssrn.3784406.
A. Dickson and C. Thomas, “Analysis of UNSW-NB15 Dataset Using Machine Learning Classifiers,” 2021, pp. 198–207. doi: 10.1007/978-981-16-0419-5_16.
S. Kumar and D. N. K. Pathak, “Evaluation Of Machine Learning Algorithms For Intrusion Detection Utilizing UNSW-NB15 Dataset,” J. Pharm. Negat. Results, pp. 4819–4832, Jan. 2022, doi: 10.47750/pnr.2022.13.S08.630%20.
K. Pramilarani and P. Vasanthi Kumari, “Cost based Random Forest Classifier for Intrusion Detection System in Internet of Things,” Appl. Soft Comput., vol. 151, p. 111125, Jan. 2024, doi: 10.1016/j.asoc.2023.111125.
M. Ge, J. B. Hong, W. Guttmann, and D. S. Kim, “A framework for automating security analysis of the internet of things,” J. Netw. Comput. Appl., vol. 83, pp. 12–27, Apr. 2017, doi: 10.1016/j.jnca.2017.01.033.
M. S. Mahdavinejad, M. Rezvan, M. Barekatain, P. Adibi, P. Barnaghi, and A. P. Sheth, “Machine learning for internet of things data analysis: a survey,” Digit. Commun. Networks, vol. 4, no. 3, pp. 161–175, Aug. 2018, doi: 10.1016/j.dcan.2017.10.002.
M. Baich, T. Hamim, N. Sael, and Y. Chemlal, “Machine Learning for IoT based networks intrusion detection: a comparative study,” Procedia Comput. Sci., vol. 215, pp. 742–751, Jan. 2022, doi: 10.1016/j.procs.2022.12.076.
S. V. N. Santhosh Kumar, M. Selvi, and A. Kannan, “A Comprehensive Survey on Machine Learning‐Based Intrusion Detection Systems for Secure Communication in Internet of Things,” Comput. Intell. Neurosci., vol. 2023, no. 1, p. 8981988, Jan. 2023, doi: 10.1155/2023/8981988.
A. Sajid, H. Abbas, and K. Saleem, “Cloud-Assisted IoT-Based SCADA Systems Security: A Review of the State of the Art and Future Challenges,” IEEE Access, vol. 4, pp. 1375–1384, Sep. 2016, doi: 10.1109/ACCESS.2016.2549047.
G. Samara et al., “A Comprehensive Review of Machine Learning-Based Intrusion Detection Techniques for IoT Networks,” in Artificial Intelligence, Internet of Things, and Society 5.0, vol. 1113, A. Hannoon and A. Mahmood, Eds. Cham: Springer Nature Switzerland, 2023, pp. 465–473. doi: 10.1007/978-3-031-43300-9_38.
B. Xu, L. Sun, X. Mao, R. Ding, and C. Liu, “IoT Intrusion Detection System Based on Machine Learning,” Electronics, vol. 12, no. 20, p. 4289, Oct. 2023, doi: 10.3390/electronics12204289.
C. Yin, Y. Zhu, J. Fei, and X. He, “A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks,” IEEE Access, vol. 5, pp. 21954–21961, Jan. 2017, doi: 10.1109/ACCESS.2017.2762418.
I. Mbona and J. H. P. Eloff, “Detecting Zero-Day Intrusion Attacks Using Semi-Supervised Machine Learning Approaches,” IEEE Access, vol. 10, pp. 69822–69838, Jan. 2022, doi: 10.1109/ACCESS.2022.3187116.
J. M. Beaver, R. C. Borges-Hink, and M. A. Buckner, “An Evaluation of Machine Learning Methods to Detect Malicious SCADA Communications,” in 2013 12th International Conference on Machine Learning and Applications, Dec. 2013, vol. 2, pp. 54–59. doi: 10.1109/ICMLA.2013.105.
C. S. Htwe, Z. T. T. Myint, and Y. M. Thant, “IoT Security Using Machine Learning Methods with Features Correlation,” J. Comput. Theor. Appl., vol. 2, no. 2, pp. 151–163, Aug. 2024, doi: 10.62411/jcta.11179.
D. R. I. M. Setiadi, H. M. M. Islam, G. A. Trisnapradika, and W. Herowati, “Analyzing Preprocessing Impact on Machine Learning Classifiers for Cryotherapy and Immunotherapy Dataset,” J. Futur. Artif. Intell. Technol., vol. 1, no. 1, pp. 39–50, Jun. 2024, doi: 10.62411/faith.2024-2.
M. I. Akazue, I. A. Debekeme, A. E. Edje, C. Asuai, and U. J. Osame, “UNMASKING FRAUDSTERS: Ensemble Features Selection to Enhance Random Forest Fraud Detection,” J. Comput. Theor. Appl., vol. 1, no. 2, pp. 201–211, Dec. 2023, doi: 10.33633/jcta.v1i2.9462.
D. R. Chirra, “Deep Learning Techniques for Anomaly Detection in IoT Devices: Enhancing Security and Privacy,” Rev. Intel. Artif. en Med., vol. 14, no. 1, pp. 529–552, Jan. 2023, [Online]. Available: http://redcrevistas.com/index.php/Revista/article/view/214
A. Jaramillo-Alcazar, J. Govea, and W. Villegas-Ch, “Anomaly Detection in a Smart Industrial Machinery Plant Using IoT and Machine Learning,” Sensors, vol. 23, no. 19, p. 8286, Oct. 2023, doi: 10.3390/s23198286.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Jean Pierre Ntayagabiri, Youssef Bentaleb, Jeremie Ndikumagenge, Hind El Makhtoum
This work is licensed under a Creative Commons Attribution 4.0 International License.
