Adaptive Cyber Defense using Advanced Deep Reinforcement Learning Algorithms: A Real-Time Comparative Analysis

Atheer Alaa Hammad; Firas Tarik Jasim

doi:10.62411/jcta.12560

Authors

Atheer Alaa Hammad Ministry of Education, Anbar Education Directorate
Firas Tarik Jasim Al-Dour Technical Institute

DOI:

https://doi.org/10.62411/jcta.12560

Keywords:

A3C, Adaptive Defense, Cybersecurity, Deep Reinforcement Learning, Intrusion Detection Systems, Network Security, SAC

Abstract

Cybersecurity is continuously challenged by increasingly sophisticated and dynamic cyber-attacks, necessitating advanced adaptive defense mechanisms. Deep Reinforcement Learning (DRL) has emerged as a promising approach, offering significant advantages over traditional intrusion detection methods through real-time adaptability and self-learning capabilities. This paper presents an advanced adaptive cybersecurity framework utilizing five prominent DRL algorithms: Deep Q-Network (DQN), Proximal Policy Optimization (PPO), Twin Delayed DDPG (TD3), Soft Actor-Critic (SAC), and Asynchronous Advantage Actor-Critic (A3C). The effectiveness of these algorithms is evaluated within complex, realistic simulation environments using live-streaming data, emphasizing key metrics such as accuracy (AUC-ROC), response latency, and network throughput. Experimental results demonstrate that the SAC algorithm consistently achieves superior detection accuracy (95% AUC-ROC) and minimal disruption to network performance compared to other approaches. Additionally, A3C provides the fastest response times suitable for real-time defense scenarios. This comprehensive comparative analysis addresses critical research gaps by integrating both traditional and novel DRL techniques and validates their potential to substantially improve cybersecurity defense strategies in realistic operational settings.

Author Biographies

Atheer Alaa Hammad, Ministry of Education, Anbar Education Directorate

Ministry of Education, Anbar Education Directorate, Al Anbar, Iraq

Firas Tarik Jasim, Al-Dour Technical Institute

Department of Medical Instrumentation Technique, Northern Technical University, Al-Dour Technical Institute, Iraq

References

A. Çetin and S. Öztürk, “Comprehensive Exploration of Ensemble Machine Learning Techniques for IoT Cybersecurity Across Multi-Class and Binary Classification Tasks,” J. Futur. Artif. Intell. Technol., vol. 1, no. 4, pp. 371–384, Feb. 2025, doi: 10.62411/faith.3048-3719-51.

J. P. Ntayagabiri, Y. Bentaleb, J. Ndikumagenge, and H. El Makhtoum, “OMIC: A Bagging-Based Ensemble Learning Framework for Large-Scale IoT Intrusion Detection,” J. Futur. Artif. Intell. Technol., vol. 1, no. 4, pp. 401–416, Feb. 2025, doi: 10.62411/faith.3048-3719-63.

C. S. Htwe, Z. T. T. Myint, and Y. M. Thant, “IoT Security Using Machine Learning Methods with Features Correlation,” J. Comput. Theor. Appl., vol. 2, no. 2, pp. 151–163, Aug. 2024, doi: 10.62411/jcta.11179.

M. A. Rahman, G. A. Francia, and H. Shahriar, “Leveraging GANs for Synthetic Data Generation to Improve Intrusion Detection Systems,” J. Futur. Artif. Intell. Technol., vol. 1, no. 4, pp. 429–439, Feb. 2025, doi: 10.62411/faith.3048-3719-52.

P. H. Hussan and S. M. Mangj, “BERTPHIURL : A Teacher-Student Learning Approach Using DistilRoBERTa and RoBERTa for Detecting Phishing Cyber URLs,” J. Futur. Artif. Intell. Technol., vol. 1, no. 4, 2025, doi: 10.62411/faith.3048-3719-71.

A. Alaa Hammad, M. Adnan Falih, S. Ali Abd, and A. Rashid Ahmed, “Detecting Cyber Threats in IoT Networks: A Machine Learning Approach,” Int. J. Comput. Digit. Syst., vol. 17, no. 1, pp. 1–25, Jan. 2025, doi: 10.12785/ijcds/1571020041.

N. Khlif, N. Khraief, and S. Belghith, “Comparative Analysis of Modified Q-Learning and DQN for Autonomous Robot Navigation,” J. Futur. Artif. Intell. Technol., vol. 1, no. 3, pp. 296–308, Dec. 2024, doi: 10.62411/faith.3048-3719-49.

M. A. Setiawan, D. R. I. M. Setiadi, E. Z. Astuti, T. Sutojo, and N. A. Setiyanto, “Exploring Deep Q-Network for Autonomous Driving Simulation Across Different Driving Modes,” J. Futur. Artif. Intell. Technol., vol. 1, no. 3, pp. 217–227, Oct. 2024, doi: 10.62411/faith.3048-3719-31.

S. Nugroho, D. R. I. M. Setiadi, and H. M. M. Islam, “Exploring DQN-Based Reinforcement Learning in Autonomous Highway Navigation Performance Under High-Traffic Conditions,” J. Comput. Theor. Appl., vol. 1, no. 3, pp. 274–286, Feb. 2024, doi: 10.62411/jcta.9929.

A. Pathirana et al., “A Reinforcement Learning-Based Approach for Promoting Mental Health Using Multimodal Emotion Recognition,” J. Futur. Artif. Intell. Technol., vol. 1, no. 2, pp. 124–142, Sep. 2024, doi: 10.62411/faith.2024-22.

A. A. Hammad, S. R. Ahmed, M. K. Abdul-Hussein, M. R. Ahmed, D. A. Majeed, and S. Algburi, “Deep Reinforcement Learning for Adaptive Cyber Defense in Network Security,” in Proceedings of the Cognitive Models and Artificial Intelligence Conference, May 2024, pp. 292–297. doi: 10.1145/3660853.3660930.

H. Shen, K. Zhang, M. Hong, and T. Chen, “Towards Understanding Asynchronous Advantage Actor-Critic: Convergence and Linear Speedup,” IEEE Trans. Signal Process., vol. 71, pp. 2579–2594, 2023, doi: 10.1109/TSP.2023.3268475.

J. Adamczyk, V. Makarenko, S. Tiomkin, and R. V. Kulkarni, “Average-Reward Reinforcement Learning with Entropy Regularization,” arXiv. Jan. 15, 2025. [Online]. Available: http://arxiv.org/abs/2501.09080

Israa Saad Mohammed, “DCITD: A Deep Q-Network Approach for Cyber Image Threats Detection,” J. Inf. Syst. Eng. Manag., vol. 10, no. 17s, pp. 436–449, Mar. 2025, doi: 10.52783/jisem.v10i17s.2748.

W. Yang, A. Acuto, Y. Zhou, and D. Wojtczak, “A Survey for Deep Reinforcement Learning Based Network Intrusion Detection,” arXiv. Sep. 25, 2024. [Online]. Available: http://arxiv.org/abs/2410.07612

Y. Badr, “Enabling intrusion detection systems with dueling double deep Q -learning,” Digit. Transform. Soc., vol. 1, no. 1, pp. 115–141, Aug. 2022, doi: 10.1108/DTS-05-2022-0016.

E. Walter, K. Ferguson-Walter, and A. Ridley, “Incorporating Deception into CyberBattleSim for Autonomous Defense,” arXiv. Aug. 31, 2021. [Online]. Available: http://arxiv.org/abs/2108.13980

F. M. Zennaro and L. Erdődi, “Modelling penetration testing with reinforcement learning using capture‐the‐flag challenges: Trade‐offs between model‐free learning and a priori knowledge,” IET Inf. Secur., vol. 17, no. 3, pp. 441–457, May 2023, doi: 10.1049/ise2.12107.

Q. Yao, Y. Wang, X. Xiong, P. Wang, and Y. Li, “Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach,” Entropy, vol. 25, no. 4, p. 605, Apr. 2023, doi: 10.3390/e25040605.

C. Lei, D.-H. Ma, and H.-Q. Zhang, “Optimal Strategy Selection for Moving Target Defense Based on Markov Game,” IEEE Access, vol. 5, pp. 156–169, 2017, doi: 10.1109/ACCESS.2016.2633983.

E. Muhati and D. B. Rawat, “Asynchronous Advantage Actor-Critic (A3C) Learning for Cognitive Network Security,” in 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), Dec. 2021, pp. 106–113. doi: 10.1109/TPSISA52974.2021.00012.

L. Chavali, T. Gupta, and P. Saxena, “SAC-AP: Soft Actor Critic based Deep Reinforcement Learning for Alert Prioritization,” arXiv. Jul. 27, 2022. [Online]. Available: http://arxiv.org/abs/2207.13666

R. Ozalp, A. Ucar, and C. Guzelis, “Advancements in Deep Reinforcement Learning and Inverse Reinforcement Learning for Robotic Manipulation: Toward Trustworthy, Interpretable, and Explainable Artificial Intelligence,” IEEE Access, vol. 12, pp. 51840–51858, 2024, doi: 10.1109/ACCESS.2024.3385426.

D. S. Diop, S. Y. Luis, M. P. Esteve, S. L. T. Marín, and D. G. Reina, “Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach,” IEEE Access, vol. 12, pp. 75559–75576, 2024, doi: 10.1109/ACCESS.2024.3403790.

K. Ohashi, K. Nakanishi, N. Goto, Y. Yasui, and S. Ishii, “Orthogonal Adversarial Deep Reinforcement Learning for Discrete- and Continuous-Action Problems,” IEEE Access, vol. 12, pp. 151907–151919, 2024, doi: 10.1109/ACCESS.2024.3479089.

M. Yavuz and Ö. C. Kivanç, “Optimization of a Cluster-Based Energy Management System Using Deep Reinforcement Learning Without Affecting Prosumer Comfort: V2X Technologies and Peer-to-Peer Energy Trading,” IEEE Access, vol. 12, pp. 31551–31575, 2024, doi: 10.1109/ACCESS.2024.3370922.

T. Zhou, Y. Yakuwa, N. Okamura, H. Hochigai, T. Kuroda, and I. Eguchi Yairi, “Dueling Network Architecture for GNN in the Deep Reinforcement Learning for the Automated ICT System Design,” IEEE Access, vol. 13, pp. 21870–21879, 2025, doi: 10.1109/ACCESS.2025.3534246.