De-Obfuscating Emerging Malware Threats Using Reverse Engineering Techniques

Abstract: The rapid evolution of malware poses a significant cybersecurity challenge, as attackers increasingly employ sophisticated obfuscation techniques to evade detection. Polymorphic and metamorphic malware utilise different obfuscation techniques such as packing, encryption, and code mutation to evade traditional signature-based detection models. Conventional static and dynamic analysis tools frequently struggle to de-obfuscate these threats. This paper proposes a novel system that integrates reverse engineering techniques with ensemble learning models containing Gradient Boosting Machines, Convolutional Neural Networks, and Gated Recurrent Units, to address these challenges. Reverse engineering enables in-depth structural and behavioural analysis of malicious binaries, exposing hidden payloads and execution patterns. Ensemble learning enhances detection by combining the strengths of multiple algorithms to improve accuracy and adaptability. The proposed system not only identifies obfuscated malware with high precision but also predicts emerging variants, offering resilience against evasion tactics. By uniting explainable reverse engineering with advanced ensemble learning, the system provides scalable, real-time protection against evolving malware threats.

Malware Reverse Engineering

1. Chen, X., Li, Y., Zhang, T., & Wang, H. (2021). Malware obfuscation techniques and their impact on detection: A survey. Journal of Information Security and Applications, 58, 102–115. https://doi.org/10.1016/j.jisa.2020.102615 [Google Scholar] [Crossref]

2. Christodorescu, M., Jha, S., & Seshia, S. A. (2005). Semantics-aware malware detection. IEEE Symposium on Security and Privacy, 32–46. https://doi.org/10.1109/SP.2005.20 [Google Scholar] [Crossref]

3. Li, Y., Chen, H., Xu, J., & Zhao, Y. (2024). PowerPeeler: A dynamic framework for de-obfuscating malicious PowerShell scripts. Computers & Security, 138, 103590. https://doi.org/10.1016/j.cose.2024.103590 [Google Scholar] [Crossref]

4. Patsakis, C., Mermigas, D., & Fotiadis, P. (2024). Leveraging large language models for malware de-obfuscation and threat intelligence. Future Generation Computer Systems, 156, 18–32. https://doi.org/10.1016/j.future.2024.01.005 [Google Scholar] [Crossref]

5. Rahman, M., Saha, R., & Ahmed, T. (2023). Enhancing adversarial robustness of malware detectors with GAN-based augmentation. Expert Systems with Applications, 229, 120573. https://doi.org/10.1016/j.eswa.2023.120573 [Google Scholar] [Crossref]

6. Salehi, F., Sharif, H., & Kaur, S. (2020). Metamorphic malware detection using machine learning techniques. Journal of Computer Virology and Hacking Techniques, 16(2), 101–114. https://doi.org/10.1007/s11416-019-00349-2 [Google Scholar] [Crossref]

7. Ucci, D., Aniello, L., & Baldoni, R. (2019). Survey of machine learning techniques for malware analysis. Computers & Security, 81, 123–147. https://doi.org/10.1016/j.cose.2018.11.001 [Google Scholar] [Crossref]

8. Wu, J., Zhang, L., & Huang, M. (2023). Reinforcement learning for adaptive malware detection in dynamic environments. IEEE Transactions on Information Forensics and Security, 18, 2456–2469. https://doi.org/10.1109/TIFS.2023.3245678 [Google Scholar] [Crossref]

9. Zhou, K., Chen, Y., & Lin, Z. (2022). Integrating reverse engineering and generative adversarial networks for robust malware detection. ACM Transactions on Privacy and Security, 25(4), 1–25. https://doi.org/10.1145/3518997 [Google Scholar] [Crossref]

• Decision Tree and Automatic Linear Modeling Approaches to Predict Body Weight in Indigenous Sabi Sheep and Matebele Goat Females of Zimbabwe
• Microcontroller – Based Automatic Railway Crossing Control and Track Obstacle Monitoring System
• Modelling the Role of Absorptive Capacity in Foreign Direct Investment - Economic Growth Nexus: A Focus on South Africa’s Manufacturing Sector
• AI-Powered Wristband for Accurate BAC Monitoring Using Smart Data Fusion
• Climate Change Impacts on Different Regions