Particle Swarm Optimization of Deep Auto-Encoder Network Architectures for Anomaly-based Intrusion Detection Systems

Deep Auto-Encoder (DAE) is a neural network model that is commonly used to develop anomaly-based network intrusion detection systems. However, the architecture of existing models are all arbitrary designs with no theoretical basis or empirical formula. As a result, it will cost computational and storage resources as well as not being able to learn the best data representation space. In this paper, we propose a novel method based on particle swarm optimization (PSO) for searching the optimal architecture of DAE for anomaly-based network intrusion detection systems, named PSODAE. Our proposed model can find the optimal number of neurons on the hidden layers of DAE. Therefore, the meaningful, prominent, concise latent representation space can be learned from network data, which better supports the performance of network anomaly detectors. We conduct experiments using standard datasets including NSLKDD, UNSW-NB15, CICIDS-2017, and five scenarios in dataset CTU13. The experimental results have generated the optimal architecture of DAE models on each dataset. This study proposes an effective way of designing deep learning network architecture to develop modern network intrusion detection systems. © 2023 IEEE.