Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization

This paper tackles a central obstacle for anomaly-based intrusion detection systems: the lack of adequate, up-to-date datasets. The authors study eleven publicly available datasets released since 1998, such as DARPA98, KDD99, ISC2012, and ADFA13, and find many outdated or unreliable, suffering from limited traffic diversity and volume, missing attack varieties, anonymized payloads, or incomplete feature sets and metadata. To address this, they generate a new dataset containing benign traffic plus seven common network attack flows that meets real-world criteria and is publicly available.

Using this dataset, the paper evaluates a comprehensive set of network traffic features together with machine learning algorithms to indicate which feature sets best detect particular attack categories. By providing a realistic, publicly available benchmark and identifying effective features per attack type, the work directly addresses the dataset shortage that had left anomaly-based approaches suffering from inaccurate deployment, analysis, and evaluation.