Определение эффективности обнаружения DNS-туннелей при помощи нейронной сети в системе обнаружения вторжений

Труды Института системного программирования РАН. 2026. Т. 38. № 4. С. 123–142.

Marinin N., Getman A.

This paper investigates a method for detecting DNS tunnels in network traffic using a neural network.
To achieve this, an analysis of current detection approaches was conducted, and a dataset for neural network
training was prepared. The proposed model takes as input a sequence of characters extracted from DNS
responses. The trained model demonstrated an F1-score close to one. To validate the proposed approach,
modules of the open-source intrusion detection system Snort 3 were extended and modified. The trained model
was executed using the compatible LibML module. Experimental results demonstrate detection accuracy close
to one with an almost complete absence of false positives. The average DNS packet processing time with the
neural network-based detection module enabled increased by 13%, while for mixed traffic consisting of
multiple protocols, the processing time increased by only 2%. The analysis of the experimental data confirms
that the use of neural networks effectively complements traditional security mechanisms, enabling efficient
detection of covert channels encapsulated within DNS traffic without significantly affecting the performance
of the signature-based detection engine.

Keywords: нейронная сеть, intrusion detection system, система обнаружения вторжений, covert channels, Convolutional neural network (CNN), DNS, DNS, DNS tunneling, DNS-туннелирование, скрытые каналы