基于自注意力混合模型的电力物联网流量分类

Traffic Classification of Power IoT Based on Self-attention Hybrid Model

王聪，郑海杰，黄振，王高洲，曲海鹏^*

WANG Cong，ZHENG Haijie，HUANG Zhen，WANG Gaozhou，QU Haipeng^*

1.国网山东省电力公司信息通信公司，山东济南250013

2.中国海洋大学信息科学与工程学部，山东青岛266100

摘要(Abstract)：

网络流量分类是网络监控和分析的关键环节，用于恶意流量拦截、服务质量保证、应用瓶颈预防和恶意行为识别等。当前，在电力物联网设备应用场景中，对Modbus和消息队列遥测传输（message queuing telemetry transport,MQTT）等通信协议的网络流量进行分类时，面临着准确率低、收敛慢等挑战。针对上述问题，提出了一种基于自注意力机制的卷积神经网络-循环神经网络（convolutional neural network-recurrent neural network,CNN-RNN）混合网络架构，用于改进电力物联网设备中Modbus和MQTT通信流量的分类性能。通过模拟电力物联网环境并采集真实环境的物联网流量，获取大量Modbus和MQTT通信数据包，将流量数据转化为伪图像格式，并引入自注意力机制来增强网络对不同区域的关注和特征捕获能力。实验结果表明，相较于传统的多层感知机（multilayer perceptron,MLP）、RNN、CNN模型和现有论文中的方案，引入自注意力机制的CNN-RNN混合模型在电力物联网流量分类方面取得了显著的改进。该模型能够实现高达95%的分类准确率，并且具有更快的收敛速度和训练效率。Network traffic classification is an important part of network monitoring and analysis,which is used for malicious traffic interception,quality of service assurance,application bottleneck prevention and malicious behavior identification.Recently,there are great challenges on the low accuracy and slow convergence problems,when classifying network traffic generated by communication protocols like Modbus and message queuing telemetry transport(MQTT)in the context of power IoT devices applications.To address aforementioned issues,a convolutional neural network-recurrent neural network(CNNRNN)hybrid architecture based on self-attention mechanism is proposed to improve the classification performance of Modbus and MQTT traffic for IoT devices that already in use within the power system.By simulating and collecting the network traffic of power IoT device in the real environment,a large number of Modbus and MQTT communication data packets are obtained.Additionally,the traffic data is converted into pseudo-image format,and a self-attention mechanism is introduced to enhance the network′s attention and feature capture capabilities in different regions.The experimental results show that,compared with traditional multilayer perceptron(MLP),RNN,CNN models,the proposed CNN-RNN hybrid model with self-attention mechanism demonstrates achieved significant improvement in IoT traffic classification.The model can achieve up to 95%accuracy,demonstrating better convergence and training efficiency.

关键词(KeyWords)：深度学习；自注意力机制；网络安全；电力物联网；流量分类

deep learning；self-attention mechanism；cyber security；power Internet of Things；traffic classification

基金项目(Foundation): 国网山东省电力公司科技项目（520627220005）
Science and Technology Project of State Grid Shandong Electric Power Company（520627220005）

作者(Author): 王聪，郑海杰，黄振，王高洲，曲海鹏^*

WANG Cong，ZHENG Haijie，HUANG Zhen，WANG Gaozhou，QU Haipeng^*

DOI: 10.20097/j.cnki.issn1007-9904.2025.06.007

收稿日期(Received): 2024-06-02; 修回日期(Revised): 2024-10-24

参考文献(References)：

[1]ALI O,ISHAK M K,BHATTI M K L,et al.A comprehensive review of Internet of Things:technology stack,middlewares,and fog/edge computing interface[J].Sensors:Basel,Switzerland,2022,22(3):995.

[2]HOU R,REN G W,ZHOU C L,et al.Analysis and research on network security and privacy security in ubiquitous electricity Internet of Things[J].Computer Communications,2020,158:64-72.

[3]白尚旺，王梦瑶，胡静，等.多区域注意力的细粒度图像分类网络[J].计算机工程，2024,50(1):271-278.

BAI Shangwang,WANG Mengyao,HU Jing,et al.Multi-region attention network for fine-grained image classification[J].Computer Engineering,2024,50(1):271-278.

[4]于治平，刘彩霞，刘树新，等.基于机器学习的网络流量分类综述[J].信息工程大学学报，2023,24(4):447-453.

YU Zhiping,LIU Caixia,LIU Shuxin,et al.Overview of network traffic classification based on machine learning[J].Journal of Information Engineering University,2023,24(4):447-453.

[5]冷涛.基于深度学习的加密流量分类研究综述[J].计算机与现代化，2021(8):112-120.

LENG Tao.A survey of encrypted traffic classification based on deep learning[J].Computer and Modernization,2021(8):112-120.

[6]XIE X,YANG B,CHEN Y H,et al.Network traffic classification based on error-correcting output codes and NN ensemble[C]⫽2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery.IEEE,2009:475-479.

[7]FENG W B,HONG Z,WU L F,et al.Network protocol recognition based on convolutional neural network[J].China Communications,2020,17(4):125-139.

[8]IZADI S,AHMADI M,NIKBAZM R.Network traffic classification using convolutional neural network and ant-lion optimization[J].Computers and Electrical Engineering,2022,101:108024.

[9]STRYCZEK S,NATKANIEC M.Internet threat detection in smart grids based on network traffic analysis using LSTM,IF,and SVM[J].Energies,2022,16(1):329.

[10]ANDRESINI G,APPICE A,CAFORIO F P,et al.ROULETTE:a neural attention multi-output model for explainable network intrusion detection[J].Expert Systems with Applications,2022,201:117144.

[11]YU J,YE X J,LI H B.A high precision intrusion detection system for network security communication based on multi-scale convolutional neural network[J].Future Generation Computer Systems,2022,129:399-406.

[12]张昊，张小雨，张振友，等.基于深度学习的入侵检测模型综述[J].计算机工程与应用，2022,58(6):17-28.

ZHANG Hao,ZHANG Xiaoyu,ZHANG Zhenyou,et al.Summary of intrusion detection models based on deep learning[J].Computer Engineering and Applications,2022,58(6):17-28.

[13]SILVA C R M,SILVA F A C M.An IoT gateway for modbus and MQTT integration[C]?2019 SBMO/IEEE MTT-S International Microwave and Optoelectronics Conference(IMOC).IEEE,2019:1-3.

[14]CAUDILL M.Neural networks primer,part I[J].AI Expert,1987,2(12):46-52.

[15]季长清，高志勇，秦静，等.基于卷积神经网络的图像分类算法综述[J].计算机应用，2022,42(4):1044-1049.

JI Changqing,GAO Zhiyong,QIN Jing,et al. Review of image classification algorithms based on convolutional neural network[J].Journal of Computer Applications,2022,42(4):1044-1049.

[16]LIPTON Z C,BERKOWITZ J,ELKAN C,et al.A critical review of recurrent neural networks for sequence learning[EB/OL].[2024-05-30].https：?arxiv.org/abs/1506.00019v4

[17]赫磊，邵展鹏，张剑华，等.基于深度学习的行为识别算法综述[J].计算机科学，2020,47（增刊1）:139-147.

HE Lei,SHAO Zhanpeng,ZHANG Jianhua,et al. A survey of behavior recognition algorithms based on deep learning[J].Computer Science,2020,47(S1):139-147.

[18]FU E,ZHANG Y N,YANG F,et al.Temporal self-attention-based Conv-LSTM network for multivariate time series prediction[J].Neurocomputing,2022,501:162-173.

[19]LECUN Y,BOSER B,DENKER J S,et al.Backpropagation applied to handwritten zip code recognition[J].Neural Computation,1989,1(4):541-551.

[20]LEE C Y,GALLAGHER P W,TU Z. Generalizing pooling functions in convolutional neural networks:mixed,gated,and tree[C]⫽Artificial Intelligence and Statistics.PMLR,2016:464-472.

[21]IOFFE S,SZEGEDY C.Batch normalization:accelerating deep network training by reducing internal covariate shift[J].32nd International Conference on Machine Learning,ICML 2015,2015,1:448-456.

[22]MEHTA K,JOSHI R,KULKARNI S V,et al.Implementation of plc based software prototype for 45.6 mhz,100 kw,icrh dac using epics control system[J].International Journal of Scientific and Engineering Research,2015,6(5):1134-1137.

[23]BERKAY CELIK Z,WALLS R J,MCDANIEL P,et al. Malware traffic detection using tamper resistant features[C]⫽MILCOM2015-2015 IEEE Military Communications Conference. IEEE,2015:330-335.

[24]LONGADGE R,DONGRE S.Class imbalance problem in data mining review[EB/OL].[2024-05-30]. https：?arxiv. org/abs/1305.1707v1

[25]SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:a simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15:1929-1958.

[26]CHOLLET F. Keras[R].(2020)[2023.10]. Available:https：?github.com/fchollet/keras.

[27]KINGMA D P,BA J,HAMMAD M M.Adam:a method for stochastic optimization[EB/OL].[2024-05-30].https：?arxiv.org/abs/1412.6980v9

[28]KB T,SURESH M,RAO S.A review on deep learning approaches to real time network Intrusion detection system[C]⫽Institute of Scholars,August 7,2020.

[29]SINGH A K.A comparative study on disease classification using machine learning algorithms[C]⫽Proceedings of 2nd International Conference on Advanced Computing and Software Engineering(ICACSE).SSRN,2019:197-202.

[30]MOUSTAFA N,SLAY J.UNSW-NB15:a comprehensive data set for network intrusion detection systems(UNSW-NB15 network data set)[C]⫽2015 Military Communications and Information Systems Conference(MilCIS).IEEE,2015:1-6.

[31]TAVALLAEE M,BAGHERI E,LU W,et al.A detailed analysis of the KDD CUP 99 data set[C]⫽2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.IEEE,2009:1-6.

[32]孙水发，李小龙，李伟生，等.图神经网络应用于知识图谱推理的研究综述[J].计算机科学与探索，2023,17(1):27-52.

SUN Shuifa,LI Xiaolong,LI Weisheng,et al.Review of graph neural networks applied to knowledge graph reasoning[J].Journal of Frontiers of Computer Science and Technology,2023,17(1):27-52.

[33]BABARIA R, MADANAPALLI S C, KUMAR H, et al.FlowFormers:transformer-based models for real-time network flow classification[C]⫽2021 17th International Conference on Mobility,Sensing and Networking(MSN).IEEE,2021:231-238.

[34]MICHELENAá，DíAZ-LONGUEIRA A,TIMIRAOS M,et al.Hybrid classification model based on supervised techniques for denial of service attacks detection over CoAP protocol[C]⫽Lecture Notes in Networks and Systems.Cham:Springer Nature Switzerland,2023:1-10.