An IIoT Temporal Data Anomaly Detection Method Combining Transformer and Adversarial Training

Prediction Based Methods

When detecting temporal anomaly data, using the Recurrent Neural Network (RNN) and the variants network was a common approach. For the simplification and lightweight consideration of the model structure, Tran et al. (2022) introduced an anomaly detection model based on self-supervised learning (SSL) to analyze the temporal data. This model has a certain effect on detecting the degree of data anomalies. In addition, a convolutional encoder was applied to capture the correlation of input data. Shen et al. (2020) implemented the detection of abnormal data based on LSTM and GRU. By introducing multiple gating units, long-term dependency relationships have been learned to better achieve long-sequence data processing and anomaly detection. Zhang et al. (2022) proposed an architecture with time-frequency analysis for anomaly detection (TFAD). This model contains modules for data decomposition and expansion, which can improve accuracy to a certain extent. Meng et al. (2020) applied Transformer to data streams by masking temporal data modeling and proposed a masking strategy for anomaly detection using an attention mechanism that includes parallel update time steps. Deng & Hooi (2021) proposed a model based on graph neural networks, which can represent temporal data as graph data and then perform anomaly detection by learning feature vectors of nodes and edges. Zhou et al. (2021) introduced a model based on Transformer: Informer. The computational complexity has been improved because of the attention network, and the prediction progress and long sequence prediction efficiency have been improved.

The above prediction-based data anomaly detection methods can predict future data points by establishing mathematical models or algorithms and comparing actual data with predicted data to detect outliers. These methods have a certain degree of predictability, adaptability, and interpretability. But at the same time, there are also obvious drawbacks, such as high requirements for data volume and high computational costs. In addition, if the data distribution changes, the model may need to be adjusted or retrained. Therefore, it is necessary to consider issues such as data quality, computational cost, and model drift during use. When selecting and using this anomaly detection method, it is necessary to weigh the specific application scenarios and requirements.