Self-Supervised Transformers for Early Fault Detection in High-Velocity Authorization Streams
Keywords:
self-supervised learning, transformers, anomaly detection, edge computing, fault prediction, microsecond latencyAbstract
Real-time authorisation systems require ultra-low-latency fault prediction to make sure five-nines availability in uncertain traffic changes. This objective of this paper is to introduce a self-supervised transformer architecture for high-velocity authorisation streams that can detect non-linear temporal connections and micro-anomalies within 200 microseconds. The proposed model independently acquires systemic integrity latent structures by simulating standard temporal cycles in the absence of failure evidence.
Downloads
References
A. Vaswani et al., "Attention is All You Need," Advances in Neural Information Processing Systems (NeurIPS), pp. 5998–6008, 2017.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," arXiv preprint arXiv:1810.04805, 2018.
Z. Liu et al., "Self-supervised Learning: Generative or Contrastive," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 8, pp. 1844–1859, Aug. 2019.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep Learning," Nature, vol. 521, no. 7553, pp. 436–444, May 2015.
C. Doersch, "Tutorial on Variational Autoencoders," arXiv preprint arXiv:1606.05908, 2016.
H. Oord, Y. Li, and O. Vinyals, "Representation Learning with Contrastive Predictive Coding," arXiv preprint arXiv:1807.03748, 2018.
B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the Support of a High-Dimensional Distribution," Neural Computation, vol. 13, no. 7, pp. 1443–1471, Jul. 2001.
P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion," Journal of Machine Learning Research, vol. 11, pp. 3371–3408, Dec. 2010.
Y. LeCun et al., "Efficient BackProp," in Neural Networks: Tricks of the Trade, 2nd ed., Springer, 2012, pp. 9–48.
G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network," arXiv preprint arXiv:1503.02531, 2015.
S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1251–1258, 2017.
Y. Bengio, A. Courville, and P. Vincent, "Representation Learning: A Review and New Perspectives," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, Aug. 2013.
Y. Zhang, S. Chen, X. Xu, and T. Li, "Real-Time Anomaly Detection for Streaming Data Using Autoencoders," Proceedings of the IEEE International Conference on Data Engineering (ICDE), pp. 1576–1581, 2019.
C. Aggarwal, Outlier Analysis, 2nd ed., Springer, 2017.
P. H. Nguyen, K. L. Hoang, and S. Kim, "Fault Detection in Real-Time Streaming Data Using Statistical Methods," IEEE Transactions on Industrial Informatics, vol. 14, no. 5, pp. 2083–2092, May 2018.
Y. Guo, Q. Wang, M. K. Ng, and K. Huang, "Streaming Anomaly Detection on Time-Series Data," IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 9, pp. 1771–1783, Sep. 2019.
M. Abadi et al., "TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems," arXiv preprint arXiv:1603.04467, 2016.
J. Dean et al., "Large Scale Distributed Deep Networks," Advances in Neural Information Processing Systems (NeurIPS), pp. 1223–1231, 2012.
Y. Chen et al., "Edge Computing for Real-Time Data Processing: A Review," IEEE Internet of Things Journal, vol. 6, no. 5, pp. 7699–7711, Oct. 2019.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
