Abstract

Advancements in cloud technologies have increased the infrastructural needs of data centers due to storage needs and processing of extensive dimensional data. Many service providers envisage anomaly detection criteria to guarantee availability to avoid breakdowns and complexities caused due to large-scale operations. The streaming log data generated is associated with multi-dimensional complexity and thus poses a considerable challenge to detect the anomalies or unusual occurrences in the data. In this research, a hybrid model is proposed that is motivated by deep belief criteria and meta-heuristics. Using Search-and-Rescue—BrainStorm Optimization (SAR-BSO), a hybrid feature selection (FS) and deep belief network classifier is used to localize and detect anomalies for streaming data logs. The significant contribution of the research lies in FS, which is carried out using SAR-BSO which increases the detection power of the model as it selects the most significant variables by minimizing redundant features. The evaluation of accuracy is efficiently improved when compared with the predictable methods, such as Extract Local Outlier Factor (ELOF), Track-plus, Hybrid Distributed Batch Stream (HDBS), IForestASD, DBN, BSO-based Feature Selection with DBN, Genetic Algorithm-Deep Belief Network (GA-DBN), Mutual Information-Deep Belief Network (MI-DBN), information entropy-Deep Belief Network(I + DBN), Flat Field-Deep Belief Network (FF + DBN), African Vulture Optimization Algorithm-Deep Belief Network(AVOA + DBN), Gorilla Troop Optimizer-Deep Belief Network(GTO-DBN), and SARO-based Feature Selection with DBN. Further, the accurate detection of the anomalies in the data stream is established by the Deep Belief Neural Network (DBN) classifier. The model’s efficacy is determined using Apache, Hadoop, HDFS, Spark, and Linux datasets and evaluated against existing similar models. The model efficiency is provided using multiple evaluation metrics and is found effective. From the experimentation, the accuracy of the proposed model is found to be 93.3, 95.4, 93.6, 94.2, and 93.5% respectively for the dataset such as Apache, Hadoop, HDFS, spark, and Linux. This enhancement in accuracy is due to the selection of optimal features by the proposed SAR-BSO algorithm.