Abstract

This paper explores the implementation of a batch processing pipeline for SIM log data in the telecommunication industry using Azure cloud services. The project leverages Azure Data Lake for data storage, Azure Data Factory for automated data ingestion, and Azure Databricks for processing large volumes of data. By applying machine learning algorithms, the system identifies patterns in network usage, detects anomalies, and provides insights into customer behaviour. The results, visualized using Power BI, enable telecom operators to optimize network performance and enhance customer satisfaction.