The increase in number of vehicles on the road is being observed day by day, and the responsibilities that should be upheld by vehicle owners are frequently neglected. To ensure that the rules defined by the RTO are adhered to by every vehicle owner, the "Vehicle Surveillance System using deep learning" is proposed. This system is designed to capture vehicle data through web-app, with the identification of vehicles being achieved through the recognition of their number plates, and the extraction of (...) characters from the plates. Once the vehicle's number plate is recognized, the vehicle's information and owner's details will be verified with RTO data using Application programming interface (API). If any unfulfilled criteria are identified by the system, a notification will be generated. Vehicle detection is carried out using the Common Objects in Context Single Shot MultiBox Detection (COCOSSD) model, while the Russian model is utilized for number plate recognition. The extraction of characters from the number plate is executed through Optical Character Recognition (OCR), which includes character extraction and segmentation. (shrink)