Audio-Based Urban Safety Classification System Using Transformer Encoder based Representations

Abstract

Urban traffic management and public safety increasingly rely on real-time monitoring of traffic and environmental events. Traditional detection methods, manual surveillance, human observation, and conventional sensors suffer from limited scalability, delayed reporting, and human error, leading to inefficient interventions. To address these challenges, this research proposes an automated audio-based classification system for urban traffic and acoustic events. The system utilizes advanced Machine Learning (ML) and Deep Learning (DL) techniques to analyze urban audio signals and classify them into categories such as accidents, honking, traffic congestion, and crime-related sounds. At its core, the model employs Transformer Encoder for Representations of Audio (TERA) for feature extraction, which captures temporal and spectral patterns and converts them into high-dimensional embeddings. These embeddings are then fed into supervised classifiers, including Categorical Boosting (CatBoost), Histogram Gradient Boosting (HGB), Extra Trees Classifier (ETC), and the proposed Tree-based Generalized Additive Model (TGAM). Experimental results show that TGAM outperforms baseline ensemble models, achieving macro-average Precision, Recall, and F1-score above 90% on a balanced dataset. The system also integrates visualization tools such as confusion matrices, Receiver Operating Characteristic (ROC) curves, and waveform overlays for interpretability. Additionally, a Tkinter-based Graphical User Interface (GUI) is developed with role-based access. Administrators manage data and model training, while users perform real-time predictions. Security is ensured using TinyDB with Secure Hash Algorithm 256-bit (SHA-256) password hashing.