Development of an autonomous vehicle system for detecting road irregularities using LiDAR and depth cameras: A case study on potholes, speed breakers, and dynamic dust - Multimedia Tools and Applications

Executive Summary

This research presents the development of a miniaturized autonomous vehicle system for real-time detection and classification of road irregularities using advanced multimedia sensor fusion. The system integrates LiDAR sensors with RGB-D cameras and employs a novel lightweight ensemble of CNN and LSTM networks for enhanced road condition analysis. Our approach introduces innovative methodological contributions including dual-sensor positioning strategies, dynamic pattern recognition algorithms, and adaptive multimedia fusion techniques that combine geometric and visual information for comprehensive road surface assessment. The hybrid CNN-LSTM ensemble architecture achieves superior classification performance by leveraging both spatial feature extraction and temporal sequence modeling. Extensive testing across diverse road conditions, including varying pothole depths and speed breaker configurations, demonstrates the system’s robust detection capabilities. The multimedia fusion approach, enhanced with handcrafted features, significantly enhances classification accuracy to 98.75%, outperforming traditional single-modality methods. Additionally, the research evaluates system performance under challenging environmental conditions, revealing adaptive responses to dynamic dust scenarios with maintained operational reliability. The autonomous vehicle implements intelligent navigation strategies, including precision steering adjustments for pothole avoidance and adaptive speed modulation for obstacle management. This study contributes to the advancement of multimedia signal processing in autonomous vehicle technology by providing a scalable framework for road irregularity detection with potential applications in intelligent transportation systems and urban infrastructure monitoring.