Real-Time End-to-End Stop Sign and Traffic Light Detection Development and Vehicle Testing

With the rise of end-to-end autonomous driving, visual perception for environmental understanding has become a key research topic in advanced driver assistance system (ADAS) development. Most existing end-to-end models generate only executable control commands or planned trajectories, making the prediction process difficult to interpret. In this study, we present an end-to-end approach for traffic-light recognition and stop-sign detection built on top of the open-source openpilot framework. Instead of deploying separate object detection networks, we extend the existing backbone with two lightweight multi-task heads: a traffic-light detection and classification head, and a stop-sign detection head with confidence estimation. The modified architecture preserves openpilot’s core driving functionality by reusing shared features and incorporating compact residual and feed-forward layers. The additional perception outputs are appended to the original outputs, ensuring that the model’s performance on other driving tasks remains unaffected. The proposed model is trained under diverse scenes and lighting conditions and demonstrates high accuracy in traffic-light classification and stop-sign detection, maintaining stable and consistent behavior during on-road evaluation. Furthermore, the enhanced model is fully compatible with Comma 3X hardware and has been successfully deployed and validated on a 2025 Nissan Leaf test vehicle. This work demonstrates the feasibility of developing a compact, lightweight, and deployable perception module that integrates traffic-signal understanding directly into an end-to-end driving model with minimal architectural modification.