Real-Time End-to-end Stop Sign and Traffic Light Detection Development and Vehicle Testing

With the rise of end-to-end autonomous driving, visual perception for environmental understanding has become one of the hot topics in advanced driver assistance system development. Most existing end-to-end models only generate executable control command or a planned path, which makes the prediction process hard to interpret. In this research, an end-to-end method for stop-sign and traffic light detection along with depth estimation is built on top of the openpilot open source end-to-end model. Instead of running separate object detection models, we extend the existing openpilot backbone with two lightweight multi-task heads: traffic light detection and classification head and stop sign detection head with confidence and distance output. The modified architecture preserves openpilot’s function by reusing shared features and adding lightweighted residual blocks and feedforward layers. The new outputs are appended to the original openpilot model output, so it will not affect the model performance on dealing with other tasks. The model trained on different scenes and lighting conditions show high accurate traffic light classification, stop sign detection and stable distance estimation that remain smooth under motion in testing. Also, the stop sign traffic light detection and depth estimation model is compatible with comma 3x hardware, which is installed on a 2025 Nissan Leaf test vehicle and validated on road test. This works shows the possibility to develop a compact, lightweighted and deployable module that bring traffic sign detection and distance estimation into an end-to-end driving model with minimal architectural modification. Keywords— end-to-end driving; traffic-light recognition; stop-sign detection; depth estimation; ADAS.