As a crucial part of the intelligent transportation system, traffic signal control will realize the boundary control of the traffic area, it will also lead to delays and excessive fuel consumption when the vehicle is driving at the intersection. To tackle this challenge, this research provides an optimized control framework based on reinforcement learning method and speed guidance strategy for the connected vehicle network. Prior to entering an intersection, vehicles are focused on in a specific speed guidance area, and important factors like uniform speed, acceleration, deceleration, and parking are optimized. Conclusion, derived from deep reinforcement learning algorithm, the summation of the length of the vehicle’s queue in front of the signal light and the sum of the number of brakes are used as the reward function, and the vehicle information at the intersection is collected in real time through the road detector on the road network. Finally, the proposed method is implemented through the SUMO (Simulation of Urban Mobility) simulation platform. The results demonstrate the effectiveness of the proposed model by obtaining the space-time trajectory map of the vehicle before and after optimization, as well as the vehicle’s travel time and fuel consumption.