The centrifugal compressor is one of the most commonly used air compressors for fuel cell air supply systems, and it has the small volume, high pressure ratio and low noise. However, surge in a centrifugal compressor severely limits its stable flow range. In this paper, a mathematical model of the compressor aerodynamic performance based on the energy transfer method was established, some parameters of model were identified by experimental data, and the model was validated through experiments. Then the dynamic model of the compression system was derived based on the compressor model and the Moore-Greitzer model. The stability analysis of the compression system was conducted, and it was strictly proved that when the compression system is unstable, there is the limit cycle in this nonlinear system, namely the surge cycle. Furthermore, the simulation of the compression system was conducted and the instability condition of the compression system was presented. The results show that at almost all constant speeds, the compression system instability occurs as the opening of the throttle valve decrease to a specific value, and at some opening of the throttle valve, the compression system instability occurs as speed decrease to a specific value. At last, the effects of structural parameters of the compression system on surge characteristics were analyzed. This research can guide the operating condition match of compressors and the active surge control design of the compression system for fuel cell vehicles.