Speaker performance in Acoustic Vehicle Alerting System (AVAS) plays a crucial role for pedestrian safety. Sound radiation from AVAS speaker has obvious directivity pattern. Considering this feature is critical for accurately simulating the exterior sound field of electrical vehicles. This paper proposes a new process to characterize the sound directivity pattern of AVAS speaker. The first step of the process is to perform an acoustic testing to measure the sound pressure radiated from the speaker at a certain number of microphone locations in a free field environment. Based on the geometry of a virtual speaker, the locations of each microphone and measured sound pressure data, an inverse method, namely the inverse pellicular analysis, is adopted to recover a set of vibration pattern of the virtual speaker surface. The recovered surface vibration pattern can then be incorporated in the full vehicle numerical model as an excitation for simulating the exterior sound field. In this study, the process is illustrated in detail and validated with a production AVAS speaker. Using the virtual speaker and the extracted vibration information, the sound pressure level (SPL) at each microphone location is numerically calculated. The SPL from the direct measurement and the prediction matches very well. By using the virtual speaker and its surface vibration, the directivity pattern of its sound field is fully recovered. Furthermore, numerical analysis is performed to calculate the sound field of the AVAS speaker in a full vehicle model. The acoustic transfer functions from the speaker location to three regulatory microphone locations are calculated. Results clearly reveal the deficiency of using a monopole to represent AVAS speaker and the necessity to consider its directivity pattern.