Traffic abnormal detection is crucial in intelligent transportation systems, while the heterogeneity and weak spatio-temporal correlation of multi-source data make it difficult for traditional methods to effectively fuse and utilize multimodal information. Most of the existing studies use data-level or decision-level fusion, which fails to fully exploit the feature complementarity of multi-source data, resulting in limited detection accuracy. To this end, we propose a multi-source data fusion anomaly detection method based on graph autoencoder (GAE) and diffusion graph neural network (DiffGNN). First, a unified data preprocessing and fusion strategy is designed to perform feature-level fusion of data from on-board sensors, infrastructures, and external environments to eliminate inconsistencies in data format, temporal alignment, and spatial distribution. Then, GAE is employed for potential graph structure feature extraction to enhance the global representation of the data on the basis of dimensionality reduction. Then, DiffGNN propagates anomalous features through the dynamic diffusion mechanism to strengthen the detection ability of local anomalous behaviors. The experimental results show that GAE-DG outperforms the traditional method for anomaly detection in multi-source data environment, with an accuracy rate of 95.24%, demonstrating stronger generalization ability and detection stability.