High-speed wet clutches may experience dynamic instability between the friction plates, leading to rattling vibrations and a significant increase in drag torque. This study employs a homogeneous flow model to characterize the gas-liquid two-phase flow within a high-speed clutch. It establishes a dynamic model for the angular oscillation of friction plates. Finite-element numerical simulations and stability analyses were conducted. The results indicate that as the clutch speed difference increases, the density and viscosity of the two-phase flow decrease rapidly, leading to a sharp reduction in fluid stiffness and damping. Consequently, the friction plates become more susceptible to angular oscillation. The stability of angular oscillation is determined by two key parameters: dimensionless comprehensive stiffness and critical frequency ratio. Higher dimensionless comprehensive stiffness and a lower critical frequency ratio enhance oscillation stability. Numerical evaluations of various groove types reveal that as rotational speed and friction plate clearance increase, the fluid stiffness coefficient, damping coefficient, dimensionless comprehensive stiffness, and critical moment of inertia all decrease, thereby reducing angular oscillation stability. Among the tested groove geometries, enclosed grooves and spiral grooves exhibit superior stability due to their strong hydrodynamic effects, yielding the highest dimensionless comprehensive stiffness. The critical frequency ratio for the self-excited angular oscillation of friction plates is approximately 0.5, termed the half-frequency oscillation characteristic. Experimental data validate the proposed angular oscillation model and its frequency response, providing a theoretical foundation for performance prediction and stability optimization in high-speed clutch design.