At present, the research on fatigue driving at home and abroad mainly has the following three methods: (i) driving behavioral (vehicle-based), (ii) driver behavioral (video-based), and (iii) driver physiological signals measure. The physiology-based methods have the highest recognition result. When drivers are in a state of fatigue, the Autonomic Nervous System (ANS) activity will be reflected from the physiological signal. Most of the contact sensors are used to obtain the physiological signal information of the driver. However, the contact sensors will affect the driver's driving operation, so this paper uses the frequency-modulated continuous-wave (FMCW) radar to collect the physiological signals. A fatigue driving simulation experiment was designed to collect experimental subjects' physiological signal data and separate the steady heartbeat and respiratory signals. Perform heart rate variability (HRV) time domain and frequency domain analysis on the heartbeat signal, and get the time domain derived features: mean of heart rate (AVGHR), heart rate root mean square difference (rMSSD). Frequency domain derived features: heart rate low-frequency (LF), heart rate high-frequency (HF), ratio of heart rate low frequency to high frequency (LF/HF). Using the spectrum estimation to get the respiratory frequency and the mean of breathing, heart rate to breathing ratio are selected as the respiratory signal's time-domain derived features. Finally, a two-class model of fatigue driving is established based on the support vector machine (SVM) theory. The above seven feature indicators are used as feature vectors as the SVM input, and the classification model is trained through the k-fold cross-validation method. The test set is used for classification detection. The accuracy rates of normal and fatigued driving are 88.75% and 84.25%, respectively. We also use Random Forests for comparison experiments. The accuracy of the RF are 96.88% and 95.14% respectively.