In this work, we classify the stress state of car drivers using multimodal physiological signals and regularized deep kernel learning. Using a driving simulator in a controlled environment, we acquire electrocardiography (ECG), electrodermal activity (EDA), photoplethysmography (PPG), and respiration rate (RESP) from N = 10 healthy drivers in experiments of 25min duration with different stress states (5min resting, 10min driving, 10min driving + answering cognitive questions). We manually remove unusable segments and approximately 4h of data remain. Multimodal time and frequency features are extracted and employed to regularized deep kernel machine learning based on a fusion framework. Task-specific representations of different physiological signals are combined using intermediate fusion. Subsequently, the fused multimodal features are fed a support vector machine (SVM) and a random forest (RF) for stress classification. The experimental results show that the proposed approach can discriminate between stress states. The combination of PPG and ECG using RF as classifier yields the highest F1-score of 0.97 in the test set. PPG only and RF yield a maximum F1-score of 0.90. Furthermore, subject-specific cross-validation improves performance. ECG and PPG signals are reliable in classifying the stress state of a car driver. In summary, the proposed framework could be extended to real-time stress state assessment in driving conditions.