Next-Generation Speech Analysis for Emotion Recognition and PTSD Detection with Advanced Machine and Deep Learning Models

Abstract

Automated emotion recognition and post-traumatic stress disorder (PTSD) detection through speech analysis represent critical challenges in affective computing and mental health diagnostics. Existing approaches often rely on either handcrafted acoustic features or deep learning models trained on isolated representations, resulting in limited generalization and task-specific performance. This study aims to develop a unified hybrid framework capable of simultaneously recognizing emotions and detecting PTSD from speech signals using multi-feature and multi-task learning strategies. The proposed methodology integrates three complementary feature types: handcrafted acoustic descriptors (MFCCs, prosody, jitter), spectrogram-based convolutional features, and contextual embedding’s from the pretrained wav2vec 2.0 model. These features are processed through a CNN-BiLSTM architecture enhanced with an attention mechanism and jointly optimized using a multi-task learning objective. Experiments were conducted on the IEMOCAP dataset for emotion classification and the DAIC-WOZ dataset for PTSD detection. The hybrid model achieves an emotion recognition accuracy of 84.7% and a macro-F1 score of 0.78, significantly outperforming traditional SVM and single-stream deep learning baselines. For PTSD detection, the model attains 89.3% accuracy, 0.86 F1-score, and an AUC-ROC of 0.92. These results confirm the effectiveness of hybrid feature fusion and attention-enhanced sequence modeling in affective and clinical speech tasks. The proposed framework offers a scalable, non-invasive solution for real-world deployment in digital mental health and intelligent human-computer interaction systems