Entropy-Guided, Optimization-Tuned Fusion for Multimodal Emotion Recognition in Stress and Depression Detection: A Gap Analysis and Proposed Framework

Swati Diwakar Bhutekar; Yogini Borole; Swati Shailesh Chandurkar

doi:10.36948/ijfmr.2026.v08i05.87002

Entropy-Guided, Optimization-Tuned Fusion for Multimodal Emotion Recognition in Stress and Depression Detection: A Gap Analysis and Proposed Framework

Author(s)	Ms. Swati Diwakar Bhutekar, Dr. Yogini Borole, Ms. Swati Shailesh Chandurkar
Country	India
Abstract	Multimodal emotion recognition (MER) has become a more powerful solution than unimodal approaches for stress and depression screening or other tasks, but the literature presents several common and recurring shortcomings that hinder accuracy and application. This paper provides a systematic gap analysis of the MER literature in the unimodal (facial, speech, text, EEG and physiological) and multimodal fusion research and a detailed description of a proposed AI based deep-learning framework to tackle the identified gaps directly. The results of the gap analysis, which is carried out on sixty-six publications reviewed, result in the following six identified deficiencies: (1) limited multimodal integration by using video, EEG or physiological signals in the same framework; (2) the use of fixed or heuristically chosen fusion weights, without optimization; (3) no multimodal fusion rules mentioned in the reviewed literature based on entropy; (4) limited validation on Indian-context data; (5) scarcity of the use of metaheuristic optimization to optimize multimodal fusion; and (6) limited explicit framing of multimodal emotion recognition in the context of stress and depression detection. In the proposed method, the gaps are addressed by five components: unimodal models for modality-specific speech, facial images and EEG; a novel Deep Belief Network and Deep Recurrent Neural Network for the video and EEG/physiological branches respectively; a cross-modal heterogeneity resolution mechanism using a shared latent space across the two modalities; a novel hybridization of Bat-Rider Optimization Algorithm for the fusion rule; and a validation plan covering the DEAP benchmark and an original Indian-context multimodal dataset. Finally, the paper explicitly traces each part of the proposed approach to the gap it is filling, giving a clear foundation for the design of the framework.
Keywords	Multimodal Emotion Recognition, Gap Analysis, Deep Learning, Fusion Strategies, Renyi entropy, Metaheuristic Optimization, EEG, Stress Detection, Depression Detection.
Field	Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In	Volume 8, Issue 5, September-October 2026
Published On	2026-09-12
DOI	https://doi.org/10.36948/ijfmr.2026.v08i05.87002

View / Download PDF File

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Entropy-Guided, Optimization-Tuned Fusion for Multimodal Emotion Recognition in Stress and Depression Detection: A Gap Analysis and Proposed Framework

Share this