Design and Implementation of a Framework for Emotion Detection System Utilizing Audio and Video-Based Expressions using AI

Author(s)	Mr. Shaik Kareemulla Sha, Ch Santhi, Ms. Pinisetti Devi Sri Satya Sai
Country	India
Abstract	Emotion recognition is an important area in artificial intelligence and human-computer interaction. Traditional emotion detection systems mainly depend on a single source such as facial expressions or speech signals, which may reduce accuracy in real-time conditions. This paper proposes a multimodal emotion detection framework using audio, video, and text-based expressions with artificial intelligence techniques. The proposed system uses Convolutional Neural Networks (CNN) for facial emotion recognition, Long Short-Term Memory (LSTM) networks for speech emotion analysis, and Natural Language Processing (NLP) for understanding text queries. The system processes multimedia inputs and predicts emotions such as happiness, sadness, anger, fear, surprise, and neutral emotions. The framework is implemented using Python, TensorFlow, Keras, OpenCV, and Flask. Experimental results show that combining multiple modalities improves emotion recognition accuracy and system performance compared to traditional single-modality methods.
Keywords	Emotion Detection, Artificial Intelligence, Deep Learning, CNN, LSTM, NLP, Multimodal Emotion Recognition, Audio and Video Analysis.
Field	Engineering
Published In	Volume 8, Issue 3, May-June 2026
Published On	2026-05-17
DOI	https://doi.org/10.36948/ijfmr.2026.v08i03.78721

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research