International Journal For Multidisciplinary Research

E-ISSN: 2582-2160     Impact Factor: 9.24

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Call for Paper Volume 8, Issue 2 (March-April 2026) Submit your research before last 3 days of April to publish your research paper in the issue of March-April.

AI tool/app for Indian Sign Language (ISL) generator from audio-visual content in English/Hindi to ISL content and vice versa

Author(s) Mr. Rangana Revanth Sai, Mr. Durgam Dinesh Kumar, Mr. Chada Darwin, Mr. Isukala Nikhil, Mr. Rohit Pratap Singh
Country India
Abstract Indian Sign Language (ISL) is the primary visual communication medium for approximately 63 million deaf and hard-of-hearing individuals in India, yet computational resources for ISL processing remain severely limited compared to spoken languages. This paper presents a comprehensive framework for developing an AI-powered mobile application capable of bidirectional translation between English/Hindi audio-visual content and Indian Sign Language. The system integrates three critical components: large-scale dataset aggregation from sources including iSign benchmark (118,000 video-sentence pairs) and ISLTranslate (31,222 sentence pairs), advanced natural language processing pipelines for grammatical transformation, and computer vision techniques for sign generation and recognition. We discuss the architectural design of such systems, the processing pipeline from audio input through NLP tokenization, lemmatization, and part-of-speech tagging to final ISL video output, and present evaluation methodologies using Dynamic Time Warping and machine translation metrics. The paper incorporates findings from recent research on ISL datasets, linguistic properties unique to sign languages, and practical implementation considerations for real-world deployment on mobile platforms.
Index Terms—Indian Sign Language, bidirectional translation, natural language processing, computer vision, mobile application, accessibility technology, sign language datasets, neural machine translation.
Keywords Indian Sign Language, Speech To Sign Translation, Sign Language Recognition, Natural Language Processing, Computer Vision, Accessibility Technology, Mobile Application
Field Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In Volume 7, Issue 6, November-December 2025
Published On 2025-12-31
DOI https://doi.org/10.36948/ijfmr.2025.v07i06.65064

Share this