ReadAble , a lip reading and sign language interpretation machine learning and deep learning model

Prajyot Kawatkar; Sujal Wankhede; Dev Katre; Rafe Sheikh; Leelkanth Dewangan

ReadAble , a lip reading and sign language interpretation machine learning and deep learning model

Author(s)	Mr. Prajyot Kawatkar, Mr. Sujal Wankhede, Mr. Dev Katre, Mr. Rafe Sheikh, Prof. Leelkanth Dewangan
Country	India
Abstract	Silent communication involving lip movements and sign language continues to be greatly under-used in popular media and new media. This paper describes ReadAble, a two-part AI system consisting of a Lip Reading module and a Sign Language Detection module that transforms silent video into output that is made accessible via text and synthesized audio output (lip reading) and translated text output (sign language). The Lip Reading module first locates the lip region of the video, tracks the spatiotemporal features of the lip movement, and maps the resulting movement to phonemes/words, producing both text and speech output. The Sign Language Detection module detects the hand and body movements (e.g. ASL, ISL in this case) and interprets the movement sequences from arbitrary sign language expressions into signed sentences or phrases, before producing them in the form of subtitles. The implementation of the system was done in Python (Jupyter Notebook) utilizing multiple state-of-theart models (3D CNN, LSTM / Transformer, MediaPipe / YOLO or SSD), utilizing video data captured from user or recorded feed.Overall, we recognize several remaining challenges to deploying the system in real world noisy, occluded, and continuous settings; and we provide some thoughts on the roadmap for getting us from a lab and into a real-world setting with a better product.
Keywords	Lip Reading, Sign Language Detection, Computer Vision, Deep Learning, Accessibility, HumanComputer Interaction
Field	Engineering
Published In	Volume 8, Issue 3, May-June 2026
Published On	2026-05-29

View / Download PDF File

E-ISSN 2582-2160

doi

CrossRef DOI is assigned to each research paper published in our journal.

IJFMR DOI prefix is
10.36948/ijfmr

Downloads

Research Paper Format Copyright Permission Form and Undertaking Form Cover Page Vol 8 Isu 3 Cover Page Vol 8 Isu 2 Cover Page Vol 8 Isu 2

All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.

CC-BY-SA

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

ReadAble , a lip reading and sign language interpretation machine learning and deep learning model

Share this