Bridging the Domain Gap in Air-Written Digit Recognition: Fine-Tuned CNN Using Gesture-Based Virtual Painter for Real-Time Fingertip Trajectory Capture

Nitasha Thakur; Nandini Gupta; Sonia Sharma; Shavi Juneja

Bridging the Domain Gap in Air-Written Digit Recognition: Fine-Tuned CNN Using Gesture-Based Virtual Painter for Real-Time Fingertip Trajectory Capture

Author(s)	Ms. Nitasha Thakur, Ms. Nandini Gupta, Ms. Sonia Sharma, Ms. Shavi Juneja
Country	India
Abstract	Air-written digit recognition connects touchless interaction with image classification, but a model trained on conventional handwriting may not generalize to traces created from webcam-tracked finger movements. This study builds on an existing hand-tracking and virtual-painter application developed with MediaPipe, OpenCV and a CNN trained on MNIST. The prototype tracks the index fingertip, renders strokes on a digital canvas and displays a predicted digit with class confidence scores. The proposed research will investigate supervised fine-tuning using a custom air-written dataset. An initial collection target of 20 writers and 4,000 labelled images is defined, with writer-separated training, validation and test partitions. The evaluation design preserves the source-trained checkpoint and compares original preprocessing, standardized preprocessing and target-domain fine-tuning so that their effects can be distinguished. Accuracy, precision, recall, macro-F1 and confusion matrices will assess classification, while capture failures and recognition latency will describe practical system behavior. Existing project screenshots provide qualitative implementation evidence. Custom dataset collection and fine-tuning have not yet been completed, so performance gains are treated as hypotheses rather than experimental findings. The intended contribution is a reproducible methodology and application-specific dataset for determining whether adaptation improves recognition of unseen writers without confusing interface demonstrations, confidence scores and measured accuracy.
Keywords	Air-writing, digit recognition, domain adaptation, convolutional neural network, MediaPipe, virtual painter, fine-tuning
Field	Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In	Volume 8, Issue 5, September-October 2026
Published On	2026-10-11

View / Download PDF File

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Bridging the Domain Gap in Air-Written Digit Recognition: Fine-Tuned CNN Using Gesture-Based Virtual Painter for Real-Time Fingertip Trajectory Capture

Share this