Understanding the Limitations of Zero-Shot Large- Language Models on Hinglish and Tanglish Text

Jaganathan B; Saravanan P

Understanding the Limitations of Zero-Shot Large- Language Models on Hinglish and Tanglish Text

Author(s)	Jaganathan B, Saravanan P
Country	India
Abstract	Large Language Models (LLMs) have shown remarkable skill in a wide range of Natural Language Processing (NLP) tasks. But we still don't know how well they perform in real life when there are more than one language, especially when code is mixed in. A lot of people in India utilise Hinglish (Hindi- English) and Tanglish (Tamil-English) on social media. People regularly switch languages in the middle of sentences and use grammar, transliteration, and slang from their area that isn't particularly professional. This study conducts an empirical error analysis of zero-shot LLMs employed for sentiment classification on code-mixed Indian texts. A comparative study is performed using two publicly available datasets: Hinglish and Tanglish. Sentiment categorisation employs the BART-Large-MNLI model in a zero-shot manner, lacking any task-specific training. To see how well the model performs, we look at its accuracy, precision, recall, F1-score, and confusion matrices. The results demonstrate that zero-shot LLMs don't work very well; they only get 31.1% correct on Hinglish and 43.0% correct on Tanglish datasets. Transliteration ambiguity, slang, irony, and complex code-switching are all challenges that come up again and again, according to error analysis. The work highlights the challenges faced by modern big language models in processing code-mixed Indian languages and stresses the imperative for language-specific adaptations in multilingual natural language processing systems.
Keywords	Large Language Models, Code-Mixed Languages, Hinglish, Tanglish, Sentiment Analysis, Error Analysis, Zero- Shot Learning, Multilingual NLP.
Field	Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In	Volume 8, Issue 3, May-June 2026
Published On	2026-05-03

View / Download PDF File

E-ISSN 2582-2160

doi

CrossRef DOI is assigned to each research paper published in our journal.

IJFMR DOI prefix is
10.36948/ijfmr

Downloads

Research Paper Format Copyright Permission Form and Undertaking Form Cover Page Vol 8 Isu 2 Cover Page Vol 8 Isu 1 Cover Page Vol 7 Isu 6

All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.

CC-BY-SA

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Understanding the Limitations of Zero-Shot Large- Language Models on Hinglish and Tanglish Text

Share this